Multi-task yolov5 with detection and segmentation based on yolov5

Last update: Dec 30, 2022

Related tags

Deep Learning yolov5ds

Overview

YOLOv5DS

Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0)

decoupled head
anchor free
segmentation head

README中文

Ablation experiment

All experiments is trained on a small dataset with 47 classes ,2.6k+ images for training and 1.5k+ images for validation:

model	P	R	[email protected]	[email protected]:95
yolov5s	0.536	0.368	0.374	0.206
yolov5s+train scrach	0.452	0.314	0.306	0.152
yolov5s+decoupled head	0.555	0.375	0.387	0.214
yolov5s + decoupled head+class balance weights	0.541	0.392	0.396	0.217
yolov5s + decoupled head+class balance weights	0.574	0.386	0.403	0.22
yolov5s + decoupled head+seghead	0.533	0.383	0.396	0.212

The baseline model is yolov5s. and decoupled head, add class balance weights all helps to improve MAP.

Adding a segmentation head can still get equivalent MAP as single detection model.

Training Method

python trainds.py

As VOC dataset do not offer the box labels and mask labels, so we forward this model with a detection batch and a segmention batch , and accumulate the gradient , than update the whole model parameters.

MAP

To compare with the SSD512, we use VOC07+12 training set as the detection training set, VOC07 test data as detection test data, for segmentation ,we use VOC12 segmentation datset as training and test set.

the input size is 512(letter box).

model	VOC2007 test
SSD512	79.8
yolov5s+seghead(512)	79.2

The above results only trained less than 200 epoch, weights

demo

see detectds.py.

Train custom data

Use labelme to label box and mask on your dataset;

the box label format is voc, you can use voc2yolo.py to convert to yolo format,

the mask label is json files , you should convert to mask .png image labels,like VOC2012 segmentation labels.

see how to arrange your detection dataset with yolov5 , then arrange your segmentation dataset same as yolo files , see data/voc.yaml:


# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: .  # dataset root dir
train: VOC/det/images/train  # train images (relative to 'path') 118287 images
val: VOC/det/images/test  # train images (relative to 'path') 5000 images
road_seg_train: VOC/seg/images/train   # road segmentation data
road_seg_val: VOC/seg/images/val

# Classes
nc: 20  # number of classes
segnc: 20

names: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']  # class names

segnames: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']

change the config in trainds.py and :

python trainds.py

test image folder with :
```
python detectds.py
```

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:3723: error: (-215:Assertion failed) inv_scale_x > 0 in function 'cv::resize'

opened by zhangfx123 0

Multi-task yolov5 with detection and segmentation based on yolov5

Related tags

Overview

YOLOv5DS

Ablation experiment

Training Method

MAP

demo

Train custom data

Reference

You might also like...

a basic code repository for basic task in CV(classification,detection,segmentation)

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

Implementation of PyTorch-based multi-task pre-trained models

Drone detection using YOLOv5

YOLOv5 detection interface - PyQt5 implementation

YOLOv5 + ROS2 object detection package

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

Releases(v6.0)

v6.0(Dec 16, 2021)

Owner

:hot_pepper: R²SQL: "Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing." (AAAI 2021)

An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

This repository is for EMNLP 2021 paper: It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data

Python Rapid Artificial Intelligence Ab Initio Molecular Dynamics

Pytorch implementation of "ARM: Any-Time Super-Resolution Method"

Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning

Multi Task Vision and Language

CountDown to New Year and shoot fireworks

Easy-to-use library to boost AI inference leveraging state-of-the-art optimization techniques.

[CoRL 2021] A robotics benchmark for cross-embodiment imitation.

Real-time Neural Representation Fusion for Robust Volumetric Mapping

Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation

Implementation of Change-Based Exploration Transfer (C-BET)

TensorFlow implementation of original paper : https://github.com/hszhao/PSPNet

Code accompanying the paper Shared Independent Component Analysis for Multi-subject Neuroimaging

Real-time VIBE: Frame by Frame Inference of VIBE (Video Inference for Human Body Pose and Shape Estimation)

A pytorch implementation of faster RCNN detection framework (Use detectron2, it's a masterpiece)

This is the official Pytorch-version code of FlatGCN (Flattened Graph Convolutional Networks for Recommendation).

Domain Adaptation with Invariant RepresentationLearning: What Transformations to Learn?

Fully convolutional networks for semantic segmentation