返回模型
说明文档
MODIPHY:
基于幽灵卷积的快速YOLO多模态遮挡检测物联网系统
我们开发了"YOLO Phantom"，用于在资源受限的物联网应用中进行低光照条件和遮挡场景的检测。我们提出了创新的"幽灵卷积（Phantom Convolution）"，使YOLO Phantom在参数量和模型大小减少43%的情况下，达到与YOLOv8n相当的精度，同时GFLOPs减少了19%。通过在我们的多模态数据集上采用迁移学习，该模型在恶劣条件下展现出增强的视觉能力。我们的树莓派物联网平台配备noIR相机，并集成AWS IoT Core和SNS，与基准YOLOv8n模型相比，热成像和RGB数据检测的帧率分别提升了17%和14%。
小型模型对比
各种低光照和遮挡条件下的检测效果
如需了解更多关于MODIPHY的信息，请参阅arXiv上的预印本。
请参阅yolo_phantom获取实现代码。
下载多模态数据集
如果您觉得这项工作有用，请引用我们：
@article{mukherjee2024modiphy, title={MODIPHY: Multimodal Obscured Detection for IoT using PHantom Convolution-Enabled Faster YOLO}, author={Mukherjee, Shubhabrata and Beard, Cory and Li, Zhu}, journal={arXiv preprint arXiv:2402.07894}, year={2024} }
shubha07m/yolo_phantom
作者 shubha07m
object-detection
↓ 0
♥ 1
创建时间: 2024-02-29 15:48:40+00:00
更新时间: 2024-02-29 16:56:48+00:00
在 Hugging Face 上查看文件 (78)
.gitattributes
README.md
gflop.png
ous_images.png
yolo_phantom/cfg/__init__.py
yolo_phantom/cfg/datasets/Argoverse.yaml
yolo_phantom/cfg/datasets/DOTAv1.5.yaml
yolo_phantom/cfg/datasets/DOTAv1.yaml
yolo_phantom/cfg/datasets/GlobalWheat2020.yaml
yolo_phantom/cfg/datasets/ImageNet.yaml
yolo_phantom/cfg/datasets/Objects365.yaml
yolo_phantom/cfg/datasets/SKU-110K.yaml
yolo_phantom/cfg/datasets/VOC.yaml
yolo_phantom/cfg/datasets/VisDrone.yaml
yolo_phantom/cfg/datasets/carparts-seg.yaml
yolo_phantom/cfg/datasets/coco-pose.yaml
yolo_phantom/cfg/datasets/coco.yaml
yolo_phantom/cfg/datasets/coco128-seg.yaml
yolo_phantom/cfg/datasets/coco128.yaml
yolo_phantom/cfg/datasets/coco8-pose.yaml
yolo_phantom/cfg/datasets/coco8-seg.yaml
yolo_phantom/cfg/datasets/coco8.yaml
yolo_phantom/cfg/datasets/crack-seg.yaml
yolo_phantom/cfg/datasets/dota8.yaml
yolo_phantom/cfg/datasets/open-images-v7.yaml
yolo_phantom/cfg/datasets/package-seg.yaml
yolo_phantom/cfg/datasets/tiger-pose.yaml
yolo_phantom/cfg/datasets/xView.yaml
yolo_phantom/cfg/default.yaml
yolo_phantom/cfg/models/README.md
yolo_phantom/cfg/models/rt-detr/rtdetr-l.yaml
yolo_phantom/cfg/models/rt-detr/rtdetr-resnet101.yaml
yolo_phantom/cfg/models/rt-detr/rtdetr-resnet50.yaml
yolo_phantom/cfg/models/rt-detr/rtdetr-x.yaml
yolo_phantom/cfg/models/v3/yolov3-spp.yaml
yolo_phantom/cfg/models/v3/yolov3-tiny.yaml
yolo_phantom/cfg/models/v3/yolov3.yaml
yolo_phantom/cfg/models/v5/yolov5-p6.yaml
yolo_phantom/cfg/models/v5/yolov5.yaml
yolo_phantom/cfg/models/v6/yolov6.yaml
yolo_phantom/cfg/models/v8/yolo_phantom.yaml
yolo_phantom/cfg/models/v8/yolov8-cls-resnet101.yaml
yolo_phantom/cfg/models/v8/yolov8-cls-resnet50.yaml
yolo_phantom/cfg/models/v8/yolov8-cls.yaml
yolo_phantom/cfg/models/v8/yolov8-ghost-p2.yaml
yolo_phantom/cfg/models/v8/yolov8-ghost-p6.yaml
yolo_phantom/cfg/models/v8/yolov8-ghost.yaml
yolo_phantom/cfg/models/v8/yolov8-obb.yaml
yolo_phantom/cfg/models/v8/yolov8-p2.yaml
yolo_phantom/cfg/models/v8/yolov8-p6.yaml
yolo_phantom/cfg/models/v8/yolov8-pose-p6.yaml
yolo_phantom/cfg/models/v8/yolov8-pose.yaml
yolo_phantom/cfg/models/v8/yolov8-rtdetr.yaml
yolo_phantom/cfg/models/v8/yolov8-seg-p6.yaml
yolo_phantom/cfg/models/v8/yolov8-seg.yaml
yolo_phantom/cfg/models/v8/yolov8.yaml
yolo_phantom/cfg/trackers/botsort.yaml
yolo_phantom/cfg/trackers/bytetrack.yaml
yolo_phantom/modiphy_paper.pdf
yolo_phantom/multimodal_trained_weights/yolophantom.onnx
ONNX
yolo_phantom/multimodal_trained_weights/yolophantom.pt
yolo_phantom/nn/__init__.py
yolo_phantom/nn/autobackend.py
yolo_phantom/nn/modules/__init__.py
yolo_phantom/nn/modules/block.py
yolo_phantom/nn/modules/conv.py
yolo_phantom/nn/modules/head.py
yolo_phantom/nn/modules/transformer.py
yolo_phantom/nn/modules/utils.py
yolo_phantom/nn/tasks.py
yolo_phantom/pre-processing/bounding_box_creation.py
yolo_phantom/pre-processing/bounding_box_validation_mixdata.py
yolo_phantom/pre-processing/bounding_box_validation_unimodal.py
yolo_phantom/pre-processing/coco2yolo_conversion.py
yolo_phantom/pre-processing/final_labels.txt
yolo_phantom/pre-processing/new_class_id_creation.py
yolo_phantom/pre-processing/rgb_thermal_mixing_from_testdata.py
yolo_phantom/pre-processing/rgb_thermal_mixing_from_train_val.py