Thursday, July 28, 2022

NVIDIA Jetson YOLO Object Detection

NVIDIA Jetson YOLO Object Detection

Demos showcase how to convert pre-trained yolov3 and yolov4 models through ONNX to TensorRT engines. The code for these 2 demos has gone through some significant changes. More specifically, I have recently updated the implementation with a "yolo_layer" plugin to speed up the inference time of the yolov3/yolov4 models.

What is a YOLO object detector?

When it comes to deep learning-based object detection, there are three primary object detectors you’ll encounter:

  • R-CNN and their variants, including the original R-CNN, Fast R- CNN, and Faster R-CNN
  • Single Shot Detector (SSDs)
  • YOLO
First introduced in 2015 by Redmon et al., their paper, You Only Look Once: Unified, Real-Time Object Detection, details an object detector capable of super real-time object detection, obtaining 45 FPS on a GPU.

You Only Look Once: Unified, Real-Time Object Detection

https://arxiv.org/pdf/1506.02640v3.pdf

YOLOv4

With the original authors work on YOLO coming to a standstill, YOLOv4 was released by Alexey Bochoknovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. The paper was titled YOLOv4: Optimal Speed and Accuracy of Object Detection

Author: Alexey Bochoknovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao
Released: 23 April 2020

YOLOv3

YOLOv3 improved on the YOLOv2 paper and both Joseph Redmon and Ali Farhadi, the original authors, contributed.
Together they published YOLOv3: An Incremental Improvement

The original YOLO papers were are hosted here

Author: Joseph Redmon and Ali Farhadi
Released: 8 Apr 2018

We’ll be using YOLOv3 , YOLOv4 in this blog post, in particular, YOLO trained on the COCO dataset.

The COCO dataset consists of 80 labels.

YOLOv3-416

Image   :   4.46 FPS.

Video    :   4.35 FPS.


YOLOv3-tiny-416

Image  :  22.30 FPS.

Video   :   20.58 FPS.


YOLOv4-416

For NVIDIA Jetson Nano

Image   :  4.46 FPS.

Video   :   4.35 FPS.


For NVIDIA Jetson Xavier NX

Image   :  24.09 FPS.

Video   :   22.33 FPS.


YOLOv4-tiny-416

For NVIDIA Jetson Nano

Image  :  22.41 FPS.

Video   :  20.47 FPS.

For NVIDIA Jetson Xavier NX

Image   :  47.93 FPS.

Video   :   43.13 FPS.

Compare Performance


Hardware NVIDIA Jetson nano 4GB.

Jetpack SDK 4.6.1


Video  MP4 ( 960x540 )








Image JPG  ( 768x576 )






Hardware NVIDIA Jetson Xavier NX

Jetpack SDK 4.6.1



Compare Nano vs. Xavier NX






Run NVIDIA GPU GTX1070

Hardware
CPU :  Intel i3-9100F 3.60GHz
GPU :  NVIDIA GTX1070. ( Driver Version 470.141.03 )

Software
OS : Ubuntu 18.04.6 LTS
Python 3.6.9
CUDA v11.2.67

Python Library
numpy==1.19.4
opencv-python==4.5.1.48
tensorrt==7.2.2.3
pycuda==2019.1.2






YOLOv4-416 ( TensorRT FP16 )

Image ( 576x425 ) >  45.x fps
Video ( 960x540 ) > 40-44 fps

 

YOLOv4-tiny-416 ( TensorRT FP16 )

Image ( 576x425 ) > 110-115 fps
Video ( 960x540 ) >  86-107 fps



Compare Performance








NVIDIA Jetson Nano Object detection with YOLOv3-416
https://youtu.be/8XmceZQHWSc

NVIDIA Jetson Nano Object detection with YOLOv3-tiny-416

NVIDIA Jetson Nano Object detection with YOLOv4-416

NVIDIA Jetson Nano Object detection with YOLOv4-tiny-416

NVIDIA Jetson Xavier Object detection with YOLOv4-416

NVIDIA Jetson Xavier Object detection with YOLOv4-tiny-416


NVIDIA GPU GTX1070 Object detection with YOLOv4-tiny-416

https://youtu.be/5JSsxoGNGC4


Reference

TensorRT demos

https://github.com/jkjung-avt/tensorrt_demos


You Only Look Once: Unified, Real-Time Object Detection

https://arxiv.org/pdf/1506.02640v3.pdf


YOLOv3: An Incremental Improvement

https://arxiv.org/pdf/1804.02767.pdf


YOLOv4: Optimal Speed and Accuracy of Object Detection

https://arxiv.org/pdf/2004.10934.pdf


DarkNet YOLO

https://github.com/AlexeyAB/darknet


YOLO Object Detection

https://pyimagesearch.com/2018/11/12/yolo-object-detection-with-opencv/


Website : https://softpower.tech

Posted by Android Arduino at 12:09 PM
Labels: ,

Sunday, July 24, 2022

NVIDIA Jetson Object Detection with Jetson Inference and TensorFlow

NVIDIA Jetson Object Detection with Jetson Inference and TensorFlow

What is Jetson Inference?

Instructional guide for inference and real-time DNN vision library for NVIDIA Jetson Nano/TX1/TX2/Xavier NX/AGX Xavier/AGX Orin.


This code uses NVIDIA TensorRT for efficiently deploying neural networks onto the embedded Jetson platform, improving performance and power efficiency using graph optimizations, kernel fusion, and FP16/INT8 precision.


Vision primitives, such as imageNet for image recognition, detectNet for object detection, segNet for semantic segmentation, and poseNet for pose estimation inherit from the shared tensorNet object. Examples are provided for streaming from live camera feed and processing images. See the API Reference section for detailed reference documentation of the C++ and Python libraries.


Locating Objects with DetectNet

The previous recognition examples output class probabilities representing the entire input image. Next we're going to focus on object detection, and finding where in the frame various objects are located by extracting their bounding boxes. Unlike image classification, object detection networks are capable of detecting many different objects per frame.

The detectNet object accepts an image as input, and outputs a list of coordinates of the detected bounding boxes along with their classes and confidence values. detectNet is available to use from Python and C++. See below for various pre-trained detection models available for download. The default model used is a 91-class SSD-Mobilenet-v2 model trained on the MS COCO dataset, which achieves realtime inferencing performance on Jetson with TensorRT.

As examples of using the detectNet class, we provide sample programs for C++ and Python:

These samples are able to detect objects in images, videos, and camera feeds. For more info about the various types of input/output streams supported, see the Camera Streaming and Multimedia page.

Pre-trained Detection Models Available

Below is a table of the pre-trained object detection networks available for download, and the associated --network argument to detectnet used for loading the pre-trained models:

ModelCLI argumentNetworkType enumObject classes
SSD-Mobilenet-v1ssd-mobilenet-v1SSD_MOBILENET_V191 (COCO classes)
SSD-Mobilenet-v2ssd-mobilenet-v2SSD_MOBILENET_V291 (COCO classes)
SSD-Inception-v2ssd-inception-v2SSD_INCEPTION_V291 (COCO classes)
DetectNet-COCO-Dogcoco-dogCOCO_DOGdogs
DetectNet-COCO-Bottlecoco-bottleCOCO_BOTTLEbottles
DetectNet-COCO-Chaircoco-chairCOCO_CHAIRchairs
DetectNet-COCO-Airplanecoco-airplaneCOCO_AIRPLANEairplanes
ped-100pednetPEDNETpedestrians
multiped-500multipedPEDNET_MULTIpedestrians, luggage
facenet-120facenetFACENETfaces

note: to download additional networks, run the Model Downloader tool
             $ cd jetson-inference/tools
             $ ./download-models.sh


In this demo use only DetectNet.

NVIDIA Jetson Nano with Jetson Inference ( DetectNet )

Inference Performance

Model : SSD-Mobilenet-v2  ( TensorRT FP16 )  :  22-24 FPS  




NVIDIA Jetson Xavier NX  with Jetson Inference ( DetectNet )

Inference Performance

Model : SSD-Mobilenet-v2  ( TensorRT FP16 ) :  60 - 100 FPS



TensorFlow Object Detection API Creating accurate machine learning models capable of localizing and identifying multiple objects in a single image remains a core challenge in computer vision. The TensorFlow Object Detection API is an open source framework built on top of TensorFlow that makes it easy to construct, train and deploy object detection models. At Google we’ve certainly found this codebase to be useful for our computer vision needs, and we hope that you will as well.

TensorFlow 1 Detection Model Zoo

We provide a collection of detection models pre-trained on the COCO dataset, the Kitti dataset, the Open Images dataset, the AVA v2.1 dataset the iNaturalist Species Detection Dataset and the Snapshot Serengeti Dataset. These models can be useful for out-of-the-box inference if you are interested in categories already in those datasets. They are also useful for initializing your models when training on novel datasets.



NVIDIA Jetson Nano with TensorFlow Object Detection 

Inference Performance

Model : SSD-Mobilenet-v2  ( TensorFlow FP32 ) :   2 - 3 FPS



NVIDIA Jetson Xavier NX with TensorFlow Object Detection

Inference Performance

Model : SSD-Mobilenet-v2  ( TensorFlow FP32 )  :  5 - 7 FPS


Compare NVIDIA Jetson Nano vs. NVIDIA Jetson Xavier NX





Performance Comparison





NVIDIA Jetson Nano Inference Object Detection with DetectNet ( FP16 )
https://youtu.be/ZkYdhTRM9FM

Nvidia Jetson Xavier inference object-detection with Detectnet ( FP16 )
https://youtu.be/xu-GKSMTf84

Nvidia Jetson Nano Tensorflow Object Detection ( FP32 )
https://youtu.be/gpVDLLxj0J0

Nvidia Jetson Xavier NX Tensorflow Object Detection ( FP32 )
https://youtu.be/bhpHHe5aBWg

Reference

TensorFlow Model Zoo

Jetson Inference




Adun Nantakaew อดุลย์ นันทะแก้ว 081-6452400
LINE : adunnan

    

Posted by Android Arduino at 9:18 AM
Subscribe to: Posts (Atom)