We often see people starting their Nvidia Jetson-based projects with RTSP cameras without a real need for them. In this article, we explain the advantages of USB/CSI cameras for projects working on Nvidia Jetson Orin hardware.
If you are not familiar with USB and MIPI CSI-2, read the article to learn more.
Let us begin with why people choose RTSP over USB/CSI-2 cameras in their projects. While it is hard to give a qualified answer, we suppose the reasons are:
- Supply chain availability (you can buy RTSP CCTV cameras everywhere);
- User-friendly configuration stack based on a Web GUI;
- Lack of knowledge about USB/CSI-2 cameras based on a bias that those cameras are designed to call a buddy via Zoom;
- Non-user-friendly configuration for USB/CSI-2 cameras.
What is wrong with RTPS cameras? They are built for CCTV, not computer vision… So,
- they are frequently designed for a large FOV by using typically 2.8 or 3.6mm lenses; thus, they do not fit most computer vision tasks requiring high pixel density for a small real-world object;
- very small fraction of them allow replacing their lens with a custom one;
- they are not optimized for fast-moving objects when the task requires pixel-perfect images because of low FPS, cheap sensors with a rolling shutter, and low-quality lenses;
- they are built with wide-angle lenses having a significant distortion increasing from center to edges;
- they have an excessive software stack because they need to deliver video in H.264 or HEVC formats over the network; thus, they are typically less stable and often introduce performance decline in the long run (outside temperature, sensor, chip heating, software bugs);
- H.264/HEVC encoding and RTSP protocol result in a greater frame delivery latency, which can reach up to 2 seconds in some cases.
Does it mean that you should not use RTSP cameras in computer vision applications? It is unnecessary because they often do their job well when their properties fit the project requirements. However, when choosing a camera for an embedded, autonomous project requiring Nvidia Jetson hardware, there are better options, allowing for the implementation of a solution with higher quality, better performance, and lower complexity. The better option is to consider a USB/CSI-2 camera for the project. Let us summarize the advantages of such a choice:
- MJPEG images or raw pixel formats allow for low latency processing because the encoding is simpler, and decoding is carried out by a dedicated MJPEG unit (NVJPEG), which has abundant performance and very low latency even for 4K images;
- USB/CSI-2 cameras are connected with a simpler wiring interface without the need for network infrastructure and a separate power line; at the same time, the power is delivered to them over the interface, and users can initiate restarting just by power cycling them;
- There are affordable options for cameras with a global shutter and high fps enabling applications requiring precise recognition of fast-moving objects;
- There are affordable cameras with a very high resolution of up to 48MP (8000×6000 px);
- There are affordable options for cameras with or without a case, enabling building custom devices;
- You can customize the camera by installing custom filters (like IR-filter) or choosing opinions without filters;
- You can install any industrial-grade lens to customize FOV and pixel density to your needs.
Let us get back to NVJPEG just for a second; even for a 48MP camera, Jetson Orin NX/AGX gives you 2×600 MP/s NVJPEG decoding performance, allowing handling 25 FPS in real-time.
As you can see, USB/CSI-2 cameras better fit embedded/industrial applications requiring robust operation, low latency, and high performance. They help build specialized optical systems and significantly surpass their RTSP competitors in non-CCTV applications.
A Note On Stereo-vision and Synchronous Image Stitching
Of course, high-precision stereo vision and image stitching are impossible with RTSP cameras; even by utilizing RTCP Sender Reports, you can achieve a lag less than or equal to 1000/FPS ms, which is often insufficient. With USB/CSI-2 cameras, you can find specialized stereo-bundles like Intel Real Sense or solutions from other vendors.
