When deep neural networks are evaluated with the CUDA runtime, the input of the model and its output are allocated in the GPU memory. The next step is to extract high-level data like bounding boxes, attributes, or masks from raw GPU-allocated tensors.Continue reading A New Feature: Accelerated Model Output Post-Processing With CuPy
Transformer models become gradually more popular in computer vision. Even a couple of years ago, nobody broadly used transformers for computer vision. However, transformers have significantly changed the landscape of sophisticated deep learning solutions, primarily in natural language processing and generative AI.Continue reading Running The RT-DETR Detection Model Efficiently With Savant
Savant uses the TensorRT engine for model serving. TensorRT is an optimization ecosystem compiling models into highly efficient engines optimized for particular hardware, batch size, and precision. After that procedure, the model’s inference speed increases significantly, but the process is resource- and time-consuming.Continue reading New Feature: Neural Network Compilation Mode
Monitoring as a part of software observability is crucial for understanding the state and health of the system. Video analytical and computer vision pipelines also benefit from monitoring, allowing SRE engineers to understand and predict system operation and reason about problems based on anomalies and deviations.
Computer vision pipelines represent complex software working in a wild environment, requiring continuous observation to understand trends and correlations between internal and external factors.Continue reading Pipeline Observabilty With Prometheus And Grafana
TensorRT is a high-performance deep learning inference optimizer and runtime library developed by NVIDIA. It’s specifically designed for production environments and optimized for NVIDIA GPUs. The primary goal of TensorRT is to accelerate deep learning inference, which is the process of using a trained neural network model to make predictions based on new data.Continue reading How To Deploy And Serve Computer Vision With TensorRT Efficiently
NVIDIA DeepStream SDK is a game-changer technology for deep neural network inference in computer vision served with NVIDIA hardware. The optimized architecture accounts for the specifics of NVIDIA accelerators and edge devices, making pipelines work blazingly fast. The core of the technology is TensorRT, which consists of two major parts: the model optimizer, transforming the model into an “engine” heavily optimized for particular hardware, and the inference library, allowing for rapidly fast inference.
Another DeepStream’s killer feature is connected with the CUDA data processing model: computations are carried on with SIMD operations over the data in a separate GPU memory. The advantage is that the GPU memory is heavily optimized for such operations, but you need to pay for it by uploading the data to the GPU and downloading the results. It can be a costly process involving delays, PCI-E bus saturation, and CPU and GPU idling. In the ideal situation, you upload a moderate amount of data to the GPU, handle it intensively, and download a moderate amount of results from the GPU at the end of the processing. DeepStream is optimized and provides developers with tools for implementing such processing efficiently.
So why do developers hesitate to use DeepStream in their computer vision endeavors? There are reasons for that we will discuss in further sections, and find out how to overcome them.Continue reading Why Choose Savant Instead Of DeepStream For High-Performance Computer Vision
Savant 0.2.6 was released on November 8, 2023. The release includes multiple bug fixes, seven new demos, and many other enhancements, including documentation, benchmarking, and Jetson Orin Nano support.
Savant crossed the 300-star band on GitHub, and Discord is now active. The work on the release took 1.5 months. In the following sections, we will cover essential parts of the release in detail.Continue reading 🚀 0.2.6 Release Notes
Specific Nvidia platforms do not support the Nvidia encoder (NVENC) device. Datacenter accelerators like V100, A30, A100, and H100 cannot encode video streams. The situation also transpires to the edge: Nvidia Jetson Orin Nano does not include NVENC either.Continue reading New Feature: GPU-less Always-On RTSP Run Mode
When estimating the object’s pose, you usually deal with keypoint detection. It can met in various situations like facial recognition to align the face based on pose estimation or action recognition to estimate the actions and analyze them.Continue reading How to Implement High-Performance Keypoint Detection
In an era where data-driven decisions are paramount, the ability to accurately count and monitor people in specific areas has become invaluable. Nvidia’s PeopleNet neural network offers a solution for this. Aside from PeopleNet, other models can be used like the representatives of the YOLO family. As a component of Nvidia’s DeepStream SDK, which specializes in AI-driven video analytics, PeopleNet boasts training on extensive datasets, ensuring high precision in detecting individuals even in intricate or densely populated scenes.Continue reading How To Count People In Polygonal Areas with Savant