We are proud to present you with a new Savant version — 0.2.5. We worked on the release for more than 2.5 months. It contains significant changes, new features, and bug fixes in several fields, but primarily, we improved developer experience and deployment features.
Quick links:
- GitHub: https://github.com/insight-platform/Savant/
- Website: https://savant-ai.io/
- Join Discord, follow on X
- Documentation
Let us begin with the most exciting updates we want to share.
Brand-New DeepStream 6.3
Nvidia released a fresh DeepStream SDK — 6.3. Savant 0.2.5 is based on the release. DeepStream 6.3 introduces a couple of bug fixes we wanted to integrate into Savant, so we switched from 6.2 to 6.3:
- H.264 encoder problem fix;
- JPEG encoder now supports dGPU-assisted encoding; in the previous DS releases, JPEG encoder was software-based.
However, DS 6.3 still has annoying unfixed bugs:
- Streams with B-frames can encounter frame disordering on Nvidia Jetson (Savant implements a workaround);
- Bug in
nvstreammuxelement causing duplicate events sent (Savant implements a workaround).
Developer Experience
Our team believes that computer vision pipelines must be easy to develop, but there is always a trade between complexity and performance and simplicity and inefficiency.
DeepStream is a very complex technology: people familiar with basic computer vision solutions like PyTorch may find it painfully difficult to develop the pipelines for DeepStream. But there is a reason for it: DeepStream is a parallel computing system providing very advanced logic for stream processing. Vanilla DeepStream requires enormous efforts to develop advanced pipelines, starting from understanding GStreamer and ending with inconvenient tooling and lack of integration.
In Savant, we strive to overcome the burden of such a complex environment. To make the development experience friendly, we implemented five features to increase developer productivity. The release introduces new advanced tooling helping to develop pipelines right from PyCharm or VS Code IDE. We also extended the documentation to practically demonstrate the use of various features (incl. video guides).
OpenTelemetry Support
Savant now supports OpenTelemetry, which makes it possible to instrument every frame processed by a pipeline. It enables profiling the code, assigning logs and attributes to traces, and analyzing pipeline errors and bottlenecks. We provide a sample demonstrating the use of OpenTelemetry. In Savant, OpenTelemetry can be used in sampled or full modes. Read more on OpenTelemetry support.
DevServer
The feature enables hot Python code reloading upon change. It helps to experiment, debug, and develop faster with just IDE. You don’t need to rebuild and restart the pipeline to have an effect.
Client SDK
Client SDK is a Python library that helps establish the interaction with a pipeline (ingesting data and retrieving processing results) with a dozen lines of simple Python code. It integrates with OpenTelemetry, allowing access to the particular frame processing logs from the programmatically.
Client SDK can be used to develop unit tests, full-featured integration tests, and custom adapters. The SDK is available in sync and async implementations.
Fancy Logging
We added color-decorated logs to help developers analyze them faster.
URI-Input Script
We packed Savant with a handy script to ingest video or images to the pipeline right from the development environment without the need for launching standalone adapters.
Improvements In Adapters
We implemented several enhancements to adapters.
🆕 Kafka/Redis Adapters
We published technology preview Kafka/Redis adapters in the release. They enable scalable processing required to handle many streams in high-load, scalable environments. The adapters handle metadata with Kafka and video with Kafka or Redis (or other Redis-compatible storage like KeyDB).
🆕 Multi-Stream Source Test Adapter
To help developers measure the performance of their pipelines and tune parameters, we implemented a utility adapter ingesting multiple streams in the pipeline. The adapter can create a desired parallel workload to help understand how the pipeline behaves under the workload and discover optimal performance configuration (batching, number of streams, etc.).
Upon completion, it can send a shutdown message to a pipeline to signal that testing is over.
🆕 Custom Adapter Development Toolkit (Client SDK)
Client SDK, described in the previous section, gives excellent tools to develop custom adapters quickly.
Multi-Streaming Always-On RTSP Sink Adapter
Previously, developers needed to launch an instance of AO-RTSP Sink Adapter per stream to cast the resulting videos. Now, you can launch a single adapter and specify multiple streams simultaneously.
HEVC Encoding For Industrial GigE Cams
With the feature, streams retrieved from GigE Vision cams can be HEVC-encoded and sent with narrow band channels if necessary. The feature may improve performance when video compression doesn’t decrease the quality.
Deployment
We improved how the module Docker image signals the container environment about its initialization process. Now, developers can utilize the keep-alive features of Docker to understand when the pipeline is ready to handle data.
Health status is available with the filesystem object and HTTP URL.
Demos and Samples
We believe samples are crucial to new users who want to dive into Savant fast; thus, every new release of Savant comes with additional samples. We are improving previously published samples to reflect industry changes.
Instance Segmentation Demo
The demo features the YOLOV8M-Seg model. The pipeline can reach 100 FPS and more on a modern GPU like RTX4000 or A4000. The demo is in the Savant repo.
Facial ReID Demo
The demo features YOLOV8-Face, AdaFace, and HNSWLib to recognize people on video. The demo is in the Savant repo.
Age/Gender Recognition Demo
The demo is based on YOLOV8-Face and a custom model based on MobileNet V2. The demo is in the Savant repo. The video footage is available by link here.
Traffic Meter Sample Enhancement
We also extended the traffic meter demo to use YOLOV8S for detection. Now it supports PeopleNet, YOLOV8M, and YOLOV8S on your choice. The demo is in the Savant repo.
Utility Samples
The OpenTelemetry example shows how to use it. The sample is in the Savant repo.
The client SDK, OpenTelemetry, and DevServer features are demonstrated in the template project.
RTSP Cam Compatibility check sample. We created a sample showing how to test an RTSP cam for compatibility and stream quality.
Kafka/Redis Ecosystem Sample. The sample shows how to deploy Kafka/Redis source, sink adapters, ingest and access data.
Bugs
The release comes with 14 bug fixes.
Documentation
A vast documentation update comes with Savant 0.2.5. Among the most important updates are:
- development environment configuration for VS Code;
- developer tutorial covering our approach to develop pipelines for Savant efficiently in PyCharm and VS Code (+ video);
- advanced topics covering non-trivial problems of pipeline development;
- adapter documentation updated.
Other Enhancements
Rust-based Transport Protocol. Previous Savant versions were based on Avro. However, we have found that AVRO is not fast enough for our needs, and we have concerns about the GIL release with Python AVRO. To address the possible problems, we re-implemented the protocol with the Rkyv crate and implemented proper GIL releasing when necessary.
Element Groups. The feature is described in detail here.
Advanced Logging Configuration. Now, log levels can be configured per code scope. Find out more in the documentation.
H264 Software Encoder. Hardware encoder is unavailable in dGPUs like A100, V100, and Jetson Nano V2. To fully support these devices, we implemented support for the software-based encoder. More on the software encoder here.
Frame Sequence Validation. When real-time adapters begin to lose data because of pipeline bottleneck, a special WARN message appears in logs.