After eight months of development, we proudly present you with Savant 0.5.1. The new Savant is based on DeepStream 7.0 (Savant 0.4 is based on DeepStream 6.4) and includes new samples (34 ready-to-use real-world samples), enhancements, and bug fixes.
We recommend using Savant 0.5.x for the production environment and new projects. Version 0.4.x becomes legacy and will not get updates except for critical bug fixes.
New Features
Key-Value API/ABI. Savant focuses on efficient computer vision and video analytics. To achieve that, we implement API and programmatic interfaces, ensuring non-fast, reliable, and blocking operations to avoid extra costs.
Embedded Key-Value API provides developers with the following features:
- Access to the local Key-Value store from Python without any network-related delays;
- Access to remote Key-Value stores from Python through standard REST API, unlocking pipeline interaction with third-party components;
- A Protobuf-based protocol allows the implementation of arbitrary structures for complex data schemes.
- Non-blocking programmatic event log subscription from PyFunc.
- Remote event log subscription with Websocket API.
This feature establishes a new, highly optimized way for the pipeline to communicate with external components in two directions:
- Inbound: receive settings, updates, and external data;
- Outbound: provide metadata for third-party consumers through REST API and websocket subscription.
We implemented a new sample showing KVS.
Frame processing in original resolution. DeepStream (6.x) required that all frames be scaled to a statically configured common resolution before the processing. It was not very convenient for many reasons.
In DS7.0, processing batches can contain frames in non-unified resolutions. Thus, we added the feature to Savant as well. With Savant 0.5, you can:
- Avoid resolution scaling at all, processing the frames in the original resolution. Check out the sample implementing this approach.
- Use a user-defined shaper that triggers when a new stream appears, allowing the user to specify a custom scaling and padding based on stream properties. Check out the sample implementing this approach.
OpenTelemetry tracing enhancement. Savant has supported tracing for a while, but previously, we only supported the Jaeger trace format. Now, we support OpenTelemetry native trace format so that you can send traces not only into Jaeger but to other OpenTelemetry collectors of your choice. We extended the telemetry functionality to support secure communication with OpenTelemetry collectors with X509 authentication and encryption. Check out the sample implementing the telemetry for details.
Etcd security enhancement. Savant has supported Etcd for configuration updates for a while, but previously, we only supported Etcd without authentication and encryption. In Savant 0.5, we extended the Etcd configuration to support X509 authentication and encryption.
Low-latency configuration options. We implemented a new configuration property, switching the pipeline into a low-latency mode by enforcing NVDEC to buffer fewer frames before decoding.
Better metrics support. We moved metrics webserver implementation from Python to Rust to decrease their influence on pipeline performance. The metrics are available in Prometheus format.
Stage latency stats. We extended metrics to monitor the latency of data processing at every stage. Those stats are printed in logs and exported as Prometheus metrics.
REST Shutdown API. REST API now supports a special shutdown handler allowing users to stop/restart pipelines remotely.
Batch input. With Savant 0.5, you can ingest synchronized video data in the pipeline. You can prepare batches with Client SDK and send them as a whole to the pipeline. If the batch size is configured properly, all those frames will reside in a single processing batch so you can implement the processing required for such an operation (e.g., stereo vision).
CUDA attribute model output converter. A new converter allows getting model output layers in the form of CuPy tensors without downloading them to CPU RAM. The use of the converter is demonstrated in the NanoSAM sample (see below).
Message dump/re-play support in the buffer adapter. The buffer adapter can now record all bypassing messages to special files to replay later. It allows for the implementation of reproducible streaming environments for testing and benchmarking purposes. Read more in the article.
WSL2 support. DeepStream 7.0 supports the WSL2 environment, and Savant supports WSL2 as well. Unfortunately, we do not have enough capacity to support WSL2 as a first-tier platform, but we appreciate your input and testing results.
New Samples
NanoSAM. We implemented a new sample showing how to use NanoSAM with Savant.
Bypass model. We received several user requests about how to analyze how the Nvinfer preprocessing works, so we implemented a sample showing the identity model, allowing efficient preprocessing of the analysis.
Bugfixes and Minor Enhancements
Savant users reported multiple issues related to specific operational environments, and we fixed many of them in Savant 0.5. We appreciate user input, allowing us to make Savant a better computer vision framework.
Future Releases
We will release a number of updates with features and fixes in the 0.5.x branch.
The development will switch to the Savant 0.6 cadence, introducing a new API for working with metadata and model inferencing. Savant 0.6 is expected in 2025, but the final date is unknown.
Do not hesitate to contact us on Discord to get more information about Savant and to progress with community help quickly.