The Savant ecosystem, known for its high-performance real-time video analytics platform, has just expanded with the release of Router, a new microservice designed to simplify and optimize the routing of video streams across complex computer vision pipelines and auxiliary services.
Router documentation can be found at the link: https://insight-platform.github.io/savant-rs/services/router/index.html
Why Routing Matters in Complex Real-Time Video Analytics
The router is typically found in complex computer vision and video analytical pipelines. When the pipeline is serving a linear upstream-to-downstream processing for all streams, the router is generally not needed. However, when multiple perceptions conditionally handle streams or heuristics are applied, the router becomes a crucial element of the processing graph.
As real-time vision systems grow in scale, engineering teams often face a familiar problem: how to flexibly distribute and direct multiple video streams to the right downstream modules for specialized processing — all while keeping the system performant, scalable, and maintainable.
For example:
- Direct particular camera feeds to a vehicle recognition model when a car appears.
- Duplicate streams for parallel analytics: one for full processing, another for key frame processing.
- Load-balance multiple streams across GPUs to optimize resource use.
- Stream downsampling by message filtering and tagging.
Before the Router, implementing these patterns required custom logic or external brokers. Router changes that introduce a purpose-built, content-aware routing layer fully integrated with Savant’s architecture.
What the Savant Router Does
The router sits at the heart of a Savant deployment as a high-throughput message switch, enabling:
- Dynamic, conditional routing based on inference metadata and labels.
- Fan-in and fan-out of multiple streams — ingest many, route to many.
- Python- extensible logic hooks for custom business logic.
- Load balancing and stream duplication for horizontal scaling.
- Backpressure control to prevent overloaded pipelines.
- Fault isolation by decoupling routing logic from inference engines.
With the Router in place, teams can design flexible, modular pipelines where video ingestion, inference, routing, and archival components are all loosely coupled yet seamlessly integrated.
Built for Scale and Performance
The router is engineered for real-world deployment at scale:
- Core implemented in Rust for low-latency, multi-core performance.
- Fully compatible with ZeroMQ-based Savant protocols.
- Operates entirely in memory.
- Supports x86_64 and ARM64 (including NVIDIA Jetson devices).
- Simple Docker-based deployment with YAML/JSON configuration.
- Stateless design with minimal operational overhead.
Benchmarks show the Router can handle thousands of routed messages per second while maintaining low CPU and RAM usage, critical for modern video inference pipelines that must process hundreds of streams concurrently.
Enabling New Pipeline Architectures
With Router, ML engineers and CV developers can now easily implement:
- Selective inference pipelines that route only relevant content to required models.
- Multi-model fan-out, where one video stream feeds multiple AI models simultaneously.
- Edge-to-core distributed processing, routing streams across cluster nodes or data centers.
- Resilient pipelines, where failures in one module don’t cascade system-wide.
Router slots naturally into Savant’s growing service lineup alongside the Retina RTSP adapter for stream ingestion and Replay for video archiving, further strengthening Savant’s position as a full-stack solution for real-time AI video systems.
A Key Step Forward
Savant Router is a highly pragmatic addition to the ecosystem, addressing one of the most common pain points in real-time computer vision deployment. Rather than building and maintaining fragile custom routing logic, teams now have a battle-tested, open-source solution built specifically for their use case.
For engineering teams building scalable, distributed inference pipelines, Router may well become one of the most valuable new tools in the Savant toolbox.