Computer vision and artificial intelligence inference pipelines often begin as straightforward, statically defined conveyers, but soon begin to require additional heuristics to optimize their performance and adapt to real-life conditions. Let us consider several situations which may require pipelines to be reconfigured dynamically. We will not cover the cases that require parameters applied for good: as they can be passed to the pipeline through environment variables or configuration files.
Also, it is worth remembering that static reconfiguration requires pipeline restart to apply new parameters. Thus, for the environment where a single pipeline instance handles multiple video streams may be an unacceptable variant because during the pipeline restarts the video from all the streams stops being processed.
The Region Of Interest change. If a user can arbitrarily change the region of interest for cams it can be convenient to apply such changes dynamically, without pipeline reload.
Temporary deactivation of a particular stream. Sometimes, it may happen that the pipeline must be disabled for a certain stream for a period of time according to an external condition; other streams remain being processed. E.g., the situation may happen when you have a lighting sensor signaling that outdoor illumination is too low to allow for the pipeline to work correctly. In this situation, you would like to mark the pipeline as blacklisted until the sensor allows for processing again.
Confidence update based on an external model. In this hypothetical situation, the edge device can receive an intelligent confidence update as feedback from the work of an advanced model located in the data center.
The Problem
The obvious solution is to allow the pipeline to make active polling of an external system for updates. However, this approach is inefficient as it introduces extra active operations, blocking the execution of the main code and requiring handling various situations related to the external system state.
In Savant, another solution is used to get updates about dynamically changing parameters – Etcd. It is the state-of-art system for distributing the configuration to acting agents. The most important feature of Etcd is its capability to provide updates with callbacks watching configuration nodes. As a result, it doesn’t require the system to poll for updates but access them instantly and locally.
The Use Of Etcd In Savant
Savant’s YAML manifest supports Etcd out of the box: you can configure it as follows:
# base module parameters
parameters:
# Etcd storage to manage processing sources
etcd:
# Etcd hosts to connect to
hosts: [etcd:2379]
# Path in Etcd to watch changes
watch_path: savantFor the full list of parameters, please refer to the definition for Etcd. After the Etcd is configured you can use a special function to access the watched parameters:
from savant_rs.utils import eval_expr
(val, is_cached) = eval_expr(f'etcd("relative/path", <DEFAULT_VALUE>)')
# is_cached == True, if the result of eval_expr is cached, not actually evaluated
# is_cached == False, the result is computed
# DEFAULT_VALUE is a type to cast the value retrieved from "relative/path", e.g.
# - eval_expr(f'etcd("relative/path", 0)') - integer
# - eval_expr(f'etcd("relative/path", "true")') - string
# - eval_expr(f'etcd("relative/path", true)') - bool
#The eval_expr is part of a bigger picture that will be disclosed in future Savant releases, but as for now, you can just use a wrapper to implement accessing Etcd with it.
Ready-To-Use Sample
We have a ready-to-use sample allowing you to try how it works in practice. The sample is “conditional_video_processing“, available on Savant GitHub.
Don’t forget to subscribe to our X to receive updates on Savant. Also, we have Discord, where we help users.