In video streaming applications, reproducing the results is crucial for quality estimation, troubleshooting, and code improvement. Unfortunately, it is not easy because data is streaming. Thus, developers need utilities that allow them to record traffic and replay it at the same rate and timing to simulate the real sources. Sometimes, developers can use video files instead of live sources like RTSP or USB cameras.
However, in many situations, they must process many streams synchronously to reproduce the particular behavior they encounter in their pipelines. It is a very challenging task with live sources.
Savant now provides a solution for this situation: we implemented a new feature for a buffer adapter, allowing the dumping of passing-by traffic for later replays. When the dumps are created, the buffer adds exact timestamps to the packets, which allows the replay of those dumps as if they go from live sources. With the new feature, developers can record traffic from many RTSP or USB cameras and replay the same situation as many times as needed to optimize and troubleshoot their algorithms.
The following systems benefit from this new feature a lot:
- Autonomous systems and transportation: these systems have many difficulties related to traffic collection, synchronization, and cloud delivery.
- Enterprises with limited access: Security departments usually do not allow access to live traffic from those organizations due to compliance rules, data security, privacy protection, etc. However, they allow limited, controlled data collection, and developers must use a maximum of those options.
How To Record The Traffic
You need to deploy a buffer between live sources and the downstream pipeline. Please refer to the sample showing buffer deployment in the Savant repository. The buffer serves both the purpose of a backpressure mechanism and records the traffic for future needs. You can launch an end-to-end pipeline or just a buffer without downstream elements. The buffer does not require GPU, so it can be deployed on a regular X86 or ARM64 device.
Normally, the buffer does not record dumps; you must enable it. It is configured with the following environment variables:
MESSAGE_DUMP_ENABLED: a flag indicating whether to dump messages to a file; default isFalse;MESSAGE_DUMP_PATH: a directory to dump message segment files; default is/tmp/buffer-adapter-dump;MESSAGE_DUMP_SEGMENT_DURATION: a duration of a message segment in seconds; default is60;MESSAGE_DUMP_SEGMENT_TEMPLATE: a template for message segment file names; default isdump-%Y-%m-%d-%H-%M-%S.msgpack.
The buffer does not remove old segments automatically; you must implement it separately.
How To Replay The Traffic
We implemented a source adapter which can replay the dumps in one of two modes:
- as fast as possible;
- as it was recorded (time-synchronized).
The new adapter is the Message Dump Player Source Adapter. It accepts a playlist containing a line-separated list of segments to play and can be used anywhere you use any other source adapter.
Do not hesitate to join Discord to get more Savant-related information and help.