Performance Testing with Video Files

Introduction

When validating BrainFrame's performance for a given use-case, you may want to use video files to simulate connecting to many IP camera streams. The built-in video file support in BrainFrame is good for some kinds of testing, but is not recommended for performance testing because it has significant overhead when compared to IP cameras.

For usecases like this, BrainFrame ships with a small RTSP server utility. This utility acheives significantly lower overhead by transcoding video files in advance. However, considering the main focus of this exercise is performance testing, we highly recommend running the RTSP server on a separate machine from the one running the BrainFrame server.

Using the RTSP Server Utility

Start by creating a directory containing the video files you would like to test with. Ensure that no other types of files are present in the directory. Then, run the following command:

docker run \
  --network host \
  --volume {video file path}:/video_files \
  aotuai/brainframe_core:0.29.2 brainframe/tools/rtsp_server/main

Replace {video file path} with a fully qualified path to the video file directory you've created.

The RTSP server will start by transcoding all video files in the directory to a known good format. This is to prevent video format incompatibilities and can significantly improve streaming performance for the RTSP server. Once transcoding is complete, new .mkv files will be created in the video file directory. Transcoding will not be run again unless new video files are introduced or the .mkv files are deleted.

When all video files have been transcoded, the RTSP server will start. An RTSP URL will be printed for each video file being streamed.

INFO:root:Video traffic_back_to_front.mkv available at rtsp://0.0.0.0:8554/traffic_back_to_front
INFO:root:Video test_store.mkv available at rtsp://0.0.0.0:8554/test_store
INFO:root:Video two_cool_guys.mkv available at rtsp://0.0.0.0:8554/two_cool_guys

Be sure to replace 0.0.0.0 with the local IP address of the machine that's running the RTSP server.

Connecting to Many Streams

When doing performance testing at the level of tens to hundreds of streams, it can become burdensome to manage that many video files. Instead, it may be easier to connect BrainFrame to the same video stream multiple times.

BrainFrame does not allow you to connect to the exact same RTSP URL multiple times, as doing so during standard operation is wasteful. However, you can work around this limitation by adding dummy query parameters to the end of the RTSP URL.

rtsp://0.0.0.0:8554/test_store?dummy=1
rtsp://0.0.0.0:8554/test_store?dummy=2
rtsp://0.0.0.0:8554/test_store?dummy=3
rtsp://0.0.0.0:8554/test_store?dummy=4
...

The RTSP server utility is optimized to support many concurrent connections to the same stream. Make sure your local network has the necessary bandwidth to facilitate the scale of testing you plan to complete.