Automating a difficult and complex process in a way such that the overall efficiency is increased, while still maintaining the entirety of its previous functionality, is one of the more rewarding achievements an engineer can aspire to. The key, and subsequently the challenge, is usually to find or invent a solution that streamlines a given manual operation without sacrificing its inherent flexibility. During the second biannual Tekniksprint of 2018 a small team from SVT Produktion & Teknik managed to achieve a feat that approximates something not entirely unlike just that.
You see, we have developed LEIF
Artwork by Lina Nord
Before I go into the nitty gritty details of what LEIF is, and more importantly why it was sorely needed in our workflow, let me set the stage by presenting a brief summary of the background story.
As you may, or may not, know we transcode hundreds of videos every day that are intended for online streaming at SVT Play. The actual number of videos is steadily increasing every year and this trend is not likely to go away anytime soon. Under such circumstances, coupled with the fact that the various input sources we handle range from single clips posted by individual journalists to big budget cinematic production masters delivered by major studios, affirming video quality assurance is becoming more and more demanding.
Furthermore, in this context it is important to clarify another thing: Usually, when a reference is made about “video quality” in general, the reader assumes that the author is talking about the acquired visual quality of an encode (or more commonly the retained visual quality of a transcode), but in this case we are dealing with something much more basic. In essence, a rudimentary quality control of a transcoded video ensures that there are actual valid video- and audio streams within the file itself, that the parameters of said streams are confined within specified limits and finally that their respective durations are correct in relation to the original.
In other words; “Video quality control” in this context refers to the act of ensuring that a media file is complete, uncorrupted and technically acceptable, as opposed to having achieved some level of a quality along a given metric (i.e. VMAF, PSNR or SSIM). With that out of the way, and without further ado, let me explain what LEIF is.
LEIF (an abbreviation of LEIF Enigmatic Indication of Faults) is an automated video quality control tool developed at SVT Produktion & Teknik. That is to say, LEIF is able to compare a transcoded file with its original source and determine whether or not basic technical parameters are correct.
For instance, LEIF will detect if a file does have…
- A video stream (that is not all black)
- An audio stream
- The correct audio channel layout
- The correct frame rate
- The correct aspect ratio
- Video free from unintended black frames
- Video free from unintended duplicate frames
- Audio free from unintended digital silence
- Interlaced frames
- The correct duration
If the answer is *negative* to one or more of the above checks, LEIF will report the error/s to the relevant department and let them determine how to proceed.
At first glance, the above checks might seem painstakingly rigorous, bordering on unnecessary given that most of them should be taken care of through options in the transcoder, *should* being the operative word here. However, with the sheer volume of media files being transcoded every day for SVT Play, we have realised that in our case “should be” is far further away from the “will be” than we would like. There are a great many edge cases that exist between the myriad of codecs and formats our transcoders take as input and technical oddities that occur as part of a rounding error once every 10000th transcode. Thus there are minor incidents on a weekly basis, where a transcoder has produced an unacceptable media file without throwing an error or reporting a warning. This is made even trickier by the fact that the erroneous final media asset in question might actually be technically correct, whilst still unacceptable for streaming purposes. Let me give you a few examples to sort of highlight what I mean:
For instance, a technical mishap within a complex export process from an editing station to the transcoder, as the file passes through the production network on premise, can result in a media file with correct audio but pitch-black video. That is to say that the transcoder will happily take the source file and process it without throwing an error, since all parameters are indeed technically correct, but in reality the file lacks video content.
Another issue that can occur on rare occasion is that the transcoder starts its process too early, perhaps as a result of an unintentional interruption of the source file transfer to a given watch folder, without proper cleanup. This in turn would result in a file that is technically acceptable, but also has the wrong duration.
Finally an all too familiar kind of issue comes courtesy of classic human error. A stressed-out editor might make the easy mistake of not connecting two clips together tightly enough on the timeline in their NLE, resulting in a few pitch-black frames between those clips. This is obviously nothing major in the grand scheme of things, but nonetheless a disturbing nuisance for any viewer and thus worthy of proper resolvement.
Now that you know what LEIF does and why we need it, let me give short explanation of how it works.
The technology behind the QC-aspects of LEIF, which is arguably the heart of the tool, is actually quite simple. Based around some clever usage of the ubiquitous framework FFMPEG, LEIF is able to run relatively short and readable command lines for each check and evaluate the output accordingly. So by utilizing existing functionality within said framework we gain two important advantages. First and foremost, anyone with intermediate knowledge of its usage (coupled with a basic understanding of video encoding and media analysis of course) can contribute with new features. Secondly due to the modular nature of LEIFs architecture, it benefits from new and improved features within FFMPEG while not being entirely dependent on any one specific function that might be deprecated in the future.
To be completely transparent however, it should be noted that we have also added a minor pinch of classic SVT magic into the mix, which slightly extends the capability and performance of the the tool.
LEIF is currently fully integrated in, and working at full speed as a part of, our production pipeline (after a semi-rigorous testing period where our self-proclaimed collective hubris took a much deserved hit having to iron out disastrous bugs). The beauty of said integration is that the QC-process itself runs parallell to the main pipeline, which in turn means that all of the aforementioned checks are done without hindering or slowing down the main workflow.
We are already enjoying measurable and beneficial results such as having detected errors in a video file and thus indirectly “saved” programs, as well as clips, a few days prior to their publishing.
One might wonder if we are so pompous as to claim that we have developed a bullet-proof solution which ensures that we will never publish a corrupt video ever again. This is obviously not the case, there are still several ways in which an error can slip through the virtual cracks between components within our production workflow. However, we are of the opinion that it is immensely better that we do some rudimentary checks for each and every video file we transcode, rather than having no checks at all.
In many ways LEIF is a Highway, and LEIF always finds a way.