The Ventuz Video Engine

Introduction to the Video Engine

It is important to note that the video engine actually includes a number of elements, including audio, live video coming from external sources, streaming video, video clip playback and the entire video output system. All audio processing inside of Ventuz is also handled by the video engine.

The configuration properties for all audio and video subsystems, including their routing, is done in the Device Configuration.

Render Pipes

A Render Pipe can be described as another instance of the Ventuz Runtime Engine within the Ventuz Runtime; A second pipe is able to load and display a Scene independent of the first Pipe. A complete Scene Tree with the Layout Scene is independent of other Scenes loaded in other Pipes. Therefore, changes to the Scene Data can be made without affecting the other Pipe. This is especially useful in Ventuz Director enabling data in the Preview Pipe to be manipulated while a different scene is playing out on the Program Pipe.

Render Pipe configuration takes place in the Device Configuration window. Multiple pipes can be added to each machine's Runtime. Depending on the Ventuz license, there is no limit to the number of Pipes that can be configured. The upper limitation on Pipes are realized through the system's hardware limitations.

It is important to know, that the handling of rendering pipes is independent of the Ventuz Scene itself. There is no need to configure a Scene to work inside of a specific Pipe. If needed, the SystemID Node can be used to retrieve information regarding which Pipe a scene is loaded, is given.

It is not recommended to use extra Pipes to render different output streams or resolutions in parallel. Since Ventuz 7, it is possible to have the Runtime Engine render the same Composition multiple times. Multi Pipes are useful when working with Ventuz Director or if you want to make use of a different input Pipe mapping in the Scenes.

It is possible to easily share data between different Pipes in the same Runtime using the Data Portal Node introduced in Ventuz Version 8.

Multi Rendering

Since Ventuz 7 it is possible to render the same scene (or whole Scene Tree) multiple times by creating a custom Render Setup; One Composition can be rendered in different resolutions or shapes (e.g. rotation or dpi). Within the engine, Ventuz will automatically render your Scene multiple times to fit the desired output. Additionally, if the Allow Scaling property is enabled, Ventuz will use very efficient shaders to scale the Composition's first render pass instead of needing to render it twice.
With these settings, it is also possible to map Scene Layers to different outputs.

Stream Modes

Every stream, depending on its hardware or software capabilities, can be configured to run in the following modes:

Name	Ports	Description
External Keying	Output Key + Fill	Output two separate SDI Streams, one containing the alpha channel (key) and one containing color information (fill).
External Chroma Keying Simple	Output Key + Fill	As above, but combines foreground, background and garbage matte from separate layers and 3D scenes into a key+fill stream.
External Chroma Keying	Output 2xKey + 2xFill	Outputs 4 signals: foreground fill, foreground key, background fill and garbage matte as key.
Internal Hardware Keying	Input + Output	Let the video hardware take the Ventuz rendering and key it onto an input signal using the linear hardware keyer of the board.
Internal Hardware Keying with Input	Input + Output	Let the video hardware take the Ventuz rendering and key it onto an input signal using the linear hardware keyer of the board. In addition, the input signal can be used as a texture in the Ventuz scene.
Internal Hardware Keying with Delayed Input	Input + Output	Let the video hardware take the Ventuz rendering and key it onto an input signal using the linear hardware keyer of the board. In addition, the input signal can be used as a texture in the Ventuz scene. The texture will be synchronized with the input keyed input, allowing seamless transition from the keyers input to 3D rendering.
Internal Software Keying	Input + Output	Do the keying with the Ventuz 3D engine instead of the hardware. This allows painting directly over the input signal, not having to care about generating correct alpha/key information. The input can also be used as a texture.
Internal Software Chroma Keying	Input + Output	As above, but the input is run through a chroma key. Also combines foreground, background, garbage matte and chroma keyed input into a single output stream.

Preview

When rendering to SDI or other off screen devices, a Preview Window is shown. The Preview here refers to quality, not a preview in the sense of previewing the next graphical element that is going to appear on-air. It is only an additional output of the signal sent to the SDI board.

Video Inputs

Ventuz will support as many concurrent input devices and streams as the hardware and operating system allow. In theory it is possible to trigger hundreds of video clips and live video streams simultaneously within Ventuz, today's hardware would not be able to handle this much bandwidth. The performance of the Ventuz Video Engine scales with the hardware that is running Ventuz Runtime. Regardless of the source of the inputs, whether that be an SDI board, a video clip or a webcam – they all are routed and processed, together with audio, through the Ventuz Video Engine.

HDR and Color Spaces

The Ventuz engine supports a full HDR workflow and color management for specified color spaces, handling incoming assets, internal rendering and output transformation.

Color Gamut, Dynamic Range and Transfer Function

The human eye is able to see a wide range of colored ​light. In detail, light is defined as the visible part of the electro-magnetic spectrum in the range of ~380nm to ~780nm. This results in a wide range of perceivable color. This range defines the basic ​CIE-1931 color space.

The perceived color can be defined by three variables. For example with a red, green, and blue part or xyY, with the chromacity in xy and Y for luminosity. Via mathematical transformation, these color coordinates can be transformed to achieve other combinations.

A color space defines the range of the chromacity and luminosity. Starting with the chromacity, the computer graphic's color space is usually defined, by 3 primary colors. Red, green and blue. The triangle in-between these primaries are the supported colors. Colors outside the triangle and the color space, are called out of gamut. Each color space also defines a whitepoint for a reference white. In most cases D65.

Wider color spaces such as Rec.2020 are able to handle a broader range of chromaticity compared to standard Rec.709. The color space most commonly used for the last several decades.

Next to the chromaticity, we perceive luminosity in colors. The human eye does not perceive luminosity in a linear way. A 25% grey is not half the luminance of a 50% grey. We are much more sensitive to luminosity changes in the lower ranges, compared to higher ranges.
When calculating in a non-linear color spaces for example, a value of 128 (out of 255) does not correspond to 50% brightness but to something closer to ~21%.

The Dynamic Range is defined by the possible range of of the lowest luminosity where shades are still visible and the highest luminosity. For a screen this is the brightness of the darkest pixel compared to brightest pixel.

Today, the general pipeline for video is to capture an image with a camera, process and store as digital data, and play back via an output device. The incoming optical information is transferred to electrical data (​OETF) and the electrical data transferred to optical again (​EOTF). These transforms are described via the color space's Transfer Function. Since the old day's CRT monitors had a non-linear EOTF, the camera also needed a specific transfer function, with the inverse to the EOTF. Since it is based on a gamma function it is also called Gamma Curve and the process Gamma Correction.

Today, the main reason to use transfer functions is due to the limitation of digital processing in terms of bit and file size. The standard was, and still is, to use 8-bits for each color channel's sample. Using 255 values and linear transfer functions would result in either very little possible dynamic range and can cause strong banding in the shadows or gradients of an image. Luckily, the human eye is not as sensitive in the highlights and the non-linear transfer function can decode the shadows with higher precision than the highlights, resulting in relatively good dynamic range with no banding on the signal.

With the aim of High-Dynamic-Range, different transfer functions may be used, including (partly-) logarithmic curves and video-scene dependent functions. Together with the usage of higher bit-rates, such as 12-bit, high ranges of of dynamics in light (0.0001 nits up to 10000 nits) are possible.

Colors and Calculation in Ventuz S

The standard for Ventuz, prior to Ventuz 8, has been to render in the fixed sRGB color space with the respective primaries and transfer functions. The color information is mapped on 8-bit integer values (0-255) with the brightest pixel is fixed to 255.

To support different color spaces and high dynamic range, it was necessary to extend the render calculation to 16-bit ​float values. The precision, step size, and the minimum and maximum values are no longer fixed. Therefore calculations outside the standard range of (0 to 1) are possible.

As described above, non-linear curves are used to enhance the efficiency of the processed data. With the switch to 16-bit float values in Ventuz 8, luminosity can be encoded using linear calculations to achieve these float values.

The primaries of the color spaces are always represented with a value of 1. (1:0:0 for red). With a change of the color space, the rendered colors also change, since the primaries differ for each color space.

The color space can be set in the Project Properties. All linear color spaces render in 16-bit float values. If the output is set to 32-bit, the whole Video Engine switches and calculates with 32-bit float instead of 16-bit.

Ventuz works scene-referred for the brightness level, where (1:1:1) is the reference white in all color spaces. It is also possible to set the reference to display-referred. This encodes the brightness as the actual luminosity measured in nits, not in relation to the reference white point. In this mode, a value correspondents to a specific brightness level in nits to the output.

GPU outputs are always display-referred with a standard white (1:1:1) of 203 nits by default. To convert from a scene-referred color space to a display-referred, Ventuz needs the brightness level for the standard white. This is given, by the SDR-Level in the Project Properties. With changing the SDR-Level, all display-referred color spaces encode in a higher or lower luminosity for the output rendering. For scene-referred output streams, like all SDI streams (except using PQ - perceptual quantizer - as transfer function), the SDR-Level is ignored.

Color Processing Pipeline

With every type of input, 3D rendering, and multiple outputs, a sophisticated color processing pipeline is necessary to handle all the different input and output color spaces and blend them correctly.

All incoming video data, such as movies, live video or any texture, uses a color processing chain to be converted into the 16-bit linear color space, selected in the Project Color space. In an HDR Layer, light calculation is done with 16-bit precision with no fixed maximum brightness level. The Material System uses the Project Color Space primaries for its color settings. If a Standard Layer is used, the internal calculation is still done in 8-bit fixed integer values with the sRGB primaries, and then processed to the linear color space of the Layer Blending. (Note: The Standard Layer's color calculation looks very slightly different compared to the whole engine is in SDR mode.)

On the output side, for each output a specific color processing is done to ensure color space flexibility and accuracy.

All color processing is done via shaders in real-time, there is no pre-loading to convert into the memory. The shaders are build upon multiple mathematical-transforms and done on the GPU per frame.
In detail:

1. An incoming stream is transformed from YCbCR to RGB channels, with a channel matrix.
   • This is only necessary for non-RGB data.
2. The non-linear signal is multiplied by the inverse transfer function of the input's color space.
   • Ventuz reads out the metadata, to automatically apply the correct functions.
   • If the input stream uses a display-referred color space, the Project SDR-Level is used to set the values for the reference white (1:1:1).
3. The now linear data is converted to the Project Color Space values with a transform, including the input color space primaries and the Project Color Space primaries.
   • If the input is a wider color space compared to the Project Color Space, (Like Rec.2020 to Rec.709) possible out-of-gamut colors are clamped.
     • It is recommended to not use videos which are color graded in a wider range than the Project Color Space, except it is known that they do not exceed the range of the Project Color Space.

Practically, all this is done in one step, in real-time.

The output processing, basically inverts the process. First the linear RGB values are converted to the output color space. The output color space is set by the output device. Again, out-of-gamut colors will be clamped. The optional tone mapping can compress a higher dynamic range to the lower range of the output and prevents clamping. The tone mapping is applied for luminosity, but this also "shrinks" the wide colors in the process a bit. Different tone mapping functions are selectable.
Next, the output transfer function is applied. For display-referred out streams, like all GPU outputs, the Project's SDR Level is used to encode the data to the correct brightness values. If necessary, the color channels are converted to YCbCr. Again, this is done via a shader on the GPU, per frame, before it is displayed.

SDR workflow vs Wide Color Space / HDR workflow

The engine has two main modes for the color processing mode. The SDR mode, with the Project Color Space set to Rec.709/sRGB and the HDR workflow with all linear color spaces and wide color spaces.

The SDR mode is exactly the same as the pre Ventuz 8 rendering. The main Standard 3D Layer calculates in 8-bit integer values with Rec.709/sRGB primaries and are gamma-influenced and therefore, not linear. Also the Layer Blending is done in 8-bits per color channel.
The HDR-Layer calculates linear in 16-bit float, with the same primaries, but can be tone mapped via the Engine settings on the Layer's Root Node. For lighting via HRi maps and lighting effects like glare and bloom the HDR Layer is best to use, which is also true in SDR scenarios.

Also in the SDR workflow, all incoming assets, like movies, are color transformed as described for In Color Processing above.

For the HDR workflow, everything is calculated in linear 16-bit float except the Standard 3D-Layer, which still uses 8-bit and then is transformed to the Project Color Space. The default settings for a Layer are set to High Dynamic Range for the Engine property.

Performance

Calculating in linear colorspaces changes the engine to use 16-bit values and uses transforms in the render chain for color conversion. This has an impact on performance. Using 16-bit instead of 8-bit doubles the memory and bandwidth in use when working with textures. When using Post-Processing effects that need the frame to be stored in memory to use it the next frame(s), like feedback or delay, this should be taken into consideration.

In general incoming textures are saved are stored / held in memory in the original color format and converted via a shader when rendered and played out. With this approach, Ventuz can automatically determine the minimum processing for conversion. (So a sRGB 8bit texture is converted only when rendered to 16bit) This is done per rendered pixel, and not dependent on the whole texture. For the main conversion of sRGB to linear, GPUs have a fast processing chain which is used by Ventuz. Since modern GPUs have a decent amount of memory, bandwidth, and processing power, the performance impact will only be noticable for extensive use and/or (very) high resolutions.

Bit-depth and controllable quality/performance balance

Ventuz, by default, renders at 8-bit per RGBA channel. However, it supports up to 32bit per channel natively. This includes highly accurate and performant algorithms for color space conversion and bit depth conversion. This means that Ventuz is capable of lossless video processing including handling out-of-gamut colors.

A 10 bit YUV signal is converted to 16 or 32 bit RGB, processed by Ventuz, and back-converted to 10 bit YUV. With 32 bit RGBA, all pixels will be reconstructed perfectly. At 16 bit, out of gamut colors are clamped to the RGB color space.

Software Watchdog

Input and output queues compensate short processing spikes. The more delay can be afforded, the longer the spikes can be without dropping any frame. (see Extra Buffers)

Ventuz can be configured to disable video processing automatically when the queues threaten to over- or under-run. In disabled mode, the video signal is passed without video processing, but still with the same delay, so seamless switching from full processing to disabled mode is possible. (internal keying modes)

Similarly, when switching scenes or projects, Ventuz might drop a few frames as new resources are loaded. A smooth transition can be guaranteed by manually disabling video processing before such operation and re-enabling afterwards.

This is useful for the various keying modes with Disabled Content set to Transparent. When the software watchdog detects that frames are about to be dropped, it provides transparent frames to the keyer, or in case of internal software keying the input frames are sent to the output unmodified. The fill will disappear but the background will not drop.

When using the Internal hardware keying with input and delay, the software watchdog must be enabled.

Hardware watchdogs control physical bypass relays which serve as last line of defense against power failure and forced reboots. In such cases, the video is simply passed through on the board.

Software Keying

Traditional keying requires a key and fill signal, which is then keyed either externally or on the video board to a video stream.

With software keying, Ventuz can render directly on to the input video, relieving the often tedious and arduous requirement of generating the correct alpha channel or key.

A seamless transition from zooming, stretching and 3-dimensional transformation, to just passing through the unmodified signal is easy. Thus Ventuz can be used as an extremely capable and flexible DVE video processor.

Multiple Sources from Multiple Vendors

The Ventuz Video Engine can mix different boards from different vendors, including, but not limited to, Deltacast, Blackmagic and DVS (see Vendors). Video signals can be synchronized to the rendering or it can be asynchronous. For instance, one can use a high quality broadcast board for input and output of the main video stream, and add additional input signals with less expensive hardware.

Similarly, other sources, such as streaming video, USB-cameras, video and image files, or HDMI capture boards can be added, mixed and matched.

SDI

With the growth of Video performance and resolutions, different SMPTE standards and formats have been introduced. Generally, Ventuz supports the SDI formats of the used hardware. For a detailed overview please refer to Supported Hardware Vendors

To transmit a 1080p60 signal with SDI you need more than the 1.5 GBit/s a HD-SDI cable can handle. To do so, you can use two HD-SDI cables (dual link) or a single 3G-SDI cable.

The SMPTE ST 425-1 standard about 3G-SDI specifies Level A and Level B, which are incompatible to each other. Level A is a "proper" SDI stream while Level B emulates dual link by using two multiplexed streams over the same cable. When creating a stream, you must specify the level.

With quad-link streams 4 HD or SG links at 1920x1080 or 2048x1028 are bundeled to one 4k stream. HD is used for refresh rates up to 30 fps, 3G is used for refresh rates above 30 and up to 60 fps.

Each link carries a quadrant of the image. There is nothing special about the individual links, if you swap two links the quadrants are swapped, as there is no marker to put things together correctly. The individual links are compatible to normal SDI equipment, for instance one can use 4 HD/3G hardware keyers to key a 4k stream.

Since a quad-link 4k signal is indistinguishable from 4 separate HS/3G signals, see Auto-Detect 4k Modes in the Device Options section about telling Ventuz how to decide when such signals are detected.

Chroma Keying

There are the following chroma keying modes:

• External Chroma Keying.
• External Chroma Keying Simple.
• Internal Software Chroma Keying.

When one of these modes is active, The Ventuz scene is rendered in 3 ranks:

• Default (Background).
• Foreground.
• Garbage Matte.

Each layer can be assigned to one of the 3 ranks. Also inside of a 3D layer, parts of the hierarchy can be filtered to be in one of the 3 ranks. The ranks are rendered independently.

Setting rank in a layer	Setting rank with filters in a 3D hierarchy

In Internal Software Chroma Keying, Ventuz adds a chroma keyer to the video input. Then the garbage matte is used to cut away parts of the input that might not be covered by green screen. In a trackless environment a masking image will suffice, in a tracked studio a 3D Scene containing simple virtual boxes around where the physical green screen is can be used to mask out the borders of the green screen while the camera moves.
The four signals, input, garbage matte, background, and foreground are composited properly to form the final image.

The External Chroma Keying Simple mode is intended to provide one fill and one key signal to a simple external chroma keyer. Again the three ranks (background (fill), foreground (key+fill), garbage matte(key)) are rendered. These images are composited to a key+fill stream, the key containing the foreground key and garbage matte key, the fill containing foreground fill and background fill. This can use used in an external chroma keyer. It will create some problems with transparencies in the foreground, but usually it suffices.

The External Chroma Keying outputs the three ranks as four individual signals. The foreground fill, foreground key, background fill and garbage matte as key. These four signals can then be composited in an external chroma keyer.

Comprehensive Audio Mixing

Virtual audio cables can be routed from any video input or Windows audio input device to any other video output or audio output device. Each cable comes with delay, volume and balance control.

Each stereo pair is handled individually, so routing from input 1 channel 5/6 to output 2 channel 1/2, or routing a 5.1 signal by using 3 stereo pairs is simple.

Ventuz can add sound clips or sound from video clips to the mix, or analyze sound from input sources. Similarly, audio can be delayed or compensated in order to match video or other delays.

The delay is for more than just adjusting for video input processing times. For some video formats, the number of samples changes from frame to frame, and input and outputs may be in a different position in the cycle of changing sample counts. For asynchronous streams, additional buffering is required. The delay will calculate a small but safe amount of buffering dependent on various attributes of the streams.

The virtual audio cables operate in stereo pairs. Each input or output has its certain number of stereo pairs (e.g. SDI has 8 pairs), and via the routing in the AV Configuration editor, the delays can be mixed and matched from input to output extremely flexibly.

An audio routing can handle multiple cables per input and output, with smoothing to prevent clicks or popping when changing volume or panning.

All audio processing in Ventuz is done at a sample level at 48kHz, and conversion to other sample rates is handled within the Ffmpeg engine. Because Ventuz works with stereo pairs, 5.1 or devices with more outputs are simply divided into stereo pairs. Similarly, mono devices are converted into a stereo pair.

Copyright 2025 Ventuz Technology