Use Cases:

Pixel Streaming

Render video and stream it to web browsers in real-time over WebRTC for interactive use.

Here's what you'll need:

Changes incoming!

The contents of this page will be revised significantly once Unreal Engine 4.27 is released. The information presented here is currently accurate as at the time of writing.

Contents

Overview and history

The Unreal Engine’s Pixel Streaming system provides the ability to stream audio and video output from an Unreal application to client devices such as web browsers in real-time using WebRTC communication stack and to receive control data such as keyboard and mouse/touch inputs back from client devices. As discussed in the official Epic Games white paper Streaming Unreal Engine content to multiple platforms, Pixel Streaming allows developers to leverage cloud computing power to deliver high-quality experiences to the broadest possible range of hardware devices.

The reference implementation of the Pixel Streaming system was introduced in Unreal Engine version 4.21. Initially, the reference implementation supported only the Windows operating system. This precluded the use of Pixel Streaming inside Unreal Engine containers, since GPU accelerated Windows containers do not have access to the required video encoding APIs. Although newer versions of the Unreal Engine have subsequently introduced Linux support in order to facilitate use inside containers, it is important to note that the limitations of Windows containers still make them unsuitable for use with Pixel Streaming. Pixel Streaming cannot be used in GPU accelerated Windows containers, irrespective of the version.

Developers from the company TensorWorks added support for both Linux and GPU accelerated Linux containers to the Pixel Streaming system in custom versions of both Unreal Engine 4.23 and Unreal Engine 4.25. These custom forks of the Unreal Engine are collectively referred to as “Pixel Streaming for Linux” in order to differentiate them from the official reference implementation. Community interest in container-based Pixel Streaming eventually prompted Epic Games to officially announce Linux and container support for Pixel Streaming on their public roadmap for inclusion in the release of Unreal Engine 4.27.

Until the official release of Unreal Engine 4.27, the TensorWorks fork of Unreal Engine 4.25 remains the recommended version for developers seeking to use Pixel Streaming in containers.

Key considerations

Implementation guidelines

Basic setup

You will need two containers for a minimal deployment of Pixel Streaming:

Once you have started both of these containers together, you should be able to access your Pixel Streaming application by opening a web browser and navigating to http://127.0.0.1 (or the public IP address of the machine that is running the containers if you are accessing the Pixel Streaming application from a different machine.)

Configuring STUN and TURN

You can use the peerConnectionOptions configuration parameter for Cirrus to specify the STUN and TURN servers that should be used when establishing WebRTC connections between web browsers and your Pixel Streaming application. This makes it possible to access Pixel Streaming applications from all client devices even when firewalls or intermediate network topology would otherwise prevent the establishment of direct peer-to-peer connections. There are a variety of options available, but the following are recommended:

Deploying at scale

The Cirrus signalling server and the Matchmaker Server that ship with the reference implementation of Pixel Streaming are only designed to act as a starting point for developers to extend, and do not provide the necessary functionality for deploying Pixel Streaming applications at scale. TensorWorks is currently developing the open source Scalable Pixel Streaming Framework to address this. This section will be updated when the framework is made available to the public.