RTCRtpScriptTransformer
Limited availability
This feature is not Baseline because it does not work in some of the most widely-used browsers.
The RTCRtpScriptTransformer
interface of the WebRTC API provides a worker-side Stream API interface that a WebRTC Encoded Transform can use to modify encoded media frames in the incoming and outgoing WebRTC pipelines.
Note: This feature is available in Dedicated Web Workers.
Instance properties
RTCRtpScriptTransformer.readable
Read only-
A
ReadableStream
on which encoded frames from the WebRTC sender or receiver pipelines may be enqueued. RTCRtpScriptTransformer.writable
Read only-
A
WritableStream
that encoded frames should be piped to. RTCRtpScriptTransformer.options
Read only-
Options passed from the
RTCRtpScriptTransform
constructor, which are used to configure transform code based on whether incoming or outgoing frames are being processed.
Instance methods
RTCRtpScriptTransformer.generateKeyFrame()
-
Requests a video encoder generate a key frame. May be called by a transformer in the sender pipeline when processing outgoing frames.
RTCRtpScriptTransformer.sendKeyFrameRequest()
-
Requests the sender send a key frame. May be called by a transformer in the receiver pipeline when processing incoming encoded video frames.
Description
A RTCRtpScriptTransformer
instance is created as part of construction of an associated RTCRtpScriptTransform
, which specifies the worker in which the transformer is created and options that will be passed to it.
The transformer is made available to a worker through the rtctransform
event transformer
property.
This event is fired on construction of the associated RTCRtpScriptTransform
and when an encoded frame is enqueued on the RTCRtpScriptTransformer.readable
from a codec (outgoing) or from the packetizer (incoming).
The transformer exposes a readable
and writable
stream into the worker, along with an options
object provided to the RTCRtpScriptTransform
on construction.
When the associated RTCRtpScriptTransform
is assigned to a RTCRtpSender
or RTCRtpReceiver
, encoded media frames from the WebRTC sender or receiver pipelines are enqueued on the readable
stream.
A WebRTC Encoded Transform must read encoded frames from transformer.readable
, modify them as needed, and write them to transformer.writable
in the same order, and without any duplication.
The transformer.options
allow an appropriate transform function to be used, based on whether the encoded media frames are incoming or outgoing.
The transform is usually implemented by piping frames from the readable
through one or more TransformStream
instances to the writable
, transforming them as needed.
The interface also provides methods for a sender to generate get a video encoder to generate a new keyframe, or for a receiver to request a new key frame from the sender's encoder (video encoders commonly send a key frame containing the full information needed to construct an image, and subsequently send delta frames containing just the information that has changed since the previous frame).
These methods are required in cases where a recipient would be unable to decode incoming frames until they receive a new key frame. For example, a new recipient joining a conference call will not be able see video until they have received a new key frame, since delta frames can only be decoded if you have the last key frame and all subsequent delta frames. Similarly, if frames are encrypted for a recipient, they will only be able to decode frames once they have received their first encrypted key frame.
Examples
This example shows the code for a WebRTC Encoded Transform running in a worker.
The code uses addEventListener()
to register a handler function for the rtctransform
event, which makes the RTCRtpScriptTransformer
available as event.transformer
.
The handler creates a TransformStream
and pipes frames from the event.transformer.readable
through it to event.transformer.writable
.
The transform stream transform()
implementation is called for each encoded frame queued on the stream: it can read the data from the frame and in this case negates the bytes and then enqueues the modifiable frame on the stream.
addEventListener("rtctransform", (event) => {
const transform = new TransformStream({
start() {}, // Called on startup.
flush() {}, // Called when the stream is about to be closed.
async transform(encodedFrame, controller) {
// Reconstruct the original frame.
const view = new DataView(encodedFrame.data);
// Construct a new buffer
const newData = new ArrayBuffer(encodedFrame.data.byteLength);
const newView = new DataView(newData);
// Negate all bits in the incoming frame
for (let i = 0; i < encodedFrame.data.byteLength; ++i) {
newView.setInt8(i, ~view.getInt8(i));
}
encodedFrame.data = newData;
controller.enqueue(encodedFrame);
},
});
event.transformer.readable
.pipeThrough(transform)
.pipeTo(event.transformer.writable);
});
The only special things to note about the TransformStream
above are that it queues encoded media frames (RTCEncodedVideoFrame
or RTCEncodedAudioFrame
) rather than arbitrary "chunks", and that writableStrategy
and readableStrategy
properties are not defined (because the queuing strategy is entirely managed by the user agent).
A transform can run in either the incoming or outgoing WebRTC pipelines.
This doesn't matter in the code above, because the same algorithm might be used in the sender to negate the frames, and in the receiver to revert them.
If the sender and receiver pipelines need to apply a different transform algorithm then information about the current pipeline must be passed from the main thread.
This is done by setting an options
argument in the corresponding RTCRtpScriptTransform
constructor, which is then made available to the worker in RTCRtpScriptTransformer.options
.
Below we use the transformer.options
to choose either a sender transform or a receiver transform.
Note that the properties of the object are arbitrary (provided the values can be serialized) and it is also possible to transfer a MessageChannel
and use it to communicate with a transform at runtime in order to, for example, share encryption keys.
// Code to instantiate transform and attach them to sender/receiver pipelines.
onrtctransform = (event) => {
let transform;
if (event.transformer.options.name == "senderTransform")
transform = createSenderTransform();
// returns a TransformStream (not shown)
else if (event.transformer.options.name == "receiverTransform")
transform = createReceiverTransform();
// returns a TransformStream (not shown)
else return;
event.transformer.readable
.pipeThrough(transform)
.pipeTo(event.transformer.writable);
};
Note that the above code is part of more complete examples provided in Using WebRTC Encoded Transforms.
Specifications
Specification |
---|
WebRTC Encoded Transform # rtcrtpscripttransformer |
Browser compatibility
BCD tables only load in the browser