Turn on end-to-end encryption in a Zoom meeting and its own support documentation lists what switches off: cloud recording, live transcription, live streaming, the AI features, polling and the whiteboard. Phone and SIP participants can't join, nor can its Web App and Web SDK clients, and the meeting is capped at 1,000 people.
That list is the visible price of end-to-end encryption for group video, and our post on video deployment architecture already lists what that costs in general terms. This post covers the less visible part: what the server forwarding your call can still see, which features keep working, and the question that decides whether the claim is true at all, namely who holds the key.
Why an SFU Gets in the Way of Ordinary Encryption
Every WebRTC call is already encrypted. DTLS-SRTP protects each hop, but an SFU sits at the end of a hop. Media arrives at the server, is decrypted, and is re-encrypted for every subscriber. In a two-person call that terminates at the other browser; in a group call through an SFU it terminates at the server.
The server needs that access for ordinary work. It reads RTP headers to route packets, reads RTCP feedback to manage bandwidth, and drops layers when a subscriber's link is weak. The SFrame specification, RFC 9605, exists because full encryption would blind the server to exactly the information it needs to forward media.
SFrame's answer is a second layer. The sender encrypts each media frame with a key the server never receives, then hands the encrypted frame to the normal WebRTC stack, which wraps it in SRTP as usual. The server removes the outer layer and finds a frame it cannot read.
What SFrame Encrypts, and What the SFU Still Sees
SFrame encrypts the media payload. Almost everything around it stays visible to the server, and some of it says a lot about the call.
Item | Visible to the SFU? | What it reveals |
|---|---|---|
Media payload (the audio and video) | No | Nothing; this is what SFrame protects |
SFrame header: key ID and counter | Yes, authenticated but not encrypted | Which participants are sending, and at what frame rate |
RTP headers and timestamps | Yes | Stream structure, timing, packet sizes |
Audio level header extension | Yes | Who is speaking, packet by packet |
Signalling and room events | Depends on the platform | Who joined, when, and from where |
RFC 9605 is direct about the header: an observer can learn "which parties are sending media" from the key IDs and at what rates from the counters, and the key-ID structure can expose epoch counts and sender identifiers. The audio level extension is the sharper leak. RFC 6464 notes that because SRTP authenticates header extensions without encrypting them, "audio levels are therefore visible on a packet-by-packet basis," and an observer "might be able to infer information about the conversation." An SFU uses that extension to pick the active speaker, so it keeps receiving it.
Put together, end-to-end encryption for group video hides what was said. It does not hide who was in the call, who spoke, for how long, or when. For many threat models that is enough. For some, the metadata is the secret.
There is a second limit the specification states plainly: SFrame does not provide per-sender authentication. Every participant holds the keys needed to encrypt as any sender, so a malicious participant can impersonate another inside the call. The server is locked out; the other participants are trusted completely.
What Keeps Working: Forwarding and Layer Selection
A common assumption is that encryption stops the SFU from adapting quality. It doesn't, provided the stream is built for it.
RFC 9605 requires each simulcast layer to be encrypted separately, and each scalable (SVC) layer to be its own SFrame ciphertext, so that the SFU can still drop layers it doesn't need to forward. The server reads layer information from RTP headers and header extensions, never from the payload. If you want a refresher on how simulcast layers work, that is the mechanism being preserved.
The AV1 RTP payload specification goes further. Its Dependency Descriptor header extension was designed so an intermediary can forward frames "when the RTP payload is encrypted end-to-end," without parsing the bitstream at all.
What does break is any server logic that peeks into the codec payload. Some SFUs detect keyframes or read codec-specific descriptors from inside the frame, and that information is now ciphertext. Mozilla's WebRTC team describes the practical consequence in its guide to end-to-end encryption in all browsers: applications must know which bytes to leave unencrypted so the packetizer keeps working, and for some codecs, H.264 among them, that takes trial and error.
Two smaller costs come from encrypting whole frames. A receiver can't decode part of a frame, because the frame only authenticates once all of it has arrived. And key rotation can delay video for someone who has just joined, because they have to wait for a keyframe encrypted under a key they hold.
What Stops Working: Anything That Needs to See the Media
The list in Zoom's documentation has one cause. Every feature on it needs the server to see or hear the media, and under end-to-end encryption the server can't.
- Recording on the server produces ciphertext. A useful recording has to come from something that holds a key.
- Transcription, captions, noise processing in the cloud and any AI that has to read the media lose their input.
- Phone and SIP gateways have to decrypt to translate media into a format the other network understands, so a gateway becomes a participant with a key, or the call excludes it.
- Live streaming to a CDN needs decoded media at some point, with the same consequence.
The honest workaround is to make the component a participant. A compliance recorder that joins the call, holds the key and records it is legitimate, as long as everyone can see it is there. It also means the call is end-to-end encrypted between the participants, one of which is a recorder, which is a weaker sentence than most marketing pages use.
End-to-End Encryption Is a Key-Management Claim
SFrame encrypts frames. It deliberately says nothing about where the keys come from. RFC 9605 puts key exchange out of scope and describes two patterns an application can use: sender keys distributed over an existing end-to-end secure channel, or keys derived from an MLS group.
That gap is where most of the security lives. LiveKit's encryption documentation is a clear example: its built-in key provider uses "a single shared key for the whole room," which the server typically generates when the room is created and distributes the same way the platform hands out access tokens. Per-participant keys and rotation need a custom key provider. The documentation is careful to say LiveKit does not store or transport keys itself, and that signalling is still readable by the server.
That design is reasonable for keeping media unreadable to a cloud SFU operated by someone else. But if the same organisation runs the SFU and the service that generates keys, it can read the media by design. The test for any group-video encryption claim is one sentence long. Can the party that operates the media server obtain the key?
A strong answer to that question needs three properties:
- Keys are agreed between clients. No server generates the media key or sees it in transit.
- Keys change when membership changes. RFC 9605 says clients should rekey when participants join or leave, so a new joiner can't decrypt earlier media and a departed one can't decrypt later media.
- Identity is verified out of band. Otherwise the server can add a participant it controls, and that participant receives the key legitimately.
Messaging Layer Security, RFC 9420, is the IETF standard built for the first two. It gives a group forward secrecy and post-compromise security, and its key operations cost scales "as the log of the group size," which is what makes rekeying on every join practical for groups ranging "from two to thousands." The third property is a product decision: safety numbers, verified devices, or an identity provider the server can't impersonate.
The Browser Side in 2026
The browser API this depends on is now available everywhere. RTCRtpScriptTransform, which lets a worker see and rewrite each encoded frame, is Baseline on MDN since October 2025, after Chrome shipped WebRTC Encoded Transform in Chrome 141 on 30 September 2025. Safari and Firefox had already implemented it natively.
What browsers don't give you is SFrame itself. The native SFrameTransform interface was still "in limbo" when Mozilla's team wrote its guide in February 2024, so applications implement SFrame in script inside the transform. That puts the cryptography, the key handling and the packetizer details in your code.
The bandwidth overhead is small for video and noticeable for audio. RFC 9605 assumes 22 bytes per encrypted frame. The arithmetic, at 1,000 bits per kilobit:
Stream | Frames per second | Added bitrate | Share of codec bitrate |
|---|---|---|---|
Video at 1,500 kbps | 30 | 5.3 kbps | 0.4% |
Video at 300 kbps | 30 | 5.3 kbps | 1.8% |
Opus audio at 32 kbps, 20 ms frames | 50 | 8.8 kbps | 27.5% |
Audio sends more frames than video and each one is small, so the fixed per-frame cost weighs more. On a constrained link that is worth budgeting for.
When End-to-End Encryption Is the Wrong Tool
End-to-end encryption answers one threat: the operator of the media server reading your media. If that operator is a vendor you would rather not trust, it is the right answer, and its costs are the price of that guarantee.
If the server operator is you, the calculation changes. A regulated organisation that must record and retain calls, transcribe them, or run analytics needs a server that can see the media. Encrypting the media end to end and then adding a recorder that holds the key gives you most of the costs and a weaker version of the claim. Running the media server, relay and recorder inside your own boundary keeps every feature and keeps the media away from any third party, without pretending the server is blind.
The question to ask is who you are protecting the media from. The answer picks the architecture.
The Bottom Line
SFrame lets an SFU forward media it can't read, and layer selection and active-speaker detection survive. What it doesn't hide is who spoke and when, and what it can't protect is a key that the operator generated. An end-to-end encryption claim is only as strong as the answer to one question, which is whether whoever runs the media server can get the key.
What's Next
Keys aren't the only trust root a self-hosted media estate has to run. Next: running your own certificate authority for real-time media, from root distribution to rotation.
Frequently Asked Questions
Does end-to-end encryption for group video work with an SFU?
Yes. SFrame, defined in RFC 9605, encrypts each media frame with a key the SFU never receives, while leaving the headers the SFU needs for forwarding and layer selection readable. The server can route the media but can't decode it.
What can the server still see in an end-to-end encrypted call?
Who is sending media and at what rate, packet timing and sizes, and the audio level of each packet, which reveals who is speaking. Signalling such as joins and leaves is usually visible to the platform too. The content of the audio and video is what stays hidden.
Can you record an end-to-end encrypted video call?
Only with something that holds a key. A server-side recorder sees ciphertext, so the recording has to come from a client or a recorder that joins the call as a participant. That makes the recorder a party to the encryption, and it should be visible to everyone in the call.
Is SFrame the same as WebRTC Insertable Streams?
No. Insertable Streams, now standardised as WebRTC Encoded Transform, is the browser API that lets your code see and modify each encoded frame. SFrame is the encryption format you can implement with that API. Browsers ship the API; most applications still implement SFrame themselves.
Who should generate the keys for end-to-end encrypted video?
The clients, ideally. If the server that runs the media also generates or distributes the media key, it can decrypt the call. Group key agreement such as MLS, RFC 9420, lets clients agree keys among themselves and rekey whenever someone joins or leaves.
Do I need end-to-end encryption if I self-host the media server?
Often not. End-to-end encryption protects media from whoever operates the media server. If you need recording, transcription or analytics and want media kept away from third parties, running the media path inside your own network may serve better. Samvyo, which is based on SFU architecture, supports on-premise deployment where the media path, TURN and recording run on infrastructure you control. End-to-end encryption is available as an option, off by default and configured for customers who need it, so a deployment can choose which guarantee it wants.
