A customer reports that yesterday's all-hands was unwatchable. You pull the browser stats from one attendee and find high packet loss on the video they received. That tells you the stream arrived damaged. It can't tell you whether the presenter's uplink failed, the attendee's Wi-Fi did, or the media server between them was overloaded.
Client-side collection, covered in our post on getStats, stops at exactly that point, and its section on what client stats cannot tell you says so. SFU observability is the other half. The media server is the only component that sees every sender's uplink and every receiver's downlink at once, which makes it the only place that can say whose network failed.
What the Media Server Sees That Clients Can't
An SFU holds a separate connection to every participant, so it measures each direction independently. That gives you five levels of evidence, each answering a different question.
Level | What it measures | The question it answers |
|---|---|---|
Producer (incoming stream) | Loss, jitter, NACKs, keyframe requests, bitrate per simulcast layer | Is this sender's uplink healthy? |
Consumer (outgoing stream) | Loss and round-trip time to one receiver, the layer being sent | Is this receiver's downlink healthy? |
Transport | Estimated available outgoing bitrate, ICE and DTLS state | How much can this connection carry, and is it connected at all? |
Worker or node | CPU, process health, streams forwarded | Is the server itself the bottleneck? |
Room and session | Join latency, participants, failed subscriptions | Is the service working for people trying to use it? |
The first two rows are what a client can never get. The presenter's browser knows its own uplink, and each viewer knows their own downlink, but only the server holds both for the same stream.
SFU Observability in LiveKit, mediasoup and Janus
The common open-source SFUs expose this evidence very differently, and that decides how much of your observability is configuration and how much is code.
LiveKit ships a Prometheus endpoint. Its sample configuration notes that, when enabled, "LiveKit will expose prometheus metrics on :6789/metrics"; it is commented out by default. The metric definitions in its source cover rooms, participants and tracks, session join latency, NACK, PLI and FIR counts, packet loss, jitter, round-trip time, forwarding latency and a connection quality score. They carry node labels and no room or participant labels, which keeps the metric count manageable and means Prometheus can tell you a node is unhealthy but not which call suffered.
mediasoup gives you the raw material and no exporter. Its RTC statistics are returned by getStats() on each producer, consumer and transport, including fraction lost, NACK and PLI counts, round-trip time, bitrate per layer and available outgoing bitrate. Events report score changes, layer changes and a worker process dying. Turning that into time series is code you write.
Janus offers two routes. Its Admin API is, in the project's own words, "a poll-based protocol," so you query it on a schedule. Its event handlers push events instead, covering sessions, handles, SDP negotiation, ICE and DTLS state, media statistics and plugin events, to whatever backend you plug in.
With mediasoup or Janus, that collector is part of the running cost inside engineering time when you self-host. Whichever stack you run, the useful design splits two jobs. Low-cardinality metrics per node drive alerts. Per-session events and logs, keyed by room and participant, drive the investigation that follows.
Whose Network Failed: Sender, Receiver or Server
This is the first question of every call-quality incident, and the server can answer it if you keep sender and receiver quality apart.
mediasoup makes the split explicit. Each consumer reports a score from 0 to 10 for the stream it is sending to its receiver, a separate producerScore for the sender's stream it is forwarding, and per-layer scores when the sender uses simulcast. Where a stack doesn't package it as a score, loss and round-trip time measured separately on the incoming and outgoing streams give the same split.
Read together, the scores point at the failure:
Pattern | Most likely cause |
|---|---|
Producer score low; every consumer of that stream low too | The sender's uplink, since every receiver gets the damaged stream |
Producer score healthy; one consumer low | That receiver's downlink, since nobody else is affected |
Many consumers low across unrelated rooms on one node | The server or its network |
Scores healthy, users still complain | A client-side problem (device, decoder, CPU), so take it to getStats |
The last row matters as much as the first three. A healthy server-side picture rules out the media path quickly, which is often the most useful thing an on-call engineer can establish in the first five minutes.
Signals That Lead and Signals That Lag
Google's SRE book frames monitoring around four golden signals: latency, traffic, errors and saturation. "If you can only measure four metrics of your user-facing system, focus on these four." Mapped onto a media server, they become concrete.
Signal | On an SFU | Leads or lags? |
|---|---|---|
Latency | Join latency (how long until a participant is connected and receiving); forwarding latency through the server | Leads; the SRE book notes latency increases are "often a leading indicator of saturation" |
Traffic | Participants, published and subscribed tracks, egress bitrate | Context for the other three |
Errors | Failed ICE or DTLS, failed subscriptions, peer connections that never reach connected, worker or process deaths | A worker death is immediate; a rising failure rate leads |
Saturation | CPU per worker, egress against interface capacity, streams per worker | Leads, if alerted before the ceiling |
Two media-specific signals lead the generic ones. The first is layer downgrades. When bandwidth estimation reacts before the user notices, the server switches a receiver to a lower simulcast layer, and mediasoup emits a layerschange event when it does. A rising rate of downgrades across a node means quality is being traded away before anyone complains. The second is the keyframe request rate. PLI and FIR counts climb when receivers lose enough packets to need a fresh picture, and a jump across many consumers is an early sign of loss somewhere shared.
The lagging signals are the ones users produce: participants leaving and rejoining, low scores that have persisted for minutes, support tickets. They confirm an incident and are worth dashboards, but by the time they move, the call already went bad.
For saturation, our post on SFU scaling covers the capacity ceilings worth alerting on in detail and why bitrate and round-trip time make poor alert triggers on their own.
What SFU Observability Should Page On
The SRE book's rule is that "every page should be actionable." For a media server that excludes most single-participant problems. One attendee on a weak hotel network is real, but nobody on call can fix it at 2am, so it belongs on a dashboard or in a support ticket.
Page on conditions that affect many people and that an engineer can act on. No published thresholds exist for these, so the numbers below are design choices to tune against your own baseline:
- A worker or media process dies. Page immediately. Every call on it has just dropped or is about to.
- Join failures rise. For example, more than a few percent of join attempts failing over five minutes.
- Join latency climbs at the high percentile. For example, the 99th percentile of join latency doubling against the same hour last week. It leads saturation.
- A node approaches a hard ceiling. For example, sustained CPU per worker or egress above roughly 80% of what you have measured it can carry, so there is time to drain it before calls degrade.
- Many consumers degrade on one node at once. For example, the share of consumers with a low score or a recent downgrade crossing a set level across unrelated rooms. One room is a participant problem; many rooms on one node is yours.
Everything else, from single bad consumers to elevated loss in one room, goes to dashboards and to the investigation trail.
What the 2am Page Should Contain
A page that says "SFU node 3 unhealthy" sends the engineer hunting. A useful one arrives with the first answers attached:
- Scope. The node, how many rooms and participants are on it, and how many are affected.
- Direction. Whether producer scores, consumer scores or both have dropped, which tells the reader whether to look at senders, receivers or the server.
- Recent change. Deploys, configuration changes or certificate renewals on that node in the last hour.
- Saturation context. CPU, egress and streams per worker against their ceilings at the time of the alert.
- The trail. Links to the per-session events and to the matching client-side getStats records, joined on room and participant IDs.
That last item is why the identifiers matter. If the server logs and the client collector use the same room and participant IDs, an engineer can go from the alert to the affected sessions to the browser's own view of them in a few clicks.
The Bottom Line
The media server is the only place that sees both directions of every stream, so it is where you find out whose network failed. Keep sender and receiver quality apart, watch the signals that lead (join latency, layer downgrades, keyframe requests and saturation), and page only on conditions that affect many people and that someone can act on. Treat every threshold as a design choice and tune it against your own traffic.
What's Next
Knowing what to alert on is half the job; knowing who answers is the other. See who gets paged at 2am under build, buy and embed.
Frequently Asked Questions
What is SFU observability?
It is monitoring a selective forwarding unit from the server side: the quality of every incoming and outgoing stream, the health of each server process, and how quickly people can join. It complements client-side getStats, which only sees one connection from one participant's point of view.
What metrics should I alert on for a media server?
Worker or process deaths, join failure rate, high-percentile join latency, saturation against measured ceilings, and widespread consumer degradation on one node. Single participants with poor networks belong on dashboards rather than pages.
How can I tell if a call problem is the sender or the receiver?
Compare sender-side and receiver-side quality for the same stream. If the incoming stream is degraded, every receiver suffers and the sender's uplink is the cause. If the incoming stream is healthy and one receiver is degraded, the problem is that receiver's downlink.
Does LiveKit expose Prometheus metrics?
Yes, when enabled in its configuration it serves Prometheus metrics on port 6789. The metrics are labelled per node, so they are suited to alerting, while per-room investigation needs session-level events or logs.
Are there standard alert thresholds for WebRTC servers?
No published standard exists. Treat any threshold as a design choice: measure your own baseline, alert on sustained changes relative to it, and tune after real incidents.
Can I see server-side media metrics if I use a hosted video API?
Usually only what the provider chooses to expose. Platforms that keep the media path on infrastructure you control, such as Samvyo, which is based on SFU architecture, let you collect server-side metrics yourself and put them next to client stats for the same session.
