From my understanding, the current challenge with ultra-low-latency is scale. Will it scale to 100, 1000, 100,000, concurrent users?
At the moment, everyone in the industry is working to figure out how to provide a viewer experience that is similar, or better than the current expectations of live streaming.
Live streaming is hard. Ultra-low-latency is just as hard.
However, I'm still not sure why we need ultra-low-latency. Will it improve the live experience? Maybe.
Aside from live-action sports or significant events, do we as content consumers really need ultra-low-latency?
Ultra low latency absolutely improves the streaming experience for community streams like on Twitch. When everyone is +-20 seconds of each other the chat doesn't line up well.
> However, I'm still not sure why we need ultra-low-latency.
Simply for passive viewing probably not. For live streaming in which the viewer is also using actuators, e.g., gamepads, wheel, etc., for controlling the stream capture device, then yes.
Yeah I tried too fancy words. What I meant to say that there might be applications beyond gaming, like driving a remote car, vacuum cleaner, or who knows what.
I did coding livestreams and spoke with the audience. The lower latency the better. Others have mentioned control of things, similar idea. HLS over Cloudflare surprisingly beat YouTube and matched Twitch in my side-by-side tests—I built an Open Source self-hosted live streaming system. Currently live sports have about 40-85 seconds of latency and BBC is shooting for 1.5 for the next olympics with ip multicast techniques [1].
How low is ultra-low-latency that it's relevant for sports? Does anyone really care if their broadcast is 30 seconds delayed from live?
I subscribe to both Formula 1 and MotoGP online streaming services and couldn't care less if the broadcasts were even multiple minutes delayed. They may very well be already and I wouldn't know. As far as I can tell they both just use standard CDNs delivering DASH or similar (basically just short video files played in sequence).
That seems more than enough for sports. Multiple minutes may result in getting real time information from elsewhere (e.g., Twitter, local radio) but anywhere from 10 to 30 seconds seems fine.
I work for a video streaming company (primarily sports). There is a huge problem with latency in sports.
We call it the "Twitter effect". Essentially people either see things on social media or app notifications before it happens. Kinda ruins it for some people.
We are working on sub-30 second latency in 4K w/ server side ad insertion, but it's hard work. Especially with encoding with DRM baked in.
I had a normal latency stream (about 1.5 seconds) of the world cup on one England game. I saw the goal going in about 5-10 seconds before the cheers from the neighbours watching on DTT.
For the FA Cup final I had two feeds, one the 4K BBC version, and one on SDI from Wembley. The goal went in, and I'd completely forgotten it by the time the 4K goal went in.
The trouble is, things like twitter, whatsapp etc, will work to the lowest latency, and for a football game that's about 50-600 nanoseconds to the earliest viewer. Twitter can push the message within a few seconds (or the VAR result or whatever)
Latency needs to be in the sub-10 second goal-glass range.
Broadcast TV latency is usually in the 4-10 second range, with typical latency being around 6s. Not sure how it breaks down between cable vs satellite vs over-the-air.
If your friends are watching the same events in other places; you basically need your video to be less delayed than your friends' viewing plus the latency of them texting you (friend observes something, types something, sends, then whatever network and push delay on the way to you).
If you're not watching with someone else, it doesn't make that much difference; but 30 seconds is plenty for someone to cheer for a goal and the notification to distract you from seeing it.
That's already an issue with radio vs TV which is where I'm getting the ~10 to 30 second value. Normal cable TV already has a bunch of delay and someone listening on the radio is often 15+ seconds ahead of you. This can be disruptive. If that person is at a bar or other gathering they will disrupt the viewing because they will cheer (or not cheer) well ahead of the event so the suspense is gone. I've seen this happen.
So given that what you are competing against (cable TV) already has a significant delay I doubt a normal web stream can't reach the same or better latency with just standard HTTP download of video files.
In Santiago, during national soccer matches everyone would watch, goals would reach us through the sound of neighbors rejoicing sooner than they would from actual television coverage because the satellite dish had a latency of say 3 seconds over cable.
You're giving an example that validates my point. Existing technologies already have widely variable delays (radio vs broadcast TV vs cable TV vs sattelite TV) and yet they're all in wide use. And you can't compare directly the 30s I suggested (end-to-end latency) with the 3 seconds of latency differential between technologies. It's probably less than that but it's possible your neighbors are in a 30s delay and you are in a 33s delay.
And if the problem was actually that serious it would be easier to fix it by just adding additional delay to the faster technologies instead of having to reduce latency in internet streaming at huge cost.
Then you would have the people at stadium tweeting about the results and ruining the experience for the rest. We can and we should strive to get the best possible result if it is feasible.
Just experienced this delay being an issue for the 3rd period of this vegas-san jose playoff game. When goals pop off as fast as they do and it completely changes the game, that delay matters. Especially when my friends are either at a bar or streaming the game, it's hard to communicate and share the moment without spoiling. I basically have to keep a mental tab on who's stream is delayed and when I can get excited with who.
Ultra-low-latency is absolutely vital in sports betting industry where the odds are constantly changing based on what actions are taking place in real time.
Results in sports can switch from one side to the other end within split second, so whoever has the lowest latency will gain massive advantages. It's very similar to stock trading.
The chord you're not cutting already has a large delay from live. If you're in the US even larger because besides the delay from the technology there's also intentional broadcast delay for censorship reasons.
Google Stadia(and other online game streaming) and other systems where actuators would be used remotely like drone control systems and others would greatly benefit from ultra low latency.
I think even with turn there's lots of chatter back and forth. Connected clients can NACK for frames and ask for a bandwidth reduction. I think the expected optimal implementation is that the group of WebRTC participants normalize towards an optimal bitrate.
For 1->n streaming you just want very high quality broadcast->server and then presumably you want the downstream clients to not have the option to send packets back to the broadcaster. Transcoding is hard though, so consumers might be stuck with just the source video quality. I assume this lib just passes the frames on from the source (twitch did just that through almost all of its pre-acquisition growth).
Some of my details might be a little off, but I did spend a few months trying to build something very similar to this with gstreamer. WebRTC was hard to grok.
Actually it is not lower latency than centralized TURN server. The difference from TURN is that Ant Media Server behaves like peer to each connected clients. Not relaying the video/audio as TURN. Additionally, it can create adaptive bitrates on the server side. It can record streams and control other things on server side.
