9 ms·
What I want to know is whether Lyra takes any longer to encode than the alternatives. Because as far as I can tell, nobody cares in the slightest about latency
by rectang 6y ago
What I want to know is whether Lyra takes any longer to encode than the alternatives.
Because as far as I can tell, nobody cares in the slightest about latency.
Phone calls are getting to be like writing postcards to each other. Speak in a whole paragraph. Wait several seconds for the latency to clear. Then the other party responds with a whole paragraph, waits several seconds for the latency to clear...
Improvements to fidelity are nice-to-have, but I would like some real-time in my real-time communications, please.
- reaperducer 6y agoI had the pleasure of using a real landline just before the pandemic. Honest wire-to-wire connection between two ranches, so no silly VOIP steps between. It was fantastic. You don't appreciate how much latency is destroying our ability to communicate verbally until you go back to the old way. One example is arguing. It's no wonder people used to be able to argue with one another on a telephone. You could raise your voice and still hear the other side and adjust your speech in real time. Today it's just one party shouting over the other to drown the opponent out.
- rectang 6y agoBetween miserable latency, not-so-great fidelity, and the fecklessness of phone companies in the face of the robocall epidemic, I have come to hate phone calls. I'm rooting for something to replace phone communications. Any chance that Matrix can do better on any of those fronts? Especially on fidelity and latency since they're germane to the high-level subject of this discussion.
- colordrops 6y agoOne conspiracy theory is that tech companies have lobbied to prevent real action on robo-calls, in order to get people like you to hate calls and migrate to online services.
- eeZah7Ux 6y ago> conspiracy theory Sounds like a run-the-mill everyday business decision.
- 908B64B197 6y agoI don't think the FCC needed any lobbying to do nothing for the consumer these last few years.
- Majestic121 6y agoThat seems unlikely. The robocall phenomena don't really exists in France and many people still dislike calling
- AshamedCaptain 6y agoIt definitely exists in France. I still receive a robo call every single day on my land-line, liste rouge/Robinson or not. I have stopped bothering reporting them. The prevalence of "who called me/is it important" services makes me think it is not an uncommon problem.
- Aloha 6y agoA cell phone tbh, is about the same latency as a landline in most end to end call circumstances. Latency only really is noticeable when better than 600ms. (And only a real problem over 1000)
- lynndotpy 6y agoI think latency is noticeable at even lower values. As a basic example, try to sing a song with someone over a voice call. Consider using Airpods or similar bluetooth headphones to make it more apparent.
- kwindla 6y agoI disagree with these numbers, in general. Though of course "noticeable" is subjective and varies by use case as well as by person. For many people, end-to-end audio latency in a 1:1 conversation becomes noticeable/annoying at 200ms. And in a multi-participant conversation, talking over each other becomes noticeably more common even at 100ms compared to 50ms.
- xellisx 6y agoIIRC the Bell standard points out no more than 50ms.
- ubercow13 6y agoImagine trying to have a face-to-face conversation with 600ms of latency...
- jeffbee 6y agoThere is a huge quantity of research opposing your statements. The ITU considers 300ms round-trip latency to be catastrophic.
- acdha 6y agoDo you have a citation for that? I've heard 200ms as the key threshold and the ITU uses 100ms for their default delay sensitivity class[1]. One key concept is that this isn't fixed but situational: if you're watching a TV show, the threshold is higher than if you're trying to react to something which is higher than simply noticing a delay and speech is more forgiving than, say, music hitting a precise tempo (I believe musicians have been tested as noticing delays down into the 10-20ms range). One other big factor is consistency: if the variability is due to compressor overhead which is constant, the effect will be noticeable but less distracting than if it's varying due to something like wireless conditions. 1. https://www.itu.int/rec/T-REC-G.107-201506-I/en https://www.itu.int/rec/T-REC-G.107-201506-I/en
- LargoLasskhyfv 6y agoEver experienced european ISDN to european ISDN at 64kb/s ? I know americans said 'I still don't need(it)' but I still do miss it :-) Funny thing was I had better(and cheaper!) calls to the US using calling cards, dialing into Frankfurt, and from there to the US than using the native offer of my telco.
- hackmiester 6y agoVoIP is definitely not the issue here. Codecs can be, and are, fast. I have no idea what cellular providers are doing to mangle the voice path so bad, but it certainly isn’t inherent to VoIP.
- andai 6y agoI thought the delay would be because VoIP uses packet-based communications rather than a direct connection.
- cnorthwood 6y agoMost landline networks are also digital now for exchange to exchange communications, so even those will be packet based
- nayuki 6y agoNot all packets are created equal. ATM (Asynchronous Transfer Mode) cells have a fixed 48-byte payload, but IP (Internet Protocol) packets have longer headers and variable-length payloads up to about 1500 bytes. ATM is designed for low latency audio transfer, whereas IP incentivizes using bigger packets to reduce overhead at the expense of more latency.
- comex 6y agoHistorically, sure. But at a realistic modern internet speed of, say, 15 MB/s, 1500 bytes takes 0.1ms to transfer. Even at 1.5 MB/s it takes 1ms. Typical VoIP call latency apparently ranges from 20ms to 200ms, so packet size is not a major contributor.
- dguaraglia 6y agoI have no idea what they are doing but it sucks. I have in many occasions found that making a call using WhatsApp works much better - both in quality and latency - than making a phone call. Phone lines are so atrociously bad, they've put me off from doing a voice call unless it's absolutely necessary.
- bentcorner 6y agoI'm glad you bring up latency - I've experienced several second latency in discord and it's really terrible when it happens (diagnosed via side-channel). The worst thing is that the app does nothing to try to salvage the conversation and the app doesn't tell you it's in a degraded state, so you'd never know this was a problem.
- kroltan 6y agoIt has a quality indicator on the bottom right, and if you click it there's a nice chart that shows the latency. https://i.imgur.com/vR7NSpG.png https://i.imgur.com/vR7NSpG.png Or are you talking about something else?
- bentcorner 6y agoWell, I stand corrected. I'll take a look at that chart next time I experience something like that happening. I haven't seen this UI before. Thanks!
- 908B64B197 6y agoI remember switching to VoIP and noticing the voice quality getting better compared to the old twisted pair. No noticeable latency.
- tgv 6y agoAnother thing I clearly experienced in the beginning of the mobile phone era, when one still could compare those things, was dynamic range. When speaking via a good landline, you could hear the presence of the other, but on mobile, there was only speech or silence. Noise suppression saves bandwidth, but it is so aggressive that the nuances in the other's utterances (breathing, hesitations, etc.) simply disappear. For a simple transaction that's not a problem, but when your SO lives far away, miscommunication arises too easily.
- IshKebab 6y agoArguing is definitely affected by latency but it is at least possible to reduce latency, e.g. by getting fibre internet or using ethernet. A more annoying "feature" of many VoIP systems is that they mute the other person while you're talking. You literally can't interrupt people because they won't hear you. I presume this is done in order to reduce feedback, but it still sucks.
- mbar84 6y agoAdd to that when the other party is hearing you over speakers, and the echo cancellation kicks in whenever you start to speak so that you can't hear what they're saying when they try to interject. You just see lips moving, stop speaking and you start to hear them mid sentence.
- kevin_thibedeau 6y agoJust disable VoIP or disconnect from WiFi.
- chrisseaton 6y agoIsn't the actual phone network packet-switched and running over fibre optics now anyway? I don't think you can get a literally analog phone call anymore can you?
- jeffbee 6y agoYou probably can still get a real phone, but plugging a plain old phone into an optical network terminal, which converts to VoIP, has most of the virtue of POTS. I use a regular POTS device with Sonic fiber service and it sounds fantastic, with no noticeable latency. Most of the problems with voice over mobile networks is caused by frame drops and wacky inter-arrival times. A wired IP network just doesn't have those problems. Going back to the article/paper, I'd love to hear more about how Lyra interacts with Duo's machine-learned voice interpolation that fills in for dropped frames. Do they complement each other, or interfere?
- giantrobot 6y agoYou haven't had "analog" phone calls for decades. Your analog line was converted to a digital signal at the central office or concentrator (ugly green boxes). From the CO to a tandem through the core network your call was digitally switched to the endpoint where it was converted back to analog for the callee's phone.
- wrs 6y agoHowever, those were circuit-switched, not packet-switched. (At least for a while.)
- giantrobot 6y agoI should have been clearer, the OP doesn't want "analog" service but PSTN (TWM switched) service. Unfortunately that is dying out as carriers are moving to all IP core networks. As customers have moved away from fixed landline service it ends up costing more per customer to keep the PSTN running. The last mile equipment is the same for TWM or IP switching but they can dump that directly into an IP network rather than maintain a hierarchy of COs/tandems/toll switches with dedicated links between them.
- kixiQu 6y agoIn the article, > This trick enables Lyra to not only run on cloud servers, but also on-device on mid-range phones in real time (with a processing latency of 90ms, which is in line with other traditional speech codecs). Does that not cover it?
- regularfry 6y agoIt covers it, but it's not exactly brilliant. 200ms is the point at which conversation breaks down. If half that budget has gone on the codec, not much has to happen on the wire for it to be noticeable.
- mlyle 6y agoIf your 200ms number is round trip, 90ms + 90ms is more than half that budget.
- volta83 6y agoThe pipeline is basically: - stream your voice through an encoder: X ms - send encoded packages over the network: 20-100ms latency (fiber vs mobile phone) - potential decoding + encoding (if receiver does not support the senders codec, e.g., a landline phone using old codec) - stream packages through a decoder: Y ms If you are aiming for 60ms audio latency, which is what I would consider "good", then in the best scenario (20ms network latency; both using same codec) the latency of the encoder+decoder has to be max 40ms (e.g. 20ms for encoder, and 20ms for decoder). It should be obvious that a decoder that does not meet the 20 ms budget, but takes 90 ms instead which is > 3x the budget, can produce better audio (ideally 3-4x better). Latency wise, everything below 60 ms is really good, 60ms is good, and the 60-200ms range goes from good to unusable. That is, 200ms, which is what this new codec would hit under ideal conditions, has a latency that humans consider "unusable" because it is too high to be able to have a fluent conversation. For me, personally, if latency is higher than 120ms, I really don't care about how good a codec "sounds". I use a phone to talk to people, and if we start speaking over each other, cutting each other, etc. because latency is too high, then the function of the phone is gone. Its like having a super nice car that cannot drive. Sure its nice, but when I want to use a car, I actually want to drive with it somewhere. If it cannot drive anywhere, then it is not very useful to me.
- kwindla 6y agoThis is a fantastic question. I agree with you that we're slowing boiling the frog (and the frog is ourselves) in accepting more and more latency in our real-time communications. I think the answer for Lyra is that latency is a concern, but maybe at this stage not as much of a concern as it could be. I'm only guessing, though based on this [0]: > The basic architecture of the Lyra codec is quite simple. Features, or distinctive speech attributes, are extracted from speech every 40ms and are then compressed for transmission. That sounds like the minimum frame size for Lyra is 40ms. For Opus (the audio codec used for most WebRTC applications), the default frame size is 20ms [1], and most implementations support frame sizes of 10ms [2]. Of course, your favorite web browser might not default to 20ms frames for Opus. And by "most implementations" I meant Google Chrome. :-) [0] https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-codec-for.html https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-... [1] https://tools.ietf.org/html/rfc7587#section-6.1 https://tools.ietf.org/html/rfc7587#section-6.1 [2] https://chromium.googlesource.com/external/webrtc/+/HEAD/modules/audio_coding/codecs/opus/audio_encoder_opus.cc#63 https://chromium.googlesource.com/external/webrtc/+/HEAD/mod...
- toomim 6y agoGoogle Chrome has a latency of 20ms to just repeat back audio on the local device. That is, with no networking, and no processing, it takes 20ms for any information to from microphone back out to speakers.
- Miraste 6y agoEven the default Windows audio device "listen to yourself" option has some serious latency.
- revenant3-2 6y agooh yeah it's bad, worse than cl_loopback in csgo even. one of the reasons i decided i wanted an XLR mic + amp
- pjc50 6y agoThe default Windows Audio Engine buffer size is 10ms, so that will undoubtedly be one buffer delay in and one buffer delay out. https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio https://docs.microsoft.com/en-us/windows-hardware/drivers/au...
- TheRealSteel 6y agoI spent the last year and a half living about as far from my home city as possible (Melbourne > Edinburgh) (Currently in quarantine) and the latency of phone calls drives me nuts. The huge gaps between people speaking, and the complete change in conversation flow since you have to speak in huge continuous paragraphs. The constant "go ahead" , "no you go" etc... Ugh. But I agree with you... nobody seems to care except me.
- robmsmt 6y agoI also agree. Sometimes I speak to work colleagues who are on BT headsets and it must add what feels like 100ms to the call. I can sometimes hear the end of my sentence as I speak. Infuriating.
- Miraste 6y agoI don't understand how Bluetooth has made it this far for phone calls. It has two modes: unintelligible quality and bad latency, or decent quality and ludicrous latency. The entire standard is an advertisement for Airpods.
- Causality1 6y agoProviders just don't seem to care. My internet connection is a hundred times faster than it was in 2004 but I'm still playing games with the same 50ms-60ms of lag. Sure the speed of electrical signals in wire is a good chunk of it but there's still so much room for improvement.
- qayxc 6y ago> but I'm still playing games with the same 50ms-60ms of lag What kind of lag, though? Input lag has actually gone up in the past 15 years (e.g. due to displays and USB device polling). These things add up quickly and it doesn't even have to be just the network that introduces lag.
- ksec 6y agoDiscussed a few days ago [1] Copying my comment here Even 3G AMR, I think that was pre 2000 Speech Codec started at 5Kbps. With a latency of only ~20ms. If I am reading correctly the encode due to ML nature would take at least 40ms and up to 90ms for Lyra. I am sure there are some specific usage that would be a great use case. But for most consumer consumption I cant think of one on top of my head. One should also be aware of the current roadmap in 5G and the on going work in 6G. We still have a long way to go in maximising bandwidth / transfer per capita. i.e More bandwidth for everybody. It seems to be the case with ML they want to take these speech codec to new low bitrate. While it is fun doing it as a research, I much rather they push the envelop of at least 6 / 8Kbps if not even higher closer to perfection with even lower latency ( 10ms if not lower ). [1] https://news.ycombinator.com/item?id=26279891 https://news.ycombinator.com/item?id=26279891
- yholio 6y agoI would go the exact opposite route for this exact reason: something like 24Kbit ADPCM or 16Kbit G728 can provide an absolutely decent quality with only a 5 ms delay. This kind of bandwidth is now available in the vast majority of VoIP scenarios, an 100ms 3Kbps codec is only relevant for the extreme niches.
- morsch 6y agoThere is voip software that is designed to be low latency, eg Jamulus. It's not as easy to use as Jitsi and Zoom are, though.
- HeadsUpHigh 6y agoThis must be a local( american?) issue. It's all voip here and there's only a minor difference in latency.