14 ms·

Google open-sources the Lyra audio codec

245 points by chmaynard 6y ago

levosmetalo 6y ago
I hope this never takes off. This whole machine learning, optimization etc, story, but the end goal is that Google can easily transcribe your voice calls and store it as text. Then it can apply all shady practices that it previously was too expensive to do because storing voice and extracting information from it required huge storage costs and actual human labour. Or worst, just imagine what some government you don't trust could do with all those voice call transcripts.
- sreekotay 6y ago
  I mean...this is them open sourcing it?
- mgraczyk 6y ago
  This codec has nothing to do with what you're worried about. There's no current technical limitation preventing what you're describing. Google doesn't do it because it makes no sense for their business and because your phone calls aren't routed through Google's servers. Governments outside the US are already doing it.
- DeepYogurt 6y ago
  > This whole machine learning, optimization etc, story, but the end goal is that Google can easily transcribe your voice calls and store it as text. Can they not do that with opus?
- barbazoo 6y ago
  It sounds more like a "offline" codec, not a Google service compressing your voice so I don't immediately see how Google would violate our privacy here this time.
- rektide 6y ago
  This will make voices radically more correlatable, most likely. It's a more effective model for voice, it has run endless regressions & found better patterns to model human sounds upon. That could well make processing & comparing pieces of speech data less computationally expensive. I don't see much relation to surveillance & transcription issues. This technology does not, would not change the field of battle significantly, if such a battle were about. Which it probably is, in some countries, perhaps even applying to Google-touched, -relayed, or Google-held data.
sreekotay 6y ago
In practical terms, very impressive. Anyone know what latency is like? Feels a domain where people who have not experienced low latency full duplex cannot fully appreciate why voice has faded in everyday life...
- Ajedi32 6y ago
  Sounds like at least +40ms of latency: > features, are extracted in chunks of 40ms, then compressed and sent over the network
robert_foss 6y ago
Encoding takes >40ms? Opus takes 5-26.5ms. Apparently 150ms[1] is the generally accepted upper bound for call latency. I think the article could do with some bandwidth/quality/latency/power comparisons to other codecs. [1] https://en.wikipedia.org/wiki/Latency_(audio) https://en.wikipedia.org/wiki/Latency_(audio)
- pjc50 6y ago
  Yeah, AMR (for GSM) is 10ms as well.
- Bedon292 6y ago
  I don't think it is discussing encoding time in the article, it says "features are extracted in chunks of 40ms". My reading is that its breaking down the speech into 40ms chunks, compressing it, and sending that.
  - 0b01 6y ago
    But since the buffer size has to be 40ms then so the minimum latency is 40ms
    - zamadatix 6y ago
      Sure latency ends up being 40 ms but that's a function of needing to wait to send the encoded data + network headers at 6 Kbps not a function of the encoder being slow holding everything up.
- ksec 6y ago
  >These speech attributes, also called features, are extracted in chunks of 40ms, then compressed and sent over the network. So while Encoding doesn't take 40ms, the latency + encoding will indeed be 40ms+. 150ms is the End to End Latency, which is basically everything from Encoding + Network + Decoding. We cant beat the speed of light on our fibre network. We can certainly do something with Encoding and Decoding. And Lyra doesn't seems to help with that case here. Something I pointed out last time Lyra was on HN. I think Opus default to 20ms with option of 10ms slot ( excluding Encoding speed ) at the expense of quality. What we really need is higher bitrate, lower latency and higher quality codec. Which is sort of the exact opposite of what Lyra is offering.
jacobn 6y ago
Sounds like NVIDIA's Maxine [1], but for voice? 1: https://developer.nvidia.com/maxine https://developer.nvidia.com/maxine
unixhero 6y ago
I used to be fond of Google products.
pjc50 6y ago
One thing I'm slightly worried about "machine learning" in compression rather than conventional everything-is-sines mathematical approaches is the possibility of odd nonlinear errors. Remember the photocopier that worked by OCR and would occasionally mis-transcribe numbers? I don't mind compressing a phoneme to <unintelligible> as much as I would mind it compressing it to a clearly audible different phoneme.
- minikites 6y ago
  >Remember the photocopier that worked by OCR and would occasionally mis-transcribe numbers? For those who don't remember: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
- smnrchrds 6y ago
  This one too: https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
  - ekianjo 6y ago
    Thats not a compression algorithm
    - astrange 6y ago
      A supersampling algorithm is a (de)compression algorithm. You give it an image and it gives you an "decompressed" image. It's not a very good one though.
- rexreed 6y ago
  The OCR issue was the first thing I thought about. Machine learning is probabilistic, not deterministic, so in the case of S being converted to 5 (or 6 to 8, etc.), which definitely impacts numerical data in the case of the OCR stuff, we can expect similar voice mis-classifications. Perhaps "You're fine" might get misclassified as "you're fired".
jagger27 6y ago
I find that Lyra sounds good at first but it can chop off hard consonants in certain scenarios. It sort of sounds like slightly slurred speech. Anyone else getting that impression from their samples?
ent101 6y ago
Discontinued in 3... 2... 1...
- devops000 6y ago
  Give them at least 3-4 years ;)
whoisburbansky 6y ago
Since this is explicitly targeted at "the next billion users," do we have any sense of how well-optimized this is on non-English audio corpuses? I can't imagine that a model trained primarily on English/Western phonemes would perform as well on the rest of the world.
- PaulHoule 6y ago
  They say they tested it on 70+ languages.
  - beaconstudios 6y ago
    which is less than the number of spoken languages in India alone.
    - reaperducer 6y ago
      Or even in New York City public schools.
    - core-questions 6y ago
      Wonder if India will ever go through a forced linguistic convergence like China did
      - sudhirj 6y ago
        Unlikely, there's too much pride in each local language. Might all converge on English over a couple of generations, though, but more for commercial reasons.
  - whoisburbansky 6y ago
    Ah you're right. I couldn't find it on the original link in this post, but the post links to https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-codec-for.html https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-..., which mentions the 70+ languages statistic under the "Fairness" section. Thanks!
pessimizer 6y ago
I've been waiting for an audio codec that could actually silently change the words I've said. https://www.zdnet.com/article/xerox-scanners-alter-numbers-in-scanned-documents/ https://www.zdnet.com/article/xerox-scanners-alter-numbers-i...
- plzbo 6y ago
  Link to the website of the person who found the error in the first place: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
flohofwoe 6y ago
Why is the demo link towards the bottom of the post pointing to the Basis Universal repository (which is a texture compressor)? https://github.com/BinomialLLC/basis_universal/tree/master/webgl https://github.com/BinomialLLC/basis_universal/tree/master/w... Copy-pasta error, or did they run the post through Lyra? ;)
PaulHoule 6y ago
It looks like "the model" has 5MB worth of coefficients so it is no problem fitting it into phones, ham radios, etc. (Radio hams badly need a good digital speech codec for VHF/UHF operation)
devops000 6y ago
404: NOT FOUND https://basis-universal-webgl.vercel.app/texture/ https://basis-universal-webgl.vercel.app/texture/ Where else I can see a demo?
cbdumas 6y ago
Anyone listening to the sample audio linked to in the article should read this note from the last time this was discussed on HN: https://news.ycombinator.com/item?id=26309787 https://news.ycombinator.com/item?id=26309787 Summary: the Lyra audio samples are louder which muddies the comparison
te0006 6y ago
"Please note that there is a closed-source kernel used for math operations that is linked via a shared object called libsparse_inference.so. We provide the libsparse_inference.so library to be linked, but are unable to provide source for it. This is the reason that a specific toolchain/compiler is required.* - README
- ncmncm 6y ago
  [update: proprietary .so] They should re-implement the needed bits of libsparse_inference before releasing this thing. Otherwise it's just a distraction. Probably they should get it building with something other than Bazel, too.
  - danaliv 6y ago
    It's not a kernel module, it's a compute kernel. Nothing to do with operating systems. They provide versions for android-arm64 and linux-x86_64. The fine README says it builds and runs on Ubuntu 20.04.
    - posguy 6y ago
      Ah, so Lyra today will not work on RISC-V, i386, Power, MIPS, lower end or older ARM chips like the Allwinner H3 (very popular in Single Board Computers) and any other new architecture that comes out?
      - mark4o 6y ago
        It won't even work on Windows, macOS, or iOS.
- skybrian 6y ago
  Yes, that will have to be removed as part of the effort of porting it to new platforms.
  - ocdtrekkie 6y ago
sandGorgon 6y ago
is the training code opensource ?
ncmncm 6y ago
A more useful system would take Opus-compressed data as input and feature-extract that, presumably faster than this thing. Bonus for not requiring a proprietary library like libsparse_inference.so. Also, instead of encoding independent 40ms segments, it should be much better to encode 10ms segments given the previous 30ms.
ProAm 6y ago
Isn't Lyra the name of Facebook's cryptocurrency as well? I cannot remember if that project was shelved.
- mgraczyk 6y ago
  Are you thinking of Libra, now called Diem? https://en.wikipedia.org/wiki/Diem_(digital_currency) https://en.wikipedia.org/wiki/Diem_(digital_currency)
  - ProAm 6y ago
    Yes I was. Thanks, I had them mixed up.
rodjons 6y ago
Is there any difference with another audio codec? It's great to see that another player in the market—this time, it's machine learning that produces high-quality calls. I'll keep an eye on the impact in the future. This architecture will surely disrupt our communication industry.
londons_explore 6y ago
Google misses the mark here... Bad internet connectivity in the developing world isn't "only 56kbps" as some people think. It's "random bursts of fast with random 30 second gaps of no connectivity at all". It's routed through 3 layers of proxies and firewalls which block random stuff and not others, while disconnecting long running connections. Oh, and it'll be expensive per MB. To that end, Lyra helps with the expense of a data connection, but is unusable for long voice calls. What would help more is a text chat system like WhatsApp. Oh right - WhatsApp is already wildly popular in most of the developing world for mostly this reason.
- villasv 6y ago
  > Oh right - WhatsApp is already wildly popular in most of the developing world for mostly this reason. Not only that, but carriers will often advertise plans with "unlimited Internet for Facebook and WhatsApp" (a punch in the face of net neutrality). So not only WhatsApp has more impact with audio messages when audio calls are too unstable, audio calls already substitute the bulk of phone calls even for people who have shitty data plans. This is what my carrier says on their most basic offering: > What does WhatsApp Unlimited mean? > The benefit is granted automatically, without the need for activation. And the use of the app is unlimited to send messages, audios, photos, videos, in addition to making voice calls. Only video calls that are discounted from the internet package, as well as access to external links.
- herodoturtle 6y ago
  Heya, please could you unpack your reasoning a little bit more? You said: > WhatsApp is already wildly popular in most of the developing world for mostly this reason. I can't speak for the majority of the developing world, but here in South Africa, WhatsApp is indeed the predominant communications app. That being said, WhatsApp voice calls are also used here quite a bit. So with that in mind, and reading from the article: > Lyra compresses raw audio down to 3kbps for quality that compares favourably to other codecs To me 3kbps sounds pretty great, and might actually work out cheaper / better than one might imagine. So I'm just wondering, how does WhatsApp voice call data usage compare to Lyra? Also whilst South Africa is indeed a developing country (where, among other things, the price of data is proportionately high relative to average household income), the cellular network infrastructure is excellent. So I don't think the random bursts of connectivity you describe are as big of an issue here, whereas the price of data most certainly is. In which case, I can definitely see a market for Lyra (assuming the 3kbps is indeed vastly superior to WhatsApp's data usage for a voice call). Hope that makes sense but I'd be happy to extrapolate a little further :-)
chintan 6y ago
Now what could go wrong? 4 dots becomes 3 dots...[1] 1). Silicon Valley - Finale S6E7 https://www.youtube.com/watch?v=48Y77jSSHGU https://www.youtube.com/watch?v=48Y77jSSHGU
rektide 6y ago
There's huge wins but the grandiosity of "enabling voice calls" is grating. I don't think this will open many users to voice communication. It will reduce data-costs in a way that has an impact on a significant amount of people's bottom line. But I feel manipulated with the current headline, and by the long extended lack of ability to mix the very real hope with some measure of humility.
dang 6y ago
Recent past threads on this: Lyra audio codec enables high-quality voice calls at 3 kbps bitrate - https://news.ycombinator.com/item?id=26300229 https://news.ycombinator.com/item?id=26300229 - March 2021 (198 comments) Lyra: A New Very Low-Bitrate Codec for Speech Compression - https://news.ycombinator.com/item?id=26279891 https://news.ycombinator.com/item?id=26279891 - Feb 2021 (25 comments) Is there significant new information here? https://hn.algolia.com/?dateRange=all&page=0&prefix=false&sort=byDate&type=comment&query=%22significant%20new%20information%22%20by%3Adang https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... Edit: it seems the SNI is the open-sourcing. I've changed the title to say that now. Corporate press releases are generally an exception to HN's rules about titles and original sources: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=corporate%20press%20release%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor....
- ehsankia 6y ago
  > Is there significant new information here? The fact that it's not open-source? The blog post is posted today and from Google themselves, so I assume there's new information.
markstos 6y ago
Notable that the two post authors sign it with " - Chrome", indicating I presume they are Chrome team members.
astlouis44 6y ago
This is going to be VERY useful for WebXR social platforms.
throwaway81523 6y ago
Another reason for end-to-end speech encryption: to keep your cleartext voice signal away from these overaggressive codecs changing the words. I can understand the need for a super low bandwidth codec on top of Mt. Everest, but 64 kbit PCM was good enough for our grandparents' landlines (or 13 kbit GSM for their mobiles) and it's good enough for us.
- LeoPanthera 6y ago
  What a spectacular failure of imagination. Why change anything ever, right? I supposed dial-up modems were good enough for you too. Everyone is imagining that codecs like this will "change your words" but no-one has provided examples of that actually happening. I don't believe it.
  - andrepd 6y ago
    There are examples posted elsewhere in this thread, although from a beta version. Also, Opus already blows any requirements, either in terms of latency or in terms of size. Adding such unreliability for minimal gain seems foolish.
  - throwaway81523 6y ago
    Spectacular failure of imagination? I mean it's a speech codec, an incremental improvement over the many out there that work fine. We're no longer in a world where voice calls dominate the world's telecom bandwidth usage. We routinely receive a megabyte of Javascript and ads and crap to display a 288 character tweet. Soon there will be 5G everywhere, so we'll get 10MB of JS etc. to see the same 288 character tweet. 1MB is 10+ minutes of full-rate GSM, or a lot more than that of Opus. If Lyra is really free (no blobs) and its computational requirements don't make us churn our phone hardware yet again, then great, it can reduce the already very low cost of voice calls by another smidgen, increasing carrier margins while almost certainly not showing in lower prices to the end user. So at that end of things, it's tolerable, while it would be horrible if (say) it were patented and became a standard, so that FOSS VOIP clients became non-interoperable with what the big vendors were using. Lyra is more transformative in some extreme niche areas of extremely limited bandwidth, say spacecraft radios or handheld satellite phones or whatever. Those applications already use super low bandwidth codecs that sound like crap. So Lyra won't really save bits, but it will help intelligibility a lot by sounding better in the same bits.
squarefoot 6y ago
Doesn't seem that better compared to Codec2 which is already fully Open Source (LGPL), even taking into account that Codec2's examples originals are already of much worse quality than the ones on Lyra's website. I'd be curious to hear both working on the same set of audio samples. https://www.rowetel.com/?page_id=452 https://www.rowetel.com/?page_id=452
- Seirdy 6y ago
  Agreed; codec2 doesn't alter speech as aggressively, require proprietary components, or have as strong a connection to Google.
miohtama 6y ago
If I remember correctly the original landline audio was 64kpbs, 8000 Hz. So Lyra is 1/20 of this. And probably still sounds better.
- YarickR2 6y ago
  Original landline audio was/is analog
  - deleted 6y ago
    [deleted]
  - colanderman 6y ago
    Not an authoritative source, but (as a point of interest) analog landlines seem to be specified as 24 dB SNR and 300-3000 Hz passband [1], giving ~21.5 kbps information rate [2]. [1] https://www.tschmidt.com/writings/POTS_Modem_Impairments.htm https://www.tschmidt.com/writings/POTS_Modem_Impairments.htm [2] https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theorem https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theore...
- posguy 6y ago
  PCMU/PCMA (G.711μ and G.711a) are not original landline quality audio, but rather what Bell Systems felt they could get away with passing off as a toll quality call in 1972. Lyra will likely sound better, but the reproduction accuracy is apt to be quite a bit poorer as many others have commented. G.711 was created to require nearly no processing (its nearly raw PCM data from a sound card after all) while operating at reasonable bitrates, Lyra looks much more computationally intensive and will likely only run on smartphones in the next few years. Edit: Is Lyra a significant improvement over modern Opus at 8Kbps? You can buy a Grandstream HT802 analog telephone adapter for ~$30 and its DSP can decode Opus today, whereas Lyra will require orders of magnitude more power to decode while providing much worse reproduction accuracy.
  - nl 6y ago
    > Is Lyra a significant improvement over modern Opus at 8Kbps? It is over 6Kbps Opus[1]. > You can buy a Grandstream HT802 analog telephone adapter for ~$30 and its DSP can decode Opus today A RaspberryPi Zero will provide more than sufficient power for Lyra (it was originally implemented on a Pixel 2). That's ~$10 [1] https://www.cnx-software.com/2021/02/28/lyra-audio-codec-enables-3-kbps-bitrate-for-high-quality-voice-calls/ https://www.cnx-software.com/2021/02/28/lyra-audio-codec-ena...
Thaxll 6y ago
Can't wait to try that in ffmpeg!
deleted 6y ago
[deleted]
p1mrx 6y ago
This seems kind of unnecessary, compared to Opus at ~10 kbps. If you're sending IPv6+UDP in 40 ms chunks, that's 9.6 kbps just from the packet headers (25 Hz * 40+8 bytes). When the voice payload is smaller than the packet headers, you're well into diminishing returns territory.
- posguy 6y ago
  Opus at 8Kbps sounds better, and commodity, inexpensive hardware like the Grandstream HT802 Analog Telephone Adapter supports this codec today (along with any cheap Android phone). Lyra as it stands today will not support anything outside of x86-64 and ARM64 without rewriting the proprietary kernel it relies on.
bilal4hmed 6y ago
I have been using Duo more for audio calls lately and the call quality has been excellent. Compared to WhatsApp its much much better which often times can mimic the sound quality of a regular phone call. Ive tested in the US and with my family in India, where the connection isnt the greatest
- perryizgr8 6y ago
  Huh, for me whatsapp calls are way better quality than duo. Wonder what causes such variations.
  - bilal4hmed 6y ago
    I'm using a variety of devices with similar results - pixel 4xl, windows, Linux with Firefox and chrome and chromeos. Usually on wifi and sometimes tmo lte my family in India is 90% of the time using lte
arbirk 6y ago
https://thispersondoesnotexist.com/ https://thispersondoesnotexist.com/
swiley 6y ago
I'd hope they would, who cares about a closed codec?
walrus01 6y ago
In the side by side comparisons I've seen between opus and lyra at 6kbps, lyra sounds remarkably better.
xcodevn 6y ago
My immediate thought: train a Transformer (or Tacotron2) that transforms text to the encoded Lyra codes... And, we will finally have a good real-time open-source text-to-speech system running on mobile devices.
throwaway189262 6y ago
Does anyone know how it compares to Codec2? Opus is great down to ~12kbps but Codec2 is the real contender down at the bottom. And I bet it uses way less CPU than Lyra
SloopJon 6y ago
The Github README shows that the public API uses types from Abseil, a library that "promises no ABI stability, even from one day to the next." That seems problematic.
- ithkuil 6y ago
  Is abseil used at the codec API surface?
peter_d_sherman 6y ago
>"Lyra’s architecture is separated into two pieces, the encoder and decoder. When someone talks into their phone the encoder captures distinctive attributes from their speech. These speech attributes, also called features, are extracted in chunks of 40ms, then compressed and sent over the network. It is the decoder’s job to convert the features back into an audio waveform that can be played out over the listener’s phone speaker. The features are decoded back into a waveform via a generative model. Generative models are a particular type of machine learning model well suited to recreate a full audio waveform from a limited number of features. The Lyra architecture is very similar to traditional audio codecs, which have formed the backbone of internet communication for decades. Whereas these traditional codecs are based on digital signal processing (DSP) techniques, the key advantage for Lyra comes from the ability of the generative model to reconstruct a high-quality voice signal." PDS: Audio Codec meets Machine Learning! I love it!!!

245 points by chmaynard 6y ago

I hope this never takes off. This whole machine learning, optimization etc, story, but the end goal is that Google can easily transcribe your voice calls and store it as text. Then it can apply all shady practices that it previously was too expensive to do because storing voice and extracting information from it required huge storage costs and actual human labour. Or worst, just imagine what some government you don't trust could do with all those voice call transcripts.

sreekotay 6y ago
I mean...this is them open sourcing it?
mgraczyk 6y ago
This codec has nothing to do with what you're worried about. There's no current technical limitation preventing what you're describing. Google doesn't do it because it makes no sense for their business and because your phone calls aren't routed through Google's servers. Governments outside the US are already doing it.
DeepYogurt 6y ago
> This whole machine learning, optimization etc, story, but the end goal is that Google can easily transcribe your voice calls and store it as text. Can they not do that with opus?
barbazoo 6y ago
It sounds more like a "offline" codec, not a Google service compressing your voice so I don't immediately see how Google would violate our privacy here this time.
rektide 6y ago
This will make voices radically more correlatable, most likely. It's a more effective model for voice, it has run endless regressions & found better patterns to model human sounds upon. That could well make processing & comparing pieces of speech data less computationally expensive. I don't see much relation to surveillance & transcription issues. This technology does not, would not change the field of battle significantly, if such a battle were about. Which it probably is, in some countries, perhaps even applying to Google-touched, -relayed, or Google-held data.

sreekotay 6y ago

In practical terms, very impressive. Anyone know what latency is like? Feels a domain where people who have not experienced low latency full duplex cannot fully appreciate why voice has faded in everyday life...

Ajedi32 6y ago
Sounds like at least +40ms of latency: > features, are extracted in chunks of 40ms, then compressed and sent over the network

robert_foss 6y ago

Encoding takes >40ms? Opus takes 5-26.5ms. Apparently 150ms[1] is the generally accepted upper bound for call latency. I think the article could do with some bandwidth/quality/latency/power comparisons to other codecs. [1] https://en.wikipedia.org/wiki/Latency_(audio) https://en.wikipedia.org/wiki/Latency_(audio)

pjc50 6y ago
Yeah, AMR (for GSM) is 10ms as well.
Bedon292 6y ago
I don't think it is discussing encoding time in the article, it says "features are extracted in chunks of 40ms". My reading is that its breaking down the speech into 40ms chunks, compressing it, and sending that.
- 0b01 6y ago
  But since the buffer size has to be 40ms then so the minimum latency is 40ms
  - zamadatix 6y ago
    Sure latency ends up being 40 ms but that's a function of needing to wait to send the encoded data + network headers at 6 Kbps not a function of the encoder being slow holding everything up.
ksec 6y ago
>These speech attributes, also called features, are extracted in chunks of 40ms, then compressed and sent over the network. So while Encoding doesn't take 40ms, the latency + encoding will indeed be 40ms+. 150ms is the End to End Latency, which is basically everything from Encoding + Network + Decoding. We cant beat the speed of light on our fibre network. We can certainly do something with Encoding and Decoding. And Lyra doesn't seems to help with that case here. Something I pointed out last time Lyra was on HN. I think Opus default to 20ms with option of 10ms slot ( excluding Encoding speed ) at the expense of quality. What we really need is higher bitrate, lower latency and higher quality codec. Which is sort of the exact opposite of what Lyra is offering.

jacobn 6y ago

Sounds like NVIDIA's Maxine [1], but for voice? 1: https://developer.nvidia.com/maxine https://developer.nvidia.com/maxine

unixhero 6y ago

I used to be fond of Google products.

pjc50 6y ago

One thing I'm slightly worried about "machine learning" in compression rather than conventional everything-is-sines mathematical approaches is the possibility of odd nonlinear errors. Remember the photocopier that worked by OCR and would occasionally mis-transcribe numbers? I don't mind compressing a phoneme to <unintelligible> as much as I would mind it compressing it to a clearly audible different phoneme.

minikites 6y ago
>Remember the photocopier that worked by OCR and would occasionally mis-transcribe numbers? For those who don't remember: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
smnrchrds 6y ago
This one too: https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
- ekianjo 6y ago
  Thats not a compression algorithm
  - astrange 6y ago
    A supersampling algorithm is a (de)compression algorithm. You give it an image and it gives you an "decompressed" image. It's not a very good one though.
rexreed 6y ago
The OCR issue was the first thing I thought about. Machine learning is probabilistic, not deterministic, so in the case of S being converted to 5 (or 6 to 8, etc.), which definitely impacts numerical data in the case of the OCR stuff, we can expect similar voice mis-classifications. Perhaps "You're fine" might get misclassified as "you're fired".

jagger27 6y ago

I find that Lyra sounds good at first but it can chop off hard consonants in certain scenarios. It sort of sounds like slightly slurred speech. Anyone else getting that impression from their samples?

ent101 6y ago

Discontinued in 3... 2... 1...

devops000 6y ago
Give them at least 3-4 years ;)

whoisburbansky 6y ago

Since this is explicitly targeted at "the next billion users," do we have any sense of how well-optimized this is on non-English audio corpuses? I can't imagine that a model trained primarily on English/Western phonemes would perform as well on the rest of the world.

PaulHoule 6y ago
They say they tested it on 70+ languages.
- beaconstudios 6y ago
  which is less than the number of spoken languages in India alone.
  - reaperducer 6y ago
    Or even in New York City public schools.
  - core-questions 6y ago
    Wonder if India will ever go through a forced linguistic convergence like China did
    - sudhirj 6y ago
      Unlikely, there's too much pride in each local language. Might all converge on English over a couple of generations, though, but more for commercial reasons.
- whoisburbansky 6y ago
  Ah you're right. I couldn't find it on the original link in this post, but the post links to https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-codec-for.html https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-..., which mentions the 70+ languages statistic under the "Fairness" section. Thanks!

pessimizer 6y ago

I've been waiting for an audio codec that could actually silently change the words I've said. https://www.zdnet.com/article/xerox-scanners-alter-numbers-in-scanned-documents/ https://www.zdnet.com/article/xerox-scanners-alter-numbers-i...

plzbo 6y ago
Link to the website of the person who found the error in the first place: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

flohofwoe 6y ago

Why is the demo link towards the bottom of the post pointing to the Basis Universal repository (which is a texture compressor)? https://github.com/BinomialLLC/basis_universal/tree/master/webgl https://github.com/BinomialLLC/basis_universal/tree/master/w... Copy-pasta error, or did they run the post through Lyra? ;)

PaulHoule 6y ago

It looks like "the model" has 5MB worth of coefficients so it is no problem fitting it into phones, ham radios, etc. (Radio hams badly need a good digital speech codec for VHF/UHF operation)

devops000 6y ago

404: NOT FOUND https://basis-universal-webgl.vercel.app/texture/ https://basis-universal-webgl.vercel.app/texture/ Where else I can see a demo?

cbdumas 6y ago

Anyone listening to the sample audio linked to in the article should read this note from the last time this was discussed on HN: https://news.ycombinator.com/item?id=26309787 https://news.ycombinator.com/item?id=26309787 Summary: the Lyra audio samples are louder which muddies the comparison

te0006 6y ago

"Please note that there is a closed-source kernel used for math operations that is linked via a shared object called libsparse_inference.so. We provide the libsparse_inference.so library to be linked, but are unable to provide source for it. This is the reason that a specific toolchain/compiler is required.* - README

ncmncm 6y ago
[update: proprietary .so] They should re-implement the needed bits of libsparse_inference before releasing this thing. Otherwise it's just a distraction. Probably they should get it building with something other than Bazel, too.
- danaliv 6y ago
  It's not a kernel module, it's a compute kernel. Nothing to do with operating systems. They provide versions for android-arm64 and linux-x86_64. The fine README says it builds and runs on Ubuntu 20.04.
  - posguy 6y ago
    Ah, so Lyra today will not work on RISC-V, i386, Power, MIPS, lower end or older ARM chips like the Allwinner H3 (very popular in Single Board Computers) and any other new architecture that comes out?
    - mark4o 6y ago
      It won't even work on Windows, macOS, or iOS.
skybrian 6y ago
Yes, that will have to be removed as part of the effort of porting it to new platforms.
- ocdtrekkie 6y ago

sandGorgon 6y ago

is the training code opensource ?

ncmncm 6y ago

A more useful system would take Opus-compressed data as input and feature-extract that, presumably faster than this thing. Bonus for not requiring a proprietary library like libsparse_inference.so. Also, instead of encoding independent 40ms segments, it should be much better to encode 10ms segments given the previous 30ms.

ProAm 6y ago

Isn't Lyra the name of Facebook's cryptocurrency as well? I cannot remember if that project was shelved.

mgraczyk 6y ago
Are you thinking of Libra, now called Diem? https://en.wikipedia.org/wiki/Diem_(digital_currency) https://en.wikipedia.org/wiki/Diem_(digital_currency)
- ProAm 6y ago
  Yes I was. Thanks, I had them mixed up.

rodjons 6y ago

Is there any difference with another audio codec? It's great to see that another player in the market—this time, it's machine learning that produces high-quality calls. I'll keep an eye on the impact in the future. This architecture will surely disrupt our communication industry.

londons_explore 6y ago

Google misses the mark here... Bad internet connectivity in the developing world isn't "only 56kbps" as some people think. It's "random bursts of fast with random 30 second gaps of no connectivity at all". It's routed through 3 layers of proxies and firewalls which block random stuff and not others, while disconnecting long running connections. Oh, and it'll be expensive per MB. To that end, Lyra helps with the expense of a data connection, but is unusable for long voice calls. What would help more is a text chat system like WhatsApp. Oh right - WhatsApp is already wildly popular in most of the developing world for mostly this reason.

villasv 6y ago
> Oh right - WhatsApp is already wildly popular in most of the developing world for mostly this reason. Not only that, but carriers will often advertise plans with "unlimited Internet for Facebook and WhatsApp" (a punch in the face of net neutrality). So not only WhatsApp has more impact with audio messages when audio calls are too unstable, audio calls already substitute the bulk of phone calls even for people who have shitty data plans. This is what my carrier says on their most basic offering: > What does WhatsApp Unlimited mean? > The benefit is granted automatically, without the need for activation. And the use of the app is unlimited to send messages, audios, photos, videos, in addition to making voice calls. Only video calls that are discounted from the internet package, as well as access to external links.
herodoturtle 6y ago
Heya, please could you unpack your reasoning a little bit more? You said: > WhatsApp is already wildly popular in most of the developing world for mostly this reason. I can't speak for the majority of the developing world, but here in South Africa, WhatsApp is indeed the predominant communications app. That being said, WhatsApp voice calls are also used here quite a bit. So with that in mind, and reading from the article: > Lyra compresses raw audio down to 3kbps for quality that compares favourably to other codecs To me 3kbps sounds pretty great, and might actually work out cheaper / better than one might imagine. So I'm just wondering, how does WhatsApp voice call data usage compare to Lyra? Also whilst South Africa is indeed a developing country (where, among other things, the price of data is proportionately high relative to average household income), the cellular network infrastructure is excellent. So I don't think the random bursts of connectivity you describe are as big of an issue here, whereas the price of data most certainly is. In which case, I can definitely see a market for Lyra (assuming the 3kbps is indeed vastly superior to WhatsApp's data usage for a voice call). Hope that makes sense but I'd be happy to extrapolate a little further :-)

chintan 6y ago

Now what could go wrong? 4 dots becomes 3 dots...[1] 1). Silicon Valley - Finale S6E7 https://www.youtube.com/watch?v=48Y77jSSHGU https://www.youtube.com/watch?v=48Y77jSSHGU

rektide 6y ago

There's huge wins but the grandiosity of "enabling voice calls" is grating. I don't think this will open many users to voice communication. It will reduce data-costs in a way that has an impact on a significant amount of people's bottom line. But I feel manipulated with the current headline, and by the long extended lack of ability to mix the very real hope with some measure of humility.

dang 6y ago

Recent past threads on this: Lyra audio codec enables high-quality voice calls at 3 kbps bitrate - https://news.ycombinator.com/item?id=26300229 https://news.ycombinator.com/item?id=26300229 - March 2021 (198 comments) Lyra: A New Very Low-Bitrate Codec for Speech Compression - https://news.ycombinator.com/item?id=26279891 https://news.ycombinator.com/item?id=26279891 - Feb 2021 (25 comments) Is there significant new information here? https://hn.algolia.com/?dateRange=all&page=0&prefix=false&sort=byDate&type=comment&query=%22significant%20new%20information%22%20by%3Adang https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... Edit: it seems the SNI is the open-sourcing. I've changed the title to say that now. Corporate press releases are generally an exception to HN's rules about titles and original sources: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=corporate%20press%20release%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor....

ehsankia 6y ago
> Is there significant new information here? The fact that it's not open-source? The blog post is posted today and from Google themselves, so I assume there's new information.

markstos 6y ago

Notable that the two post authors sign it with " - Chrome", indicating I presume they are Chrome team members.

astlouis44 6y ago

This is going to be VERY useful for WebXR social platforms.

throwaway81523 6y ago

Another reason for end-to-end speech encryption: to keep your cleartext voice signal away from these overaggressive codecs changing the words. I can understand the need for a super low bandwidth codec on top of Mt. Everest, but 64 kbit PCM was good enough for our grandparents' landlines (or 13 kbit GSM for their mobiles) and it's good enough for us.

LeoPanthera 6y ago
What a spectacular failure of imagination. Why change anything ever, right? I supposed dial-up modems were good enough for you too. Everyone is imagining that codecs like this will "change your words" but no-one has provided examples of that actually happening. I don't believe it.
- andrepd 6y ago
  There are examples posted elsewhere in this thread, although from a beta version. Also, Opus already blows any requirements, either in terms of latency or in terms of size. Adding such unreliability for minimal gain seems foolish.
- throwaway81523 6y ago
  Spectacular failure of imagination? I mean it's a speech codec, an incremental improvement over the many out there that work fine. We're no longer in a world where voice calls dominate the world's telecom bandwidth usage. We routinely receive a megabyte of Javascript and ads and crap to display a 288 character tweet. Soon there will be 5G everywhere, so we'll get 10MB of JS etc. to see the same 288 character tweet. 1MB is 10+ minutes of full-rate GSM, or a lot more than that of Opus. If Lyra is really free (no blobs) and its computational requirements don't make us churn our phone hardware yet again, then great, it can reduce the already very low cost of voice calls by another smidgen, increasing carrier margins while almost certainly not showing in lower prices to the end user. So at that end of things, it's tolerable, while it would be horrible if (say) it were patented and became a standard, so that FOSS VOIP clients became non-interoperable with what the big vendors were using. Lyra is more transformative in some extreme niche areas of extremely limited bandwidth, say spacecraft radios or handheld satellite phones or whatever. Those applications already use super low bandwidth codecs that sound like crap. So Lyra won't really save bits, but it will help intelligibility a lot by sounding better in the same bits.

squarefoot 6y ago

Doesn't seem that better compared to Codec2 which is already fully Open Source (LGPL), even taking into account that Codec2's examples originals are already of much worse quality than the ones on Lyra's website. I'd be curious to hear both working on the same set of audio samples. https://www.rowetel.com/?page_id=452 https://www.rowetel.com/?page_id=452

Seirdy 6y ago
Agreed; codec2 doesn't alter speech as aggressively, require proprietary components, or have as strong a connection to Google.

miohtama 6y ago

If I remember correctly the original landline audio was 64kpbs, 8000 Hz. So Lyra is 1/20 of this. And probably still sounds better.

YarickR2 6y ago
Original landline audio was/is analog
- deleted 6y ago
  [deleted]
- colanderman 6y ago
  Not an authoritative source, but (as a point of interest) analog landlines seem to be specified as 24 dB SNR and 300-3000 Hz passband [1], giving ~21.5 kbps information rate [2]. [1] https://www.tschmidt.com/writings/POTS_Modem_Impairments.htm https://www.tschmidt.com/writings/POTS_Modem_Impairments.htm [2] https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theorem https://en.wikipedia.org/wiki/Shannon%E2%80%93Hartley_theore...
posguy 6y ago
PCMU/PCMA (G.711μ and G.711a) are not original landline quality audio, but rather what Bell Systems felt they could get away with passing off as a toll quality call in 1972. Lyra will likely sound better, but the reproduction accuracy is apt to be quite a bit poorer as many others have commented. G.711 was created to require nearly no processing (its nearly raw PCM data from a sound card after all) while operating at reasonable bitrates, Lyra looks much more computationally intensive and will likely only run on smartphones in the next few years. Edit: Is Lyra a significant improvement over modern Opus at 8Kbps? You can buy a Grandstream HT802 analog telephone adapter for ~$30 and its DSP can decode Opus today, whereas Lyra will require orders of magnitude more power to decode while providing much worse reproduction accuracy.
- nl 6y ago
  > Is Lyra a significant improvement over modern Opus at 8Kbps? It is over 6Kbps Opus[1]. > You can buy a Grandstream HT802 analog telephone adapter for ~$30 and its DSP can decode Opus today A RaspberryPi Zero will provide more than sufficient power for Lyra (it was originally implemented on a Pixel 2). That's ~$10 [1] https://www.cnx-software.com/2021/02/28/lyra-audio-codec-enables-3-kbps-bitrate-for-high-quality-voice-calls/ https://www.cnx-software.com/2021/02/28/lyra-audio-codec-ena...

Thaxll 6y ago

Can't wait to try that in ffmpeg!

deleted 6y ago

[deleted]

p1mrx 6y ago

This seems kind of unnecessary, compared to Opus at ~10 kbps. If you're sending IPv6+UDP in 40 ms chunks, that's 9.6 kbps just from the packet headers (25 Hz * 40+8 bytes). When the voice payload is smaller than the packet headers, you're well into diminishing returns territory.

posguy 6y ago
Opus at 8Kbps sounds better, and commodity, inexpensive hardware like the Grandstream HT802 Analog Telephone Adapter supports this codec today (along with any cheap Android phone). Lyra as it stands today will not support anything outside of x86-64 and ARM64 without rewriting the proprietary kernel it relies on.

bilal4hmed 6y ago

I have been using Duo more for audio calls lately and the call quality has been excellent. Compared to WhatsApp its much much better which often times can mimic the sound quality of a regular phone call. Ive tested in the US and with my family in India, where the connection isnt the greatest

perryizgr8 6y ago
Huh, for me whatsapp calls are way better quality than duo. Wonder what causes such variations.
- bilal4hmed 6y ago
  I'm using a variety of devices with similar results - pixel 4xl, windows, Linux with Firefox and chrome and chromeos. Usually on wifi and sometimes tmo lte my family in India is 90% of the time using lte

arbirk 6y ago

https://thispersondoesnotexist.com/ https://thispersondoesnotexist.com/

swiley 6y ago

I'd hope they would, who cares about a closed codec?

walrus01 6y ago

In the side by side comparisons I've seen between opus and lyra at 6kbps, lyra sounds remarkably better.

xcodevn 6y ago

My immediate thought: train a Transformer (or Tacotron2) that transforms text to the encoded Lyra codes... And, we will finally have a good real-time open-source text-to-speech system running on mobile devices.

throwaway189262 6y ago

Does anyone know how it compares to Codec2? Opus is great down to ~12kbps but Codec2 is the real contender down at the bottom. And I bet it uses way less CPU than Lyra

SloopJon 6y ago

The Github README shows that the public API uses types from Abseil, a library that "promises no ABI stability, even from one day to the next." That seems problematic.

ithkuil 6y ago
Is abseil used at the codec API surface?

peter_d_sherman 6y ago

>"Lyra’s architecture is separated into two pieces, the encoder and decoder. When someone talks into their phone the encoder captures distinctive attributes from their speech. These speech attributes, also called features, are extracted in chunks of 40ms, then compressed and sent over the network. It is the decoder’s job to convert the features back into an audio waveform that can be played out over the listener’s phone speaker. The features are decoded back into a waveform via a generative model. Generative models are a particular type of machine learning model well suited to recreate a full audio waveform from a limited number of features. The Lyra architecture is very similar to traditional audio codecs, which have formed the backbone of internet communication for decades. Whereas these traditional codecs are based on digital signal processing (DSP) techniques, the key advantage for Lyra comes from the ability of the generative model to reconstruct a high-quality voice signal." PDS: Audio Codec meets Machine Learning! I love it!!!