3 ms·

This could support extreme levels of compression, right? Send a few seconds of speech sample. Use speech to text and the reconstitute on the client.

dontwearitout 4y ago
Absolutely. These transformer models are trained to autoregressively predict the next token (text, or now audio), and so can be viewed as powerful compression schemes by combining with something like Huffman coding (see https://ml.berkeley.edu/blog/posts/dalle2/ https://ml.berkeley.edu/blog/posts/dalle2/ for a lengthy but great read on how language models can be viewed as sophisticated lossless compression algorithms). The log-likelihood loss function used to train these models can quite directly be interpreted as the average bit-per-token needed to compress the dataset. "Compression is comprehension" - Chaitin
- holoduke 4y ago
  We also need to to have a emotion sheet, just like notes. So you can define along side text the emotion type per second. Angry, happy, exited etc.
londons_explore 4y ago
There aren't awfully many use cases for needing to transmit speech at 50 bytes per second, but where you have beefy GPU's on both ends to compress/decompress. Modern communications links are getting far higher bandwidth than required for voice calls, and where they do glitch, the issue tends to be because of brief disconnections (eg. a half second gap while wifi reconnects to a different base station). Lower bandwidth won't fix that. One of the only usecases I see for this is persistent surveillance - ie. turning every mic on in every phone globally and recording everything, and being able to transmit it back to a 'national library of sounds'. 5 billion phones * 50 bytes/second * 365 days / 10 (silence/duplication/compression factor). Works out to about 200 hard drives per day of retention. I'm sure there would be a lot of governments who would love to be able to listen to any conversation anywhere, in the name of 'security'. And some governments needn't do it secretly - they can just write a law that all phones sold in the country must support always-on listening as a feature controllable by mobile networks.
- BoxOfRain 4y ago
  >There aren't awfully many use cases for needing to transmit speech at 50 bytes per second, but where you have beefy GPU's on both ends to compress/decompress. Not something I do myself but I bet the amateur radio community would love this, they're all about stuffing as much data as you can into a narrowband shortwave channel.

dontwearitout 4y ago

Absolutely. These transformer models are trained to autoregressively predict the next token (text, or now audio), and so can be viewed as powerful compression schemes by combining with something like Huffman coding (see https://ml.berkeley.edu/blog/posts/dalle2/ https://ml.berkeley.edu/blog/posts/dalle2/ for a lengthy but great read on how language models can be viewed as sophisticated lossless compression algorithms). The log-likelihood loss function used to train these models can quite directly be interpreted as the average bit-per-token needed to compress the dataset. "Compression is comprehension" - Chaitin

holoduke 4y ago
We also need to to have a emotion sheet, just like notes. So you can define along side text the emotion type per second. Angry, happy, exited etc.

londons_explore 4y ago

There aren't awfully many use cases for needing to transmit speech at 50 bytes per second, but where you have beefy GPU's on both ends to compress/decompress. Modern communications links are getting far higher bandwidth than required for voice calls, and where they do glitch, the issue tends to be because of brief disconnections (eg. a half second gap while wifi reconnects to a different base station). Lower bandwidth won't fix that. One of the only usecases I see for this is persistent surveillance - ie. turning every mic on in every phone globally and recording everything, and being able to transmit it back to a 'national library of sounds'. 5 billion phones * 50 bytes/second * 365 days / 10 (silence/duplication/compression factor). Works out to about 200 hard drives per day of retention. I'm sure there would be a lot of governments who would love to be able to listen to any conversation anywhere, in the name of 'security'. And some governments needn't do it secretly - they can just write a law that all phones sold in the country must support always-on listening as a feature controllable by mobile networks.

BoxOfRain 4y ago
>There aren't awfully many use cases for needing to transmit speech at 50 bytes per second, but where you have beefy GPU's on both ends to compress/decompress. Not something I do myself but I bet the amateur radio community would love this, they're all about stuffing as much data as you can into a narrowband shortwave channel.