3 ms·

In an example[1], it sounds decent but i noticed a fuzzy white noise whenever the voice is talking. Is this the algorithm, or compression? If it's the algorithm

In an example[1], it sounds decent but i noticed a fuzzy white noise whenever the voice is talking. Is this the algorithm, or compression? If it's the algorithm, why? [1]: https://soundcloud.com/user-565970875/pocket-article-wavernn-and-tacotron2 https://soundcloud.com/user-565970875/pocket-article-wavernn...

throwawaysea 6y ago
I actually don't hear the fuzzy white noise, but maybe it's because of my tinnitus. Is it during a certain part of the recording? To my ears this sounds surprisingly high fidelity and natural sounding.
- adkadskhj 6y ago
  It's only during when the .. "person" talks. Which makes it quite noticeable to me because it starts and stops. It is rather faint, so i might not even notice it if it was consistent.
eddyg 6y ago
I hear it as well, even when using the speaker on my phone and not headphones (where it seems like it would be even more noticeable).
xcodevn 6y ago
This is a well known problem. The noise is due to mu-law compression. The 16 bit audio samples are compressed to 8, 9, or 10 bits before feeding to the neutral net. The reason is because predicting a categorical distribution of 2^16 values requires too many parameters. The noise was also in samples from the famous Wavenet from Deepmind (they used 8 bit mu law). There are two ways to avoid this: 1. predict 8 high (coarse) bits, 8 low (fine) bits separately as in the original waveRNN paper. 2. use a mixture of logistic distributions as the predictive output as in the recent Lyra vocoder from Google.
- Tade0 6y ago
  How does the number of parameters scale with resolution? Specifically, how much slower this would be if the audio was, say, 10 bits? I recall a lab exercise in college where we were supposed to increase the resolution of a quantizer until we reached a decent tone and 10 bits were the point at which we reached satisfying quality.
  - xcodevn 6y ago
    It is a single matrix multiplication to predict probabilities of all possible outputs. For example, with a hidden state of 1024 dimensions, and 8 bits output, it is 1024x256 parameters. 10 bits will need 1024x1024 params.
erogol 6y ago
It mainly reflects the quality of the trained dataset, the earlier stages of the project and some experiments. I suggest you the check the latest uploads on soundcloud.
kdavis 6y ago
You should listen to newer examples[1]. It's improved a lot in the intervening time period. [1] https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/

throwawaysea 6y ago

I actually don't hear the fuzzy white noise, but maybe it's because of my tinnitus. Is it during a certain part of the recording? To my ears this sounds surprisingly high fidelity and natural sounding.

adkadskhj 6y ago
It's only during when the .. "person" talks. Which makes it quite noticeable to me because it starts and stops. It is rather faint, so i might not even notice it if it was consistent.

eddyg 6y ago

I hear it as well, even when using the speaker on my phone and not headphones (where it seems like it would be even more noticeable).

xcodevn 6y ago

This is a well known problem. The noise is due to mu-law compression. The 16 bit audio samples are compressed to 8, 9, or 10 bits before feeding to the neutral net. The reason is because predicting a categorical distribution of 2^16 values requires too many parameters. The noise was also in samples from the famous Wavenet from Deepmind (they used 8 bit mu law). There are two ways to avoid this: 1. predict 8 high (coarse) bits, 8 low (fine) bits separately as in the original waveRNN paper. 2. use a mixture of logistic distributions as the predictive output as in the recent Lyra vocoder from Google.

Tade0 6y ago
How does the number of parameters scale with resolution? Specifically, how much slower this would be if the audio was, say, 10 bits? I recall a lab exercise in college where we were supposed to increase the resolution of a quantizer until we reached a decent tone and 10 bits were the point at which we reached satisfying quality.
- xcodevn 6y ago
  It is a single matrix multiplication to predict probabilities of all possible outputs. For example, with a hidden state of 1024 dimensions, and 8 bits output, it is 1024x256 parameters. 10 bits will need 1024x1024 params.

erogol 6y ago

It mainly reflects the quality of the trained dataset, the earlier stages of the project and some experiments. I suggest you the check the latest uploads on soundcloud.

kdavis 6y ago

You should listen to newer examples[1]. It's improved a lot in the intervening time period. [1] https://erogol.github.io/ddc-samples/ https://erogol.github.io/ddc-samples/