5 ms·
Translatotron: An End-to-End Speech-to-Speech Translation Model
- bowmessage 7y agoIs it just me or are the final results (Translatotron translation) not playable, while the initial audio samples work fine?
- barrystaes 7y agoNot just you, same problem here. (Firefox on Windows) Workaround: If you look at the WAV file the player loads (right click > Inspect Element), that URL is playable in a new tab/browser window.
- fartcannon 7y agoDid you by bchance notice what is different regarding the other files?
- gattilorenz 7y agoThe original (source) is a PCM S16 LE mono file, 8kHz/16 bit. The translations are 32 bits float LE mono file, 24kHz/32 bit. Firefox supports 8-bit and 16-bit PCM only ( https://support.mozilla.org/en-US/kb/html5-audio-and-video-firefox#w_open-media https://support.mozilla.org/en-US/kb/html5-audio-and-video-f... ). I don't know why, exactly.
- londons_explore 7y agoThats silly... 16 bit audio really doesn't have enough dynamic range for many uses now. Listen to a bit of classical music with very loud and very quiet bits and you'll hear quantization hiss in the quiet bits. 16 bits allows a dynamic range of 96dB while sounds you might come across in everyday life can peak over 130dB briefly.
- deleted 7y ago[deleted]
- bildung 7y agoWhile the actual analog sounds may indeed have a higher signal to noise ratio than 96db, most ADCs haven't, i.e. for most digital recordings 16bit should be more than enough.
- jbay808 7y agoSince audio is usually sampled around 44 kHz, which is slow by signal processing standards, most ADCs can exceed 16 bits by supersampling and averaging the results.
- mcemilg 7y agoSame.
- jonathankchang 7y agoHey mcemilg, You asked about NREC earlier but I failed to set up email alerts correctly (and that post is locked now) so I'm replying here if that's ok. Hiring process takes around a month or two, depending on how schedules line up. There would be a basic phone/skype screen, then a in-depth technical screen on your areas of expertise, and finally an on-site interview. We try to get back to you on each stage within a week of each stage. We have a bunch of employees that are on H1-B's right now. Let me know if you have any other questions and sorry for the late reply!
- MrTrvp 7y agoWorks in chrome
- solarkraft 7y agoThe one official web runtime environment.
- gpvos 7y ago:angry face:
- a_imho 7y agoSame FF on Ubuntu.
- kozak 7y agoA perfect illustration for how Chromium's monopoly is killing the open web. The sample audio doesn't play in Firefox, because who cares about anything other than Chrome, right?
- londons_explore 7y agoTo be fair, this looks like a firefox bug rather than google inventing their own 'standards'. It's a bog-standard HTML5 audio tag with a WAV file. One file is 8khz and the other 24khz, both standard frequencies. One is Microsoft PCM format, while the other is IEEE float. I suspect the latter is the issue - even though it's been around for decades, I bet it isn't a well tested codepath in firefox. It makes sense they use floats for machine learning outputs. Unless someone specifically thought to quantize the data to a specific bit depth, whatever wav file writing library google used probably thought it was being helpful by using the 'right' encoding.
- kozak 7y agoThe problem is that they only tested it on Chrome.
- gattilorenz 7y agoMaybe. On the other hand, the audio tag is pretty standard and the format is standard too, and if you look at w3c support page [1] it looks like ffox supports wav files (and it does, just not all of them. And they might just have tested if the first file plays (or even the second). I would say it's a firefox "bug" instead, there's not a good reason not to support this wav format. And I say that as a Firefox user that never betrayed the fox for that shiny metallic look. [1] https://www.w3schools.com/html/html5_audio.asp https://www.w3schools.com/html/html5_audio.asp
- londons_explore 7y agoEspecially since firefox probably ought to be using the underlying OS libraries for media decoding (and on windows, mac, and linux all the major libraries support this format). Firefoxes "we need to invent it ourself so we aren't at the whim of the platform" has bitten them here. Also, a patch to support this format is probably only 10 lines of code... Simply a for loop over every sample converting to 16 bit PCM.
- londons_explore 7y agoThere seem to be lots of questionable translations in their source data - missing emphasis, wrong words, and sometimes stuttering or mistaken vocalisations. If they could fix that, I think results could be much better.
- akie 7y agoI'm just going to say that this is flat out amazing. Really super impressive.
- ttctciyf 7y agoAm I reading this correctly, that there's no explicit semantic representation going on at any stage, it's purely audio input frequencies -> ML -> audio output frequencies? If so, that's ... so much for Jerry Fodor and his Language of Thought Hypothesis, eh? (Yeah, mostly joking but still..)
- makomk 7y agoIf I'm understanding this right, the actual translation goes from audio frequencies -> ML -> audio frequencies, but the training process for the ML algorithm relies on text transcripts of the speech.
- Nr7 7y agoCool stuff. I wonder how long it will take until speech synthesis is at the point where it can be used to create dialogue for video games. Obviously it would (at least initially) replace just the less important stuff spoken by background characters or less important NPCs, but even that could be a big improvement. Just imagine how much more immersive a game like Skyrim would be if the writers could just write the lines and then run it through a speech synthesizer to get finished dialogue. No need to hire multiple actors and get them to a studio to record their lines. It would be so much easier and faster to create a massive amount of unique dialogue and you wouldn't have to listen to the same "arrow to the knee" line spoken by the same couple of actors over and over again everywhere you go. It would improve user created mods as well since just about anyone could then just create new characters with completely custom voices and dialogue.
- Mirioron 7y agoScrew Skyrim. Imagine what indie developers could do! It would be particularly amazing if this voice could be generated on the fly. But this is still a ways away.
- sansnomme 7y agoImagine dwarf fortress with realistic speech...
- indy 7y agoWe could see a resurgence in text adventures, except this time the input would be voice based and the output would be to your headphones. Downside to this would be people on public transport saying stuff like: "Use magic spell on trolls"
- logicchains 7y agoSounds like a pretty big upside to me.
- deleted 7y ago[deleted]
- lostmsu 7y agoIs the model available yet?