17 ms·
Online speech recognition with wav2letter anywhere
- dvduval 7y agoSo by open sourced I assume this means there are absolutely no Facebook dependencies where the voice is passing through a Facebook server? Sorry, have to ask, as my trust level is low. Otherwise, awesome!
- notduncansmith 7y agoThe repository (https://github.com/facebookresearch/wav2letter https://github.com/facebookresearch/wav2letter) claims to come with pre-trained models for automated speech recognition.
- snissn 7y agothat's cool! I wonder how it works against podcasts
- throwawayhhakdl 7y ago> Trained models: ... I think I remember a similar thing happening with previous wav2letter releases. I would love for a simple tutorial on just using a pretrained model but that feels unlikely to ever happen
- snippyhollow 7y agoThe models are here: https://github.com/facebookresearch/wav2letter/tree/master/recipes/models/sota/2019 https://github.com/facebookresearch/wav2letter/tree/master/r... https://github.com/facebookresearch/wav2letter/tree/master/recipes/models/streaming_convnets/librispeech https://github.com/facebookresearch/wav2letter/tree/master/r...
- vineelkpratap 7y agoYes, nothing goes through Facebook servers. The model will be run locally on the machine.
- MiroF 7y agoNot only does it mean that, it means that you can look at the source code yourself rather than asking the question on HN
- ColanR 7y agoSo what's the efficiency of this model? Can I use it instead of pocketsphinx on a raspberry pi?
- sp332 7y agoAccording to https://research.fb.com/wp-content/uploads/2020/01/Scaling-up-online-speech-recognition-using-ConvNets.pdf https://research.fb.com/wp-content/uploads/2020/01/Scaling-u... the benchmarks were run on "Intel Skylake CPUs with 18 physical cores and 64GB of RAM."
- yorwba 7y agoThe real-time factor for a single audio stream looks like 0.1 (based on eyeballing the graph), so it should be possible to achieve acceptable speeds even with a slower CPU (maybe not a Pi). The memory requirements for the intermediate results are likely to be substantial, though. They say they have "carefully optimized memory use", but don't give any figures.
- sp332 7y agoThey benchmarked several configurations, but I couldn't match up which configuration of models produced which results. I was trying to figure out if you drop the CPU power so throughput just handles one person talking at a normal pace, whether latency would necessarily get crazy high.
- fermenflo 7y agoSo no, lol
- sp332 7y agoThey say they're coming out with Android and iOS versions soon, so maybe take a look after that point to see how they've tweaked the models and if the error rates are a lot higher.
- gliese1337 7y agoOnline speech recognition for English. The framework should be generalizable, but the models they are making available are only for English. Actually adapting this for any other language would be a huge amount of additional work.
- moron4hire 7y agoThis is one of the reasons why we're using Azure Cognitive Services. https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/language-support https://docs.microsoft.com/en-us/azure/cognitive-services/sp... That, and (at least for English) the results are the most accurate I've ever seen.
- londons_explore 7y ago> would be a huge amount of additional work. A small amount of additional work, and a huge amount of money to pay for dataset collection and compute for training.
- gliese1337 7y agoThe dataset collection is going to be work for someone.
- Jnr 7y agoHow does this compare to Mozilla's DeepSpeech? And does anyone know when Mozilla will release the updated Common Voice dataset from https://voice.mozilla.org https://voice.mozilla.org ?
- zachruss92 7y agoI've been following DeepSpeech for a while. They have a WER IN THE 7% range and wave2letters SOTA model is at around 5%. I haven't used wav2letter, but I can run DeepSpeech on my (low powered) laptop with faster than real-time transcription with just the CPU.
- Nimitz14 7y agoIt is misleading to use librispeech WER as a general guide to real world WER. Don't do that.
- londons_explore 7y agoWord error rate depends heavily on dataset. All modern models get to ~human level when tested on individual phrases or sentences. None yet get to human level when the source is many paragraphs long, because the human benefits from context and 'getting used to the accent', which ML has so far not achieved.
- dodobirdlord 7y agowav2letter outperforms, in large part because it seems to make better use of more training data. Facebook’s original paper shows that wav2letter outperforms when trained with their very large internal dataset, and they included a reproduction on a smaller open source dataset with worse overall accuracy but again wav2letter outperforming.
- faitswulff 7y agoSpeaking as a Facebook user, I'm a bit confused - where do they use speech recognition? Or is this just purely research oriented?
- valvar 7y agoSubtitles for videos is my guess.
- melling 7y agoVoice user interfaces are becoming more common. Ignoring this technology is a bad idea. FB has VR devices and their portal device: https://www.theverge.com/2019/9/18/20870866/facebook-portal-mini-new-price-release-date-whatsapp-support https://www.theverge.com/2019/9/18/20870866/facebook-portal-... Sure, the average HN reader will tell you that they don't see the point, etc. But Amazon and Google have sold hundreds of millions of those little voice devices.
- yalok 7y agobetter ads targeting based on user-generated audio/video?
- jauer 7y agoPortal: https://portal.facebook.com/ https://portal.facebook.com/
- yayitswei 7y agoThey could use microphone audio for ad targeting. There's ongoing speculation around whether they're currently doing this, but it could be an ongoing area of research. I'd imagine efficient on-device transcription would help in this regard. https://newatlas.com/computers/facebook-not-secretly-listening-conversations/ https://newatlas.com/computers/facebook-not-secretly-listeni... https://www.wired.com/story/facebooks-listening-smartphone-microphone/ https://www.wired.com/story/facebooks-listening-smartphone-m...
- CosmicShadow 7y agoDidn't they "get caught" for having their apps listening all the time and everyone was wondering how they kept getting ads for things they only ever talked about?
- jwineinger 7y agoI'd love a tutorial that shows a normal guy like me how to use this tool with the pre-trianed models to transcribe my audio files. Not finding anything of that kind included there.
- vineelkpratap 7y agoCheck out the tutorial here - https://github.com/facebookresearch/wav2letter/wiki/Inference-Run-Examples https://github.com/facebookresearch/wav2letter/wiki/Inferenc...
- deleted 7y ago[deleted]
- isoos 7y agoI'd be really interested in the accuracy of this tool to solve Google audio captchas. I'm assuming the price of solving captchas will go further down.
- londons_explore 7y agoI wish recaptcha would let me disable audio captchas - I'm pretty sure all the spammers solve them that way. The volume of real users that want to sign up to my site, and are blind, and don't have a Google account, and clear their cookies frequently enough to get an extra recaptcha challenge, and can't just call support to make an account for them, is probably zero. Yet the number of spammers that come in that way must be in the millions by now.
- cookie_monsta 7y agoI'm not a spammer but I always use the audio option. Typing one word is way less onerous than clicking through multiple screens hunting for level crossings, fire hydrants, etc
- Semaphor 7y agoI’m thankful they don’t. Recaptcha already makes me usually close the tab instead, without audio captcha it would make me close it 100% of the time. I’m not spending 20 minutes hunting for dumb images.
- phkahler 7y agoWhich OSS license?
- trakout 7y agoBSD
- gok 7y agoThe preprint: https://research.fb.com/wp-content/uploads/2020/01/Scaling-up-online-speech-recognition-using-ConvNets.pdf https://research.fb.com/wp-content/uploads/2020/01/Scaling-u... Interestingly, the baselines are all systems that model grapheme forms instead of acoustic (phonemes) directly.
- yellow_lead 7y agoThe name must be a nod to Word2Vec[1]. A cool naming scheme IMO. [1] https://en.m.wikipedia.org/wiki/Word2vec https://en.m.wikipedia.org/wiki/Word2vec
- CrazyStat 7y ago___2___ has been a common naming scheme for conversion utilities for a long time.
- nmfisher 7y agoFacebook Research actually have another toolkit called wav2vec that's based on the same principle as word2vec (self-supervised discriminative pretraining). word2vec learns good feature representations for words by looking at a context window and adjusting weights for the ngram that fits within that window, simultaneously doing so in the opposite direction for randomly sampled ngrams. wav2vec does something similar, learning good feature representations of audio by distinguishing a "real" sample frame from a randomly sampled frame. I can't remember whether wav2vec operates on the raw waveform, the Fourier transform or MFCCs, but the underlying principle is the same. These learned feature vectors have been shown to dramatically improve downstream tasks.
- rexreed 7y agoGiven that this uses a beam search decoder to find the most likely word pattern, is it possible small perturbations in audio could cause it to improperly decode certain word strings? Sort of like the audio equivalent of adversarial attacks, but on ASR?
- cproctor 7y agoI'm about to start as a professor in CS education, and am hoping we're getting close to the point where I can easily transcribe interviews and high-quality dialogue audio using open-sourced models running on machines in my lab. I'm tired of paying $1/minute for human transcription that's not great anyway, and would love to undertake research that would require processing a lot more audio than is affordable on those terms. I haven't kept up with developments over the last two years--anyone have a sense of whether this is close to being a reality? (I've taken a bunch of Stanford's graduate AI courses on NLP and speech recognition; I can read documentation and deploy/configure models but don't have much appetite for getting into the weeds.)
- kick 7y agoEarlier this year the Media Lab did an absolutely ginormous automated transcription project. Off the top of my head, it was ~2.8 billion words. 13.1% error rate (vs. ~7% error rate for Google's proprietary solution). Found the paper on it: https://arxiv.org/pdf/1907.07073.pdf https://arxiv.org/pdf/1907.07073.pdf
- woodson 7y agoSadly they don’t (well, can’t) release the audio+transcripts as dataset, as they clearly don’t own the rights.
- kick 7y agoThey did release it as a dataset. I have a copy of it. It's massive. I'd recommend reading the paper, it has a link to a place where you can download it, and aside from that, it's also fascinating.
- woodson 7y agoIncluding the audio? I downloaded the transcripts from their bucket, but couldn’t find any information on how to obtain the corresponding recordings. At Interspeech 2019, the author basically told me that they couldn’t share it.
- starpilot 7y agoDo the pretrained models work decently on landline phone quality recordings? I can see massive value for this if it can transcribe corporate call center audio.
- woodson 7y agoThey can’t, because they are trained on more-or-less high-quality recordings of people reading books out loud. Phone conversations are very different, not just in audio quality but in the way people speak.
- amluto 7y agoI wonder if this would be a good engine to plug in to rhasspy.
- tingletech 7y agoAll I see is """Sorry, this content isn't available right now The link you followed may have expired, or the page may only be visible to an audience you're not in. Go back to the previous page · Go to News Feed · Visit our Help Center""" Edit: found a link that works https://github.com/facebookresearch/wav2letter https://github.com/facebookresearch/wav2letter
- deleted 7y ago[deleted]
- vineelkpratap 7y agoHere is the direct link - https://ai.facebook.com/blog/online-speech-recognition-with-wav2letteranywhere https://ai.facebook.com/blog/online-speech-recognition-with-...
- rgj 7y agoSame error (from NL)
- vineelkpratap 7y agoFixed now !
- jonathanleroux 7y agoIf I may insert a relevant plug: we (MERL) just put out a paper last week with SOTA 7.0 % WER on LibriSpeech test-other (vs wav2letter@anywhere's 7.5%) with 590 ms theoretical latency using joint CTC-Transformer with parallel time-delayed LSTM and triggered attention. Check it out: https://arxiv.org/abs/2001.02674 https://arxiv.org/abs/2001.02674
- jonathanleroux 7y agoCorrection: no PTDLSTM here but time-restricted self-attention (duh). PTDLSTM was our previous encoder setup published at ASRU.
- z3t4 7y agoFor any project like this, please post exactly the sound configuration used for the model. eg. the rate (Hz), channels, and format.