13 ms·
Show HN: LibreASR – An On-Premises, Streaming Speech Recognition System
- pw6hv 6y agoIs the transcription of Macron's speech completely off or am I not understanding what the two shown texts represent?
- jiehong 6y agoSeems off indeed. Plus, the readme says French is not supported yet. Not the best demo IMO.
- iceychris 6y agoI have not yet trained a french model. Also, the gif shows Macron speaking to the congress with his english accent [0] [0] https://www.youtube.com/watch?v=RqUc1h7bZQ4 https://www.youtube.com/watch?v=RqUc1h7bZQ4
- iceychris 6y agoThe upper transcript is YouTube's automatic transcription. Below is the web app transcribing live. And yes, it is actually missing a few words.
- saurik 6y ago(Anyone know how the transcription quality compares to the various cloud offerings from AWS, Google, and IBM?)
- iceychris 6y agoAs I commented above, very poorly. It's still early days.
- iceychris 6y agoHey HN! I've been working on this for a while now. While there are other on-premise solutions using older models such as DeepSpeech [0], I haven't found a deployable project supporting multiple languages using the recent RNN-T Architecture [1]. Please note that this does not achieve SotA performance. Also, I've only trained it on one GPU so there might be room for improvement. Edit: Don't expect good performance :D this is still in early stage development. I am looking for contributers :) [0] https://github.com/mozilla/DeepSpeech https://github.com/mozilla/DeepSpeech [1] https://arxiv.org/abs/1811.06621 https://arxiv.org/abs/1811.06621
- the_biot 6y agoWhy on earth would you put a demo front and center that shows your software doing a terrible, terrible job?
- brmgb 6y agoThat's called setting up expectations. If you know your project might interest people but needs work, why pretends it's good when it's not? They seem to be courting contributors more than users anyway. I found the video to be funny. It nicely highlights both the current limitations and the ambition of the projects. Bold choice certainly but I think it works.
- dcsan 6y agoand it's maybe a dig at Macron's accent at the same time :D although the author is a student in germany. Anyway you should join the Discord, we discussed this there too... https://discord.gg/pqTMeP5D3g https://discord.gg/pqTMeP5D3g
- th3h4mm3r 6y agoHi! What should you need to implement other language i.e. Italian or French? I mean: it's a problem due to the less of datas or what? Another question: could you use for example mozilla voice data to train/test?
- iceychris 6y agoData and compute are the largest hurdles. I only have one GPU and training one model takes 3+ days, so I am limited by that. Also, scraping from YouTube takes time and a lot of storage (multiple TBs). Mozilla Common Voice data is already used for training.
- jack_pp 6y agoWhy does it take a lot of data? Afaik you can select lower quality in youtube-dl but you don't even need video do you?
- jiehong 6y agoWhat datasets are used to train the models?
- iceychris 6y agoLibriSpeech, Tatoeba, Common Voice and scraped YouTube videos.
- blackcat201 6y agoDo you get good results when adding scraped youtube audio? My model performance on LibriSpeech dev drops a bit when adding youtube audio to the training dataset ( my guess is likely due to poor alignment from auto generated captions ).
- iceychris 6y agoI haven't trained on LibriSpeech exclusively, but yes, the perf on LibriSpeech dev is quite bad, around ~60.0 WER. If the poor alignment of yt captions is the issue, maybe concatenating multiple samples helps a bit.
- lunixbochs 6y agoYou should consider realignment; maybe start with something like DSAlign or my wav2train project.
- dcsan 6y agowould it be possible to train on any of the more recent Text to speech engines out there? some of them are very realistic. this would give you absolutely perfect sync down to the word, I assume... I don't know about the cost if you paid ratecard though, perhaps you can do some partnership with them since yours is a symetrical product
- ahoka 6y agoIn case someone wondered what does the title have to do with logic, it’s just probably a (common) malapropism. premise (noun) a previous statement or proposition from which another is inferred or follows as a conclusion. premises (noun) a house or building, together with its land and outbuildings, occupied by a business or considered in an official context.
- iceychris 6y agoRight, fixed it, thank you :D
- Proven 6y agoIt's not "mal" anything, many simply prefer to use "on-premise" or on-prem for "on-premises". You didn't have any issue understanding the original title.
- yjftsjthsd-h 6y ago> You didn't have any issue understanding the original title. Humans are very good at live error correction, but that doesn't make it not wrong.
- blackcat201 6y agoCool project, seems like your model have similar WER as mine (4th reference in readme). Do you plan to do any pre-training on the encoder part in the future? Maybe something like this[1] [1] https://ai.facebook.com/blog/wav2vec-20-learning-the-structure-of-speech-from-raw-audio/ https://ai.facebook.com/blog/wav2vec-20-learning-the-structu...
- iceychris 6y agoHey blackcat! Your project [0] helped me a lot! Pre-training the encoder sounds great, I'll maybe add it in the future. [0] https://github.com/theblackcat102/Online-Speech-Recognition https://github.com/theblackcat102/Online-Speech-Recognition
- bravura 6y agoHey black cat, I have some work in preprint for a NeuroIPS workshop, demonstrating negative results of different audio distances on pitch tasks. There is one particular w2v result I'd like your feedback on. Do you mind emailing me? Lastname at gmail dot com (see my profile for my name)
- OnlyMortal 6y agoHaving worked on ViaVoice OSX back in the day, we had to have models for different varieties of English. The US model couldn’t understand my northern English (think GoT) accent. It’s why the product came out with a UK localisation. Wondering if you might have a better reco of the French President if you have a model per dialect of English?
- iceychris 6y agoYes, probably. The data I trained on mostly reflects UK and US accents.
- OnlyMortal 6y agoIBM kept their US and UK models apart. May have been historic or dataset size. As a FYI, I was told “the money” was in specific “dictionaries” for medical professionals and so forth. Apparently, doctors liked to dictate straight into text. Might be worth trying that $$$€€€£££?
- guillem_lefait 6y agoNice project and bold to demo with a French native speaking English. On a side project, I'm looking at the best interface to facilitate further edition (correction) of the recognized text. Target is local councils and regional parliament, where sessions are usually recorded but without transcripts. If xx% accuracy is enough to identify keywords, manual edition is still required to not distort precise meaning. Nothing special in the interface, but two features seems interesting: 1. Be able to collaborate in real-time. Maybe using Etherpad API to merge multiple editions. 2. Easily validate text and label speakers so as to generate new training data. Pointers to similar existing solutions would be very appreciated.
- nsomaru 6y agoI’m really interested in this project too. Been thinking about similar solutions for a while now. I looked into Kaldi and Mozilla Deep Speech but the former seems geared at ASR experts and the latter didn’t seem suited for my particular application (longer recorded audio or real time stream)
- whimsicalism 6y agoUse wav2letter
- posguy 6y agoMozilla DeepSpeech has streaming audio support as of a few releases ago, the word error rate has also improved. I would recommend looking at Vosk too, it converts speech to text much faster than Mozilla DeepSpeech while having slightly better results: https://alphacephei.com/vosk/ https://alphacephei.com/vosk/
- fareesh 6y agoIs there an open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a system like this? I looked at Twilio but they seem to only offer a means to do it on their VOIP/SIP product.
- Y_Y 6y agoGoogle Meet does this
- dcsan 6y agohow real-time do you need it? if you use a streaming API you can even use google and there isn't too much lag, and it's continuous. Agora also talk about this, but I haven't used it myself https://www.agora.io/en/ https://www.agora.io/en/
- whimsicalism 6y ago> open-source or paid SDK/API that I can use to create a group voice chat mobile app with "live" transcription? Or something that can plug-in to a system like this? Yes, Google, Amazon, Microsoft all offer streaming solutions (wouldn't recommend Amazon's however, might recommend Microsoft over Google). wav2letter from FB is the only open-source framework worth looking at, deepspeech is not a seriously usable framework.
- woodson 6y agoCheck out Kaldi. It's a toolkit rather than a ready-to-deploy service but has some solid pretrained models and recipes for training your own. You can use various existing projects for deployment, e.g. vosk-server (also for on-device) which comes with models for various languages and accents and has an excellent support channel via telegram. Quite frankly, despite not being "end-to-end", you'll get much much better results in practice.
- whimsicalism 6y agoI collected custom audio and had it transcribed by hand for cash, then evaluated it on wav2letter and vosk. At least for that domain, wav2letter outperforms vosk.
- donpdonp 6y agoThe README could use a section as to what CPU platform and storage requirements are necessary to run this app.
- lunixbochs 6y agoCan you post WER per dataset? Bucketing all of the WER together means you can only directly compare to models that are validated on the exact same combination datasets. This excludes all other ASR systems from comparison, as well as your own models if you decide to add validation data in the future.
- woodson 6y agoThis is somewhat off-topic, but as someone who has worked on various speech processing and ASR projects, I'm curious to learn from people who have specific problems and applications for this technology where it can make a difference. That is to say, what are areas where you think ASR can enable new products or make common and tedious tasks much more efficient?
- crawfordcomeaux 6y agoMy partner and I are raising our child in unique ways with a focus on avoiding certain linguistic patterns. I'm interested in filming/recording our home, automating transcription, and creating a system for voice-driven video editing we can do on the fly to create highlights and capture discussions. I see this as a way to create a dataset they can use in the future to diagnose any trauma caused by our choices.
- 72deluxe 6y agoYou're effectively experimenting on your child? Don't you think the repurcussions might be severe?
- crawfordcomeaux 6y agoEvery parent is, though many will deny it. The repercussions of doing so without intention and without admitting it are already severe. Every parent is likely to traumatize their child in some way, including the trauma of protecting them from trauma to the point that they don't learn to heal through it. I'm ok with intentionally experimenting and normalizing healing within our family. Also, it's already paying off tremendously. When repercussions can be severe, so can rewards. We have a 2-year old who is incredibly emotionally aware, has a huge vocabulary, enjoys eating anything, explores freely, is learning to play multiple instruments, draws with a pencil grip in both hands, can sit to actively listen to music for 20+ minutes at a time, and learns lyrics incredibly fast. If you have specific fears, I'm interested in hearing them because that gives us an opportunity to prepare.
- kkielhofner 6y agoI LOVE that you provided a sample application targeting the ESP32-LyraT! While the ESP8266/ESP32 get plenty of love on HN (and elsewhere) I think the ESP ADF (audio development framework) and various boards dev boards (Lyra, Korvo, etc) are really under appreciated and essentially unknown. I enjoy a Raspberry Pi, Jetson nano, Arduino, whatever as much as the next person but the seemingly endless stream of projects and resulting blog posts, etc featuring them can get a little old. Great work!
- iceychris 6y agoAudio boards based on ESP32 boards are quite under the radar and have lovely features for just a few bucks. Running LibreASR on a RPi should also be feasible soon. Thank you for your kind words! :)
- sagz 6y agoThis is great! would you be able to integrate it with Live Transcribe to make a great FOSS solution for the deaf and hard of hearing? :) https://github.com/google/live-transcribe-speech-engine https://github.com/google/live-transcribe-speech-engine
- nshm 6y agoThe secret of RNN-T is that it is __extremely__ hard to train, it is very unstable. On a single GPU you'll spend years to train reasonable quality model, moreover a streaming one. Streaming training requires teacher-student setup. Thats why there are dozen RNNT projects around, some of them more reasonable, some less, but most of them struggle to demonstrate even a good librispeech WER. You have a long way to go.