9 ms·
Show HN: Ermine.ai – Record and transcribe speech, 100% client-side (WASM)
- nunobrito 4y agoSo nice, it worked. Is there a version of this we can run on ESP32 (arduino) devices?
- vishnumenon 4y agoThanks! Not sure if it'll work on an Arduino, but maybe take a look at https://github.com/ggerganov/whisper.cpp https://github.com/ggerganov/whisper.cpp -- it works on a Raspberry Pi at least, so resource requirements are fairly minimal
- nunobrito 4y agoKeep up the good work!
- maven29 4y agoYou're only going to be able to do wake-words and command recognition locally on a microcontroller. The maximum external SPI RAM on an ESP32 is 4MB. https://github.com/espressif/esp-skainet https://github.com/espressif/esp-skainet https://github.com/tomzbj/dhrystone_score https://github.com/tomzbj/dhrystone_score
- voicedYoda 4y agoNice! Also enjoy the ffmpeg usage. Gonna give it a try!
- apineda 4y agoIs there a github link? I tried clicking the logo but it didn't work.
- vishnumenon 4y agoSorry, it looks like the modal background overlaps the logo so it only works once you've loaded the model -- here's a direct link: https://github.com/vishnumenon/ermine-ai https://github.com/vishnumenon/ermine-ai
- maxloh 4y agoWill it be released in an open source license?
- vishnumenon 4y agoJust added an MIT license!
- wdb 4y agoDoesn't seem to work in Safari
- vishnumenon 4y agoThanks for the feedback! I tested in Chrome and Firefox but unfortunately I run Fedora so I haven't been able to test in Safari. I'll look into it!
- supermatt 4y agoDoesnt work in firefox for me (latest version on windows).
- wazoox 4y agoDoesn't work on Firefox Linux for me either ( error is Uncaught (in promise) DOMException: AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported. index-0dae94e71b526640.js:1 )
- vishnumenon 4y agoThanks, appreciate the error message, will look into it ASAP!
- mbushey 4y agoSafari was forked from Konqueror. While a lot has changed since then it might be worth it to try Konqueror.
- evulhotdog 4y agoWorked on safari on iOS for me. Normally I expect a lot of things that push the limit to not work on iOS, but this one did!
- calculated 4y agoWorked for me on Safari
- 1attice 4y agodoes not work in Firefox, but works draw-droppingly well in Chrome
- psychoslave 4y agoSame observation on Firefox 112.0b2 (64-bit) here.
- noduerme 4y agoI hope you didn't crack your draw when you were speaking your comment....
- rado 4y agoWorks great, thanks
- NiekvdMaas 4y agoNice! Are you aware that whisper.cpp has a WASM-version as well? See https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.wasm https://github.com/ggerganov/whisper.cpp/tree/master/example... - demo at https://whisper.ggerganov.com/ https://whisper.ggerganov.com/
- vishnumenon 4y agoYup! Love the whisper.cpp (and llama.cpp) projects, they're incredible -- I mostly ended up using transformers.js for this just because the JS interface is a bit more straightforward/well-documented and its the same underlying model
- quickthrower2 4y agoRelated : https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_...
- leodriesch 4y agoBe wary that some implementations of the API use server-side transcription and Firefox does not support it at all.
- quickthrower2 4y agoEew! That definitely goes against my expectations of a user agent.
- infruset 4y agoVery nice. Would it be easy to add other languages than English? Also, as others have notes, I had to open it in Chrome to make it work, Firefox didn't work.
- NayamAmarshe 4y ago> Would it be easy to add other languages than English? I think so, yes. The website uses whisper-tiny.en, so I suppose all they need to do is switch to whisper-tiny instead.
- vishnumenon 4y agoIn theory, yes -- but tbh in practice, I tried using the multilingual version of whisper-tiny and saw significantly worse performance, so that's why it's english-only at the moment. Possible that there's some sort of parameter/tuning that could improve performance though, I'm looking in to it!
- viraptor 4y agoI'm after something that can transcribe medical notes and unfortunately it does not work well for that case. (almost nothing does though) There's quite a few people interested in something that doesn't turn "laparoscopic" into "leper as cop it". Maybe the current progress will help though. Models adjusted by your own dictionary or from postprocessing fixes would be amazing.
- dilek 4y agopicovoice processes on the device and you can fine-tune the models https://picovoice.ai/platform/cat/ https://picovoice.ai/platform/cat/
- johtso 4y agoHave you tried https://speechmatics.com/ https://speechmatics.com/ ? I think they have a specially tuned medical version, and quite a generous free allowance.
- viraptor 4y agoI have not, not will do tomorrow. Thanks for the link.
- giovannibonetti 4y agoYou'll probably need a custom model tailored to medical content.
- anonymouse008 4y agoHave you tried Siri Dictation? The transcription process doesn’t leave iOS 14 and on - and I’ve been pleasantly surprised. Laparoscopic - that worked fine ;) Edit: if you have one available, mind sending over a sample deidentified note to my email in profile? I’m working on something.
- viraptor 4y ago> Have you tried Siri Dictation? No, there's no Apple hardware available in my scenario. Also in medical context, if I can't tell where the data goes, the solution is not usable.
- APock 4y agoI want this but let me upload a file first.
- technocratius 4y agoLove this idea! Tried a 10 sec clip on Firefox for Android. App seems to be stuck on Transcribing... for few mins now...
- joshgev 4y agoSeems to fail on Desktop Firefox (I'm on Linux) as well.
- bartislartfast 4y agocan we add a "new" button? I have to refresh to do it a second time
- vishnumenon 4y agoDefinitely, I'll add one ASAP! Thanks for the suggestion
- senko 4y agoWhen I try this on Firefox on Linux (not incognito) I get the following error: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported. index-0dae94e71b526640.js:1:2992 Uncaught (in promise) DOMException: AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported. index-0dae94e71b526640.js:1 Media resource blob:https://www.ermine.ai/e762a6f1-f292-4b23-96e0-8059a7f9d635 could not be decoded. www.ermine.ai Media resource blob:https://www.ermine.ai/e762a6f1-f292-4b23-96e0-8059a7f9d635 could not be decoded, error: Error Code: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) (also, the weights json doesn't download at all in Firefox incognito). Would be good if you could pop some kind of alert (literally alert() might do the trick) on an exception just so people don't wait for a couple of minutes before realizing something's gone wrong :)
- denysonique 4y agoI also get this error. Unfortunately nowadays most developers forget about Firefox, this includes big sites such as Microsoft Azure Console not working under Firefox and Google Cloud Console being slow on Firefox.
- vishnumenon 4y agoThanks for the detailed error report! I was under the impression that it worked in FF, but I guess there's some additional testing needed -- I'll definitely look into it!
- emadda 4y agoI released a similar web WASM transcription tool recently: https://bigwav.app https://bigwav.app
- GTP 4y agoWhich languages does your tool support?
- ibnbutlAn 4y agoSo this is all client-side and my speech is not sent antwhere or is ut "client-side ui" for some API? I will have a look at the repository to find out, but maybe someone already looked into it.
- Garcia98 4y agoIt's all client side, it downloads a model and performs the inference using ONNX runtime (which uses WASM).
- bethecloud 4y agoHow are you currently distributing the audio download? Any interest in using a distributed CDN layer – happy to get help get it funded
- vishnumenon 4y agoThere's actually no server-side component, the audio file is generated entirely client-side in JS!
- vrglvrglvrgl 4y ago[dead]
- dmix 4y agoCan it identify speakers? For examples SPEAKER A: blah blah SPEAKER B: blah blah So it can be used for transcribing phone calls?
- vishnumenon 4y agoUnfortunately not at the moment, but that feature (speaker diarization) is something I'm looking in to! AFAIK the Whisper model (which this uses currently) doesn't support that functionality, but I'm experimenting with adding an auxiliary model just for diarization.
- MuffinFlavored 4y agoWow, it worked on an iPhone despite it saying it wouldn't work on Safari (needed Chrome). The little "replay your audio capture" <audio> HTML element says "error" but the transcription actually worked.
- bloominggarden 4y agoThis is pretty cool! Just yesterday I finished a similar demo, also using transformers.js. I am currently in the process of adding real-time transcription, do you plan on adding that?
- vishnumenon 4y agoThanks! I actually initially had it doing transcription in 5- and 10-second chunks for close-to-realtime results, but the CPU usage on my laptop (which admittedly doesn't have the best specs) was a bit higher than i wanted. 30-second blocks gave me the best balance of semi-real-time and good performance, especially since the whisper model is built for 30-second chunks. If you get real-time working smoothly though, i'd love to take a look!
- RecycledEle 4y agoI need a Windows executable that takes a directory of audio files, and transcribes them to similarly named text files, so they can be searched. Example: 20230406115923.mp3 ==> 20230406115923.txt 20230406083110.m4a ==> 20230406083110.txt I wish someone would build one and sell it for $10 a copy.