10 ms·

Show HN: State-of-the-art German speech recognition in 284 lines of C++

231 points by fxtentacle 4y ago

pkaye 4y ago
Includes a standard C++ library called tensorflow...
- fxtentacle 4y ago
  ... like all the other speech recognition projects that you might want to compare the LOC against.
  - numpad 4y ago
    ...which however don't clickbait by using LoC as a primary selling point.
    - fxtentacle 4y ago
      I wouldn't call it clickbait. Mozilla's DeepSpeech is so large that you can't really read it and understand it. This one is 10x less code while recognition quality is 75% better (lower relative word error rate). So this one is small enough that you can read the source code if you want to, while DeepSpeech is not.
      - numpad 4y ago
        Good point – calling this clickbait might be too cynical. From this perspective I can definitely get behind advertising the project with the LoC measurement. Subjectively, I still find this to be a bit of a "not telling the whole truth", however I also only ever toyed around with speech recognition ai.
  - Evidlo 4y ago
    You could have maybe said "284 lines of code plus weights" to avoid all this criticism that misses the point of the project.
    - fxtentacle 4y ago
      That wouldn't have fit into the 80 character limit. I also wanted to mention that it's private and offline, but couldn't say it compact enough.
- deleted 4y ago
  [deleted]
fxtentacle 4y ago
And for the true "Show HN" experience: wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.001 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.002 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.003 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.004 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.005 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." cat tevr_asr_tool-1.0.0-Linux-x86_64.zip.00* > tevr_asr_tool-1.0.0-Linux-x86_64.zip unzip tevr_asr_tool-1.0.0-Linux-x86_64.zip sudo dpkg -i tevr_asr_tool-1.0.0-Linux-x86_64.deb tevr_asr_tool --target_file=test_audio.wav and then you'll be greeted with some TensorFlow Lite diagnostics, followed by the intermediate states of the beam-search decoder, followed by the hopefully correct transcription result. And if that piques your curiosity, here's a short overview over the code: https://github.com/DeutscheKI/tevr-asr-tool#how-does-this-work https://github.com/DeutscheKI/tevr-asr-tool#how-does-this-wo...
OrangeMonkey 4y ago
I can do it in 1 line of code if it includes downloading your github repo.
- mrweasel 4y ago
  Using operating system provided functionality, a GUI library or anything in libc, isn't something I'd hold against anyone when counting lines of code. Relying on an entire external project however, that's a bit misleading.
  - hardware2win 4y ago
    Where is the boundry then? You cannot "outsource" the "core"?
    - dahfizz 4y ago
      You can do whatever you want. Just don't brag about your line count if all your project does is import dependencies.
      - fxtentacle 4y ago
        It doesn't. It implements a new way of decoding the logits which improves performance by a relative 16% over the previously best German speech recognition, which was Facebook's wav2vec2. And the size is relevant to people in the industry because DeepSpeech uses 2000+ LOCs for implementing their decoding, so this works better and is 10x less code.
        agileAlligator 4y ago
        > It implements a new way of decoding the logits which improves performance by a relative 16% Then you should have mentioned that in the title rather than the LOCs
    - mrweasel 4y ago
      I don't know. It's not that it isn't impressive and it is very illustrative of how much you can build, utilizing the tools that are available to all of us. It's just at what point are you writing code and when are you just initializing and configuring an existing tool. It's also how you present it. If you said: "Building a TensorFlow backed German speech recognition system in only a few hundred lines of code." Then I feel like you're being more honest.
BurningPenguin 4y ago
How well does it do with accent or slight dialect influences?
- deleted 4y ago
  [deleted]
- fxtentacle 4y ago
  We tested it on CommonVoice German, which is what Mozilla used for DeepSpeech German, too. The idea behind that dataset is that arbitrary people on the internet submit their recordings (hence the "common" in the name) and then if enough other people upvote it as "understandable", it gets included in the dataset. As such, the AI works well with a variety of accents.
  - junkerm 4y ago
    I suspect that "works well" means that the model will output words in "official german" and kind of corrects pronounciation errors? I am asking because I had the use case to automatically give feedback to non native german speakers.
    - fxtentacle 4y ago
      There's a command-line parameter "--use_language_model=0" to disable the spellchecker.
- deleted 4y ago
  [deleted]
nottorp 4y ago
And if you add the dependent libraries it's 3 million lines?
- deleted 4y ago
  [deleted]
tjungblut 4y ago
I'm sure you can save another line while checking for mückenstiche in your dictionary.
dahfizz 4y ago
Guys I just wrote a json parser in one line of code! import json
- petercooper 4y ago
  And all algorithms are O(f(n)), where f is someone else's responsibility.
- deleted 4y ago
  [deleted]
captainmuon 4y ago
This looks pretty cool, especially since it is offline and free! The only caveat is that it is probably not feasible to train yourself, right? I should play with this if I have some time. I've had the idea for a while to build a voice assistent which can switch modes or datasets while you are speaking. If you say "computer, play ...." for example it would load a recognizer that is specialized on song names. The idea is that you can mix English song names in a German prompt, and it will not be confused. Every voice assistent I know gets confused, presumably because they convert speech to plain text and only then act on the text.
- fxtentacle 4y ago
  > I've had the idea for a while to build a voice assistent which can switch modes or datasets while you are speaking. If you say "computer, play ...." for example it would load a recognizer that is specialized on song names. I would say by now the generic recognizers are so good that this is becoming less and less useful. For example, this tool handles non-existing German words quite well. That said, the tool has a "--data_folder_path" parameter where you can specify a different acoustic and language model. BTW, I also want to build an offline voice assistant :) That's how I got started on this journey. You might be interested in my next project, where I try to do offline real-time English recognition with a WebRTC API to make it easy for developers to connect my AI module with their own task logic. Here's the waiting list: https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5/join https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5...
postalrat 4y ago
And 27 includes plus a huge model. "284 lines of C++" is something you could fit on a small microcontroller. This isn't 284 lines of C++.
- fxtentacle 4y ago
  In my opinion, the focus here should be on the fact that this is a state of the art AI which beats Facebook's wav2vec2 by a relative 16% improvement, Scribosermo (based on NVIDIA NeMo) by a relative 44% and Mozilla's DeepSpeech German by a relative 75% improvement. People usually don't share their production-quality tools ;) That said, I wrote "284 lines of C++" to indicate that this is compact enough for people to actually read and understand the source code. Also, compiling my implementation is super easy and straightforward ... something which can't be said for Kaldi, Vosk, or DeepSpeech. If you try to read the CTC beam search decoder from Mozilla's DeepSpeech [1], that alone is about 2000 LOC in multiple files. If you try to read the pyctcdecode source that is used by HuggingFace [2], that's 1000+ LOC of Python. But this implementation is all the client-side, i.e. the entire "native_client" folder hierarchy in DeepSpeech [3], narrowed down to a mere 284 lines. Also, both DeepSpeech and HuggingFace Transformers use TensorFlow as a dependency, i.e. just like me. So in my opinion, it doesn't make sense to include TF in the LOC comparison if all the AI speech recognition systems use it. That would be like including libstdc++, too. [1] https://github.com/mozilla/DeepSpeech/tree/master/native_client/ctcdecode https://github.com/mozilla/DeepSpeech/tree/master/native_cli... [2] https://github.com/kensho-technologies/pyctcdecode https://github.com/kensho-technologies/pyctcdecode [3] https://github.com/mozilla/DeepSpeech/tree/master/native_client https://github.com/mozilla/DeepSpeech/tree/master/native_cli...
  - kylebgorman 4y ago
    A speech recognizer is not an "AI".
    - fxtentacle 4y ago
      Well, the acoustic model is created using deep learning loss-minimization by gradient descent, which is what people usually call AI these days.
vintermann 4y ago
284 lines of C++, and a 1.53 GB wav2vec2 model. But still very nice!
- fxtentacle 4y ago
  Actually the model is TEVR, which is only the wav2vec2 feature extraction but with a modified encoder that improves performance by exploiting redundancies in the German language.
- deleted 4y ago
  [deleted]
cgeier 4y ago
I was quite interested in this, but after seeing that it includes three big other repositories, my interest was immediately gone. I'm not saying this isn't an interesting project, but perhaps lead with something different?
- eternalban 4y ago
  How to assemble a German speech recognition program in under 300 lines. - ps: I skimmed the paper cited and what I wrote above is -not- correct. The project is not simply assembling a pipeline, it is claiming ('papers with code') an innovation: "This paper presents TEVR, a speech recognition model designed to minimize the variation in token entropy w.r.t. to the language model. This takes advantage of the fact that if the language model will reliably and accurately predict a token anyway, then the acoustic model doesn't need to be accurate in recognizing it." ... "We have shown that when combined with an appropriately tuned language model, the TEVR-enhanced model outperforms the best German automated speech recognition result from literature by a relative 44.85% reduction in word error rate. It also outperforms the best self-reported community model by a relative 16.89% reduction in word error rate."
  - fxtentacle 4y ago
    Yes, we actually did research and spent a significant amount of GPU time. Thanks to my luck of stumbling into the right people at the right time, I could afford cloud-scale training because OVH granted me very generous rebates... The main innovation is that we prevent loss mis-allocation during training of the acoustic AI model by pre-weighting things with the loss of the language model. Or in short: We don't train what you don't need to hear If you want to play around with the TEVR tokenizer design, here's the source for that: https://huggingface.co/fxtentacle/tevr-token-entropy-predictor-de https://huggingface.co/fxtentacle/tevr-token-entropy-predict...
    - cgeier 4y ago
      > The main innovation is that we prevent loss mis-allocation during training of the acoustic AI model by pre-weighting things with the loss of the language model. Or in short: > We don't train what you don't need to hear This does sound a lot more interesting than the ~280 lines of code.
kevin_b_er 4y ago
So, what caught my interest was this quote at the bottom: "Alternatively, you can donate roughly 2 weeks of A100 GPU credits to me and I'll train a suitable recognition model and upload it to HuggingFace." This takes 2 weeks of A100 GPU time to train? Current GC spot pricing puts that at about $300.
- fxtentacle 4y ago
  "This takes 2 weeks of A100 GPU time to train?" Yes, but only for me in this specific case. That's because this German model was derived based on a model which was pre-trained for months on hundreds of thousands of hours of YouTube audio in various languages. So if I now train an English recognizer, I don't start from scratch. Instead, I start from a checkpoint which can already almost flawlessly recognize any human utterance. (Or all phonemes in the IPA alphabet, to be more precise) The "training" then only learns the mapping between IPA phonemes and text notation.
  - deleted 4y ago
    [deleted]
  - mciancia 4y ago
    What other languages can you "easily" do like that?
    - fxtentacle 4y ago
      I believe for English, German, Dutch, Spanish, French, Italian, Portuguese, Polish there's enough data in CommonVoice to get good results.
      - gogusrl 4y ago
        Any chance for Romanian ?
        fxtentacle 4y ago
henrydark 4y ago
There are so many comments about the "real" length of the program. The number 284 means something to people who work in speech recognition - these are the people that know how much _they_ write when they try to compete with this library. This number isn't meant for people who are disinterested (have no stake) in speech recognition.
- deleted 4y ago
  [deleted]
- prewett 4y ago
  "X in N lines of code" is usually used for compressing an algorithm down to its bare essentials. "Ray tracing in 100 lines of code". In that case, it's disingenuous to have 100 lines of glue code that calls into pov-ray. If I compile 284 lines of code, I expect a few kb executable (including libraries that aren't language-libraries like libc). This hasn't really simplified the speech-recognition algorithm, it still requires large amounts of training data and time, and still does the speech recognition by solving a large matrix with Tensorflow. It's not interesting to me that you can hook up Tensorflow in a few lines of code; I know Tensorflow can solve the problem. However, I would be interested in 284 lines of code that replace Tensorflow, which is what the title suggested to me. A better title would be to focus on how the model and/or the handling of the data produces better results, because that's what this code seems to be about.
  - fxtentacle 4y ago
    If you compare this with Mozilla's DeepSpeech repository, you will find that correctly calling TensorFlow Lite and handling the results in only 284 lines of code is impressive, too. Also, the C++ code is mostly a custom beam search decoder based on the research for my paper, so it's not like TensorFlow is doing all the heavy lifting here, because precisely that TEVR token decoder causes the relative 16% performance improvement of this speech recognition AI over others.
lynndotpy 4y ago
To those criticizing the title, how would you improve it? Dependencies do the heavy lifting , but that's true even for a hello-world. It seems obvious enough that I wouldn't see a reason to clarify. Instead, to me, this reads, "Hey C++ fans, you don't need Python for nice things. Look at what you can do in 284 lines of code!" I don't have C++ chops, so this is nontrivial for me. I appreciate OP sharing this!
- itcrowd 4y ago
  Show HN: State-of-the-Art German Speech Recognition in C++
  - lynndotpy 4y ago
    That makes sense, but knowing it's 284 LOC means I can casually read it and learn something from that. That would be different at 2840 LOC or very very different at 284000 LOC.
    - fxtentacle 4y ago
      Yes, this is precisely why I included it. I wanted to highlight for people who have experience with speech recognition in general that this is a magnitude easier to read than Mozilla's DeepSpeech.
- UncleEntity 4y ago
  > To those criticizing the title, how would you improve it? By not making false assertions? Judging by the comments the 284 lines is in addition to many hours of GPU training time plus some magic and a huge library. I didn’t even click the link because I knew it was 284 lines of plumbing code on top of something else. I once wrote a entire renderman compatible renderer in a couple hundred lines of python — which was really a super dumb script which generated ctypes bindings from the header for an actual renderer (pixie if anyone is curious).
  - fxtentacle 4y ago
fwsgonzo 4y ago
Is it possible to get speech recognition into a statically linked executable? ldd myprogram should say "not a dynamic executable" Are there any libraries for those of us in embedded?
- fxtentacle 4y ago
  My code does precisely that, the only dependencies of the finished executable are Debian OS libraries: $ lddtree build/tevr_asr_tool tevr_asr_tool => build/tevr_asr_tool (interpreter => /lib64/ld-linux-x86-64.so.2) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
  - fwsgonzo 4y ago
    I guess I will give it a try then.
ale42 4y ago
284 lines of C++, and a big external dependency (tensorflow)... plus the model (but this was expected).
- deleted 4y ago
  [deleted]
gok 4y ago
The "TEVR" token encoding idea is interesting. Any chance you compared it with other subword tokenization techniques like BPE?
- fxtentacle 4y ago
  Sorry, not really. Training this pipeline was already quite expensive so I compared against all models and papers I could find online, but I couldn't afford to train a full new model just to check wav2vec2 with BPE. That said, I did check against exhaustively allowing all 1-4 character tokens, which is pretty similar to BPE, and that performed worse in every situation.
  - gok 4y ago
    Without even actually retraining everything, I would be curious how different the tokens are with your technique compared to using an off-the-shelf solution like SentencePiece with the same output vocabulary size.
a2128 4y ago
This is very impressive work. Could this model be capable of doing streamed input for live speech recognition?
- fxtentacle 4y ago
  No. The best you can do with this AI architecture would be something like NVIDIA's NeMo, meaning you group the input audio stream into 2s blocks with 1s overlap and then run the speech recognition on that. If you want real-time speech recognition with less than 0.5s of delay between speaking the word and it being fully recognized, then one needs to implement a different architecture. And that one is much more difficult and expensive to train that this one (which was already expensive). That said, I want a fully offline and privacy-respecting voice assistant myself. So attempting to build the AI for real-time streamed live English speech recognition will be my next project. I plan to ship it as an OpenGL-accelerated binary with WebRTC server so that others can easily combine my recognition with their logic. But it probably won't be free since I'm looking at >$100k in compute costs to build it. In any case, here's the waiting list: https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5/join https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5...
  - a2128 4y ago
    Thanks for the information! I've been looking to build an accessibility tool for a deaf community so that they can see live captions of conversations, but some of the existing solutions I've tried seem to lag behind in conversational speech accuracy, or they're difficult/impossible to fine-tune with the community-specific words and phrases.
    - fxtentacle 4y ago
      This type of "Digital Therapeutics" might be paid for by German health insurance companies. For example, https://gaia-group.com/en/ https://gaia-group.com/en/ appears to be a successful provider of medical apps. If you don't mind, please email me at moin@deutscheki.de and explain in a bit more detail what the needs of that deaf community are. Maybe I can forward that to the right people to get my government to pay for the app that you wish to see developed. I mean I agree with you, it certainly would increase life satisfaction for deaf people if they could "listen in" into conversations to know what others are gossiping about.
snickerbockers 4y ago
*and also a bunch of git submodules
- deleted 4y ago
  [deleted]
einpoklum 4y ago
I think the author should probably use some C++ <algorithm>'s in lines 241 and later, avoiding raw loops and making the code clearer. Also, yeah, it's not 284 lines of C++ by any stretch of the imagination, but the main file is pretty readable, which is nice.
- fxtentacle 4y ago
  I just used the Linux CLOC on the C++ files that I wrote. Yes, I didn't count my 3 dependencies KenLM, Wave, or TensorFlow, because those are used by pretty much all speech recognition projects. For comparing the complexity of my code to Mozilla's DeepSpeech, it makes sense to ignore the LOCs for shared dependencies.
  - einpoklum 4y ago
    It was a cute gimmick for the title, nm. But you should try to avoid raw loops in favor of established iteration patterns / standard library algorithms. I found this talk on the subject by Sean Parent to be educational: https://www.youtube.com/watch?v=qH6sSOr-yk8 https://www.youtube.com/watch?v=qH6sSOr-yk8
    - fxtentacle 4y ago
      Thanks for sharing the talk :)
- deleted 4y ago
  [deleted]
nshm 4y ago
This model seems strongly overtrained on CV test set. Usually improvement from LM rescoring is just 10% relative. In the paper https://arxiv.org/pdf/2206.12693.pdf https://arxiv.org/pdf/2206.12693.pdf the improvement is from 10.1% WER to 3.64% WER (Table 6). Such a big improvement suggests that LM is biased. Also, the perplexity of provided ngram LM on CV test set is just 86 and most of 5-gram histories are already in the LM. This also suggests bias.
- rob74 4y ago
  Sounds interesting, although as someone not that deeply into ML these terms don't say a lot to me. What would "bias" mean in this case? That the model would recognize a "standard German" speaker, but not someone from Bavaria? Because that happens to a lot of (non-Bavarian) Germans too.
  - nshm 4y ago
    The model knows the recognition text well and demonstrates good results because of it. If you test the same model on some unrelated speech which model didn't see yet the results will not be that great. Error rate might be significantly worse than other systems.
    - Evidlo 4y ago
      Wouldn't that more commonly be called generalizability?
- fxtentacle 4y ago
  Also in Table 6, you see that Facebook's wav2vec 2.0 XLS-R went from 12.06% without LM to 4.38% with 5-gram LM. In comparison to that, I found TEVR going from 10.10% to 3.64% unproblematic. The core assumption of my paper is that for German specifically, the language model is very important due to conserved (and usually mumbled) word endings. Anyway, it's roughly a 64% reduction for both wav2vec2 XLS-R and TEVR. So if your criticism that I overtrained the TEVR model turns out to be correct, then that would suggest that the Zimmermeister 2022 wav2vec2 XLS-R was equally over-trained, which would still make it a fair comparison w.r.t. the 16% relative improvement in WER. Or are you suggesting that all wav2vec2 -derived AI models are strongly overtrained for CommonVoice? Because they seem to do very well on LibriSpeech and GigaSpeech, too. Could you explain what you mean by "perplexity" here? Can you recommend a paper about it? I haven't read about that in any of the ASR papers I studied, so this sounds like an exciting new technique for me to learn :) BTW, regardless of the metrics, this is the model that "works for me" in production. BTW, BTW, it would be really helpful for research if Vosk could also publish a paper. As you can see, PapersWithCode.com currently doesn't list any Vosk WERs for CommonVoice German, despite the website reporting 11.99% for vosk-model-de-0.21.
maratc 4y ago
I'd like to know its accuracy on the sound of Rhabarberbarbara[0] or Nackenzacken[1]. [0]https://www.youtube.com/watch?v=l3_tRPRt9x8 https://www.youtube.com/watch?v=l3_tRPRt9x8 [1]https://www.youtube.com/watch?v=Mngm8k4_qAc https://www.youtube.com/watch?v=Mngm8k4_qAc
- marshray 4y ago
  Bet it'd do just fine - maybe better than humans. People just get confused by large numbers of similar syllables because we have to buffer them for more processing. I suspect a pure speech-to-text model doesn't need to worry so much about context and can just take the syllables one by one.
  - fxtentacle 4y ago
    Actually, this model uses attention layers which are kind of like a query/key=>value look-up and allow the model to merge the knowledge from neighboring time-steps into the current logit prediction. The result is that this model performs worse for highly repetitive words, just like humans do.
    - marshray 4y ago
      This stuff is so fascinating, I really need to learn more about it. :-)
nice_byte 4y ago
> 284 lines of c++ > includes the entirety of tensorflow as a submodule
- deleted 4y ago
  [deleted]
woah 4y ago
284 lines of C++ and how many trained weights? Touting the "low number of lines" on a neural network project seems kind of silly, since the logic is encoded in the weights. Kind of like if I said "Doom in 5 lines of JS", but those 5 lines just downloaded and ran 130kb of WASM
- munk-a 4y ago
  If you want to be really lean I wrote a program with identical performance as the linked one in a single line of shell code. ./tevr_asr_tool Lines of code began and persists as an absolutely awful way to measure anything.
  - fxtentacle 4y ago
    I mentioned them to make it obvious that this is A LOT LESS code than Mozilla's DeepSpeech, which uses the same dependencies (plus some).
- fxtentacle 4y ago
  The novelty here is the TEVR tokens, which is in the c++ beam search decoder. Also, https://news.ycombinator.com/item?id=32411566 https://news.ycombinator.com/item?id=32411566
  - joe__f 4y ago
    Just delete all the whitespace and you'll have the whole program in one line of code
    - fxtentacle 4y ago
      Actually, I used "cloc" to calculate the number and I believe it'll split at semicolons to prevent this. But great hack anyway :)
      - joe__f 4y ago
        Hahaha ok you got me
codeflo 4y ago
I don’t want to take anything away from the achievement, because this looks very useful, but the headline misled me a bit. I’m note sure whether it’s my affinity for code golf, but when I see someone bragging about about line count, I don’t expect the use of multi million line nonstandard libraries. :) For anyone wondering, most of the 284 lines are (of course, some might say) calls into Tensorflow. Still, I think this is really nice, just not what I expected from the title.
- thaumasiotes 4y ago
  > I’m not sure whether it’s my affinity for code golf, but when I see someone bragging about about line count, I don’t expect the use of multi million line nonstandard libraries. I wouldn't say the headline is "misleading" - no one is about to be fooled into thinking 300 lines of C++ could be capable of state-of-the-art speech recognition. The headline is squarely in the territory of "complete nonsense, but you can tell that without having to read further than the headline".
  - eurasiantiger 4y ago
    Are you saying state-of-the-art speech recognition cannot fit in 284 lines of C++ without any third party libraries?
    - thaumasiotes 4y ago
      Yes, assuming the lines are of reasonable length.
      - eurasiantiger 4y ago
        I suppose we have a challenge in our hands.
        fxtentacle 4y ago
        As soon as you succeed, I'm pretty sure someone will complain that the sequence of matrix multiplications in the AI parameter file also counts as "code" in the wider sense.
olliej 4y ago
284 loc + however many LoC in tensor flow + however many training weights. I think "X in Y LoC" should be limited to where Y is the LoC to do the work, not the LoC to setup/interface with some other library. We're getting ever closer to "SQL DB in only 200 LoC" that simply forwards to sqlite or some such.
- fxtentacle 4y ago
  The unique work that makes this speech recognition superior to other tools is in those 284 lines of code: https://github.com/DeutscheKI/tevr-asr-tool/blob/master/tevr_asr_tool.cc#L57-L160 https://github.com/DeutscheKI/tevr-asr-tool/blob/master/tevr... That's a custom-designed beam search decoder implemented in C++ and based on the research for my TEVR paper. It increases performance by a relative 16% reduction in word error rate.
- deleted 4y ago
  [deleted]
CamperBob2 4y ago
LOL, 284 lines of C++, plus: #include "absl/flags/parse.h" #include "absl/flags/flag.h" #include "absl/flags/usage.h" #include "absl/flags/internal/commandlineflag.h" #include "absl/flags/internal/private_handle_accessor.h" #include "absl/flags/reflection.h" #include "absl/flags/usage_config.h" #include "absl/memory/memory.h" #include "absl/strings/match.h" #include "absl/strings/str_cat.h" #include "absl/strings/string_view.h" #include "tensorflow/lite/c/common.h" #include "tensorflow/lite/delegates/hexagon/hexagon_delegate.h" #include "tensorflow/lite/interpreter.h" #include "tensorflow/lite/interpreter_builder.h" #include "tensorflow/lite/kernels/kernel_util.h" #include "tensorflow/lite/kernels/register.h" #include "tensorflow/lite/model_builder.h" #include "tensorflow/lite/testing/util.h" #include "tensorflow/lite/tools/benchmark/benchmark_utils.h" #include "tensorflow/lite/interpreter.h" #include "tensorflow/lite/kernels/register.h" #include "tensorflow/lite/model.h" #include "tensorflow/lite/optional_debug_tools.h" #include "kenlm/lm/ngram_query.hh" #include "wave/file.h" #include "tensorflow/lite/minimal_logging.h"
- deleted 4y ago
  [deleted]
amelius 4y ago
How many lines for an English version? And how many lines can be re-used in a version that recognizes both German and English?
fxtentacle 4y ago
Original poster here, I just wanted to say thank you! Despite some snark - which I totally deserve, but I can't change the title anymore - I also received some very helpful advice, learned something new, and got introduced to people who plan to use this technology to help others. I can't think of any better outcome for me publishing my research. Thanks :)

231 points by fxtentacle 4y ago

pkaye 4y ago

Includes a standard C++ library called tensorflow...

fxtentacle 4y ago
... like all the other speech recognition projects that you might want to compare the LOC against.
- numpad 4y ago
  ...which however don't clickbait by using LoC as a primary selling point.
  - fxtentacle 4y ago
    I wouldn't call it clickbait. Mozilla's DeepSpeech is so large that you can't really read it and understand it. This one is 10x less code while recognition quality is 75% better (lower relative word error rate). So this one is small enough that you can read the source code if you want to, while DeepSpeech is not.
    - numpad 4y ago
      Good point – calling this clickbait might be too cynical. From this perspective I can definitely get behind advertising the project with the LoC measurement. Subjectively, I still find this to be a bit of a "not telling the whole truth", however I also only ever toyed around with speech recognition ai.
- Evidlo 4y ago
  You could have maybe said "284 lines of code plus weights" to avoid all this criticism that misses the point of the project.
  - fxtentacle 4y ago
    That wouldn't have fit into the 80 character limit. I also wanted to mention that it's private and offline, but couldn't say it compact enough.
deleted 4y ago
[deleted]

fxtentacle 4y ago

And for the true "Show HN" experience: wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.001 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.002 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.003 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.004 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." wget "https://github.com/DeutscheKI/tevr-asr-tool/releases/download/v1.0.0/tevr_asr_tool-1.0.0-Linux-x86_64.zip.005 https://github.com/DeutscheKI/tevr-asr-tool/releases/downloa..." cat tevr_asr_tool-1.0.0-Linux-x86_64.zip.00* > tevr_asr_tool-1.0.0-Linux-x86_64.zip unzip tevr_asr_tool-1.0.0-Linux-x86_64.zip sudo dpkg -i tevr_asr_tool-1.0.0-Linux-x86_64.deb tevr_asr_tool --target_file=test_audio.wav and then you'll be greeted with some TensorFlow Lite diagnostics, followed by the intermediate states of the beam-search decoder, followed by the hopefully correct transcription result. And if that piques your curiosity, here's a short overview over the code: https://github.com/DeutscheKI/tevr-asr-tool#how-does-this-work https://github.com/DeutscheKI/tevr-asr-tool#how-does-this-wo...

OrangeMonkey 4y ago

I can do it in 1 line of code if it includes downloading your github repo.

mrweasel 4y ago
Using operating system provided functionality, a GUI library or anything in libc, isn't something I'd hold against anyone when counting lines of code. Relying on an entire external project however, that's a bit misleading.
- hardware2win 4y ago
  Where is the boundry then? You cannot "outsource" the "core"?
  - dahfizz 4y ago
    You can do whatever you want. Just don't brag about your line count if all your project does is import dependencies.
    - fxtentacle 4y ago
      It doesn't. It implements a new way of decoding the logits which improves performance by a relative 16% over the previously best German speech recognition, which was Facebook's wav2vec2. And the size is relevant to people in the industry because DeepSpeech uses 2000+ LOCs for implementing their decoding, so this works better and is 10x less code.
      - agileAlligator 4y ago
        > It implements a new way of decoding the logits which improves performance by a relative 16% Then you should have mentioned that in the title rather than the LOCs
  - mrweasel 4y ago
    I don't know. It's not that it isn't impressive and it is very illustrative of how much you can build, utilizing the tools that are available to all of us. It's just at what point are you writing code and when are you just initializing and configuring an existing tool. It's also how you present it. If you said: "Building a TensorFlow backed German speech recognition system in only a few hundred lines of code." Then I feel like you're being more honest.

BurningPenguin 4y ago

How well does it do with accent or slight dialect influences?

deleted 4y ago
[deleted]
fxtentacle 4y ago
We tested it on CommonVoice German, which is what Mozilla used for DeepSpeech German, too. The idea behind that dataset is that arbitrary people on the internet submit their recordings (hence the "common" in the name) and then if enough other people upvote it as "understandable", it gets included in the dataset. As such, the AI works well with a variety of accents.
- junkerm 4y ago
  I suspect that "works well" means that the model will output words in "official german" and kind of corrects pronounciation errors? I am asking because I had the use case to automatically give feedback to non native german speakers.
  - fxtentacle 4y ago
    There's a command-line parameter "--use_language_model=0" to disable the spellchecker.
deleted 4y ago
[deleted]

nottorp 4y ago

And if you add the dependent libraries it's 3 million lines?

deleted 4y ago
[deleted]

tjungblut 4y ago

I'm sure you can save another line while checking for mückenstiche in your dictionary.

dahfizz 4y ago

Guys I just wrote a json parser in one line of code! import json

petercooper 4y ago
And all algorithms are O(f(n)), where f is someone else's responsibility.
deleted 4y ago
[deleted]

captainmuon 4y ago

This looks pretty cool, especially since it is offline and free! The only caveat is that it is probably not feasible to train yourself, right? I should play with this if I have some time. I've had the idea for a while to build a voice assistent which can switch modes or datasets while you are speaking. If you say "computer, play ...." for example it would load a recognizer that is specialized on song names. The idea is that you can mix English song names in a German prompt, and it will not be confused. Every voice assistent I know gets confused, presumably because they convert speech to plain text and only then act on the text.

fxtentacle 4y ago
> I've had the idea for a while to build a voice assistent which can switch modes or datasets while you are speaking. If you say "computer, play ...." for example it would load a recognizer that is specialized on song names. I would say by now the generic recognizers are so good that this is becoming less and less useful. For example, this tool handles non-existing German words quite well. That said, the tool has a "--data_folder_path" parameter where you can specify a different acoustic and language model. BTW, I also want to build an offline voice assistant :) That's how I got started on this journey. You might be interested in my next project, where I try to do offline real-time English recognition with a WebRTC API to make it easy for developers to connect my AI module with their own task logic. Here's the waiting list: https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5/join https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5...

postalrat 4y ago

And 27 includes plus a huge model. "284 lines of C++" is something you could fit on a small microcontroller. This isn't 284 lines of C++.

fxtentacle 4y ago
In my opinion, the focus here should be on the fact that this is a state of the art AI which beats Facebook's wav2vec2 by a relative 16% improvement, Scribosermo (based on NVIDIA NeMo) by a relative 44% and Mozilla's DeepSpeech German by a relative 75% improvement. People usually don't share their production-quality tools ;) That said, I wrote "284 lines of C++" to indicate that this is compact enough for people to actually read and understand the source code. Also, compiling my implementation is super easy and straightforward ... something which can't be said for Kaldi, Vosk, or DeepSpeech. If you try to read the CTC beam search decoder from Mozilla's DeepSpeech [1], that alone is about 2000 LOC in multiple files. If you try to read the pyctcdecode source that is used by HuggingFace [2], that's 1000+ LOC of Python. But this implementation is all the client-side, i.e. the entire "native_client" folder hierarchy in DeepSpeech [3], narrowed down to a mere 284 lines. Also, both DeepSpeech and HuggingFace Transformers use TensorFlow as a dependency, i.e. just like me. So in my opinion, it doesn't make sense to include TF in the LOC comparison if all the AI speech recognition systems use it. That would be like including libstdc++, too. [1] https://github.com/mozilla/DeepSpeech/tree/master/native_client/ctcdecode https://github.com/mozilla/DeepSpeech/tree/master/native_cli... [2] https://github.com/kensho-technologies/pyctcdecode https://github.com/kensho-technologies/pyctcdecode [3] https://github.com/mozilla/DeepSpeech/tree/master/native_client https://github.com/mozilla/DeepSpeech/tree/master/native_cli...
- kylebgorman 4y ago
  A speech recognizer is not an "AI".
  - fxtentacle 4y ago
    Well, the acoustic model is created using deep learning loss-minimization by gradient descent, which is what people usually call AI these days.

vintermann 4y ago

284 lines of C++, and a 1.53 GB wav2vec2 model. But still very nice!

fxtentacle 4y ago
Actually the model is TEVR, which is only the wav2vec2 feature extraction but with a modified encoder that improves performance by exploiting redundancies in the German language.
deleted 4y ago
[deleted]

cgeier 4y ago

I was quite interested in this, but after seeing that it includes three big other repositories, my interest was immediately gone. I'm not saying this isn't an interesting project, but perhaps lead with something different?

eternalban 4y ago
How to assemble a German speech recognition program in under 300 lines. - ps: I skimmed the paper cited and what I wrote above is -not- correct. The project is not simply assembling a pipeline, it is claiming ('papers with code') an innovation: "This paper presents TEVR, a speech recognition model designed to minimize the variation in token entropy w.r.t. to the language model. This takes advantage of the fact that if the language model will reliably and accurately predict a token anyway, then the acoustic model doesn't need to be accurate in recognizing it." ... "We have shown that when combined with an appropriately tuned language model, the TEVR-enhanced model outperforms the best German automated speech recognition result from literature by a relative 44.85% reduction in word error rate. It also outperforms the best self-reported community model by a relative 16.89% reduction in word error rate."
- fxtentacle 4y ago
  Yes, we actually did research and spent a significant amount of GPU time. Thanks to my luck of stumbling into the right people at the right time, I could afford cloud-scale training because OVH granted me very generous rebates... The main innovation is that we prevent loss mis-allocation during training of the acoustic AI model by pre-weighting things with the loss of the language model. Or in short: We don't train what you don't need to hear If you want to play around with the TEVR tokenizer design, here's the source for that: https://huggingface.co/fxtentacle/tevr-token-entropy-predictor-de https://huggingface.co/fxtentacle/tevr-token-entropy-predict...
  - cgeier 4y ago
    > The main innovation is that we prevent loss mis-allocation during training of the acoustic AI model by pre-weighting things with the loss of the language model. Or in short: > We don't train what you don't need to hear This does sound a lot more interesting than the ~280 lines of code.

kevin_b_er 4y ago

So, what caught my interest was this quote at the bottom: "Alternatively, you can donate roughly 2 weeks of A100 GPU credits to me and I'll train a suitable recognition model and upload it to HuggingFace." This takes 2 weeks of A100 GPU time to train? Current GC spot pricing puts that at about $300.

fxtentacle 4y ago
"This takes 2 weeks of A100 GPU time to train?" Yes, but only for me in this specific case. That's because this German model was derived based on a model which was pre-trained for months on hundreds of thousands of hours of YouTube audio in various languages. So if I now train an English recognizer, I don't start from scratch. Instead, I start from a checkpoint which can already almost flawlessly recognize any human utterance. (Or all phonemes in the IPA alphabet, to be more precise) The "training" then only learns the mapping between IPA phonemes and text notation.
- deleted 4y ago
  [deleted]
- mciancia 4y ago
  What other languages can you "easily" do like that?
  - fxtentacle 4y ago
    I believe for English, German, Dutch, Spanish, French, Italian, Portuguese, Polish there's enough data in CommonVoice to get good results.
    - gogusrl 4y ago
      Any chance for Romanian ?
      - fxtentacle 4y ago

henrydark 4y ago

There are so many comments about the "real" length of the program. The number 284 means something to people who work in speech recognition - these are the people that know how much _they_ write when they try to compete with this library. This number isn't meant for people who are disinterested (have no stake) in speech recognition.

deleted 4y ago
[deleted]
prewett 4y ago
"X in N lines of code" is usually used for compressing an algorithm down to its bare essentials. "Ray tracing in 100 lines of code". In that case, it's disingenuous to have 100 lines of glue code that calls into pov-ray. If I compile 284 lines of code, I expect a few kb executable (including libraries that aren't language-libraries like libc). This hasn't really simplified the speech-recognition algorithm, it still requires large amounts of training data and time, and still does the speech recognition by solving a large matrix with Tensorflow. It's not interesting to me that you can hook up Tensorflow in a few lines of code; I know Tensorflow can solve the problem. However, I would be interested in 284 lines of code that replace Tensorflow, which is what the title suggested to me. A better title would be to focus on how the model and/or the handling of the data produces better results, because that's what this code seems to be about.
- fxtentacle 4y ago
  If you compare this with Mozilla's DeepSpeech repository, you will find that correctly calling TensorFlow Lite and handling the results in only 284 lines of code is impressive, too. Also, the C++ code is mostly a custom beam search decoder based on the research for my paper, so it's not like TensorFlow is doing all the heavy lifting here, because precisely that TEVR token decoder causes the relative 16% performance improvement of this speech recognition AI over others.

lynndotpy 4y ago

To those criticizing the title, how would you improve it? Dependencies do the heavy lifting , but that's true even for a hello-world. It seems obvious enough that I wouldn't see a reason to clarify. Instead, to me, this reads, "Hey C++ fans, you don't need Python for nice things. Look at what you can do in 284 lines of code!" I don't have C++ chops, so this is nontrivial for me. I appreciate OP sharing this!

itcrowd 4y ago
Show HN: State-of-the-Art German Speech Recognition in C++
- lynndotpy 4y ago
  That makes sense, but knowing it's 284 LOC means I can casually read it and learn something from that. That would be different at 2840 LOC or very very different at 284000 LOC.
  - fxtentacle 4y ago
    Yes, this is precisely why I included it. I wanted to highlight for people who have experience with speech recognition in general that this is a magnitude easier to read than Mozilla's DeepSpeech.
UncleEntity 4y ago
> To those criticizing the title, how would you improve it? By not making false assertions? Judging by the comments the 284 lines is in addition to many hours of GPU training time plus some magic and a huge library. I didn’t even click the link because I knew it was 284 lines of plumbing code on top of something else. I once wrote a entire renderman compatible renderer in a couple hundred lines of python — which was really a super dumb script which generated ctypes bindings from the header for an actual renderer (pixie if anyone is curious).
- fxtentacle 4y ago

fwsgonzo 4y ago

Is it possible to get speech recognition into a statically linked executable? ldd myprogram should say "not a dynamic executable" Are there any libraries for those of us in embedded?

fxtentacle 4y ago
My code does precisely that, the only dependencies of the finished executable are Debian OS libraries: $ lddtree build/tevr_asr_tool tevr_asr_tool => build/tevr_asr_tool (interpreter => /lib64/ld-linux-x86-64.so.2) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
- fwsgonzo 4y ago
  I guess I will give it a try then.

ale42 4y ago

284 lines of C++, and a big external dependency (tensorflow)... plus the model (but this was expected).

deleted 4y ago
[deleted]

gok 4y ago

The "TEVR" token encoding idea is interesting. Any chance you compared it with other subword tokenization techniques like BPE?

fxtentacle 4y ago
Sorry, not really. Training this pipeline was already quite expensive so I compared against all models and papers I could find online, but I couldn't afford to train a full new model just to check wav2vec2 with BPE. That said, I did check against exhaustively allowing all 1-4 character tokens, which is pretty similar to BPE, and that performed worse in every situation.
- gok 4y ago
  Without even actually retraining everything, I would be curious how different the tokens are with your technique compared to using an off-the-shelf solution like SentencePiece with the same output vocabulary size.

a2128 4y ago

This is very impressive work. Could this model be capable of doing streamed input for live speech recognition?

fxtentacle 4y ago
No. The best you can do with this AI architecture would be something like NVIDIA's NeMo, meaning you group the input audio stream into 2s blocks with 1s overlap and then run the speech recognition on that. If you want real-time speech recognition with less than 0.5s of delay between speaking the word and it being fully recognized, then one needs to implement a different architecture. And that one is much more difficult and expensive to train that this one (which was already expensive). That said, I want a fully offline and privacy-respecting voice assistant myself. So attempting to build the AI for real-time streamed live English speech recognition will be my next project. I plan to ship it as an OpenGL-accelerated binary with WebRTC server so that others can easily combine my recognition with their logic. But it probably won't be free since I'm looking at >$100k in compute costs to build it. In any case, here's the waiting list: https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5/join https://madmimi.com/signups/f0da3b13840d40ce9e061cafea6280d5...
- a2128 4y ago
  Thanks for the information! I've been looking to build an accessibility tool for a deaf community so that they can see live captions of conversations, but some of the existing solutions I've tried seem to lag behind in conversational speech accuracy, or they're difficult/impossible to fine-tune with the community-specific words and phrases.
  - fxtentacle 4y ago
    This type of "Digital Therapeutics" might be paid for by German health insurance companies. For example, https://gaia-group.com/en/ https://gaia-group.com/en/ appears to be a successful provider of medical apps. If you don't mind, please email me at moin@deutscheki.de and explain in a bit more detail what the needs of that deaf community are. Maybe I can forward that to the right people to get my government to pay for the app that you wish to see developed. I mean I agree with you, it certainly would increase life satisfaction for deaf people if they could "listen in" into conversations to know what others are gossiping about.

snickerbockers 4y ago

*and also a bunch of git submodules

deleted 4y ago
[deleted]

einpoklum 4y ago

I think the author should probably use some C++ <algorithm>'s in lines 241 and later, avoiding raw loops and making the code clearer. Also, yeah, it's not 284 lines of C++ by any stretch of the imagination, but the main file is pretty readable, which is nice.

fxtentacle 4y ago
I just used the Linux CLOC on the C++ files that I wrote. Yes, I didn't count my 3 dependencies KenLM, Wave, or TensorFlow, because those are used by pretty much all speech recognition projects. For comparing the complexity of my code to Mozilla's DeepSpeech, it makes sense to ignore the LOCs for shared dependencies.
- einpoklum 4y ago
  It was a cute gimmick for the title, nm. But you should try to avoid raw loops in favor of established iteration patterns / standard library algorithms. I found this talk on the subject by Sean Parent to be educational: https://www.youtube.com/watch?v=qH6sSOr-yk8 https://www.youtube.com/watch?v=qH6sSOr-yk8
  - fxtentacle 4y ago
    Thanks for sharing the talk :)
deleted 4y ago
[deleted]

nshm 4y ago

This model seems strongly overtrained on CV test set. Usually improvement from LM rescoring is just 10% relative. In the paper https://arxiv.org/pdf/2206.12693.pdf https://arxiv.org/pdf/2206.12693.pdf the improvement is from 10.1% WER to 3.64% WER (Table 6). Such a big improvement suggests that LM is biased. Also, the perplexity of provided ngram LM on CV test set is just 86 and most of 5-gram histories are already in the LM. This also suggests bias.

rob74 4y ago
Sounds interesting, although as someone not that deeply into ML these terms don't say a lot to me. What would "bias" mean in this case? That the model would recognize a "standard German" speaker, but not someone from Bavaria? Because that happens to a lot of (non-Bavarian) Germans too.
- nshm 4y ago
  The model knows the recognition text well and demonstrates good results because of it. If you test the same model on some unrelated speech which model didn't see yet the results will not be that great. Error rate might be significantly worse than other systems.
  - Evidlo 4y ago
    Wouldn't that more commonly be called generalizability?
fxtentacle 4y ago
Also in Table 6, you see that Facebook's wav2vec 2.0 XLS-R went from 12.06% without LM to 4.38% with 5-gram LM. In comparison to that, I found TEVR going from 10.10% to 3.64% unproblematic. The core assumption of my paper is that for German specifically, the language model is very important due to conserved (and usually mumbled) word endings. Anyway, it's roughly a 64% reduction for both wav2vec2 XLS-R and TEVR. So if your criticism that I overtrained the TEVR model turns out to be correct, then that would suggest that the Zimmermeister 2022 wav2vec2 XLS-R was equally over-trained, which would still make it a fair comparison w.r.t. the 16% relative improvement in WER. Or are you suggesting that all wav2vec2 -derived AI models are strongly overtrained for CommonVoice? Because they seem to do very well on LibriSpeech and GigaSpeech, too. Could you explain what you mean by "perplexity" here? Can you recommend a paper about it? I haven't read about that in any of the ASR papers I studied, so this sounds like an exciting new technique for me to learn :) BTW, regardless of the metrics, this is the model that "works for me" in production. BTW, BTW, it would be really helpful for research if Vosk could also publish a paper. As you can see, PapersWithCode.com currently doesn't list any Vosk WERs for CommonVoice German, despite the website reporting 11.99% for vosk-model-de-0.21.

maratc 4y ago

I'd like to know its accuracy on the sound of Rhabarberbarbara[0] or Nackenzacken[1]. [0]https://www.youtube.com/watch?v=l3_tRPRt9x8 https://www.youtube.com/watch?v=l3_tRPRt9x8 [1]https://www.youtube.com/watch?v=Mngm8k4_qAc https://www.youtube.com/watch?v=Mngm8k4_qAc

marshray 4y ago
Bet it'd do just fine - maybe better than humans. People just get confused by large numbers of similar syllables because we have to buffer them for more processing. I suspect a pure speech-to-text model doesn't need to worry so much about context and can just take the syllables one by one.
- fxtentacle 4y ago
  Actually, this model uses attention layers which are kind of like a query/key=>value look-up and allow the model to merge the knowledge from neighboring time-steps into the current logit prediction. The result is that this model performs worse for highly repetitive words, just like humans do.
  - marshray 4y ago
    This stuff is so fascinating, I really need to learn more about it. :-)

nice_byte 4y ago

> 284 lines of c++ > includes the entirety of tensorflow as a submodule

deleted 4y ago
[deleted]

woah 4y ago

284 lines of C++ and how many trained weights? Touting the "low number of lines" on a neural network project seems kind of silly, since the logic is encoded in the weights. Kind of like if I said "Doom in 5 lines of JS", but those 5 lines just downloaded and ran 130kb of WASM

munk-a 4y ago
If you want to be really lean I wrote a program with identical performance as the linked one in a single line of shell code. ./tevr_asr_tool Lines of code began and persists as an absolutely awful way to measure anything.
- fxtentacle 4y ago
  I mentioned them to make it obvious that this is A LOT LESS code than Mozilla's DeepSpeech, which uses the same dependencies (plus some).
fxtentacle 4y ago
The novelty here is the TEVR tokens, which is in the c++ beam search decoder. Also, https://news.ycombinator.com/item?id=32411566 https://news.ycombinator.com/item?id=32411566
- joe__f 4y ago
  Just delete all the whitespace and you'll have the whole program in one line of code
  - fxtentacle 4y ago
    Actually, I used "cloc" to calculate the number and I believe it'll split at semicolons to prevent this. But great hack anyway :)
    - joe__f 4y ago
      Hahaha ok you got me

codeflo 4y ago

I don’t want to take anything away from the achievement, because this looks very useful, but the headline misled me a bit. I’m note sure whether it’s my affinity for code golf, but when I see someone bragging about about line count, I don’t expect the use of multi million line nonstandard libraries. :) For anyone wondering, most of the 284 lines are (of course, some might say) calls into Tensorflow. Still, I think this is really nice, just not what I expected from the title.

thaumasiotes 4y ago
> I’m not sure whether it’s my affinity for code golf, but when I see someone bragging about about line count, I don’t expect the use of multi million line nonstandard libraries. I wouldn't say the headline is "misleading" - no one is about to be fooled into thinking 300 lines of C++ could be capable of state-of-the-art speech recognition. The headline is squarely in the territory of "complete nonsense, but you can tell that without having to read further than the headline".
- eurasiantiger 4y ago
  Are you saying state-of-the-art speech recognition cannot fit in 284 lines of C++ without any third party libraries?
  - thaumasiotes 4y ago
    Yes, assuming the lines are of reasonable length.
    - eurasiantiger 4y ago
      I suppose we have a challenge in our hands.
      - fxtentacle 4y ago
        As soon as you succeed, I'm pretty sure someone will complain that the sequence of matrix multiplications in the AI parameter file also counts as "code" in the wider sense.

olliej 4y ago

284 loc + however many LoC in tensor flow + however many training weights. I think "X in Y LoC" should be limited to where Y is the LoC to do the work, not the LoC to setup/interface with some other library. We're getting ever closer to "SQL DB in only 200 LoC" that simply forwards to sqlite or some such.

fxtentacle 4y ago
The unique work that makes this speech recognition superior to other tools is in those 284 lines of code: https://github.com/DeutscheKI/tevr-asr-tool/blob/master/tevr_asr_tool.cc#L57-L160 https://github.com/DeutscheKI/tevr-asr-tool/blob/master/tevr... That's a custom-designed beam search decoder implemented in C++ and based on the research for my TEVR paper. It increases performance by a relative 16% reduction in word error rate.
deleted 4y ago
[deleted]

CamperBob2 4y ago

LOL, 284 lines of C++, plus: #include "absl/flags/parse.h" #include "absl/flags/flag.h" #include "absl/flags/usage.h" #include "absl/flags/internal/commandlineflag.h" #include "absl/flags/internal/private_handle_accessor.h" #include "absl/flags/reflection.h" #include "absl/flags/usage_config.h" #include "absl/memory/memory.h" #include "absl/strings/match.h" #include "absl/strings/str_cat.h" #include "absl/strings/string_view.h" #include "tensorflow/lite/c/common.h" #include "tensorflow/lite/delegates/hexagon/hexagon_delegate.h" #include "tensorflow/lite/interpreter.h" #include "tensorflow/lite/interpreter_builder.h" #include "tensorflow/lite/kernels/kernel_util.h" #include "tensorflow/lite/kernels/register.h" #include "tensorflow/lite/model_builder.h" #include "tensorflow/lite/testing/util.h" #include "tensorflow/lite/tools/benchmark/benchmark_utils.h" #include "tensorflow/lite/interpreter.h" #include "tensorflow/lite/kernels/register.h" #include "tensorflow/lite/model.h" #include "tensorflow/lite/optional_debug_tools.h" #include "kenlm/lm/ngram_query.hh" #include "wave/file.h" #include "tensorflow/lite/minimal_logging.h"

deleted 4y ago
[deleted]

amelius 4y ago

How many lines for an English version? And how many lines can be re-used in a version that recognizes both German and English?

fxtentacle 4y ago

Original poster here, I just wanted to say thank you! Despite some snark - which I totally deserve, but I can't change the title anymore - I also received some very helpful advice, learned something new, and got introduced to people who plan to use this technology to help others. I can't think of any better outcome for me publishing my research. Thanks :)