4 ms·
yeesh it's an RNN and not even something like a transformer
by dnautics 5y ago
yeesh it's an RNN and not even something like a transformer
- cs702 5y ago...which means that even a modestly sized transformer model could do significantly better.
- dnautics 5y agowith a big heap of "maybe". Signal could be a limiting factor, but also come on, more than 99% correct? That's above the level of a non-professional human transcriber in most situations. I'm not sure that typing I get more than 99% correct, I've made at least six errors typing this sentence alone (though it's quick to fix). The real improvement, it seems would be to speed and latency, looking at the diagram, the sampling is over the course of 3 seconds, which is butt-slow. A good NN would be able to compressed patterns in the data stream and blat out more than one letter, or contextually learn letters in the word, or learn whole words, etc. But that is not, it appears, supported by the model.