3 ms·
I spent at least five years trying to use statistical ml and mlps to do NLP on social media comments from about 2003. Nothing like a transformer (or an rnn even
by sgt101 7y ago
I spent at least five years trying to use statistical ml and mlps to do NLP on social media comments from about 2003. Nothing like a transformer (or an rnn even) occurred to me.
I have a belief that someone in the USSR worked out a way of doing fluid dynamics that has enabled the Russians to develop hypersonics and super cavitation. This is probably rather straightforward - in the style of NS - of you know the principles. No one in the West ( or China) knows those principles, so Western torpedos and reentry vehicles are rather poor Vs Russian ones. Once you grasp how something works the fact that it's rather easy to apply in comparison to the process of getting the insight shouldn't detract from the value of the insight .
- roenxi 7y agoThe wiki page on RNN says the early groundwork was done in 1986 and the LSTM was a 1997 innovation. If they didn't occur to you in 2003 that doesn't imply much, they are not suprising concepts. The surprise was that in the mid-2000s suddenly GPU became so powerful that LSTM could be used to achieve interesting results. The story here isn't the models, it is the computers running the models.