4 ms·
Weird that a team led by Andrew Ng couldn't do better with the 11 billion parameters model than with the 1 billion one. This is becoming accessible for everyon
by visarga 13y ago
Weird that a team led by Andrew Ng couldn't do better with the 11 billion parameters model than with the 1 billion one.
This is becoming accessible for everyone. It's both exciting and terrifying. It has the potential to save humanity from itself or condemn us to totalitarianism. I am sure machine learning and NLP and statistical models are what enables them to analyze the data they collect on us.
Big, fast noSQL tables, clustering technology (map-reduce) and machine learning are what allows these guys to do what they do. Our most prized toys became our enemies.
- dumitrue 13y agoIt's possible that the underlying model is just not particularly good at learning from data. 11B parameters is a lot of free parameters to learn -- for instance, the main competitor to that paradigm is the work by Krizhevsky et al., which are convolutional networks with lots of parameter sharing, and I think they get better performance (on a comparable task) with ~60M free parameters.