4 ms·
Meh, I thought Google was always on the cutting edge of ML. They just tried to pimp up a two year old dysfunctional show pony and sell it as a new racehorse. I
by lowdose 6y ago
Meh, I thought Google was always on the cutting edge of ML. They just tried to pimp up a two year old dysfunctional show pony and sell it as a new racehorse. I would like to see a trained GPT3 that's is currently up for rent from the AI shop delivering actual innovation.
- lawrenceyan 6y agoGPT-3 costs $4.6 million dollars to train currently[0]. Not only is your comment unnecessarily negative, it's just generally a pretty bad take. [0]: https://lambdalabs.com/blog/demystifying-gpt-3/ https://lambdalabs.com/blog/demystifying-gpt-3/
- lowdose 6y agoWhat value is added by training a model yourself? Why don't they provide that model? Isn't the real innovation here to give people a $4.3 million model for free? Instead of letting people burn unnecessary compute power to train a model 2 years behind state of the art.
- lacker 6y agoThe main value is that you can choose your own training data. For example, if you wanted to train a text-generation model on purely C++ code, this would be a good way to do it.
- krageon 6y agoThey don't provide that model because then it can't be rented out to people with less upfront cash for insane profit.
- boulos 6y agoDisclosure: I work on Google Cloud. The link here is just to someone’s personal Colab notebook (which Google hosts for free at colab.research.google.com). This doesn’t appear to be a Google project. It’s more like a link to a GitHub repo.
- zachthewf 6y agoThis isn't a Google research project, it's just hosted on Colab. Also this is from April, pre-GPT3. Also given that there's tons of cutting-edge pre-trained models available for free but relatively few resources on how to actually train such large models on a hobbyist's budget, this type of work is really important.