3 ms·
I'm so excited that an open source community is trying to recreate GPT3 in the open: https://docs.google.com/document/d/1wfCZBd18DMNt6YcC6boPNMd9qzzH3zpHHfKj4de
by etaioinshrdlu 6y ago
I'm so excited that an open source community is trying to recreate GPT3 in the open: https://docs.google.com/document/d/1wfCZBd18DMNt6YcC6boPNMd9qzzH3zpHHfKj4dezk0g/edit https://docs.google.com/document/d/1wfCZBd18DMNt6YcC6boPNMd9...
I'd urge anyone interested to support this project.
- p1esk 6y agoYou don't really need to recreate GPT3. Just focus on building a better dataset. Much cheaper and a lot more useful.
- etaioinshrdlu 6y agoCan you elaborate on this terse statement? What parameter size would you train to? What will such a model be able to do that GPT3 cannot, and vice versa? What are you picturing?
- p1esk 6y agoI mean reproduce and improve/expand the dataset instead of training models.
- AgentME 6y agoWhat are you going to do with the dataset if not train models on it?
- p1esk 6y agoThe point is: training GPT-3 is too expensive to be a hobby project, but creating a good dataset isn't.
- etaioinshrdlu 6y agoIt's not too expensive for a group of hobbyists!
- p1esk 6y agoIt costs ~$4.5M to train GPT-3. How big is your group of hobbyists?