5 ms·
Ask HN: Training my own chat gpt
Seeing all the buzz around GPT has got me interested in training my own AI on data from my field ( game development).
Any tips on how to get started?
- brrrrrm 4y agoChatGPT uses a modified version of GPT3 (details are sparse, but here are some: https://beta.openai.com/docs/model-index-for-researchers https://beta.openai.com/docs/model-index-for-researchers). There’s a logbook associated with the training of OPT (similar size to GPT-3): https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec... This article estimated the cost of training GPT3 to be over $4M: https://heits.digital/articles/gpt3-overview https://heits.digital/articles/gpt3-overview
- tru3_power 4y agoAny idea on how to do something like this? https://news.ycombinator.com/item?id=33795263 https://news.ycombinator.com/item?id=33795263
- ozten 4y agoI think there are different ways to get involved: 1) Creative applications of existing models Start playing with https://huggingface.co/ https://huggingface.co/ 2) MLOps/data engineering - Deploying, automating, uptime, cost control Classic SRE / SysAdmin skills 3) Creating new ML models (4 - 8 months investment) You probably want to understand some fundamentals. https://www.deeplearning.ai/ https://www.deeplearning.ai/ 4) ML Research - Like #3 but 4 - 8 years of investment, heavy Sciences