ChatGPT uses a modified version of GPT3 (details are sparse, but here are some: https://beta.openai.com/docs/model-index-for-researchers https://beta.openai.com

ChatGPT uses a modified version of GPT3 (details are sparse, but here are some: https://beta.openai.com/docs/model-index-for-researchers https://beta.openai.com/docs/model-index-for-researchers). There’s a logbook associated with the training of OPT (similar size to GPT-3): https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec... This article estimated the cost of training GPT3 to be over $4M: https://heits.digital/articles/gpt3-overview https://heits.digital/articles/gpt3-overview