OpenAI's mentioned that it was basically getting a lot of little details right. Some of the bigger details: * Huge compute scale for the pre-training * Huge

OpenAI's mentioned that it was basically getting a lot of little details right. Some of the bigger details: * Huge compute scale for the pre-training * Huge data scale * Well processed and cleaned data (GDB has alluded to this) * Fine tuning with large amounts of code snippets and examples * Massive amounts of RLHF * Continual ongoing RLHF