GPT-3 was out for years already for Googlw to see, but with what OpenAI saw of it they began training an expensive GPT-4, before chatgpt success, maybe started

GPT-3 was out for years already for Googlw to see, but with what OpenAI saw of it they began training an expensive GPT-4, before chatgpt success, maybe started before RLHF.