3 ms·

How do we know it’s just been scale up to before GPT-4? As I said, OpenAI hasn’t told us why ChatGPT is so much better than other models (Bloom, LaMDA, LLaMa) a

by EGreg 4y ago

How do we know it’s just been scale up to before GPT-4? As I said, OpenAI hasn’t told us why ChatGPT is so much better than other models (Bloom, LaMDA, LLaMa) and yet we know that they employ thousands of people to do RLHF, including for the coding models: https://www.semafor.com/article/01/27/2023/openai-has-hired-an-army-of-contractors-to-make-basic-coding-obsolete https://www.semafor.com/article/01/27/2023/openai-has-hired-... Doesn’t quite sound like it’s “just scale”. I asked ChatGPT about its training and corpus and it explicitly disavows having that information.

famouswaffles 4y ago
>OpenAI hasn’t told us why ChatGPT is so much better than other models Here's the thing.... It's not. Unless you restrict the pool to models you know about. https://crfm.stanford.edu/helm/latest/?group=core_scenarios https://crfm.stanford.edu/helm/latest/?group=core_scenarios And Again, chatGPT was not the general SOTA LLM. https://arxiv.org/abs/2210.11416 https://arxiv.org/abs/2210.11416 So in the research world with detailed papers explaining what and what not, we had models that were better.