3 ms·
Ask HN: What's GPT-4's secret sauce that makes it so much better than others?
Especially for coding, I've had absolutely fantastic results with GPT-4. I've tried ChatGPT, Claude, LlaMA, and others, but GPT-4 is the only LLM that performs superb in terms of coding. Most times I get the correct response at the first try.
There's a rumor about the number of GPT-4's parameters (1 trillion?). But I'm not sure. To me, it's just an amazing piece of technology that works very well and somehow I don't know how it achieved this performance.
- sdfas34366 3y ago[flagged]
- tikkun 3y agoOpenAI's mentioned that it was basically getting a lot of little details right. Some of the bigger details: * Huge compute scale for the pre-training * Huge data scale * Well processed and cleaned data (GDB has alluded to this) * Fine tuning with large amounts of code snippets and examples * Massive amounts of RLHF * Continual ongoing RLHF