3 ms·
Right, but the intuition I am going for isn't "Are 10 high-school students who collaborate together smarter than 1 high-school student working on their own". T
by zoogeny 3y ago
Right, but the intuition I am going for isn't "Are 10 high-school students who collaborate together smarter than 1 high-school student working on their own".
The intuition I am asking for is more like: "Are 10, 100, or 1000 GPT-3s capable of producing the equivalent intelligence of 1 GPT-4". That kind of reasoning expanded to GPT-N.
And further, if no number of GPT-3s can reach the level of intelligence of a GPT-4, then what set of problems can be solved by 100s of GPT-3s in a mixed model and what problems require the single GPT-4.
- visarga 3y agoYou can generally amplify models by using more tokens, more calls or relying on some oracle of correctness. I see rapid growth of synthetic datasets from now on, we need the diversity of synthetic data to progress further.