3 ms·
I don't think parent meant training specifically on that test data, but rather in that kind of task. Think more "LLM trained only on images of art" doing badly
by viraptor 3y ago
I don't think parent meant training specifically on that test data, but rather in that kind of task. Think more "LLM trained only on images of art" doing badly on this test, but then "LLM trained on images of art and IQ puzzles" doing better on this test.
It's kind of like asking what even is IQ if you can learn how to solve Mensa puzzles and improve your score. Does it mean you're more intelligent?
- firtoz 3y agoOh, I see. One guess for GPT4 is that it's an amalgamation of models. So you would have various models trained more specifically like the ones you mentioned, and you ask them all to start answering the query, then you choose which one produces better results, and then you present that to the user. Alternatively you can have a decider model that knows what kind of queries goes to what specialised one, then have that as a "hidden layer", whether it be in the application level or "neuron layers".
- broast 3y agoIf you've checked out the docs for the assistant api, you can intuit that there is a higher level system which decides which subsystems to use to respond. The assistant determines whether to use a tool (vision, code interpreter, search, retrieval), as well as which code or text language model to use to generate the response.
- pixl97 3y agoYour last statement reminds me of the folktale of John Henry. That of generalized capability versus that of specialization. Under a narrow range of tests specialization is pretty much guaranteed to win. Specialization in all cases I know of comes at a cost to general capability. It's like one of those qips "Pick any two: fast, cheap, good", the more you pull at one category, the costs rase in the other categories.