3 ms·
Not strictly related to their study, but if we consider Solomonoff induction to be the most general model of all (albeit not computable), then I’d say yes to th
by Xcelerate 3y ago
Not strictly related to their study, but if we consider Solomonoff induction to be the most general model of all (albeit not computable), then I’d say yes to the question, if only because it will select the most specialized model(s) for the particular problem at hand automatically.
One could argue universal general intelligence is simply the ability to optimally specialize as necessary.
I think one aspect that is overlooked when people are involved is that we create specialized or fine-tuned models precisely for the reason that a general approach didn’t work well enough. If it had, we would have stopped there. So there’s a selection bias in that almost all fine-tuned models are initially better than general models, at least until the general models catch up.
- doingtheiroming 3y agoThere are lots of ways to define “good enough” as well. What are the costs of inferring several small experts contributing to a decomposed workflow versus using GPT4 for example. If you want to run multiple instances for different teams or departments, how do the costs escalate. Do you want to include patient data and have concerns about using a closed source model to do so. Etc. There’s little doubt that GPT4 is going to be most capable at most tasks either OotB or with prompt engineering as here. But that doesn’t mean it’s the right approach to use now.
- cyanydeez 3y agoI assume the next step is to find the right model size for semantic specialism then create a model of model that's filtering down to specialized models. same will happen for temporal information that stratifies and changes, ie, asking what a 1950s scientist understands about aquantim physics.
- VirusNewbie 3y agoI don't understand the reasoning. if you merged two of the most specialized models together and simply doubled the size/parameter count, there is no reason the smaller one can generalize better than the larger one. It would simply have two different submodels inside its ubermodel.
- Davidzheng 3y agoYour argument has a very concrete example. We are a generalist intelligence. And when we encountered chess, we eventually developed specialist intelligence to be the best at chess.
- marvin 3y agoGeneral intelligence builds tools as required, maybe having a whole bunch of them built-in already. Maybe the ones that are most frequently useful. Maybe a more general intelligence can build more of these tools into itself as required. Interesting philosophic topic how to ensure objectives aren't affected too much. You can make this mind experiment even as a human. There are parts of our goal system we might want to adjust, and parts we'd be horrified to, being very reluctant to accidentally risk affecting it.
- quickthrower2 3y agoThis is another level though. GPT4 wont do that. But maybe whatever is coming next can.