3 ms·
There is an obvious trend to scaling and performance by making models hierarchical ( not strictly but an element of local learning and global tuned connections
by DrScientist 3y ago
There is an obvious trend to scaling and performance by making models hierarchical ( not strictly but an element of local learning and global tuned connections ).
The obvious next step is to have specialised models loosely connected ( and trained [1] ) as a whole.
[1] We've had multiple models connected with text->concept and concept-> image etc
but I'm not sure if that connection is trained yet.
- seanhunter 3y agoThat’s sort of what happens in the “toolformer” paper[1] which sets out the basic architecture by which things like chatGPT plugins work. Language models can learn to use “tools” provided by plugins. Those tools might themselves bee the specialized models you are describing (although for some plugins they could be a non-AI tool such as web search or whatever). [1] https://arxiv.org/pdf/2302.04761.pdf https://arxiv.org/pdf/2302.04761.pdf