25 ms·
I’ve been wondering about variations on panel of experts designs that use a “mob” of small models or possibly a network of them. Sort of like crowdsourcing with
by api 3y ago
I’ve been wondering about variations on panel of experts designs that use a “mob” of small models or possibly a network of them. Sort of like crowdsourcing with baby AIs.
Could we have libraries of small models that are all experts on niche subjects that basically collaborate?
The advantage is that it’s easier to scale and distribute lots of small models on commodity hardware than it is to run giant models that require heaps of contiguous ram.
I’d love to have some time to play with versions of this.
- nickpsecurity 3y agoThe best design we know is how God made humans. Our brains have specialized parts that work together with analog components. We’re raised on knowledge of increasing complexity as humans, given general-purpose heuristics, and then we specialize. From there, small teams of specialists interacting typically do the best jobs. So, I’d try to do stuff like that first since it’s proven in the only general intelligence that exists. If I do experiments, I’m making models specialized from foundational to programming to specific language to specific tasks. Each pass will have training data so it picks up general concepts. Then, it will be trained to focus more on what it needs. Eventually, what’s slushing around in its neurons should be just the right information. Or it won’t work. Anyway, my concepts that I published are at this link: https://gethisword.com/tech/exploringai/index.html https://gethisword.com/tech/exploringai/index.html (See alternative models section.)