7 ms·
Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated m
by amazingamazing 2y ago
Smaller reinforced models are what will win. Look at our civilization, your company, heck even your team: lots of specialized folks, not some singular bloated mastermind.
The issue though is the overemphasis on the current benchmarks. Ideally the thing benchmarked is against user kpis.
- ericd 2y agoI’m not sure about that, I think the current setup is an artifact of human limits, which an immortal shared ML model doesn’t share. Also, cross-disciplinary knowledge is frequently incredibly helpful (if rare, because currently it’s the intersection of multiple uncommon specialties).
- noman-land 2y agoThe question that's screaming at me right now is that if humans used specialization + cooperation to achieve global dominance, what happens when you pair unlimited global masterminds + cooperation?
- jazzyjackson 2y agoCan't wait for the parliament of competing super intelligent political parties producing propaganda to convince cloud investors to reallocate compute.
- 6510 2y agoWe turn into cats.
- smnrg 2y agoI like optimists.
- UltraSane 2y agoBest case scenario is you get The Culture where society is ruled by hyper-intelligent but kind and benevolent AIs that solve all economic problems and that keep a population of quadrillions of very happy, healthy, and rich humans as pets.
- 6510 2y agoAs bandwidth is still crap my bet would be that, like with humans, the sum of the parts isn't greater than the greatest mastermind in the group or it will take an increasingly long time to come up with increasingly original thought. If it scales reasonably linear it would be worth waiting for a year or a decade to produce a single result. Also a funny thought, with enough of a mind it should be able to figure out how to get very limited storage and barely usable computation at very low speed out of some extremely cheap substrate. Something like growing salt crystals in a lake (or ocean) Have a write pump flashing a light onto salt water and a read pump discarding undesirable results to be overwritten. You know, like, it doesn't matter if you encode randomly, if you do it often enough you will get the entire set. You don't have to read accurately either, as long as you get at least some results out. wait, did I just explain the universe?
- evilduck 2y agoWhatever the human equivalent of beehives and ant colonies will be.
- nomel 2y ago> lots of specialized folks This isn't what I see in R&D. The broad and deep folks are the ones with all the patents, and design wins, because they're the only ones that can see outside the box that everyone else is specialized in.
- jazzyjackson 2y agoNah, it takes a mastermind to be creative. The tiny models IME are good at following explicit instructions for data transformations (conforming to a JSON schema is especially helpful) but I wouldn't expect one to solve a problem I haven't already solved myself like how I use the .6T full size models.
- EE84M3i 2y agoI thought conforming to a JSON schema was a solved problem for even the most garbage models by restricting the tokens it can generate?
- jazzyjackson 2y agoYes that's what I mean, with a well described task and a JSON schema, the tiny models are suitable (though still hallucinatory - even llama3 7B was making up phone numbers when I tasked it with extracting contact details from email signature blocks) I was pushing back against the parent's notion that tiny models would serve general purpose needs.