4 ms·
I don't agree with the transformer=transistor analogy. Nothing is to say we don't find a new model 10x better than transformers. That would be like perfecting v
by quickthrower2 3y ago
I don't agree with the transformer=transistor analogy. Nothing is to say we don't find a new model 10x better than transformers. That would be like perfecting very tiny and cheap transistors, before even making integrated circuits!
- jpmoral 3y agoI was very confused until I realised you weren't talking about electrical transformers.
- muzani 3y agoThe way I see it, GPT-3 is like someone with 70 IQ, while GPT-4 is like 120 IQ. It's not omnipotent, but having this block of human-level intelligence that you can just call through an API is world changing to say the least. What if they create something of effectively 200 IQ? Or 5000 IQ, where we just abstract IQ like horsepower? I would guess Amdahl's Law applies here. You get diminishing gains, to the point that a 10x better thing or 10 giga transistors on a mobile phone doesn't really make a difference. But to use the transistor analogy, the mates working on transistor or quantum theory didn't envision we'd have this world where gig economy influences elections or strangers arguing with each other at the speed of light using only their thumbs. So that's where Gustafson's Law kicks in. The 1x or 10x thing unlocks a new thing, which unlocks new levels of things all with their variations of speed, where it's a race to unlock the next next level thing. The current level thing is AI agents. What happens when you put them in a community? Will that unlock new things? There's probably a certain speed and cost restriction, then you have AI nanobot cities giving you the power to regenerate limbs.
- shinryuu 3y agoI think you misunderstand what IQ represents.