3 ms·
In my head I picture these models like if you built a massive scaffold. Just boxes upon boxes, enough to fill a whole school gym, or even cover a football field
by jvm___ 4y ago
In my head I picture these models like if you built a massive scaffold. Just boxes upon boxes, enough to fill a whole school gym, or even cover a football field. Everything is bolted together.
You walk up to one side and Say "write me a poem on JVM". The signals race through the cube and your answer appears on the other side. You want to change something, go back and say another thing - new answer on the other side.
But it's all fixed together like metal scaffolding. The network doesn't change. Sure, it's massive and has a bajillion routes through it, but it's all fixed.
The next step is to make the grid flexible. It can mold and reshape itself based on inputs and output results. I think the challenge is to keep the whole thing together, while allowed it to shape-shift. Too much movement and your network looses parts of itself, or collapses altogether.
Just because we can build a complex, but fixed, scaffolding system, doesn't mean we can build one that adapts and stays together. Broken is a more likely outcome than AGI.
- yesenadam 4y ago> it's massive and has a bajillion routes through it, but it's not fixed. I think you meant to write "but it's fixed."
- jvm___ 4y agoThanks
- urthor 4y ago> The next step is to make the grid flexible. But... we can do exactly that??? Model retraining is very transparent and a very straightforward way to redirect it. The core problem is that the analogies just don't hold up. Artificial Intelligence is not a problem that can be resolved via analogies and heuristics. Because artificial intelligence simply looks nothing like what our human heuristics imagine it to be. It can only be solved by understanding the actual, genuine detail.
- jvm___ 4y agoIs there a 3d structure that you think is more analogous? From my understanding it's all matrix math, which to me is a grid. Maybe not a scaffolding one, and more like a bundle of interconnected wires so that each concept has multiple related concepts, but I think a rigid grid is easier to explain to lay people.