3 ms·
Hard to overstate how exciting I find the fields of hypernetworks and metalearning. For good or for evil, this I feel is almost certainly how we create AGI. GNN
by mkaic 5y ago
Hard to overstate how exciting I find the fields of hypernetworks and metalearning. For good or for evil, this I feel is almost certainly how we create AGI. GNNs make perfect sense as a method of accomplishing it too -- after all, neural networks are just graphs anyways.
I don't believe AGI is going to conform to our rigid, symmetrical, boxy human-made architectures with their nice, neat flow patterns and powers-of-two factors everywhere -- those architectures work great for what they're designed to do, but achieving sentience/true intelligence just really doesn't feel like it's going to come out of a feedforward MLP, or even a transformer (in my opinion).
The most complicated and capable neural networks we know of are the organic, messy, squishy, yet organized electric meat lumps in our own skulls, produced by billions of years of being exposed to the same loss function over and over and over, but with unlimited ways to adapt in order to minimize that loss function. Nature makes awesome, fractally, insanely complex networks that we could never design by hand. I think it's only sensible to assume that AGI will work similarly.
Of course, this is all just armchair speculation. I'm not an expert, just an enthusiastic data scientist who thinks these next twenty years in AI are going to be a heck of a freaking ride.
- drdeca 5y agoIf we do achieve AGI through a setup which is not only neural network based, but where even the architecture by which the nodes are connected is beyond our understanding, then, that sounds very bad for interpretability and therefore for safety.
- mkaic 5y agoSure. I'm all for safety, but I also think that unsafe AGI, like the atom bomb, is an inevitability. It's not a matter of if, it's a matter of when we invent it and release it. The future is already murky but it's about to get a heck of a lot more so -- sentient AIs could be the thing that exalts humanity or the thing that destroys us. Break out the popcorn, because I'd wager we're going to see one of those outcomes in the next century.
- riskneutral 5y agoIf we achieve AGI, then the AGI should be able to produce a formal verification proof of its own safety, similar to how humans use produce formal verification proofs for critical software using automated theorem provers today.
- Icko 5y agoCan you provide formal verification proof of your own safety?
- drdeca 5y agoWell, to have a formal proof of its own safety, we first would need a correct formal definition of what it means for it to be safe (or, rather, a formal definition of a condition which is truly a sufficient condition for it to be safe. It needn't be a necessary condition.) But, I suppose arriving at such a formal definition doesn't really require that the thing being described be particularly amenable to our understanding. However, I'm not sure that "if we ran it, if it is safe, it could prove itself to be safe" is sufficient to address the concerns. If it isn't safe, then running at all may spell doom, so, "if it is safe, it could demonstrate that after we turn it on" doesn't really address that, because it doesn't give us any assurance before we turn it on. Now, if we started with something with substantially sub-human overall intelligence, and which wasn't really agent-y (so that, if it was a little unsafe, it would at least not be catastrophically unsafe), but which was more equipped to formally prove things about itself and potential modifications of itself than humans are equipped to formally prove things about it, then we could task that thing with formally proving safety properties about itself, and of also proving safety properties about its successor, and do that before running its successor, and iterate this process to produce increasingly intelligent programs and perhaps also eventually agentic ones, while always having a safety proof of each before we run it.. But, I'm not really sure how plausible this route is? Like, even assuming we do reach AGI, safe or unsafe, I'm not sure this is a plausible route of getting there safely.
- yusefnapora 5y agoWell, we don't have anything like a coherent understanding of the architecture of biological general intelligence either, so at least this won't be a particularly new state of affairs.