Here's one way it could happen: Let's say there's some circuit that does problem solving of the kind we call intelligence. We dont know what this circuit look

Here's one way it could happen: Let's say there's some circuit that does problem solving of the kind we call intelligence. We dont know what this circuit looks like, but it exists in our brain. Doing regression on outputs from the brain (e.g. internet text) with enough parameters, we can "fit" our model to this circuit. But if you try to fit it with fewer parameters than it needs, you're just going to get some linear approximation.