The core reason is that nobody understands how these models work, and hence what they are/will be capable of. It's a little unnerving to think about, since the

The core reason is that nobody understands how these models work, and hence what they are/will be capable of. It's a little unnerving to think about, since the approach is pretty much "Lets build the framework, plant the seeds, and then see what comes out the other side". If AGI is achieved, it will almost certainly be a surprise rather than be intentional.