I mean...to some extent, that trope is real. We can feed a prompt to any of the current ML-based generators (whether text, image, or whatever), and the details

I mean...to some extent, that trope is real. We can feed a prompt to any of the current ML-based generators (whether text, image, or whatever), and the details of how it produces the particular output it does are entirely obscured by the black-box functioning of the neural net and its trained model. Like, you can describe in general How A Neural Network Functions to produce output from input, but you can't explain why, say, asking Stable Diffusion for a picture of the sun rising over marshmallow peeps produced an image with just a few weird-looking marshmallows, but no sun (actual example from my own fiddling around with it). But..."We can't explain exactly how it gets from point A to point B" is not the same thing as "it's self-aware". Not in the slightest. And you're absolutely right that the current ML-based approaches are either in completely the wrong direction, or several orders of magnitude too simple, to even be worth talking about in the same sentence as what most laypeople mean when they talk about "AI".