Given an LLM, you can construct a perceptron-based model that approximates it, but that doesn't mean there's any obvious way of training a perceptron model that

Given an LLM, you can construct a perceptron-based model that approximates it, but that doesn't mean there's any obvious way of training a perceptron model that would produce something like an LLM. What the universal approximation theorem tells you is that there's some algorithm you could use to train a (sufficiently-large) perceptron model to behave arbitrarily-closely to the equivalently-trained LLM – which, yeah, of course there is. There's a way of training an array of floating point numbers to do that, too. It's called the transformer architecture. (Your misunderstandings are similar to the sorts of misunderstandings I had before I got in the habit of trying to (abstractly) implement every CS concept I came across. Kudos for trying to reason about AI from first principles, but you have to do a lot of reasoning before you stop being much wronger than the empiricists.)