This seems a bit reductionist. It is hard to cut through the hype and counter hype but it can makes sense to reason about the theoretical limits of a technology

This seems a bit reductionist. It is hard to cut through the hype and counter hype but it can makes sense to reason about the theoretical limits of a technology. For a toy example we have something like the halting problem which was proven undecidable decades ago and this is still true despite computers getting way better. We use theoretical bounds all the time. The term AI is nebulous enough such that new, unrelated, technologies can be invented and called AI but if you are talking about analyzing LLMs specifically you could probably figure out some meaningful theoretical bounds. This appears to be what these researchers are starting to try to do.