It is a somewhat though provoking question for me where you LLM's fall in automata theory? Are they non-deterministic finite automatons which would make regex'

It is a somewhat though provoking question for me where you LLM's fall in automata theory? Are they non-deterministic finite automatons which would make regex's a cousin? Or does it require pushdown automata levels of capability?