I tried something like that some time ago. The problem with that strategy is the lack of backtracking. Let's say I prompt my LLM to exclusively use the letters

I tried something like that some time ago. The problem with that strategy is the lack of backtracking. Let's say I prompt my LLM to exclusively use the letters 'aefghilmnoprst' and the LLM generates "that's one small step for a man, one giant leap for man-"[1]. Since the next token with the highest probability ("-kind") isn't allowed, it may very well be that the next appropriate word is something really generic or, if your grammar is really restrictive, straight up nonsense because nothing fits. And then there's pathological stuff like "... one giant leap for man, one small step for a man, one giant leap for man- ...". [1] Toy example - I'm sure these specific rules are not super restrictive and "management" is right there.