I think a big part of the success of LLMs is they know what is good, and what would be absurd natural language. Most every domain will need that capability. Man

I think a big part of the success of LLMs is they know what is good, and what would be absurd natural language. Most every domain will need that capability. Many sentences can be interpreted multiple ways but once you can perceive which interpretations are non-sensical, the LLMs are on the right track to provide reasonable correct answers. Chomsky famously said "Colorless green ideas sleep furiously". An LLM can detect that such a sentence, such an interpretation, or translation, would make no sense --even if syntactically proper-- because its training materials never include such sentences (except perhaps this particular one). Then knowing it cannot be that "Colorless green ideas sleep furiously", it can prune out many otherwise possible interpretations. This is just my hunch, something like this may be going on. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously#:~:text=Colorless%20green%20ideas%20sleep%20furiously%20was%20composed%20by%20Noam%20Chomsky,%2Dformed%2C%20but%20semantically%20nonsensical https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu....