The core misunderstanding here is that the LLM is not “following instructions”, it’s just adding text onto an existing document. It’s generating a likely contin

The core misunderstanding here is that the LLM is not “following instructions”, it’s just adding text onto an existing document. It’s generating a likely continuation of the existing text. That text might look like a conversation or a tutorial or a poem, but it’s just a big blob of text that gets appended to by the LLM. If it looks like it’s “following instructions” that’s because it generated text that would likely follow a request in a transcript of a chat. There is no AI, it’s just text generation. Everything else that happens is due to code put in place by humans. Unfortunately, if it takes action based on probabilistically generated human language, there’s a lot of unpredictable ways that can go. SQL has a limited syntax. Human language does not.