3 ms·
LLMs aren't trained solely on natural language. They also ingest formal notation from every domain and at every level (from preschool to PhD); they see code and
by TeMPOraL 2y ago
LLMs aren't trained solely on natural language. They also ingest formal notation from every domain and at every level (from preschool to PhD); they see code and markup in every language even remotely popular. They see various encodings, binary dumps, and nowadays also diagrams. The training data has all that's needed to teach them great many formal languages and how to use them.