It is only difficult for a LLM to sort a list of numbers if the list is longer than half of the context window. (Source: I tested this myself[1]). The sorts are

It is only difficult for a LLM to sort a list of numbers if the list is longer than half of the context window. (Source: I tested this myself[1]). The sorts are not error-free every time, but with sufficient training they become error-free the vast majority of the time, even for long lists. This is not especially surprising because transformers are capable of directly representing sorting programs.[2] [1] https://jbconsulting.substack.com/p/its-not-just-statistics-gpt-4-does https://jbconsulting.substack.com/p/its-not-just-statistics-... [2] https://arxiv.org/abs/2106.06981 https://arxiv.org/abs/2106.06981