You could, and it would be an interesting challenge to come up with the sorting which makes the data most compressible. (The question of how to sort data for op

You could, and it would be an interesting challenge to come up with the sorting which makes the data most compressible. (The question of how to sort data for optimum compressibility is something I've messed around with a bit: https://www.gwern.net/Archiving-URLs#sort---key-compression-trick https://www.gwern.net/Archiving-URLs#sort---key-compression-... An interesting tool designed just for this is 'binsort': http://neoscientists.org/~tmueller/binsort/ http://neoscientists.org/~tmueller/binsort/ ) But it would also be completely different from (and easier than) the existing benchmark of wikitext, which is what everyone uses and judges NN natural language modeling progress by, so there wouldn't be too much research interest in it, and it's not clear how useful it would be. After all, AIs don't get to reorganize the entire universe to make inputs come in the most convenient order for compression.