The underlying reason is closer to your second observation. Statistically chosen tokens are most likely to be reusable and composable. Much less to do with hard

The underlying reason is closer to your second observation. Statistically chosen tokens are most likely to be reusable and composable. Much less to do with hardware architecture. Token vocabulary happens before any quantization where processor word size may play a role.