in go X tokens. Out comes a probability distribution across every possible next token. In your analogy, the bus could be any width. In practice people tend to

in go X tokens. Out comes a probability distribution across every possible next token. In your analogy, the bus could be any width. In practice people tend to trade "bus width" for heads, but it need not be that way. I'm not sure understand the last part. It's quite easy to try stuff with LLMs/transformers. The fact that a paper hasn't been written on every combination doesn't mean the haven't been tried in some way. It's not as though the architecture is the only thing.