The ratio between the tokenizer and the model is constant for both training and inference. And I believe the ratio is that big - you just use a lot of compute

The ratio between the tokenizer and the model is constant for both training and inference. And I believe the ratio is that big - you just use a lot of compute for training, and that's why it's so expensive.