I don't think there has been any exponential in terms on inference costs in the last couple of years. In fact, they have worsened as the same relevant hardware

I don't think there has been any exponential in terms on inference costs in the last couple of years. In fact, they have worsened as the same relevant hardware is more expensive and so is energy - and to top it off, to stay SotA companies are using larger models with higher running costs. But for some reason people are conflating the improvements in models with the cost of inference.