3 ms·
Feels like we are incredibly early into pricing LLM usage. I suspect we are going to look back in the days these scripts were useful with some rose tinted glass
by martinald 1y ago
Feels like we are incredibly early into pricing LLM usage. I suspect we are going to look back in the days these scripts were useful with some rose tinted glasses.
I assume they have very peaky demand, especially when Europe + N American office hours overlap (though I'm assuming b2b demand is higher than b2c). I'm also assuming Asian demand is significantly less than "the west", which I guess would be true given the reluctance to serve Chinese users (and Chinese users using locally hosted models?).
I know OpenAI and Anthropic have 'batch' pricing but that's slightly different as it's asynchronous and not well suited for a lot of code tasks. Think a more dynamic model for users would make a lot more sense - for example, a cheaper tier giving "Max" usage but you can only use it 8pm-6am Eastern time, otherwise you are on Pro limits.