3 ms·
Isn’t this an arbitrage opportunity? Offer to pay a fraction of the cost per token but accept that your tokens will only be processed when the batch window isn’
by philipodonnell 1y ago
Isn’t this an arbitrage opportunity? Offer to pay a fraction of the cost per token but accept that your tokens will only be processed when the batch window isn’t big enough, then resell that for a markup to people who need non-time sensitive inference?
- pama 1y agoYou may have already noticed that many providers have separate, much lower, prices for offline inference.