3 ms·
Awesome! Curious how you guys manage to make it "75% cheaper"?
by mrv_asura 1y ago
Awesome! Curious how you guys manage to make it "75% cheaper"?
- aikin-nivedit 1y agoBunch of optimisations for large data, including: Smart Batching, Prompt Caching, Custom Routing Algorithms and hardware planning for batch.