3 ms·
Awesome! Curious how you guys manage to make it "75% cheaper"?
by mrv_asura 2y ago
Awesome! Curious how you guys manage to make it "75% cheaper"?
- aikin-nivedit 2y agoBunch of optimisations for large data, including: Smart Batching, Prompt Caching, Custom Routing Algorithms and hardware planning for batch.