3 ms·
Ant Group – scaling to 1.37M QPS on Ray
- george_123 4y agoThis is a guest engineering blog post from Ray contributors at Ant Group, discussing how Ant Group implemented scalable Ray Serving architecture atop Ray, deploying 240,000 cores for model serving, scaling by 3.5x from previous year, and reaching 1.37 million TPS during peak times. (resubmission, URL redirected to /Engineering last time)