3 ms·How fast are coldstarts, and how do you compare against other gpu providers (runpod modal etc)by Nevin1901 3y agoHow fast are coldstarts, and how do you compare against other gpu providers (runpod modal etc)xena 3y agoThe slowest part is loading weights into vram in my experience. I haven't done benchmarking on that. What kind of benchmark would you like to see?ipsum2 3y agoI would like to see time to first inference for typical models (llama-7b first token, SDXL 1 step, etc)