3 ms·
This is the best setup I've seen so far, only problem is if you want to integrate it into your product you'll need a cloud GPU which can be quite expensive.
by cranberryturkey 2y ago
This is the best setup I've seen so far, only problem is if you want to integrate it into your product you'll need a cloud GPU which can be quite expensive.
- BoredPositron 2y agoYou can use serverless endpoints on runpod or similar services the delay is a bit high for the first inference but for concurrent requests it's fine with a reasonable keep alive.