4 ms·
Try doing it at scale for a whole office. Not trivial.
by akie 4mo ago
Try doing it at scale for a whole office. Not trivial.
- ReptileMan 4mo agoYou could probably do with couple of instances. People rarely use ai 24/7, so right now you can oversubscribe and still have acceptable latency and high utilization rate.
- arjunchint 4mo agoThere are plenty of US based hosters racing to optimize and drive efficiencies Literal race on twitter posting to increase token throughput and drive down costs on these Chinese open source models