2 ms·
Interested in knowing how you are running BERT model with $35/month? Cheapest GPU instance costs $200-300/month AFAIK.
by YetAnotherNick 3y ago
Interested in knowing how you are running BERT model with $35/month? Cheapest GPU instance costs $200-300/month AFAIK.
- aaronvg 3y ago(Other author of this blog post here) We actually do CPU inference. The SBERT models have a pretty small memory footprint -- you can fit a couple models on a t2.medium instance. On a C6.Large you can get 75ms inference. T2.medium is more around 100-200ms