3 ms·
“100% guaranteed downtime when you least can afford it and the support tickets are your problem.” We’ve a hybrid shop, including hosting our own ML infra, and
by bevekspldnw 2mo ago
“100% guaranteed downtime when you least can afford it and the support tickets are your problem.”
We’ve a hybrid shop, including hosting our own ML infra, and we save a ton from cloud spend with local ML. Easily one million USD over past three years. But it’s not “free”, you are shifting a lot of labor into your plate.
- hypfer 2mo agoAnd with that also gain institutional knowledge, skill up your workers and attract talent that wants to work on this stuff. All boils down to short-term/long-term thinking.
- gruturo 2mo agoThis. People WANT to work on this stuff. And having skilled workers is a precious advantage.
- bevekspldnw 2mo agoStill has to break even on the balance sheet, especially at a bootstrapped startup. We actually made most of the financial windfall in translation API fees oddly enough. For our own model training we needed to do some large scale translation tasks of a large dataset (1M or so documents, 10 or so target languages), running full-size NLLB on-prem saved us an absurd amount of money vs Google Translate API. (For reference doing 1M target docs into a single language in Google Translate API is roughly $120k list price. You can run full size NLLB on an 48GB NVIDIA A600 and the major difference for us was speed, but for this task time to completion wasn’t an issue.)