4 ms·
The main issue is the models you need to do the job are too big for most consumers, even people with nice video cards. You'll need a couple L40S GPUs at minimum
by teaearlgraycold 2y ago
The main issue is the models you need to do the job are too big for most consumers, even people with nice video cards. You'll need a couple L40S GPUs at minimum. Maybe a few H100s more realistically.
So do you want to buy tens of thousands of dollars in GPUs or do you want to rent them second-by-second? Most people will choose the latter. I understand you don't trust the infrastructure and that's reasonable. If self-hosting was viable it would be more popular.