3 ms·
How do you define 'small amount of capital'? Assuming you have to buy the processing from one of the big 3 (likely not possible to own the hardware outright), y
by briHass 4y ago
How do you define 'small amount of capital'? Assuming you have to buy the processing from one of the big 3 (likely not possible to own the hardware outright), you're looking at 10s of millions of dollars per training run.
It's certainly not outside reality for the well-known software companies to have divisions working on it, but I don't think we're quite at the point where garage startups can afford the compute.
- smoldesu 4y ago> likely not possible to own the hardware outright Why not? The majority of these models are running on fairly ubiquitous datacenter GPUs. They're being trained on highly available hardware. You're right that there's a compute barrier, but you get around that by throwing money at the problem. If someone was sufficiently motivated, they could probably dethrone OpenAI in 2 months with enough funding.
- jonnyreiss 4y ago> If someone was sufficiently motivated, they could probably dethrone OpenAI in 2 months with enough funding. I'd argue Google is sufficiently motivated to dethrone OpenAI --- so why haven't they? They have plenty of money to throw at the problem as well.