No, you can get away with under $20k these days. The models can now distribute adequately well, so you can get away with a swarm of thirty two 12GB 3060 Ti card

No, you can get away with under $20k these days. The models can now distribute adequately well, so you can get away with a swarm of thirty two 12GB 3060 Ti cards. 32*$400 = $12,800. Add in computers to stick them in and interlink, and that's sort of where you land. When you needed this in one machine, the cost to build a box capable of running OPT was around $50-$150k. An 80GB GPU runs around $10k-$20k, and you need five of them. So you can get away with a good bit less than an NVidia DGX Station. Unless you're a FAANG SWE of a high level of some kind, it's well outside of hobbyist use, but it's not an unreasonable expense for most business use. The way I think of this expense is it's 10% of the annual cost of a SWE. If it improves productivity by more than 10%, it's worth it for a one-person team. With a 10-person team, it's worth it at a 1% improvement.