4 ms·

Someone might host DeepSeek for you but you'll pay through the nose for it and it'll be frozen in time because the training cost doesn't have the revenue to kee

• wongarsu 1y ago
  There are already inference providers like DeepInfra or inference.net whose entire business model is hosted inference of open-source models. They promise not to keep or use any of the data and their business model has no scaling effects, so I assume they are already charging a fair market rate where the price covers the costs and returns a profit. The GPU market will probably take a hit. But the flip side of that is that the market will be flooded with second-hand enterprise-grade GPUs. And if Nvidia needs sales from consumer GPUs again we might see more attractive prices and configurations there too. In the short term a market shock might be great for hobby-scale inference, and maybe even training (at the 7B scale). In the long term it will hurt, but if all else fails we still have AMD who are somehow barely invested in this AI boom