3 ms·
> LLMs aren't that expensive to train for big organizations like Google and Meta that have the capital to train foundation models. Gpt-4 cost $100 million and G
by kypro 3y ago
> LLMs aren't that expensive to train for big organizations like Google and Meta that have the capital to train foundation models. Gpt-4 cost $100 million and Google makes 18b in profit a quarter and already has massive pools of computational capital.
The cost of training of the last generation of LLM models was ~$100 million, but companies like Amazon, Microsoft, Google and Meta still need to buy the GPUs to build out their data centres. Unlike big tech OpenAI just had to pay for the compute, not for the physical GPUs which have rocketed in price. Plus, next gen models are likely to cost even more to train and big tech companies are not just working on one or two of next gen models, but many of them. The cost is easily in the billions.
https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-meta-is-spending-billions-on-nvidia-ai-chips.html https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-me...
> That these still hugely profitable systems will continue to exist but that the only people allowed to realize the value of them might be shareholders deeply disgusts me and makes me think if this gets worse, we should nationalize a lot of these firms.
If I had to guess we'll see regulators get increasingly aggressive with big tech companies. My guess is that they will continue to grow more profitable in the coming years, but also increasingly likely to be broken up.
This isn't tech specific, but the future in general worries me in regards to equality. Many people naively think that the reason workers grew richer in the 20th century was because workers realised for the first time in history they could protest for worker's rights. Similarly people naively think the reason women entered the work force was because of similar protests.
In reality technology made reduced the physical requirements of work, and each worker become far more productive through technology so companies could compensate them increasingly more. At the end of the day it only ever makes sense for a company to compensate you equal or less than your productivity as a worker, and prior to the 20th century worker productivity had little changed since the development of agriculture.
Today technological advancements aren't making us that much more productive as workers, but it is making companies far more productivity through automation. I don't see this trend changing and it's only going to accelerate with AI. There really is no way to stop the rich growing richer, and trying to stop this by giving that power to the government is risky for other reasons.