4 ms·

> The main cause here is that training AI models is extremely expensive so companies are cutting costs where they can so they can make these big investments wit

by __loam 3y ago

> The main cause here is that training AI models is extremely expensive so companies are cutting costs where they can so they can make these big investments without upsetting investors. I highly highly doubt this. Shareholders put pressure on big tech companies to do layoffs because they perceived that these companies had more people than they needed and interest rates rose. LLMs aren't that expensive to train for big organizations like Google and Meta that have the capital to train foundation models. Gpt-4 cost $100 million and Google makes 18b in profit a quarter and already has massive pools of computational capital. I doubt inference is a big expense for these orgs too but who knows. > Additionally, we should remember that software engineering isn't like traditional engineering in the sense that software doesn't require maintenance like a physical building or bridge would. This is false. > Similarly, SaaS companies that built out software products over the last decade are now enabling companies to build complex software products with much fewer engineers. A decade ago it was quite common for companies to have their own eCommerce products, for example. Today most will just use something like Shopify and hire a small team of engineers for the integrations and customisations. This is a good point I'm worried you're right about most of the building being done but I can't help but be hugely bitter about it. The software industry has been pretty good about expanding the franchise and giving employees some ownership over the systems they built. That these still hugely profitable systems will continue to exist but that the only people allowed to realize the value of them might be shareholders deeply disgusts me and makes me think if this gets worse, we should nationalize a lot of these firms.

kypro 3y ago
> LLMs aren't that expensive to train for big organizations like Google and Meta that have the capital to train foundation models. Gpt-4 cost $100 million and Google makes 18b in profit a quarter and already has massive pools of computational capital. The cost of training of the last generation of LLM models was ~$100 million, but companies like Amazon, Microsoft, Google and Meta still need to buy the GPUs to build out their data centres. Unlike big tech OpenAI just had to pay for the compute, not for the physical GPUs which have rocketed in price. Plus, next gen models are likely to cost even more to train and big tech companies are not just working on one or two of next gen models, but many of them. The cost is easily in the billions. https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-meta-is-spending-billions-on-nvidia-ai-chips.html https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-me... > That these still hugely profitable systems will continue to exist but that the only people allowed to realize the value of them might be shareholders deeply disgusts me and makes me think if this gets worse, we should nationalize a lot of these firms. If I had to guess we'll see regulators get increasingly aggressive with big tech companies. My guess is that they will continue to grow more profitable in the coming years, but also increasingly likely to be broken up. This isn't tech specific, but the future in general worries me in regards to equality. Many people naively think that the reason workers grew richer in the 20th century was because workers realised for the first time in history they could protest for worker's rights. Similarly people naively think the reason women entered the work force was because of similar protests. In reality technology made reduced the physical requirements of work, and each worker become far more productive through technology so companies could compensate them increasingly more. At the end of the day it only ever makes sense for a company to compensate you equal or less than your productivity as a worker, and prior to the 20th century worker productivity had little changed since the development of agriculture. Today technological advancements aren't making us that much more productive as workers, but it is making companies far more productivity through automation. I don't see this trend changing and it's only going to accelerate with AI. There really is no way to stop the rich growing richer, and trying to stop this by giving that power to the government is risky for other reasons.

by __loam 3y ago

kypro 3y ago

> LLMs aren't that expensive to train for big organizations like Google and Meta that have the capital to train foundation models. Gpt-4 cost $100 million and Google makes 18b in profit a quarter and already has massive pools of computational capital. The cost of training of the last generation of LLM models was ~$100 million, but companies like Amazon, Microsoft, Google and Meta still need to buy the GPUs to build out their data centres. Unlike big tech OpenAI just had to pay for the compute, not for the physical GPUs which have rocketed in price. Plus, next gen models are likely to cost even more to train and big tech companies are not just working on one or two of next gen models, but many of them. The cost is easily in the billions. https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-meta-is-spending-billions-on-nvidia-ai-chips.html https://www.cnbc.com/2024/01/18/mark-zuckerberg-indicates-me... > That these still hugely profitable systems will continue to exist but that the only people allowed to realize the value of them might be shareholders deeply disgusts me and makes me think if this gets worse, we should nationalize a lot of these firms. If I had to guess we'll see regulators get increasingly aggressive with big tech companies. My guess is that they will continue to grow more profitable in the coming years, but also increasingly likely to be broken up. This isn't tech specific, but the future in general worries me in regards to equality. Many people naively think that the reason workers grew richer in the 20th century was because workers realised for the first time in history they could protest for worker's rights. Similarly people naively think the reason women entered the work force was because of similar protests. In reality technology made reduced the physical requirements of work, and each worker become far more productive through technology so companies could compensate them increasingly more. At the end of the day it only ever makes sense for a company to compensate you equal or less than your productivity as a worker, and prior to the 20th century worker productivity had little changed since the development of agriculture. Today technological advancements aren't making us that much more productive as workers, but it is making companies far more productivity through automation. I don't see this trend changing and it's only going to accelerate with AI. There really is no way to stop the rich growing richer, and trying to stop this by giving that power to the government is risky for other reasons.