5 ms·

It's not sustainable. Copilot+ runs locally, but has the quality of 2 years ago, it's basically useless for anything but shitposts and spam. The cloud AI tool

It's not sustainable. Copilot+ runs locally, but has the quality of 2 years ago, it's basically useless for anything but shitposts and spam. The cloud AI tools all burn hideous amounts of money, all ran at a loss. AGI is a red herring, and will not happen. The architecture of generative-AI systems simply doesn't permit the required logic and reasoning capability. Even the "Actually, Indians" concept of outsourcing the tertiary sector to the developing world by way of having low-skill workers clean up AI generated trash is unviable. (It both doesn't work, and is politically doomed.) What's going on here is that tech companies are tearing up everything to pump their stock prices after the covid-tech-boom and ZIRP ended. Burn down their core products to keep the bubble going just a bit longer.

fnordpiglet 2y ago
I’m not sure why you think cloud AI runs at a loss. This is simply not true. The cloud providers are public companies and they’re definitely not losing money in AI. The bigger AI companies like OpenAI and Anthropic are not losing money in their monetized APIs. Nvidia isn’t selling GPUs at a loss. Every place I’ve worked is absolutely reducing costs and doing new / more business as a result of their LLM use. So I’m not entirely sure my first hand experience and the economics in actual reality jibe with your opinion.
- victor106 2y ago
  I agree Nvidia is for sure making money. For the other companies in your list and more specifically the ones that are developing LLM's they are investing billions of dollars in the hope that they will make all of those back and then some in the future. Meta for example said they invested $10bn into AI just last quarter alone.
  - fnordpiglet 2y ago
    Note I didn’t discuss the fully loaded costs I discussed the inference costs. At this point I see training costs as basic research and R&D, partially and not entirely oriented at -making training scale- and cost optimization for next generations of foundational and fine tuned models. For example Falcon is literally basic research funding. That’s relevant because if they stopped with say Claude Opus and 4o and did no more training they would be handsomely profitable indefinitely because the product is that useful. However it’s an arms race and the limit of effectiveness hasn’t been reached as training costs fall dramatically. So it’s not the right time to stop because whoever stops first loses everything to whoever doesn’t stop. More they’re feeding off each other in a virtuous cycle. Once diminishing returns kill the race whoever is left in the race will have a handsome business indefinitely as the moat to build such a return diminished model is probably enormous. But if inference optimizations keep going as they’re going it’ll be crazy cheap to operate. The second layer market of tools that constrain, optimize, and effectively apply the models in effective ways will be the first to really turn a profit. Many hype wagon AI companies are already being snapped up by larger companies to bootstrap their internal work.
    - victor106 2y ago
      good points, thank you
- ADeerAppeared 2y ago
  The people selling the proverbial shovels are turning a profit. > The bigger AI companies like OpenAI and Anthropic are not losing money in their monetized APIs. [CITATION NEEDED] Inference is more expensive than claimed, used extensively as a 'slot machine' with users trained to just keep re-generating until they get something useful, and only keeps getting more expensive as model quality has to go up. And in practice, training the model is far less one-off than claimed. Current tools are not sufficient. > Every place I’ve worked is absolutely reducing costs and doing new / more business as a result of their LLM use. Unless you are working in SEO, Marketing, or spam, I don't believe you. LLMs aren't reliable enough to replace actual human labour. While it's true many companies are fooling themselves into believing they're reducing costs, in practice other staff is picking up the slack. This is unsustainable unless your company has massively overhired. Things like "AI generated software tests" are a farce. The consequences aren't immediate, but will show up long term.
  - fnordpiglet 2y ago
    I work closely with both companies quite a lot so you can either take my word for it or not - it doesn’t bother me either way to be honest. They are pricing inference to make money. I don’t feel like you really have much experience using LLMs in business. However an example of where they’re very powerful is in summarization. For instance we have a pretty complex customer support model for our fraud and other cases with various disparate data sets including prior cases, related possible fraudsters identified via our fraud models, etc. We built a copilot LLM multi agent system that has access to various functions as sub agents that are prompted and context aware of how to summarize their specified data sets. They also have the ability to render widgets on demand or if their context implies it’s relevant. This allows quite a lot of complex high cognitive load information to be distilled rapidly and the investigators to interrogate the copilot on a case. As the copilot develops “answers” as a summary it dynamically renders an appropriate contextual dashboard with the relevant visualization. By structuring the application as a multi agent model we can constrain the LLM to pretty well specified tasks with fine tunings and very specific contexts for their specific task. This almost entirely eliminates hallucination and forgetfulness. Even if it were to do so the actual ground truth is visualized for the investigator. Prior systems either dumped massive amounts of cognitive load in the investigators face or took man years of effort to create a specific workflow, and in an adversarial dynamic space like fraud you need a much more dynamic approach to different types of new attacks. We aren’t replacing anyone. That’s not our goal. In fact we grew our investigator footprint because both our precision and recall have grown dramatically making our losses much less. We hire more skilled investigators and greater number to address more suspected cases faster and better. Listen. When John Henry battled the steam drill he did win, but it killed him. Go to any modern bore site and you won’t see less people working on the tunnel but more people - people who aren’t there for their strong back and ability to swing a pick but because they’re highly trained experts. They’re just building more complex tunnels that don’t collapse and don’t lose dozens of workers per dig. This form of automation is no different in my experience so far. So, if all you can see is SEO and grift, it might be a lack of imagination and experience on your part and some magical AI thinking sprinkled in. All your points about LLMs failures are true but they also all have solutions that don’t require slot machines as you say or imply it’s all a scam. They’re a tool like any other and they require handling in specific ways to be most effective. Even if chatgpt is a pretty unconstrained interface and that leads to issues doesn’t mean that’s the only way to use the tech. Use of LLMs to generate software is dumb. Although a protio, LLMs are actually pretty remarkable at generating Cucumber tests as Gherkin is a natural language grammar that plays into their native strength better than producing computer language grammars. This is useful if say you have business people or whatever writing effectiveness or whatever testing where they can provide a specification of policy and a well prompted LLM can generate pretty exhaustive cucumber tests (which can be pretty redundant and formulaic when asserting positive and negative cases exhaustively) which can then be revised by hand as needed. Since they’re natural language as well the business people tend to be pretty good at debugging the tests up front and with a large set of cucumber tests written by hand you’ll see tons of errors anyways. The LLM tests tend to be much much higher quality than the human written ones.
brcmthrowaway 2y ago
What would be the stock market play here
galdosdi 2y ago
If so, what happens when the music stops and they run out of furniture to burn for heat? Is the job market going to explode again because suddenly they desperately need IT professionals to rebuild the mess they've made? Is it going to tank further because so much capital has been destroyed that there's less to work on?
raincole 2y ago
> The cloud AI tools all burn hideous amounts of money, all ran at a loss. Press X to doubt. I highly doubt if ChatGPT API is losing money. Yes, I've read claims saying so. No, I haven't seen a credible one. And it's getting cheaper and cheaper, currently even faster than Moore's law (gpt-4o is 6x cheaper than gpt-4).
tymscar 2y ago
I agree with everything you’ve said, and to add to this why they do this without caring about the company future is because in 99.9% of the times the ones taking decisions aren’t founders who have a soft spot for the company. Those are long retired in the Bahamas. The leaders come and go quite frequently and the good ones that have been here for a while are ready to retire which means the bigger the boom is before they leave the better, with 0 regard to what comes after them.
CaptainFever 2y ago
The simple answer is that AI Paint runs locally. That's it. It's not some kind of grand conspiracy by the "elites".

fnordpiglet 2y ago

I’m not sure why you think cloud AI runs at a loss. This is simply not true. The cloud providers are public companies and they’re definitely not losing money in AI. The bigger AI companies like OpenAI and Anthropic are not losing money in their monetized APIs. Nvidia isn’t selling GPUs at a loss. Every place I’ve worked is absolutely reducing costs and doing new / more business as a result of their LLM use. So I’m not entirely sure my first hand experience and the economics in actual reality jibe with your opinion.

victor106 2y ago
I agree Nvidia is for sure making money. For the other companies in your list and more specifically the ones that are developing LLM's they are investing billions of dollars in the hope that they will make all of those back and then some in the future. Meta for example said they invested $10bn into AI just last quarter alone.
- fnordpiglet 2y ago
  Note I didn’t discuss the fully loaded costs I discussed the inference costs. At this point I see training costs as basic research and R&D, partially and not entirely oriented at -making training scale- and cost optimization for next generations of foundational and fine tuned models. For example Falcon is literally basic research funding. That’s relevant because if they stopped with say Claude Opus and 4o and did no more training they would be handsomely profitable indefinitely because the product is that useful. However it’s an arms race and the limit of effectiveness hasn’t been reached as training costs fall dramatically. So it’s not the right time to stop because whoever stops first loses everything to whoever doesn’t stop. More they’re feeding off each other in a virtuous cycle. Once diminishing returns kill the race whoever is left in the race will have a handsome business indefinitely as the moat to build such a return diminished model is probably enormous. But if inference optimizations keep going as they’re going it’ll be crazy cheap to operate. The second layer market of tools that constrain, optimize, and effectively apply the models in effective ways will be the first to really turn a profit. Many hype wagon AI companies are already being snapped up by larger companies to bootstrap their internal work.
  - victor106 2y ago
    good points, thank you
ADeerAppeared 2y ago
The people selling the proverbial shovels are turning a profit. > The bigger AI companies like OpenAI and Anthropic are not losing money in their monetized APIs. [CITATION NEEDED] Inference is more expensive than claimed, used extensively as a 'slot machine' with users trained to just keep re-generating until they get something useful, and only keeps getting more expensive as model quality has to go up. And in practice, training the model is far less one-off than claimed. Current tools are not sufficient. > Every place I’ve worked is absolutely reducing costs and doing new / more business as a result of their LLM use. Unless you are working in SEO, Marketing, or spam, I don't believe you. LLMs aren't reliable enough to replace actual human labour. While it's true many companies are fooling themselves into believing they're reducing costs, in practice other staff is picking up the slack. This is unsustainable unless your company has massively overhired. Things like "AI generated software tests" are a farce. The consequences aren't immediate, but will show up long term.
- fnordpiglet 2y ago
  I work closely with both companies quite a lot so you can either take my word for it or not - it doesn’t bother me either way to be honest. They are pricing inference to make money. I don’t feel like you really have much experience using LLMs in business. However an example of where they’re very powerful is in summarization. For instance we have a pretty complex customer support model for our fraud and other cases with various disparate data sets including prior cases, related possible fraudsters identified via our fraud models, etc. We built a copilot LLM multi agent system that has access to various functions as sub agents that are prompted and context aware of how to summarize their specified data sets. They also have the ability to render widgets on demand or if their context implies it’s relevant. This allows quite a lot of complex high cognitive load information to be distilled rapidly and the investigators to interrogate the copilot on a case. As the copilot develops “answers” as a summary it dynamically renders an appropriate contextual dashboard with the relevant visualization. By structuring the application as a multi agent model we can constrain the LLM to pretty well specified tasks with fine tunings and very specific contexts for their specific task. This almost entirely eliminates hallucination and forgetfulness. Even if it were to do so the actual ground truth is visualized for the investigator. Prior systems either dumped massive amounts of cognitive load in the investigators face or took man years of effort to create a specific workflow, and in an adversarial dynamic space like fraud you need a much more dynamic approach to different types of new attacks. We aren’t replacing anyone. That’s not our goal. In fact we grew our investigator footprint because both our precision and recall have grown dramatically making our losses much less. We hire more skilled investigators and greater number to address more suspected cases faster and better. Listen. When John Henry battled the steam drill he did win, but it killed him. Go to any modern bore site and you won’t see less people working on the tunnel but more people - people who aren’t there for their strong back and ability to swing a pick but because they’re highly trained experts. They’re just building more complex tunnels that don’t collapse and don’t lose dozens of workers per dig. This form of automation is no different in my experience so far. So, if all you can see is SEO and grift, it might be a lack of imagination and experience on your part and some magical AI thinking sprinkled in. All your points about LLMs failures are true but they also all have solutions that don’t require slot machines as you say or imply it’s all a scam. They’re a tool like any other and they require handling in specific ways to be most effective. Even if chatgpt is a pretty unconstrained interface and that leads to issues doesn’t mean that’s the only way to use the tech. Use of LLMs to generate software is dumb. Although a protio, LLMs are actually pretty remarkable at generating Cucumber tests as Gherkin is a natural language grammar that plays into their native strength better than producing computer language grammars. This is useful if say you have business people or whatever writing effectiveness or whatever testing where they can provide a specification of policy and a well prompted LLM can generate pretty exhaustive cucumber tests (which can be pretty redundant and formulaic when asserting positive and negative cases exhaustively) which can then be revised by hand as needed. Since they’re natural language as well the business people tend to be pretty good at debugging the tests up front and with a large set of cucumber tests written by hand you’ll see tons of errors anyways. The LLM tests tend to be much much higher quality than the human written ones.

brcmthrowaway 2y ago

What would be the stock market play here

galdosdi 2y ago

If so, what happens when the music stops and they run out of furniture to burn for heat? Is the job market going to explode again because suddenly they desperately need IT professionals to rebuild the mess they've made? Is it going to tank further because so much capital has been destroyed that there's less to work on?

raincole 2y ago

> The cloud AI tools all burn hideous amounts of money, all ran at a loss. Press X to doubt. I highly doubt if ChatGPT API is losing money. Yes, I've read claims saying so. No, I haven't seen a credible one. And it's getting cheaper and cheaper, currently even faster than Moore's law (gpt-4o is 6x cheaper than gpt-4).

tymscar 2y ago

I agree with everything you’ve said, and to add to this why they do this without caring about the company future is because in 99.9% of the times the ones taking decisions aren’t founders who have a soft spot for the company. Those are long retired in the Bahamas. The leaders come and go quite frequently and the good ones that have been here for a while are ready to retire which means the bigger the boom is before they leave the better, with 0 regard to what comes after them.

CaptainFever 2y ago

The simple answer is that AI Paint runs locally. That's it. It's not some kind of grand conspiracy by the "elites".