3 ms·

Pretty bad decision on his part. I've been telling other engineers within my company who felt threatened by AI that this would happen. That prices would rise an

Pretty bad decision on his part. I've been telling other engineers within my company who felt threatened by AI that this would happen. That prices would rise and the marginal cost for changes to big codebases would start to exceed the cost of an engineer's salary. API credits are expensive, especially for huge contexts, and sometimes the model will use $200 in credits trying to solve a problem that could be fixed in an hour by a good engineer with enough context. It kind of reminds me of the joke where a plumber charges $500 for a 5 minute visit. When the client complains the plumber says it's $50 for labor and $450 for knowing how to fix the problem.

christoph 6mo ago
A good lesson for all - I always really liked the Picasso version: In a bustling restaurant, an excited patron recognized the famous artist Picasso dining alone. Seizing the moment, the patron approached Picasso with a simple request. With a plain napkin and a big smile, he asked the artist for a drawing. He promised payment for his troubles. Picasso, ever the creator, didn’t hesitate. From his pocket, he produced a charcoal pencil and he brought to life a stunning sketch of a goat on the napkin—a clear mark of his unique style. Proudly, he presented it to the patron. The artwork mesmerized the patron, who reached out to take it, only to be stopped by Picasso’s firm hand. “That will be $100,000,” Picasso declared. Astonished, the patron balked at the sum. “But it took you just a few seconds to draw this!” With a calm demeanor, Picasso took back the napkin, crumpled it, and tucked it away into his pocket, replying, “No, it has taken me a lifetime.”
- airstrike 6mo ago
  [flagged]
  - wat10000 6mo ago
    It actually matches up well with the current AI scene, except backwards. We use these model which cost ridiculous amounts of money to train, and all of that effort goes into producing the outputs we use, but we're paying something not too far above the marginal cost of inference when we use them.
    - airstrike 6mo ago
      So not applicable at all
      - wat10000 6mo ago
        Extremely applicable to illustrate the difference between people (time is precious, training and experience amortize across a relatively small amount of paid work) and software (can replicate infinitely, time is cheap, startup costs can amortize across billions of hours of paid work).
  - kshacker 6mo ago
    A good engineer and / or a tenured engineer could very well be compared to Picasso in this story. A tenured engineer did not just sit their entire career drawing that painting on the napkin, they delivered other results too. But at the end of it, they are able to deliver a Picasso at a moment's notice.
breton 6mo ago
> the model will use $200 in credits trying to solve a problem that could be fixed in an hour by a good engineer with enough context So the price for fixing the problem is equal. Sounds like a great argument for AI.
- OlivOnTech 6mo ago
  Most good engineers are way cheaper than that. The world is bigger than the united states.
- malinens 6mo ago
  99% of software developers earn less than 200 USD a hour
  - d1sxeyes 6mo ago
    That “with enough context” is doing a lot of work here. If you take a great engineer, drop them in front of an unfamiliar codebase, it’ll take them more than an hour to do most non-trivial tasks.
- collingreen 6mo ago
  Equal sounds like a terrible argument given all the other problems with replacing engineering thought with ai. I don't know where the line is but I expect it's far beyond equal AND there needs to be a level of "this can debug effectively in production" before that makes any sense for a real business case.
charcircuit 6mo ago
>That prices would rise Competition will prevent that from happening. When anyone can host open models and there is giant demand for LLMs companies can not easily raise token prices without sending a lot of traffic to their competitors.
- paganel 6mo ago
  > When anyone can host open models They'd still need to pay the actual power costs.
  - charcircuit 6mo ago
    I didn't say that inference would be free, but that everything to do inference is a commodity which means that competition is easy to do.
jrpear 6mo ago
Even if you take it as true that prices have risen recently, and may continue to rise as the VC subsidies dry up, they will fall again long-term. Inference will get more power efficient with model-on-chip solutions like Taalas and God willing we will get cheaper and cheaper renewable energy. Despite this I don't think engineers should feel threatened. As long as there is a need for a human in the loop, as today, there will still be engineering jobs. And if demand for engineering effort is elastic enough, there could easily be even more jobs tomorrow. Rather than threatened, I think engineers should feel exposed. To danger, yes, but opportunity as well.
- travoc 6mo ago
  Increased demand will not drive down energy costs.
  - lobf 6mo ago
    Why not?
  - jrpear 6mo ago
    Of course not necessarily, but I keep seeing articles about how wind and especially solar power just keep getting cheaper.
wat10000 6mo ago
It seems very unlikely that prices would rise in the long term. Yes, RAM and GPU prices are suddenly going up due to the demand spike and OpenAI's shenanigans, but I doubt it's going to last very long. Some combination of new capacity and reduced demand will most likely put things back on the usual course where this stuff gradually gets cheaper over time. And models are getting better, so next year you can probably get the same results for less compute. That $200 in credits becomes $150, then $100, then....
deleted 6mo ago
[deleted]

christoph 6mo ago

A good lesson for all - I always really liked the Picasso version: In a bustling restaurant, an excited patron recognized the famous artist Picasso dining alone. Seizing the moment, the patron approached Picasso with a simple request. With a plain napkin and a big smile, he asked the artist for a drawing. He promised payment for his troubles. Picasso, ever the creator, didn’t hesitate. From his pocket, he produced a charcoal pencil and he brought to life a stunning sketch of a goat on the napkin—a clear mark of his unique style. Proudly, he presented it to the patron. The artwork mesmerized the patron, who reached out to take it, only to be stopped by Picasso’s firm hand. “That will be $100,000,” Picasso declared. Astonished, the patron balked at the sum. “But it took you just a few seconds to draw this!” With a calm demeanor, Picasso took back the napkin, crumpled it, and tucked it away into his pocket, replying, “No, it has taken me a lifetime.”

airstrike 6mo ago
[flagged]
- wat10000 6mo ago
  It actually matches up well with the current AI scene, except backwards. We use these model which cost ridiculous amounts of money to train, and all of that effort goes into producing the outputs we use, but we're paying something not too far above the marginal cost of inference when we use them.
  - airstrike 6mo ago
    So not applicable at all
    - wat10000 6mo ago
      Extremely applicable to illustrate the difference between people (time is precious, training and experience amortize across a relatively small amount of paid work) and software (can replicate infinitely, time is cheap, startup costs can amortize across billions of hours of paid work).
- kshacker 6mo ago
  A good engineer and / or a tenured engineer could very well be compared to Picasso in this story. A tenured engineer did not just sit their entire career drawing that painting on the napkin, they delivered other results too. But at the end of it, they are able to deliver a Picasso at a moment's notice.

breton 6mo ago

> the model will use $200 in credits trying to solve a problem that could be fixed in an hour by a good engineer with enough context So the price for fixing the problem is equal. Sounds like a great argument for AI.

OlivOnTech 6mo ago
Most good engineers are way cheaper than that. The world is bigger than the united states.
malinens 6mo ago
99% of software developers earn less than 200 USD a hour
- d1sxeyes 6mo ago
  That “with enough context” is doing a lot of work here. If you take a great engineer, drop them in front of an unfamiliar codebase, it’ll take them more than an hour to do most non-trivial tasks.
collingreen 6mo ago
Equal sounds like a terrible argument given all the other problems with replacing engineering thought with ai. I don't know where the line is but I expect it's far beyond equal AND there needs to be a level of "this can debug effectively in production" before that makes any sense for a real business case.

charcircuit 6mo ago

>That prices would rise Competition will prevent that from happening. When anyone can host open models and there is giant demand for LLMs companies can not easily raise token prices without sending a lot of traffic to their competitors.

paganel 6mo ago
> When anyone can host open models They'd still need to pay the actual power costs.
- charcircuit 6mo ago
  I didn't say that inference would be free, but that everything to do inference is a commodity which means that competition is easy to do.

jrpear 6mo ago

Even if you take it as true that prices have risen recently, and may continue to rise as the VC subsidies dry up, they will fall again long-term. Inference will get more power efficient with model-on-chip solutions like Taalas and God willing we will get cheaper and cheaper renewable energy. Despite this I don't think engineers should feel threatened. As long as there is a need for a human in the loop, as today, there will still be engineering jobs. And if demand for engineering effort is elastic enough, there could easily be even more jobs tomorrow. Rather than threatened, I think engineers should feel exposed. To danger, yes, but opportunity as well.

travoc 6mo ago
Increased demand will not drive down energy costs.
- lobf 6mo ago
  Why not?
- jrpear 6mo ago
  Of course not necessarily, but I keep seeing articles about how wind and especially solar power just keep getting cheaper.

wat10000 6mo ago

It seems very unlikely that prices would rise in the long term. Yes, RAM and GPU prices are suddenly going up due to the demand spike and OpenAI's shenanigans, but I doubt it's going to last very long. Some combination of new capacity and reduced demand will most likely put things back on the usual course where this stuff gradually gets cheaper over time. And models are getting better, so next year you can probably get the same results for less compute. That $200 in credits becomes $150, then $100, then....

deleted 6mo ago

[deleted]