6 ms·

AI can be an amazing productivity multiplier for people who know what they're doing. This result reminded me of the C compiler case that Anthropic posted recen

by cpard 8mo ago

AI can be an amazing productivity multiplier for people who know what they're doing. This result reminded me of the C compiler case that Anthropic posted recently. Sure, agents wrote the code for hours but there was a human there giving them directions, scoping the problem, finding the test suites needed for the agentic loops to actually work etc etc. In general making sure the output actually works and that it's a story worth sharing with others. The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. It works great for creating impressions and building brand value but also does a disservice to the actual researchers, engineers and humans in general, who do the hard work of problem formulation, validation and at the end, solving the problem using another tool in their toolbox.

elzbardico 8mo ago
Actually, the results were far worse and way less impressive than what the media said.
- cpard 8mo ago
  the c compiler results or the physics results this post is about?
  - NewsaHackO 8mo ago
    His point is going to be some copium like since the c compiler is not as optimized as gcc, it was not impressive.
    - elzbardico 8mo ago
      You probably don’t know what you’re talking about.
      - NewsaHackO 8mo ago
        Why wasn't the C compiler it made impressive to you?
        kewscombinator 8mo ago
        I found this was the least impressive bit about it https://github.com/anthropics/claudes-c-compiler/issues/1 https://github.com/anthropics/claudes-c-compiler/issues/1
        NewsaHackO 8mo ago
        >I found this was the least impressive bit about it https://github.com/anthropics/claudes-c-compiler/issues/1 https://github.com/anthropics/claudes-c-compiler/issues/1 So, I just skimmed the discussion thread, but I am not seeing how this shows that CCC is not impressive. Is the point you're making that the person who opened the issue is not impressive?
        kortilla 8mo ago
        It didn’t work without gcc and it was significantly worse than gcc with gcc optimizations disabled.
        NateEag 8mo ago
        Like everything genAI, it was amazing yet surprisingly crappy. Yes, the bear is definitely dancing. But a few feet away there's a world-class step dancer doing intricate rhythms they've perfected over twenty years of hard work. The bear's kind of shuffling along to the beat like a stoner in a club. It's amazing it can do it at all... but the resulting compiler is not actually good enough to be worth using.
        NewsaHackO 8mo ago
        >It's amazing it can do it at all... but the resulting compiler is not actually good enough to be worth using. No one has made that assertion; however, the fact that it can create a functioning C compiler with minimal oversight is the impressive part, and it shows a path to autonomous GenAI use in software development.
  - elzbardico 8mo ago
    The C compiler.
    - cpard 8mo ago
      of course the results were much worse than what was communicated on the media, it was content marketing not an attempt to build a better c compiler.
fragmede 8mo ago
Right. If it hadn't been Nicholas Carlini driving Claude, with his decades of experience, there wouldn't be a Claude c compiler. It still required his expertise and knowledge for it to get there.
- deleted 8mo ago
  [deleted]
decidu0us9034 8mo ago
I'm not sure you can call something an optimizing C compiler if it doesn't optimize or enforce C semantics (well, it compiles C but also a lot of things that aren't syntactically valid C). It seemed to generate a lot of code (wow!) that wasn't well-integrated and didn't do what it promised to, and the human didn't have the requisite expertise to understand that. I'm not a theoretical physicist but I will hold to my skepticism here, for similar reasons.
- cpard 8mo ago
  sure, I won't argue on this, although it did manage to deliver the marketing value they were looking for, at the end their goal was not to replace gcc but to make people talk about AI and Anthropic. What I said in my original comment is that AI delivers when it's used by experts, in this case there was someone who was definitely not a C compiler expert, what would happen if there was a real expert doing this?
  - BrouteMinou 8mo ago
    Deliver what exactly? False hope and lies? https://github.com/anthropics/claudes-c-compiler/issues/228 https://github.com/anthropics/claudes-c-compiler/issues/228
jonahx 8mo ago
> The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. It's also a legitimate concern. We happen to be in a place where humans are needed for that "last critical 10%," or the first critical 10% of problem formulation, and so humans are still crucial to the overall system, at least for most complex tasks. But there's no logical reason that needs to be the case. Once it's not, humans will be replaced.
- cpard 8mo ago
  The reason there is a marketing opportunity is because, to your point, there is a legitimate concern. Marketing builds and amplifies the concern to create awareness. When the systems turn into something trivial to manage with the new tooling, humans build more complex or add more layers on the existing systems.
- krethh 8mo ago
  The logical reason is that humans are exceptionally good at operating at the edge of what the technology of the time can do. We will find entire classes of tech problems which AI can't solve on its own. You have people today with job descriptions that even 15 years ago would have been unimaginable, much less predictable. To think that whatever the AI is capable of solving is (and forever will be) the frontier of all problems is deeply delusional. AI got good at generating code, but it still can't even do a fraction of what the human brain can do.
  - jonahx 8mo ago
    > To think that whatever the AI is capable of solving is (and forever will be) the frontier of all problems is deeply delusional. AI got good at generating code, but it still can't even do a fraction of what the human brain can do. AGI means fully general, meaning everything the human brain can do and more. I agree that currently it still feels far (at least it may be far), but there is no reason to think there's some magic human ingredient that will keep us perpetually in the loop. I would say that is delusional. We used to think there was human-specific magic in chess, in poker, in Go, in code, and in writing. All those have fallen, the latter two albeit only in part but even that part was once thought to be the exclusive domain of humans.
kylehotchkiss 8mo ago
> for people who know what they're doing. I worry we're not producing as many of those as we used to
- blks 8mo ago
  We will be producing them even less. I fear for the future graduates, hell even for school children, who are now uncontrollably using ChatGPT for their homework. Next level brainrot
deleted 8mo ago
[deleted]
supern0va 8mo ago
>AI can be an amazing productivity multiplier for people who know what they're doing. >[...] >The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. You're sort of acting like it's all or nothing. What about the the humans that used to be that "force multiplier" on a team with the person guiding the research? If a piece of software required a team of ten to people, and instead it's built with one engineer overseeing an AI, that's still 90% job loss. For a more current example: do you think all the displaced Uber/Lyft drivers aren't going to think "AI took my job" just because there's a team of people in a building somewhere handling the occasional Waymo low confidence intervention, as opposed to being 100% autonomous?
- bagacrap 8mo ago
  Well those Uber drivers are usually pretty quick to note that Uber is not their job, just a side hustle. It's too bad I won't know what they think by then since we won't be interacting any more.
- Human-Cabbage 8mo ago
  The optimistic case is that instead of a team of 10 people working on one project, you could have those 10 people using AI assistants to work on 10 independent projects. That, of course, assumes that there are 9 other projects that are both known (or knowable) and worth doing. And in the case of Uber/Lyft drivers, there's a skillset mismatch between the "deprecated" jobs and their replacements.
- cpard 8mo ago
  there's 90% job loss assuming that this is a zero sum type of thing where humans and agents compete for working on a fixed amount of work. I'm curious why you think I'm acting like it's all or nothing. What I was trying to communicate is the exact opposite, that it's not all or nothing. Maybe it's the way I articulate things, I'm genuinely interested what makes it sound like this.
  - ramathornn 8mo ago
    Fully agree with your og comment and I didn’t get the same read as the person above at all. This is a bizarre time to be living in, on one hand these tools are capable of doing more and more of the tasks any knowledge worker today handles, especially when used by an experienced person in X field. On the other, it feels like something is about to give. All the superbowl ads, AI in what feels like every single piece of copy coming out these days. AI CEOs hopping from one podcast to another warning about the upcoming career apocalypse…I’m not fully buying it.
ece 8mo ago
Everytime I see a RL startup, a data startup or even a startup focused on a specific vertical, I think this exact same thing about LLMs.
hananova 8mo ago
AI is indeed an amazing productivity multiplier! Sadly that multiplier is in the range [0; 1).

by cpard 8mo ago

elzbardico 8mo ago

Actually, the results were far worse and way less impressive than what the media said.

cpard 8mo ago
the c compiler results or the physics results this post is about?
- NewsaHackO 8mo ago
  His point is going to be some copium like since the c compiler is not as optimized as gcc, it was not impressive.
  - elzbardico 8mo ago
    You probably don’t know what you’re talking about.
    - NewsaHackO 8mo ago
      Why wasn't the C compiler it made impressive to you?
      - kewscombinator 8mo ago
        I found this was the least impressive bit about it https://github.com/anthropics/claudes-c-compiler/issues/1 https://github.com/anthropics/claudes-c-compiler/issues/1
        NewsaHackO 8mo ago
        >I found this was the least impressive bit about it https://github.com/anthropics/claudes-c-compiler/issues/1 https://github.com/anthropics/claudes-c-compiler/issues/1 So, I just skimmed the discussion thread, but I am not seeing how this shows that CCC is not impressive. Is the point you're making that the person who opened the issue is not impressive?
      - kortilla 8mo ago
        It didn’t work without gcc and it was significantly worse than gcc with gcc optimizations disabled.
      - NateEag 8mo ago
        Like everything genAI, it was amazing yet surprisingly crappy. Yes, the bear is definitely dancing. But a few feet away there's a world-class step dancer doing intricate rhythms they've perfected over twenty years of hard work. The bear's kind of shuffling along to the beat like a stoner in a club. It's amazing it can do it at all... but the resulting compiler is not actually good enough to be worth using.
        NewsaHackO 8mo ago
        >It's amazing it can do it at all... but the resulting compiler is not actually good enough to be worth using. No one has made that assertion; however, the fact that it can create a functioning C compiler with minimal oversight is the impressive part, and it shows a path to autonomous GenAI use in software development.
- elzbardico 8mo ago
  The C compiler.
  - cpard 8mo ago
    of course the results were much worse than what was communicated on the media, it was content marketing not an attempt to build a better c compiler.

fragmede 8mo ago

Right. If it hadn't been Nicholas Carlini driving Claude, with his decades of experience, there wouldn't be a Claude c compiler. It still required his expertise and knowledge for it to get there.

deleted 8mo ago
[deleted]

decidu0us9034 8mo ago

I'm not sure you can call something an optimizing C compiler if it doesn't optimize or enforce C semantics (well, it compiles C but also a lot of things that aren't syntactically valid C). It seemed to generate a lot of code (wow!) that wasn't well-integrated and didn't do what it promised to, and the human didn't have the requisite expertise to understand that. I'm not a theoretical physicist but I will hold to my skepticism here, for similar reasons.

cpard 8mo ago
sure, I won't argue on this, although it did manage to deliver the marketing value they were looking for, at the end their goal was not to replace gcc but to make people talk about AI and Anthropic. What I said in my original comment is that AI delivers when it's used by experts, in this case there was someone who was definitely not a C compiler expert, what would happen if there was a real expert doing this?
- BrouteMinou 8mo ago
  Deliver what exactly? False hope and lies? https://github.com/anthropics/claudes-c-compiler/issues/228 https://github.com/anthropics/claudes-c-compiler/issues/228

jonahx 8mo ago

> The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. It's also a legitimate concern. We happen to be in a place where humans are needed for that "last critical 10%," or the first critical 10% of problem formulation, and so humans are still crucial to the overall system, at least for most complex tasks. But there's no logical reason that needs to be the case. Once it's not, humans will be replaced.

cpard 8mo ago
The reason there is a marketing opportunity is because, to your point, there is a legitimate concern. Marketing builds and amplifies the concern to create awareness. When the systems turn into something trivial to manage with the new tooling, humans build more complex or add more layers on the existing systems.
krethh 8mo ago
The logical reason is that humans are exceptionally good at operating at the edge of what the technology of the time can do. We will find entire classes of tech problems which AI can't solve on its own. You have people today with job descriptions that even 15 years ago would have been unimaginable, much less predictable. To think that whatever the AI is capable of solving is (and forever will be) the frontier of all problems is deeply delusional. AI got good at generating code, but it still can't even do a fraction of what the human brain can do.
- jonahx 8mo ago
  > To think that whatever the AI is capable of solving is (and forever will be) the frontier of all problems is deeply delusional. AI got good at generating code, but it still can't even do a fraction of what the human brain can do. AGI means fully general, meaning everything the human brain can do and more. I agree that currently it still feels far (at least it may be far), but there is no reason to think there's some magic human ingredient that will keep us perpetually in the loop. I would say that is delusional. We used to think there was human-specific magic in chess, in poker, in Go, in code, and in writing. All those have fallen, the latter two albeit only in part but even that part was once thought to be the exclusive domain of humans.

kylehotchkiss 8mo ago

> for people who know what they're doing. I worry we're not producing as many of those as we used to

blks 8mo ago
We will be producing them even less. I fear for the future graduates, hell even for school children, who are now uncontrollably using ChatGPT for their homework. Next level brainrot

deleted 8mo ago

[deleted]

supern0va 8mo ago

>AI can be an amazing productivity multiplier for people who know what they're doing. >[...] >The "AI replaces humans in X" narrative is primarily a tool for driving attention and funding. You're sort of acting like it's all or nothing. What about the the humans that used to be that "force multiplier" on a team with the person guiding the research? If a piece of software required a team of ten to people, and instead it's built with one engineer overseeing an AI, that's still 90% job loss. For a more current example: do you think all the displaced Uber/Lyft drivers aren't going to think "AI took my job" just because there's a team of people in a building somewhere handling the occasional Waymo low confidence intervention, as opposed to being 100% autonomous?

bagacrap 8mo ago
Well those Uber drivers are usually pretty quick to note that Uber is not their job, just a side hustle. It's too bad I won't know what they think by then since we won't be interacting any more.
Human-Cabbage 8mo ago
The optimistic case is that instead of a team of 10 people working on one project, you could have those 10 people using AI assistants to work on 10 independent projects. That, of course, assumes that there are 9 other projects that are both known (or knowable) and worth doing. And in the case of Uber/Lyft drivers, there's a skillset mismatch between the "deprecated" jobs and their replacements.
cpard 8mo ago
there's 90% job loss assuming that this is a zero sum type of thing where humans and agents compete for working on a fixed amount of work. I'm curious why you think I'm acting like it's all or nothing. What I was trying to communicate is the exact opposite, that it's not all or nothing. Maybe it's the way I articulate things, I'm genuinely interested what makes it sound like this.
- ramathornn 8mo ago
  Fully agree with your og comment and I didn’t get the same read as the person above at all. This is a bizarre time to be living in, on one hand these tools are capable of doing more and more of the tasks any knowledge worker today handles, especially when used by an experienced person in X field. On the other, it feels like something is about to give. All the superbowl ads, AI in what feels like every single piece of copy coming out these days. AI CEOs hopping from one podcast to another warning about the upcoming career apocalypse…I’m not fully buying it.

ece 8mo ago

Everytime I see a RL startup, a data startup or even a startup focused on a specific vertical, I think this exact same thing about LLMs.

hananova 8mo ago

AI is indeed an amazing productivity multiplier! Sadly that multiplier is in the range [0; 1).