3 ms·

> "I gave it the same prompt, and it came out different" I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a diffe

> "I gave it the same prompt, and it came out different" I wrote a program in C and and gave it to gcc. Then I gave the same program to clang and I got a different result. I guess C code isn't programming.

cocoto 8mo ago
If there is no error on the compiler implementation and no undefined behavior the resulting program is equivalent and the few differences are mostly just implementation defined stuff which are left to the compiler to decide (but often gcc and clang do the same). The performance might differ also. It’s clearly not comparable to the many differences you can get from LLM’s output.
- hackyhacky 8mo ago
  It just depends what level of abstraction you're willing to pretend doesn't matter. gcc and clang produce different assembly code, but it "does the same thing," for certain definitions of "same" and "thing." Claude and Gemini produce different Rust code, but it "does the same thing," for certain definitions of "same" and "thing." The issue is that the ultimate beneficiary of AI is the business owner. He's not a programmer, and he has a much looser definition of "same."
  - ebb_earl_co 8mo ago
    No, the ultimate beneficiary of LLM-created code is the toll collectors who stole as much intellectual property as they could (and continue to do so), fleecing everyone else that they are Promethean for having done so and for continuing to do so.
  - danelski 8mo ago
    It has to stop somewhere. Business owner can also hire a different company to create the product and get a result different by as little as 5% performance difference or something with clearly inferior maintainability and UX. You'd hardly argue that it's 'the same' when they followed the same spec, which will never be fully precise at the business level. I agree that talking to an LLM is akin to using the business oriented logic at the module or even function level.
  - runarberg 8mo ago
    Your logic sounds like willful ignorance. You are relying on some odd definitions of "definitions", "equivalence", and "procedures". These are all rigorously defined in the underlying theory of computer science (using formal logic, lambda calculus, etc.) Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds). The C Standard has been rigorously written to uphold certain principles such that the same code (following its axioms) will always produce the same results (under specified conditions) for any standard compliant compiler. There exists no such standard (and no axioms nor conditions to speak of) where the same is true of Claude and Gemini. If you are interested, you can read the standard here (after purchasing access): https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en
    - hackyhacky 8mo ago
      > Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds). True, but none of that is relevant to the non-programmer end user. > You are relying on some odd definitions of "definitions", "equivalence", and "procedures" These terms have rigorous definitions for programmers. The person making software in the future is a non-programmer and doesn't care about any of that. They care only that the LLM can produce what they asked for. > The C Standard has been rigorously written to uphold certain principles I know what a standard is. The point is that the standard is irrelevant if you never look at the code.
      - runarberg 8mo ago
        It is indeed extremely relevant to the end user. For websites the end user is not the creator of the web site who pushes it to the server, it is the user who opens it on a browser. And for that user it matters a great deal if a button is green or blue, if it responds to keyboard events, if it says “submit” or “continue”, etc. It also matters to the user whether their information is sent to a third party, whether their password is leaked, etc. Your argument here (if I understand you correctly) is the same argument that to build a bridge you do not need to know all the laws of physics that prevents it from collapsing. The project manager of the construction team doesn’t need to know it, and certainly not the bicyclists who cross it. But the engineer who draws the blueprints needs to know it, and it matters that every detail on those blueprints are rigorously defined, such that the project manager of the construction team follows them to the letter. If the engineer does not know the laws of physics, or the project manager does not follow the blueprints to the letter, the bridge will likely collapse, killing the end user, that poor bicyclist.
svieira 8mo ago
Note that the prompt wasn't fed to another LLM, but to the same one. "I wrote a program in C and gave it to GCC. Then I gave the same program to GCC again and I got a different result" would be more like it.
- hackyhacky 8mo ago
  > Then I gave the same program to GCC again and I got a different result" would be more like it. This is a completely realistic scenario, given variance between compiler output based on optimization level, target architecture, and version. Sure, LLMs are non-deterministic, but that doesn't matter if you never look at the code.
  - Retric 8mo ago
    Optimization level, target architecture, etc are just information fed to the compiler. If it’s still nondeterministic with everything kept the same your compiler is broken.
    - hackyhacky 8mo ago
      You're missing the point. I'm not saying that compilers are nondeterminitsic or that LLMs are deterministic. I'm saying that it doesn't matter. No one cares about deterministic results except programmers. The non-technical user who will make software in the future just knows that he gets what he asked for.
      - Retric 8mo ago
        Systems randomly failing is of significant concern to non programmers, that’s inherent to the non-deterministic nature of LLM’s. I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
        hackyhacky 8mo ago
measurablefunc 8mo ago
The output will conform to the standard & it will be semantically equivalent. You're making several category errors w/ your comparison.
- hackyhacky 8mo ago
  > You're making several category errors w/ your comparison. I don't think I am. If you ask an LLM for a burger web site, you will get a burger web site. That's the only category that matters.
  - measurablefunc 8mo ago
    I think you are b/c you lack actual understanding of how compilers work & what it would mean to compile the same C code w/ two different conformant C compilers & get semantically different results.
    - hackyhacky 8mo ago
      > you lack actual understanding of how compilers work My brother in Christ, please get off your condescending horse. I have written compilers. I know how they work. And also you've apparently never heard of undefined behavior. The point is that the output is different at the assembly level, but that doesn't matter to the user. Just as output from an LLM but differ from another, but the user doesn't care.
      - measurablefunc 8mo ago
        You don't sound like you have written any code at all actually. What you do sound like is someone who is pretending like they know what it means to program which happens a lot on the internet.
        hackyhacky 8mo ago
        > You don't sound like you have written any code at all actually Well, you sound like an ignorant troll who came here to insult people and start fights. Which also happens a lot on the internet. Take your abrasive ego somewhere else. HN is not for you.

cocoto 8mo ago

If there is no error on the compiler implementation and no undefined behavior the resulting program is equivalent and the few differences are mostly just implementation defined stuff which are left to the compiler to decide (but often gcc and clang do the same). The performance might differ also. It’s clearly not comparable to the many differences you can get from LLM’s output.

hackyhacky 8mo ago
It just depends what level of abstraction you're willing to pretend doesn't matter. gcc and clang produce different assembly code, but it "does the same thing," for certain definitions of "same" and "thing." Claude and Gemini produce different Rust code, but it "does the same thing," for certain definitions of "same" and "thing." The issue is that the ultimate beneficiary of AI is the business owner. He's not a programmer, and he has a much looser definition of "same."
- ebb_earl_co 8mo ago
  No, the ultimate beneficiary of LLM-created code is the toll collectors who stole as much intellectual property as they could (and continue to do so), fleecing everyone else that they are Promethean for having done so and for continuing to do so.
- danelski 8mo ago
  It has to stop somewhere. Business owner can also hire a different company to create the product and get a result different by as little as 5% performance difference or something with clearly inferior maintainability and UX. You'd hardly argue that it's 'the same' when they followed the same spec, which will never be fully precise at the business level. I agree that talking to an LLM is akin to using the business oriented logic at the module or even function level.
- runarberg 8mo ago
  Your logic sounds like willful ignorance. You are relying on some odd definitions of "definitions", "equivalence", and "procedures". These are all rigorously defined in the underlying theory of computer science (using formal logic, lambda calculus, etc.) Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds). The C Standard has been rigorously written to uphold certain principles such that the same code (following its axioms) will always produce the same results (under specified conditions) for any standard compliant compiler. There exists no such standard (and no axioms nor conditions to speak of) where the same is true of Claude and Gemini. If you are interested, you can read the standard here (after purchasing access): https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en https://www.iso.org/obp/ui/#iso:std:iso-iec:9899:ed-5:v1:en
  - hackyhacky 8mo ago
    > Claude and Gemini do not "do the same thing" in the same way in which Clang and GCC does the same thing with the same code (as long as certain axioms of the code holds). True, but none of that is relevant to the non-programmer end user. > You are relying on some odd definitions of "definitions", "equivalence", and "procedures" These terms have rigorous definitions for programmers. The person making software in the future is a non-programmer and doesn't care about any of that. They care only that the LLM can produce what they asked for. > The C Standard has been rigorously written to uphold certain principles I know what a standard is. The point is that the standard is irrelevant if you never look at the code.
    - runarberg 8mo ago
      It is indeed extremely relevant to the end user. For websites the end user is not the creator of the web site who pushes it to the server, it is the user who opens it on a browser. And for that user it matters a great deal if a button is green or blue, if it responds to keyboard events, if it says “submit” or “continue”, etc. It also matters to the user whether their information is sent to a third party, whether their password is leaked, etc. Your argument here (if I understand you correctly) is the same argument that to build a bridge you do not need to know all the laws of physics that prevents it from collapsing. The project manager of the construction team doesn’t need to know it, and certainly not the bicyclists who cross it. But the engineer who draws the blueprints needs to know it, and it matters that every detail on those blueprints are rigorously defined, such that the project manager of the construction team follows them to the letter. If the engineer does not know the laws of physics, or the project manager does not follow the blueprints to the letter, the bridge will likely collapse, killing the end user, that poor bicyclist.

svieira 8mo ago

Note that the prompt wasn't fed to another LLM, but to the same one. "I wrote a program in C and gave it to GCC. Then I gave the same program to GCC again and I got a different result" would be more like it.

hackyhacky 8mo ago
> Then I gave the same program to GCC again and I got a different result" would be more like it. This is a completely realistic scenario, given variance between compiler output based on optimization level, target architecture, and version. Sure, LLMs are non-deterministic, but that doesn't matter if you never look at the code.
- Retric 8mo ago
  Optimization level, target architecture, etc are just information fed to the compiler. If it’s still nondeterministic with everything kept the same your compiler is broken.
  - hackyhacky 8mo ago
    You're missing the point. I'm not saying that compilers are nondeterminitsic or that LLMs are deterministic. I'm saying that it doesn't matter. No one cares about deterministic results except programmers. The non-technical user who will make software in the future just knows that he gets what he asked for.
    - Retric 8mo ago
      Systems randomly failing is of significant concern to non programmers, that’s inherent to the non-deterministic nature of LLM’s. I can send specific LLM output to QA, I can’t ask QA to validate that this prompt will always produce bug free code even for future versions of the AI.
      - hackyhacky 8mo ago

measurablefunc 8mo ago

The output will conform to the standard & it will be semantically equivalent. You're making several category errors w/ your comparison.

hackyhacky 8mo ago
> You're making several category errors w/ your comparison. I don't think I am. If you ask an LLM for a burger web site, you will get a burger web site. That's the only category that matters.
- measurablefunc 8mo ago
  I think you are b/c you lack actual understanding of how compilers work & what it would mean to compile the same C code w/ two different conformant C compilers & get semantically different results.
  - hackyhacky 8mo ago
    > you lack actual understanding of how compilers work My brother in Christ, please get off your condescending horse. I have written compilers. I know how they work. And also you've apparently never heard of undefined behavior. The point is that the output is different at the assembly level, but that doesn't matter to the user. Just as output from an LLM but differ from another, but the user doesn't care.
    - measurablefunc 8mo ago
      You don't sound like you have written any code at all actually. What you do sound like is someone who is pretending like they know what it means to program which happens a lot on the internet.
      - hackyhacky 8mo ago
        > You don't sound like you have written any code at all actually Well, you sound like an ignorant troll who came here to insult people and start fights. Which also happens a lot on the internet. Take your abrasive ego somewhere else. HN is not for you.