5 ms·

> Writing and programming are both a form of working at a problem through text… Whoa whoa whoa hold your horses, code has a pretty important property that ordi

by Uehreka 8mo ago

> Writing and programming are both a form of working at a problem through text… Whoa whoa whoa hold your horses, code has a pretty important property that ordinary prose doesn’t have: it can make real things happen even if no one reads it (it’s executable). I don’t want to read something that someone didn’t take the time to write. But I’ll gladly use a tool someone had an AI write, as long as it works (which these things increasingly do). Really elegant code is cool to read, but many tools I use daily are closed source, so I have no idea if their code is elegant or not. I only care if it works.

NuclearPM 8mo ago
AI is excellent for helping to write things like tech specs, procedures and manuals.
- djeastm 8mo ago
  As long as we're all aware that it'll probably mostly be AI reading them...
JohnMakin 8mo ago
Sometimes (or often) things with horrible security flaws "work" but not in the way that they should and are exposing you to risk.
- rescripting 8mo ago
  If you refuse to run AI generated code for this reason, then you should refuse to run closed source code for the same reason.
  - JohnMakin 8mo ago
    I don't see how the two correlate - commercial, closed source software usually have teams of professionals behind them with a vested and shared interest in not shipping crap that will blow up in their customers' face. I don't think the motivations of "guy who vibe coded a shitty app in an afternoon" are the same. And to answer you more directly, generally, in my professional world, I don't use closed source software often for security reasons, and when I do, it's from major players with oodles of more resources and capital expenditure than "some guy with a credit card paid for a gemini subscription."
pixl97 8mo ago
Hell, I'd read an instruction manual that AI wrote as long as it accurately describes. I see a lot of these discussions where a person gets feelings/feels mad about something and suddenly a lot of black and white thinking starts happening. I guess that's just part of being human.
1shooner 8mo ago
>Code has a pretty important property that ordinary prose doesn’t have But isn't this the distinction that language models are collapsing? There are 'prose' prompt collections that certainly make (programmatic) things happen, just as there is significant concern about the effect of LLM-generated prose on social media, influence campaigns, etc.
fhd2 8mo ago
Users typically don't read code, developers (of the software) do. If it's not worth reading something where the writer didn't take the time to write it, by extension that means nobody read the code. Which means nobody understands it, beyond the external behaviour they've tested. I'd have some issues with using such software, at least where reliability matters. Blackbox testing only gets you so far. But I guess as opposed to other types of writing, developers _do_ read generated code. At least as soon as something goes wrong.
- tptacek 8mo ago
  Developers do not in fact tend to read all the software they use. I have never once looked at the code for jq, nor would I ever want to (the worst thing I could learn about that contraption is that the code is beautiful, and then live out the rest of my days conflicted about my feelings about it). This "developers read code" thing is just special pleading.
  - hexaga 8mo ago
    You're a user of jq in the sense of the comment you're replying to, not a developer. The developer is the developer _of jq_, not developers in general.
    - tptacek 8mo ago
      We're talking about Show HN here.
    - fhd2 8mo ago
      Yes, that's exactly how I meant it. I might _rarely_ peruse some code if I'm really curious about it, but by and large I just trust the developers of the software I use and don't really care how it works. I care about what it does.
      - agentultra 8mo ago
        As a developer of software I often have to care because it matters and so I read the code. Source code is often written for other humans first and foremost.
arscan 8mo ago
> But I’ll gladly use a tool someone had an AI write, as long as it works (which these things increasingly do). It works, sure, but is it worth your time to use? I think a common blind spot for software engineers is understanding how hard it is to get people to use software they aren’t effectively forced to use (through work or in order to gain access to something or ‘network effects’ or whatever). Most people’s time and attention is precious, their habits are ingrained, and they are fundamentally pretty lazy. And people that don’t fall into the ‘most people’ I just described, probably won’t want to use software you had an LLM write up when they could have just done it themselves to meet their exact need. UNLESS it’s something very novel that came from a bit of innovation that LLMs are incapable of. But that bit isn’t what we are talking about here, I don’t think.
- mindcrime 8mo ago
  probably won’t want to use software you had an LLM write up when they could have just done it themselves to meet their exact need Sure... to a point. But realistically, the "use an LLM to write it yourself" approach still entails costs, both up-front and on-going, even if the cost may be much less than in the past. There's still reason to use software that's provided "off the shelf", and to some extent there's reason to look at it from a "I don't care how you wrote it, as long as it works" mindset. came from a bit of innovation that LLMs are incapable of. I think you're making an overly binary distinction on something that is more of a continuum, vis-a-vis "written by human vs written by LLM". There's a middle ground of "written by human and LLM together". I mean, the people building stuff using something like SpecKit or OpenSpec still spend a lot of time up-front defining the tech stack, requirements, features, guardrails, etc. of their project, and iterating on the generated code. Some probably even still hand tune some of the generated code. So should we reject their projects just because they used an LLM at all, or ?? I don't know. At least for me, that might be a step further than I'd go.
  - arscan 8mo ago
    > There's a middle ground of "written by human and LLM together". Absolutely, but I’d categorize that ‘bit’ as the innovation from the human. I guess it’s usually just ongoing validation that the software is headed down a path of usefulness which is hard to specify up-front and by definition something only the user (or a very good proxy) can do (and even they are usually bad at it).
exit 8mo ago
similarly, i think that something that someone took the time to proof-read/verify can be of value, even if they did not directly write it. this is the literary equivalent of compiling and running the code.
rubslopes 8mo ago
I agree with your sentiment, and it touches on one of the reasons I left academia for IT. Scientific research is preoccupied with finding the truth, which is beautiful but very stressful. If you're a perfectionist, you're always questioning yourself: "Did I actually find something meaningful, or is it just noise? Did I gaslight myself into thinking I was just exploring the data when I was actually p-hacking the results?" This took a real toll on my mental health. Although I love science, I'm much happier building programs. "Does the program do what the client expects with reasonable performance and safety? Yes? Ship it."
nicbou 8mo ago
It makes sense. A vibe-coded tool can sometimes do the job, just like some cheap Chinese-made widget. Not every task requires hand-crafted professional grade tools. For example, I have a few letter generators on my website. The letters are often verified by a lawyer, but the generator could totally be vibe-coded. It's basically an HTML form that fills in the blanks in the template. Other tools are basically "take input, run calculation, show output". If I can plug in a well-tested calculation, AI could easily build the rest of the tool. I have been staunchly against using AI in my line of work, but this is an acceptable use of it.
- Brian_K_White 8mo ago
  "A vibe-coded tool can sometimes do the job" How do you know it ever does the job? I don't know either for most code that I use, but I do have reason to trust that the author does know. I don't really trust any code itself, only the people and processes (organizational, not computer) that generated it. I have no reason to trust that ai generated code is doing the correct thing. I know enough about the way code works to know that merely observing it seem to work in a test case means absolutely nothing at all. Multiply that zero by a million more test cases and it's the same zero. The only thing I trust is that someone actually understood a problem they were trying to solve, and cares about avoiding edge cases, and tries to develop logic to make unintended outcomes impossible etc... It's not possible for an ai to do any of that regardless what the prompts are. But what they can do is emit stuff that some person once wrote which did exhibit these qualities, and so looks ok, and causes idiots to think they found the cheat code to life, and worse, foist that shit off on everyone else. My mom does not have my awareness that any of this is going on. She's just out there in the world running into this crap blindly as an unwitting end-user who has no idea how badly she's being served these days when she uses basically any app or service. Thanks for that vibe coders of the world.
  - nicbou 8mo ago
    > How do you know it ever does the job? Because the part of the job it automates is simple, and can be tested. I cannot overstate how simple the tools I am thinking of are. Think tipping calculator. Neither new nor creative nor complex. The real value here is being familiar with the problem. You are missing the point here. I am talking about people who were not served at all by software developers. The alternative is not craftsmanship, but at best duct taping wordpress plugins together.
ethmarks 8mo ago
I guess it depends on whether you're only executing the code or if you're submitting it for humans to review. If your use case is so low-stakes that a review isn't required, then vibe coding is much more defensible. But if code quality matters even slightly, such that you need to review the code, then you run into the same problems that you do with AI-generated prose: nobody wants to read what you couldn't be bothered to write.
- josephg 8mo ago
  There’s lots of times where I just don’t care how it’s implemented. I got Claude to make a test suite the other day for a couple RFCs so I could check for spec compliance. It made a test runner and about 300 tests. And an html frontend to view the test results in a big table. Claude and I wrote 8500 lines of code in a day. I don’t care how the test runner works, so long as it works. I really just care about the test results. Is it finding real bugs? Well, we went though the 60 or so failing tests. We changed 3 tests, because Claude had misunderstood the rfc. The rest were real bugs. I’m sure the test runner would be more beautiful if I wrote it by hand. But I don’t care. I’ve written test runners before. They’re not interesting. I’m all for beautiful, artisanal code. I love programming. But sometimes I just want to get a job done. Sometimes the code isn’t for reading. It’s for running.
  - layer8 8mo ago
    What is it about the tests that make you not care about how it’s implemented versus other code? Just the fact that it isn’t interesting to you? I happen to like spec-testing code and find it an interesting design space. It is vital to review it thoroughly for correctness. But I also think that whether I find something interesting or not doesn’t have much bearing on whether code quality is important. Code quality can even be more important for uninteresting code, because due to the lack of interest I’ll have a lower attention span when reviewing it.
    - josephg 8mo ago
      > What is it about the tests that make you not care about how it’s implemented versus other code? Huh this is a thought provoking question. I think there's a few reasons. In a test suite: - I don't care about performance. - I don't care (as much) about reliability. My users aren't affected by crashes and other failures in my tests. - I don't care (as much) about correctness. Erroneously failing tests will get human attention. Tests that erroneously pass are a bigger problem, but my test suite is not the last line of defence against bugs reaching users. - If I had infinite time, I'd love every line of code to be a mathematically beautiful work of art. But I don't. Writing this test suite by hand would have taken me about 3 weeks. Instead, I did it in 1 day with claude. This let me spend 14 productive days working on other things. I would rather have a good-enough test suite and 14 days of productive work than an excellent test suite and nothing else. I could spend those 14 days fixing all the bugs it found. Or writing more tests. Or getting claude to write more tests. Or going outside with my friends. These are all better uses of my time. If I was writing the core of a new game engine, the scheduler of an operating system or the data storage engine for a new database, then I would think hard about every line of code. But not all code is like that. We must adapt ourselves to the project at hand. Some lines of code matter a lot more than others. Our workflow should take that into account.
zahlman 8mo ago
> it can make real things happen even if no one reads it (it’s executable). "One" is the operative word here, supposing this includes only humans and excludes AI agents. When code is executed, it does get read (by the computer). Making that happen is a conscious choice on the part of a human operator. The same kind of conscious choice can feed writing to an LLM to see what it does in response. That is much the same kind of "execution", just non-deterministic (and, when given any tools beyond standard input and standard output, potentially dangerous in all the same ways, but worse because of the nondeterminism).
morgoths_bane 8mo ago
> I only care if it works Okay but it is probably not going to be a tool that will be reliable or work as expected for too long depending on how complex it is, how easily it can be understood, and how it can handle updates to libraries, etc. that it is using. Also, what is our trust with this “tool”? E.g. this is to be used in a brain surgery that you’ll undergo, would you still be fine with using something generated by AI? Earlier you couldn’t even read something it generated, but we’ll trust a “tool” it created because we believe it works? Why do we believe it will work? Because a computer created it? That’s our own bias towards computing that we assume that it is impartial but this is a probabilistic model trained on data that is just as biased as we are. I cannot imagine that you have not witnessed these models creating false information that you were able to identify. Understanding their failure on basic understandings, how then could we trust it with engineering tasks? Just because “it works”? What does that mean and how can we be certain? QA perhaps but ask any engineer here if companies are giving a single shit about QA while they’re making them shove out so much slop, and the answer is going to be disappointing. I don’t think we should trust these things even if we’re not developers. There isn’t anyone to hold accountable if (and when) things go wrong with their outputs. All I have seen AI be extremely good at is deceiving people, and that is my true concern with generative technologies. Then I must ask, if we know that its only effective use case is deception, why then should I trust ANY tool it created? Maybe the stakes are quite low, maybe it is just a video player that you use to watch your Sword and Sandal flicks. Ok sure, but maybe someone uses that same video player for an exoscope and the data it is presenting to your neurosurgeon is incorrect causing them to perform an action they otherwise would have not done if provided with the correct information. We should not be so laissez-faire with this technology.
aeturnum 8mo ago
> even if no one reads it I gotta disagree with you there! Code that isn't read doesn't do anything. Code must be read to be compiled, it must be read to be interpreted, etc. I think this points to a difference in our understanding of "read" means, perhaps? To expand my pithy "not gonna read if you didn't write" bit: The idea that code stands on its own is a lie. The world changes around code and code must be changed to keep up with the world. Every "program" (is the git I run the same as the git you run?) is a living document that people maintain as need be. So when we extend the "not read / didn't write" it's not using the program (which I guess is like taking the lessons from a book) it's maintaining the program. So I think it's possible that I could derive benefit from someone else reading an llm's text output (they get an idea) - but what we are trying to talk about is the work of maintaining a text.
- williamcotton 8mo ago
  Correct me if I’m wrong, but if you wrote a dependency-free recursive descent parser in C89 thirty years ago it should still compile and return the same AST.
  - aeturnum 8mo ago
    I mean....it will compile and return the same AST on the OS and hardware from 30 years ago. But if you want to get the same result today on modern hardware / software you may discover you need to make some changes (or rather people have been making little changes for 30 years to ensure you can still get the same AST). Generally software has either had little bits and bobs added and removed to keep it relevant or its fallen away and been forgotten.
  - pjmlp 7mo ago
    Well, if it made use of any UB alongside its code, and it gets compiled with the latest version of a modern compiler in -O3, it might, or might not.
- deleted 8mo ago
  [deleted]
panny 8mo ago
>but many tools I use daily are closed source I wonder if this is a major differentiator between AI fans and detractors. I dislike and actively avoid anything closed source. I fully agree with the premise of the submission as well.
- layer8 8mo ago
  I don’t think so. With closed source, it depends on whether you trust the authors. As someone who grew up with shareware (which is generally closed source), you learned to trust authors based on their documentation and reputation. AI-generated code you simply can’t trust as-is. You have to trust whoever reviews it. Of course, with open source you can be the one who reviews it. But typically few people do, and usually almost no one does as thoroughly as the original authors and maintainers — except if they are an AI.
onel 8mo ago
I am a software engineer, but I think code pales in comparison to the impact of every book, speech and essay had on our society throughout history.
jpfromlondon 8mo ago
>Whoa whoa whoa hold your horses, code has a pretty important property that ordinary prose doesn’t have: it can make real things happen even if no one reads it (it’s executable). When your boss (assuming you have one) tells you to do something, do you just ignore it?

by Uehreka 8mo ago

NuclearPM 8mo ago

AI is excellent for helping to write things like tech specs, procedures and manuals.

djeastm 8mo ago
As long as we're all aware that it'll probably mostly be AI reading them...

JohnMakin 8mo ago

Sometimes (or often) things with horrible security flaws "work" but not in the way that they should and are exposing you to risk.

rescripting 8mo ago
If you refuse to run AI generated code for this reason, then you should refuse to run closed source code for the same reason.
- JohnMakin 8mo ago
  I don't see how the two correlate - commercial, closed source software usually have teams of professionals behind them with a vested and shared interest in not shipping crap that will blow up in their customers' face. I don't think the motivations of "guy who vibe coded a shitty app in an afternoon" are the same. And to answer you more directly, generally, in my professional world, I don't use closed source software often for security reasons, and when I do, it's from major players with oodles of more resources and capital expenditure than "some guy with a credit card paid for a gemini subscription."

pixl97 8mo ago

Hell, I'd read an instruction manual that AI wrote as long as it accurately describes. I see a lot of these discussions where a person gets feelings/feels mad about something and suddenly a lot of black and white thinking starts happening. I guess that's just part of being human.

1shooner 8mo ago

>Code has a pretty important property that ordinary prose doesn’t have But isn't this the distinction that language models are collapsing? There are 'prose' prompt collections that certainly make (programmatic) things happen, just as there is significant concern about the effect of LLM-generated prose on social media, influence campaigns, etc.

fhd2 8mo ago

Users typically don't read code, developers (of the software) do. If it's not worth reading something where the writer didn't take the time to write it, by extension that means nobody read the code. Which means nobody understands it, beyond the external behaviour they've tested. I'd have some issues with using such software, at least where reliability matters. Blackbox testing only gets you so far. But I guess as opposed to other types of writing, developers _do_ read generated code. At least as soon as something goes wrong.

tptacek 8mo ago
Developers do not in fact tend to read all the software they use. I have never once looked at the code for jq, nor would I ever want to (the worst thing I could learn about that contraption is that the code is beautiful, and then live out the rest of my days conflicted about my feelings about it). This "developers read code" thing is just special pleading.
- hexaga 8mo ago
  You're a user of jq in the sense of the comment you're replying to, not a developer. The developer is the developer _of jq_, not developers in general.
  - tptacek 8mo ago
    We're talking about Show HN here.
  - fhd2 8mo ago
    Yes, that's exactly how I meant it. I might _rarely_ peruse some code if I'm really curious about it, but by and large I just trust the developers of the software I use and don't really care how it works. I care about what it does.
    - agentultra 8mo ago
      As a developer of software I often have to care because it matters and so I read the code. Source code is often written for other humans first and foremost.

arscan 8mo ago

> But I’ll gladly use a tool someone had an AI write, as long as it works (which these things increasingly do). It works, sure, but is it worth your time to use? I think a common blind spot for software engineers is understanding how hard it is to get people to use software they aren’t effectively forced to use (through work or in order to gain access to something or ‘network effects’ or whatever). Most people’s time and attention is precious, their habits are ingrained, and they are fundamentally pretty lazy. And people that don’t fall into the ‘most people’ I just described, probably won’t want to use software you had an LLM write up when they could have just done it themselves to meet their exact need. UNLESS it’s something very novel that came from a bit of innovation that LLMs are incapable of. But that bit isn’t what we are talking about here, I don’t think.

mindcrime 8mo ago
probably won’t want to use software you had an LLM write up when they could have just done it themselves to meet their exact need Sure... to a point. But realistically, the "use an LLM to write it yourself" approach still entails costs, both up-front and on-going, even if the cost may be much less than in the past. There's still reason to use software that's provided "off the shelf", and to some extent there's reason to look at it from a "I don't care how you wrote it, as long as it works" mindset. came from a bit of innovation that LLMs are incapable of. I think you're making an overly binary distinction on something that is more of a continuum, vis-a-vis "written by human vs written by LLM". There's a middle ground of "written by human and LLM together". I mean, the people building stuff using something like SpecKit or OpenSpec still spend a lot of time up-front defining the tech stack, requirements, features, guardrails, etc. of their project, and iterating on the generated code. Some probably even still hand tune some of the generated code. So should we reject their projects just because they used an LLM at all, or ?? I don't know. At least for me, that might be a step further than I'd go.
- arscan 8mo ago
  > There's a middle ground of "written by human and LLM together". Absolutely, but I’d categorize that ‘bit’ as the innovation from the human. I guess it’s usually just ongoing validation that the software is headed down a path of usefulness which is hard to specify up-front and by definition something only the user (or a very good proxy) can do (and even they are usually bad at it).

exit 8mo ago

similarly, i think that something that someone took the time to proof-read/verify can be of value, even if they did not directly write it. this is the literary equivalent of compiling and running the code.

rubslopes 8mo ago

I agree with your sentiment, and it touches on one of the reasons I left academia for IT. Scientific research is preoccupied with finding the truth, which is beautiful but very stressful. If you're a perfectionist, you're always questioning yourself: "Did I actually find something meaningful, or is it just noise? Did I gaslight myself into thinking I was just exploring the data when I was actually p-hacking the results?" This took a real toll on my mental health. Although I love science, I'm much happier building programs. "Does the program do what the client expects with reasonable performance and safety? Yes? Ship it."

nicbou 8mo ago

It makes sense. A vibe-coded tool can sometimes do the job, just like some cheap Chinese-made widget. Not every task requires hand-crafted professional grade tools. For example, I have a few letter generators on my website. The letters are often verified by a lawyer, but the generator could totally be vibe-coded. It's basically an HTML form that fills in the blanks in the template. Other tools are basically "take input, run calculation, show output". If I can plug in a well-tested calculation, AI could easily build the rest of the tool. I have been staunchly against using AI in my line of work, but this is an acceptable use of it.

Brian_K_White 8mo ago
"A vibe-coded tool can sometimes do the job" How do you know it ever does the job? I don't know either for most code that I use, but I do have reason to trust that the author does know. I don't really trust any code itself, only the people and processes (organizational, not computer) that generated it. I have no reason to trust that ai generated code is doing the correct thing. I know enough about the way code works to know that merely observing it seem to work in a test case means absolutely nothing at all. Multiply that zero by a million more test cases and it's the same zero. The only thing I trust is that someone actually understood a problem they were trying to solve, and cares about avoiding edge cases, and tries to develop logic to make unintended outcomes impossible etc... It's not possible for an ai to do any of that regardless what the prompts are. But what they can do is emit stuff that some person once wrote which did exhibit these qualities, and so looks ok, and causes idiots to think they found the cheat code to life, and worse, foist that shit off on everyone else. My mom does not have my awareness that any of this is going on. She's just out there in the world running into this crap blindly as an unwitting end-user who has no idea how badly she's being served these days when she uses basically any app or service. Thanks for that vibe coders of the world.
- nicbou 8mo ago
  > How do you know it ever does the job? Because the part of the job it automates is simple, and can be tested. I cannot overstate how simple the tools I am thinking of are. Think tipping calculator. Neither new nor creative nor complex. The real value here is being familiar with the problem. You are missing the point here. I am talking about people who were not served at all by software developers. The alternative is not craftsmanship, but at best duct taping wordpress plugins together.

ethmarks 8mo ago

I guess it depends on whether you're only executing the code or if you're submitting it for humans to review. If your use case is so low-stakes that a review isn't required, then vibe coding is much more defensible. But if code quality matters even slightly, such that you need to review the code, then you run into the same problems that you do with AI-generated prose: nobody wants to read what you couldn't be bothered to write.

josephg 8mo ago
There’s lots of times where I just don’t care how it’s implemented. I got Claude to make a test suite the other day for a couple RFCs so I could check for spec compliance. It made a test runner and about 300 tests. And an html frontend to view the test results in a big table. Claude and I wrote 8500 lines of code in a day. I don’t care how the test runner works, so long as it works. I really just care about the test results. Is it finding real bugs? Well, we went though the 60 or so failing tests. We changed 3 tests, because Claude had misunderstood the rfc. The rest were real bugs. I’m sure the test runner would be more beautiful if I wrote it by hand. But I don’t care. I’ve written test runners before. They’re not interesting. I’m all for beautiful, artisanal code. I love programming. But sometimes I just want to get a job done. Sometimes the code isn’t for reading. It’s for running.
- layer8 8mo ago
  What is it about the tests that make you not care about how it’s implemented versus other code? Just the fact that it isn’t interesting to you? I happen to like spec-testing code and find it an interesting design space. It is vital to review it thoroughly for correctness. But I also think that whether I find something interesting or not doesn’t have much bearing on whether code quality is important. Code quality can even be more important for uninteresting code, because due to the lack of interest I’ll have a lower attention span when reviewing it.
  - josephg 8mo ago
    > What is it about the tests that make you not care about how it’s implemented versus other code? Huh this is a thought provoking question. I think there's a few reasons. In a test suite: - I don't care about performance. - I don't care (as much) about reliability. My users aren't affected by crashes and other failures in my tests. - I don't care (as much) about correctness. Erroneously failing tests will get human attention. Tests that erroneously pass are a bigger problem, but my test suite is not the last line of defence against bugs reaching users. - If I had infinite time, I'd love every line of code to be a mathematically beautiful work of art. But I don't. Writing this test suite by hand would have taken me about 3 weeks. Instead, I did it in 1 day with claude. This let me spend 14 productive days working on other things. I would rather have a good-enough test suite and 14 days of productive work than an excellent test suite and nothing else. I could spend those 14 days fixing all the bugs it found. Or writing more tests. Or getting claude to write more tests. Or going outside with my friends. These are all better uses of my time. If I was writing the core of a new game engine, the scheduler of an operating system or the data storage engine for a new database, then I would think hard about every line of code. But not all code is like that. We must adapt ourselves to the project at hand. Some lines of code matter a lot more than others. Our workflow should take that into account.

zahlman 8mo ago

> it can make real things happen even if no one reads it (it’s executable). "One" is the operative word here, supposing this includes only humans and excludes AI agents. When code is executed, it does get read (by the computer). Making that happen is a conscious choice on the part of a human operator. The same kind of conscious choice can feed writing to an LLM to see what it does in response. That is much the same kind of "execution", just non-deterministic (and, when given any tools beyond standard input and standard output, potentially dangerous in all the same ways, but worse because of the nondeterminism).

morgoths_bane 8mo ago

> I only care if it works Okay but it is probably not going to be a tool that will be reliable or work as expected for too long depending on how complex it is, how easily it can be understood, and how it can handle updates to libraries, etc. that it is using. Also, what is our trust with this “tool”? E.g. this is to be used in a brain surgery that you’ll undergo, would you still be fine with using something generated by AI? Earlier you couldn’t even read something it generated, but we’ll trust a “tool” it created because we believe it works? Why do we believe it will work? Because a computer created it? That’s our own bias towards computing that we assume that it is impartial but this is a probabilistic model trained on data that is just as biased as we are. I cannot imagine that you have not witnessed these models creating false information that you were able to identify. Understanding their failure on basic understandings, how then could we trust it with engineering tasks? Just because “it works”? What does that mean and how can we be certain? QA perhaps but ask any engineer here if companies are giving a single shit about QA while they’re making them shove out so much slop, and the answer is going to be disappointing. I don’t think we should trust these things even if we’re not developers. There isn’t anyone to hold accountable if (and when) things go wrong with their outputs. All I have seen AI be extremely good at is deceiving people, and that is my true concern with generative technologies. Then I must ask, if we know that its only effective use case is deception, why then should I trust ANY tool it created? Maybe the stakes are quite low, maybe it is just a video player that you use to watch your Sword and Sandal flicks. Ok sure, but maybe someone uses that same video player for an exoscope and the data it is presenting to your neurosurgeon is incorrect causing them to perform an action they otherwise would have not done if provided with the correct information. We should not be so laissez-faire with this technology.

aeturnum 8mo ago

> even if no one reads it I gotta disagree with you there! Code that isn't read doesn't do anything. Code must be read to be compiled, it must be read to be interpreted, etc. I think this points to a difference in our understanding of "read" means, perhaps? To expand my pithy "not gonna read if you didn't write" bit: The idea that code stands on its own is a lie. The world changes around code and code must be changed to keep up with the world. Every "program" (is the git I run the same as the git you run?) is a living document that people maintain as need be. So when we extend the "not read / didn't write" it's not using the program (which I guess is like taking the lessons from a book) it's maintaining the program. So I think it's possible that I could derive benefit from someone else reading an llm's text output (they get an idea) - but what we are trying to talk about is the work of maintaining a text.

williamcotton 8mo ago
Correct me if I’m wrong, but if you wrote a dependency-free recursive descent parser in C89 thirty years ago it should still compile and return the same AST.
- aeturnum 8mo ago
  I mean....it will compile and return the same AST on the OS and hardware from 30 years ago. But if you want to get the same result today on modern hardware / software you may discover you need to make some changes (or rather people have been making little changes for 30 years to ensure you can still get the same AST). Generally software has either had little bits and bobs added and removed to keep it relevant or its fallen away and been forgotten.
- pjmlp 7mo ago
  Well, if it made use of any UB alongside its code, and it gets compiled with the latest version of a modern compiler in -O3, it might, or might not.
deleted 8mo ago
[deleted]

panny 8mo ago

>but many tools I use daily are closed source I wonder if this is a major differentiator between AI fans and detractors. I dislike and actively avoid anything closed source. I fully agree with the premise of the submission as well.

layer8 8mo ago
I don’t think so. With closed source, it depends on whether you trust the authors. As someone who grew up with shareware (which is generally closed source), you learned to trust authors based on their documentation and reputation. AI-generated code you simply can’t trust as-is. You have to trust whoever reviews it. Of course, with open source you can be the one who reviews it. But typically few people do, and usually almost no one does as thoroughly as the original authors and maintainers — except if they are an AI.

onel 8mo ago

I am a software engineer, but I think code pales in comparison to the impact of every book, speech and essay had on our society throughout history.

jpfromlondon 8mo ago

>Whoa whoa whoa hold your horses, code has a pretty important property that ordinary prose doesn’t have: it can make real things happen even if no one reads it (it’s executable). When your boss (assuming you have one) tells you to do something, do you just ignore it?