10 ms·

I'm puzzled how anyone trusts ChatGPT for code

• delichon 3y ago
  I'm puzzled why anyone would incorporate code on trust. I certainly don't trust LLMs for code, but I use code from LLMs on most work days ... not because I trust it, but because I understand it and tested it. So, the same as code from anyone, including from myself six months ago. Trust doesn't enter into it. Clear and workable solutions do, and I've gotten an impressive number of those.
  • GenerocUsername 3y ago
    Agree. The snippets from ChatGPT are essentially open-source as in they are fully inspectable and can be freely modified before execution. There is zero trust required.
  • yevlasenko 3y ago
    I would back this one up as well. The code I generate in chat gpt is rather "dumb" (mostly react components, nestjs modules, etc) and in this case I trust myself to test and use it.
  • argc 3y ago
    Yeah, to use it effectively you use it for inspiration and generation of template code. The code may need a lot of changes or may need relatively few. But don't copy it expecting it to work without validating and testing it.
  • fassssst 3y ago
    Right, hopefully no one is sprinkling eval(prompt) into their codebase.
    • 1f60c 3y ago
      I had an idea about that the other day! What if you could write something like: @implement_this def prime_sieve(n: int) -> list[int]: pass And the decorator reads the function name and optional docstring, runs it through an LLM and replaces the function with one implementing the desired behavior (hopefully correctly). I remember there was something like this for StackOverflow.
      • deleted 3y ago
        [deleted]
      • adhamsalama 3y ago
        https://github.com/retrage/gpt-macro https://github.com/retrage/gpt-macro
      • matsemann 3y ago
        I made that as a joke in javascript 8 years ago. Not using LLM, but using javascript proxies so that if a function doesn't exist it tries to implement it runtime based on the name of the function using a sql like grammar. I really hope to not see something like that in real use, heh. https://github.com/Matsemann/Declaraoids https://github.com/Matsemann/Declaraoids Maybe I should make a LLM version of this.
        • LegionMammal978 3y ago
          The funny part is, Spring Data JPA has a quite serious take on this, in the form of query methods [0]. You create a repository interface with certain method names, and it dynamically creates an implementation that queries the columns of the underlying table, according to the name of the method. [0] https://docs.spring.io/spring-data/jpa/reference/jpa/query-methods.html https://docs.spring.io/spring-data/jpa/reference/jpa/query-m...
      • Nodejsmith 3y ago
        I've never tried it myself, but Prefect does have something like this with their Marvin AI library for Python. https://github.com/PrefectHQ/marvin?tab=readme-ov-file#-build-ai-powered-functions https://github.com/PrefectHQ/marvin?tab=readme-ov-file#-buil...
  • gordon_freeman 3y ago
    yes. LLMs are great at giving a template or code block for specific usecase and then I use it after understanding, modifying (if needed) and testing it. I am not sure about ChatGPT but Gemini Advanced even allows modifying code and running it within the answer itself which is convenient to test out minor changes. Also, getting template of 100s of LOC saves time in writing it from scratch even if I know exactly what I'd need to write.
    • chrisjj 3y ago
      > LLMs are great at giving a template or code block for specific use case and then I use it after understanding, I suspect the typical ChatGPT user is using it for code that's beyond his ability to write. That being the whole point of his using it. It follows that such code is likely beyond his ability to understand. But worse, beyond his ability to understand he doesn't understand it.
      • Brian_K_White 3y ago
        I can read and modify all kinds of code I couldn't write in the first place. I haven't used any llm for code yet but most code I ever touched was mostly written by others, and I wouldn't have been able to write it from scratch, but was able to modify it, take only chunks out of the whole to use for something else, take only the structure and not the verbatim code and use in some other language, refactor to satisfy my own priorities vs the original, get useful results in languages I don't even know, etc etc. It doesn't seem like using some pattern-matched uber-autocorrect code would be much different, especially once you understand that understandingless nature of what your getting.
    • repelsteeltje 3y ago
      > Also, getting template of 100s of LOC saves time in writing it from scratch even if I know exactly what I'd need to write. I rarely know exactly what I need to write and writing it usually isn't the problem. Sans LLM, there have always been tools and techniques you can lean on, like syntax highlighting, auto completion, your compiler feedback, or unit tests. I find that writing code consumes only a small percentage of my time. And while writing code is obviously where bugs originate, it's not where the big problems come from. Making software good is the easy part, making the right software is where the trouble lies. It's why I spend most time not coding, but talking to people. Figuring out what needs to be done, how, when and why.
  • burnte 3y ago
    > I'm puzzled why anyone would incorporate code on trust. Really? You can't understand why people do things based on trust? Do you trust no one?
    • simonw 3y ago
      Trust but verify. Code review exists because even the best developers make mistakes.
      • burnte 3y ago
        I totally agree, but let's say Jane writes the code, Frank reviews the code and checks it in. You're trusting both of them. Trust can't be eliminated.
  • 2OEH8eoCRo0 3y ago
    So you've audited all code that you're running on your system?
  • chrisjj 3y ago
    > I'm puzzled why anyone would incorporate code on trust. Because almost every programmer these days has learned by a route that relies on incorporating code on trust. E.g. using someone else's compiler. > I use code from LLMs on most work days ... not because I trust it, but because I understand it and tested it. The snag there is that one can rarely verify test coverage is sufficient. > the same as code from anyone, including from myself six months ago. But it is not the same. ChatGPT delivers completely untested code. Not even syntax-checked in my case. Would you accept such code from any human? > Trust doesn't enter into it. Clear and workable solutions do Plenty of clear and workable solutions are discovered unsafe. I suspect that's much more likely when the code comes from a source such as ChatGPT that delivers code in a completely untested state.
    • Brian_K_White 3y ago
      Syntactically incorrect code is not much different than an outline or pseudocode. There are way too countless many stories of people successfully accomplishing tasks by getting an llm to give them a starting point or outline or even a complete solution that only needed a few fixups, hardly any different from debugging your own first draft, to say it doesn't or can't work. That's already sailed. The fact that it doesn't produce finished correct code, or even appropriate outline for every single case, doesn't seem especially remarkable or damning to me. It is what it is, it's not what it's not. (I haven't used any myself, but I'm not primarily a coder who is obligated to take advantage of every available tool, and don't like how any of the llm companies are training on open source code without either consent or attribution and then selling that. And don't like how ai and even mere speech to text voice interface before that is being added to everything in general.)
      • dllthomas 3y ago
        > hardly any different from debugging your own first draft I find this take surprising. Leaving AI aside entirely, debugging my first draft seems very different than debugging your first draft.
        • 3y ago
  • zouhair 3y ago
    I am no programmer and I love how chatgpt can explain a bash script to me line by line, makes reading those way easier and more enjoyable.
• FaridIO 3y ago
  Quality very much depends on the language I've noticed. I don't use ChatGPT but I do use Github's Copilot. The main way I've been using it lately is to make manually written algorithms more efficient. So I'll write an algorithm, select it and ask it to figure out a more time/space efficient way of doing it. Still makes a mistake every now and then, but at the very least it helps me quickly see things from a different perspective.
  • dawnerd 3y ago
    I mainly use copilot for its ability to figure out patterns. It’s pretty nice when working with frontend code with tailwind. Not perfect but has sped development up a bit. One thing I like is if I put a rem size in css and wanted to comment what the equivalent pixel value is it does the math.
    • ptx 3y ago
      But the equivalent pixel value depends on the root element font size, so the comment will be wrong when that changes. If you leave the math to the browser dev tools you'll get accurate results without any AI figuring out patterns.
      • dawnerd 3y ago
        Yep but in our workflow we’ve never deviated from 16px base. The comments in the code are purely to help when translating designs to rem in particular with tailwind.
• mindcrime 3y ago
  I'm with @delichon... I don't "trust" LLM generated code, I review it and decide how appropriate (if at all) it is, possibly modify it, and include it if it makes sense. Trust isn't really part of the equation.
  • jaredsohn 3y ago
    I also don't trust code from stackoverflow, colleagues, or even code I had written myself in the past. Is helpful to always be skeptical of code and try to get an understanding of it. Another way I think of it is I find it easier to revise than to write (both in English and code). So when I write something I just try to get my ideas out there and then skeptically reread it and recompose. LLMs let me just speed up that process.
    • mindcrime 3y ago
      or even code I had written myself in the past. Aaawwww hellz no. I hate looking at my old code. I've had too many of those surreal experiences where something breaks, I start looking at how to fix it, and am suddenly stunned to realize "this could never have worked in the first place." There's one or two of those I'm still puzzling over, decades later. :-(
      • zer00eyz 3y ago
        I have been writing go for a decade now. Good go is like brutalist architecture. Strip out all the cute, the smart the stupid coder tricks. Write ugly, blocky, verbose, spelled out, functional code. Test the ever loving shit out of it. Go is fast enough that if your going back to fix a "performance problem" you either did something head smackingly stupid (we all have) or you have a real bottleneck and it's time to roll up your sleeves and write that elegant solution. I tend to revisit code more often to add features or deal with edged case bugs than anything else. That clear and functional code without any magic is a win. Edit: you can do the above with almost any language but those that choose packages over standard libs make it HARD.
  • nemothekid 3y ago
• xhkkffbf 3y ago
  Here's what I usually get from them: accurate function header accurate glue code wacky invocation of some API that may or may not exist accurate glue code It's rare for me to get anything really useful. I suppose the accurate glue code can be helpful.
• exe34 3y ago
  I don't trust code from stack overflow either - I always use the found code in a minimal working example - where I fix the obvious issues, convince myself it's doing what I want, and not doing anything that I don't understand, and then put that into my main project. I've had a lot of success with chat gpt where it told me what libraries to use, what classes/functions I need, etc, where it would have taken me ages to find out by reading documentation and even longer to go through tutorials that leave out the more interesting stuff.
• analyte123 3y ago
  Are you using GPT-4 or the free version?
  • chrisjj 3y ago
    3.5.
• pprotas 3y ago
  Sounds like GPT 3.5 to me, version 4 is much better at code. Also, as a general rule you shouldn't trust its code output, but it's a good way to brainstorm or get started on coding something. It can give useful tips or replace stack overflow. I almost never ask it to generate code for me, but I ask it questions about existing code very often. That seems to work very well for me.
  • bearjaws 3y ago
    GPT4 still sucks if you are using anything modern. It's knowledge of sveltekit still seems weighted to pre 1.0, it hallucinates libraries for shadcn, or radix-ui. You have to prefix your prompts with "Please use modern JS" otherwise it won't use .filter or .reduce sometimes. I agree its a good replacement for Stackoverflow, but it is often pitched as a replacement for coding. I've found using aider as the best compromise, let it build something small and revert if it doesn't work.
    • water9 3y ago
      So basically, you’re trying to use stuff that came later than the data set ChatGPT was trained on and complaining it doesn’t know things that are happening in the future..
      • iamacyborg 3y ago
        How is a user supposed to know what it is and is not trained on?
        • creesch 3y ago
          In this context, it is safe to assume it isn't trained on stuff that was not available when it was trained. OpenAI lists that cutoff date next to each model: https://platform.openai.com/docs/models/continuous-model-upgrades https://platform.openai.com/docs/models/continuous-model-upg...
      • not2b 3y ago
• rhelz 3y ago
  Treat it like the best educated, most well-informed intern in the world. Usually gives good answers, but still, not a seasoned professional yet, so check the work accordingly.
  • chrisjj 3y ago
    But interns learn from corrections. This product does not.
    • rhelz 3y ago
      Have you heard the expression "Hire a teenager while they still know everything" :-)
• fwlr 3y ago
  Mismatched brackets seems like a lower-level job - better done by a linter, not an LLM. Likewise, I wouldn’t rely on an LLM to tell me what properties are on the response object - that job is better done by e.g. TypeScript. Analogously, “convert this function from Python to lisp” seems like a higher-level job, better done by a programmer human (or at least, it’s too high for the current level of available code LLMs). There is still a valuable level that code LLMs can operate at, in my experience - something like a cross between “a more advanced autocomplete” and “a templating/scaffolding/boilerplate engine”.
  • chrisjj 3y ago
    > Mismatched brackets seems like a lower-level job - better done by a linter, not an LLM. Mismatched brackets is not any /job/. And /matched/ brackets are part of the job this LMM claimed to do. Writing runnable code.
• j45 3y ago
  A key is to not let it code something you don't understand and can't code. GPT 4 is much better than the others. You should try something like Aider to get better output
• jgb1984 3y ago
  ChatGPT, when I was forced to use it often got confused which is the bigger number, 2 or 3. I don't care how many lines of bottom of the barrel code it can throw up, I'm not using any of it. AI generated spaghetti debt. LLM's will turn out to be nothing more than another glittering gadget. Except this one will obliterate any coding skills of future generations. Not to mention the gigatons of AI generated garbage text, images and music that it's filling the world with. What a waste.
  • groby_b 3y ago
    You... might want to inform yourself a bit more before making proclamations about the future. LLMs have fairly well understood failure modes. They have incredible strengths, too. And they differ very much by what you use. If the ChatGPT you were forced to use was GPT3.5 (and the behavior sounds very much like it was), you literally have no idea what modern LLMs can actually do.
  • superfrank 3y ago
    > ChatGPT, when I was forced to use it often got confused which is the bigger number, 2 or 3 I mean... yeah. ChatGPT doesn't know what numbers or what logic is. It wasn't "confused". It just doesn't do that. You're using it for something it's not intended to be used for and, unsurprisingly, it's bad at it. If I said I'm giving up on using hammers because they were terrible for screwing screws into the wall, you'd tell me I'm using the wrong tool. This is no different. ChatGPT has its uses, but it isn't magic. Just like any tool it has limitations.
  • sdenton4 3y ago
    User: which is the larger number, 3 or 2? ChatGPT: The larger number is 3. User: If I have one apple, pick another apple and then get an apple from a friend, and Alice has two apples, who has more apples? ChatGPT: You have more apples. You started with one apple, picked another, and got one more from a friend, so you have three apples. Alice has two apples. [edit]: User: what does good will hunting say to the know-it-all at the bar? ChatGPT: After Will Hunting outsmarts the know-it-all student in the bar, he spots the student's female friend outside a shop later on. To tease the student further and show his wit, Will taps on the window to get her attention and, holding up his phone number, famously asks, "Do you like apples?" After she nods, Will triumphantly says, "Well, I got her number. How do you like them apples?" This line became quite iconic and is one of the memorable quotes from the movie.
• bretpiatt 3y ago
  Did you try to use GPT 3.5? Our testing is it isn't great, using GPT 4 or some of the specialized trained versions of GPT 4 (there's one with good reviews called Lisp Engineer) our experience has been different. It is not replacing engineers, it isn't one where you give it a broad set of requirements and it just goes and builds, it is helping increase productivity, to get folks through areas where they need to bounce ideas off of someone. We're coding mostly in Python, C++, and .NET Core where I do expect it'll have a much deeper set of training data than it will for Lisp (and even for those languages we're getting marginally better performance from specialized engines than we are general GPT 4). The non-OpenAI other coding AIs so far are all performing worse for us than GPT4. We've done testing against LeetCode challenges and a bunch of other things.
  • chrisjj 3y ago
    > Did you try to use GPT 3.5? Yes. And succeeded! :) > We're coding mostly in Python, C++, and .NET Core where I do expect it'll have a much deeper set of training data than it will for Lisp I can't imagine how malformed bracketing is due to insufficient training set. And nor it seems can ChatGPT: "I've encountered numerous examples of Lisp code during my training, covering different applications and techniques within the language. Whether it's simple examples or more complex implementations, I've seen quite a bit of Lisp code."
    • lolinder 3y ago
      You tried the very first iteration of an LLM-based chat assistant, were unsatisfied with it because it couldn't match Lisp parentheses, and went on to form an opinion about the value of these tools and implicitly the intelligence of the people who use them. That speaks more to your preconceptions than it does to the state of better tools like Copilot or GPT-4. You didn't label it (which, btw, is a faux pas), but it's obvious from your replies that this wasn't an Ask HN, it was a Tell HN. You have absolutely no interest in what the rest of us have to say. Nevertheless, I'll try once more for luck: Basing your opinions about LLMs on your experience with GPT-3.5 is a mistake. If you don't want to use LLMs at all because you have preconceptions, that's fine, but don't pretend that you've sampled LLMs and found them lacking for professional coding when you haven't tried the professional tools.
• cirrus3 3y ago
  I don't think anyone who has used it frequently has any assumptions it is correct. You almost always have to edit it either for bugs or just plain incorrect... but it can still be a very helpful tool to get you a nice starting point.
• toddmorey 3y ago
  I think of it as not a solutions engine but a suggestions engine. Sometimes what it suggests won’t work at all. But other times it opens my thinking to an approach I had too much tunnel vision to see. I end up writing the code ultimately. There was only once a tiny function I could paste in directly, unedited. But still helpful. One great technique can be to ask it for options: “List five ways this code could be improved / more readable.” Usually about 1 in 5 suggestions will lead to something interesting.
• m12k 3y ago
  I find it to be a great way to look up syntax or APIs, and generate simple examples of using these. So for example "In [language] how do you parse JSON?" will give me a usable example to follow much faster than trying to find the anointed Stack Overflow question and then sift through outdated answers to find the modern one. If you need to revive a language you haven't coded in for a while (or regex or sql), ChatGPT can really help get you off the ground much faster, though you'll still need to understand the output it gives you and compose it together yourself.
• simonw 3y ago
  It's more reliable for code than it is for anything else. If it hallucinates something, the code won't run - or it won't work correctly when you test it. Much easier to "fact check" code than prose.
• Sohcahtoa82 3y ago
  If you're expecting to just copy/paste from ChatGPT without understanding what it generates, you're really doing it wrong. I've only used it for basic snippets where I know what the code I need is, but it's faster to describe it than write it out myself (for example, "Using Python and boto3, take all the files in the current directory that end in .csv or .pdf and upload them to a bucket named "reports". Prepend the S3 keys with "YYYY-MM-DD/" using today's date"). For simple things like that, it's been great.
  • NewsaHackO 3y ago
    Yep. OP had to ask it fix mismatched brackets, and is a perfect example of the type of person who definitely should not use LLM for code.
    • peteradio 3y ago
      I've seen self described "product owners" claim to build using LLM outputs without knowing how to code.
    • boredtofears 3y ago
      I’m sure most people can learn to use LLMs correctly if they can code, it doesn’t matter what “type” of person you are.
    • chrisjj 3y ago
      Actually I didn't have to. I chose to. So I could give it another chance to succeed. But do tell. What type of person does not have to ask it to fix its mismatched brackets?
  • paulmd 3y ago
    It’s also constrained by how many examples it’s seen of a particular language or data format, and the general complexity of the format (even without AI you’ll see diff breaking on json files for much the same reason, unless you go out of your way to avoid it - brackets are semantic white noise and there’s nothing unique for it to lock onto between those bracket tokens). At a minimum it’s going to absolutely chew through tokens to maintain that much context. Obscure languages and operating systems are also a weakness - for example I could not get llama3 to reply with FreeBSD commands even after repeatedly telling it not to - there just aren’t that many SO/Reddit/forum posts about BSD in comparison to Linux. And evidently that’s what they’ve trained it on. There isn’t even anything semantically odd about an rc.conf, it spits out correct statements, just for the wrong file in the wrong OS. Everyone acknowledges things like multiplication and counting are hard for LLMs because they aren’t statistically encoded. But counting lisp brackets is also basically the same problem. And unlike YAML it’s not really self-healing or obviously correctable once it’s fucked up (as you noticed). I think the implication is there’s clearly some syntaxes that are much less generateable simply because they require intense amounts of context. (also, as far as counting… I’m sure far smarter people have look at this, but maybe multiplication is something that could benefit from log-addition techniques, that might reduce the complexity quite a bit vs a naive “multiply column by column like a human fourth-grader” approach.)
• furyofantares 3y ago
  I read the code. And I would have just added the bracket myself if the code was otherwise correct.
  • chrisjj 3y ago
    As I said, it wasn't.
• DSpinellis 3y ago
  ChatGPT can generate wrong code but it can also be a great helper. I've summarized the good, and bad, and the ugly in a recently published IEEE Software column: https://doi.org/10.1109/MS.2024.3363848 https://doi.org/10.1109/MS.2024.3363848
• BugsJustFindMe 3y ago
  I trust ChatGPT as much as I trust my co-workers. That is to say, I don't need to trust them, because I have eyes and a brain. "This code doesn't do what you say it will" is not a review comment reserved exclusively for machine-generated code.
  • chrisjj 3y ago
    > I trust ChatGPT as much as I trust my co-workers. That is to say, I don't need to trust them, because I have eyes and a brain. ... and 100 hours a day to review and test their code? :)
    • deleted 3y ago
      [deleted]
    • BugsJustFindMe 3y ago
      It takes them longer to write the wrong code than it takes me to look at it and tell them where it's wrong. And yes, reviewing code is part of my job. And the reason it's part of my job is that people who get hired as programmers are often very bad at not making obvious mistakes.
      • chrisjj 3y ago
        It had not ocurred to me that anyone would hire a programmer who submitted code that fails even to compile, as per ChatGPT.
        • BugsJustFindMe 3y ago
          > It had not ocurred to me that anyone would hire a programmer who submitted code that fails even to compile, as per ChatGPT. I would say that this points more to the narrowness of your industry experience than anything else. Failing to compile is one of a million different ways that code doesn't produce the claimed or desired or correct behavior. It's not particularly special. All code needs to be reviewed, regardless of who/what wrote it and how.
• superfrank 3y ago
  I trust it in the same was I trust a junior developer or intern. Give it tasks that I know how to do, can confirm whether it's done right, but I don't want to spend time doing it. That's the sweet spot.
• deleted 3y ago
  [deleted]
• boredemployee 3y ago
  i feel sorry for those it doesn't, for me it works like a charm
• jasonjmcghee 3y ago
  As others have mentioned - code should rarely involve trust. This is what code review is for. Even your mentor / cracked engineer makes mistakes. Code review and tests are always critical. Whether it's a human or an AI, code and the code that tests it needs to be reviewed / validated.
• shinycode 3y ago
  Our intern uses copilot extensively and its code is riddled with errors. What a pain to review, a lot of time wasted. This is really concerning for newer generations that are not professionals yet but they trust it because the code looks better than what they are able to do. Worse, they outsource their brain and don’t sharpen their senses. How will they become professionals this way ? As a help it’s okay, as a cheating tool that make them faster …
  • zamadatix 3y ago
    An intern using LLMs to generate bad untested code is no different than an intern using Stack Overflow to generate bad untested code in that the problem isn't Stack Overflow or the LLM or whatever tool the intern is using rather the lack of application of quality control and code review by the intern on their own code. Making them write everything from scratch isn't an instant cure either, you can write some really badly designed stuff right from scratch and push it immediately on up too. This doesn't make you mentally sharper along the way, it makes you confidently sloppy. A better path is to ensure they are spending significantly more time reviewing, testing, and incrementally improving their code than they are typing/generating it. That's where you really sharpen your brain/senses... and also where you keep from driving everyone else nuts with your PRs. Be it LLM or senior dev if you just say "one shot this functionality so I can push it to main" you're going to have a bad day.
    • robryan 3y ago
      At least with stack overflow you can be fairly confident the code at least works correctly in the context it was provided in if it has a lot of votes.
      • chrisjj 3y ago
        Absolutely. Yet where's the "AI" that can do such voting?
    • shinycode 3y ago
      The main difference with SO is that the code from SO cannot always be c/p, the context is different so we had to rethink it. With copilot you have the sense that it’s contextualized and it’s the right fit. With pair programming I saw him accept multiple times the autocompleted code without reading it, he just said « wow it got it fast ». That for me is even worse than SO in that regard. For the rest I agree
• chx 3y ago
  Every time... https://hachyderm.io/@inthehands/112006855076082650 https://hachyderm.io/@inthehands/112006855076082650 > You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to. > Alas, that does not remotely resemble how people are pitching this technology.
  • chrisjj 3y ago
    > I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” Great. Solved the world shortage of bullshit. Oh wait...
• jokethrowaway 3y ago
  1. I don't trust code from humans too 2. You can't trust LLM for anything. They are a Large Language Model, you could consider LLMs having knowledge to be a bug. Their architecture is not meant to reason. 3. ChatGPT and co really needs a feedback loop with compilers to drastically improve(http://devin.ai/ http://devin.ai/ is working on that) 4. If you pick a popular language you get good results because they trained it more with those. JS and Python are ok, Rust or Haskell are never correct
• TachyonicBytes 3y ago
  In an ironic twist, the original Lisp paper has mismatched parentheses.
• wolpoli 3y ago
  For context, ChatGPT doesn't seem to be very good at less popular languages. In my experience, it's able to generate good C# code, but it is very poor at generating Autohotkey code. So it's a similiar situation here with LISP.
• alkonaut 3y ago
  Try making a 200 line chunk of boilerplate for something mundane like a big table in a react page. This is where it saves you googling parameters and doing copy paste. And you don’t need to trust it just test it after it’s done.
  • onkarshahi1 2y ago
    creating doctor dashboard to see their patients with appointments for new consultation or follow-up or review test reports. Doctor should be able see the complete timeline of patient's encounters, symptoms, doctor observations, medication - current or past and test reports for diagnostics or radiology. They should be able to compare reports to see the progress of medication.
• jeroenhd 3y ago
  It works fine for C# and Java. Usually works fine for Rust. The context window is too small for anything more than a quick demo and it often just assumes that certain types and classes exist, but it's good enough that you can get a working stub in no time. This is especially useful when dealing with (open source) libraries that lack decent documentation. Obviously you inspect and validate the bullshit it generates, but generally it produces some very useful bullshit. The output is as good as the data set available on the input side. I don't think many people use Lisp, so the Lisp output will be worse. Other uncommon languages have the exact same problem.
• clarkdale 3y ago
  Sometimes I forget a parenthesis here and there.
• speak_plainly 3y ago
  I’ve had great success with ChatGPT 4 writing some simple Python scripts and it has really helped speed up my workflow and helped me address issues as they cropped up. I did have trouble when I tried to get it to integrate and combine all these scripts into one larger one but I’m sure with some work I could get things working. Perhaps there are issues with the code (and I’m a certain kind of user) but I really just care about the output and having this assistance has been a godsend for me.
• jghn 3y ago
  In the before times, would you copy/paste something directly from StackOverflow or the like? No of course not. You'd look it over, assess if it does what you think is the right thing, and consider if that's really what you want. But more likely you look at it, decide it's not quite what you need, and modify it as appropriate. LLMs are no different.
  • itronitron 3y ago
    There are a lot of comments equating stuff posted on SO with the stuff that ChatGPT produces, but in the before times code posted on SO was likely written with a specific objective in mind, while the GPT generated stuff is not. The best analogy I can think of is the distinction between aiming a gun (SO, spitball examples) and waving a gun (AI/ChatGPT). Both require you to double check where the gun is pointed but with the former you at least have some confidence that it's pointed in the right direction.
    • jghn 3y ago
      Sure, these differences exist. I don't find the quality to be much different between the two sources. I treat both of them as a probably wrong, likely inexperienced developer. Sometimes they pleasantly surprise me. More often than not they don't but in both cases I make use of those resources because they at least move me in the right direction.
  • chrisjj 3y ago
    > In the before times, would you copy/paste something directly from StackOverflow or the like? No. I'd first check the votes and comments from other users. There's no such option with ChaptGPT.
  • chrisjj 3y ago
    > In the before times, would you copy/paste something directly from StackOverflow or the like? No. I'd first check the votes and comments from other users. > LLMs are no different. One difference is no votes and comments from other users.
• stephc_int13 3y ago
  From my experience, the more ones learn about how the sausage is made, the less you want that stuff in your codebase. I think that this tech could be useful at point, with a lot of well-engineered guardrails around it, maybe. And this will mostly be for boilerplate and the most generic parts of your code.
• rickcarlino 3y ago
  From “All You Need is Cheating”: https://sourcegraph.com/blog/cheating-is-all-you-need https://sourcegraph.com/blog/cheating-is-all-you-need One of the craziest damned things I hear devs say about LLM-based coding help is that they can’t “trust” the code that it writes, because it “might have bugs in it”. Ah me, these crazy crazy devs. Can you trust code you yeeted over from Stack Overflow? NO! Can you trust code you copied from somewhere else in your code base? NO! Can you trust code you just now wrote carefully by hand, yourself? NOOOO! All you crazy MFs are completely overlooking the fact that software engineering exists as a discipline because you cannot EVER under any circumstances TRUST CODE. That’s why we have reviewers. And linters. And debuggers. And unit tests. And integration tests. And staging environments. And runbooks. And all of goddamned Operational Excellence. And security checkers, and compliance scanners, and on, and on and on! So the next one of you to complain that “you can’t trust LLM code” gets a little badge that says “Welcome to engineering motherfucker”. You’ve finally learned the secret of the trade: Don’t. Trust. Anything! Peeps, let’s do some really simple back-of-envelope math. Trust me, it won’t be difficult math. You get the LLM to draft some code for you that’s 80% complete/correct. You tweak the last 20% by hand. How much of a productivity increase is that? Well jeepers, if you’re only doing 1/5th the work, then you are… punches buttons on calculator watch… five times as productive.
• deleted 3y ago
  [deleted]
• aranchelk 3y ago
  Big picture, IMO most devs (especially more junior ones) are far better at writing code than reading it, that’s part of the allure of green field projects. No, you don’t trust any LLM output, just as you shouldn’t trust code you get from Google, SO, etc. You read it, understand it, and decide to use it, alter it, or toss it. Before LLMs put everyone out of the job, I think code literacy along with (obviously) LLM-specific skills are going to be very important.
• elicksaur 3y ago
  People nitpicking the word “trust” are missing the point. There’s an obvious and growing dichotomy in experience using LLMs to assist in programming. One camp claims it 2-10x’s their productivity. The other claims it is trash and useless. A much more interesting thread to pull on would be how these subjective experiences can be so far apart while using the same products.
  • chrisjj 3y ago
    The difference is surely QA. High productivity is easy where the quality bar is low.
• 77pt77 3y ago
  I asked it to write code to calculate the arithmetic average of a list of ints in OCaml. It used only integer operators. Nowhere were floats involved. I asked "why is there no +. or /. operator" It completely bullshitted me saying + is polymorphic in Ocaml. Surprised, I asked since when. It game me a caml version over 20 years old. I fired a repl and obviously + only works on ints. When I called it out zero fs were given. I'll pass, thank you very much.
• mst 3y ago
  There's a long tradition of sharing illustrative code snippets on IRC by typing it straight into a pastebin linked to the channel. The odds of them containing at least one stupid and one subtle mistake are very high, but they're immensely effective at illustration a general shape to another developer in a way they can pick up, cross reference with the docs, and turn into working code in their project. Treating LLM generated code the same way has been working out well for me so far.
  • chrisjj 3y ago
    I guess the problem then could be ChatGPT's success at encouraging treating its output as more.
    • deleted 3y ago
      [deleted]
• certkindausr 3y ago
  took a bit of trial and error to have "G" write a local server with live reload. it works and all security issues were taken care of. faith can be replaced by iterating until you don't need to trust anymore. "what security issues might become an issue?" "how can i test that fix?" "explain this part of the code." "browse the net for security issues for this part." "how does lisp use brackets and indentation?" "use what you just taught me and fix your mistakes in your translation of these 12 lines of python code."
• lolinder 3y ago
  FWIW, I'd expect mismatched parentheses to be particularly likely to occur in Lisp. There are very few forms of writing where two tokens () account for 30-50% of the total token size of the text. It's unsurprising that with even a tiny bit of temperature the model gets overzealous and places one too early or too late. Add to that the fact that matching parentheses is hard for humans to reliably do correctly—especially when typing code into a textbox on Stack Overflow—and now you have a model whose architecture struggles with knowing when to stop in highly repetitive sequences and that was trained on data that is disproportionately likely to have incorrect syntax for the language you're using. Other languages don't have those features, and I've never found Copilot to give me a syntax error in, say, JavaScript. That said, I also don't know any professional developer who trusts LLMs to code. Everyone I know who uses them uses them as a tool and takes responsibility for cleaning up their output.
  • Leherenn 3y ago
    Copilot likes to swallow my } in C++ for some reasons. E.g. if you generate a new function at the end of a namespace block, it will usually use the bracket of the namespace block at the terminator of the function, leaving you one short.
  • chrisjj 3y ago
    > FWIW, I'd expect mismatched parentheses to be particularly likely to occur in Lisp. FWIW, this example had one more ) than ( ! I'll post it as a comment to the root.
    • lolinder 3y ago
      Yep, that's entirely unsurprising. Generative AI in general has a hard time knowing when to stop highly repetitive patterns—image generators struggle with getting the right number of fingers, and transformers will often get stuck in loops if things get too repetitive. Lisp is just a uniquely hard language for a transformer to cope with—not that it doesn't make mistakes in other languages, but I've almost never seen a syntax mistake.
• jug 3y ago
  Key is to not trust it, but use AI as an assistant that you don't trust. Even then, it can save you hours of work.
• lejohnq 3y ago
  I wouldn’t say I trust ChatGPT without verification yet, but it definitely accelerates me by generating good enough code for me to quickly debug. I am a much faster debugger than a from scratch generator, so ChatGPT/etc helps accelerate me through those moments even when I know exactly what I want to write. This makes it useful enough a tool to use anytime I’m writing code
• Harmohit 3y ago
  I am not a LLM lover but I think you are thinking about it the wrong way. I think that most of the value that LLMs provide for writing code is in syntax suggestions and being slightly better autocomplete.
• renewiltord 3y ago
  You don't have faith or trust or anything. You use it like a tool. I don't trust my IDE and select the first autocomplete solution. It's just a tool. When Google came out a lot of people were like "How can you trust Google? I searched for X and it gave me crap". Yeah, you can't trust Google like that.
  • chrisjj 3y ago
    > I don't trust my IDE You trust it not to prang your program, right? > and select the first autocomplete solution. I have never seen an IDE mess up Lisp brackets.
• notatoad 3y ago
  Don't have faith in LLMs. don't trust them. treat them like a new hire - you've got to take some time to judge what their capabilities are, how they're going to fit into your workflow, and how you can make their output useful. they're going to be bad at some things, and they're going to be good at some things. and maybe they're not going to be a good fit at all. but sometimes they will be really helpful, and ignoring that and deciding to do everything yourself instead is going to make you less productive in the long run.
  • chrisjj 3y ago
    > Don't have faith in LLMs. don't trust them. > treat them like a new hire I expect a new hire to learn post-hire. These LLM's learn nothing post-hire, right?
    • fragmede 3y ago
      there's fine tuning happening in some systems, so without knowing which system you're using, a "yes" answer could be incorrect.
• calderknight 3y ago
  What model were you using? What prompt did you use? You can learn to trust it for some tasks that it's reliably good at. But for the most part, you don't trust it - you read through it and check it.
  • chrisjj 3y ago
    ChatGPT. Translate the following Python program to Lisp. Yes, I forgot to add "... and no crap." ;)
    • calderknight 3y ago
      ChatGPT is just the brand. I guess you're using GPT-4, but if you're using the default model (GPT-3.5) that would certainly explain below-expectation results.
      • chrisjj 3y ago
        > ChatGPT is just the brand. "ChatGPT" is a model, says its provider. But yes, the web page says model GPT-3.5.
        • calderknight 3y ago
          Mystery solved! The answer to your OP is that the reason anyone can trust ChatGPT for code is that they use a much better model than the one you're using! GPT-3.5 is ancient and way behind GPT-4. In fact, there are now tens of organisations who have developed model classes way ahead of GPT-3.5.
          • chrisjj 3y ago
            > the reason anyone can trust ChatGPT for code is that they use a much better model than the one you're using! GPT-3.5 is ancient and way behind GPT-4. Testing that theory now using Bing Copilot GPT-4. Failed. unmatched close parenthesis Line: 11, Column: 11, File-Position: 408 Input is: Translate this to Common Lisp. def solve(graph): a = 0 while a < len(graph): for b in range(len(graph)): if b > a and graph[a].intersection(graph[b]): graph[b] = graph[a] | graph[b] graph.pop(a);a-=1 break a+=1 return graph a = [{1, 2}, {4, 5}, {1, 3}, {1, 4}] print((solve( a ))) Output is: (defun solve (graph) (let ((a 0)) (loop while (< a (length graph)) do (loop for b from 0 below (length graph) do (when (and (> b a) (intersection (nth a graph) (nth b graph))) (setf (nth b graph) (union (nth a graph) (nth b graph))) (setf graph (remove-if (lambda (x) (equal x (nth a graph))) graph)) (decf a) (return)))) (incf a)) graph)) (let ((a (list (list 1 2) (list 4 5) (list 1 3) (list 1 4)))) (format t "~a~%" (solve a)))
• itronitron 3y ago
  IDEs used to be very capable development environments. They could do magical things (by today's standards) because they understood the structure of the language as well as available resources. The current batch of AI doesn't seem to take this into consideration, whether out of laziness, incompetence, or some strategic or cynical view.
• andrewstuart 3y ago
  You're holding it wrong. ChatGPT does not do your job for you. It is an assistant - you have to do the debugging and adapting and understanding and verification and fixing. And, ChatGPT is better at coding in programming languages that have more training material, so the more popular the languages, the more likely ChatGPT is to give a meaningful result.
• deleted 3y ago
  [deleted]
• SkyPuncher 3y ago
  You basically picked the worse possible task for ChatGPT and are applying it as a broad conclusion. ChatGPT is _very_ good at taking human-language descriptions and transforming that into code. It works even better when instructions are concise and clear. I theoretically believe this stems from the fact that there’s a lot of human description of code for the LLM to train on. However, there’s relatively very little machine to machine code available. The tasks you’re looking to accomplish will likely have much better success if you break it into two “human-to-code” steps. 1. Ask ChatGPT to explain what the Python code is doing, line-by-line 2. Then ask ChatGPT to write the equivalent Lisp code for the description.
  • ssl-3 3y ago
    Yep. It often "reasons", as much as it does, best with regular language. And sometimes, just like with real people, it works better if one or more intermediate steps are interposed. But it doesn't necessarily "know" this -- just as real people mightn't. It's not unfair to say "Yeah, well if the bot was so smart then why didn't it just take care of that intermediate step for me?" But it's also not unfair to retort "Well, if you were so smart then why did you expect perfect results from an imperfect tool without expending any real effort to understand and/or explain the thing on your part?"
    • Jensson 3y ago
      > But it's also not unfair to retort "Well, if you were so smart... But it was the tool being evaluated, not the person. If the person has to code in natural language then why not just code it directly?
  • ptx 3y ago
    Like XMLVM[0] with English instead of XML. [0] http://www.xmlvm.org/overview/ http://www.xmlvm.org/overview/
  • chrisjj 3y ago
    > You basically picked the worse possible task for ChatGPT Wow, what were the odds on that?? :) I'd be interested in verification, so will start a new thread for that. > The tasks you’re looking to accomplish will likely have much better success if you break it into two “human-to-code” steps. > 1. Ask ChatGPT to explain what the Python code is doing, line-by-line The answer is "However, there's a logical flaw and a few issues in the implementation:" followed by points every one of which is in error, showing ChaptGPT has deeply mis"understood" the code. > 2. Then ask ChatGPT to write the equivalent Lisp code for the description The result fails with "odd number of &KEY arguments".
• icedchai 3y ago
  Do you trust yourself or your coworkers to write correct code, without being tested or even run? Do you really expect an LLM to do better?
  • chrisjj 3y ago
    I do not deliver untested code. This "AI" does.
    • icedchai 3y ago
      I really don't see the problem. It's a tool, not a coworker. It's up to you to test it. If you find a solution from googling, you still need to test it. It's the same thing.
      • chrisjj 3y ago
        The coworker comparison is not mine. I too consider it invalid.
• tardisx 3y ago
  Using LLM's for software development is cargo cult programming, automated. The problem is not "incorrect" (syntactically) code. The problem is the developer trusting the AI to understand the nuance of the problem space, and to write the code the "right way". As someone wise once said "writing the code was never the hard part". A colleague recently used an LLM to complete some work, which required some concurrency - which they were clearly unfamiliar with. The LLM created a byzantine mess of multiple channels (which immediately deadlocked) when all that was needed was a single mutex. A two minute human conversation would have turned up "you just need a mutex". Even in the unlikely scenario that the AI generated solution worked, did the developer learn anything? No. Was the codebase made more or less maintainable? Definitely less. I keep hearing that the LLM's will get better at understanding nuance and writing code that fits the problem space. It sounds an awful lot like how the crypto bros kept telling us that blockchain was the future of money and we'd soon all be rich. The half-on-the-fence AI users tell me "I agree, but I just use it to autocomplete functions or write small things". I can do that myself - and I get the opportunity for my own brain to do it's own concurrency and interrupt me "actually, maybe this is not the right way to do this".
• nforgerit 3y ago
  The best thing I can say about evaluating LLMs for code is that they reminded me of the infinite monkey theorem.
• deathanatos 3y ago
  > I wonder how much user faith in ChatGPT is based on examples in which the errors are not apparent ... to a certain kind of user. I think this is it, basically. A certain segment of programmers are simply not skeptical enough. The same sort of skepticism that will keep jepsen.io going until the heat death of the universe debugging faulty distributed consensus algos. My coworker recently generated a bash script with it. The shell was terrible, full of novice shell anti-patterns. But this makes "sense" — what was it trained on, after all? Most shell in its training set would be of low quality, because most shell is of low quality. The script was "determine lines added/removed by author in git repo". The `git log` call it made was bizarre — emitting way more information than it needed to — which it then subsequently filtered out extraneous bits with `grep`. … but it could have just never outputted them in the first place. Things like `| grep | awk`, lack of quoting where needed, etc. Multiple loops through the log output where one would do (and having to slurp it into memory because of that, where a simple git log | <process output> would suffice). The regexes to match the log output were wrong, resulting in the wrong output — but that didn't stop multiple people from going "nice" or "cool script" first.
• deleted 3y ago
  [deleted]
• minikomi 3y ago
  Which model, which code. We're going to need receipts before going forward.
• deleted 3y ago
  [deleted]
• paradite 3y ago
  One thing that matters a lot is the prompt. A good prompt can dramatically improve the performance and quality of the code output from ChatGPT. It took me months of trial and error to optimize the prompt for using ChatGPT to write code. I wrote a blog post summarizing the best practices: https://16x.engineer/2024/02/03/chatgpt-coding-best-practices.html https://16x.engineer/2024/02/03/chatgpt-coding-best-practice...
  • chrisjj 3y ago
    Thanks. My prompt followed all the best practice offered there. However that practice appears to assume program generation not translation.
• GregDavidson 3y ago
  Thanks to influence from Dijkstra, Wirth, et al in my formative years, when I program I construct a (usually informal) correctness proof simultaneously (well, kind of interleaved) with my construction of the code. The two constructions assist one another and converge to procedures which solve the required problem. I find this approach more productive than an ad hoc (hack & debug until done) approach. The correctness proof supports the correctness of the solution and the solution is usually more elegant (simpler, etc.). I annotate the code with elements of the proof to assist with maintenance. I think that most programmers do this to some degree, i.e. have an internal argument about why what they're doing will work and include some assertions and comments, etc. When I'm given some hunk of complex procedural code lacking strong types, preconditions, postconditions, invariants, arguments bridging such, etc. I don't find it very useful. Proving the correctness of complex ad hoc code is often harder than just solving the problem again from scratch. There are AI automated reasoning systems (based on incremental theorem provers) that can help me write good code and I follow the evolution of such tools. So far, the code I'm seeing from llm systems seems like a maintenance and reliability nightmare.
• zzz999 3y ago
  [dead]
• petabyt 3y ago
  The utility of chatgpt is to spit up common code that it was trained on. It will write a mandelbrot renderer just fine and can also explain it. Try to get it to generate any Brain*ck and it will fail miserably.
• fragmede 3y ago
  You're holding a hammer, and hit your thumb. do you swear off hammers as the devil, or do you learn to use the tool better? LLMs work on tokens. I wouldn't expect them to be good at matching parens. Even the very best hammer is a terrible screwdriver.
  • chrisjj 3y ago
    A parens is a token, last time I looked. And even Wikipedia agrees.
• cnotv 3y ago
  If you are used to planning your code, and writing what you need to do, with lists and everything, it usually does a good job of creating simple functions, RegEx rules, and finding the snippet of documentation you need. Even if I could code everything with not a single issue, it saves time. If you have not understood yet where it helps, maybe you should play around with it a bit more. To write unit tests is often perfect and way better than some of my colleagues, especially the old ones who don't trust and have never written tests.
• rokkitmensch 3y ago
  This is a very Lisper objection. The thing is a token predictor, and can't count levels of depth. Think "autistic junior engineer, whose work needs lots of testing but is also prolific at writing tests" instead of "Godlike text generator", it's much more productive.
  • chrisjj 3y ago
    > This is a very Lisper objection. Why thank you! :) > The thing is a token predictor, and can't count levels of depth. Er, the average C program has more depth than this small Lisp one. If it can't handle levels of depth, then why is it even claiming ability to code?
• yencabulator 3y ago
  It's funny how many LLM enthusiasts think they can notice bugs by reading code. More likely you're just nodding your head along and missing the bugs. When someone posted LLM-generated elisp on HN, it used a non-existent escape code in a regexp. Huge conversation, nobody but me realized that bug.
  • lolinder 3y ago
    I don't think I can notice bugs by reading code. I think I can notice bugs by: * Writing code in a language with a static type system. * Reading the code and "playing computer". And, last but by far the most important: * Actually testing the code Turns out that using an LLM to generate code doesn't preclude testing it!
    • yencabulator 3y ago
      Writing tests after the fact is way harder than writing code to be testable. You're pretty much guaranteed to miss edge conditions, which tend to be the cases LLMs often screw up. The rampant optimism is just weird to me.
      • lolinder 3y ago
        You seem to be assuming that we're generating large amounts of code with LLMs that we don't read and adapt and make our own. I can definitely see that as a risk for overconfident juniors, but that is not the MO for senior developers who are using these tools. 95% of the code that I add from an LLM is less than a line, a nice autocomplete. The other 5% is a function that I have ChatGPT get started for me and then I read it and refactor it until I understand it as well as any code I've written myself. The rampant fearmongering is just weird to me. It's a tool, and like any tool some people abuse it, but it's weird that some people simply can't believe that there are people who use it well.
        • yencabulator 3y ago
          I see people overestimating their ability to notice bugs.
• err4nt 3y ago
  The people who trust it are the people who either don't know the difference, or the code ChatGPT outputs is comparable or better than what they can output. It seems to work okay the other direction though, if code is well-named and well-commented, ChatGPT can explain what it does in natural language tailored to people of varying backgrounds and levels of experience. But it's not so good at taking natural language and making it into code that has to be much stricter.
• chrisjj 3y ago
  PS Using chatgpt.com ChatGPT 3.5, input was: Translate this to Common Lisp. def solve(graph): a = 0 while a < len(graph): for b in range(len(graph)): if b > a and graph[a].intersection(graph[b]): graph[b] = graph[a] | graph[b] graph.pop(a);a-=1 break a+=1 return graph a = [{1, 2}, {4, 5}, {1, 3}, {1, 4}] print((solve( a ))) Output today is: (defun solve (graph) (let ((a 0)) (loop while (< a (length graph)) do (loop for b below (length graph) do (when (and (> b a) (intersection (elt graph a) (elt graph b))) (setf (elt graph b) (union (elt graph a) (elt graph b))) (setf graph (remove (elt graph a) graph)) (decf a) (return))) (incf a)) graph)) (defparameter *a* '((1 2) (4 5) (1 3) (1 4))) (format t "~a" (solve *a*))) and though this differs from the original attempt's output, it fails to compile the same way - "unmatched close parenthesis" on graph)). Using Bing.com GPT-4 the output again differs, but again fails the same.