9 ms·

I've used GPT-4 to do what I had said. I pasted the errors I was given, and did so for 2-3 more iterations, and it successfully ported critical in-house infrast

I've used GPT-4 to do what I had said. I pasted the errors I was given, and did so for 2-3 more iterations, and it successfully ported critical in-house infrastructure from Python 3 to Go.

sergiotapia 2y ago
you're never going to convince people that are in an ideological battle against AI.
- bigstrat2003 2y ago
  And you're never going to convince anyone if you assume without evidence that they are ideologically opposed to AI. Lots of people have tried these tools with an open mind and found them to not be useful, you need to address those criticisms rather than using a dismissive insult.
  - HaZeust 2y ago
    What evidence would you like? You're posting on a thread that hyperlinks to a list of code and Claude Artifacts for pet-projects that can make thousands a month with some low-effort PPC and an AdWords embed, and some mid-size projects that can be anything from grounds to a promotion at a programming role - to the MVP for a PMF-stage startup. What, specifically, would pivot your pre-conceived notions?
    - inexcf 2y ago
      Did you even look at the artifacts? Its a bunch of things a beginner would do on their first day programming. How do you make thousands a month from 1 library call to solve a qr code. A promotion for building an input field and calling a json to yaml converter library?
      - newswasboring 2y ago
        > Its a bunch of things a beginner would do on their first day programming. Is this an exaggeration? Because this is absolutely not true. I'm a beginner in JavaScript and other web stuff and I absolutely can't build it in many days.
        inexcf 2y ago
        You better check the code, mate. The meat of what most of it does is a one liner calling jsQR or some other imported lib to do the real work. I am not exaggerating in the slightest.
        newswasboring 2y ago
        Dude. I don't judge my knowledge after the answer is given to me. If I was the junior programmer assigned to the author and they were having this chat with me I am telling you as a beginner I wouldn't be able to do it. Of course if you show me the answers I will think I can do it easy, because answers in programming are always easy (good answers anyways). It's the process of finding the answer that is hard. And I'm not a bad programmer either, I'm at least mediocre, I'm just unfamiliar with web technology.
        inexcf 2y ago
        I am of the firm believe that you can put "JavaScript scan qr code" in a search engine and arrive at your goal. The answers range from libraries to code snippets basically the same as those created by Claude. Using the same libraries. I feel like googling every step would be faster than trying to get it right with LLMs, but that is a different point. I've seen a complete no-code person install whisper x with a virtual Python environment and use it for realtime speech to text in their Japanese lessons, in less than 3 hours. You can do a simple library call in JavaScript.
        simonw 2y ago
        "I feel like googling every step would be faster than trying to get it right with LLMs" Why don't you give that a go? See if you can knock out a QR code reading UI in JavaScript in less than 3 minutes, complete with drag-and-drop file opening support. (I literally built this one in a separate browser tab while I was actively taking notes in a meeting) I say three minutes because my first message in https://gist.github.com/simonw/c2b0c42cd1541d6ed6bfe5c17d638039 https://gist.github.com/simonw/c2b0c42cd1541d6ed6bfe5c17d638... was at 2:45pm and the final reply from Claude was at 2:47pm.
    - achierius 2y ago
      Are you serious about "thousands a month"? I don't mean to be hostile, I'm just truly surprised -- if the bar were that low (not that these apps aren't impressive, but most engineers write useful apps from time to time) I would expect the market to be rather packed
      - HaZeust 2y ago
        Nah, most are hundreds a month - a few golden geese can break the thousand barrier, though. But, regardless, have a few of those sites up, and you're making good side income.
    - tharant 2y ago
      > What, specifically, would pivot your pre-conceived notions? A live or unedited demonstration of how a non-trivial (doesn’t have to be complex, but should be significantly more interesting than the “getting started” tutorials that litter the web) pet-project was implemented using these models.
      - simonw 2y ago
        The point of my post here was to provide 14 of those. Some of them are trivial but I'd argue that a couple of them - the OpenAI Audio one and the LLM pricing calculator - go a bit beyond "getting started". If you want details of more complex projects I've written using Claude here are a few - in each case I provide the full chat transcript: - https://simonwillison.net/2024/Aug/8/django-http-debug/ https://simonwillison.net/2024/Aug/8/django-http-debug/ - https://simonwillison.net/2024/Aug/27/gemini-chat-app/ https://simonwillison.net/2024/Aug/27/gemini-chat-app/ - https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/ https://simonwillison.net/2024/Aug/26/gemini-bounding-box-vi... - https://simonwillison.net/2024/Aug/16/datasette-checkbox/ https://simonwillison.net/2024/Aug/16/datasette-checkbox/ - https://simonwillison.net/2024/Oct/6/svg-to-jpg-png/ https://simonwillison.net/2024/Oct/6/svg-to-jpg-png/
        tharant 2y ago
        Thank you! I have an ugly JS/content filter running that mogrifies some websites such that I miss the formatting completely; I didn’t recognize you had chat session content included on the page. That said, after looking at a couple of your sessions, I don’t see anything you’re doing that I’m not—at least in terms of prompting. Your prompts are a bit more terse than mine (I can be long-winded so I’ll give brevity a try with my next project) but the structure and design descriptions are still there. That would suggest the differences in our experience boils down to the languages with which we choose or are required to work; maybe there’s a stylistic or cultural difference in how one should prompt a model in order to generate a Python project and how one should prompt for a Haskel or Scala/Java project; surely not though, right? I’m not giving up and I’ll keep playing with these models but for now, given my use-case at least, they still seem to be far more capable at rubber-ducking with me than they are as a pair programming partner.
  - epolanski 2y ago
    Issue is, it takes time to learn how to interact with these tools and get the best out of them. And they get better quite fast.
    - unit149 2y ago
      claude-to-sql parser is particularly useful in LLM implementation
  - sergiotapia 2y ago
    you are replying to a submission with a dozen or more examples of real tangible stuff, and you still argue? pointless.
  - farts_mckensy 2y ago
    No need to address the criticisms. Just have chat gpt do it.
- versteegen 2y ago
  Humans have a massive pro-human bias. Don't ask one whether AI can replace humans and expect a fair answer.
  - n0id34 2y ago
    Well, obviously. The only ones happy about all of our potential replacements would be those that have the power to do the replacing and save themselves a shitload of money. It's hardly like everyone is going to rejoice at the rapid advancement of AI that can potentially make most of us jobless....unless, as I said, you're the one in charge, then it's wonderful.
- Workaccount2 2y ago
  "It is difficult to get a man to understand something when his salary depends upon his not understanding it." - Upton Sinclair.
- fhdsgbbcaA 2y ago
  There’s no ideological battle here. The first self-driving DARPA grand challenge was passed in 2005, everybody thought we’d have self driving on the road within a decade. 20 years later that’s still not the case, because it turns out NN/ML can do some very impressive things at the 99% correct level. The other 1% ranges in severity from “weird lane change” to “a person riding a bicycle gets killed”. GPT-3.5 was the DARPA grand challenge moment, we’re still years away from LLM being reliable - and they may never be fully trustworthy.
  - josephg 2y ago
    > they may never be fully trustworthy. So? Neither are humans. Neither is google search. Chatgpt doesn't write bug free code, but neither do I. The question isn't "when will it be perfect". The question is "when will it be useful?". Or, "When is it useful enough that you're not employable?" I don't think its so far away. Everyone I know with a spark in their eye has found weird and wonderful ways to make use of chatgpt & claude. I've used it to do system design, help with cooking, practice improv, write project proposals, teach me history, translate code, ... all sorts of things. Yeah, the quality is lower than that of an expert human. But I don't need a 5 star chef to tell me how long to put potatoes in the oven, make suggestions for characters to play, or listen to me talk about encryption systems and make suggestions. Its wildly useful today. Seriously, anyone who says otherwise hasn't tried it or doesn't understand how to make proper use of it. Between my GF and I, we average about 1-2 conversations with chatgpt per day. That number will only go up.
    - fhdsgbbcaA 2y ago
      I find it very interesting the primary rebuttals to people criticizing LLM from the “converted” tends to result in implicit suggestions the critique is rooted in old fashioned thinking. That’s not remotely true. I am an expert, and it’s incredibly clear to me how bad LLM are. I still use them heavily, but I don’t trust any output that doesn’t conform to my prior expert knowledge and they are constantly wrong. I think what is likely happening is many people aren’t an expert in anything, but the LLM makes them feel like they are and they don’t want that feeling to go away and get irrationally defensive at cogent criticism of the technology. And that’s all it is, a new technology with a lot of hype and a lot of promise, but it’s not proven, it’s not reliable, and I do think it is messing with people’s heads in a way that worries me greatly.
VirusNewbie 2y ago
But how do you know those were the only errors?
- HaZeust 2y ago
  What's this question even mean? Because they're the only ones that came up in the debugger portion of the IDE, the output serves the intended purposes, the logging and error handling that I wanted to include were in the initial write-up prompt, and I could read the code it wrote because I partially knew the outputted language - and when I wasn't sure of a line, I asked it for clarification and a source from a reputable knowledgebase of the language, and GPT provided it?
  - majormajor 2y ago
    I would've expected an answer involving "an exhaustive suite of test cases still passed" - "it looks right" is a low bar for any complex software project these days. It's the long, long, long tail of edge cases - not just porting them, but even identifying them to test - that slow or doom most real-world human rewrites, after all.
    - josephg 2y ago
      True - but you can ask the chatbot to write a test suite too.
      - what 2y ago
        This doesn’t really make sense? If I can’t trust the code it writes, why should I trust that it can write a comprehensive test suite?
        josephg 2y ago
        Because the bugs in its code and the bugs in its test suites usually don't line up and cancel each other out.
        2y ago
IshKebab 2y ago
I have also tried to do this and it didn't work as smoothly as you claim. I don't think either of you are wrong; it just heavily depends on the complexity of the app and how familiar LLMs are with it. E.g. rewriting a web scraper, CRUD backend or a build script? Sure, maybe. Rewriting a bootloader, compiler or GUI app? No chance.
- josephg 2y ago
  Its funny seeing the goalposts move in real time. "Yes, AI can make human sounding sentences, but can it play chess?" "Well yes, it can play chess. But no computer can beat a human grandmaster at chess." "Well it beat Kasperov - but it has no hope of beating a human at Go." "Its funny - it can beat humans at go but still can't speak as well as a toddler." "Alright it can write simple problems, but it introduces bugs in anything nontrivial, and it can't fix those bugs!" I write bugs in anything nontrivial too! My human advantages are currently that I'm better at handling a large context, and I can iterate better than the computer can. But - seriously, do you think innovation will stop here? Did the improvements ever stop? It seems like a pretty trivial engineering problem to hook an AI up to a compiler / runtime so it can iterate just like we can. Anthropic is clearly already starting to try that. I agree with you, today. I used claude to help translate some rust code into typescript. I needed to go through the output with a fine toothed comb to fix a lot of obvious bugs and clean up the output. But the improvement over what was possible with GPT3.5 is totally insane. At the current rate of change, I give it 5-10 years before we can ask chatgpt to make a working compiler from scratch for a novel language.
  - wrtasfg 2y ago
    Of course it can stop, once legislation catches up and forbids IP theft using a thinly disguised probabilistic and compressed database of other people's code.
    - josephg 2y ago
      You really think those laws are coming? That the US and Chinese governments will force AI companies to put the genie back in the bottle? I think you're going to be very disappointed.
swat535 2y ago
GPT-4 has no understanding of logic what-so-ever, let's stop pretending it does. If it gives you a solution that is wrong, you have to point it at, then it will give you a second version , if that is also wrong, it will then slightly modify the same solutions over and over again instead of actually fixing the issue. It gets stuck in a loop of giving you 2-3 versions of the same solution with the slightly different outputs. It's only useful for boilerplate code and even then, you have to clean it up..
- GaggiX 2y ago
  Then you should try Claude, I have never seen it get stuck in a loop, at some point it would just rewrite everything if it came to that.
tharant 2y ago
I feel like I’ve been gaslit by the entire GenAI industry that I’m just bad at prompt engineering. When I’m talking to an LLM about stuff unrelated to code-generation, I can get sane and reasonable responses—engaging and useful even. The same goes for image generation and even the bit of video generation I’ve tried. For me however, getting any of these models to produce reasonably sane code has proven elusive. Claude is a bit better than others IME but I can’t even get it to describe a usable project template and directory structure for anything other than very simple Scala, Java, or Python projects. The code I’m able to generate always needs dramatic and manual changes; even trying to get a model to refactor a method in the code it wrote within the current context window results in bugs and broken business logic. I dearly wish I knew how others are able to accomplish things like “it successfully ported critical in-house infrastructure from Python 3 to Go.”. To date, I’ve seen no actual evidence (aside from what are purported to be LLM-generated artifacts) that anything beyond generating (or RAG-ing existing code) is even possible. What am I missing? Is it unrealistic for me to assume that prompt engineering such a seemingly dramatic LLM-generated code rewrite is something that I could learn by example from others? If not, can somebody recommend resources related to learning how to accomplish non-trivial code generation?
- malfist 2y ago
  I think it's level of expertise. You are an expert in coding (10,000 hours and all that) so you know when the code is wrong. Everything else you put into it and get plausible sounding response is just as incorrect as the plausible sounding responses to coding questions, just you know enough to spot the errors. LLMs are insidious, it feeds into "everything is simple" concept a lot of us have of the world. We ask an LLM for a project plan and it looks so good we're willing to fire our TPM, or a TPM asks the LLM for code and it gives them code that looks so good they question the value of an engineer. In reality, the LLM cannot do either role's job well.
  - tharant 2y ago
    > You are an expert in coding (10,000 hours and all that) so you know when the code is wrong. While I appreciate the suggestion that I might be an expert, I am decidedly not. That said, I’ve been writing what the companies I’ve worked for would consider “mission critical” code (mostly Java/Scala, Python, and SQL) for about twenty years, I’ve been a Unix/Linux sysadmin for over thirty years, and I’ve been in IT for almost forty years. Perhaps the modernity and/or popularity of the languages are my problem? Are the models going to produce better code if I target “modern” languages like Go/Rust, and the various HTML/JS/FE frameworks instead of “legacy” languages like Java or SQL? Or maybe my experience is too close to bare metal and need to focus on more trivial projects with higher-level or more modern languages? (fwiw, I don’t actually consider Go/Rust/JS/etc to be higher-level or more “modern” languages than the JVM languages with which I’m experienced; I’m open to arguments though) > LLMs are insidious, it feeds into "everything is simple" concept a lot of us have of the world. Yah, that’s what I mean when I say I feel gaslit. > In reality, the LLM cannot do either role's job well. I am aware of this. I’m not looking for an agent. That said, am I being too simplistic or unreasonable in expecting that I too could leverage these models (albeit perhaps after acquiring some missing piece of knowledge) as assistants capable of reasoning about my code or even the code they generate? If so, how are others able to get LLMs to generate what they claim are “deployable” non-trivial projects or refactorings of entire “critical” projects from the Python language to Go? Is someone lying or do I just need (seemingly dramatically) deeper knowledge of how to “correctly” prompt the models? Have I simply been victim of (again, seemingly dramatically) overly optimistic marketing hype?
fhdsgbbcaA 2y ago
That just means the bugs are so subtle you haven’t found them yet, they are there and unspooling the damage may be very painful.
- HaZeust 2y ago
  That's rather assuming of you, they're there no less than they would be for a human's programming - and VERY likely no more.
  - kuhewa 2y ago
    But one is trying to write good-enough code. The other is trying to write good-enough-looking code. The probability of pain arising from the bugs of the latter is probably greater.
    - HaZeust 2y ago
      I'd actually love to see a benchmark on this - we're just speculating now.
      - kuhewa 2y ago
        The work demonstrating the Frankfurtian Bullshit nature of generated prose would suggest as much, given the architecture is the same for code outputs it seems like a fair assumption until it is demonstrated otherwise.
  - tharant 2y ago
    > they're there no less than they would be for a human's programming - and VERY likely no more. This is VERY different from my own experience. The bugs introduced by the code I’ve tried to generate via LLMs (Mostly Claude, some GPT-4o and o1-preview, and lots of one-off fiddling with local models to see if they’re any better/worse than commercial products) are considerably more numerous (and often more subtle) than what my fellow engineers—juniors included—tend to introduce. I /want/ these tools to be useful; they haven’t been so far though and I’m kinda stuck on understanding if I’m just not using ‘em right or if they’re even capable of what I want to do. Like I said in a previous comment; I don’t know if I’m being gaslit or if I’m being naive but it feels a lot more like gaslighting.
ants_everywhere 2y ago
GPT-4 is pretty bad at generating Python. It kind of works as well as combining 2-3 stack overflow questions, but it can't tell that the combination is sane. I mostly agree with what the others are saying. It can generate boilerplate and it can generate simple API calls when there are lots of examples in the training set. Generating Go is probably easier because at least you get compiler feedback. Right now the only place it saves me time are with languages I don't know at all and with languages like Bash and SQL where I just can't bring myself to care enough to remember the long tail of more esoteric points that I don't use every day.

I've used GPT-4 to do what I had said. I pasted the errors I was given, and did so for 2-3 more iterations, and it successfully ported critical in-house infrastructure from Python 3 to Go.

sergiotapia 2y ago

you're never going to convince people that are in an ideological battle against AI.

bigstrat2003 2y ago
And you're never going to convince anyone if you assume without evidence that they are ideologically opposed to AI. Lots of people have tried these tools with an open mind and found them to not be useful, you need to address those criticisms rather than using a dismissive insult.
- HaZeust 2y ago
  What evidence would you like? You're posting on a thread that hyperlinks to a list of code and Claude Artifacts for pet-projects that can make thousands a month with some low-effort PPC and an AdWords embed, and some mid-size projects that can be anything from grounds to a promotion at a programming role - to the MVP for a PMF-stage startup. What, specifically, would pivot your pre-conceived notions?
  - inexcf 2y ago
    Did you even look at the artifacts? Its a bunch of things a beginner would do on their first day programming. How do you make thousands a month from 1 library call to solve a qr code. A promotion for building an input field and calling a json to yaml converter library?
    - newswasboring 2y ago
      > Its a bunch of things a beginner would do on their first day programming. Is this an exaggeration? Because this is absolutely not true. I'm a beginner in JavaScript and other web stuff and I absolutely can't build it in many days.
      - inexcf 2y ago
        You better check the code, mate. The meat of what most of it does is a one liner calling jsQR or some other imported lib to do the real work. I am not exaggerating in the slightest.
        newswasboring 2y ago
        Dude. I don't judge my knowledge after the answer is given to me. If I was the junior programmer assigned to the author and they were having this chat with me I am telling you as a beginner I wouldn't be able to do it. Of course if you show me the answers I will think I can do it easy, because answers in programming are always easy (good answers anyways). It's the process of finding the answer that is hard. And I'm not a bad programmer either, I'm at least mediocre, I'm just unfamiliar with web technology.
        inexcf 2y ago
        I am of the firm believe that you can put "JavaScript scan qr code" in a search engine and arrive at your goal. The answers range from libraries to code snippets basically the same as those created by Claude. Using the same libraries. I feel like googling every step would be faster than trying to get it right with LLMs, but that is a different point. I've seen a complete no-code person install whisper x with a virtual Python environment and use it for realtime speech to text in their Japanese lessons, in less than 3 hours. You can do a simple library call in JavaScript.
        simonw 2y ago
        "I feel like googling every step would be faster than trying to get it right with LLMs" Why don't you give that a go? See if you can knock out a QR code reading UI in JavaScript in less than 3 minutes, complete with drag-and-drop file opening support. (I literally built this one in a separate browser tab while I was actively taking notes in a meeting) I say three minutes because my first message in https://gist.github.com/simonw/c2b0c42cd1541d6ed6bfe5c17d638039 https://gist.github.com/simonw/c2b0c42cd1541d6ed6bfe5c17d638... was at 2:45pm and the final reply from Claude was at 2:47pm.
  - achierius 2y ago
    Are you serious about "thousands a month"? I don't mean to be hostile, I'm just truly surprised -- if the bar were that low (not that these apps aren't impressive, but most engineers write useful apps from time to time) I would expect the market to be rather packed
    - HaZeust 2y ago
      Nah, most are hundreds a month - a few golden geese can break the thousand barrier, though. But, regardless, have a few of those sites up, and you're making good side income.
  - tharant 2y ago
    > What, specifically, would pivot your pre-conceived notions? A live or unedited demonstration of how a non-trivial (doesn’t have to be complex, but should be significantly more interesting than the “getting started” tutorials that litter the web) pet-project was implemented using these models.
    - simonw 2y ago
      The point of my post here was to provide 14 of those. Some of them are trivial but I'd argue that a couple of them - the OpenAI Audio one and the LLM pricing calculator - go a bit beyond "getting started". If you want details of more complex projects I've written using Claude here are a few - in each case I provide the full chat transcript: - https://simonwillison.net/2024/Aug/8/django-http-debug/ https://simonwillison.net/2024/Aug/8/django-http-debug/ - https://simonwillison.net/2024/Aug/27/gemini-chat-app/ https://simonwillison.net/2024/Aug/27/gemini-chat-app/ - https://simonwillison.net/2024/Aug/26/gemini-bounding-box-visualization/ https://simonwillison.net/2024/Aug/26/gemini-bounding-box-vi... - https://simonwillison.net/2024/Aug/16/datasette-checkbox/ https://simonwillison.net/2024/Aug/16/datasette-checkbox/ - https://simonwillison.net/2024/Oct/6/svg-to-jpg-png/ https://simonwillison.net/2024/Oct/6/svg-to-jpg-png/
      - tharant 2y ago
        Thank you! I have an ugly JS/content filter running that mogrifies some websites such that I miss the formatting completely; I didn’t recognize you had chat session content included on the page. That said, after looking at a couple of your sessions, I don’t see anything you’re doing that I’m not—at least in terms of prompting. Your prompts are a bit more terse than mine (I can be long-winded so I’ll give brevity a try with my next project) but the structure and design descriptions are still there. That would suggest the differences in our experience boils down to the languages with which we choose or are required to work; maybe there’s a stylistic or cultural difference in how one should prompt a model in order to generate a Python project and how one should prompt for a Haskel or Scala/Java project; surely not though, right? I’m not giving up and I’ll keep playing with these models but for now, given my use-case at least, they still seem to be far more capable at rubber-ducking with me than they are as a pair programming partner.
- epolanski 2y ago
  Issue is, it takes time to learn how to interact with these tools and get the best out of them. And they get better quite fast.
  - unit149 2y ago
    claude-to-sql parser is particularly useful in LLM implementation
- sergiotapia 2y ago
  you are replying to a submission with a dozen or more examples of real tangible stuff, and you still argue? pointless.
- farts_mckensy 2y ago
  No need to address the criticisms. Just have chat gpt do it.
versteegen 2y ago
Humans have a massive pro-human bias. Don't ask one whether AI can replace humans and expect a fair answer.
- n0id34 2y ago
  Well, obviously. The only ones happy about all of our potential replacements would be those that have the power to do the replacing and save themselves a shitload of money. It's hardly like everyone is going to rejoice at the rapid advancement of AI that can potentially make most of us jobless....unless, as I said, you're the one in charge, then it's wonderful.
Workaccount2 2y ago
"It is difficult to get a man to understand something when his salary depends upon his not understanding it." - Upton Sinclair.
fhdsgbbcaA 2y ago
There’s no ideological battle here. The first self-driving DARPA grand challenge was passed in 2005, everybody thought we’d have self driving on the road within a decade. 20 years later that’s still not the case, because it turns out NN/ML can do some very impressive things at the 99% correct level. The other 1% ranges in severity from “weird lane change” to “a person riding a bicycle gets killed”. GPT-3.5 was the DARPA grand challenge moment, we’re still years away from LLM being reliable - and they may never be fully trustworthy.
- josephg 2y ago
  > they may never be fully trustworthy. So? Neither are humans. Neither is google search. Chatgpt doesn't write bug free code, but neither do I. The question isn't "when will it be perfect". The question is "when will it be useful?". Or, "When is it useful enough that you're not employable?" I don't think its so far away. Everyone I know with a spark in their eye has found weird and wonderful ways to make use of chatgpt & claude. I've used it to do system design, help with cooking, practice improv, write project proposals, teach me history, translate code, ... all sorts of things. Yeah, the quality is lower than that of an expert human. But I don't need a 5 star chef to tell me how long to put potatoes in the oven, make suggestions for characters to play, or listen to me talk about encryption systems and make suggestions. Its wildly useful today. Seriously, anyone who says otherwise hasn't tried it or doesn't understand how to make proper use of it. Between my GF and I, we average about 1-2 conversations with chatgpt per day. That number will only go up.
  - fhdsgbbcaA 2y ago
    I find it very interesting the primary rebuttals to people criticizing LLM from the “converted” tends to result in implicit suggestions the critique is rooted in old fashioned thinking. That’s not remotely true. I am an expert, and it’s incredibly clear to me how bad LLM are. I still use them heavily, but I don’t trust any output that doesn’t conform to my prior expert knowledge and they are constantly wrong. I think what is likely happening is many people aren’t an expert in anything, but the LLM makes them feel like they are and they don’t want that feeling to go away and get irrationally defensive at cogent criticism of the technology. And that’s all it is, a new technology with a lot of hype and a lot of promise, but it’s not proven, it’s not reliable, and I do think it is messing with people’s heads in a way that worries me greatly.

VirusNewbie 2y ago

But how do you know those were the only errors?

HaZeust 2y ago
What's this question even mean? Because they're the only ones that came up in the debugger portion of the IDE, the output serves the intended purposes, the logging and error handling that I wanted to include were in the initial write-up prompt, and I could read the code it wrote because I partially knew the outputted language - and when I wasn't sure of a line, I asked it for clarification and a source from a reputable knowledgebase of the language, and GPT provided it?
- majormajor 2y ago
  I would've expected an answer involving "an exhaustive suite of test cases still passed" - "it looks right" is a low bar for any complex software project these days. It's the long, long, long tail of edge cases - not just porting them, but even identifying them to test - that slow or doom most real-world human rewrites, after all.
  - josephg 2y ago
    True - but you can ask the chatbot to write a test suite too.
    - what 2y ago
      This doesn’t really make sense? If I can’t trust the code it writes, why should I trust that it can write a comprehensive test suite?
      - josephg 2y ago
        Because the bugs in its code and the bugs in its test suites usually don't line up and cancel each other out.
      - 2y ago

IshKebab 2y ago

I have also tried to do this and it didn't work as smoothly as you claim. I don't think either of you are wrong; it just heavily depends on the complexity of the app and how familiar LLMs are with it. E.g. rewriting a web scraper, CRUD backend or a build script? Sure, maybe. Rewriting a bootloader, compiler or GUI app? No chance.

josephg 2y ago
Its funny seeing the goalposts move in real time. "Yes, AI can make human sounding sentences, but can it play chess?" "Well yes, it can play chess. But no computer can beat a human grandmaster at chess." "Well it beat Kasperov - but it has no hope of beating a human at Go." "Its funny - it can beat humans at go but still can't speak as well as a toddler." "Alright it can write simple problems, but it introduces bugs in anything nontrivial, and it can't fix those bugs!" I write bugs in anything nontrivial too! My human advantages are currently that I'm better at handling a large context, and I can iterate better than the computer can. But - seriously, do you think innovation will stop here? Did the improvements ever stop? It seems like a pretty trivial engineering problem to hook an AI up to a compiler / runtime so it can iterate just like we can. Anthropic is clearly already starting to try that. I agree with you, today. I used claude to help translate some rust code into typescript. I needed to go through the output with a fine toothed comb to fix a lot of obvious bugs and clean up the output. But the improvement over what was possible with GPT3.5 is totally insane. At the current rate of change, I give it 5-10 years before we can ask chatgpt to make a working compiler from scratch for a novel language.
- wrtasfg 2y ago
  Of course it can stop, once legislation catches up and forbids IP theft using a thinly disguised probabilistic and compressed database of other people's code.
  - josephg 2y ago
    You really think those laws are coming? That the US and Chinese governments will force AI companies to put the genie back in the bottle? I think you're going to be very disappointed.

swat535 2y ago

GPT-4 has no understanding of logic what-so-ever, let's stop pretending it does. If it gives you a solution that is wrong, you have to point it at, then it will give you a second version , if that is also wrong, it will then slightly modify the same solutions over and over again instead of actually fixing the issue. It gets stuck in a loop of giving you 2-3 versions of the same solution with the slightly different outputs. It's only useful for boilerplate code and even then, you have to clean it up..

GaggiX 2y ago
Then you should try Claude, I have never seen it get stuck in a loop, at some point it would just rewrite everything if it came to that.

tharant 2y ago

I feel like I’ve been gaslit by the entire GenAI industry that I’m just bad at prompt engineering. When I’m talking to an LLM about stuff unrelated to code-generation, I can get sane and reasonable responses—engaging and useful even. The same goes for image generation and even the bit of video generation I’ve tried. For me however, getting any of these models to produce reasonably sane code has proven elusive. Claude is a bit better than others IME but I can’t even get it to describe a usable project template and directory structure for anything other than very simple Scala, Java, or Python projects. The code I’m able to generate always needs dramatic and manual changes; even trying to get a model to refactor a method in the code it wrote within the current context window results in bugs and broken business logic. I dearly wish I knew how others are able to accomplish things like “it successfully ported critical in-house infrastructure from Python 3 to Go.”. To date, I’ve seen no actual evidence (aside from what are purported to be LLM-generated artifacts) that anything beyond generating (or RAG-ing existing code) is even possible. What am I missing? Is it unrealistic for me to assume that prompt engineering such a seemingly dramatic LLM-generated code rewrite is something that I could learn by example from others? If not, can somebody recommend resources related to learning how to accomplish non-trivial code generation?

malfist 2y ago
I think it's level of expertise. You are an expert in coding (10,000 hours and all that) so you know when the code is wrong. Everything else you put into it and get plausible sounding response is just as incorrect as the plausible sounding responses to coding questions, just you know enough to spot the errors. LLMs are insidious, it feeds into "everything is simple" concept a lot of us have of the world. We ask an LLM for a project plan and it looks so good we're willing to fire our TPM, or a TPM asks the LLM for code and it gives them code that looks so good they question the value of an engineer. In reality, the LLM cannot do either role's job well.
- tharant 2y ago
  > You are an expert in coding (10,000 hours and all that) so you know when the code is wrong. While I appreciate the suggestion that I might be an expert, I am decidedly not. That said, I’ve been writing what the companies I’ve worked for would consider “mission critical” code (mostly Java/Scala, Python, and SQL) for about twenty years, I’ve been a Unix/Linux sysadmin for over thirty years, and I’ve been in IT for almost forty years. Perhaps the modernity and/or popularity of the languages are my problem? Are the models going to produce better code if I target “modern” languages like Go/Rust, and the various HTML/JS/FE frameworks instead of “legacy” languages like Java or SQL? Or maybe my experience is too close to bare metal and need to focus on more trivial projects with higher-level or more modern languages? (fwiw, I don’t actually consider Go/Rust/JS/etc to be higher-level or more “modern” languages than the JVM languages with which I’m experienced; I’m open to arguments though) > LLMs are insidious, it feeds into "everything is simple" concept a lot of us have of the world. Yah, that’s what I mean when I say I feel gaslit. > In reality, the LLM cannot do either role's job well. I am aware of this. I’m not looking for an agent. That said, am I being too simplistic or unreasonable in expecting that I too could leverage these models (albeit perhaps after acquiring some missing piece of knowledge) as assistants capable of reasoning about my code or even the code they generate? If so, how are others able to get LLMs to generate what they claim are “deployable” non-trivial projects or refactorings of entire “critical” projects from the Python language to Go? Is someone lying or do I just need (seemingly dramatically) deeper knowledge of how to “correctly” prompt the models? Have I simply been victim of (again, seemingly dramatically) overly optimistic marketing hype?

fhdsgbbcaA 2y ago

That just means the bugs are so subtle you haven’t found them yet, they are there and unspooling the damage may be very painful.

HaZeust 2y ago
That's rather assuming of you, they're there no less than they would be for a human's programming - and VERY likely no more.
- kuhewa 2y ago
  But one is trying to write good-enough code. The other is trying to write good-enough-looking code. The probability of pain arising from the bugs of the latter is probably greater.
  - HaZeust 2y ago
    I'd actually love to see a benchmark on this - we're just speculating now.
    - kuhewa 2y ago
      The work demonstrating the Frankfurtian Bullshit nature of generated prose would suggest as much, given the architecture is the same for code outputs it seems like a fair assumption until it is demonstrated otherwise.
- tharant 2y ago
  > they're there no less than they would be for a human's programming - and VERY likely no more. This is VERY different from my own experience. The bugs introduced by the code I’ve tried to generate via LLMs (Mostly Claude, some GPT-4o and o1-preview, and lots of one-off fiddling with local models to see if they’re any better/worse than commercial products) are considerably more numerous (and often more subtle) than what my fellow engineers—juniors included—tend to introduce. I /want/ these tools to be useful; they haven’t been so far though and I’m kinda stuck on understanding if I’m just not using ‘em right or if they’re even capable of what I want to do. Like I said in a previous comment; I don’t know if I’m being gaslit or if I’m being naive but it feels a lot more like gaslighting.

ants_everywhere 2y ago

GPT-4 is pretty bad at generating Python. It kind of works as well as combining 2-3 stack overflow questions, but it can't tell that the combination is sane. I mostly agree with what the others are saying. It can generate boilerplate and it can generate simple API calls when there are lots of examples in the training set. Generating Go is probably easier because at least you get compiler feedback. Right now the only place it saves me time are with languages I don't know at all and with languages like Bash and SQL where I just can't bring myself to care enough to remember the long tail of more esoteric points that I don't use every day.