5 ms·

I'm a retired programmer. I can't imagine trusting code generated by probablities for anything mission critical. If it were close and just needed minor tweaks I

I'm a retired programmer. I can't imagine trusting code generated by probablities for anything mission critical. If it were close and just needed minor tweaks I could understand that. But I don't have experience with it. My comment is mainly to say LLMs are amazing in areas that are not coding, like brainstorming, blue sky thinking, filling in research details, asking questions that make me reflect. I treat the LLM like a thinking partner. It does make mistakes, but those can be caught easily by checking other sources, or even having another LLM review the conclusions.

garciasn 1y ago
Well; I can't speak to your specific experience (current or past) but I'm telling you that while I'm skeptical as hell about EVERYTHING, it's blowing my expectations away in every conceivable way. I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now. The most impressive thing is that it can do all of the things that I absolutely can do, just faster. But the most impressive thing is that it can do all the things I cannot possibly do and would have had to hire up/contract out to accomplish--for far less money, time, and with faster iterations than if I had to communicate with another human being. It's not perfect and it's incredibly frustrating at times (hardcoding values into the code when I have explicitly told it not to; outright lying that it made a particular fix, when it actually changed something else entirely unrelated), but it is a game changer IMO.
- gyomu 1y ago
  > I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now Would love to see it!
- 98eb1d0ff7fb96 1y ago
  See, your comment is a good example of what's going wrong. The OP specifically mentioned "mission critical things" - My interpretation of that would be things that are not allowed to break, because otherwise people might die, in the worst case - and you were talking about just SOMETHING that got "done" faster. No mention about anything critical. Of course, I was playing around with claude code, too, and I was fascinated how fun it can be and yes, you can get stuff done. But I have absolutely no clue what the code is doing and if there are some nasty mistakes. So it kinda worked, but I would not use that for anything "mission critical" (whatever this means).
  - CharlesW 1y ago
    > So it kinda worked, but I would not use that for anything "mission critical" (whatever this means). It means projects like Cloudflare's new OAuth provider library. https://github.com/cloudflare/workers-oauth-provider https://github.com/cloudflare/workers-oauth-provider > "This library (including the schema documentation) was largely written with the help of Claude, the AI model by Anthropic. Claude's output was thoroughly reviewed by Cloudflare engineers with careful attention paid to security and compliance with standards. Many improvements were made on the initial output, mostly again by prompting Claude (and reviewing the results)."
    - g-b-r 1y ago
      https://neilmadden.blog/2025/06/06/a-look-at-cloudflares-ai-coded-oauth-library https://neilmadden.blog/2025/06/06/a-look-at-cloudflares-ai-...
    - blibble 1y ago
      > It means projects like Cloudflare's new OAuth provider library. the one that's a few weeks old and already has several CVEs due to the poor implementation?
  - labrador 1y ago
    The mission of a professional programmer is to deliver code that works according to the design specs, handles edge cases, fails gracefully and doesn't contain performance bottlenecks. It could be software for a water plant, or software that incurs charges to accomplish it's task and could bankrupt you if there is a mistake. It doesn't have to be a matter of life or death.
    - SoftTalker 1y ago
      But there are a lot of projects and problem domains that don't even demand that much or have any real consequences for failure. I look at all the self-service stuff my employer has for HR, benefits, policy compliance, it's all half-broken, nobody ever seems to get held to fixing anything, and the only answers are "try it again later." Professional programmers built this stuff too, or maybe it was vibe-coded but since it's been like that for years I think probably not. But we don't know where on the spectrum of "people might die" to "try again later" most of these programmers who claim great productivity gains from LLMs lie. Maybe it is making them 10x faster at churning out shit, who knows? They might not even realise it themselves.
svdr 1y ago
I've been programming for 40 years and have started using LLM's a few months ago, and it has really changed the way I work. I let it write pieces of code (pasting error messages from logs mostly result in a fix in less then a minute), but also brainstorming about architecture or new solutions. Of course I check the code it writes, but I'm still almost daily amazed at the intelligence and accuracy. (Very much unlike crypto).
- labrador 1y ago
  That's good to know if I ever get an idea for a side project. Anything to relieve the tedius aspects of programming would be very welcome.
deleted 1y ago
[deleted]
anon-3988 1y ago
There's one thing that I find LLM extremely good at: data science. Since the IO is well defined, you can easily verify that the output is correct. You can even ask it write tests for you given that you know certain properties of the data. The problem is that the LLM needs context of what you are doing, contexts that you won't (or too lazy) to give in a chat with it ala ChatGPT. This is where Claude Code changes the game. For example, you have PCAP file where each UDP packet contain multiple messages. How do you filter the IP/port/protocol/time? Use LLM, check the output How do you find the number of packets that have patterns A, AB, AAB, ABB.... Use LLM, check the output How to create PCAPs that only contain those packets for testing? Use LLM, check the output Etc etc Since it can read your code, it is able to infer (because lets be honest, you work aint special) what you are trying to do at a much better rate. In any case, the fact that you can simply ask "Please write a unit test for all of the above functions" means that you can help it verify itself.
fleebee 1y ago
I tried the "thinking partner" approach for a while and for a moment I thought it worked well, but at some point the cracks started to show and I called the bluff. LLMs are extremely good at creating an illusion that they know things and are capable of reasoning, but they really don't do a good job of cultivating intellectual conversation. I think it's dangerously easy to get misled when trying to prod LLMs for knowledge, especially if it's a field you're new to. If you were using a regular search engine, you could look at the source website to determine the trustworthiness of its contents, but LLMs don't have that. The output can really be whatever, and I don't agree it's necessarily that easy to catch the mistakes.
- labrador 1y ago
  You don't say what LLM you are using. I'm using ChatGPT 4o. I'm getting great results, but I review the output with a skeptical eye similar to how I read Wikipedia articles. Like Wikipedia, GPT 4o is great for surfacing new topics for research and does it quickly, which makes stream of thought easier.
- selfhoster11 1y ago
  This is very model-dependent. If you use something heavy on sycophancy and low on brain cells (like GPT-4o, the default paid ChatGPT model), you'll get lots and lots of cracks because these models are optimised for engagement. That said, don't use model output directly. Use it to extract "shibboleth" keywords and acronyms in that domain, then search those up yourself with a classical search engine (or in a follow-up LLM query). You'll access a lot of new information that way, simply because you know how to surface it now.
- edg5000 1y ago
  Mostly agree. What works better is seeing the AI as a "high level to low level converter", in the same way Java is converted to machine code when it's ran. You describe exactly what you want it to report or do, and steer it whenever there are ambiguities. It does "grunt work" for you. With the bar of what grunt work means being moved up. Grunt work used to be doing the dishes or calculating numbers by hand on a paper spreasheet. Decades ago we automated those these. Now we've automated searching for information, summarization, implementing fully specified technical desings for software, the list goes on.
deleted 1y ago
[deleted]
shash 1y ago
I say this as an LLM skeptic. All code, including stuff that we experienced coders write is inherently probabilistic. That’s why we have code reviews, unit tests, pair programming, guidelines and guardrails in any critical project. If you’re using LLM output uncritically, you’re doing it wrong, but if you’re using _human_ output uncritically you’re doing it wrong too. That said, they are not magic, and my fear is that people use copilots and agentic models and all the rest to hide poor engineering practice, building more and more boilerplate instead of refactoring or redesigning for efficiency or safety or any of the things that matter in the long run.
- danielbln 1y ago
  LLMs turn foot guns into foot bazookas. Still, if you learn to aim them away from your body they do make a bigger better boom.
edg5000 1y ago
Treating it like a thinking partner is good. When programming, I don't treat it like a person, but rather a very high programming language. Imagine exactly how you want the code to be. Then find a way to express that unambiguously in natural language; the idea is that you'll still have a bit of work to do writing things out, but it will be a lot quicker than typing out all code by hand. Combined with iterations of feedback as well as having the AI build and run your program (at least as a sanity check); and asking the AI to check the program behaviour in the same way you would, gets you quite far. A limitation is the lack of memory. If you steer it from style A to B using multiple points of feedback, if this is not written down, the next AI session you'll have to reexplain this all over. Deepseek is about 1TB in weights; maybe that is why LLMs don't remember things across sessions yet. I think everybody can have their personal AI (hosted remote unless you own lots of compute); it should remember what happened yesterday; in particular the feedback it was given when developing. As an AI layman I do think this is the next step.
danielbln 1y ago
You don't trust the code coming out of the probabilistic machine. You build a validation cage around it with hard interfaces and you also review the output.

garciasn 1y ago

Well; I can't speak to your specific experience (current or past) but I'm telling you that while I'm skeptical as hell about EVERYTHING, it's blowing my expectations away in every conceivable way. I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now. The most impressive thing is that it can do all of the things that I absolutely can do, just faster. But the most impressive thing is that it can do all the things I cannot possibly do and would have had to hire up/contract out to accomplish--for far less money, time, and with faster iterations than if I had to communicate with another human being. It's not perfect and it's incredibly frustrating at times (hardcoding values into the code when I have explicitly told it not to; outright lying that it made a particular fix, when it actually changed something else entirely unrelated), but it is a game changer IMO.

gyomu 1y ago
> I built something in less than 24h that I'm sure would have taken us MONTHS to just get off the ground, let alone to the polished version it's at right now Would love to see it!
98eb1d0ff7fb96 1y ago
See, your comment is a good example of what's going wrong. The OP specifically mentioned "mission critical things" - My interpretation of that would be things that are not allowed to break, because otherwise people might die, in the worst case - and you were talking about just SOMETHING that got "done" faster. No mention about anything critical. Of course, I was playing around with claude code, too, and I was fascinated how fun it can be and yes, you can get stuff done. But I have absolutely no clue what the code is doing and if there are some nasty mistakes. So it kinda worked, but I would not use that for anything "mission critical" (whatever this means).
- CharlesW 1y ago
  > So it kinda worked, but I would not use that for anything "mission critical" (whatever this means). It means projects like Cloudflare's new OAuth provider library. https://github.com/cloudflare/workers-oauth-provider https://github.com/cloudflare/workers-oauth-provider > "This library (including the schema documentation) was largely written with the help of Claude, the AI model by Anthropic. Claude's output was thoroughly reviewed by Cloudflare engineers with careful attention paid to security and compliance with standards. Many improvements were made on the initial output, mostly again by prompting Claude (and reviewing the results)."
  - g-b-r 1y ago
    https://neilmadden.blog/2025/06/06/a-look-at-cloudflares-ai-coded-oauth-library https://neilmadden.blog/2025/06/06/a-look-at-cloudflares-ai-...
  - blibble 1y ago
    > It means projects like Cloudflare's new OAuth provider library. the one that's a few weeks old and already has several CVEs due to the poor implementation?
- labrador 1y ago
  The mission of a professional programmer is to deliver code that works according to the design specs, handles edge cases, fails gracefully and doesn't contain performance bottlenecks. It could be software for a water plant, or software that incurs charges to accomplish it's task and could bankrupt you if there is a mistake. It doesn't have to be a matter of life or death.
  - SoftTalker 1y ago
    But there are a lot of projects and problem domains that don't even demand that much or have any real consequences for failure. I look at all the self-service stuff my employer has for HR, benefits, policy compliance, it's all half-broken, nobody ever seems to get held to fixing anything, and the only answers are "try it again later." Professional programmers built this stuff too, or maybe it was vibe-coded but since it's been like that for years I think probably not. But we don't know where on the spectrum of "people might die" to "try again later" most of these programmers who claim great productivity gains from LLMs lie. Maybe it is making them 10x faster at churning out shit, who knows? They might not even realise it themselves.

svdr 1y ago

I've been programming for 40 years and have started using LLM's a few months ago, and it has really changed the way I work. I let it write pieces of code (pasting error messages from logs mostly result in a fix in less then a minute), but also brainstorming about architecture or new solutions. Of course I check the code it writes, but I'm still almost daily amazed at the intelligence and accuracy. (Very much unlike crypto).

labrador 1y ago
That's good to know if I ever get an idea for a side project. Anything to relieve the tedius aspects of programming would be very welcome.

deleted 1y ago

[deleted]

anon-3988 1y ago

There's one thing that I find LLM extremely good at: data science. Since the IO is well defined, you can easily verify that the output is correct. You can even ask it write tests for you given that you know certain properties of the data. The problem is that the LLM needs context of what you are doing, contexts that you won't (or too lazy) to give in a chat with it ala ChatGPT. This is where Claude Code changes the game. For example, you have PCAP file where each UDP packet contain multiple messages. How do you filter the IP/port/protocol/time? Use LLM, check the output How do you find the number of packets that have patterns A, AB, AAB, ABB.... Use LLM, check the output How to create PCAPs that only contain those packets for testing? Use LLM, check the output Etc etc Since it can read your code, it is able to infer (because lets be honest, you work aint special) what you are trying to do at a much better rate. In any case, the fact that you can simply ask "Please write a unit test for all of the above functions" means that you can help it verify itself.

fleebee 1y ago

I tried the "thinking partner" approach for a while and for a moment I thought it worked well, but at some point the cracks started to show and I called the bluff. LLMs are extremely good at creating an illusion that they know things and are capable of reasoning, but they really don't do a good job of cultivating intellectual conversation. I think it's dangerously easy to get misled when trying to prod LLMs for knowledge, especially if it's a field you're new to. If you were using a regular search engine, you could look at the source website to determine the trustworthiness of its contents, but LLMs don't have that. The output can really be whatever, and I don't agree it's necessarily that easy to catch the mistakes.

labrador 1y ago
You don't say what LLM you are using. I'm using ChatGPT 4o. I'm getting great results, but I review the output with a skeptical eye similar to how I read Wikipedia articles. Like Wikipedia, GPT 4o is great for surfacing new topics for research and does it quickly, which makes stream of thought easier.
selfhoster11 1y ago
This is very model-dependent. If you use something heavy on sycophancy and low on brain cells (like GPT-4o, the default paid ChatGPT model), you'll get lots and lots of cracks because these models are optimised for engagement. That said, don't use model output directly. Use it to extract "shibboleth" keywords and acronyms in that domain, then search those up yourself with a classical search engine (or in a follow-up LLM query). You'll access a lot of new information that way, simply because you know how to surface it now.
edg5000 1y ago
Mostly agree. What works better is seeing the AI as a "high level to low level converter", in the same way Java is converted to machine code when it's ran. You describe exactly what you want it to report or do, and steer it whenever there are ambiguities. It does "grunt work" for you. With the bar of what grunt work means being moved up. Grunt work used to be doing the dishes or calculating numbers by hand on a paper spreasheet. Decades ago we automated those these. Now we've automated searching for information, summarization, implementing fully specified technical desings for software, the list goes on.

deleted 1y ago

[deleted]

shash 1y ago

I say this as an LLM skeptic. All code, including stuff that we experienced coders write is inherently probabilistic. That’s why we have code reviews, unit tests, pair programming, guidelines and guardrails in any critical project. If you’re using LLM output uncritically, you’re doing it wrong, but if you’re using _human_ output uncritically you’re doing it wrong too. That said, they are not magic, and my fear is that people use copilots and agentic models and all the rest to hide poor engineering practice, building more and more boilerplate instead of refactoring or redesigning for efficiency or safety or any of the things that matter in the long run.

danielbln 1y ago
LLMs turn foot guns into foot bazookas. Still, if you learn to aim them away from your body they do make a bigger better boom.

edg5000 1y ago

Treating it like a thinking partner is good. When programming, I don't treat it like a person, but rather a very high programming language. Imagine exactly how you want the code to be. Then find a way to express that unambiguously in natural language; the idea is that you'll still have a bit of work to do writing things out, but it will be a lot quicker than typing out all code by hand. Combined with iterations of feedback as well as having the AI build and run your program (at least as a sanity check); and asking the AI to check the program behaviour in the same way you would, gets you quite far. A limitation is the lack of memory. If you steer it from style A to B using multiple points of feedback, if this is not written down, the next AI session you'll have to reexplain this all over. Deepseek is about 1TB in weights; maybe that is why LLMs don't remember things across sessions yet. I think everybody can have their personal AI (hosted remote unless you own lots of compute); it should remember what happened yesterday; in particular the feedback it was given when developing. As an AI layman I do think this is the next step.

danielbln 1y ago

You don't trust the code coming out of the probabilistic machine. You build a validation cage around it with hard interfaces and you also review the output.