5 ms·

95% of real dev work is digging through piles of brownfield spaghetti to fix a bug or implement a feature. human-provided descriptions of the bug to fix or the

95% of real dev work is digging through piles of brownfield spaghetti to fix a bug or implement a feature. human-provided descriptions of the bug to fix or the feature to implement are often so vague that even a human developer can barely comprehend them. also more often than not, a visual link is required between the developer and the product being fixed/improved. hypothetical AI will replace humans, sure. LLMs probably won't. original GPT4 was the best coding LLM I dealt with, and it was 1.5+ years ago. the context size was pitiful, yes, but it was a lot smarter than the current crop of big models.

015a 2y ago
> original GPT4 was the best coding LLM I dealt with, and it was 1.5+ years ago. the context size was pitiful, yes, but it was a lot smarter than the current crop of big models. Anytime someone says "this is the worst it'll ever be" I can't help but think "oh sweet summer child"; and then I ask them, what's your favorite operating system you've ever used? The answer heavily depends on the person's age. Windows 10 or 7 dominates the conversation, and for slightly older folks, XP. "MacOS Snow Leopard" gets said a lot, that was a beloved MacOS release because they very famously said "no new features we're just fixing the old ones" and they actually did it. No one ever says "MacOS Sequoia" or "Windows 11". Man, the discourse surrounding the new iOS 18 Photos app is reaching a fever pitch. Sorry for the weird music on this, but its the first clip I could find of even, gasp, Elon Musk, stating this very real fact [1]: Technology does not automatically improve. In fact, it seems like especially software loves to get worse, and it takes a focused effort, from talented individuals, as far removed from money and business and agilescrum as possible to keep things operationally stable. But, sure, definitely the new metaversecryptoai bubblescam will escape that and be built to the highest possible quality. There definitely won't be investors and men in suits in every room saying "we're losing billions is there a knob we can turn to reduce electricity usage?" OpenAI is definitely raising billions to fund the R&D of the next level of intelligence, and certainly not because without that capital they'd be bankrupt in three months. [1] https://www.youtube.com/watch?v=aDC6kWC8y2Y https://www.youtube.com/watch?v=aDC6kWC8y2Y
- shepherdjerred 2y ago
  I would pick Windows 11 (LTSC) over any prior version of Windows, and I would pick macOS Sequoia over any prior version of macOS.
  - 015a 2y ago
    [flagged]
    - shepherdjerred 2y ago
      It's uncharitable of you to consider my opinion as being contrarian versus being genuine preference. Alternatively you could have chosen to be curious about my response that deviates from your understanding. I've used Windows 97 onwards, macOS around Mavericks, and various desktop Linux distros from around 2013. I would have told you that I like Windows 11 (LTSC) because it is much more visually pleasing than 7/8/10. The OS works reliably and I overall enjoy using it much more than the Linux distros I tried out before the LTSC build was made available. I like macOS Sequoia (and iOS 18) because Siri has gotten substantially better and I'm excited about Apple Intelligence. iOS 18 adds RCS which has been great and I actually do like Photos quite a bit, but I only moved to iCloud Photos while I was on the beta, so I can't compare to the old Photos app.
      - Retric 2y ago
        That built in advertising is definitely aesthetically pleasing. It just makes the whole UI pop.
        shepherdjerred 2y ago
        This is another unnecessarily snarky reply. I’ve specifically mentioned the LTSC build when mentioning Windows 11, which does not have the normal bloat. The consumer version of Windows 11 is terrible IMO, but that’s more a business decision than a technical one.
lokar 2y ago
Even just the ability to reproduce a non-trivial bug, and then work out what general area of the codebase might be in seems out of reach for an LLM
- skeptrune 2y ago
  Yes! I think I'm going to make LLM bug reproduction my primary response to AI developer maximalists.
  - beeflet 2y ago
    Maybe the solution isn't to train LLMs on human-written code, but to train some sort of text-to-AST generator on descriptions of programs and then the output of programs. That way it doesn't learn to include human-inspired bugs at least.
    - mnky9800n 2y ago
      I always wondered why bother having an AI generate human readable code. Isn't it a machine? Why not have it generate machine code? of course, it wouldn't be readable, but would it be performant? Maybe. I guess my point is that if an AI is doing computing why would it doing computing resemble a person doing it? It's much closer to the machine then me. Why not take advantage of that?
      - delapid 2y ago
        Isn’t that kind of what neural networks are in a way? Large blobs of computation we cant really parse fully
        mnky9800n 2y ago
        Yea but there are a bunch of layers, lol, between that specification and the machine and I'm wondering if there's a way to reduce that assuming it's a bottleneck. But I'm not sure it is I'm just guessing.
moi2388 2y ago
I hate to break it to you, but that’s exactly what ChatGPT o1 excels at. I swear to god, it’s better than our junior devs fresh out of college. Like, I let the LLM solve a PBI and it’s just faster with better code. Actual devs will still have a job, but the low hanging write some code guys? They’re done. And a lot of companies had a bunch of them..
- n_ary 2y ago
  I hate to break it to you but o1 is like far off of anything what you describe. Yesterday, I was testing some stuff and told it in very precise prompt to write a dripping bucket rate limiting middleware and it just literally named a var “bucket” with a timeout and then implemented something that will wait for the next function to timeout! I then said, heh and asked it to implement a circuit breaker pattern using axios, and it again implemented a timeout and added it to axios options! Could be am dumb or o1 was hallucinating on me intentionally, but the thing is very far away from a Jr dev fresh out of college.
  - oefnak 2y ago
    I don't even understand half those words. And I'm human btw.
    - n_ary 2y ago
      It is fine, if those terms are not relevant to you. We learn when we need to. My point was that, a Jr dev will do some research and then try to implement a solution, while o1 being praised as immediately replacement of Jr devs is not even close and only shows as much correct implementation as its training data.
  - wruza 2y ago
    very far away from a Jr dev fresh out of college Feels like your Jr position claims 3 YoE in hardcore backend. Last time I’ve seen them, they discussed == vs === and “in” vs “of”. Wouldn’t hold my breath asking them to implement a rate limiter mw or any sort of debouncer. Unless humanity boosted their average intelligence very recently, ofc, where you’re an async networking specialist fresh outta school.
refurb 2y ago
It would be interesting if AI could be used to isolate bugs? Have it run through all the branches of spaghetti code and isolate the problematic sections along with proposed changes.
scruple 2y ago
And that's to say nothing of the fact that the programs we build exist in an ecosystem of other programs, and sometimes you're lucky if you even have a copy of the source available... And that's without mentioning that it's all networked and you're dependent on other systems all the way down. So much of the work I do today is black box debugging where I can only make broad assumptions about the behavior of systems that are distant to my own based on their past performance, based on some notes someone took from a meeting 6 months ago, based on what the engineers on that team believe their systems are doing, etc... A lot of times at work, support teams modify their systems and don't bother communicating what they're doing, they seem to rely on scream tests to figure out who is even using their services and for what purpose. It's a total fucking mess and I laugh whenever I hear someone suggest that "AI" is going to replace what I do when some massive % of what I do has fucking nothing to do with code at all.

015a 2y ago

> original GPT4 was the best coding LLM I dealt with, and it was 1.5+ years ago. the context size was pitiful, yes, but it was a lot smarter than the current crop of big models. Anytime someone says "this is the worst it'll ever be" I can't help but think "oh sweet summer child"; and then I ask them, what's your favorite operating system you've ever used? The answer heavily depends on the person's age. Windows 10 or 7 dominates the conversation, and for slightly older folks, XP. "MacOS Snow Leopard" gets said a lot, that was a beloved MacOS release because they very famously said "no new features we're just fixing the old ones" and they actually did it. No one ever says "MacOS Sequoia" or "Windows 11". Man, the discourse surrounding the new iOS 18 Photos app is reaching a fever pitch. Sorry for the weird music on this, but its the first clip I could find of even, gasp, Elon Musk, stating this very real fact [1]: Technology does not automatically improve. In fact, it seems like especially software loves to get worse, and it takes a focused effort, from talented individuals, as far removed from money and business and agilescrum as possible to keep things operationally stable. But, sure, definitely the new metaversecryptoai bubblescam will escape that and be built to the highest possible quality. There definitely won't be investors and men in suits in every room saying "we're losing billions is there a knob we can turn to reduce electricity usage?" OpenAI is definitely raising billions to fund the R&D of the next level of intelligence, and certainly not because without that capital they'd be bankrupt in three months. [1] https://www.youtube.com/watch?v=aDC6kWC8y2Y https://www.youtube.com/watch?v=aDC6kWC8y2Y

shepherdjerred 2y ago
I would pick Windows 11 (LTSC) over any prior version of Windows, and I would pick macOS Sequoia over any prior version of macOS.
- 015a 2y ago
  [flagged]
  - shepherdjerred 2y ago
    It's uncharitable of you to consider my opinion as being contrarian versus being genuine preference. Alternatively you could have chosen to be curious about my response that deviates from your understanding. I've used Windows 97 onwards, macOS around Mavericks, and various desktop Linux distros from around 2013. I would have told you that I like Windows 11 (LTSC) because it is much more visually pleasing than 7/8/10. The OS works reliably and I overall enjoy using it much more than the Linux distros I tried out before the LTSC build was made available. I like macOS Sequoia (and iOS 18) because Siri has gotten substantially better and I'm excited about Apple Intelligence. iOS 18 adds RCS which has been great and I actually do like Photos quite a bit, but I only moved to iCloud Photos while I was on the beta, so I can't compare to the old Photos app.
    - Retric 2y ago
      That built in advertising is definitely aesthetically pleasing. It just makes the whole UI pop.
      - shepherdjerred 2y ago
        This is another unnecessarily snarky reply. I’ve specifically mentioned the LTSC build when mentioning Windows 11, which does not have the normal bloat. The consumer version of Windows 11 is terrible IMO, but that’s more a business decision than a technical one.

lokar 2y ago

Even just the ability to reproduce a non-trivial bug, and then work out what general area of the codebase might be in seems out of reach for an LLM

skeptrune 2y ago
Yes! I think I'm going to make LLM bug reproduction my primary response to AI developer maximalists.
- beeflet 2y ago
  Maybe the solution isn't to train LLMs on human-written code, but to train some sort of text-to-AST generator on descriptions of programs and then the output of programs. That way it doesn't learn to include human-inspired bugs at least.
  - mnky9800n 2y ago
    I always wondered why bother having an AI generate human readable code. Isn't it a machine? Why not have it generate machine code? of course, it wouldn't be readable, but would it be performant? Maybe. I guess my point is that if an AI is doing computing why would it doing computing resemble a person doing it? It's much closer to the machine then me. Why not take advantage of that?
    - delapid 2y ago
      Isn’t that kind of what neural networks are in a way? Large blobs of computation we cant really parse fully
      - mnky9800n 2y ago
        Yea but there are a bunch of layers, lol, between that specification and the machine and I'm wondering if there's a way to reduce that assuming it's a bottleneck. But I'm not sure it is I'm just guessing.

moi2388 2y ago

I hate to break it to you, but that’s exactly what ChatGPT o1 excels at. I swear to god, it’s better than our junior devs fresh out of college. Like, I let the LLM solve a PBI and it’s just faster with better code. Actual devs will still have a job, but the low hanging write some code guys? They’re done. And a lot of companies had a bunch of them..

n_ary 2y ago
I hate to break it to you but o1 is like far off of anything what you describe. Yesterday, I was testing some stuff and told it in very precise prompt to write a dripping bucket rate limiting middleware and it just literally named a var “bucket” with a timeout and then implemented something that will wait for the next function to timeout! I then said, heh and asked it to implement a circuit breaker pattern using axios, and it again implemented a timeout and added it to axios options! Could be am dumb or o1 was hallucinating on me intentionally, but the thing is very far away from a Jr dev fresh out of college.
- oefnak 2y ago
  I don't even understand half those words. And I'm human btw.
  - n_ary 2y ago
    It is fine, if those terms are not relevant to you. We learn when we need to. My point was that, a Jr dev will do some research and then try to implement a solution, while o1 being praised as immediately replacement of Jr devs is not even close and only shows as much correct implementation as its training data.
- wruza 2y ago
  very far away from a Jr dev fresh out of college Feels like your Jr position claims 3 YoE in hardcore backend. Last time I’ve seen them, they discussed == vs === and “in” vs “of”. Wouldn’t hold my breath asking them to implement a rate limiter mw or any sort of debouncer. Unless humanity boosted their average intelligence very recently, ofc, where you’re an async networking specialist fresh outta school.

refurb 2y ago

It would be interesting if AI could be used to isolate bugs? Have it run through all the branches of spaghetti code and isolate the problematic sections along with proposed changes.

scruple 2y ago

And that's to say nothing of the fact that the programs we build exist in an ecosystem of other programs, and sometimes you're lucky if you even have a copy of the source available... And that's without mentioning that it's all networked and you're dependent on other systems all the way down. So much of the work I do today is black box debugging where I can only make broad assumptions about the behavior of systems that are distant to my own based on their past performance, based on some notes someone took from a meeting 6 months ago, based on what the engineers on that team believe their systems are doing, etc... A lot of times at work, support teams modify their systems and don't bother communicating what they're doing, they seem to rely on scream tests to figure out who is even using their services and for what purpose. It's a total fucking mess and I laugh whenever I hear someone suggest that "AI" is going to replace what I do when some massive % of what I do has fucking nothing to do with code at all.