6 ms·

This is ominous and very depressing given what we've recently learned / reconfirmed about LLMs sapping our ability to persist through difficult problems: > The

This is ominous and very depressing given what we've recently learned / reconfirmed about LLMs sapping our ability to persist through difficult problems: > There were 2 or 3 bugs that stumped me, and after 20 min or so of debugging I asked Claude for some advice. But most of the debugging was by hand! Twenty whole minutes. Us old-timers (I am 39) are chortling. I am not trying to knock the author specifically. But he was doing this for education, not for work. He should have spent more like 6 hours before desperately reaching for the LLM. I imagine after 1 hour he would have figured it out on his own.

sho_hn 6mo ago
Now imagine someone else reading this and genuinely considering 20 minutes a long time to wait :-)
alemwjsl 6mo ago
Yep and after 6 hours don't reach for LLM, instead: * Ask someone to come over and look * Come back the next day, work on something else * Add comment # KNOWN-ISSUE: ...., and move on and forget about it. But year spent days on a bug at work before ha ha!
- justonceokay 6mo ago
  You say this as if the LLM isn’t committing things it doesn’t even recognize as bugs if you don’t babysit it. I’d rather have a codebase with a few very well marked evil zones, rather than a codebase no one has read. All code contains demons and it’s good to have an understanding of their locations and relative power
- moregrist 6mo ago
  > Come back the next day, work on something else This is a tried and true way of working on puzzles and other hard problems. I generally have 2-4 important things in flight, so I find myself doing this a lot when I get stuck.
  - ignoramous 6mo ago
    > This is a tried and true way of working on puzzles and other hard problems ... generally have 2-4 important things in flight Just a note that, for chronic procrastinators, having 2 to 4 important things going on is a trigger & they'd rather not complete anything. I wonder, for such folks, if SoTA LLMs help with procrastination?
- calvinmorrison 6mo ago
  so many eureka moments of mine were simply sitty on the MTA
Trasmatta 6mo ago
YES. I don't know how many multi WEEK sessions of debugging I've been through in my career. Frustrating, but so many valuable lessons learned in the process. LLMs are absolutely causing us to lose something very important.
- voidfunc 6mo ago
  If I told someone I spent a week debugging a problem these days I think I would get laughed out of the call. Even a day might hit somw chuckles. If you cant fix the bug just slop some code over it so its more hidden. This is all gonna be fascinating in 5-10 years.
  - seanw444 6mo ago
    This really does feel like a mass hysteria event. Bizarre to have to live through it.
    - lstodd 6mo ago
      It's third major on my memory: first, cagw/global warming, then covid, and now this. Many minor happened along, like crypto+nft stuff or renaming master branches and adding codes of conduct. I think it's just human nature. Fascinating nevertheless.
  - SlinkyOnStairs 6mo ago
    This does depend on who you are; If you're a senior with 10+ years of experience, it's a failure of your abilities to cut your losses or know when to seek help if you take far too long debugging something. But for juniors, it's invaluable experience. And as a field we're already seeing problems resulting from the new generations of juniors being taught with modern web development, whose complexity is very obstructing of debugging.
    - badc0ffee 6mo ago
      There are definitely situations where you can't ask for help and you can't turn your back on the bug. I worked on a project that depended on an open source but deprecated/unmaintained Linux kernel module that we used for customers running RHEL[1]. There were a number of serious bugs causing panics that we encountered, but only for certain customers with high VFS workloads. I spent days to a week+ on each one, reading kernel code, writing userland utilities to repro the problem, and finally committing fixes to the module. I was the only one on the team up to the task. We couldn't tell the customers to upgrade, we couldn't write an alternative module in a reasonable timeframe, and they paid us a lot of money, so I did what I had to do. I'm sure there are lots of other examples like this out there. [1] Known for its use of ancient kernels with 10000 patches hand-picked by Red Hat. At least at the time (5-10 years ago).
Gigachad 6mo ago
Often when LLMs give me some command option or advice I haven’t seen before I try to independently verify it. And I’ve often been frustrated just how hard it is to find this info from the source documents. Though a lot of the time this is more an inefficiency of the documentation and Google rather than something only LLMs could do.
- nyarlathotep_ 6mo ago
  As the rate of 'hallucinations' seems to have dropped dramatically (at least IME as regards non-existent flags and the like), I'm more concerned with usage. I often use grep.app/GH code search to look for usage examples as a sanity check when things look "off", for exactly the reason you described--there's often a total lack of good documentation on things like that, especially on "younger" tools/stuff.
- skydhash 6mo ago
  Most established projects has quite good documentation. And if they don’t, that’s because they consider the code as the documentation and just provide with an overview. The source code is the main truth and there’s a trick to reading it quickly. But that’s acquired by playing around with a lot of projects
derangedHorse 6mo ago
I'm sure the author will encounter problems where the only way to solve them will be the marginal effort provided by a human. At that point he won't be just be solving problems to work his brain, but also to accomplish a goal.
usernametaken29 6mo ago
I’ve worked in financial modelling before where you need to make sure results are correct, not approximate. One time there was a nasty bug in pandas multiindexes (admittedly we banned pandas for all new code because it just can’t do semver). Spent 9 days to debug three lines of code. Endurance and patience are learned skills and sometimes they’re the only way you can get a correct verifiable solution.
- BodyCulture 6mo ago
  What are you using instead of pandas? Thanks!
  - usernametaken29 6mo ago
    Either nothing (a lot of functionality of pandas can be done with simple plain python) or polars for complex queries. Also look at the statistics module which has a lot of useful things in there
JuniperMesos 6mo ago
Why shouldn't someone consult some kind of external resource for help, after struggling with a specific coding problem for 20 minutes? Why is 6 hours the right amount of time to timebox this to?
- thrance 6mo ago
  The struggle is the point, that's how you learn. If you offload your task to someone/something else after barely 20 minutes of head scratching, you've missed the plot entirely.
  - SoftTalker 6mo ago
    [dead]
- th0ma5 6mo ago
  [dead]
- demorro 6mo ago
  20 minutes is not enough time to drive you into a state of desperation, where you may be forced to try something novel which will expand your mind and future capabilities in unknown and unexpected ways. You might be driven to contact another human being, for example.
  - YesBox 6mo ago
    I was typing up a long and somewhat boring story. So, the short of it is that this is a great insightful comment that I can back up with my own experience in making a game from scratch over the last 4+ years.
- bhelkey 6mo ago
  There wasn't always an external resource to go to for help. Especially for legacy pieces of software, it was easy to become the person with most context on the team.
raw_anon_1111 6mo ago
Why? I’m as old timer as old timer can get - started programming as a hobby in 1986 in assembly on an Apple //e in 65C02 assembly language. But just today a bug was reported by a customer (we are still in testing not a production bug). I implemented this project myself from an empty git repo and an empty AWS account including 3 weeks of pre implementation discovery. I reproduced the issue and through the problem at Claude with nothing but two pieces of information - the ID of the event showing the bug and the description. It worked backwards looking at the event stream in the database, looking at the code that stored the event stream, looking at the code that generated the event stream (separate Lambda), looking at the actual config table and found the root cause in 3 minutes. After looking at the code locally, it even looked at the cached artifacts of my build and verified that what was deployed was the same thing that I had locally (same lambda deployment version in AWS as my artifacts). I had it document the debug steps it took in an md file. Why make life harder on myself? Even if it were something I was doing as a hobby, I have a wife who I want to spend time with, I’m a gym rat and I’m learning Spanish. Why would I waste 6 hours doing something that a computer could do for me in 5 minutes? Assuming he has a day job and gets off at 6, he would be spending all of his off time chasing down a bug that he could be using doing something else.
- grebc 6mo ago
  It’s always the journey that matters. If you’re experienced as you are, you’re not learning the same way a junior assigned this might learn from it.
  - raw_anon_1111 6mo ago
    So the project I mentioned while I did write every single line of app code and IAC, made every architectural decision, etc., I did come on an off the project over the course of a year and I couldn’t even remember some of the decisions I made. I also used Codex and asked questions about how the codebase worked to refresh my own memory. Why wouldn’t a junior developer do the same? I mentioned that I had Codex describe in detail how it debugged it. It walked through each query it did, the lines of code it looked at and the IAC. It jogged my memory about code I wrote a year ago and after being on other projects
j1elo 6mo ago
I just grabbed an Android remaster of "Broken Sword: Shadow of the Templars", a 90's point-and-click adventure that has been added a hints system which pops up automatically after a timeout of the player not progressing. This can be set as far as 1h of being stuck. Can also be 5 minutes. But by default it is 30 seconds. My inner kid was screaming "that's cheating!" :-D but on second thought it is a very cool feature for us busy adults, however it's sad the extremes that gamedevs have to go in order to appease the short-term mindless consumers of today's tik-toks. But more seriously, where's the joy of generating long-standing memories of being stuck for a while on a puzzle that will make you remember that scene for 30 years? An iconic experience that separates this genre from just being an animated movie with more steps. I couldn't imagine "Monkey Island II but every 30 seconds we push you forward". Gimme that monkey wrench. TFA and this comment just made me have this thought about today's pace of consumption, work, and even gaming.
Tanoc 6mo ago
Often times the fastest way to debug is to write it wrong, write it wrong again, find an example where somebody wrote it right, write that wrong in your own file, then figure out what you changed to adapt it that made it go wrong. If anyone remembers middleschool mathematics this is the coding example of the teacher making you write out the equations in their longest form instead of shortcutting. It's done this way because it shows you your exact train of thought and where you went wrong. That sticks in your head. You understand the problem by understanding yourself. Giving up after twenty minutes instead of stopping, clearing your active cognitive load, and then coming back erases your ability to understand that train of thought. For a comparison it's like being in first person view in a videogame, and the only thing you have is the ability to look behind you, versus being able to bring up a map that has an overhead view. In first person you're likely to lose where exactly you went to get where you are, while with the overhead view map you can orient your traveled route according to landmarks and distance.

sho_hn 6mo ago

Now imagine someone else reading this and genuinely considering 20 minutes a long time to wait :-)

alemwjsl 6mo ago

Yep and after 6 hours don't reach for LLM, instead: * Ask someone to come over and look * Come back the next day, work on something else * Add comment # KNOWN-ISSUE: ...., and move on and forget about it. But year spent days on a bug at work before ha ha!

justonceokay 6mo ago
You say this as if the LLM isn’t committing things it doesn’t even recognize as bugs if you don’t babysit it. I’d rather have a codebase with a few very well marked evil zones, rather than a codebase no one has read. All code contains demons and it’s good to have an understanding of their locations and relative power
moregrist 6mo ago
> Come back the next day, work on something else This is a tried and true way of working on puzzles and other hard problems. I generally have 2-4 important things in flight, so I find myself doing this a lot when I get stuck.
- ignoramous 6mo ago
  > This is a tried and true way of working on puzzles and other hard problems ... generally have 2-4 important things in flight Just a note that, for chronic procrastinators, having 2 to 4 important things going on is a trigger & they'd rather not complete anything. I wonder, for such folks, if SoTA LLMs help with procrastination?
calvinmorrison 6mo ago
so many eureka moments of mine were simply sitty on the MTA

Trasmatta 6mo ago

YES. I don't know how many multi WEEK sessions of debugging I've been through in my career. Frustrating, but so many valuable lessons learned in the process. LLMs are absolutely causing us to lose something very important.

voidfunc 6mo ago
If I told someone I spent a week debugging a problem these days I think I would get laughed out of the call. Even a day might hit somw chuckles. If you cant fix the bug just slop some code over it so its more hidden. This is all gonna be fascinating in 5-10 years.
- seanw444 6mo ago
  This really does feel like a mass hysteria event. Bizarre to have to live through it.
  - lstodd 6mo ago
    It's third major on my memory: first, cagw/global warming, then covid, and now this. Many minor happened along, like crypto+nft stuff or renaming master branches and adding codes of conduct. I think it's just human nature. Fascinating nevertheless.
- SlinkyOnStairs 6mo ago
  This does depend on who you are; If you're a senior with 10+ years of experience, it's a failure of your abilities to cut your losses or know when to seek help if you take far too long debugging something. But for juniors, it's invaluable experience. And as a field we're already seeing problems resulting from the new generations of juniors being taught with modern web development, whose complexity is very obstructing of debugging.
  - badc0ffee 6mo ago
    There are definitely situations where you can't ask for help and you can't turn your back on the bug. I worked on a project that depended on an open source but deprecated/unmaintained Linux kernel module that we used for customers running RHEL[1]. There were a number of serious bugs causing panics that we encountered, but only for certain customers with high VFS workloads. I spent days to a week+ on each one, reading kernel code, writing userland utilities to repro the problem, and finally committing fixes to the module. I was the only one on the team up to the task. We couldn't tell the customers to upgrade, we couldn't write an alternative module in a reasonable timeframe, and they paid us a lot of money, so I did what I had to do. I'm sure there are lots of other examples like this out there. [1] Known for its use of ancient kernels with 10000 patches hand-picked by Red Hat. At least at the time (5-10 years ago).

Gigachad 6mo ago

Often when LLMs give me some command option or advice I haven’t seen before I try to independently verify it. And I’ve often been frustrated just how hard it is to find this info from the source documents. Though a lot of the time this is more an inefficiency of the documentation and Google rather than something only LLMs could do.

nyarlathotep_ 6mo ago
As the rate of 'hallucinations' seems to have dropped dramatically (at least IME as regards non-existent flags and the like), I'm more concerned with usage. I often use grep.app/GH code search to look for usage examples as a sanity check when things look "off", for exactly the reason you described--there's often a total lack of good documentation on things like that, especially on "younger" tools/stuff.
skydhash 6mo ago
Most established projects has quite good documentation. And if they don’t, that’s because they consider the code as the documentation and just provide with an overview. The source code is the main truth and there’s a trick to reading it quickly. But that’s acquired by playing around with a lot of projects

derangedHorse 6mo ago

I'm sure the author will encounter problems where the only way to solve them will be the marginal effort provided by a human. At that point he won't be just be solving problems to work his brain, but also to accomplish a goal.

usernametaken29 6mo ago

I’ve worked in financial modelling before where you need to make sure results are correct, not approximate. One time there was a nasty bug in pandas multiindexes (admittedly we banned pandas for all new code because it just can’t do semver). Spent 9 days to debug three lines of code. Endurance and patience are learned skills and sometimes they’re the only way you can get a correct verifiable solution.

BodyCulture 6mo ago
What are you using instead of pandas? Thanks!
- usernametaken29 6mo ago
  Either nothing (a lot of functionality of pandas can be done with simple plain python) or polars for complex queries. Also look at the statistics module which has a lot of useful things in there

JuniperMesos 6mo ago

Why shouldn't someone consult some kind of external resource for help, after struggling with a specific coding problem for 20 minutes? Why is 6 hours the right amount of time to timebox this to?

thrance 6mo ago
The struggle is the point, that's how you learn. If you offload your task to someone/something else after barely 20 minutes of head scratching, you've missed the plot entirely.
- SoftTalker 6mo ago
  [dead]
th0ma5 6mo ago
[dead]
demorro 6mo ago
20 minutes is not enough time to drive you into a state of desperation, where you may be forced to try something novel which will expand your mind and future capabilities in unknown and unexpected ways. You might be driven to contact another human being, for example.
- YesBox 6mo ago
  I was typing up a long and somewhat boring story. So, the short of it is that this is a great insightful comment that I can back up with my own experience in making a game from scratch over the last 4+ years.
bhelkey 6mo ago
There wasn't always an external resource to go to for help. Especially for legacy pieces of software, it was easy to become the person with most context on the team.

raw_anon_1111 6mo ago

Why? I’m as old timer as old timer can get - started programming as a hobby in 1986 in assembly on an Apple //e in 65C02 assembly language. But just today a bug was reported by a customer (we are still in testing not a production bug). I implemented this project myself from an empty git repo and an empty AWS account including 3 weeks of pre implementation discovery. I reproduced the issue and through the problem at Claude with nothing but two pieces of information - the ID of the event showing the bug and the description. It worked backwards looking at the event stream in the database, looking at the code that stored the event stream, looking at the code that generated the event stream (separate Lambda), looking at the actual config table and found the root cause in 3 minutes. After looking at the code locally, it even looked at the cached artifacts of my build and verified that what was deployed was the same thing that I had locally (same lambda deployment version in AWS as my artifacts). I had it document the debug steps it took in an md file. Why make life harder on myself? Even if it were something I was doing as a hobby, I have a wife who I want to spend time with, I’m a gym rat and I’m learning Spanish. Why would I waste 6 hours doing something that a computer could do for me in 5 minutes? Assuming he has a day job and gets off at 6, he would be spending all of his off time chasing down a bug that he could be using doing something else.

grebc 6mo ago
It’s always the journey that matters. If you’re experienced as you are, you’re not learning the same way a junior assigned this might learn from it.
- raw_anon_1111 6mo ago
  So the project I mentioned while I did write every single line of app code and IAC, made every architectural decision, etc., I did come on an off the project over the course of a year and I couldn’t even remember some of the decisions I made. I also used Codex and asked questions about how the codebase worked to refresh my own memory. Why wouldn’t a junior developer do the same? I mentioned that I had Codex describe in detail how it debugged it. It walked through each query it did, the lines of code it looked at and the IAC. It jogged my memory about code I wrote a year ago and after being on other projects

j1elo 6mo ago

I just grabbed an Android remaster of "Broken Sword: Shadow of the Templars", a 90's point-and-click adventure that has been added a hints system which pops up automatically after a timeout of the player not progressing. This can be set as far as 1h of being stuck. Can also be 5 minutes. But by default it is 30 seconds. My inner kid was screaming "that's cheating!" :-D but on second thought it is a very cool feature for us busy adults, however it's sad the extremes that gamedevs have to go in order to appease the short-term mindless consumers of today's tik-toks. But more seriously, where's the joy of generating long-standing memories of being stuck for a while on a puzzle that will make you remember that scene for 30 years? An iconic experience that separates this genre from just being an animated movie with more steps. I couldn't imagine "Monkey Island II but every 30 seconds we push you forward". Gimme that monkey wrench. TFA and this comment just made me have this thought about today's pace of consumption, work, and even gaming.

Tanoc 6mo ago

Often times the fastest way to debug is to write it wrong, write it wrong again, find an example where somebody wrote it right, write that wrong in your own file, then figure out what you changed to adapt it that made it go wrong. If anyone remembers middleschool mathematics this is the coding example of the teacher making you write out the equations in their longest form instead of shortcutting. It's done this way because it shows you your exact train of thought and where you went wrong. That sticks in your head. You understand the problem by understanding yourself. Giving up after twenty minutes instead of stopping, clearing your active cognitive load, and then coming back erases your ability to understand that train of thought. For a comparison it's like being in first person view in a videogame, and the only thing you have is the ability to look behind you, versus being able to bring up a map that has an overhead view. In first person you're likely to lose where exactly you went to get where you are, while with the overhead view map you can orient your traveled route according to landmarks and distance.