3 ms·

wait, did you see the part where the person you are replying to said that writing the code themself was essential to correctly solving the problem? Because the

wait, did you see the part where the person you are replying to said that writing the code themself was essential to correctly solving the problem? Because they didn't understand the architecture or the domain models otherwise. Perhaps in your case you do have strong hands-on experience with the domain models, which may indeed have shifted you job requirements to supervising those implementing the actual models. I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job? If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to? Funny story-- I asked a LLM to review a call transcript to see if the caller was an existing customer. The LLM said True. It was only when I looked closer that I saw that the LLM mean "True-- the caller is an existing customer of one of our competitors". Not at all what I meant.

vidarh 9mo ago
I saw that part and I disagreed with the very notion, hence why I wrote what I did. > Because they didn't understand the architecture or the domain models otherwise. My point is that requiring or expecting an in-depth understanding of all the algorithms you rely on is not a productive use of developer time, because outside narrow niches it is not what we're being paid for. It is also not something the vast majority of us do now, or have done for several decades. I started with assembler, but most developers have never-ever worked less than a couple of abstractions up, often more, and leaned heavily on heaps of code they do not understand because it is not necessary. Sometimes it is. But for the vast majority of us pretending it is necessary all the time or even much of the time is a folly. > I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job? Growing the people under me involves teaching them to solve problems, and already long before AI that typically involved teaching developers to stop obsessing over details with low ROI for the work they were actually doing in favour of understanding and solving the problems of the business. Often that meant making them draw a line between what actually served the needs they were paid to solve rather than the ones that were personally fun to them (I've been guilty of diving into complex low-level problems I find fun rather than what solves the highest ROI problems too - ask me about my compilers, my editor, my terminal - I'm excellent at yak shaving, but I work hard to keep that away from my work) > If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to? For AI use: Tests. Tests. More tests. And, yes, skills and agents. Not primarily even to verify that it understands the specs, but to create harnesses to run them in agent loops without having to babysit them every step of the way. If you use AI and spend your time babysitting them, you've become a glorified assistant to the machine.
- lazide 9mo ago
  Most of the tests are BS too. And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field. Especially if it’s doing it inline somewhere. The underlying issue with AI slop, is that it’s harder to recognize unless you look closely, and then you realize the whole thing is bullshit.
  - smallerfish 9mo ago
    > Most of the tests are BS too. Only if you don't constrain the tests. If you use agents adversarially in generating test cases, tests and review of results, you can get robust and tight test cases. Unless you're in research, most of what we do in our day jobs is boilerplate. Using these tools is not yet foolproof, but with some experience and experimentation you can get excellent results.
    - lazide 9mo ago
      I don’t have to do boilerplate, generally. And with all the stacking LLMs against each other, that just sounds like more work than just… writing the damn tests.
      - smallerfish 9mo ago
        > I don’t have to do boilerplate, generally. I meant this more in the sense of there is nothing new under the sun, and that LLMs have been trained on essentially everything that's available online "under the sun". Sure, there are new SaaS ideas every so often, but the software to produce the idea is rarely that novel (in that you can squint and figure out roughly how it works without thinking too hard), and is in that sense boilerplate.
        lazide 9mo ago
        hahaha, oh boy. that is roughly as useful or accurate as saying that all machines are just combinations of other machines, and hence there is nothing unique about any machine.
empath75 9mo ago
If your product has code on it that can only be understood and worked on by the person that wrote it, then your code is too complex and underdocumented and/or doesn't have enough test coverage. Your time would be better spent, in a permanent code base, trying to get that LLM to understand something than it would be trying to understand the thing yourself. It might be the case that you need to understand the thing more thoroughly yourself so you can explain it to the LLM, and it might be the case that you need to write some code so that you can understand it and explain it, but eventually the LLM needs to get it based on the code comments and examples and tests.

vidarh 9mo ago

I saw that part and I disagreed with the very notion, hence why I wrote what I did. > Because they didn't understand the architecture or the domain models otherwise. My point is that requiring or expecting an in-depth understanding of all the algorithms you rely on is not a productive use of developer time, because outside narrow niches it is not what we're being paid for. It is also not something the vast majority of us do now, or have done for several decades. I started with assembler, but most developers have never-ever worked less than a couple of abstractions up, often more, and leaned heavily on heaps of code they do not understand because it is not necessary. Sometimes it is. But for the vast majority of us pretending it is necessary all the time or even much of the time is a folly. > I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job? Growing the people under me involves teaching them to solve problems, and already long before AI that typically involved teaching developers to stop obsessing over details with low ROI for the work they were actually doing in favour of understanding and solving the problems of the business. Often that meant making them draw a line between what actually served the needs they were paid to solve rather than the ones that were personally fun to them (I've been guilty of diving into complex low-level problems I find fun rather than what solves the highest ROI problems too - ask me about my compilers, my editor, my terminal - I'm excellent at yak shaving, but I work hard to keep that away from my work) > If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to? For AI use: Tests. Tests. More tests. And, yes, skills and agents. Not primarily even to verify that it understands the specs, but to create harnesses to run them in agent loops without having to babysit them every step of the way. If you use AI and spend your time babysitting them, you've become a glorified assistant to the machine.

lazide 9mo ago
Most of the tests are BS too. And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field. Especially if it’s doing it inline somewhere. The underlying issue with AI slop, is that it’s harder to recognize unless you look closely, and then you realize the whole thing is bullshit.
- smallerfish 9mo ago
  > Most of the tests are BS too. Only if you don't constrain the tests. If you use agents adversarially in generating test cases, tests and review of results, you can get robust and tight test cases. Unless you're in research, most of what we do in our day jobs is boilerplate. Using these tools is not yet foolproof, but with some experience and experimentation you can get excellent results.
  - lazide 9mo ago
    I don’t have to do boilerplate, generally. And with all the stacking LLMs against each other, that just sounds like more work than just… writing the damn tests.
    - smallerfish 9mo ago
      > I don’t have to do boilerplate, generally. I meant this more in the sense of there is nothing new under the sun, and that LLMs have been trained on essentially everything that's available online "under the sun". Sure, there are new SaaS ideas every so often, but the software to produce the idea is rarely that novel (in that you can squint and figure out roughly how it works without thinking too hard), and is in that sense boilerplate.
      - lazide 9mo ago
        hahaha, oh boy. that is roughly as useful or accurate as saying that all machines are just combinations of other machines, and hence there is nothing unique about any machine.

empath75 9mo ago

If your product has code on it that can only be understood and worked on by the person that wrote it, then your code is too complex and underdocumented and/or doesn't have enough test coverage. Your time would be better spent, in a permanent code base, trying to get that LLM to understand something than it would be trying to understand the thing yourself. It might be the case that you need to understand the thing more thoroughly yourself so you can explain it to the LLM, and it might be the case that you need to write some code so that you can understand it and explain it, but eventually the LLM needs to get it based on the code comments and examples and tests.