3 ms·

Someone on my team complained to me about some seemingly relatively easy task yesterday. They claimed I was pushing more work onto them as I'm working on the b

Someone on my team complained to me about some seemingly relatively easy task yesterday. They claimed I was pushing more work onto them as I'm working on the backend and they are working on the frontend. This puzzled me so I tried it and ended up doing the work in about 1.5h I did struggle through the poor docs of a relatively new library, but it wasn't hard. This got me wondering: maybe they have become so dependent on AI copilots that what should have been an easy task was seen as insurmountably hard because the LLM didn't have info on this new-ish library.

smugglerFlynn 2y ago
Have you considered extra time it would take for some person other than yourself to get onboarded into the problem you are solving? It can quickly add 2-3 extra hours on top of that [seemingly easy] work.
- aprilthird2021 2y ago
  Maybe, but most people used to trust that their coworkers had the ability to quickly learn what needed to be learned to solve a small problem. AI has made people dependent on lose that skill a bit. I've seen myself people confronted with something they need to learn about and come back with something not much more than "AI said this, so here you go"
  - klabb3 2y ago
    I think this is very real, and will act as a force towards center of gravity of the LLMs which means popular and established tech. It might even create a divide between ”prompt engineer engineers” and those who understand. That assumes using LLMs will not improve your understanding compared to traditional means which isn’t necessarily true. I dislike LLMs as coders but for learning new topics quickly they are better than obscure literature or googling for deep blog posts. That said the pure prompt engineers might suffer similar fate as Stack Overflow engineers, for another reason: the limiting factor of building software is not shitting out code or even tests, it’s curbing the accumulated total complexity in the project as it grows. This is incredibly hard even for humans but the best engineers can reduce it. These most difficult problems have at least 2-3 properties that make them almost impossible for LLMs today: they’re non-local, hard to quantify and have enough constraints to mandate solutions that end in the fringes of training data set. Even simple self-contained LLM solutions introduce more complexity than necessary.
    - CharlieDigital 2y ago
      > I think this is very real, and will act as a force towards center of gravity of the LLMs which means popular and established tech Maybe it means devs will invest more time in putting together quality docs because that'll be the deciding factor if an LLM can figure out how to use your library; interesting!
      - klabb3 2y ago
        Sure thing. But this reminds me of another thought I’ve had. That LLMs are likely better at writing (and refactoring) docs and docstrings than code – certainly not better than the best humans but the average bar is very low here. So that’s a force that might improve total software quality if it can be integrated to workflows effectively.
      - aprilthird2021 2y ago
        I think there will be SEO but focused on getting AIs to gobble up your content. Back to the days of tons of hidden text in multiple websites
- CharlieDigital 2y ago
  I also didn't know the library nor have any exposure (yesterday was literally my first time looking at the docs for this particular library) and still got it done in 1.5h because someone has to do it. And they've been at the company working with the FE stack longer than me by a few months! I'm not even on the frontend team and decided to take the matter into my hands because I was pretty sure my ask wasn't too onerous so I wanted to double check. It had an out-of-the-box, first party add on package to do exactly what we needed to get data in the right shape on the backend, but the dev made it seem like I was pushing more work on FE. I'm just trying to get the right data format...
hakaneskici 2y ago
Nice example. If I were that person; from my perspective, I could say the EXACT same thing about my frustration with LLM. That's my mental model, I am trying to make my AI peer to build something and it is complaining about not knowing this new-ish library. They just didn't RTFM :)
- CharlieDigital 2y ago
  The thing is, even I hadn't seen the docs before yesterday, but I could intuit through experience working with similar libraries that my ask wasn't that onerous. The gist of it was to take the JSON representation of an editor and convert it to Markdown. Every popular editor library has an add on or option to export Markdown as well as import Markdown. But on import/export, you then need to almost always write a small transformer for any custom visual elements encoded as text. Why MD? Because the user is writing a prompt so we need it as MD so it makes sense to transact this data in MD. It just so happens that the library is newish and docs are sparse in some areas. But totally manageable just by looking at the source how to do it. I can't say for certain where the disconnect is in this whole thing, but to me it felt like "this isn't easy (because the LLM can't do it), so we shouldn't do it this way".

smugglerFlynn 2y ago

Have you considered extra time it would take for some person other than yourself to get onboarded into the problem you are solving? It can quickly add 2-3 extra hours on top of that [seemingly easy] work.

aprilthird2021 2y ago
Maybe, but most people used to trust that their coworkers had the ability to quickly learn what needed to be learned to solve a small problem. AI has made people dependent on lose that skill a bit. I've seen myself people confronted with something they need to learn about and come back with something not much more than "AI said this, so here you go"
- klabb3 2y ago
  I think this is very real, and will act as a force towards center of gravity of the LLMs which means popular and established tech. It might even create a divide between ”prompt engineer engineers” and those who understand. That assumes using LLMs will not improve your understanding compared to traditional means which isn’t necessarily true. I dislike LLMs as coders but for learning new topics quickly they are better than obscure literature or googling for deep blog posts. That said the pure prompt engineers might suffer similar fate as Stack Overflow engineers, for another reason: the limiting factor of building software is not shitting out code or even tests, it’s curbing the accumulated total complexity in the project as it grows. This is incredibly hard even for humans but the best engineers can reduce it. These most difficult problems have at least 2-3 properties that make them almost impossible for LLMs today: they’re non-local, hard to quantify and have enough constraints to mandate solutions that end in the fringes of training data set. Even simple self-contained LLM solutions introduce more complexity than necessary.
  - CharlieDigital 2y ago
    > I think this is very real, and will act as a force towards center of gravity of the LLMs which means popular and established tech Maybe it means devs will invest more time in putting together quality docs because that'll be the deciding factor if an LLM can figure out how to use your library; interesting!
    - klabb3 2y ago
      Sure thing. But this reminds me of another thought I’ve had. That LLMs are likely better at writing (and refactoring) docs and docstrings than code – certainly not better than the best humans but the average bar is very low here. So that’s a force that might improve total software quality if it can be integrated to workflows effectively.
    - aprilthird2021 2y ago
      I think there will be SEO but focused on getting AIs to gobble up your content. Back to the days of tons of hidden text in multiple websites
CharlieDigital 2y ago
I also didn't know the library nor have any exposure (yesterday was literally my first time looking at the docs for this particular library) and still got it done in 1.5h because someone has to do it. And they've been at the company working with the FE stack longer than me by a few months! I'm not even on the frontend team and decided to take the matter into my hands because I was pretty sure my ask wasn't too onerous so I wanted to double check. It had an out-of-the-box, first party add on package to do exactly what we needed to get data in the right shape on the backend, but the dev made it seem like I was pushing more work on FE. I'm just trying to get the right data format...

hakaneskici 2y ago

Nice example. If I were that person; from my perspective, I could say the EXACT same thing about my frustration with LLM. That's my mental model, I am trying to make my AI peer to build something and it is complaining about not knowing this new-ish library. They just didn't RTFM :)

CharlieDigital 2y ago
The thing is, even I hadn't seen the docs before yesterday, but I could intuit through experience working with similar libraries that my ask wasn't that onerous. The gist of it was to take the JSON representation of an editor and convert it to Markdown. Every popular editor library has an add on or option to export Markdown as well as import Markdown. But on import/export, you then need to almost always write a small transformer for any custom visual elements encoded as text. Why MD? Because the user is writing a prompt so we need it as MD so it makes sense to transact this data in MD. It just so happens that the library is newish and docs are sparse in some areas. But totally manageable just by looking at the source how to do it. I can't say for certain where the disconnect is in this whole thing, but to me it felt like "this isn't easy (because the LLM can't do it), so we shouldn't do it this way".