11 ms·

>and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the ha

>and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs. You are getting confused here... LLMs are terrible at doing software engineering jobs. They fall on their face trying to work in sprawling codebases. However, they are excellent at writing small bespoke programs on behalf of regular people. Becky doesn't need ChatGPT to one-shot Excel.exe to keep track of the things she sells at the craft fair that afternoon. This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers.

Avicebron 1y ago
That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough. This smells like "Self-checkouts", "perfect bespoke solution so that everyone can check their own groceries out, no need to wait for a slow/expensive _human_ to get the job done, except you end up basically just doing the cashiers job and waiting around inconveniently whenever you want to get some beer
- satvikpendem 1y ago
  And yet I'd much rather self checkout than have a cashier do it for me.
  - kulahan 1y ago
    It feels like “virtual stuck behind a bus”, then doing the company a favor for the experience.
  - apt-apt-apt-apt 1y ago
    At Costco, they forced me to empty out my very fat cart and scan each item individually. That's when I realized the value of cashiers.
  - libraryatnight 1y ago
    The only time, for me, this is true, is when it's empty and I have very few items. If there's a line, I'd rather wait for the cashier. Self-checkout seems to get clogged up with people waiting for the supervising human the busier it is, so to have a human or two (one ringing one bagging) when I've got a cart and there's a line, is advantageous. I'm also pretty anti-social and actually prefer the robotic banter of quickly checking out with a person to the anxious nightmare of just trying to buy something real quick and now waiting because the machine hates life more than me. Oh and many of the folks doing the bagging at some of the stores are disabled, and I dunno - I hope we're taking care of people in those jobs.
    - satvikpendem 1y ago
      For ninety nine out of a hundred scenarios, I've never had to wait for a supervising human at all. Maybe it's where I shop, but generally speaking, I've almost never had a problem with people needing to see my checked out items.
  - lmm 1y ago
    It's a marginal improvement perhaps, but nothing to write home about. And for other things (e.g. booking travel) the self-service experience can be worse.
    - satvikpendem 1y ago
      Not sure about that, I prefer to know all aspects of booking travel that I would not trust to another agent, human or otherwise. Therefore, I believe your statement applies to some people and not others.
- bgwalter 1y ago
  And often you are still supervised by sometimes rude security guards who demand the receipts. So you have the worst of all worlds: You pay the supermarket, do the work and are treated like dirt.
- Agentlien 1y ago
  On a tangent: what about self scanning? Here in Sweden that is the norm. For the last ten years I've brought my own bags, packed everything like I want it as I grab it - with a quick blip from the scanner - and then checkout is no line, no cashier, no unpacking and packing things again. Just scanning my card and going through a brief screen to check that the summary looks right. Instead of a guard there's a scanner by the exit gate where you scan your ticket. In my case just a small stub since the actual ticket is sent digitally. I think it works so smoothly, much better than before this system was introduced. My recent experience buying groceries in Belgium was very similar.
  - LtWorf 1y ago
    Here in sweden it is not "the norm" by any definition of "norm". Also it's incredibly slower and not "one blip away". And the scanner to exit is terrible for fire safety and accessibility. Also I've seen it break with the result that you had to trigger the alarm to leave. It does not work smoothly at all, but perhaps you live in a different sweden than I.
    - munksbeer 1y ago
      It works pretty well in the UK. The nice thing is, I can pack my bags as I shop, so I don't feel stressed when I get to the checkout. I use this method of shopping whenever available.
      - LtWorf 1y ago
        In sweden you have to unpack and re-pack at the checkout. And if you have a backpack you can't place stuff inside it. No. You have to unpack, place everything on a scale and then place everything in your backpack again. And for items like bread or so you need to navigate a number of menus to find it. And of course you are asked if you are a member, if you'd like to become a member, if you want to do a donation, and a number of other useless questions.
        deleted 1y ago
        [deleted]
- Workaccount2 1y ago
  >That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough. That's exactly why she asks an LLM to do it for her. A program like this would be <1k LOC, well within even the current LLM one-shot-working-app domain.
theptip 1y ago
Right. Jevons Paradox. The lower barrier to entry might mean the average quality of software goes WAY down. But this could be great if it means that there are 10x as many little apps that make one person happy by doing only as much as they need. The above is also consistent with the quantity of excellent software going way up as well (just slower than the below-average stuff).
- LtWorf 1y ago
  What the android/apple appstores have taught me compared to the old symbian appstore is that we don't need more apps, we need decent ones. The current appstores have more.
  - graemep 1y ago
    One of the advantages of FDroid is that although tiny compared to Play Store, it is much better curated.
  - theptip 1y ago
    Discovery definitely becomes more important in this world. Fortunately AI helps with that too.
    - LtWorf 1y ago
      Why do you think it will not soon be biased as search is?
      - theptip 1y ago
        I think AI gives us the opportunity to write our own ranking algorithms. Of course most people will pick an off-the-shelf editorial perspective but I expect these to be much more configurable than the current state with a tiny number of algorithmic gatekeepers.
        LtWorf 1y ago
disambiguation 1y ago
>. They fall on their face trying to work in sprawling codebases. >However, they are excellent at writing small bespoke programs For programmers I predict a future of a million micro services. Sprawling has always been an undesirable and unnecessary byproduct of growing code bases, but there's been no easy solution to that. I wonder if LLMs would perform better on a mono repo of many small services than one sprawling monolith.
- api 1y ago
  Micro service systems are just huge sprawling code bases with more glue code. Calling something over a network instead of via a local function call is still calling something.
  - libraryatnight 1y ago
    For years one of my favorite experiences (amusing given your username) was being on calls for incidents where they get the dev on for X thing and an exec goes "I thought we got rid of that" and a bunch of people sheepishly explain it wasn't really retired... it was repurposed as an API. I especially loved it when the "retirement" of the broken thing was the execs big achievement. (The comedic nuance often being the thing could have been retired for real, but they demanded a timeline that necessitated the "fake" retirement) It still happens but it's not a favorite experience anymore. It's just a source of loathing for MBA culture.
    - api 1y ago
      The pure play MBA is the capitalist West's equivalent of the Soviet apparatchik: https://en.wikipedia.org/wiki/Apparatchik https://en.wikipedia.org/wiki/Apparatchik Been saying that for years. Private equity is perhaps analogous to the Politburo.
  - disambiguation 1y ago
    Another way to look at it is - why force the agent to grapple with the whole code base when they can rapidly standup many single purpose services instead?
tsimionescu 1y ago
Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully? Programming ecosystems are generally very unfriendly to this type of use case, with plenty of setup required to get something usable out even if you have the code available. Especially if you'd like to run the results on your phone. Sure, an LLM might be able to guide you through the steps, and even help when you stumble, but you still have to follow a hundred little steps exactly with no intuition whether things will work out at the end. I very much doubt that this is something many people will even think to ask for, and then follow-through with. Especially since the code quality of LLMs is nowhere near what you make it out to be. You'll still need to bear with them and go through a lot of trial and error to get anything running, especially if you have no knowledge of the terms of art, nor any clue of what might be going wrong if it goes wrong. If you've ever seen consumer bug reports, you might have some idea of the quality of feedback the LLM may be getting back if something is not working perfectly the first time. It's very likely to be closer to "the button you added is not working" than to "when I click the button, the application crashes" or "when I click the button, the application freezes for a few seconds and then pops up an error saying that input validation failed" or "the button is not showing up on the screen" [...]
- azan_ 1y ago
  > Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully? I’m radiologist, I’ve been paying for software that sped up my reporting like 200 usd per month. I’ve remade all the functionality I need in one evening with cursor and added some things that I’ve found missing from the original software.
- Workaccount2 1y ago
  My company is a non-tech company that is in manufacturing. I'm not a programmer and niether is anyone else here. So far I have created 7 programs that are now used daily in production. One of them replaces a $1k/yr/usr CAD package, and another we used to bring in a contractor to write. The others a miscellaneous apps for automating/simplifying our in houses processes. None of the programs are more than 6k LOC.
ben_w 1y ago
> This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers. Kinda. They're good, and I like them, but I think of them like a power tool: just because you can buy an angle grinder or a plasma cutter from a discount supermarket, doesn't mean the tool is safe in the hands of an untrained amateur. Someone using it to help fix up a spreadsheet? Probably fine. But you should at least be able to read the code to the level you don't get this deliberate bad example to illustrate the point: #!/usr/bin/python3 total_sales = 0.0 def add_sale(total_sales, amount): total_sales = total_sales + amount print("total_sales: " + str(total_sales)) Still useful, still sufficiently advanced technology that for most people it is (Clarketech) magic, but also still has some rough edges.

Avicebron 1y ago

That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough. This smells like "Self-checkouts", "perfect bespoke solution so that everyone can check their own groceries out, no need to wait for a slow/expensive _human_ to get the job done, except you end up basically just doing the cashiers job and waiting around inconveniently whenever you want to get some beer

satvikpendem 1y ago
And yet I'd much rather self checkout than have a cashier do it for me.
- kulahan 1y ago
  It feels like “virtual stuck behind a bus”, then doing the company a favor for the experience.
- apt-apt-apt-apt 1y ago
  At Costco, they forced me to empty out my very fat cart and scan each item individually. That's when I realized the value of cashiers.
- libraryatnight 1y ago
  The only time, for me, this is true, is when it's empty and I have very few items. If there's a line, I'd rather wait for the cashier. Self-checkout seems to get clogged up with people waiting for the supervising human the busier it is, so to have a human or two (one ringing one bagging) when I've got a cart and there's a line, is advantageous. I'm also pretty anti-social and actually prefer the robotic banter of quickly checking out with a person to the anxious nightmare of just trying to buy something real quick and now waiting because the machine hates life more than me. Oh and many of the folks doing the bagging at some of the stores are disabled, and I dunno - I hope we're taking care of people in those jobs.
  - satvikpendem 1y ago
    For ninety nine out of a hundred scenarios, I've never had to wait for a supervising human at all. Maybe it's where I shop, but generally speaking, I've almost never had a problem with people needing to see my checked out items.
- lmm 1y ago
  It's a marginal improvement perhaps, but nothing to write home about. And for other things (e.g. booking travel) the self-service experience can be worse.
  - satvikpendem 1y ago
    Not sure about that, I prefer to know all aspects of booking travel that I would not trust to another agent, human or otherwise. Therefore, I believe your statement applies to some people and not others.
bgwalter 1y ago
And often you are still supervised by sometimes rude security guards who demand the receipts. So you have the worst of all worlds: You pay the supermarket, do the work and are treated like dirt.
Agentlien 1y ago
On a tangent: what about self scanning? Here in Sweden that is the norm. For the last ten years I've brought my own bags, packed everything like I want it as I grab it - with a quick blip from the scanner - and then checkout is no line, no cashier, no unpacking and packing things again. Just scanning my card and going through a brief screen to check that the summary looks right. Instead of a guard there's a scanner by the exit gate where you scan your ticket. In my case just a small stub since the actual ticket is sent digitally. I think it works so smoothly, much better than before this system was introduced. My recent experience buying groceries in Belgium was very similar.
- LtWorf 1y ago
  Here in sweden it is not "the norm" by any definition of "norm". Also it's incredibly slower and not "one blip away". And the scanner to exit is terrible for fire safety and accessibility. Also I've seen it break with the result that you had to trigger the alarm to leave. It does not work smoothly at all, but perhaps you live in a different sweden than I.
  - munksbeer 1y ago
    It works pretty well in the UK. The nice thing is, I can pack my bags as I shop, so I don't feel stressed when I get to the checkout. I use this method of shopping whenever available.
    - LtWorf 1y ago
      In sweden you have to unpack and re-pack at the checkout. And if you have a backpack you can't place stuff inside it. No. You have to unpack, place everything on a scale and then place everything in your backpack again. And for items like bread or so you need to navigate a number of menus to find it. And of course you are asked if you are a member, if you'd like to become a member, if you want to do a donation, and a number of other useless questions.
      - deleted 1y ago
        [deleted]
Workaccount2 1y ago
>That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough. That's exactly why she asks an LLM to do it for her. A program like this would be <1k LOC, well within even the current LLM one-shot-working-app domain.

theptip 1y ago

Right. Jevons Paradox. The lower barrier to entry might mean the average quality of software goes WAY down. But this could be great if it means that there are 10x as many little apps that make one person happy by doing only as much as they need. The above is also consistent with the quantity of excellent software going way up as well (just slower than the below-average stuff).

LtWorf 1y ago
What the android/apple appstores have taught me compared to the old symbian appstore is that we don't need more apps, we need decent ones. The current appstores have more.
- graemep 1y ago
  One of the advantages of FDroid is that although tiny compared to Play Store, it is much better curated.
- theptip 1y ago
  Discovery definitely becomes more important in this world. Fortunately AI helps with that too.
  - LtWorf 1y ago
    Why do you think it will not soon be biased as search is?
    - theptip 1y ago
      I think AI gives us the opportunity to write our own ranking algorithms. Of course most people will pick an off-the-shelf editorial perspective but I expect these to be much more configurable than the current state with a tiny number of algorithmic gatekeepers.
      - LtWorf 1y ago

disambiguation 1y ago

>. They fall on their face trying to work in sprawling codebases. >However, they are excellent at writing small bespoke programs For programmers I predict a future of a million micro services. Sprawling has always been an undesirable and unnecessary byproduct of growing code bases, but there's been no easy solution to that. I wonder if LLMs would perform better on a mono repo of many small services than one sprawling monolith.

api 1y ago
Micro service systems are just huge sprawling code bases with more glue code. Calling something over a network instead of via a local function call is still calling something.
- libraryatnight 1y ago
  For years one of my favorite experiences (amusing given your username) was being on calls for incidents where they get the dev on for X thing and an exec goes "I thought we got rid of that" and a bunch of people sheepishly explain it wasn't really retired... it was repurposed as an API. I especially loved it when the "retirement" of the broken thing was the execs big achievement. (The comedic nuance often being the thing could have been retired for real, but they demanded a timeline that necessitated the "fake" retirement) It still happens but it's not a favorite experience anymore. It's just a source of loathing for MBA culture.
  - api 1y ago
    The pure play MBA is the capitalist West's equivalent of the Soviet apparatchik: https://en.wikipedia.org/wiki/Apparatchik https://en.wikipedia.org/wiki/Apparatchik Been saying that for years. Private equity is perhaps analogous to the Politburo.
- disambiguation 1y ago
  Another way to look at it is - why force the agent to grapple with the whole code base when they can rapidly standup many single purpose services instead?

tsimionescu 1y ago

Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully? Programming ecosystems are generally very unfriendly to this type of use case, with plenty of setup required to get something usable out even if you have the code available. Especially if you'd like to run the results on your phone. Sure, an LLM might be able to guide you through the steps, and even help when you stumble, but you still have to follow a hundred little steps exactly with no intuition whether things will work out at the end. I very much doubt that this is something many people will even think to ask for, and then follow-through with. Especially since the code quality of LLMs is nowhere near what you make it out to be. You'll still need to bear with them and go through a lot of trial and error to get anything running, especially if you have no knowledge of the terms of art, nor any clue of what might be going wrong if it goes wrong. If you've ever seen consumer bug reports, you might have some idea of the quality of feedback the LLM may be getting back if something is not working perfectly the first time. It's very likely to be closer to "the button you added is not working" than to "when I click the button, the application crashes" or "when I click the button, the application freezes for a few seconds and then pops up an error saying that input validation failed" or "the button is not showing up on the screen" [...]

azan_ 1y ago
> Do you have any proof whatsoever that non-programmers are using LLMs to write small bespoke apps for them successfully? I’m radiologist, I’ve been paying for software that sped up my reporting like 200 usd per month. I’ve remade all the functionality I need in one evening with cursor and added some things that I’ve found missing from the original software.
Workaccount2 1y ago
My company is a non-tech company that is in manufacturing. I'm not a programmer and niether is anyone else here. So far I have created 7 programs that are now used daily in production. One of them replaces a $1k/yr/usr CAD package, and another we used to bring in a contractor to write. The others a miscellaneous apps for automating/simplifying our in houses processes. None of the programs are more than 6k LOC.

ben_w 1y ago

> This is a huge blind-spot I keep seeing people miss. LLM "programmers" are way way more useful to regular people than they are to career programmers. Kinda. They're good, and I like them, but I think of them like a power tool: just because you can buy an angle grinder or a plasma cutter from a discount supermarket, doesn't mean the tool is safe in the hands of an untrained amateur. Someone using it to help fix up a spreadsheet? Probably fine. But you should at least be able to read the code to the level you don't get this deliberate bad example to illustrate the point: #!/usr/bin/python3 total_sales = 0.0 def add_sale(total_sales, amount): total_sales = total_sales + amount print("total_sales: " + str(total_sales)) Still useful, still sufficiently advanced technology that for most people it is (Clarketech) magic, but also still has some rough edges.