4 ms·

This kind of mirror my experience with LLMs. If I ask them non-original problems (make this API, write this test, update this function (that must be written 100

by 101008 2y ago

This kind of mirror my experience with LLMs. If I ask them non-original problems (make this API, write this test, update this function (that must be written 100s of time by develoeprs around the world, etc), it works very well. Some minors changes here and there but it saves time. When I ask them to code things that they never heard of (I am working on a online sport game), it fails catastrophically. The LLM should know the sport, and what I ask is pretty clear for anyone who understand the game (I tested against actual people and it was obvious what to expect), but the LLM failed miserably. Even worse when I ask them to write some designs in CSS for the game. It seems if you take them outside the 3-columsn layout or bootstrap or the overused landing page, LLMs fails miserably. It works very well for the known cases, but as soon as you want them to do something original, they just can't.

jppope 2y ago
completely agree with this. The problem I'm starting to run into is that I do a lot of things that have been done before so I get really efficient at doing that stuff with LLMs... and now I'm slow to solve the stuff I used to be really fast at.
anonzzzies 2y ago
I ask llms things like I would spec it out for my team, which is to say, when I write a task, I would not include things about the sport, I would explain the logic etc one would need to write. No one needs to know what it is about in abstract as I see the same issue with humans; they get confused when using real world things to see the connection with what they need to do. I mean I don't flesh it out to the extend I might as well write the code, but enough that you don't need to know anything about the underlying subject. We are doing a large EU healthcare project and things differ per country obviously; if I would assume people to have a modicum of world knowledge or interest in it and look it up, nothing would get done. It is easier to deliver excel sheets with the proper wording and formulas and leave out what it is even for. Works fine with LLMs. Disclaimer; the people in my company know everything about the subject matter; the people (and LLMs) that implement it are usually not ours: we just manage them and in my experience, programmers at big companies are borderline worthless, hence, in the past 35 years, I have taken to write tasks as concrete as I can; we found that otherwise we either get garbage commits or tons of meetings with questions and then garbage commits. And now that comes in handy as this works much better on LLMs too.
- sagarm 2y ago
  Breaking a problem down to the extent you are describing should only have to be done for fresh grads IMO. LLMs are indeed better at tasks broken down to this degree in my experience. I'd expect more senior engineers to handle increasingly complex/ambiguous projects, while taking the business context into account as necessary.
steve_adams_86 2y ago
This has made me wonder just how few developers have been doing even remotely interesting things. I encounter a lot of situations where the LLMs catastrophically fail, but I don’t think it’s for lack of trying to guide it through to solutions properly or sufficiently. I give as much or as little context as I can think to, give partial solutions or zero solutions, try different LLMs with the same prompts, etc. Maybe we were mostly actually doing the exact same stuff and these things are trained on a whole lot of the same. Like with react components, these things are amazing at pumping out the most basic varieties of them.
- uludag 2y ago
  My thoughts exactly. I honestly think that the majority of SWE work is completely unoriginal. But then there are those idiosyncrasies that every company has, a finicky build system, strange hacks to get certain things to work, extensive webs of tribal knowledge, that will be the demise for any LLM SWE application besides being a tool for skilled professionals.
  - bamboozled 2y ago
    The whole world is wrinkly, that's the way it is, everyone, and everything is slightly different.
- bugglebeetle 2y ago
  > This has made me wonder just how few developers have been doing even remotely interesting things. There’s not much to ponder on here as the majority of human thought and action is unoriginal and irrelevant, so software development is not some special exception to this. This doesn’t mean it lacks meaning or value for those doing it, since that can be self-generated, extrinsically motivated, or both. But, by definition, most things done are within the boundaries of a normal distribution and not exceptional, transcendent, or sublime. You can strive for those latter categories, but it’s a path strewn with much failure, misery, and quite questionable rewards, even where it is achieved. Most prefer more achievable comforts or security to it, or, at most, dull-minded imitations thereof, such as amassing great wealth.
- 2y ago
PaulRobinson 2y ago
> as soon as you want them to do something original, they just can't. An LLM is, famously, a "stochastic parrot". It has no reasoning. It has no creativity. It has no basis or mechanism for true originality: it is regurgitating what it has seen before, based on probabilities, nothing more. It looks more impressive, but that's all is behind the curtain. It surprises me that so many people expect an LLM to do something "clever" or "original". They're compression machines for knowledge, a tool for summarisation, rephrasing, not for creation. Why is this not that widely known and accepted across the industry yet, and why do so many people have such high expectations?
- redlock 2y ago
  Because it isn't true according to Hinton, Sutskever, and other leading AI researchers.
  - FroshKiller 2y ago
    Most foxes are herbivores according to the fellow guarding my henhouse.
    - redlock 2y ago
      They do make compelling arguments if you listened to them. Also, from my coding with llms using cursor they obviously understand the code and my request more often than not. Mechanistic interpretability research shows evidence of concepts being represented within the layers. Golden gate claude is evidence of this: https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude To me this proves that llms learn concepts and multilayeres representations within its network and not just some dumb statistical inference. Even famous llm skeptic like Francois Chollet doesn't invoke this stochastic parrots anymore and has moved on to arguing that they don't generalize well and are just memorizing. With GPT-2 and 3 I was of the same opinion that they seemed like just sophisticated stochastic parrots, but current llms are a different class from early gpts. Now that o3 has beaten memory resisting benchmark like ARC-AGI I think we can confidently move on from this stochastic parrots notion. (And before you argue that o3 is not an llm, here is an openai researcher stating that it is an llm https://x.com/__nmca__/status/1870170101091008860?s=46&t=eTe6EMTslOxUfPDx9s6IIQ https://x.com/__nmca__/status/1870170101091008860?s=46&t=eTe... )
deleted 2y ago
[deleted]

by 101008 2y ago

jppope 2y ago

completely agree with this. The problem I'm starting to run into is that I do a lot of things that have been done before so I get really efficient at doing that stuff with LLMs... and now I'm slow to solve the stuff I used to be really fast at.

anonzzzies 2y ago

I ask llms things like I would spec it out for my team, which is to say, when I write a task, I would not include things about the sport, I would explain the logic etc one would need to write. No one needs to know what it is about in abstract as I see the same issue with humans; they get confused when using real world things to see the connection with what they need to do. I mean I don't flesh it out to the extend I might as well write the code, but enough that you don't need to know anything about the underlying subject. We are doing a large EU healthcare project and things differ per country obviously; if I would assume people to have a modicum of world knowledge or interest in it and look it up, nothing would get done. It is easier to deliver excel sheets with the proper wording and formulas and leave out what it is even for. Works fine with LLMs. Disclaimer; the people in my company know everything about the subject matter; the people (and LLMs) that implement it are usually not ours: we just manage them and in my experience, programmers at big companies are borderline worthless, hence, in the past 35 years, I have taken to write tasks as concrete as I can; we found that otherwise we either get garbage commits or tons of meetings with questions and then garbage commits. And now that comes in handy as this works much better on LLMs too.

sagarm 2y ago
Breaking a problem down to the extent you are describing should only have to be done for fresh grads IMO. LLMs are indeed better at tasks broken down to this degree in my experience. I'd expect more senior engineers to handle increasingly complex/ambiguous projects, while taking the business context into account as necessary.

steve_adams_86 2y ago

This has made me wonder just how few developers have been doing even remotely interesting things. I encounter a lot of situations where the LLMs catastrophically fail, but I don’t think it’s for lack of trying to guide it through to solutions properly or sufficiently. I give as much or as little context as I can think to, give partial solutions or zero solutions, try different LLMs with the same prompts, etc. Maybe we were mostly actually doing the exact same stuff and these things are trained on a whole lot of the same. Like with react components, these things are amazing at pumping out the most basic varieties of them.

uludag 2y ago
My thoughts exactly. I honestly think that the majority of SWE work is completely unoriginal. But then there are those idiosyncrasies that every company has, a finicky build system, strange hacks to get certain things to work, extensive webs of tribal knowledge, that will be the demise for any LLM SWE application besides being a tool for skilled professionals.
- bamboozled 2y ago
  The whole world is wrinkly, that's the way it is, everyone, and everything is slightly different.
bugglebeetle 2y ago
> This has made me wonder just how few developers have been doing even remotely interesting things. There’s not much to ponder on here as the majority of human thought and action is unoriginal and irrelevant, so software development is not some special exception to this. This doesn’t mean it lacks meaning or value for those doing it, since that can be self-generated, extrinsically motivated, or both. But, by definition, most things done are within the boundaries of a normal distribution and not exceptional, transcendent, or sublime. You can strive for those latter categories, but it’s a path strewn with much failure, misery, and quite questionable rewards, even where it is achieved. Most prefer more achievable comforts or security to it, or, at most, dull-minded imitations thereof, such as amassing great wealth.
2y ago

PaulRobinson 2y ago

> as soon as you want them to do something original, they just can't. An LLM is, famously, a "stochastic parrot". It has no reasoning. It has no creativity. It has no basis or mechanism for true originality: it is regurgitating what it has seen before, based on probabilities, nothing more. It looks more impressive, but that's all is behind the curtain. It surprises me that so many people expect an LLM to do something "clever" or "original". They're compression machines for knowledge, a tool for summarisation, rephrasing, not for creation. Why is this not that widely known and accepted across the industry yet, and why do so many people have such high expectations?

redlock 2y ago
Because it isn't true according to Hinton, Sutskever, and other leading AI researchers.
- FroshKiller 2y ago
  Most foxes are herbivores according to the fellow guarding my henhouse.
  - redlock 2y ago
    They do make compelling arguments if you listened to them. Also, from my coding with llms using cursor they obviously understand the code and my request more often than not. Mechanistic interpretability research shows evidence of concepts being represented within the layers. Golden gate claude is evidence of this: https://www.anthropic.com/news/golden-gate-claude https://www.anthropic.com/news/golden-gate-claude To me this proves that llms learn concepts and multilayeres representations within its network and not just some dumb statistical inference. Even famous llm skeptic like Francois Chollet doesn't invoke this stochastic parrots anymore and has moved on to arguing that they don't generalize well and are just memorizing. With GPT-2 and 3 I was of the same opinion that they seemed like just sophisticated stochastic parrots, but current llms are a different class from early gpts. Now that o3 has beaten memory resisting benchmark like ARC-AGI I think we can confidently move on from this stochastic parrots notion. (And before you argue that o3 is not an llm, here is an openai researcher stating that it is an llm https://x.com/__nmca__/status/1870170101091008860?s=46&t=eTe6EMTslOxUfPDx9s6IIQ https://x.com/__nmca__/status/1870170101091008860?s=46&t=eTe... )

deleted 2y ago

[deleted]