4 ms·

Randomly, my advice: don't sleep on this. Three or four weeks ago I was posting how LLMs were useful for one-off questions but I wouldn't trust them on my code

Randomly, my advice: don't sleep on this. Three or four weeks ago I was posting how LLMs were useful for one-off questions but I wouldn't trust them on my codebase. Then I spent my week's holiday messing around on them for some personal projects. I am now a fairly committed Roo user. There are lots of problems, but there is incredible value here. Try it and see if you're still a hold-out.

rationalfaith 1y ago
[dead]
phito 1y ago
I will definitely sleep on agents. Normal LLM use, fine, but I am not giving up reasoning.
- bananapub 1y ago
  this is kind of a weird position to take. you're the captain, you're the person reviewing the code the LLM (agent or not) generates, you're the one asking for the code you want, you're in charge of deciding how much effort to put in to things, and especially which things are most worth your effort. all this agent stuff sounded stupid to me until I tried it out in the last few weeks, and personally, it's been great - I give a not-that-detailed explanation for what I want, point it at the existing code and get back a patch to review once I'm done making my coffee. sometimes it's fine to just apply, sometimes I don't like a variable name or whatever, sometimes it doesn't fit in with the other stuff so I get it to try again, sometimes (<< 10% of the time) it's crap. the experience is pretty much like being a senior dev with a bunch of very eager juniors who read very fast. anyway, obviously do whatever you want, but deriding something you've not looked in to isn't a hugely thoughtful process for adapting to a changing world.
  - phito 1y ago
    If I have to review all code code it's writing, I'd rather write it myself (maybe with the help of an LLM). > anyway, obviously do whatever you want, but deriding something you've not looked in to isn't a hugely thoughtful process for adapting to a changing world. I have tried it. Not sure I want to be part of such world, unfortunately. > the experience is pretty much like being a senior dev with a bunch of very eager juniors who read very fast. I... don't want that. Juniors just slow me down because I have to check what they did and fix their mistakes. (this is in the context of professional software development, not making scripts, tinkering etc)
    - BeetleB 1y ago
      > I... don't want that. Juniors just slow me down because I have to check what they did and fix their mistakes. > (this is in the context of professional software development, not making scripts, tinkering etc) I understand the sentiment. A few months ago they wanted us to move fast and dumped us (originally 2 developers) with 4 new people who have very little real world coding experience. Not fun, and very stressful. However, keep in mind that in many workplaces, handling junior devs poorly means one of two things: 1. If you have some abstruse domain expertise, and it's OK that only 1-2 people work on it, you'll be relegated to doing that. Sadly, most workplaces don't have such tasks. 2. You'll be fantastic in your output. Your managers will like you. But they will not promote you. After some point, they expect you to be a leverage multiplier - if you can get others to code really well, the overall team productivity will exceed that of any superstar (and no, I don't believe 10x programmers exist in most workplaces).
- simonw 1y ago
  What's your definition of "agents" there?
- BeetleB 1y ago
  > Normal LLM use, fine, but I am not giving up reasoning. Ouch! Reminds me of: - I'm never going to use cell phones. I care about voice quality (me decades ago) - I'm never going to use VoIP. I care about voice quality (everyone but me 2 decades ago). - I'm never going to use a calculator. I am not going to give up on reasoning. - I'm never going to let my kids play with <random other ethnicity>. I care about good manners. https://en.wikipedia.org/wiki/False_dilemma https://en.wikipedia.org/wiki/False_dilemma
  - phito 1y ago
    Sure, keep slowly offloading more and more of your brain to technology. Until you won't be needed anymore.
    - BeetleB 1y ago
      If that is the ultimate outcome, I assure you that your insistence on not using not only is not going to protect you - you'll be the first to be unemployed. I don't really get the sentiment. If you're in engineering/computers, you are fundamentally in the business of replacing people with technology. Cloud solutions resulted in people in IT/dev ops losing their jobs. So much SW out there has displaced real people in the last 5 decades. Do you want to go back to the days of dealing with switchboard operators to make phone calls?
      - hansmayer 1y ago
        > Cloud solutions resulted in people in IT/dev ops losing their jobs. This happened only in the phantasy world of deluded MBAs who were sold on the narrative of the cloud operators taking over their infrastructure management. If you ever worked on any of the hyperscaler-based infrastructure, it is actually more complex than the more simple on-premise setups people used to have. I am not saying it is bad that way, I actually enjoy working with the cloud more than with on-premise stuff, but just to highlight you are just making such a bad comparison here.
  - blibble 1y ago
    I'm never going to use the metaverse. I'm never going to use use Blockchain. I'm never going to use use NFTs.
vultour 1y ago
I spent a good part of yesterday attempting to use ChatGPT to help me choose an appropriate API gateway. Over and over it suggested things that literally do not exist, and the only reason I could tell was that I spent a good amount of time in the actual documentation. This has been my experience roughly 80% of the time when trying to use an LLM. I would like to know what is the magical prompt engineering technique that makes it stop confidently hallucinating about literally everything.
- yunwal 1y ago
  Did you try giving it the docs to read?
- petesergeant 1y ago
  Sure, this was exactly how I felt three weeks ago, and I could have written that comment myself. The agentic approach where it works out it made something up by looking at the errors the type-check generates is what makes the difference.
- spacechild1 1y ago
  I'm having a very good experience with ChatGPT at the moment. I'm mostly using it for little tasks where I don't remember the exact library functions. Examples: "C++ question: how do I get the unqualified local system time and turn into an ISO time string?" "Python question: how do I serialize a C struct over a TCP socket with asyncio?" "JS question: how do I dynamically show/hide an HTML element?" (I obviously don't write a lot of JS :-D) ChatGPT gave me the correct answers on the first try. I have been a sceptic, but I'm now totally sold on AI assisted coding, at least as a replacement for Google and StackOverflow. For me there is no point anymore in wading through all the blog spam and SEO crap just to find a piece of information. Stack Overflow is still occasionally useful, but the writing is on the wall... EDIT: Important caveat: stay critical! I have been playing around asking ChatGPT more complex questions where I actually know the correct answer resp. where I can immediately spot mistakes. It sometimes gives me answers that would look correct to a non-expert, but are hilariously wrong.
  - deleted 1y ago

rationalfaith 1y ago

[dead]

phito 1y ago

I will definitely sleep on agents. Normal LLM use, fine, but I am not giving up reasoning.

bananapub 1y ago
this is kind of a weird position to take. you're the captain, you're the person reviewing the code the LLM (agent or not) generates, you're the one asking for the code you want, you're in charge of deciding how much effort to put in to things, and especially which things are most worth your effort. all this agent stuff sounded stupid to me until I tried it out in the last few weeks, and personally, it's been great - I give a not-that-detailed explanation for what I want, point it at the existing code and get back a patch to review once I'm done making my coffee. sometimes it's fine to just apply, sometimes I don't like a variable name or whatever, sometimes it doesn't fit in with the other stuff so I get it to try again, sometimes (<< 10% of the time) it's crap. the experience is pretty much like being a senior dev with a bunch of very eager juniors who read very fast. anyway, obviously do whatever you want, but deriding something you've not looked in to isn't a hugely thoughtful process for adapting to a changing world.
- phito 1y ago
  If I have to review all code code it's writing, I'd rather write it myself (maybe with the help of an LLM). > anyway, obviously do whatever you want, but deriding something you've not looked in to isn't a hugely thoughtful process for adapting to a changing world. I have tried it. Not sure I want to be part of such world, unfortunately. > the experience is pretty much like being a senior dev with a bunch of very eager juniors who read very fast. I... don't want that. Juniors just slow me down because I have to check what they did and fix their mistakes. (this is in the context of professional software development, not making scripts, tinkering etc)
  - BeetleB 1y ago
    > I... don't want that. Juniors just slow me down because I have to check what they did and fix their mistakes. > (this is in the context of professional software development, not making scripts, tinkering etc) I understand the sentiment. A few months ago they wanted us to move fast and dumped us (originally 2 developers) with 4 new people who have very little real world coding experience. Not fun, and very stressful. However, keep in mind that in many workplaces, handling junior devs poorly means one of two things: 1. If you have some abstruse domain expertise, and it's OK that only 1-2 people work on it, you'll be relegated to doing that. Sadly, most workplaces don't have such tasks. 2. You'll be fantastic in your output. Your managers will like you. But they will not promote you. After some point, they expect you to be a leverage multiplier - if you can get others to code really well, the overall team productivity will exceed that of any superstar (and no, I don't believe 10x programmers exist in most workplaces).
simonw 1y ago
What's your definition of "agents" there?
BeetleB 1y ago
> Normal LLM use, fine, but I am not giving up reasoning. Ouch! Reminds me of: - I'm never going to use cell phones. I care about voice quality (me decades ago) - I'm never going to use VoIP. I care about voice quality (everyone but me 2 decades ago). - I'm never going to use a calculator. I am not going to give up on reasoning. - I'm never going to let my kids play with <random other ethnicity>. I care about good manners. https://en.wikipedia.org/wiki/False_dilemma https://en.wikipedia.org/wiki/False_dilemma
- phito 1y ago
  Sure, keep slowly offloading more and more of your brain to technology. Until you won't be needed anymore.
  - BeetleB 1y ago
    If that is the ultimate outcome, I assure you that your insistence on not using not only is not going to protect you - you'll be the first to be unemployed. I don't really get the sentiment. If you're in engineering/computers, you are fundamentally in the business of replacing people with technology. Cloud solutions resulted in people in IT/dev ops losing their jobs. So much SW out there has displaced real people in the last 5 decades. Do you want to go back to the days of dealing with switchboard operators to make phone calls?
    - hansmayer 1y ago
      > Cloud solutions resulted in people in IT/dev ops losing their jobs. This happened only in the phantasy world of deluded MBAs who were sold on the narrative of the cloud operators taking over their infrastructure management. If you ever worked on any of the hyperscaler-based infrastructure, it is actually more complex than the more simple on-premise setups people used to have. I am not saying it is bad that way, I actually enjoy working with the cloud more than with on-premise stuff, but just to highlight you are just making such a bad comparison here.
- blibble 1y ago
  I'm never going to use the metaverse. I'm never going to use use Blockchain. I'm never going to use use NFTs.

vultour 1y ago

I spent a good part of yesterday attempting to use ChatGPT to help me choose an appropriate API gateway. Over and over it suggested things that literally do not exist, and the only reason I could tell was that I spent a good amount of time in the actual documentation. This has been my experience roughly 80% of the time when trying to use an LLM. I would like to know what is the magical prompt engineering technique that makes it stop confidently hallucinating about literally everything.

yunwal 1y ago
Did you try giving it the docs to read?
petesergeant 1y ago
Sure, this was exactly how I felt three weeks ago, and I could have written that comment myself. The agentic approach where it works out it made something up by looking at the errors the type-check generates is what makes the difference.
spacechild1 1y ago
I'm having a very good experience with ChatGPT at the moment. I'm mostly using it for little tasks where I don't remember the exact library functions. Examples: "C++ question: how do I get the unqualified local system time and turn into an ISO time string?" "Python question: how do I serialize a C struct over a TCP socket with asyncio?" "JS question: how do I dynamically show/hide an HTML element?" (I obviously don't write a lot of JS :-D) ChatGPT gave me the correct answers on the first try. I have been a sceptic, but I'm now totally sold on AI assisted coding, at least as a replacement for Google and StackOverflow. For me there is no point anymore in wading through all the blog spam and SEO crap just to find a piece of information. Stack Overflow is still occasionally useful, but the writing is on the wall... EDIT: Important caveat: stay critical! I have been playing around asking ChatGPT more complex questions where I actually know the correct answer resp. where I can immediately spot mistakes. It sometimes gives me answers that would look correct to a non-expert, but are hilariously wrong.
- deleted 1y ago