5 ms·

2 lessons to learn from this blog: > these LLMs won’t replace software engineers anytime soon, because it requires a strong engineering background to recognize

2 lessons to learn from this blog: > these LLMs won’t replace software engineers anytime soon, because it requires a strong engineering background to recognize what is actually a good idea, along with other constraints that are domain specific. > One issue with my experiments is that I’m benchmarking code improvement using Python, which isn’t the coding language developers consider when hyperoptimizing performance.

lostmsu 2y ago
TBH I'm not sure how he arrived at "won’t replace software engineers anytime soon" The LLM solved his task. With his "improved prompt" the code is good. The LLM in his setup was not given a chance to actually debug its code. It only took him 5 "improve this code" commands to get to the final optimized result, which means the whole thing was solved (LLM execution time) in under 1 minute.
- shahzaibmushtaq 2y ago
  Did you read the two paragraphs written above and the one where he made that statement? My comments on "what you are not sure" is that Max is a software engineer (I am sure a good one) and he kept iterating the code until it reached close to 100x faster code because he knew what "write better code" looked like. Now ask yourself this question: Is there any chance a no-code/low-code developer will come to a conclusion deduced by Max (he is not the only one) that you are not sure about? An experienced software engineer/developer is capable of improving LLM written code into better code with the help of LLM.
  - minimaxir 2y ago
    > Max is a software engineer (I am sure a good one) Opinions are mixed.
    - lostmsu 2y ago
      I'd hear one where somebody thought you failed at software engineering.
    - shahzaibmushtaq 2y ago
      Then stop stirring the opinion pot, or should I say Max is a software developer and that means you.
  - lostmsu 2y ago
    I mean from this sample of 2 you could just ask it repeatedly for up to 5 times restarting the counter whenever performance improves.
- simonw 2y ago
  A non-engineer would not be able to interpret ANY of what he did here, or fix any of the bugs.
  - lostmsu 2y ago
    A non-engineer by definition would not be able to fix bugs. But why does it matter that they won't be able to interpret anything? Just like with real engineers you can ask AI to provide an explanation digestible by an eloi.
    - simonw 2y ago
      By the time a non-engineer has waded through all of those necessary explanations they'll deserve to call themselves an engineer.
      - lostmsu 2y ago
        You did not really answer the question. The part of the comment you replied to was there to illustrate the need for the question in the first place.
    - shahzaibmushtaq 2y ago
      A patient cannot become a doctor by asking AI to explain and prescribe treatment based on symptoms. That's what Simon is trying to say.
      - lostmsu 2y ago
        That statement is not being discussed as it is obvious. The question is "can AI be a developer", not "am I a developer if I use an AI who is a developer".
        shahzaibmushtaq 2y ago
        The answer to your question is "can AI be a doctor".

lostmsu 2y ago

TBH I'm not sure how he arrived at "won’t replace software engineers anytime soon" The LLM solved his task. With his "improved prompt" the code is good. The LLM in his setup was not given a chance to actually debug its code. It only took him 5 "improve this code" commands to get to the final optimized result, which means the whole thing was solved (LLM execution time) in under 1 minute.

shahzaibmushtaq 2y ago
Did you read the two paragraphs written above and the one where he made that statement? My comments on "what you are not sure" is that Max is a software engineer (I am sure a good one) and he kept iterating the code until it reached close to 100x faster code because he knew what "write better code" looked like. Now ask yourself this question: Is there any chance a no-code/low-code developer will come to a conclusion deduced by Max (he is not the only one) that you are not sure about? An experienced software engineer/developer is capable of improving LLM written code into better code with the help of LLM.
- minimaxir 2y ago
  > Max is a software engineer (I am sure a good one) Opinions are mixed.
  - lostmsu 2y ago
    I'd hear one where somebody thought you failed at software engineering.
  - shahzaibmushtaq 2y ago
    Then stop stirring the opinion pot, or should I say Max is a software developer and that means you.
- lostmsu 2y ago
  I mean from this sample of 2 you could just ask it repeatedly for up to 5 times restarting the counter whenever performance improves.
simonw 2y ago
A non-engineer would not be able to interpret ANY of what he did here, or fix any of the bugs.
- lostmsu 2y ago
  A non-engineer by definition would not be able to fix bugs. But why does it matter that they won't be able to interpret anything? Just like with real engineers you can ask AI to provide an explanation digestible by an eloi.
  - simonw 2y ago
    By the time a non-engineer has waded through all of those necessary explanations they'll deserve to call themselves an engineer.
    - lostmsu 2y ago
      You did not really answer the question. The part of the comment you replied to was there to illustrate the need for the question in the first place.
  - shahzaibmushtaq 2y ago
    A patient cannot become a doctor by asking AI to explain and prescribe treatment based on symptoms. That's what Simon is trying to say.
    - lostmsu 2y ago
      That statement is not being discussed as it is obvious. The question is "can AI be a developer", not "am I a developer if I use an AI who is a developer".
      - shahzaibmushtaq 2y ago
        The answer to your question is "can AI be a doctor".