4 ms·

Peaked? Nah, it's barely started. Wait till we get decent SWE agents reliably writing good code, probably later this year or next. Once AI moves beyond simple b

Peaked? Nah, it's barely started. Wait till we get decent SWE agents reliably writing good code, probably later this year or next. Once AI moves beyond simple boilerplate, the productivity boost will be huge. Too soon to call hype when we've barely scratched the surface.

bluefirebrand 2y ago
I asked copilot to write me a Typescript function today I had two defined types, both with the exact same field names. The only difference is one has field names written in snake_case, and the other had names written in camelCase. Otherwise the exact same I wanted a function that would take an object of the snake_case type, and output an object of the camelCase type. The object only had about 10 fields It missed about half of the fields, and inserted fields that didn't even exist on either object You cannot convince me that AI is anywhere near to this level if it cannot even generate a function that can convert "is_enabled" to "isEnabled" inside an object Every time I try this stuff I'm so disappointed with it. It makes me think anyone who is hyped about it is an absolute fraud that does not know at all what they are doing
- leosanchez 2y ago
  Typescript is the language it probably has most data on...
- knowaveragejoe 2y ago
  You get out what you put in. Of course if you provide one sentence of context(and some implicit context from the environment) you aren't going to get magical result.
  - bluefirebrand 2y ago
    The test was "can I get it to generate this while spending less effort than it would take me to just write it" and it failed miserably. And this was a super low effort, small boilerplate problem to solve. This is the sort of problem it has to solve to be remotely useful If it cannot do that, then why is anyone saying it is a productivity booster?
    - knowaveragejoe 2y ago
      My response could only possibly be that I haven't had that issue. I've asked for relatively complex changes to codebases(mainly python), and had very little in the way of trouble.
      - imiric 2y ago
        ... That you're aware of. The more code you ask it to generate, the higher the chances that it will introduce an issue. Even if the code compiles, subtle bugs can easily creep in. Not unlike a human programmer, you might say, but a human programmer wouldn't hallucinate APIs. LLMs make entirely unique types of errors, and do so confidently. You really need to carefully review every single line, which sometimes takes more effort than just writing it yourself. I would be particularly wary of generated code for a dynamic language like Python.
- parliament32 2y ago
  This mirrors my experience, and I've tried nearly every popular model available, numerous times over the last year or so. I'm having trouble understanding where all the hype is coming from -- is it literally just all marketing? I refuse to believe any half-decent SWE can be bullish on AI-for-code after novelty of the first week wears off.

bluefirebrand 2y ago

I asked copilot to write me a Typescript function today I had two defined types, both with the exact same field names. The only difference is one has field names written in snake_case, and the other had names written in camelCase. Otherwise the exact same I wanted a function that would take an object of the snake_case type, and output an object of the camelCase type. The object only had about 10 fields It missed about half of the fields, and inserted fields that didn't even exist on either object You cannot convince me that AI is anywhere near to this level if it cannot even generate a function that can convert "is_enabled" to "isEnabled" inside an object Every time I try this stuff I'm so disappointed with it. It makes me think anyone who is hyped about it is an absolute fraud that does not know at all what they are doing

leosanchez 2y ago
Typescript is the language it probably has most data on...
knowaveragejoe 2y ago
You get out what you put in. Of course if you provide one sentence of context(and some implicit context from the environment) you aren't going to get magical result.
- bluefirebrand 2y ago
  The test was "can I get it to generate this while spending less effort than it would take me to just write it" and it failed miserably. And this was a super low effort, small boilerplate problem to solve. This is the sort of problem it has to solve to be remotely useful If it cannot do that, then why is anyone saying it is a productivity booster?
  - knowaveragejoe 2y ago
    My response could only possibly be that I haven't had that issue. I've asked for relatively complex changes to codebases(mainly python), and had very little in the way of trouble.
    - imiric 2y ago
      ... That you're aware of. The more code you ask it to generate, the higher the chances that it will introduce an issue. Even if the code compiles, subtle bugs can easily creep in. Not unlike a human programmer, you might say, but a human programmer wouldn't hallucinate APIs. LLMs make entirely unique types of errors, and do so confidently. You really need to carefully review every single line, which sometimes takes more effort than just writing it yourself. I would be particularly wary of generated code for a dynamic language like Python.
parliament32 2y ago
This mirrors my experience, and I've tried nearly every popular model available, numerous times over the last year or so. I'm having trouble understanding where all the hype is coming from -- is it literally just all marketing? I refuse to believe any half-decent SWE can be bullish on AI-for-code after novelty of the first week wears off.