5 ms·

I think it's already surpassed 10x, and is closer to 25x or 30x. You just have to know what to ask it to do.

by resters 2y ago

vundercind 2y ago
I should probably get around to actually using one of these things.
- gadflyinyoureye 2y ago
  I’d love to see people going through a real world walk through using such things. Some people who do cookie cutter common tasks find that an LLM can produce their whole code base. But ask it to make a change and what happens?
  - abdullin 2y ago
    This is an example of my prompt to o1 a few days ago. First request produced refactoring suggestions. Second request (yes, I like results, implement them) produced multiple files that have just worked. —- Take a look at this code from my multi-mode (a la vim or old terminal apps) block-based content editor. I want to build on the keyboard interface and introduce a simple way to have simple commands with small popup. E.g. after doing "A" in "view" mode, show user a popup that expects H,T,I, or V. Or, after pressing "P" in view mode - show a small popup that has an text input waiting for the permission role for the page. Don't implement the changes, just think through how to extend existing code to make logic like that simple. Remember, I like simple code, I don't like spaghetti code and many small classes/files.
  - falcolas 2y ago
    IME: I will either get a repeat of something already written (this happens all the time with java getter/setter boilerplate if the class isn't the one I'm currently working in), or I get some junior level attempt at what I ask it to do. The second case is the best case, and it still requires a code review with a fine toothed comb. For example, it loves to suggest plausible looking but non-existant methods on popular libraries. It's one of those things where a language aware autocomplete will often save me more time, because it doesn't try and second guess me.
    - resters 2y ago
      sometimes one must give it current docs to avoid that. Ironically gpt-4 always tries to use the old chat completions api when writing code to consume the open-ai api.
  - JHonaker 2y ago
    I've been looking for such a walkthrough for months. I can't find anyone actually showing them working and building something. There are plenty of, "look at this code this thing spit out that successfully builds," but I've not been able to find anything I would consider truly showing someone building something novel. If anyone has a YT video or a detailed article, please show me!
    - resters 2y ago
      give me an example of the kind of thing you are curious about how to build in this way, and I will give you a step by step.
mewpmewp2 2y ago
It's very circumstance specific. In my side projects, where I work alone and have the perfect setup for AI tools, for sure it's magnitudes in difference, but at actual work I'm constantly blocked by other teams, bureaucracy, legacy limitations, decisions, so my hands are tied with or without AI. Short, one-off projects independent of others also massive difference.
aerhardt 2y ago
It's doing the code of 30 devs equal to your level? What kinds of things do you program?
- maest 2y ago
  It's more like: are there tasks that would take me 30h but now only take me 1h? Maybe, but they are rare. Still a welcome boost, though.
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
freetanga 2y ago
I think it’s peak at whatever X, but when you weight all activities in a software delivery project, the averaged impact is diluted, which might be in line with the author and studies presented (I have not read them yet) There is a significant chunk of time spent on how to address your problem, how to break it down, thinking how it will perform in real life concurrently, etc. Sitting down and coding like you are on caffeine overdrive is not the full range of “programming” (at least in my neck of the woods). I see a lot of people mentioning 20x in here. Would they fire 20 devs tomorrow and eat their own dogfood?
nonameiguess 2y ago
Statements like this make no sense to me. I get that your personal experience may be you can type 30x as much code per unit of time, but where is the 30x better software? Where even is the 30x more software? I've never been part of any project I can think where how fast you can produce code was even a meaningful bottleneck to delivery compared to all the other impediments. Granted, I've largely worked in fairly high-security environments, but still.
- resters 2y ago
  For types of code that LLMs are "good at", the results are quite good if the LLM is told pretty much how to solve the problem. Generally they can map English language text into working code modules, can write tests for the modules, etc. You still have to plan out the approach and keep track of how it's supposed to be working or else they can be quite error prone and arbitrarily remove needed code or randomly forget important details. But once those constraints are understood it can often do quite well. At this point if we get larger context windows and LLMs become able to break down problems to include fetching relevant code into context when needed, and perhaps with the addition of some oversight training about not removing functionality, the qualty will get much, much better. But already they are a massive time saver and they are good at some of the more boring aspects of writing code, leaving the engineer to do more systems thinking and problem solving.

I think it's already surpassed 10x, and is closer to 25x or 30x. You just have to know what to ask it to do.

vundercind 2y ago

I should probably get around to actually using one of these things.

gadflyinyoureye 2y ago
I’d love to see people going through a real world walk through using such things. Some people who do cookie cutter common tasks find that an LLM can produce their whole code base. But ask it to make a change and what happens?
- abdullin 2y ago
  This is an example of my prompt to o1 a few days ago. First request produced refactoring suggestions. Second request (yes, I like results, implement them) produced multiple files that have just worked. —- Take a look at this code from my multi-mode (a la vim or old terminal apps) block-based content editor. I want to build on the keyboard interface and introduce a simple way to have simple commands with small popup. E.g. after doing "A" in "view" mode, show user a popup that expects H,T,I, or V. Or, after pressing "P" in view mode - show a small popup that has an text input waiting for the permission role for the page. Don't implement the changes, just think through how to extend existing code to make logic like that simple. Remember, I like simple code, I don't like spaghetti code and many small classes/files.
- falcolas 2y ago
  IME: I will either get a repeat of something already written (this happens all the time with java getter/setter boilerplate if the class isn't the one I'm currently working in), or I get some junior level attempt at what I ask it to do. The second case is the best case, and it still requires a code review with a fine toothed comb. For example, it loves to suggest plausible looking but non-existant methods on popular libraries. It's one of those things where a language aware autocomplete will often save me more time, because it doesn't try and second guess me.
  - resters 2y ago
    sometimes one must give it current docs to avoid that. Ironically gpt-4 always tries to use the old chat completions api when writing code to consume the open-ai api.
- JHonaker 2y ago
  I've been looking for such a walkthrough for months. I can't find anyone actually showing them working and building something. There are plenty of, "look at this code this thing spit out that successfully builds," but I've not been able to find anything I would consider truly showing someone building something novel. If anyone has a YT video or a detailed article, please show me!
  - resters 2y ago
    give me an example of the kind of thing you are curious about how to build in this way, and I will give you a step by step.

mewpmewp2 2y ago

It's very circumstance specific. In my side projects, where I work alone and have the perfect setup for AI tools, for sure it's magnitudes in difference, but at actual work I'm constantly blocked by other teams, bureaucracy, legacy limitations, decisions, so my hands are tied with or without AI. Short, one-off projects independent of others also massive difference.

aerhardt 2y ago

It's doing the code of 30 devs equal to your level? What kinds of things do you program?

maest 2y ago
It's more like: are there tasks that would take me 30h but now only take me 1h? Maybe, but they are rare. Still a welcome boost, though.

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

freetanga 2y ago

I think it’s peak at whatever X, but when you weight all activities in a software delivery project, the averaged impact is diluted, which might be in line with the author and studies presented (I have not read them yet) There is a significant chunk of time spent on how to address your problem, how to break it down, thinking how it will perform in real life concurrently, etc. Sitting down and coding like you are on caffeine overdrive is not the full range of “programming” (at least in my neck of the woods). I see a lot of people mentioning 20x in here. Would they fire 20 devs tomorrow and eat their own dogfood?

nonameiguess 2y ago

Statements like this make no sense to me. I get that your personal experience may be you can type 30x as much code per unit of time, but where is the 30x better software? Where even is the 30x more software? I've never been part of any project I can think where how fast you can produce code was even a meaningful bottleneck to delivery compared to all the other impediments. Granted, I've largely worked in fairly high-security environments, but still.

resters 2y ago
For types of code that LLMs are "good at", the results are quite good if the LLM is told pretty much how to solve the problem. Generally they can map English language text into working code modules, can write tests for the modules, etc. You still have to plan out the approach and keep track of how it's supposed to be working or else they can be quite error prone and arbitrarily remove needed code or randomly forget important details. But once those constraints are understood it can often do quite well. At this point if we get larger context windows and LLMs become able to break down problems to include fetching relevant code into context when needed, and perhaps with the addition of some oversight training about not removing functionality, the qualty will get much, much better. But already they are a massive time saver and they are good at some of the more boring aspects of writing code, leaving the engineer to do more systems thinking and problem solving.