5 ms·

Being a developer in the age of reasoning AI

63 points by QueensGambit 2y ago

deleted 2y ago
[deleted]
dartos 2y ago
a little melodramatic at the end. As with every openai release, I’d take it with try a grain of salt until it sees wide public usage. So far every iteration of gpt since 4 has been pushed as a replacement for developers, but I have yet to feel that these tools are even consistently useful for my day to day.
- josephg 2y ago
  I think they definitely add some value. For example, I haven’t written complex sql queries since I was in school 20 years ago. Recently Claude helped me write a recursive database query which did an in-database graph traversal. It would have taken me way more time to relearn how to do that from scratch. But I don’t think many people are that worried about gpt4. We’re worried about what it’ll look like with 5-10 more years of progress. O3 is apparently already better than 99% of competition programmers or something - it just costs thousands of dollars of compute per query. But give them a few short years.
  - stanislavb 2y ago
    Yes, I think this is the major factor to consider. If it’s only the cost that’s stopping it from replacing real people, then it’s a matter of a few to max several years to be more than cost effective.
  - silisili 2y ago
    Complex SQL and regex are pretty much my only current use cases for it. Saves tons of time doing things you don't do every day so rusty, but still can tell whether it's doing what you were asking for.
    - jordanb 2y ago
      It's basically a replacement for Stack Overflow, made possible by scraping Stack Overflow.
  - dartos 2y ago
    I’ll believe what they say about O3 when it starts seeing use in the wild.
  - rootnod3 2y ago
    On the other hand one could argue that you should understand the code you write. A super complex in-database graph traversal should probably be properly understood in case performance problems pop up. Or the problem itself should be broken down differently.
    - josephg 2y ago
      Once Claude gave me the query, I read it, asked questions about how it works and asked about potential changes. “Why is this bit here? Couldn’t we do it like this instead?” And so on. We went through about 5 variants of the same query, Claude and I. They didn’t all work. I’m still no expert. But I have a lot more confidence in that query & how it works than I would have from cobbling something together from stackoverflow.
- righthand 2y ago
  I have needed 0 LLM tools to efficiently do my job since any release of any models. This predates OpenAI. The code suggestion/autocomplete has been sometimes nice but rarely is what I’m going to type out. I don’t discourage use of LLMs but I don’t encourage it either. If you need to pay OpenAI to get your work done then that’s on you.
  - wawhal 2y ago
    > but rarely is what I’m going to type out. Just curious, does it matter? For some simple boilerplate code, simple test cases and simple skaffolding, which is needed in most applications, do you even care?
    - righthand 2y ago
      But simple scaffolding, test cases, boilerplate was around before LLMs via code generators/scaffolders, so yes, it’s revised history to pretend like an LLM invented those things. None of those things are complex ideas either. I’m not sure I understand what you’re asking though.
      - anonzzzies 2y ago
        While true, LLMs are just a more advanced version of it. Let's take Go as example as that is a very boilerplatey language; with claude, I only have to write the meat, for instance; result, err:= doSomething() and it'll generate the rest around it. Every time. Faster than I would type it and with more eye for detail (I am lazy, I will forget things as I did it now the 10004th time). So you can imagine some boilerplate scaffold thing that would do this err handling for you, but if you have a custom one, for 'old' tools you would have to tell the tool; now it just does it. It especially becomes clear in frontend tasks: it just generates 100+ lines of react that work and look good; who likes writing they type of thing manually? I know 0 people who do: after that you will want to tweak it, but the 100k+ lines of react we generally have in a project are not something we would want to write these days from scratch. Which old school scaffold etc tool does that? So that it is 80-90% there for SaaS first shot? I know nothing (doesn't exist) without the lovely verboseness of having to type a million tags? It saves us a lot of time we don't have to spend on boring stuff. We only have to write the actual business logic and data models; the rest drops out. And of course we only have to write that very loosely, not formally.
- dawnerd 2y ago
  Lately copilot has been giving me outright u correct information and when you tell it, it’ll admit it lied and give another incorrect answer. Not even that obscure of a question either. I have zero trust in it being able to generate anything more than bare bones boiler plate (and even then). I also feel they’ve made responses fairly wordy to make it appear to know stuff.
  - sdesol 2y ago
    Can you ask for the prompt when chatting?
skydhash 2y ago
> Users think with data. Imagine you are a cashier at a grocery store. You learn how to calculate the amount by watching the owner do it for a couple of customers. No one does that. You go to school and learn numbers and basic arithmetic, which gives you all the formulas you need. And even illiterate people work out quickly the minimum they need to do transactions. > This ability to abstract logic is what separates developers from users Then everyone is a developer as everyone can write a serie of instructions which abstract action from context. It just that the computer is very limited and you need to be very precise. But what it can do, it can do it very fast. What is instant for us is an eternity in CPU time, and you can do a lot with simple operations if you can do a great number of them. > Just like a developer's brain can think about a problem and come up with a solution using code, o3 generates a program (i.e., metadata) on the fly to solve the problem Again that's not happening. You think about a problem and solves it by constraining yourself to the basic operations you have available (the APIs, languages, libraries, platforms). The code is just the written form of that solution. You can't invent operations that does not exist, you can only group them under a new name. Each set of names is another layer of abstraction that make it easier to write the code. But the constraints don't go away. The code is not how you create a solution. It's how you instruct the machine to replicate the solution you came up with.
- austin-cheney 2y ago
  I disagree with everything you commented. As a self-taught developer I absolutely promise you don't need to go to any school to learn the requisite skills to achieve a variety of complex things. You absolutely can learn from both trial and observation. Yes, everyone actually is potentially a developer, because there are no baselines to the contrary. I have met many developers that should never have been there professionally. Not everybody is capable of actual engineering though, as engineers follow processes and measure things. So many people writing code for a living absolutely cannot measure things. I think your fundamental problem is loss of empathy. Perhaps you understand your own limitations but do not seem to understand other people learn differently than you do or may have wildly different experiences than you have.
  - skydhash 2y ago
mediumsmart 2y ago
The interesting development of course being the approaching Age of reasonable software made by mere mortals and AI
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
feverzsj 2y ago
These articles are like web3/crypto spams.
- righthand 2y ago
  Praying for the day we can move past talking about how cool autocomplete tricks are.
Willingham 2y ago
Wouldn’t the end game here be that the AI just writes straight to machine code? It’s not like some devs are safe, those who maintain coding languages would be out of a job too. I think AI would find it awfully silly to have to code up a language, maintain it, then turn around and use that language to code with. Bare metal AI seems like the end game to me.
- efitz 2y ago
  We’ve lots of programming languages; most businesses need only an OS (Linux- free), an application language (python, java, C*), a database language (SQL), and a front end be language (js + html). No need to go to bare metal in most cases as generative AI is already good at these. The mostly unsolved problems right now are around build, deployment, and operations.
- im3w1l 2y ago
  No I don't think so. Both human and AI benefits from building on top of existing abstractions: langugages, frameworks and libraries.
- righthand 2y ago
  Code is written to be auditable by humans, if you just generate machine code how will you know the level of correctness? Would you save the prompts then I have to prompt review before merge? That would end up being much wordier than a code file.
  - tasuki 2y ago
    Humans write the tests. AI writes the implementation. Just write good tests: property based. Or actual formal specification...
    - episteme 2y ago
      It's much quicker to write code than a formal spec for that code.
efitz 2y ago
Why do developers continue to work on AI? Sooner or later (probably sooner) it will be adequate enough to remove the need for many or most developers. I listened to this week’s All In podcast and the podcasters (all wildly successful tech entrepreneurs) were quite emphatic that they were seeking a world where they were not beholden to developers. They envision a world where non- or semi-technical employees create the desired software via AI; other AI agents would then figure out how to deploy it. I feel like we are at a similar point to many other professions in the past facing a rapid technological change. But in the case of software development, developers themselves are hastening their own demise. Just something to think on.
- fourside 2y ago
  The people directly responsible for advancing the latest AI tech are both small in number relative to the entire population of devs and they also have a chance of building generational wealth as a result of it. The rest of us are trying to keep up with AI because it’s both interesting and because we don’t have a choice.
  - righthand 2y ago
    You have a choice. What is preventing choice?
- jarsin 2y ago
  I find it fascinating how people that successful don't take that thought process one step further. If they use AI to build an app by just telling it what to do then everyone on the planet can look at how their app works and describe it to their AI thus making a clone. Even Eric Schmidt said at Standford that we would be able use AI to copy TikTok and steal all of it's content and users in the next few years. Interestingly though he didn't even think to mention Youtube...
  - deleted 2y ago
    [deleted]
- ewoodrich 2y ago
rurban 2y ago
I wouldn't call o3 a reasoning AI yet. It gets better with its abstraction abilities but that's a still a very far step from reasoning and logic. Something like 10%

63 points by QueensGambit 2y ago

deleted 2y ago

[deleted]

a little melodramatic at the end. As with every openai release, I’d take it with try a grain of salt until it sees wide public usage. So far every iteration of gpt since 4 has been pushed as a replacement for developers, but I have yet to feel that these tools are even consistently useful for my day to day.

josephg 2y ago
I think they definitely add some value. For example, I haven’t written complex sql queries since I was in school 20 years ago. Recently Claude helped me write a recursive database query which did an in-database graph traversal. It would have taken me way more time to relearn how to do that from scratch. But I don’t think many people are that worried about gpt4. We’re worried about what it’ll look like with 5-10 more years of progress. O3 is apparently already better than 99% of competition programmers or something - it just costs thousands of dollars of compute per query. But give them a few short years.
- stanislavb 2y ago
  Yes, I think this is the major factor to consider. If it’s only the cost that’s stopping it from replacing real people, then it’s a matter of a few to max several years to be more than cost effective.
- silisili 2y ago
  Complex SQL and regex are pretty much my only current use cases for it. Saves tons of time doing things you don't do every day so rusty, but still can tell whether it's doing what you were asking for.
  - jordanb 2y ago
    It's basically a replacement for Stack Overflow, made possible by scraping Stack Overflow.
- dartos 2y ago
  I’ll believe what they say about O3 when it starts seeing use in the wild.
- rootnod3 2y ago
  On the other hand one could argue that you should understand the code you write. A super complex in-database graph traversal should probably be properly understood in case performance problems pop up. Or the problem itself should be broken down differently.
  - josephg 2y ago
    Once Claude gave me the query, I read it, asked questions about how it works and asked about potential changes. “Why is this bit here? Couldn’t we do it like this instead?” And so on. We went through about 5 variants of the same query, Claude and I. They didn’t all work. I’m still no expert. But I have a lot more confidence in that query & how it works than I would have from cobbling something together from stackoverflow.
righthand 2y ago
I have needed 0 LLM tools to efficiently do my job since any release of any models. This predates OpenAI. The code suggestion/autocomplete has been sometimes nice but rarely is what I’m going to type out. I don’t discourage use of LLMs but I don’t encourage it either. If you need to pay OpenAI to get your work done then that’s on you.
- wawhal 2y ago
  > but rarely is what I’m going to type out. Just curious, does it matter? For some simple boilerplate code, simple test cases and simple skaffolding, which is needed in most applications, do you even care?
  - righthand 2y ago
    But simple scaffolding, test cases, boilerplate was around before LLMs via code generators/scaffolders, so yes, it’s revised history to pretend like an LLM invented those things. None of those things are complex ideas either. I’m not sure I understand what you’re asking though.
    - anonzzzies 2y ago
      While true, LLMs are just a more advanced version of it. Let's take Go as example as that is a very boilerplatey language; with claude, I only have to write the meat, for instance; result, err:= doSomething() and it'll generate the rest around it. Every time. Faster than I would type it and with more eye for detail (I am lazy, I will forget things as I did it now the 10004th time). So you can imagine some boilerplate scaffold thing that would do this err handling for you, but if you have a custom one, for 'old' tools you would have to tell the tool; now it just does it. It especially becomes clear in frontend tasks: it just generates 100+ lines of react that work and look good; who likes writing they type of thing manually? I know 0 people who do: after that you will want to tweak it, but the 100k+ lines of react we generally have in a project are not something we would want to write these days from scratch. Which old school scaffold etc tool does that? So that it is 80-90% there for SaaS first shot? I know nothing (doesn't exist) without the lovely verboseness of having to type a million tags? It saves us a lot of time we don't have to spend on boring stuff. We only have to write the actual business logic and data models; the rest drops out. And of course we only have to write that very loosely, not formally.
dawnerd 2y ago
Lately copilot has been giving me outright u correct information and when you tell it, it’ll admit it lied and give another incorrect answer. Not even that obscure of a question either. I have zero trust in it being able to generate anything more than bare bones boiler plate (and even then). I also feel they’ve made responses fairly wordy to make it appear to know stuff.
- sdesol 2y ago
  Can you ask for the prompt when chatting?

skydhash 2y ago

> Users think with data. Imagine you are a cashier at a grocery store. You learn how to calculate the amount by watching the owner do it for a couple of customers. No one does that. You go to school and learn numbers and basic arithmetic, which gives you all the formulas you need. And even illiterate people work out quickly the minimum they need to do transactions. > This ability to abstract logic is what separates developers from users Then everyone is a developer as everyone can write a serie of instructions which abstract action from context. It just that the computer is very limited and you need to be very precise. But what it can do, it can do it very fast. What is instant for us is an eternity in CPU time, and you can do a lot with simple operations if you can do a great number of them. > Just like a developer's brain can think about a problem and come up with a solution using code, o3 generates a program (i.e., metadata) on the fly to solve the problem Again that's not happening. You think about a problem and solves it by constraining yourself to the basic operations you have available (the APIs, languages, libraries, platforms). The code is just the written form of that solution. You can't invent operations that does not exist, you can only group them under a new name. Each set of names is another layer of abstraction that make it easier to write the code. But the constraints don't go away. The code is not how you create a solution. It's how you instruct the machine to replicate the solution you came up with.

austin-cheney 2y ago
I disagree with everything you commented. As a self-taught developer I absolutely promise you don't need to go to any school to learn the requisite skills to achieve a variety of complex things. You absolutely can learn from both trial and observation. Yes, everyone actually is potentially a developer, because there are no baselines to the contrary. I have met many developers that should never have been there professionally. Not everybody is capable of actual engineering though, as engineers follow processes and measure things. So many people writing code for a living absolutely cannot measure things. I think your fundamental problem is loss of empathy. Perhaps you understand your own limitations but do not seem to understand other people learn differently than you do or may have wildly different experiences than you have.
- skydhash 2y ago

mediumsmart 2y ago

The interesting development of course being the approaching Age of reasonable software made by mere mortals and AI

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

feverzsj 2y ago

These articles are like web3/crypto spams.

righthand 2y ago
Praying for the day we can move past talking about how cool autocomplete tricks are.

Willingham 2y ago

Wouldn’t the end game here be that the AI just writes straight to machine code? It’s not like some devs are safe, those who maintain coding languages would be out of a job too. I think AI would find it awfully silly to have to code up a language, maintain it, then turn around and use that language to code with. Bare metal AI seems like the end game to me.

efitz 2y ago
We’ve lots of programming languages; most businesses need only an OS (Linux- free), an application language (python, java, C*), a database language (SQL), and a front end be language (js + html). No need to go to bare metal in most cases as generative AI is already good at these. The mostly unsolved problems right now are around build, deployment, and operations.
im3w1l 2y ago
No I don't think so. Both human and AI benefits from building on top of existing abstractions: langugages, frameworks and libraries.
righthand 2y ago
Code is written to be auditable by humans, if you just generate machine code how will you know the level of correctness? Would you save the prompts then I have to prompt review before merge? That would end up being much wordier than a code file.
- tasuki 2y ago
  Humans write the tests. AI writes the implementation. Just write good tests: property based. Or actual formal specification...
  - episteme 2y ago
    It's much quicker to write code than a formal spec for that code.

efitz 2y ago

Why do developers continue to work on AI? Sooner or later (probably sooner) it will be adequate enough to remove the need for many or most developers. I listened to this week’s All In podcast and the podcasters (all wildly successful tech entrepreneurs) were quite emphatic that they were seeking a world where they were not beholden to developers. They envision a world where non- or semi-technical employees create the desired software via AI; other AI agents would then figure out how to deploy it. I feel like we are at a similar point to many other professions in the past facing a rapid technological change. But in the case of software development, developers themselves are hastening their own demise. Just something to think on.

fourside 2y ago
The people directly responsible for advancing the latest AI tech are both small in number relative to the entire population of devs and they also have a chance of building generational wealth as a result of it. The rest of us are trying to keep up with AI because it’s both interesting and because we don’t have a choice.
- righthand 2y ago
  You have a choice. What is preventing choice?
jarsin 2y ago
I find it fascinating how people that successful don't take that thought process one step further. If they use AI to build an app by just telling it what to do then everyone on the planet can look at how their app works and describe it to their AI thus making a clone. Even Eric Schmidt said at Standford that we would be able use AI to copy TikTok and steal all of it's content and users in the next few years. Interestingly though he didn't even think to mention Youtube...
- deleted 2y ago
  [deleted]
ewoodrich 2y ago

rurban 2y ago

I wouldn't call o3 a reasoning AI yet. It gets better with its abstraction abilities but that's a still a very far step from reasoning and logic. Something like 10%