4 ms·

… because programming languages are the right level of precision for specifying a program you want. Natural language isn’t it. Of course you need to review and

… because programming languages are the right level of precision for specifying a program you want. Natural language isn’t it. Of course you need to review and edit what it generates. Of course it’s often easier to make the change yourself instead of describing how to make the change. I wonder if the independent studies that show Copilot increasing the rate of errors in software have anything to do with this less bold attitude. Most people selling AI are predicting the obsolescence of human authors.

soulofmischief 1y ago
Transformers can be used to automate testing, create deeper and broader specification, accelerate greenfield projects, rapidly and precisely expand a developer's knowledge as needed, navigate unfamiliar APIs without relying on reference, build out initial features, do code review and so much more. Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual. > Most people selling AI are predicting the obsolescence of human authors. It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace. This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step. That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.
- interstice 1y ago
  > Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual. It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
  - recursive 1y ago
    > It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. That's not true at all. ... It's only pretending to be happy.
  - soulofmischief 1y ago
    That's a combination of current context limitations and a lack of quality tooling and prompting. A well-designed agent can absolutely roll back code if given proper context and access to tooling such as git. Even flushing context/message history becomes viable for agents if the functionality is exposed to them.
    - jashmatthews 1y ago
      Can we demonstrate them doing that? Absolutely. Will they fail to do it in practice once they poison their own context hallucinating libraries or functions that don’t exist? Absolutely. That’s the tricky part of working with agents.
  - jcalvinowens 1y ago
    This is it for me. If you ask these models to write something new, the result can be okay. But the second you start iterating with them... the codebase goes to shit, because they never delete code. Never. They always bolt new shit on to solve any problem, even when there's an incredibly obvious path to achieve the same thing in a much more maintainable way with what already exists. Show me a language model that can turn rube goldberg code into good readable code, and I'll suddenly become very interested in them. Until then, I remain a hater, because they only seem capable of the opposite :)
JoeOfTexas 1y ago
Doesn't AI have diminishing returns on it's pseudo creativity? Throw all the training output of LLM into a circle. If all input comes from other LLM output, the circle never grows. Humans constantly step outside the circle. Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.
- svachalek 1y ago
  I think you're either imagining the circle too small or overestimating how often humans step outside it. The typical programming job involves lots and lots of work, and yet none of it creating wholly original computer science. Current LLMs can customize well known UI/IO/CRUD/REST patterns with little difficulty, and these make up the vast majority of commercial software development.
  - sarchertech 1y ago
    Frameworks and low code systems have been able to do that for years. The reason they haven’t replaced programmers is that every system eventually becomes a special unique snowflake as long as it has time and users. I’m getting maybe a 10-20% productivity boost using AI on mature codebases. Nice but not life changing.
    - spongebobstoes 1y ago
      a 20% boost is huge, for 3 years since chatgpt. even if it stopped there, that's 20% fewer people that need to be in your role, which is at least tens of thousands of jobs
      - alternatex 1y ago
        If devs produce 20% more, won't companies hire more since the gain/loss equation is starting to tilt their way even more? I find it odd that people think productivity increases lead to layoffs.
      - benjaminwootton 1y ago
JamesBarney 1y ago
Right level for exactly specifying program behavior in a global domain without context. But once you add repo context, domain knowledge etc... programming languages are far too verbose.
msgodel 1y ago
In my experience the best use of AI is to stay in the flow state when you get blocked by an API you don't understand or a feature you don't want to implement for whatever reason.
al_borland 1y ago
I wasn’t in the industry to see it first hand, but was this same criticism levied against higher level languages when they first hit the scene? Something to the effect of high level languages not being the right level of precision, because the programmer wanted to directly manipulate what was happening in memory, and the high level languages are not the right level of precision for that? The issue with natural language isn’t that it’s impossible to be precise, it’s that most people aren’t, or they are precise about what they want it to do for them, but not what the computer needs to do to make it happen. This leads to a lot of guessing by engineerings as they try to translate the business requirements into code. Now the LLM is doing that guessing, often with less context about the broader business objectives, or an understanding of the people writing those requirements.
- agentultra 1y ago
  > was this same criticism levied against higher level languages when they first hit the scene? No. Some were concerned that the output of compilers couldn’t match the quality of what could be done by a competent programmer at the time. That was true for a time. Then compilers got better. Nobody was concerned that compilers were going to be used by capitalists to lay them off and seize the means of producing programs by turning it into property.
giancarlostoro 1y ago
> Of course you need to review and edit what it generates. Treating LLMs as a scaffolding tool yields better results at least for me personally. I just brain dump what I'm thinking of building and ask for it to give me models, and basic controllers using said models, then I just worry about the views and business logic.

soulofmischief 1y ago

Transformers can be used to automate testing, create deeper and broader specification, accelerate greenfield projects, rapidly and precisely expand a developer's knowledge as needed, navigate unfamiliar APIs without relying on reference, build out initial features, do code review and so much more. Even if code is the right medium for specifying a program, transformers act as an automated interface between that medium and natural language. Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual. > Most people selling AI are predicting the obsolescence of human authors. It's entirely possible that we do become obsolete for a wide variety of programming domains. That's simply a reality, just as weavers saw massive layoffs in the wake of the automated loom, or scribes lost work after the printing press, or human calculators became pointless after high-precision calculators became commonplace. This replacement might not happen tomorrow, or next year, or even in the next decade, but it's clear that we are able to build capable models. What remains to be done is R&D around things like hallucinations, accuracy, affordability, etc. as well as tooling and infrastructure built around this new paradigm. But the cat's out of the bag, and we are not returning to a paradigm that doesn't involve intelligent automation in our daily work; programming is literally about automating things and transformers are a massive forward step. That doesn't really mean anything, though; You can still be as involved in your programming work as you'd like. Whether you can find paid, professional work depends on your domain, skill level and compensation preferences. But you can always program for fun or personal projects, and decide how much or how little automation you use. But I will recommend that you take these tools seriously, and that you aren't too dismissive, or you could find yourself left behind in a rapidly evolving landscape, similarly to the advent of personal computing and the internet.

interstice 1y ago
> Modern high-end transformers have no problem producing code, while benefiting from a wealth of knowledge that far surpasses any individual. It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. I've yet to see one that can back itself out of a logical corner.
- recursive 1y ago
  > It will also still happily turn your whole codebase into garbage rather than undo the first thing it tried to try something else. That's not true at all. ... It's only pretending to be happy.
- soulofmischief 1y ago
  That's a combination of current context limitations and a lack of quality tooling and prompting. A well-designed agent can absolutely roll back code if given proper context and access to tooling such as git. Even flushing context/message history becomes viable for agents if the functionality is exposed to them.
  - jashmatthews 1y ago
    Can we demonstrate them doing that? Absolutely. Will they fail to do it in practice once they poison their own context hallucinating libraries or functions that don’t exist? Absolutely. That’s the tricky part of working with agents.
- jcalvinowens 1y ago
  This is it for me. If you ask these models to write something new, the result can be okay. But the second you start iterating with them... the codebase goes to shit, because they never delete code. Never. They always bolt new shit on to solve any problem, even when there's an incredibly obvious path to achieve the same thing in a much more maintainable way with what already exists. Show me a language model that can turn rube goldberg code into good readable code, and I'll suddenly become very interested in them. Until then, I remain a hater, because they only seem capable of the opposite :)

JoeOfTexas 1y ago

Doesn't AI have diminishing returns on it's pseudo creativity? Throw all the training output of LLM into a circle. If all input comes from other LLM output, the circle never grows. Humans constantly step outside the circle. Perhaps LLM can be modified to step outside the circle, but as of today, it would be akin to monkeys typing.

svachalek 1y ago
I think you're either imagining the circle too small or overestimating how often humans step outside it. The typical programming job involves lots and lots of work, and yet none of it creating wholly original computer science. Current LLMs can customize well known UI/IO/CRUD/REST patterns with little difficulty, and these make up the vast majority of commercial software development.
- sarchertech 1y ago
  Frameworks and low code systems have been able to do that for years. The reason they haven’t replaced programmers is that every system eventually becomes a special unique snowflake as long as it has time and users. I’m getting maybe a 10-20% productivity boost using AI on mature codebases. Nice but not life changing.
  - spongebobstoes 1y ago
    a 20% boost is huge, for 3 years since chatgpt. even if it stopped there, that's 20% fewer people that need to be in your role, which is at least tens of thousands of jobs
    - alternatex 1y ago
      If devs produce 20% more, won't companies hire more since the gain/loss equation is starting to tilt their way even more? I find it odd that people think productivity increases lead to layoffs.
    - benjaminwootton 1y ago

JamesBarney 1y ago

Right level for exactly specifying program behavior in a global domain without context. But once you add repo context, domain knowledge etc... programming languages are far too verbose.

msgodel 1y ago

In my experience the best use of AI is to stay in the flow state when you get blocked by an API you don't understand or a feature you don't want to implement for whatever reason.

al_borland 1y ago

I wasn’t in the industry to see it first hand, but was this same criticism levied against higher level languages when they first hit the scene? Something to the effect of high level languages not being the right level of precision, because the programmer wanted to directly manipulate what was happening in memory, and the high level languages are not the right level of precision for that? The issue with natural language isn’t that it’s impossible to be precise, it’s that most people aren’t, or they are precise about what they want it to do for them, but not what the computer needs to do to make it happen. This leads to a lot of guessing by engineerings as they try to translate the business requirements into code. Now the LLM is doing that guessing, often with less context about the broader business objectives, or an understanding of the people writing those requirements.

agentultra 1y ago
> was this same criticism levied against higher level languages when they first hit the scene? No. Some were concerned that the output of compilers couldn’t match the quality of what could be done by a competent programmer at the time. That was true for a time. Then compilers got better. Nobody was concerned that compilers were going to be used by capitalists to lay them off and seize the means of producing programs by turning it into property.

giancarlostoro 1y ago

> Of course you need to review and edit what it generates. Treating LLMs as a scaffolding tool yields better results at least for me personally. I just brain dump what I'm thinking of building and ask for it to give me models, and basic controllers using said models, then I just worry about the views and business logic.