4 ms·

It seems like a solid set of criteria for how easily a task can be automated by AI agents is: - extent to which correctness of solution be easily specified and

It seems like a solid set of criteria for how easily a task can be automated by AI agents is: - extent to which correctness of solution be easily specified and checked - extent to which new potential solutions can be implemented as text - extent to which prior art exists online This basically maps to software engineering and math. I think a fair bit of AI hype comes from the fact that the very architects of AI are the people whose jobs are most easily automated by AI. They think, “if my job receives this much of a boost from AI, surely every job will be the same”. Ironically it couldn’t be further from the truth… and likewise the predictions of widespread labor obsolescence

richardbarosky 3mo ago
Interesting take! I feel like 2 of them are maybe overstated: > - extent to which correctness of solution be easily specified and checked I don't think most software is like solving a math problem or series of math problems. Algorithmic problems are very narrow and might be more like this though, where an oracle that verifies answers as either correct or incorrect exists beforehand. The correctness function of most software is how much users want to use/pay for it, which is a pretty fuzzy problem. Since the cost of copying software is effectively zero, software systems also tend to be be unique rather than being exactly like something else, and don't converge to be like another software system but rather diverge. The prior art point is an interesting one. At least for applications as a whole, there isn't really prior art for a material amount of all the problems/tradeoffs a non-trivial software application embodies. For a todo list app or make a social network project, there's plenty of prior art to be sufficient to build something with an LLM system, but probably not most apps. That's my initial intuition anyway.
- virgildotcodes 3mo ago
  > how much users want to use/pay for it, which is a pretty fuzzy problem Isn’t this quantifiable by revenue?
  - srdjanr 3mo ago
    It's a very lagging metric, and also influenced by sales, market conditions for your customers etc.
- nextaccountic 3mo ago
  > The correctness function of most software is how much users want to use/pay for it, which is a pretty fuzzy problem. This is indeed a problem, but we (well we humans, but also agents) correct this by introducing partial verifiers like tests, running lints, type checkers, etc that both verify that at least some requirements are met, and also verify that the software is internally self-consistent. And more importantly, breaking down the software into small modules that are more clearly defined The task to get user requirements and decide the right architecture requires more creativity and is fuzzier than implementing each module. Perhaps over time there will be models specialized for each task. And actually this approach mirrors math, if agents ever develop new math on the fly to solve a problem. Developing new math is analogous to deciding how to break up the software in modules (and also analogous to designing a language, if we subscribe to the SICP notion that libraries actually are mini-languages on their own) That is, LLMs for math is still in the phase where they are used to prove stuff (which by Curry-Howard, is like writing code), rather than deciding what to prove (which is like deciding the signatures of each function, again by Curry-Howard)
- plaidfuji 3mo ago
  > I don't think most software is like solving a math problem or series of math problems. I agree with you when talking about high level software design. As you say it ultimately boils down to building something people will pay for, which is a fuzzy correctness function that is hard to measure within an agentic sandbox. But unlike other professions, there are a lot of sub-problems within software development that are able to be fully specified and tested via text generation. And I think the developers of AI overestimate how many such problems exist for other professions. What I’m saying is most other professions tend to be “fuzzy all the way down”… which incidentally is why they select for people with fuzzier skillsets. Or in other cases, like physical engineering, the correctness is quantitative, but the necessary I/O integrations and physical automation lower the ROI of agentic workflows considerably.
ianm218 3mo ago
> Ironically it couldn’t be further from the truth… and likewise the predictions of widespread labor obsolescence Could you explain what you mean here? It feels like there is one bucket of verifiable work - programming, math etc that AI will clearly excel at. There is another large bucket of like law/ accounting/ financial analysis where I don’t have any reason to think AI won’t be super human at, but the work is more on bringing all the domain expertise into harnesses and software. Is there aspects of knowledge work that you think AI wouldn’t excel at in the long run?
- nsz65 3mo ago
  Anything where unpredictability is a daily part of the job.
SubiculumCode 3mo ago
Human: I specialize in tasks in which the correctness of my solution cannot be easily specified or checked. Truuuuussssttttt meeeeee. I'm just saying, let's not get painted into that corner completely as a species :)
pianopatrick 3mo ago
Let's assume AI is only good at software and math. If we can use AI to write lots of good software for cheap, that software can automate away a lot of jobs. So AI does not have to take the jobs directly, software written by AI can. Especially if that works for software for robots.
- semiquaver 3mo ago
  Yup. There’s infinite latent demand for software, we just don’t know yet.
- dominotw 3mo ago
  why the middle man when it can generate software on the fly
aurareturn 3mo ago
Many white collar jobs are verifiable. Make a robot and suddenly real world tasks are verifiable too.
dominotw 3mo ago
per dwarkesh its also 'grindablity' in training .
wyager 3mo ago
> the very architects of AI are the people whose jobs are most easily automated by AI Think very hard about what this implies for the future pace of AI R&D
- tripzilch 3mo ago
  It's going to turn into slop, exponentially fast, any moment now. No but seriously it could imply grinding to a halt, turning into a laughing stock of slop, or, imho much less likely given the current state of AI (barring huge technological breakthrough and a couple of other assumptions) you were probably implying an intelligence explosion.
kertoip_1 3mo ago
The job of a programmer isn't to write code, but to automate things. Code itself doesn't have any value unless it solves some real problem not related to coding. So if the work of a programmer can be automated then this means that any work can be automated. So no, it's not about software engineering only.
- emil-lp 3mo ago
  > if the work of a programmer can be automated then this means that any work can be automated This is a very typical programmer thing to say.
  - imdsm 3mo ago
    i think most things can be automated, but not everything i think AI will just replace jobs with new jobs, and humans will continue with open-ended goal setting, and probably jobs where being human is necessary: responsibility and authenticity, so judges, politics, leaders, chefs, etc. so in a way, the above commenter was correct, anything a programmer can automate, AI can do
    - hoppp 3mo ago
      It is a little bit suspicious that the big replacement didn't happen, yet. If it was so valuable then AI companies would directly create valuable things, not just sell API tokens. A proof like this is a good effort at trying to provide value directly, but still far away from real use. When OpenAI cures cancer I'll accept the fate, but till then I'm still seeing a lot of gambling going on.
      - deebosong 3mo ago
        Reminds me of crypto. A lotta tokens. A lotta gambling. Very little supplanting of crypto from fiat. Very few use-cases that revolutionize society, etc.
        3mo ago
petra 3mo ago
- the extent to which solutions could be implemented as text: not sure about that. AlphaFold is basically a mechanical/geometrical/Chemical problem. There are other scientific transformer based models. - the extent which solutions exist online - if you have a strong verification tool, you can generate examples, you can generate feedback, i think you could start with small/smaller prior art - the extent which solutions could be specified and checked - if you have a lot of priort art, maybe llm's can find the good "patterns" and compare against them, and at least get close to a good results - but you'd still need human verification.
nbardy 3mo ago
These are true and it does make theses fields the first to fall, but also the hype comes form the fact that it can escape these conditions as well: - generalize to non verifiable domains (https://arxiv.org/abs/2507.17746 https://arxiv.org/abs/2507.17746)[This is on going work but has had steady progress in many angles of attack) - Visual Reasoning is alive and well in video diffusion and image models(see recent works around using diffusion model priors as world model features for physics reasoning) - Prior art exists for anything humans can do online?(Is this one even a hold back?)

richardbarosky 3mo ago

Interesting take! I feel like 2 of them are maybe overstated: > - extent to which correctness of solution be easily specified and checked I don't think most software is like solving a math problem or series of math problems. Algorithmic problems are very narrow and might be more like this though, where an oracle that verifies answers as either correct or incorrect exists beforehand. The correctness function of most software is how much users want to use/pay for it, which is a pretty fuzzy problem. Since the cost of copying software is effectively zero, software systems also tend to be be unique rather than being exactly like something else, and don't converge to be like another software system but rather diverge. The prior art point is an interesting one. At least for applications as a whole, there isn't really prior art for a material amount of all the problems/tradeoffs a non-trivial software application embodies. For a todo list app or make a social network project, there's plenty of prior art to be sufficient to build something with an LLM system, but probably not most apps. That's my initial intuition anyway.

virgildotcodes 3mo ago
> how much users want to use/pay for it, which is a pretty fuzzy problem Isn’t this quantifiable by revenue?
- srdjanr 3mo ago
  It's a very lagging metric, and also influenced by sales, market conditions for your customers etc.
nextaccountic 3mo ago
> The correctness function of most software is how much users want to use/pay for it, which is a pretty fuzzy problem. This is indeed a problem, but we (well we humans, but also agents) correct this by introducing partial verifiers like tests, running lints, type checkers, etc that both verify that at least some requirements are met, and also verify that the software is internally self-consistent. And more importantly, breaking down the software into small modules that are more clearly defined The task to get user requirements and decide the right architecture requires more creativity and is fuzzier than implementing each module. Perhaps over time there will be models specialized for each task. And actually this approach mirrors math, if agents ever develop new math on the fly to solve a problem. Developing new math is analogous to deciding how to break up the software in modules (and also analogous to designing a language, if we subscribe to the SICP notion that libraries actually are mini-languages on their own) That is, LLMs for math is still in the phase where they are used to prove stuff (which by Curry-Howard, is like writing code), rather than deciding what to prove (which is like deciding the signatures of each function, again by Curry-Howard)
plaidfuji 3mo ago
> I don't think most software is like solving a math problem or series of math problems. I agree with you when talking about high level software design. As you say it ultimately boils down to building something people will pay for, which is a fuzzy correctness function that is hard to measure within an agentic sandbox. But unlike other professions, there are a lot of sub-problems within software development that are able to be fully specified and tested via text generation. And I think the developers of AI overestimate how many such problems exist for other professions. What I’m saying is most other professions tend to be “fuzzy all the way down”… which incidentally is why they select for people with fuzzier skillsets. Or in other cases, like physical engineering, the correctness is quantitative, but the necessary I/O integrations and physical automation lower the ROI of agentic workflows considerably.

ianm218 3mo ago

> Ironically it couldn’t be further from the truth… and likewise the predictions of widespread labor obsolescence Could you explain what you mean here? It feels like there is one bucket of verifiable work - programming, math etc that AI will clearly excel at. There is another large bucket of like law/ accounting/ financial analysis where I don’t have any reason to think AI won’t be super human at, but the work is more on bringing all the domain expertise into harnesses and software. Is there aspects of knowledge work that you think AI wouldn’t excel at in the long run?

nsz65 3mo ago
Anything where unpredictability is a daily part of the job.

SubiculumCode 3mo ago

Human: I specialize in tasks in which the correctness of my solution cannot be easily specified or checked. Truuuuussssttttt meeeeee. I'm just saying, let's not get painted into that corner completely as a species :)

pianopatrick 3mo ago

Let's assume AI is only good at software and math. If we can use AI to write lots of good software for cheap, that software can automate away a lot of jobs. So AI does not have to take the jobs directly, software written by AI can. Especially if that works for software for robots.

semiquaver 3mo ago
Yup. There’s infinite latent demand for software, we just don’t know yet.
dominotw 3mo ago
why the middle man when it can generate software on the fly

aurareturn 3mo ago

Many white collar jobs are verifiable. Make a robot and suddenly real world tasks are verifiable too.

dominotw 3mo ago

per dwarkesh its also 'grindablity' in training .

wyager 3mo ago

> the very architects of AI are the people whose jobs are most easily automated by AI Think very hard about what this implies for the future pace of AI R&D

tripzilch 3mo ago
It's going to turn into slop, exponentially fast, any moment now. No but seriously it could imply grinding to a halt, turning into a laughing stock of slop, or, imho much less likely given the current state of AI (barring huge technological breakthrough and a couple of other assumptions) you were probably implying an intelligence explosion.

kertoip_1 3mo ago

The job of a programmer isn't to write code, but to automate things. Code itself doesn't have any value unless it solves some real problem not related to coding. So if the work of a programmer can be automated then this means that any work can be automated. So no, it's not about software engineering only.

emil-lp 3mo ago
> if the work of a programmer can be automated then this means that any work can be automated This is a very typical programmer thing to say.
- imdsm 3mo ago
  i think most things can be automated, but not everything i think AI will just replace jobs with new jobs, and humans will continue with open-ended goal setting, and probably jobs where being human is necessary: responsibility and authenticity, so judges, politics, leaders, chefs, etc. so in a way, the above commenter was correct, anything a programmer can automate, AI can do
  - hoppp 3mo ago
    It is a little bit suspicious that the big replacement didn't happen, yet. If it was so valuable then AI companies would directly create valuable things, not just sell API tokens. A proof like this is a good effort at trying to provide value directly, but still far away from real use. When OpenAI cures cancer I'll accept the fate, but till then I'm still seeing a lot of gambling going on.
    - deebosong 3mo ago
      Reminds me of crypto. A lotta tokens. A lotta gambling. Very little supplanting of crypto from fiat. Very few use-cases that revolutionize society, etc.
      - 3mo ago

petra 3mo ago

- the extent to which solutions could be implemented as text: not sure about that. AlphaFold is basically a mechanical/geometrical/Chemical problem. There are other scientific transformer based models. - the extent which solutions exist online - if you have a strong verification tool, you can generate examples, you can generate feedback, i think you could start with small/smaller prior art - the extent which solutions could be specified and checked - if you have a lot of priort art, maybe llm's can find the good "patterns" and compare against them, and at least get close to a good results - but you'd still need human verification.

nbardy 3mo ago

These are true and it does make theses fields the first to fall, but also the hype comes form the fact that it can escape these conditions as well: - generalize to non verifiable domains (https://arxiv.org/abs/2507.17746 https://arxiv.org/abs/2507.17746)[This is on going work but has had steady progress in many angles of attack) - Visual Reasoning is alive and well in video diffusion and image models(see recent works around using diffusion model priors as world model features for physics reasoning) - Prior art exists for anything humans can do online?(Is this one even a hold back?)