3 ms·

A year ago I would have agreed wholeheartedly and I was a self confessed skeptic. Then Gemini got good (around 2.5?), like I-turned-my-head good. I started to

A year ago I would have agreed wholeheartedly and I was a self confessed skeptic. Then Gemini got good (around 2.5?), like I-turned-my-head good. I started to use it every week-ish, not to write code. But more like a tool (as you would a calculator). More recently Opus 4.5 was released and now I'm using it every day to assist in code. It is regularly helping me take tasks that would have taken 6-12 hours down to 15-30 minutes with some minor prompting and hand holding. I've not yet reached the point where I feel letting is loose and do the entire PR for me. But it's getting there.

spaceywilly 9mo ago
I would strongly recommend this podcast episode with Andrej Karpathy. I will poorly summarize it by saying his main point is that AI will spread like any other technology. It’s not going to be a sudden flash and everything is done by AI. It will be a slow rollout where each year it automates more and more manual work, until one day we realize it’s everywhere and has become indispensable. It sounds like what you are seeing lines up with his predictions. Each model generation is able to take on a little more of the responsibilities of a software engineer, but it’s not as if we suddenly don’t need the engineer anymore. https://www.dwarkesh.com/p/andrej-karpathy https://www.dwarkesh.com/p/andrej-karpathy
- sheeh 9mo ago
  AI first of all is not a technology. Can people get their words straight before typing?
  - shawabawa3 9mo ago
    Is LLM a technology? Are you complaining about the use of AI to mean LLM? Because I think that ship has sailed
- daxfohl 9mo ago
  Though I think it's a very steep sigmoid that we're still far on the bottom half of. For math it just did its first "almost independent" Erdos problem. In a couple months it'll probably do another, then maybe one each month for a while, then one morning we'll wake up and find whoom it solved 20 overnight and is spitting them out by the hour. For software it's been "curiosity ... curiosity ... curiosity ... occasionally useful assistant ... slightly more capable assistant" up to now, and it'll probably continue like that for a while. The inflection point will be when OpenAI/Anthropic/Google releases an e2e platform meant to be driven primarily by the product team, with engineering just being co-drivers. It probably starts out buggy and needing a lot of hand-holding (and grumbling) from engineering, but slowly but surely becomes more independently capable. Then at some point, product will become more confident in that platform than their own engineering team, and begin pushing out features based on that alone. Once that process starts (probably first at OpenAI/Anthropic/Google themselves, but spreading like wildfire across the industry), then it's just a matter of time until leadership declares that all feature development goes through that platform, and retains only as many engineers as is required to support the platform itself.
  - nullpoint420 9mo ago
    And then what? Am I supposed to be excited about this future?
    - daxfohl 9mo ago
      Hard to say. In business we'll still have to make hard decisions about unique situations, coordinate and align across teams and customers, deal with real world constraints and complex problems that aren't suitable to feed to an LLM and let it decide. In particular, deciding whether or not to trust an LLM with a task will itself always be a human decision. I think there will always be a place for analytical thinking in business even if LLMs do most of the actual engineering. If nothing else, the speed at which they work will require an increase in human analytical effort, to maximize their efficacy while maintaining safety and control. In the academic world, and math in particular, I'm not sure. In a way, you could say it doesn't change anything because proofs already "exist" long before we discover them, so AI just streamlines that discovery. Many mathematicians say that asking the right questions is more important than finding the answers. In which case, maybe math turns into something more akin to philosophy or even creative writing, and equivalently follows the direction that we set for AI in those fields. Which is, perhaps less than one would think: while AI can write a novel and it could even be pretty good, part of the value of a novel is the implicit bond between the author and the audience. "Meaning" has less value coming from a machine. And so maybe math continues that way, computers solving the problems but humans determining the meaning. Or maybe it all turns to shit and the sheer ubiquity of "masterpieces" of STEM/art everything renders all human endeavor pointless. Then the only thing that's left worth doing is for the greedy, the narcissists, and the power hungry to take the world back to the middle ages where knowledge and search for meaning take a back seat to tribalism and war mongering until the datacenters power needs destroy the planet. I'm hoping for something more like the former, but, it's anybody's guess.
    - suddenlybananas 9mo ago
      You have to remember that half these people think they are building god.
    - user34283 9mo ago
kstrauser 9mo ago
> I was a self confessed skeptic. I think that's the key. Healthy skepticism is always appropriate. It's the outright cynicism that gets me. "AI will never be able to [...]", when I've been sitting here at work doing 2/3rds of those supposedly impossible things. Flawlessly? No, of course not! But I don't do those things flawlessly on the first pass, either. Skepticism is good. I have no time or patience for cynics who dismiss the whole technology as impossible.
- sublinear 9mo ago
  I think the concern expressed as "impossible" is whether it can ever do those things "flawlessly" because that's what we actually need from its output. Otherwise a more experienced human is forced to do double work figuring out where it's wrong and then fixing it. This is not a lofty goal. It's what we always expect from a competent human regardless of the number of passes it takes them. This is not what we get from LLMs in the same amount time it takes a human to do the work unassisted. If it's impossible then there is no amount of time that would ever get this result from this type of AI. This matters because it means the human is forced to still be in the loop, not saving time, and forced to work harder than just not using it. I don't mean "flawless" in the sense that there cannot be improvements. I mean that the result should be what was expected for all possible inputs, and when inspected for bugs there are reasonable and subtle technical misunderstandings at the root of them (true bugs that are possibly undocumented or undefined behavior) and not a mess of additional linguistic ones or misuse. This is the stronger definition of what people mean by "hallucination", and it is absolutely not fixed and there has been no progress made on it either. No amount of prompting or prayer can work around it. This game of AI whack-a-mole really is a waste of time in so many cases. I would not bet on statistical models being anything more than what they are.
cameronh90 9mo ago
I'm now putting more queries into LLMs than I am into Google Search. I'm not sure how much of that is because Google Search has worsened versus LLMs having improved, but it's still a substantial shift in my day-to-day life. Something like finding the most appropriate sensor ICs to use for a particular use case requires so much less effort than it used to. I might have spent an entire day digging through data sheets before, and now I'll find what I need in a few minutes. It feels at least as revolutionary as when search replaced manually paging through web directories.
dmux 9mo ago
I feel like I'm living in a totally different world or I'm being gaslit by LLMs when I read stuff like this and other similar comments in this thread. Do you mind mentioning _what_ language / tech stack you're in? At my current job, we have a large Ruby on Rails codebase and just this week Gemini 2.5 and 3 struggled to even identify what classes inherited from another class.

spaceywilly 9mo ago

I would strongly recommend this podcast episode with Andrej Karpathy. I will poorly summarize it by saying his main point is that AI will spread like any other technology. It’s not going to be a sudden flash and everything is done by AI. It will be a slow rollout where each year it automates more and more manual work, until one day we realize it’s everywhere and has become indispensable. It sounds like what you are seeing lines up with his predictions. Each model generation is able to take on a little more of the responsibilities of a software engineer, but it’s not as if we suddenly don’t need the engineer anymore. https://www.dwarkesh.com/p/andrej-karpathy https://www.dwarkesh.com/p/andrej-karpathy

sheeh 9mo ago
AI first of all is not a technology. Can people get their words straight before typing?
- shawabawa3 9mo ago
  Is LLM a technology? Are you complaining about the use of AI to mean LLM? Because I think that ship has sailed
daxfohl 9mo ago
Though I think it's a very steep sigmoid that we're still far on the bottom half of. For math it just did its first "almost independent" Erdos problem. In a couple months it'll probably do another, then maybe one each month for a while, then one morning we'll wake up and find whoom it solved 20 overnight and is spitting them out by the hour. For software it's been "curiosity ... curiosity ... curiosity ... occasionally useful assistant ... slightly more capable assistant" up to now, and it'll probably continue like that for a while. The inflection point will be when OpenAI/Anthropic/Google releases an e2e platform meant to be driven primarily by the product team, with engineering just being co-drivers. It probably starts out buggy and needing a lot of hand-holding (and grumbling) from engineering, but slowly but surely becomes more independently capable. Then at some point, product will become more confident in that platform than their own engineering team, and begin pushing out features based on that alone. Once that process starts (probably first at OpenAI/Anthropic/Google themselves, but spreading like wildfire across the industry), then it's just a matter of time until leadership declares that all feature development goes through that platform, and retains only as many engineers as is required to support the platform itself.
- nullpoint420 9mo ago
  And then what? Am I supposed to be excited about this future?
  - daxfohl 9mo ago
    Hard to say. In business we'll still have to make hard decisions about unique situations, coordinate and align across teams and customers, deal with real world constraints and complex problems that aren't suitable to feed to an LLM and let it decide. In particular, deciding whether or not to trust an LLM with a task will itself always be a human decision. I think there will always be a place for analytical thinking in business even if LLMs do most of the actual engineering. If nothing else, the speed at which they work will require an increase in human analytical effort, to maximize their efficacy while maintaining safety and control. In the academic world, and math in particular, I'm not sure. In a way, you could say it doesn't change anything because proofs already "exist" long before we discover them, so AI just streamlines that discovery. Many mathematicians say that asking the right questions is more important than finding the answers. In which case, maybe math turns into something more akin to philosophy or even creative writing, and equivalently follows the direction that we set for AI in those fields. Which is, perhaps less than one would think: while AI can write a novel and it could even be pretty good, part of the value of a novel is the implicit bond between the author and the audience. "Meaning" has less value coming from a machine. And so maybe math continues that way, computers solving the problems but humans determining the meaning. Or maybe it all turns to shit and the sheer ubiquity of "masterpieces" of STEM/art everything renders all human endeavor pointless. Then the only thing that's left worth doing is for the greedy, the narcissists, and the power hungry to take the world back to the middle ages where knowledge and search for meaning take a back seat to tribalism and war mongering until the datacenters power needs destroy the planet. I'm hoping for something more like the former, but, it's anybody's guess.
  - suddenlybananas 9mo ago
    You have to remember that half these people think they are building god.
  - user34283 9mo ago

kstrauser 9mo ago

> I was a self confessed skeptic. I think that's the key. Healthy skepticism is always appropriate. It's the outright cynicism that gets me. "AI will never be able to [...]", when I've been sitting here at work doing 2/3rds of those supposedly impossible things. Flawlessly? No, of course not! But I don't do those things flawlessly on the first pass, either. Skepticism is good. I have no time or patience for cynics who dismiss the whole technology as impossible.

sublinear 9mo ago
I think the concern expressed as "impossible" is whether it can ever do those things "flawlessly" because that's what we actually need from its output. Otherwise a more experienced human is forced to do double work figuring out where it's wrong and then fixing it. This is not a lofty goal. It's what we always expect from a competent human regardless of the number of passes it takes them. This is not what we get from LLMs in the same amount time it takes a human to do the work unassisted. If it's impossible then there is no amount of time that would ever get this result from this type of AI. This matters because it means the human is forced to still be in the loop, not saving time, and forced to work harder than just not using it. I don't mean "flawless" in the sense that there cannot be improvements. I mean that the result should be what was expected for all possible inputs, and when inspected for bugs there are reasonable and subtle technical misunderstandings at the root of them (true bugs that are possibly undocumented or undefined behavior) and not a mess of additional linguistic ones or misuse. This is the stronger definition of what people mean by "hallucination", and it is absolutely not fixed and there has been no progress made on it either. No amount of prompting or prayer can work around it. This game of AI whack-a-mole really is a waste of time in so many cases. I would not bet on statistical models being anything more than what they are.

cameronh90 9mo ago

I'm now putting more queries into LLMs than I am into Google Search. I'm not sure how much of that is because Google Search has worsened versus LLMs having improved, but it's still a substantial shift in my day-to-day life. Something like finding the most appropriate sensor ICs to use for a particular use case requires so much less effort than it used to. I might have spent an entire day digging through data sheets before, and now I'll find what I need in a few minutes. It feels at least as revolutionary as when search replaced manually paging through web directories.

dmux 9mo ago

I feel like I'm living in a totally different world or I'm being gaslit by LLMs when I read stuff like this and other similar comments in this thread. Do you mind mentioning _what_ language / tech stack you're in? At my current job, we have a large Ruby on Rails codebase and just this week Gemini 2.5 and 3 struggled to even identify what classes inherited from another class.