3 ms·

That time dilation feels a bit like what METR reported: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ https://metr.org/blog/2025-07-

• paulgerhardt 1y ago
  This was the study with a population size of 16?
  • sarchertech 1y ago
    16 developers doing hundreds of tasks. And the effect size was big enough for the result to be statistically significant.
  • fmbb 1y ago
    Do you have any better research we can read?
  • filoeleven 1y ago
    This was the only study that's been done so far?
  • deleted 1y ago
    [deleted]
• logicchains 1y ago
  It depends on how the AI is used; there's a huge difference in productivity between a structured workflow with repeated automated feedback to the AI, and just ad-hoc prompting. For instance, Gemini 2.5 coding up a spec-compliant HTTP2.0 server in two weeks: https://outervationai.substack.com/p/building-a-100-llm-written-standards https://outervationai.substack.com/p/building-a-100-llm-writ... . 15k lines of code and 30k lines of tests; no human coder could produce something like that so fast.
  • djeastm 1y ago
    Would that application not exist in the training data already?
    • logicchains 1y ago
      The are around three Golang HTTP2.0 servers on Github, but it wasn't just regurgitating the application from memory, as if it was then it would have been mostly correct first try, and it wouldn't have needed to spend 80%+ of the development time in a code-compile-test cycle of fixing bugs identified by integration tests and spec conformance tests.
      • codingdave 1y ago
        So you are saying that LLMs have no capacity to re-use existing work, and will just burn hours and money re-inventing wheels?
        • bavell 1y ago
          Welcome to The Future™!
        • simonw 1y ago
          If you tell it to reinvent the wheel, it will. If you tell it to use existing libraries (and they are in its training data) it will do that instead. If you tell it about libraries it hasn't encountered before it can use those instead.
  • akcih 1y ago
    It's cool that it made something that works, but that code is pretty bad. Trial and error is not the way to develop something like that.
  • fmbb 1y ago
    I would not say that is ”how the AI is used”. That problem space is one where humans have spent unusually much time defining a spec and writing a test suite. A million monkeys randomly typing could actually complete that task as well.
• layer8 1y ago
  That’s reminiscent of how when people are “in the flow”, they feel productive but also tend to not notice how quickly time passes.
• ares623 1y ago
  Reminds me of that quaalude scene from the movie The Wolf of Wall Street.