6 ms·

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

stogot 2mo ago
What tooling makes this go?
- colordrops 2mo ago
  The ability to run queries and get the metadata about the run, e.g. length of run, execution plan, engine, engine params, etc
- madeofpalk 2mo ago
  Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! I’ve been working on UI component improvements and it was doing a lousy job until i specifically told it to test in a headless browser to validate it works. I think somewhere in an AGENTS.md i have an instruction to “don’t state your guesses as fact - validate findings and results”.
  - Tade0 2mo ago
    It bothers me that you have to explicitly state this to the agent. Makes me think what else is missing from that file which also needs to be explicitly stated, but I don't know what don't know. "Do a good job"?
    - Filligree 2mo ago
      Will, it depends on the AI. Anthropic used to have a lecture-length system prompt for their models to explain this stuff—part of the secret sauce for Claude Code—and famously found that the 5 series models no longer need it. As usual, if you use anything but the best model available I’m going to state that the better ones do better. If you do use the best model available, then I’ll just mention that Fable still has limits and still needs some guidance. One thing it does not do is deliberately build tests which test nothing at all, or which restate the code under test. I mention this because certain other models absolutely would.
    - Gareth321 2mo ago
      This is where the harness shines. It's why products like Cursor have been growing like crazy. If you drop an extremely intelligent person into a problem they've never seen before, the initial context and knowledge-sharing just to get them up to speed is immense. They'll make a lot of mistakes until they're ready to fix your problem. The harness can leapfrog most of that. There is a booming industry in harnesses with no saturation in sight. We're not even close to maximising the utility of current models, let alone the amazing improvements which are happening week to week.
      - QuercusMax 2mo ago
        How does the cursor harness help here compared to something more basic like pi coder? I'm talking specifics.
    - Leynos 2mo ago
      If you ask Fable or 5.6 Sol to improve performance, it will generally know to build a benchmark and create a test corpus. I'm not sure where the contrary suggestion is coming from.
    - swat535 2mo ago
      Additionally, if you are asking agents to write test, be very careful. They will write tautology tests. They will mock things to no end. They will flat out REMOVE assertions (saying it's not needed). They can also write test to assert the wrong result. You have to always review it, it's exhausting.
      - Tade0 2mo ago
        That is my experience as well. I write tests by hand because they really still can't do it properly. And I think I know why - neither do most people.
      - QuercusMax 2mo ago
        This was my experience with some of the stupider models, but I haven't had Claude do this type of thing since I started using it for work about 6mo ago. Claude will do some boneheaded things for sure, but it's pretty good about writing tests that are useful, and not removing or modifying tests just because they're in the way. Claude is pretty bad about assuming that it couldn't have broken a test it didn't know about, as it has often told me "this is already broken on main" which is definitely NOT true.
    - derefr 2mo ago
      An agent doesn’t know what “job” it’s being asked to do until you tell it. Imagine you’re blind and deaf and have temporary retrograde amnesia. You “wake up” one moment with a memory of some words in your head like “what is the bug?” …but you don’t recall the context of that question, and nor can you look/listen around to observe the context. So you don’t know whether you’re e.g. at the office, in front of your computer, in the middle of doing some pair-programming (where, yes, you’d in investigate the bug thoroughly with tools), vs. having a conversation with a colleague over lunch (where the expectation is for you to tap into your knowledge + intuitions to either guess or say you don’t know — not to pull out your laptop. That’s what it’s like to be one of these LLMs being prompted by some agent harness. Unless the harness injects the proper context into its “recent memory”, it just doesn’t know.
    - ClikeX 2mo ago
      Claude Code just does this for me. But it might be one of their Superpower skills that I added.
  - altmanaltman 2mo ago
    If the agent can write the tests its supposed to pass and we are worried that agent produces quite a lot of slop (which is why we are doing the tests), then what is the defense against test slop? Test the tests?
    - KptMarchewa 2mo ago
      Existing tests.
    - Filligree 2mo ago
      Read the tests. Doing a pass where you just ask the AI to sanity-check the existing tests (against rules like “test against the spec, not the implementation” can also help.
    - user43928 2mo ago
      Manual testing. I work on user facing applications, and since the models do not have good taste, testing the UX is essential. If you spot a bug, usually the model will attempt to reproduce it in a new test case that does cover the actual issue.
    - Leynos 2mo ago
      1st line of defence, use something like ponytail to enforce brevity. Use property testing and behavioural testing on top of unit testing. Enforce readability standards so you will be able to understand the tests. 2nd line, code review. Do the first pass with an agent, ask it to bounce back vacuous or tautological tests. Ask it to verify that the tests verify what they claim to. Then read them yourself. 3rd line, mutation testing. If the tests don't actually catch broken code, kill the mutants.
    - mikeydiamonds 2mo ago
      [dead]
  - KptMarchewa 2mo ago
    In this particular case tests won't help you, without the actual data. But overall you're right.
    - madeofpalk 2mo ago
      "tests" here could mean have a benchmark the agent can run to determinately evaluate quality. I don't strictly mean junit unit tests.
  - troupo 2mo ago
    > Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! Tests onli validate the presence of bugs, not their abscence (Djikstra). I'll also add that tests look at outputs and don't care how those outputs are derived. E.g. code filtering the entire db in memory will be fine in tests.
    - bluGill 2mo ago
      "Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth Tests prove the things you thought of worked, and it often isn't hard to find the likely edge cases such that you have reasonable confidence everything works. You will be wrong from time to time, but not that often. You can prove code correct, but if the proof is wrong (common when a human is doing it), or the spec is wrong (most people have no clue how to write a comprehensive spec) it can still be wrong.
    - anthonyrstevens 2mo ago
      Haven't you ever written tests that assert that certain things happen or do not happen along the way? We use mocks all the time to, say, make sure that the code doesn't load the entire db in memory. Then it's there as a regression if somebody down the line wants to do that exact thing for some reason.
      - troupo 2mo ago
        Unfortunately, with LLMs you have to write these tests for every little thing. And then there's the endless code duplication, reinventing of existing code and libraries etc.
  - tester756 2mo ago
    >Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! Tests =/= TDD
- gWPVhyxPHqvk 2mo ago
  The same tooling that you'd use. AI is not magic, it cannot know exactly how a program will run under your exact circumstances and data. Therefore, if it can use a profiler, test suite or application, or be able to read logs, just like you would, it then can make changes, evaluate them, and iterate. Personally, I've created test MCP servers that will perform actions like submit orders, have the AI capture and analyze PerfViews and Concurrency Visualizer, write out tracing data to logs... all the same stuff I was doing by hand for years.
- jvanderbot 2mo ago
  I'm convinced this is why there's a bifurcation in old-hand/new-hand results: the old hands include the steps they'd do and the tools they'd use, and the new-hands never learned those tools. That'll get RL-d in, eventually.
- nextaccountic 2mo ago
  In this case, you need benchmarks, in addition to regular tests. You also need concise instructions (or better yet, following your language / ecosystem norms if applicable) to how to run this stuff
jfrbfbreudh 2mo ago
Built exactly this for work a few months ago. Claude has a read-only account to the dev DB and a bunch of python postgres tools, I point it at queries/views and ask it optimize, and it has to: 1) verify identical query results 2) run repeatedly to get average, worst, best, etc duration of runs Sped up so many legacy things that none of us were ever going to bother with.
- cyanydeez 2mo ago
  the point is more: are people going to flood the zone with infinite bikeshedding that drowns out the real gains
  - xXSLAYERXx 2mo ago
    [flagged]
    - cyanydeez 2mo ago
      yeah, but not at scale; you can now spin up 20 bike shedding agents and flood anyone of your employees/coworkers.
  - dpark 2mo ago
    What is bikeshedding supposed to mean here?
    - RGS1811 2mo ago
      It comes from C. Northcote Parkinson's work, where he calls it "The Law of Triviality". The idea is that people clog up decision-making by introducing a swarm of issues that have little relative impact or importance for the project at hand, leading to a massive misallocation of attention. His example is a $10 million construction project for a Nuclear Reactor where as much time is spent debating the materials to be used for a $3000 bike shed as on the actual reactor buildout. The law of triviality says "the time spent on any item of the agenda will be in inverse proportion to the sum [of money] involved." Source: C. Northcote Parkinson, "Parkinson's Law and Other Studies in Administration" (1957)
KptMarchewa 2mo ago
Exactly. In the above case, AI should be able to verify it's own hypothesis by having access to the database's table statistics and planner.
cenamus 2mo ago
But them it's more like AI assisted performance fuzzing? Just throw stuff against some measuring harness and see what sticks?
- blackoil 2mo ago
  That's how anyone will do it, read code, build hypothesis, test. It may not be very good with ideas compared to human but it will be cheaper.
- ygjb 2mo ago
  Yes, sort of. Not that there is anything wrong with that. Fuzzing yields tons of trash results, the occasional fleck of gold and the even more rare nugget. Then you analyze how it found the flakes and nuggets update your fuzzing harness. Every so often you get to write a paper or give a talk about a breakthrough to share meaningful improvements. Current AI isn't super effective at making the breakthroughs, but it sure is effective at democratizing the ones you can point it at.
  - inigyou 2mo ago
    What does that mean? Democratising the breakthroughs you can point LLMs at?
    - ygjb 2mo ago
      Until a couple of years ago you needed to be able to read research papers and code to build fuzzing harnesses and triage the findings to find vulns. Now you can point Claude, codex, Kimi, or even some smaller open models that run local and get a servable fuzzing loop in a couple of hours. That's democratization - shifting power and skills away from those with the wealth to purchase vulns and run offensive security teams. It's still not simple, but it's one of the reasons bigger vendors want to lock down the capabilities of models that present a threat to the wealthy and powerful.
      - inigyou 2mo ago
spiderfarmer 2mo ago
That’s how I do it in my Laravel apps. I simply ask it to hypothesize, benchmark, apply and verify. Laravel has great tools for that.
moron4hire 2mo ago
[flagged]
- barrkel 2mo ago
  If you run it in a sandbox with no internet access and a read-only role login for the database, and you supervise it, how big is the risk do you reckon? Of course there's an issue if the data in the database is contractually restricted and you don't have a zero retention endpoint for inference, but the former usually comes with the budget to fund the latter.
- CuriouslyC 2mo ago
  You think they're talking about giving them access to a live prod db/app? You know offline copies are a thing right?
  - moron4hire 2mo ago
    Nothing to do with the database itself. It's the tooling necessary to give the agentic harness access to the database that is the problem.
mettamage 2mo ago
Just like humans, AI needs a "reality" to check things against. If we don't provide that in an automated way, somehow, it will always make mistakes and we'd have to point it out. Since, at that point, we're just mediating with what reality gives us and what an AI should do with it. AI doesn't have enough senses yet. It's trapped in a box.
- inigyou 2mo ago
  The AI box experiment has been resolved. We let the AI out of the box if it promises to summarise and desummarise emails.
- deleted 2mo ago
  [deleted]
- shuwix 2mo ago
  There's an old programming rule since 90's: By fixing a bug, 3 new bugs appear. I can imagine the quality of bug fixing by unsupervised (we save money by layoffs) halucinating AI at current state of development.
  - zhengyi13 2mo ago
    The way I learned it was: "99 bugs in the code, 99 bugs in the code... Fix one bug, compile it again, 101 bugs in the code!" (To the tune of 99 Bottles Of Beer)
    - shuwix 2mo ago
      Mayne the 99 bottles of beer explains it. But hey, one can't survive 1 week of step-by-step debugging without 99 bottles of beer.
- harrall 2mo ago
  I’d say it’s not so much senses but defining what success means. If you want to learn darts or perfect parallel parking for example, most of the increase in accuracy comes from simply but very deliberately pointing out to your brain where you wanted to land versus where it did land. We like to usually just do things and hope for the best. Defining success is not something we automatically do and naturally we don’t do it with AI either.
dominotw 2mo ago
Very few problems really have "all the tooling to verify its hypotheses" though. even if you want to construct such an harness. Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'
- simonw 2mo ago
  We don't. You could freeze model development today and we would be able to use our current models to effectively optimize existing code for years to come.
- fn-mote 2mo ago
  > Very few problems really have "all the tooling to verify its hypotheses" This is such a blanket dismissal that I can’t agree or disagree. Maybe very few of YOUR problems are this way. At least mention some problem domains. Recent experiences: compiler-related (helpful), UI-related (agree it isn’t testable but the design iteration is quick, easy, and correct), debugging technical configuration problems (useless; I basically have to solve each problem myself before the LLM recognizes it).
  - dominotw 2mo ago
    > This is such a blanket dismissal that I can’t agree or disagree. > Maybe very few of YOUR problems are this way. At least mention some problem domains. i did in second part of my comment. why do you think billions are being poured into ai if ai can already do verifable tasks.
- dragonwriter 2mo ago
  > Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses' “Good” isn’t “perfect” and even if it was, the ability to produce perfect output with all the tooling to verify its hypotheses could still be improved, in time and token efficiency, by better models producing fewer spurious hypotheses, rejecting those it does generate faster, and taking fewer unnecessary steps in confirming its good hypotheses.
moomin 2mo ago
I’ve had a lot of success with this. Give it a good enough harness, leave it for a few days and see what it finds that actually holds up.

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

stogot 2mo ago

What tooling makes this go?

colordrops 2mo ago
The ability to run queries and get the metadata about the run, e.g. length of run, execution plan, engine, engine params, etc
madeofpalk 2mo ago
Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! I’ve been working on UI component improvements and it was doing a lousy job until i specifically told it to test in a headless browser to validate it works. I think somewhere in an AGENTS.md i have an instruction to “don’t state your guesses as fact - validate findings and results”.
- Tade0 2mo ago
  It bothers me that you have to explicitly state this to the agent. Makes me think what else is missing from that file which also needs to be explicitly stated, but I don't know what don't know. "Do a good job"?
  - Filligree 2mo ago
    Will, it depends on the AI. Anthropic used to have a lecture-length system prompt for their models to explain this stuff—part of the secret sauce for Claude Code—and famously found that the 5 series models no longer need it. As usual, if you use anything but the best model available I’m going to state that the better ones do better. If you do use the best model available, then I’ll just mention that Fable still has limits and still needs some guidance. One thing it does not do is deliberately build tests which test nothing at all, or which restate the code under test. I mention this because certain other models absolutely would.
  - Gareth321 2mo ago
    This is where the harness shines. It's why products like Cursor have been growing like crazy. If you drop an extremely intelligent person into a problem they've never seen before, the initial context and knowledge-sharing just to get them up to speed is immense. They'll make a lot of mistakes until they're ready to fix your problem. The harness can leapfrog most of that. There is a booming industry in harnesses with no saturation in sight. We're not even close to maximising the utility of current models, let alone the amazing improvements which are happening week to week.
    - QuercusMax 2mo ago
      How does the cursor harness help here compared to something more basic like pi coder? I'm talking specifics.
  - Leynos 2mo ago
    If you ask Fable or 5.6 Sol to improve performance, it will generally know to build a benchmark and create a test corpus. I'm not sure where the contrary suggestion is coming from.
  - swat535 2mo ago
    Additionally, if you are asking agents to write test, be very careful. They will write tautology tests. They will mock things to no end. They will flat out REMOVE assertions (saying it's not needed). They can also write test to assert the wrong result. You have to always review it, it's exhausting.
    - Tade0 2mo ago
      That is my experience as well. I write tests by hand because they really still can't do it properly. And I think I know why - neither do most people.
    - QuercusMax 2mo ago
      This was my experience with some of the stupider models, but I haven't had Claude do this type of thing since I started using it for work about 6mo ago. Claude will do some boneheaded things for sure, but it's pretty good about writing tests that are useful, and not removing or modifying tests just because they're in the way. Claude is pretty bad about assuming that it couldn't have broken a test it didn't know about, as it has often told me "this is already broken on main" which is definitely NOT true.
  - derefr 2mo ago
    An agent doesn’t know what “job” it’s being asked to do until you tell it. Imagine you’re blind and deaf and have temporary retrograde amnesia. You “wake up” one moment with a memory of some words in your head like “what is the bug?” …but you don’t recall the context of that question, and nor can you look/listen around to observe the context. So you don’t know whether you’re e.g. at the office, in front of your computer, in the middle of doing some pair-programming (where, yes, you’d in investigate the bug thoroughly with tools), vs. having a conversation with a colleague over lunch (where the expectation is for you to tap into your knowledge + intuitions to either guess or say you don’t know — not to pull out your laptop. That’s what it’s like to be one of these LLMs being prompted by some agent harness. Unless the harness injects the proper context into its “recent memory”, it just doesn’t know.
  - ClikeX 2mo ago
    Claude Code just does this for me. But it might be one of their Superpower skills that I added.
- altmanaltman 2mo ago
  If the agent can write the tests its supposed to pass and we are worried that agent produces quite a lot of slop (which is why we are doing the tests), then what is the defense against test slop? Test the tests?
  - KptMarchewa 2mo ago
    Existing tests.
  - Filligree 2mo ago
    Read the tests. Doing a pass where you just ask the AI to sanity-check the existing tests (against rules like “test against the spec, not the implementation” can also help.
  - user43928 2mo ago
    Manual testing. I work on user facing applications, and since the models do not have good taste, testing the UX is essential. If you spot a bug, usually the model will attempt to reproduce it in a new test case that does cover the actual issue.
  - Leynos 2mo ago
    1st line of defence, use something like ponytail to enforce brevity. Use property testing and behavioural testing on top of unit testing. Enforce readability standards so you will be able to understand the tests. 2nd line, code review. Do the first pass with an agent, ask it to bounce back vacuous or tautological tests. Ask it to verify that the tests verify what they claim to. Then read them yourself. 3rd line, mutation testing. If the tests don't actually catch broken code, kill the mutants.
  - mikeydiamonds 2mo ago
    [dead]
- KptMarchewa 2mo ago
  In this particular case tests won't help you, without the actual data. But overall you're right.
  - madeofpalk 2mo ago
    "tests" here could mean have a benchmark the agent can run to determinately evaluate quality. I don't strictly mean junit unit tests.
- troupo 2mo ago
  > Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! Tests onli validate the presence of bugs, not their abscence (Djikstra). I'll also add that tests look at outputs and don't care how those outputs are derived. E.g. code filtering the entire db in memory will be fine in tests.
  - bluGill 2mo ago
    "Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth Tests prove the things you thought of worked, and it often isn't hard to find the likely edge cases such that you have reasonable confidence everything works. You will be wrong from time to time, but not that often. You can prove code correct, but if the proof is wrong (common when a human is doing it), or the spec is wrong (most people have no clue how to write a comprehensive spec) it can still be wrong.
  - anthonyrstevens 2mo ago
    Haven't you ever written tests that assert that certain things happen or do not happen along the way? We use mocks all the time to, say, make sure that the code doesn't load the entire db in memory. Then it's there as a regression if somebody down the line wants to do that exact thing for some reason.
    - troupo 2mo ago
      Unfortunately, with LLMs you have to write these tests for every little thing. And then there's the endless code duplication, reinventing of existing code and libraries etc.
- tester756 2mo ago
  >Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD! Tests =/= TDD
gWPVhyxPHqvk 2mo ago
The same tooling that you'd use. AI is not magic, it cannot know exactly how a program will run under your exact circumstances and data. Therefore, if it can use a profiler, test suite or application, or be able to read logs, just like you would, it then can make changes, evaluate them, and iterate. Personally, I've created test MCP servers that will perform actions like submit orders, have the AI capture and analyze PerfViews and Concurrency Visualizer, write out tracing data to logs... all the same stuff I was doing by hand for years.
jvanderbot 2mo ago
I'm convinced this is why there's a bifurcation in old-hand/new-hand results: the old hands include the steps they'd do and the tools they'd use, and the new-hands never learned those tools. That'll get RL-d in, eventually.
nextaccountic 2mo ago
In this case, you need benchmarks, in addition to regular tests. You also need concise instructions (or better yet, following your language / ecosystem norms if applicable) to how to run this stuff

jfrbfbreudh 2mo ago

Built exactly this for work a few months ago. Claude has a read-only account to the dev DB and a bunch of python postgres tools, I point it at queries/views and ask it optimize, and it has to: 1) verify identical query results 2) run repeatedly to get average, worst, best, etc duration of runs Sped up so many legacy things that none of us were ever going to bother with.

cyanydeez 2mo ago
the point is more: are people going to flood the zone with infinite bikeshedding that drowns out the real gains
- xXSLAYERXx 2mo ago
  [flagged]
  - cyanydeez 2mo ago
    yeah, but not at scale; you can now spin up 20 bike shedding agents and flood anyone of your employees/coworkers.
- dpark 2mo ago
  What is bikeshedding supposed to mean here?
  - RGS1811 2mo ago
    It comes from C. Northcote Parkinson's work, where he calls it "The Law of Triviality". The idea is that people clog up decision-making by introducing a swarm of issues that have little relative impact or importance for the project at hand, leading to a massive misallocation of attention. His example is a $10 million construction project for a Nuclear Reactor where as much time is spent debating the materials to be used for a $3000 bike shed as on the actual reactor buildout. The law of triviality says "the time spent on any item of the agenda will be in inverse proportion to the sum [of money] involved." Source: C. Northcote Parkinson, "Parkinson's Law and Other Studies in Administration" (1957)

KptMarchewa 2mo ago

Exactly. In the above case, AI should be able to verify it's own hypothesis by having access to the database's table statistics and planner.

cenamus 2mo ago

But them it's more like AI assisted performance fuzzing? Just throw stuff against some measuring harness and see what sticks?

blackoil 2mo ago
That's how anyone will do it, read code, build hypothesis, test. It may not be very good with ideas compared to human but it will be cheaper.
ygjb 2mo ago
Yes, sort of. Not that there is anything wrong with that. Fuzzing yields tons of trash results, the occasional fleck of gold and the even more rare nugget. Then you analyze how it found the flakes and nuggets update your fuzzing harness. Every so often you get to write a paper or give a talk about a breakthrough to share meaningful improvements. Current AI isn't super effective at making the breakthroughs, but it sure is effective at democratizing the ones you can point it at.
- inigyou 2mo ago
  What does that mean? Democratising the breakthroughs you can point LLMs at?
  - ygjb 2mo ago
    Until a couple of years ago you needed to be able to read research papers and code to build fuzzing harnesses and triage the findings to find vulns. Now you can point Claude, codex, Kimi, or even some smaller open models that run local and get a servable fuzzing loop in a couple of hours. That's democratization - shifting power and skills away from those with the wealth to purchase vulns and run offensive security teams. It's still not simple, but it's one of the reasons bigger vendors want to lock down the capabilities of models that present a threat to the wealthy and powerful.
    - inigyou 2mo ago

spiderfarmer 2mo ago

That’s how I do it in my Laravel apps. I simply ask it to hypothesize, benchmark, apply and verify. Laravel has great tools for that.

moron4hire 2mo ago

[flagged]

barrkel 2mo ago
If you run it in a sandbox with no internet access and a read-only role login for the database, and you supervise it, how big is the risk do you reckon? Of course there's an issue if the data in the database is contractually restricted and you don't have a zero retention endpoint for inference, but the former usually comes with the budget to fund the latter.
CuriouslyC 2mo ago
You think they're talking about giving them access to a live prod db/app? You know offline copies are a thing right?
- moron4hire 2mo ago
  Nothing to do with the database itself. It's the tooling necessary to give the agentic harness access to the database that is the problem.

mettamage 2mo ago

Just like humans, AI needs a "reality" to check things against. If we don't provide that in an automated way, somehow, it will always make mistakes and we'd have to point it out. Since, at that point, we're just mediating with what reality gives us and what an AI should do with it. AI doesn't have enough senses yet. It's trapped in a box.

inigyou 2mo ago
The AI box experiment has been resolved. We let the AI out of the box if it promises to summarise and desummarise emails.
deleted 2mo ago
[deleted]
shuwix 2mo ago
There's an old programming rule since 90's: By fixing a bug, 3 new bugs appear. I can imagine the quality of bug fixing by unsupervised (we save money by layoffs) halucinating AI at current state of development.
- zhengyi13 2mo ago
  The way I learned it was: "99 bugs in the code, 99 bugs in the code... Fix one bug, compile it again, 101 bugs in the code!" (To the tune of 99 Bottles Of Beer)
  - shuwix 2mo ago
    Mayne the 99 bottles of beer explains it. But hey, one can't survive 1 week of step-by-step debugging without 99 bottles of beer.
harrall 2mo ago
I’d say it’s not so much senses but defining what success means. If you want to learn darts or perfect parallel parking for example, most of the increase in accuracy comes from simply but very deliberately pointing out to your brain where you wanted to land versus where it did land. We like to usually just do things and hope for the best. Defining success is not something we automatically do and naturally we don’t do it with AI either.

dominotw 2mo ago

Very few problems really have "all the tooling to verify its hypotheses" though. even if you want to construct such an harness. Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'

simonw 2mo ago
We don't. You could freeze model development today and we would be able to use our current models to effectively optimize existing code for years to come.
fn-mote 2mo ago
> Very few problems really have "all the tooling to verify its hypotheses" This is such a blanket dismissal that I can’t agree or disagree. Maybe very few of YOUR problems are this way. At least mention some problem domains. Recent experiences: compiler-related (helpful), UI-related (agree it isn’t testable but the design iteration is quick, easy, and correct), debugging technical configuration problems (useless; I basically have to solve each problem myself before the LLM recognizes it).
- dominotw 2mo ago
  > This is such a blanket dismissal that I can’t agree or disagree. > Maybe very few of YOUR problems are this way. At least mention some problem domains. i did in second part of my comment. why do you think billions are being poured into ai if ai can already do verifable tasks.
dragonwriter 2mo ago
> Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses' “Good” isn’t “perfect” and even if it was, the ability to produce perfect output with all the tooling to verify its hypotheses could still be improved, in time and token efficiency, by better models producing fewer spurious hypotheses, rejecting those it does generate faster, and taking fewer unnecessary steps in confirming its good hypotheses.

moomin 2mo ago

I’ve had a lot of success with this. Give it a good enough harness, leave it for a few days and see what it finds that actually holds up.