4 ms·

Have it run fuzz and test suites. Get with it man. Most of my LLM projects have massive test suites that do a far better job then I ever would have.

• jbstack 3mo ago
  So you're fine with your code having unintended behaviour, as long as that unintended behaviour passes a test that the agent wrote? My preferred approach is to read and understand everything the LLM produces AND have it create test suites (which I also read and understand). The LLM can help you with that too - just have it breakdown and explain the code at each iteration.
  • energy123 3mo ago
    If it's code for a pacemaker, it should be reviewed by multiple humans. If it's code for an unimportant side project, I will never read it. Between those extremes are shades of grey. Part of our job in this new era is to understand the worst-case consequences of a bug given how the code interacts with the world, then allocate our effort based on that understanding. This can only be done on a case by case basis.
    • hansvm 3mo ago
      If you don't, at a bare minimum, read the code once though, you don't know how it interacts with the world.
      • energy123 3mo ago
        [dead]
• rimliu 3mo ago
  Did you look at those test suites? Sometimes they test nothing.
  • esikich 3mo ago
    Yes, but not all - it has been a long time in my experience that frontier LLMs have written nonsense tests.