4 ms·

There's no AI tool today that will resolve incidents to anyone's satisfaction. People need to be in the loop not only to take responsibility but to make sure th

There's no AI tool today that will resolve incidents to anyone's satisfaction. People need to be in the loop not only to take responsibility but to make sure the right actions are performed.

tptacek 1y ago
Nobody disputes this. Weakly posits a bright line between agents suggesting active steps and agents actually performing active steps. The problem is that during incident investigations, some active steps make a lot of sense for agents to perform, and others don't; the line isn't where she seems to claim it is.
- cmiles74 1y ago
  Understood. To your example about the logs, my concern would be be that the AI chooses the wrong thing to focus on and people decide there’s nothing of interest in the logs, thus overlooking a vital clue.
  - tptacek 1y ago
    You wouldn't anticipate using AI tools to one-shot complex incidents, just to rapidly surface competing hypotheses.
kookamamie 1y ago
Exactly. There seems to be this fantasy in which you can somehow string different kinds of agents together, one designing and one reviewing, and that finally producing something superior as output - I just don't buy that. Sounds like heuristics added on top of statistics, which is trying to remedy some root problem with another hack.
- rusticpenn 1y ago
  The whole field of metaheuristic algorithms rests on a similar idea. a lot of stupid "agents" finding a good solution. by metaheuristics i mean genetic algorithms, PSO, ACO etc.
- yunohn 1y ago
  Hmm, but this provably works right now though? All LLMs perform better with roleplay direction and focused scope. Using coding agents with plan then execute makes noticeable quality improvements.
  - kookamamie 1y ago
    Why isn't this the de facto then? Anyone packaging such commercial solutions?
    - yunohn 1y ago
      Most agents solutions have modes or roles already. There’s no standard, but this is already being used IRL. Heck, even system prompts are role play too.

There's no AI tool today that will resolve incidents to anyone's satisfaction. People need to be in the loop not only to take responsibility but to make sure the right actions are performed.

tptacek 1y ago

Nobody disputes this. Weakly posits a bright line between agents suggesting active steps and agents actually performing active steps. The problem is that during incident investigations, some active steps make a lot of sense for agents to perform, and others don't; the line isn't where she seems to claim it is.

cmiles74 1y ago
Understood. To your example about the logs, my concern would be be that the AI chooses the wrong thing to focus on and people decide there’s nothing of interest in the logs, thus overlooking a vital clue.
- tptacek 1y ago
  You wouldn't anticipate using AI tools to one-shot complex incidents, just to rapidly surface competing hypotheses.

kookamamie 1y ago

Exactly. There seems to be this fantasy in which you can somehow string different kinds of agents together, one designing and one reviewing, and that finally producing something superior as output - I just don't buy that. Sounds like heuristics added on top of statistics, which is trying to remedy some root problem with another hack.

rusticpenn 1y ago
The whole field of metaheuristic algorithms rests on a similar idea. a lot of stupid "agents" finding a good solution. by metaheuristics i mean genetic algorithms, PSO, ACO etc.
yunohn 1y ago
Hmm, but this provably works right now though? All LLMs perform better with roleplay direction and focused scope. Using coding agents with plan then execute makes noticeable quality improvements.
- kookamamie 1y ago
  Why isn't this the de facto then? Anyone packaging such commercial solutions?
  - yunohn 1y ago
    Most agents solutions have modes or roles already. There’s no standard, but this is already being used IRL. Heck, even system prompts are role play too.