4 ms·
Interesting. When it comes to auditing, I think there is also another layer to be verified - the intent. We have tooling for static analysis and automated tests
by jolaflow 2mo ago
Interesting. When it comes to auditing, I think there is also another layer to be verified - the intent. We have tooling for static analysis and automated tests, but drift in intent or vision is hard to detect.
After leaving agents unsupervised for longer periods of time, the biggest questions tend to be "how did it arrive at this conclusion?" and "did we drift from the initial vision?". Typically these questions are hard to answer as you only review the final state of the workflow.
I recently built an issue tracker that does offer time-traveling as a core feature. Somewhat unexpectedly this turned out to provide a much sought for overview allowing for a better workflow audit, as detailed here:
https://dev.to/ljtn/vision-drift-addressing-the-next-problem-in-agentic-workflows-2gfb https://dev.to/ljtn/vision-drift-addressing-the-next-problem...
Would be interested to hear your thoughts. I guess vision drift would classify as something that is hard to reverse, because you often don't realize it has happened until much later.
- ithkuil 2mo agoI see vision drift happen routinely in most (all) human led projects.
- jolaflow 2mo agoThat is also very true, and I think time-travel capabilities would help detect that too.
- steve1977 2mo agoWould behaviour driven development be helpful here?
- jolaflow 2mo agoI think behavior driven development probably can make intent more explicit and standardized. However, I think they operate at different levels, or on a different axis if you will. BDD helps make each point in time easier to reason about wile being able to traverse the time axis helps understand how the workflow evolved.