3 ms·
Posts like this kinda-sorta grind my gears, like... I get it, but also... accounting, like many real world tasks, is fundamentally a chain of precise and constr
by neom 1y ago
Posts like this kinda-sorta grind my gears, like... I get it, but also... accounting, like many real world tasks, is fundamentally a chain of precise and constrained and auditable operations. Humans approach these tasks through structured processes... we use roles, and we have checkpoints precisely because complexity compounds quickly and becomes unmanageable if tackled as one giant block. Expecting a single AI model to handle an e2e workflow seamlessly without similarly explicit segmentation and oversight misunderstands not only the model but also the nature of the workflow itself.
I wanna see someone take long horizon tasks, recongnize they're not linear, and design and test a better system: structured orchestration, transparent auditability, and disciplined modularity, I think that would be considerably more interesting personally.
- andy99 1y agoIt's a useless benchmark if everyone aces it. If some models do better than others and none saturate it, then is has some value, no? Permitting comparison is the point.
- neom 1y agoI agree, hence the heavy couching, so to your point I'm def just ranting a bit, because: I just think it would be more valuable to see some kind of MoA, I guess what I'm talking about is a bit of a different measurement, thinking in terms of economic outlook and understanding where we are and what can be done. I suspect more of this will shape our ability to understand how frontier models will impact the economy. Maybe I should just do my own evaluation, heh. :) Edit: although to argue against myself, I suppose once a model can one-shot this stuff, my MoA comments become moot.