Most of my work is data analysis, and it's the area I find LLMs are most difficult to use, and potentially dangerous! I really worry when I think of how less e

Most of my work is data analysis, and it's the area I find LLMs are most difficult to use, and potentially dangerous! I really worry when I think of how less experienced colleagues and Excel power-users might approach a pandas/matplotlib workflow with an LLM. Very normal example from the other week. I've got a big csv of housing data, poking around I spot a bunch of very weird things in the prices: cheap places where I'd expect high prices, a strange bicameral distribution. I spend a few hours thinking about it before taking it up with the colleague who'd provided the data. There's a bug in the scraper that's messed up a bunch of values! It'd be super-easy to naively run a simple LLM-driven analysis and miss this sort of thing. When you're "building things" with AI, feedback is immediate. You press a button, it doesn't work as expected, the charts you can quickly run up don't give the same signals.