3 ms·
Humans aren't very diligent in the long term. If an LLM does something correctly enough times in a row (or close enough), humans are likely to stop checking its
by phire 8mo ago
Humans aren't very diligent in the long term. If an LLM does something correctly enough times in a row (or close enough), humans are likely to stop checking its work throughly enough.
This isn't exactly a new problem we do it with any bit of new software/hardware, not just LLMs. We check its work when it's new, and then tend to trust it over time as it proves itself.
But it seems to be hitting us worse with LLMs, as they are less consistent than previous software. And LLM hallucinations are partially dangerous, because they are often plausible enough to pass the sniff test. We just aren't used to handling something this unpredictable.
- potatoman22 8mo agohttps://en.wikipedia.org/wiki/Automation_bias https://en.wikipedia.org/wiki/Automation_bias
- Waterluvian 8mo agoIt’s a core part of the job and there’s simply no excuse for complacency.
- pixl97 8mo agoThe words on the page are just a medium to sell ads. If shit gets ad views then producing shit is part of the job... unless you're the one stepping up to cut the checks.
- Marsymars 8mo agoArs also sells ad-free subscriptions.
- jatora 8mo agoThere's not a human alive that isnt complacent in many ways.
- intended 8mo agoThis is a first degree expectation of most businesses. What the OP pointed out is a fact of life. We do many things to ensure that humans don’t get “routine fatigue”- like pointing at each item before a train leaves the station to ensure you don’t eyes glaze over during your safety check list. This isn’t an excuse for the behavior. Its more about what the problem is and what a corresponding fix should address.
- Waterluvian 8mo agoI agree. The role of an editor is in part to do this train pointing. I think it slips because the consequences of sloppy journalism aren’t immediately felt. But as we’re witnessing in the U.S., a long decay of journalistic integrity contributes to tremendous harm. It used to be that to be a “journalist” was a sacred responsibility. A member of the Fourth Estate, who must endeavour to maintain the confidence of the people.
- zahlman 8mo agoThere's a weird inconsistency among the more pro-AI people that they expect this output to pass as human, but then don't give it the review that an outsourced human would get.
- vidarh 8mo agoThe irony is that while from perfect, an LLM-based fact-checking agent is likely to be far more dilligent (but still needs human review as well) by nature of being trivial to ensure it has no memory of having done a long list of them (if you pass e.g. Claude a long list directly in the same context, it is prone to deciding the task is "tedious" and starting to take shortcuts). But at the same time, doing that makes it even more likely the human in the loop will get sloppy, because there'll be even fewer cases where their input is actually needed. I'm wondering if you need to start inserting intentional canaries to validate if humans are actually doing sufficiently torough reviews.