3 ms·
To me there's an intermediate and more-immediate kind of distrust going on: Everyone should distrust and avoid any tool which unpredictably fails in serious way
by Terr_ 3y ago
To me there's an intermediate and more-immediate kind of distrust going on: Everyone should distrust and avoid any tool which unpredictably fails in serious ways that are hidden from the user.
For example, if you were digitizing important blueprints and financial/legal documents before trashing the originals, but the scanning process was unpredictably changing numbers inside text [0] so that your copies were wrong in small but incredibly serious ways.
In the case of LLMs (or the example scanner) the tool will eventually yield answer which looks correct but is dangerously wrong, and the amount of continuous human fact-checking--itself another form of distrust!--needed can be so high that any labor-savings are wiped out.
[0] http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...