3 ms·
I don't disagree with most of your comment, it seems like a fair assessment, all in all. Where I fundamentally differ in my value judgement is this: > 80% rig
by qsort 3y ago
I don't disagree with most of your comment, it seems like a fair assessment, all in all.
Where I fundamentally differ in my value judgement is this:
> 80% right-first-time
Even taking that number at face value, that is really bad. Really, really bad. Not because of the number itself, but for its consequences:
(a) It's downright harmful as a teacher or tutor. If I'm learning something with little or no prior knowledge, I'm in the worst possible position to question what I'm being taught, and those numbers mean it's going to routinely make mistakes.
(b) If these numbers stay in the same ballpark, it's impossible to compose tasks. (80%) ^ 10 ~= 10%. Make that 95%, (95%) ^ 10 ~= 59%. Unless we see a 3 order of magnitude improvement in the error rate, interactive usage constantly supervised by an expert is the only possible way to use this technology.
(c) Even if it's something I know very well, is basic enough that the model can reasonably do, and not critical enough that an error will be catastrophic, a 20% error rate means I have to keep my eyes wide open.
Unless we see fundamentally new developments, it's a nice to have, but not a game changer. Intellisense level of useful, not git level of useful, let alone Internet level of useful.
- ben_w 3y agoMm, (b) is why it's not already made us redundant. I don't know how far we are from that. But yes, for now it's a footgun for whoever tries to use it as a replacement rather than an assistant. For how long, I know not. But I suspect it is on par with git, despite the flaws: LLMs make you the manager, git helps you collaborate with other humans.