3 ms·
I try to adjust expectations away from the extremes with these things. Both "it's just a stochastic parrot" (and more recently "artificial stupidity, learns not
by ben_w 3y ago
I try to adjust expectations away from the extremes with these things. Both "it's just a stochastic parrot" (and more recently "artificial stupidity, learns nothing") as a common trope in one group, and also "this is already enough to render $random_profession entirely redundant!" in the opposite.
Anyone who tells me the latter: nope, it's as good as a university student in an industrial placement/internship.
Anyone who tells me the former: nope, the algorithm definitely had to learn a lot to be able to produce even a half-baked mess, and even 3.5 clearly knows more than I do about almost every subject.
So…
> it's fundamentally limited in what it can produce,
Indeed
> and the massive productivity benefits you see (30x faster!) are for trivial tasks, not difficult tasks.
For many people, programming at all is difficult.
Even for us, switching to a completely different language and API isn't a matter of moments.
Don't get me wrong, it absolutely screws up; but at 80% right-first-time (in my experience) going up to 90% just from giving it back the error messages from the compiler or interpreter, that's better than most students and much better than coming from a different domain.
The same in reverse means you should never ever rely on it for anything critical, even though it can still save time if you can perform fast reliable tests on the output.
- qsort 3y agoI don't disagree with most of your comment, it seems like a fair assessment, all in all. Where I fundamentally differ in my value judgement is this: > 80% right-first-time Even taking that number at face value, that is really bad. Really, really bad. Not because of the number itself, but for its consequences: (a) It's downright harmful as a teacher or tutor. If I'm learning something with little or no prior knowledge, I'm in the worst possible position to question what I'm being taught, and those numbers mean it's going to routinely make mistakes. (b) If these numbers stay in the same ballpark, it's impossible to compose tasks. (80%) ^ 10 ~= 10%. Make that 95%, (95%) ^ 10 ~= 59%. Unless we see a 3 order of magnitude improvement in the error rate, interactive usage constantly supervised by an expert is the only possible way to use this technology. (c) Even if it's something I know very well, is basic enough that the model can reasonably do, and not critical enough that an error will be catastrophic, a 20% error rate means I have to keep my eyes wide open. Unless we see fundamentally new developments, it's a nice to have, but not a game changer. Intellisense level of useful, not git level of useful, let alone Internet level of useful.
- ben_w 3y agoMm, (b) is why it's not already made us redundant. I don't know how far we are from that. But yes, for now it's a footgun for whoever tries to use it as a replacement rather than an assistant. For how long, I know not. But I suspect it is on par with git, despite the flaws: LLMs make you the manager, git helps you collaborate with other humans.