3 ms·
One of the problems with ChatGPT is that for the most part, the things that people are most likely to feed it are also the things it is most likely to have trai
by jerf 4y ago
One of the problems with ChatGPT is that for the most part, the things that people are most likely to feed it are also the things it is most likely to have training data for.
So, you say, "Write me a heap for Java", and it spits out amazing, wonderful code that is almost right. Well, yeah, that's cool and all, though I would point out that its training data probably included a heap that was entirely correct, but, still, it's a common question.
But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that.
But it deceives people, because the very questions they are most likely to ask as a test are the exact questions it knows the answer to, for the very reason that they are also the most likely questions to be in the training data.
(This article kind of doubles as great set of examples of just how quickly you get into confident nonsense if you stick a toe slightly out of line in your query. For instance, even if it knows "What is Y" and "What is Z", it can't necessarily combine them in any correct way. It'll spew out a lot of words that sound very confident, but it'll make mistakes without any indication they were mistakes.)
- alfalfasprout 4y ago> But I'm not writing a heap for Java. I'm taking business object X and stuffing through system Y and feeding the result to system Z, all internal to the company I work for. Good luck with getting ChatGPT to do that. Exactly. Nearly every example I've seen is something that I could easily find an answer for on SO for anyways. ChatGPT customizes it as needed which is nice (though it often makes glaring and not so glaring mistakes).
- phatfish 4y agoIt's interesting, StackOverflow will give you (after sufficient searching) an existing solution close to what you want. It's pretty certain to be accurate/working in the context of the question asked on SO. You as the "problem solver" just have to adapt the similar solution to your own problem. ChatGPT will try to respond with exactly what you want, but with (multiple) errors that must be fixed. I'll take the working example and adapt it myself. There was a blog post on SO about this recently, they are banning accounts posting ChatGPT answers because they appear reasonable at first glance, but try and execute the code and it has syntax errors or is just plain wrong.
- adamsmith143 4y ago"Guys the first version of a machine to generate code hasn't put me out of a job, it's obviously bollocks." !remindme 10 years
- tombert 4y agoI cannot speak for anyone else, but the disappointment for me came from abundance of clickbaitey articles/videos saying stuff like “ChatGPT rewrote my website 10x better than me!!!” I don’t dispute that it’s super cool technology made by some equally cool people, and honestly I would be perfectly fine with it putting me out of the job if it gets to that point.
- adamsmith143 4y agoTest it for yourself. The beta is free. Whats the point in reading about it when you can easily try it out
- jerf 4y ago"Guys it's really easy to win an argument when I just make up what the other side is saying." !remindme all of human history This is a specific point about people being deceived by the very questions they ask, not a generalized comment about the AI.