3 ms·

ChatGPT probably makes stack overflow unecessesary, I can see why they don't like it.

by LandR 4y ago

shadowofneptune 4y ago
The article already states why this is such an issue: The results could be correct or incorrect (the model has no idea either way), but are delivered in the style of a well-researched answer. They can also be automated, so there's tons of these deceptive answers.
- blacksmith_tb 4y ago
  Right, it sounds like the scale is a particular problem, you'd expect the feedback mechanisms to gradually weed out nonsense answers, but if 90% of all answers were nonsense, humans couldn't keep up with.
ludamad 4y ago
I have trialled live coding with it and while amazing it suddenly generates half outputs and gets stuck. Right no, no, its swamping their volunteers, not suddenly making their service better.
StreamBright 4y ago
I don't think that you have the data to back this up.
Micoloth 4y ago
I am big fan of Gpt and not a fan of restricting its usage, but i think this decision is completely correct. Considering that, stack overflow is definitely one of the source materials for Gpt training, and probably a very significant one for like Codex. If i ask on SO it’s because i want a human answer. What is the point of getting a Gpt one? If I wanted the Got opinion, I can __just ask gpt__ (and btw I already do it always, now). So the decision is very reasonable.. If anything, it’s sad that i cannot see any way in which it can actually be enforced
MrBoomixer 4y ago
I don't see it this way; Without curated information we wouldn't have a resource of valuable Q and A's for AI to base itself off of. I'm ignorant on what particular text the model was trained on but I have to assume it had a lot of details from online resources. It knows about coding languages and common problems so it has trained it's information form something. Without stack overflow or some kind of relationship between openAI ChatGPT suggested answers and confirmation that it was valid the information would becomes pointless for future frameworks since everyone would be stuck using ChatGPT for answers and it would have no place to find the answers. Maybe I'm missing something a context of some kind but I don't see it providing valid answers without the wealth knowledge like a place that stackoverflow can provide.
nickvincent 4y ago
There's a big concern in the long run: if ChatGPT seriously reduces the number of people who visit StackExchange domains, there will be no dataset no for GPT 4 / ChatGPT 2. E.g. what if a brand new programming language gains popularity, or new libraries with very different patterns of use? This "paradox of reuse" is a really big deal, IMO (blog post on the topic: https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language-models-edition/ https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language...) It's actually a separate concern from the spam / content bloat described in the linked post, but they complement each other in creating amplified harm for StackExchange-like platforms: some fraction of users may stop visiting the site (because ChatGPT answered without links) and some other fraction of users will submit spam to try and earn points.
- nickvincent 4y ago
  Also, based on all the public info about InstructGPT (the closest ChatGPT "family member"), all of StackExchange is definitely in the training via OpenAI's "filtered Common Crawl", if it isn't also included as a special over-weighted training set (English Wikipedia, for instance, was over-weighted in GPT 3 training).
adamckay 4y ago
Until it stops bullshitting, absolutely not. I've asked it to write me some Python code for the Redlock pattern [1] and whilst it generated text that explained it correctly the code wouldn't run - it assumed the class it needed was called `RedLock` when instead it is simply `Lock`. I've also asked it how you can make a Flask application not write any cookies on a specific route. First it told me I can use the `Flask-NoCookies` package, gave me the `pip install flask-nocookies` command and showed me how to use its decorator. Great! Except there's no such package called `Flask-NoCookies`. It's just made it up. I've just repeated that question now and it's told me I "can use the `@app.route()` decorator and specify the `no_cookie` option", again, this doesn't exist and it's just made it up. There are also examples people have posted to Twitter where it's returned references to books, papers and URLs that it's entirely fabricated, too. 1 - https://redis.com/redis-best-practices/communication-patterns/redlock/ https://redis.com/redis-best-practices/communication-pattern...
- amazon_illegal 4y ago

ChatGPT probably makes stack overflow unecessesary, I can see why they don't like it.

by LandR 4y ago

ChatGPT probably makes stack overflow unecessesary, I can see why they don't like it.

The article already states why this is such an issue: The results could be correct or incorrect (the model has no idea either way), but are delivered in the style of a well-researched answer. They can also be automated, so there's tons of these deceptive answers.

blacksmith_tb 4y ago
Right, it sounds like the scale is a particular problem, you'd expect the feedback mechanisms to gradually weed out nonsense answers, but if 90% of all answers were nonsense, humans couldn't keep up with.

ludamad 4y ago

I have trialled live coding with it and while amazing it suddenly generates half outputs and gets stuck. Right no, no, its swamping their volunteers, not suddenly making their service better.

StreamBright 4y ago

I don't think that you have the data to back this up.

Micoloth 4y ago

I am big fan of Gpt and not a fan of restricting its usage, but i think this decision is completely correct. Considering that, stack overflow is definitely one of the source materials for Gpt training, and probably a very significant one for like Codex. If i ask on SO it’s because i want a human answer. What is the point of getting a Gpt one? If I wanted the Got opinion, I can __just ask gpt__ (and btw I already do it always, now). So the decision is very reasonable.. If anything, it’s sad that i cannot see any way in which it can actually be enforced

MrBoomixer 4y ago

I don't see it this way; Without curated information we wouldn't have a resource of valuable Q and A's for AI to base itself off of. I'm ignorant on what particular text the model was trained on but I have to assume it had a lot of details from online resources. It knows about coding languages and common problems so it has trained it's information form something. Without stack overflow or some kind of relationship between openAI ChatGPT suggested answers and confirmation that it was valid the information would becomes pointless for future frameworks since everyone would be stuck using ChatGPT for answers and it would have no place to find the answers. Maybe I'm missing something a context of some kind but I don't see it providing valid answers without the wealth knowledge like a place that stackoverflow can provide.

nickvincent 4y ago

There's a big concern in the long run: if ChatGPT seriously reduces the number of people who visit StackExchange domains, there will be no dataset no for GPT 4 / ChatGPT 2. E.g. what if a brand new programming language gains popularity, or new libraries with very different patterns of use? This "paradox of reuse" is a really big deal, IMO (blog post on the topic: https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language-models-edition/ https://nmvg.mataroa.blog/blog/the-paradox-of-reuse-language...) It's actually a separate concern from the spam / content bloat described in the linked post, but they complement each other in creating amplified harm for StackExchange-like platforms: some fraction of users may stop visiting the site (because ChatGPT answered without links) and some other fraction of users will submit spam to try and earn points.

nickvincent 4y ago
Also, based on all the public info about InstructGPT (the closest ChatGPT "family member"), all of StackExchange is definitely in the training via OpenAI's "filtered Common Crawl", if it isn't also included as a special over-weighted training set (English Wikipedia, for instance, was over-weighted in GPT 3 training).

adamckay 4y ago

Until it stops bullshitting, absolutely not. I've asked it to write me some Python code for the Redlock pattern [1] and whilst it generated text that explained it correctly the code wouldn't run - it assumed the class it needed was called `RedLock` when instead it is simply `Lock`. I've also asked it how you can make a Flask application not write any cookies on a specific route. First it told me I can use the `Flask-NoCookies` package, gave me the `pip install flask-nocookies` command and showed me how to use its decorator. Great! Except there's no such package called `Flask-NoCookies`. It's just made it up. I've just repeated that question now and it's told me I "can use the `@app.route()` decorator and specify the `no_cookie` option", again, this doesn't exist and it's just made it up. There are also examples people have posted to Twitter where it's returned references to books, papers and URLs that it's entirely fabricated, too. 1 - https://redis.com/redis-best-practices/communication-patterns/redlock/ https://redis.com/redis-best-practices/communication-pattern...

amazon_illegal 4y ago