3 ms·

Take a look at the answers on this post if you want to know how their most recent genAI experiment went: https://meta.stackoverflow.com/questions/425162/we-are

Take a look at the answers on this post if you want to know how their most recent genAI experiment went: https://meta.stackoverflow.com/questions/425162/we-are-seeking-functional-feedback-for-the-formatting-assistant https://meta.stackoverflow.com/questions/425162/we-are-seeki... In short, it was a complete disaster. They create a tool to provide formatting suggestions on your new questions by piping the entire question into ChatGPT (or one of the comparable services). The internal prompt was figured out within minutes. And there are plenty of examples of this tool messing up questions entirely. And even when it doesn't destroy the questions, it still edits far too much minor and irrelevant stuff. I don't understand how this experiment could make it this far, they actually put it into a live test on the site. So a part of the new users asking questions got this tool in this broken state. And the way this failed was entirely predictable when you just pipe content into genAI.

lamontcg 3y ago
I found it useful to clean up a question I wrote recently. I had to do a few round trips through it, and did need to proof read and fix what it suggested, but nothing too major. The result was something that was much tighter than my usual aspy prose, and got positive responses and suggestions and didn't seem to trigger anyone being pissy.
bhouston 3y ago
> The internal prompt was figured out within minutes. So what? > And there are plenty of examples of this tool messing up questions entirely. And even when it doesn't destroy the questions, it still edits far too much minor and irrelevant stuff. If it can usually help but sometimes doesn't, then this is just an optional suggestion which can be skipped. This seems like a decent design for rolling this out.
- fabian2k 3y ago
  The users that are the most likely to benefit from this kind of tool are also the least likely to be able to judge the quality of the edits it produces. Especially if you're not a native English speaker you might not notice that it changed the meaning of the text in some way. It wasn't hard to trigger problems here, it isn't like this mostly works. And it also breaks code in a way that invalidates the question. With no visible trail of the original question being available.
  - falcor84 3y ago
    > if you're not a native English speaker you might not notice that it changed the meaning of the text in some way Many questions (especially, but not only, from non-native English speakers) are very unclear and ambiguous already, so I wouldn't be that concerned if the meaning is slightly changed - at the very least, GenAI text tends to be self-consistent. > it also breaks code What is this referring to? As I understand it, the tool being discussed only makes suggestions about the title.
LordDragonfang 3y ago
I think this experiment would have gone over much better if SO leadership wasn't so disconnected from the actual users of the site, and had been willing to say "here's an early prototype, please red-team it and show us where it breaks so we can hopefully get it to a state where it's actually useful in a way that makes your jobs easier"

lamontcg 3y ago

I found it useful to clean up a question I wrote recently. I had to do a few round trips through it, and did need to proof read and fix what it suggested, but nothing too major. The result was something that was much tighter than my usual aspy prose, and got positive responses and suggestions and didn't seem to trigger anyone being pissy.

bhouston 3y ago

> The internal prompt was figured out within minutes. So what? > And there are plenty of examples of this tool messing up questions entirely. And even when it doesn't destroy the questions, it still edits far too much minor and irrelevant stuff. If it can usually help but sometimes doesn't, then this is just an optional suggestion which can be skipped. This seems like a decent design for rolling this out.

fabian2k 3y ago
The users that are the most likely to benefit from this kind of tool are also the least likely to be able to judge the quality of the edits it produces. Especially if you're not a native English speaker you might not notice that it changed the meaning of the text in some way. It wasn't hard to trigger problems here, it isn't like this mostly works. And it also breaks code in a way that invalidates the question. With no visible trail of the original question being available.
- falcor84 3y ago
  > if you're not a native English speaker you might not notice that it changed the meaning of the text in some way Many questions (especially, but not only, from non-native English speakers) are very unclear and ambiguous already, so I wouldn't be that concerned if the meaning is slightly changed - at the very least, GenAI text tends to be self-consistent. > it also breaks code What is this referring to? As I understand it, the tool being discussed only makes suggestions about the title.

LordDragonfang 3y ago

I think this experiment would have gone over much better if SO leadership wasn't so disconnected from the actual users of the site, and had been willing to say "here's an early prototype, please red-team it and show us where it breaks so we can hopefully get it to a state where it's actually useful in a way that makes your jobs easier"