4 ms·
From TFA - In this process, we have found that in long, extended chat sessions of 15 or more questions, Bing can become repetitive or be prompted/provoked to g
by thunderbong 4y ago
From TFA -
In this process, we have found that in long, extended chat sessions of 15 or more questions, Bing can become repetitive or be prompted/provoked to give responses that are not necessarily helpful or in line with our designed tone. We believe this is a function of a couple of things:
1. Very long chat sessions can confuse the model on what questions it is answering and thus we think we may need to add a tool so you can more easily refresh the context or start from scratch
2. The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend.This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.
I'm guessing most of the crazy responses being reported are because of one of these points.
- rideontime 4y agoWhat is "TFA"?
- wetmore 4y agoThe Fucking/Featured Article
- recuter 4y agoI must ask in that case what does the F in RTFM stand for?
- rom-antics 4y agoRead The Friendly Manual
- recuter 4y agoEgads some ne'er-do-well rapscallions have defaced this one: https://en.wikipedia.org/wiki/RTFM https://en.wikipedia.org/wiki/RTFM
- rideontime 4y agoSeem redundant, what other article would we be discussing?
- basch 4y agoIn my experience, the repetitiveness is also a function of human input. As you ask it to iterate, it repeats most of the previous answer etc. The repetition caused by the human causes it to weigh its own responses more heavily for next time. Think of its short term memory as a sum of everything in the chat window. Suddenly certain phrases are ascribed undue weight. You can fight this in a couple ways. Ask a variety of questions. And search the web. Web searches for some reason appear to reset its prompt, at least partially (i would assume this may be an internal safeguard designed to prevent its search results from overwhelming and outweighing the initial prompt.) Another way to "fix" it midway through chat is to ask it "is it possible for you to respond without repeating what I just said?" and then answer affirmative if that is what you want. It'll then settle back down. I've written elsewhere, that I have had almost no problems with it becoming aggressive, because I choose not to feed it any negative emotions or disrespect. If Microsoft wants to combat that, I would think some sort of preprocessor would be easy, first have a separate instance of a transformer rephrase the input as more respectful. Some of the worst part of the product from my perspective is that it is attached to bing. If i ask it a question, I get a response from a crappy website. If i ask it a question from its internal memory without search, I get a similar but much better answer. If I swap out its rule to use google first, I get better answers. If I let it read articles without searching first, I can control exactly what text is input into its memory. It's honestly a little too bad it steers your travel through bing. It also has an incredibly poor understanding of copyright. It is constantly confused about what it can and cannot do due to copyright restrictions, sometimes telling you it cant parody a song out of respect for the author, but then parodying a different song by the same author. Itll say it cant summarize an article because of copyright, but then say it can give you a "brief overview." It also for some reason is under the assumption that volume of consensus is a substitute for validity. If you talk to it about the possibility that Satan was right for tempting Eve with and gifting her knowledge, itll say no because everybody says so, citing answersingenesis among others in the process.
- iggldiggl 4y ago> It also for some reason is under the assumption that volume of consensus is a substitute for validity. That's simply how those language models work, isn't it? The more often something appears in its training materials, the more likely the final model will respond in that direction. The training process has no innate way of autonomously generating generating some concept of validity that'd automatically let the model up- or downrank certain sources. So the only thing that's left is the developers manually up- or downweighting certain sources during training, but due to the gigantic amounts of text that procedure doesn't really scale to really fine-grained control.
- Denzel 4y agoYes, this is why some people are afraid of Artificial General Intelligence (AGI). We can't even control or predict simple, by comparison, LLMs and yet we have the hubris to believe we'll learn how to control or predict AGIs.
- deleted 4y ago[deleted]
- mistermann 4y agoI wonder if it would help if they could somehow expose the context and allow the user to modify it or apply weights to different parts. I've certainly noticed that chat GPT seems to sometimes simply forget what is going on, reply with the same answer that I've already rejected, etc.