15 ms·
> The training data If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?
by throw310822 8mo ago
> The training data
If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?
- qsera 8mo agoJust using a scaled up and cleverly tweaked version of linear regression analysis...
- red75prime 8mo agoThat is, the probability distribution that the network should learn is defined by which probability distribution the network has learned. Brilliant!
- cbovis 8mo agoThe prompt is unique but the tokens aren't. Type "owejdpowejdojweodmwepiodnoiwendoinw welidn owindoiwendo nwoeidnweoind oiwnedoin" into ChatGPT and the response is "The text you sent appears to be random or corrupted and doesn’t form a clear question." because the prompt doesnt correlate to training data.
- hmmmmmmmmmmmmmm 8mo ago...? what is the response supposed to be here?
- HDThoreaun 8mo agoOr because the text you send was random and doesnt form a clear quesiton?
- newswasboring 8mo ago> The prompt is unique but the tokens aren't. The tokens aren't unique, but the sequence is. Every input this model sees in unique. Even tokens are not as simple as they seem If you type "ejst os th xspitsl of fermaby?" in ChatGPT it responds with > It looks like you typed “ejst os th xspitsl of fermaby?”, which seems like a garbled version of: > "What is the capital of Germany?” > The capital of Germany is Berlin. > If you meant to ask something else, feel free to clarify!" edit: formatting
- ajam1507 8mo agoThe prompt does correlate to its training data. In this case, since you sent random text, it generated the most likely response to random text.
- hmmmmmmmmmmmmmm 8mo agoHamiltonian paths and previous work by Donald Knuth is more than likely in the training data.
- red75prime 8mo agoThe specific sequence of tokens that comprise the Knuth's problem with an answer to it is not in the training data. A naive probability distribution based on counting token sequences that are present in the training data would assign 0 probability to it. The trained network represents extremely non-naive approach to estimating the ground-truth distribution (the distribution that corresponds to what a human brain might have produced).
- qsera 8mo ago>the distribution that corresponds to what a human brain might have produced.. But the human brain (or any other intelligent brain) does not work by generating probability distribution of the next word. Even beings that does not have a language can think and act intelligent.