4 ms·

How can they be so sure the model isn’t just hallucinating? It can also hallucinate real facts from the training data. However, that doesn’t mean the entire out

by Nevin1901 3y ago

How can they be so sure the model isn’t just hallucinating? It can also hallucinate real facts from the training data. However, that doesn’t mean the entire output is directly from the training data. Also, is there any real world use case? I couldn’t think of a case where this would be able to extract something meaningful and relevant to what the attackers were trying to accomplish.

LeoPanthera 3y ago
> How can they be so sure the model isn’t just hallucinating? This is explicitly covered in the article, if you scroll down.
Solvency 3y ago
They can't.
- Aachen 3y ago
  If you flip a coin and generate a bitcoin address from it, there's two possible keys. Two coin flips, four possibilities. After 80 coin flips, you've got more possible keys than a regular computer can loop through in a lifetime. After some 200 coin flips, the amount of energy to check all of them (if you're guessing the generated private key for this address) exceeds what the sun outputs in a year iirc (or maybe all sunlight that hits the earth — either way, you get the idea: incomputable with contemporary technology). Exponentials are a bitch or a boon, depending on what you're trying to achieve. Per another comment <https://news.ycombinator.com/item?id=38467969 https://news.ycombinator.com/item?id=38467969>, existing bitcoin addresses is what they found being generated. There is physically no way that's a coincidence. Perhaps it live queries the web, that's an alternative explanation you could prove if the authors are wrong (science is a set of testable theories, after all). The simplest explanation, given what we know of how this tech works, is that it's training data.
PUSH_AX 3y ago
They cover this in the article, they verified that the output matched data found on the internet, 100% verbatim.
mettamage 3y ago
They have 10TB on internet data and could find huge swaths of texts verbatim in it.

LeoPanthera 3y ago

> How can they be so sure the model isn’t just hallucinating? This is explicitly covered in the article, if you scroll down.

Solvency 3y ago

They can't.

Aachen 3y ago
If you flip a coin and generate a bitcoin address from it, there's two possible keys. Two coin flips, four possibilities. After 80 coin flips, you've got more possible keys than a regular computer can loop through in a lifetime. After some 200 coin flips, the amount of energy to check all of them (if you're guessing the generated private key for this address) exceeds what the sun outputs in a year iirc (or maybe all sunlight that hits the earth — either way, you get the idea: incomputable with contemporary technology). Exponentials are a bitch or a boon, depending on what you're trying to achieve. Per another comment <https://news.ycombinator.com/item?id=38467969 https://news.ycombinator.com/item?id=38467969>, existing bitcoin addresses is what they found being generated. There is physically no way that's a coincidence. Perhaps it live queries the web, that's an alternative explanation you could prove if the authors are wrong (science is a set of testable theories, after all). The simplest explanation, given what we know of how this tech works, is that it's training data.

PUSH_AX 3y ago

They cover this in the article, they verified that the output matched data found on the internet, 100% verbatim.

mettamage 3y ago

They have 10TB on internet data and could find huge swaths of texts verbatim in it.