3 ms·
What this experiment shows is that making stuff up off the top of your head, using memory alone, isn't as hard as it looks. But when doing actual research, we
by skybrian 4y ago
What this experiment shows is that making stuff up off the top of your head, using memory alone, isn't as hard as it looks.
But when doing actual research, we check our work against the real world. For example, that's how you get a list of real references rather than fake ones.
Suppose we played a guessing game: given a title, does the Wikipedia article exist or not? You could fairly confidently say that "Apple" exists and "wjifdvq" does not, but given a plausible-looking name of a person or place that you don't recognize, you'd have a harder time. It's not a problem in practice though, because you can look it up.
- benreesman 4y agoI sometimes fine tune new language models on the HN comment corpus for fun. It generates comments arguing that ML models are just parrots.
- Swizec 4y agoAs a bird owner, they are not “just parrots”. There is clear intent to communicate. Even species who can’t say human words use distinct vocalizations for specific purposes and situations.
- benreesman 4y agoI didn’t say that fine-tuned language models have enlightened opinions about living things, just that they produce comments indistinguishable to me from the GP :)
- IIAOPSW 4y agohttps://i.imgur.com/xRaUzUGh.jpg https://i.imgur.com/xRaUzUGh.jpg