4 ms·
ASIOF spoiler below! > At the Tower of Joy, Ned Stark defeated three members of the Kingsguard and discovered his dying sister, Lyanna, who made him promise to
by zamnos 3y ago
ASIOF spoiler below!
> At the Tower of Joy, Ned Stark defeated three members of the Kingsguard and discovered his dying sister, Lyanna, who made him promise to protect her son, Jon Snow, whose true parentage remained a closely guarded secret.
Seems like ChatGPT-3 already knows, unless there's a deeper secret that I'm not deep enough into ASIOF fandom to know.
- jokteur 3y agoBut this is becaude ASIOF was in the training dataset. Chatgpt wouldn't be able to say anything about this book if it wasn't in his dataset, and you wouldn't be able to have enough tokens to present the whole book to chatgpt.
- nl 3y agoThinking of it as "the training dataset" vs "the context window" is the wrong way of looking at it. There's a bunch of prior art for adaption techniques for getting new data into a trained model (fine tuning, RLHF etc). There's no real reason to think there won't be more techniques that turn what think of now as the context window into something that alters the weights in the model and is serialized back to disk.
- dcl 3y agoIt's a reasonable way to look at it given that's how pretty much all 'deployed' versions of LLM's work?
- dcl 3y agoExactly. But also, not just ASIOF is in the training set, but presumably lots of discussion about it and all the interesting events in the book.