3 ms·
> Authors actually have an unexpected superpower in the age of AI: they can accurately assess a model’s mastery of a 400-page book better than any collection of
by tkgally 2y ago
> Authors actually have an unexpected superpower in the age of AI: they can accurately assess a model’s mastery of a 400-page book better than any collection of human or software raters, which turns out to be a remarkably valuable way of testing the model’s overall sophistication.
As it happens, that is one of the first ways I tested NotebookLM when I started using it a couple of months ago. I gave it the entire texts of one book I wrote and another I edited, and I asked questions about details scattered throughout the books and about overall themes. The answers were excellent. The generated podcasts also reflected the books’ content extremely well.
- ben_w 2y agoHuh. I gave NotebookLM a much shorter text, one of my own blog posts, and it mixed in accurate statements with a noticeable quantity of made-up statements. I'd give it 80% on accuracy, as I've had similar from real flesh humans, but it's also not an aspect I was particularly impressed with despite that. https://github.com/BenWheatley/Timeline-of-the-near-future https://github.com/BenWheatley/Timeline-of-the-near-future