8 ms·

> LLMs are great for this, for the plot and character questions, etc. The article links to a clear, direct counterexample of this claim. By Amazon, even. http

• dcre 11mo ago
  It's not that direct a counterexample. We have no idea what underlying data from the Fallout show they gave to the model to summarize. Surely it wasn't the scripts of the episodes. The nature of the error makes me think it might have been given stills of the show to analyze visually. In this case we know it is the text of the book.
  • ceejayoz 11mo ago
    > It's not that direct a counterexample. Amazon made a video with AI summarizing their own show, and got it broadly wrong. Why would we expect their book analysis to be dramatically better - especially as far fewer human eyes are presumably on the summaries of some random book that sold 500 copies than official marketing pushes for the Fallout show.
    • catgary 11mo ago
      Because text analysis is substantially easier than video analysis?
      • ceejayoz 11mo ago
        Amazon has the Fallout scripts, subtitles, internal show bibles, etc. all available to them.
    • dcre 11mo ago
      For the reason I gave in my answer: it would be answering based on the text of the book. I don't expect it to be particularly great regardless because these features always use cheap models.
      • ceejayoz 11mo ago
        > For the reason I gave in my answer: it would be answering based on the text of the book. Why would that not also be true for the Fallout season one recap video?
        • dcre 11mo ago
          Did you read a word past the part of my answer you quoted?
          • ceejayoz 11mo ago
            Of course. I’m just not sure how “the Kindle feature’s cheap models are gonna be even worse at the task” helps your point.