4 ms·

I also could not find any mention of the methodology.

• sitkack 2y ago
  Without this detail, the whole body of work is anecdotal.
  • benreesman 2y ago
    To be fair, most of the commentary on both sides of the LLM conversation are pretty anecdotal, which is increasingly looking like a structural problem given that any solid evidence goes in the training set in about an hour.
  • gctwnl 2y ago
    Definitely. Otherwise it would have required a lot more than a single blog post. It is an observation, not anything rigorous with a large number of examples, and decent statistics.
    • sitkack 2y ago
      It doesn’t require a whole more. 1) the full transcript of the llm exchange and 2) version numbers for the llm would go a long way. It is basically a long winded way of saying in a bug report, “it doesn’t work”.