3 ms·

Do we still have Turing Test competitions? Now that everybody's used to LLMs, and detecting their style, it might be interesting. I'm not sure the LLMs would do

• viraptor 2y ago
  "detecting their style" is trivial only because people take the default output and run with it. You can significantly alter it if you want and it would be an obvious thing to do for such test. Right now the default output styles of public LLMs have no reason to try avoiding detection and is not trained to do it.
  • card_zero 2y ago
    I didn't exactly mean superficial style, but properties like creativity, liveliness, and consistency. I hesitate to say that these are things LLMs can't do, because I'll provoke somebody into telling me that they totally can, but weaknesses like this could be ways for human intuition to detect an LLM. It might have to be a long competition, because an unusual set of prompts (or training, even) is likely to initially impress the humans: time should be allowed for novelty to wear off.
    • viraptor 2y ago
      > properties like creativity, liveliness, and consistency I don't believe we train The LLMs for those things specifically. It will be interesting to see if some datasets for this appear. I think we can still make huge improvements by just caring about that more. We've got Samantha though, so I hope we will see those attributes too. https://erichartford.com/meet-samantha https://erichartford.com/meet-samantha