4 ms·

I think we should start smaller and make them able to count first.

by dtx1 2y ago

Terr_ 2y ago
Yeah, you can train an LLM to recognize the vocabulary and grammatical features of logical fallacies... Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose. Heck, I think detecting sarcasm would be an easier goal, and still tricky.
- idle_zealot 2y ago
  > Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose Well that's actually good news. With a large enough labelled dataset of actually-sound and fallacious text with similar grammatical features you should be able to train a discriminator to distinguish between them using some other metric. Good luck with getting that data set though.
  - Terr_ 2y ago
    > you should be able to train a discriminator to distinguish between them using some other metric Not when the better metrics are likely alien/incompatible to the discriminator's core algorithm! Then it's rather inconvenient news, because it means you have to develop something separate and novel. As the other poster already mentioned, if we can't even get them to reliably count how many objects are being referred to, how do you expect them to also handle logical syllogisms?
  - nyrikki 2y ago
    The Entscheidungsproblem tends to rear its ugly problem Remember NP is equivalent to second order logic with existential quantified. E.g. for any X there exists a Y And that only gets you to truthy Trues, co-NP is another problem. ATP is hard, and while we get lucky with some constrained problems like type inference, which is pathological in its runtime, but decidable, Pressburger arithmetic is the highest form we know is decidable. It is a large reason CS uses science and falsification vs proofs. Gödel and the difference between Symantec and syntactic completeness is another rat hole.

I think we should start smaller and make them able to count first.

by dtx1 2y ago

I think we should start smaller and make them able to count first.

Yeah, you can train an LLM to recognize the vocabulary and grammatical features of logical fallacies... Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose. Heck, I think detecting sarcasm would be an easier goal, and still tricky.

idle_zealot 2y ago
> Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose Well that's actually good news. With a large enough labelled dataset of actually-sound and fallacious text with similar grammatical features you should be able to train a discriminator to distinguish between them using some other metric. Good luck with getting that data set though.
- Terr_ 2y ago
  > you should be able to train a discriminator to distinguish between them using some other metric Not when the better metrics are likely alien/incompatible to the discriminator's core algorithm! Then it's rather inconvenient news, because it means you have to develop something separate and novel. As the other poster already mentioned, if we can't even get them to reliably count how many objects are being referred to, how do you expect them to also handle logical syllogisms?
- nyrikki 2y ago
  The Entscheidungsproblem tends to rear its ugly problem Remember NP is equivalent to second order logic with existential quantified. E.g. for any X there exists a Y And that only gets you to truthy Trues, co-NP is another problem. ATP is hard, and while we get lucky with some constrained problems like type inference, which is pathological in its runtime, but decidable, Pressburger arithmetic is the highest form we know is decidable. It is a large reason CS uses science and falsification vs proofs. Gödel and the difference between Symantec and syntactic completeness is another rat hole.