4 ms·

I wonder if this could one day be how we settle disagreements with no solid answer, like was William Shakespeare really the author of all those plays.

TimTheTinker 4y ago
The word "knowledge" as used in TA shouldn't be taken as philosophically defined knowledge (i.e. a justified true belief). Language models might indeed help us find potential sources of validation for various hypotheses, but deciding whether a hypothesis is true is another matter entirely.
- didericis 4y ago
  This is an extremely important point. These are very large, very complex fuzzy language maps, and are best used like a crystal ball for autocompletion suggestions, not definitive answers. I’m unsurprised but also very disappointed by people’s desire to give these fuzzy answers final word. The desire to cede final authority on truth to these machines just because they’re giant and impressive is incredibly misguided and a major step backwards. These machines are literal embodiments of group think. They can only add value if people understand their limitations. If people get carried away with them and start assuming they’re authoritative we’re in for serious trouble.
goto11 4y ago
William Shakespeare was the author of those plays, if you mean plays like Hamlet, Romeo and Juliet etc. The answer is completely settled, since there is no serious argument that we wasn't, outside of fanciful conspiracy theories. There are legitimate discussions about some of the plays which were likely collaborations between multiple authors (e.g. Edward III, Henry IV 1), and machine learning has been used to try identify the different authors and authorship of different passages. This is an area of open research.
- boppo1 4y ago
  If you run ML on the historian-uncontested[0] plays the same way as on the ones with likely different/multiple authors, what is the result? Does it say 'this is clearly just one guy'? [0] You sound confident about the consensus. Can you share the material that led you to that? I'm currently under the impression they were all written by multiple authors. HOWEVER, I recognize I'm very low-info on this (or low-quality info at least) and my belief would be easily changed by something reputable and/or with good methodology.
  - goto11 4y ago
    > Does it say 'this is clearly just one guy'? It says the majority of the plays are likely written by the same person, since Shakespeare have a distinct "voice" which is clearly different form other contemporary playwrights. But there is a number of pitfalls in the statistical approach because an author might try to mimic the writing style of another author, especially since these collaborations was somewhat like master-apprentice relationships. The Oxford Complete Shakespeare credits Marlowe as co-author of Henry VI part 1, but this is highly controversial since some believe Shakespeare was just strongly inspired by Marlowe in his early plays. Here is a podcast with a bit a bout the controversy: https://www.folger.edu/shakespeare-unlimited/christopher-marlowe-attribution-henry-vi https://www.folger.edu/shakespeare-unlimited/christopher-mar... As for the general authorship question, you can just read anything which isn't written by a crackpot. But Oxford has "William Shakespeare - A Very Short Introduction" which is a pocket-size introduction and biography. If you want something meatier, Marjorie Garbers "Shakespeare After All" is a great book.
- theGnuMe 4y ago
  What about stylometry.. that should be enough…
  - jameshart 4y ago
    But what if we can train a language model to produce output that, according to stylometric analysis, is with 90% confidence written by the same person who wrote Hamlet?
    - totetsu 4y ago
      I want to watch a performance of this nouveau Shakespeare.
jameshart 4y ago
I’m reminded of Charles Babbage’s comments on being asked about the capabilities of his difference engine: On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. Where do you think a large language model is going to acquire the information to establish the truth value of a statement, whose truth value is unknown outside the context of the language model?
- totetsu 4y ago
  Well that’s why its ‘latent’ knowledge. And I’m not thinking about deductive truth so much as the possibility of a new epistemological position that uh.. just takes a sufficiently large models word for it.
  - jameshart 4y ago
    A formal definition of ‘common knowledge’ or ‘common sense’? “Anything that would be regarded as true by a language model trained on the Common Crawl dataset”. Be an interesting philosophical position to test out in, say, a judicial setting. We could replace the legal concept of a ‘reasonable person’ with ‘a language model with its temperature set to a reasonable level’. And perhaps instead of a jury of your peers, the fairest way to reach a verdict would be to present the facts of a case to a large language model, and ask it ’guilty or not guilty?’ twelve times..
  - didericis 4y ago
    > just takes a sufficiently large models word for it The renaissance and enlightenment happened because we stopped doing this. The church was that era’s big model, and doctrine was similarly impenetrably complex and indecipherable. You’d just take the church’s word for something. We’re going backwards.
    - Dylan16807 4y ago
      "sufficiently large model" is really really not how churches work. > We’re going backwards. ...it was an off the cuff hypothetical.

TimTheTinker 4y ago

The word "knowledge" as used in TA shouldn't be taken as philosophically defined knowledge (i.e. a justified true belief). Language models might indeed help us find potential sources of validation for various hypotheses, but deciding whether a hypothesis is true is another matter entirely.

didericis 4y ago
This is an extremely important point. These are very large, very complex fuzzy language maps, and are best used like a crystal ball for autocompletion suggestions, not definitive answers. I’m unsurprised but also very disappointed by people’s desire to give these fuzzy answers final word. The desire to cede final authority on truth to these machines just because they’re giant and impressive is incredibly misguided and a major step backwards. These machines are literal embodiments of group think. They can only add value if people understand their limitations. If people get carried away with them and start assuming they’re authoritative we’re in for serious trouble.

goto11 4y ago

William Shakespeare was the author of those plays, if you mean plays like Hamlet, Romeo and Juliet etc. The answer is completely settled, since there is no serious argument that we wasn't, outside of fanciful conspiracy theories. There are legitimate discussions about some of the plays which were likely collaborations between multiple authors (e.g. Edward III, Henry IV 1), and machine learning has been used to try identify the different authors and authorship of different passages. This is an area of open research.

boppo1 4y ago
If you run ML on the historian-uncontested[0] plays the same way as on the ones with likely different/multiple authors, what is the result? Does it say 'this is clearly just one guy'? [0] You sound confident about the consensus. Can you share the material that led you to that? I'm currently under the impression they were all written by multiple authors. HOWEVER, I recognize I'm very low-info on this (or low-quality info at least) and my belief would be easily changed by something reputable and/or with good methodology.
- goto11 4y ago
  > Does it say 'this is clearly just one guy'? It says the majority of the plays are likely written by the same person, since Shakespeare have a distinct "voice" which is clearly different form other contemporary playwrights. But there is a number of pitfalls in the statistical approach because an author might try to mimic the writing style of another author, especially since these collaborations was somewhat like master-apprentice relationships. The Oxford Complete Shakespeare credits Marlowe as co-author of Henry VI part 1, but this is highly controversial since some believe Shakespeare was just strongly inspired by Marlowe in his early plays. Here is a podcast with a bit a bout the controversy: https://www.folger.edu/shakespeare-unlimited/christopher-marlowe-attribution-henry-vi https://www.folger.edu/shakespeare-unlimited/christopher-mar... As for the general authorship question, you can just read anything which isn't written by a crackpot. But Oxford has "William Shakespeare - A Very Short Introduction" which is a pocket-size introduction and biography. If you want something meatier, Marjorie Garbers "Shakespeare After All" is a great book.
theGnuMe 4y ago
What about stylometry.. that should be enough…
- jameshart 4y ago
  But what if we can train a language model to produce output that, according to stylometric analysis, is with 90% confidence written by the same person who wrote Hamlet?
  - totetsu 4y ago
    I want to watch a performance of this nouveau Shakespeare.

jameshart 4y ago

I’m reminded of Charles Babbage’s comments on being asked about the capabilities of his difference engine: On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. Where do you think a large language model is going to acquire the information to establish the truth value of a statement, whose truth value is unknown outside the context of the language model?

totetsu 4y ago
Well that’s why its ‘latent’ knowledge. And I’m not thinking about deductive truth so much as the possibility of a new epistemological position that uh.. just takes a sufficiently large models word for it.
- jameshart 4y ago
  A formal definition of ‘common knowledge’ or ‘common sense’? “Anything that would be regarded as true by a language model trained on the Common Crawl dataset”. Be an interesting philosophical position to test out in, say, a judicial setting. We could replace the legal concept of a ‘reasonable person’ with ‘a language model with its temperature set to a reasonable level’. And perhaps instead of a jury of your peers, the fairest way to reach a verdict would be to present the facts of a case to a large language model, and ask it ’guilty or not guilty?’ twelve times..
- didericis 4y ago
  > just takes a sufficiently large models word for it The renaissance and enlightenment happened because we stopped doing this. The church was that era’s big model, and doctrine was similarly impenetrably complex and indecipherable. You’d just take the church’s word for something. We’re going backwards.
  - Dylan16807 4y ago
    "sufficiently large model" is really really not how churches work. > We’re going backwards. ...it was an off the cuff hypothetical.