7 ms·

"First of all, just to say that this is really serious stuff in terms of what was done." "The probability that McCartney wrote it was .018" "In situations lik

"First of all, just to say that this is really serious stuff in terms of what was done." "The probability that McCartney wrote it was .018" "In situations like this, you'd better believe the math because it's much more reliable than people's recollections." The probability was .018 under their model. This doesn't mean that this is the true probability. Naive Bayes probabilities are typically not very reliable [1]. I have not read the paper, but I think his confident wording makes me question how believable this is. [1] Ensembles of models are better, like random forest.

svrt 8y ago
I could not find a paper (I was interested in how they constructed the dataset), but only an extended abstract of the talk [1] It seems the dataset consisted of 70 songs. Since they don't specify the distribution, there is no saying if 80% accuracy is good or worse than average guessing. An average of 35 samples per class is a serious few-shot constraint, which makes resorting to interpretable and simple Bayesian analysis a sane step. Note how the dataset is grossly underdefined too: 70 samples and 149 features, which can cause problems for more complex algo's. I think we have to reconstruct this article from behind the pop-sci glasses this was presented with. > this is really serious stuff in terms of what was done Sure, it is no joke paper, nor meandering about music theory without any hard proofs. > "The probability that McCartney wrote it was .018" Probably this meant to say: The model predicts a 0.018 probability, but that is too careful for a pop-sci article. We can then question the validity of the model. > you'd better believe the math because it's much more reliable than people's recollections "Ha-ha! People did a lot of drugs in the 60s!" nothing more... > And 10 years later, here we are talking about the discovery. Also cute, in that it makes it sound like one of the authors spend 10 years working on this very basic analysis. Meta: This article would have been published 10 years ago too (since the finding is interesting from a pop-sci view), but I doubt they would even dare describing the maths behind the publication. Now we are in techno-fetish era and write about the number of layers, GPU hours, BoW, and value networks, just to fool the reader into getting a glimpse of modern AI. While the methods in this article can be implemented in 1 hour of downloading Midi files and another hour of implementing Graham's 2002: A plan for spam. [2] [1] https://www.amstat.org/asa/files/pdfs/pressreleases/JSM2018-Lennon-McCartney.pdf https://www.amstat.org/asa/files/pdfs/pressreleases/JSM2018-... [2] http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html
- svrt 8y ago
  Needlessly negative/critical, but I could not edit my post. Apologies, I really liked this effort. I dig stylometry, that's why I change my username so much. Their future research may be really cool, tracing popular chord progressions all through pop history.
- coldtea 8y ago
  >Sure, it is no joke paper, nor meandering about music theory without any hard proofs. Well, similarly to how one comments without having read the paper, but just the article. >"Ha-ha! People did a lot of drugs in the 60s!" nothing more... Points to a common fact: people's memories from their drug-fueled periods are hazy, and a historical fact: the Beatles did a lot of drugs during the mid-to-late sixties.
deleted 8y ago
[deleted]
yuliyp 8y ago
> The probability was .018 under their model. This doesn't mean that this is the true probability. Well of course different models will give you different probabilities. But given that this is a past event, talking about "true probability" is a bit weird. The only "true" probabilities are 0 and 1. There's nothing nondeterministic about the past. Someone wrote the song.
- azernik 8y ago
  That's not how Bayesian statistics work. In this model (on which statistical learning and information theory based) a probability does not require nondeterminism, just incomplete information; and a probability is defined with respect to a certain amount/type of information. (Actually information is defined in terms of things that change probability, and probability is the more "core" concept.) So for example, if someone rolls a six-sided die, hides it from you, and asks what the probability of a "6" is, under Bayesian theory it's 1/6; if they then tell you that the number rolled was odd, the probability of a "6" drops to zero, even though nothing physically changed, purely because you now have more information. This is generally a much more useful and intuitive definition of probability than to say "the probability of it being a 6 is either 0 or 1, I just don't know yet".
  - gweinberg 8y ago
    Bayeseian probability is always probability within a model. For things like dice where it's pretty uncontroversial to say the only sensible model is one where the a priori probability of the die landing on each face is equal. For questions like "who wrote the song" it is much less clear what the proper model should be.
    - nonbel 8y ago
      >"Bayeseian probability is always probability within a model." Is there a conception of probability that is not conditional on a model?
      - 8y ago

svrt 8y ago

I could not find a paper (I was interested in how they constructed the dataset), but only an extended abstract of the talk [1] It seems the dataset consisted of 70 songs. Since they don't specify the distribution, there is no saying if 80% accuracy is good or worse than average guessing. An average of 35 samples per class is a serious few-shot constraint, which makes resorting to interpretable and simple Bayesian analysis a sane step. Note how the dataset is grossly underdefined too: 70 samples and 149 features, which can cause problems for more complex algo's. I think we have to reconstruct this article from behind the pop-sci glasses this was presented with. > this is really serious stuff in terms of what was done Sure, it is no joke paper, nor meandering about music theory without any hard proofs. > "The probability that McCartney wrote it was .018" Probably this meant to say: The model predicts a 0.018 probability, but that is too careful for a pop-sci article. We can then question the validity of the model. > you'd better believe the math because it's much more reliable than people's recollections "Ha-ha! People did a lot of drugs in the 60s!" nothing more... > And 10 years later, here we are talking about the discovery. Also cute, in that it makes it sound like one of the authors spend 10 years working on this very basic analysis. Meta: This article would have been published 10 years ago too (since the finding is interesting from a pop-sci view), but I doubt they would even dare describing the maths behind the publication. Now we are in techno-fetish era and write about the number of layers, GPU hours, BoW, and value networks, just to fool the reader into getting a glimpse of modern AI. While the methods in this article can be implemented in 1 hour of downloading Midi files and another hour of implementing Graham's 2002: A plan for spam. [2] [1] https://www.amstat.org/asa/files/pdfs/pressreleases/JSM2018-Lennon-McCartney.pdf https://www.amstat.org/asa/files/pdfs/pressreleases/JSM2018-... [2] http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html

svrt 8y ago
Needlessly negative/critical, but I could not edit my post. Apologies, I really liked this effort. I dig stylometry, that's why I change my username so much. Their future research may be really cool, tracing popular chord progressions all through pop history.
coldtea 8y ago
>Sure, it is no joke paper, nor meandering about music theory without any hard proofs. Well, similarly to how one comments without having read the paper, but just the article. >"Ha-ha! People did a lot of drugs in the 60s!" nothing more... Points to a common fact: people's memories from their drug-fueled periods are hazy, and a historical fact: the Beatles did a lot of drugs during the mid-to-late sixties.

deleted 8y ago

[deleted]

yuliyp 8y ago

> The probability was .018 under their model. This doesn't mean that this is the true probability. Well of course different models will give you different probabilities. But given that this is a past event, talking about "true probability" is a bit weird. The only "true" probabilities are 0 and 1. There's nothing nondeterministic about the past. Someone wrote the song.

azernik 8y ago
That's not how Bayesian statistics work. In this model (on which statistical learning and information theory based) a probability does not require nondeterminism, just incomplete information; and a probability is defined with respect to a certain amount/type of information. (Actually information is defined in terms of things that change probability, and probability is the more "core" concept.) So for example, if someone rolls a six-sided die, hides it from you, and asks what the probability of a "6" is, under Bayesian theory it's 1/6; if they then tell you that the number rolled was odd, the probability of a "6" drops to zero, even though nothing physically changed, purely because you now have more information. This is generally a much more useful and intuitive definition of probability than to say "the probability of it being a 6 is either 0 or 1, I just don't know yet".
- gweinberg 8y ago
  Bayeseian probability is always probability within a model. For things like dice where it's pretty uncontroversial to say the only sensible model is one where the a priori probability of the die landing on each face is equal. For questions like "who wrote the song" it is much less clear what the proper model should be.
  - nonbel 8y ago
    >"Bayeseian probability is always probability within a model." Is there a conception of probability that is not conditional on a model?
    - 8y ago