4 ms·

This is an interesting subject and I would love to read more on it! Can anyone point me to other publications on the same issue? Preferably something with some

by zibzab 6y ago

This is an interesting subject and I would love to read more on it! Can anyone point me to other publications on the same issue? Preferably something with some real-world examples?

disgruntledphd2 6y ago
So, the thing that I think is most important about the article (which was wonderful, btw) is the double descent loss curve. Originally demonstrated for boosting in 1989, it seems to have made a comeback since 2016. That being said, this kind of stuff is gonna mostly be in the papers, so I suggest following the interesting references from the article, and repeating until you feel you understand :)
- GregarianChild 6y ago
  Have you got a reference for the 1989 discovery of double descent? I brief Googling didn't unearth it. I'm not "citation needed"-ing you. I think a lot of decision-tree and boosting work is being rediscovered by the DL community. I would love to see the original discovery.
  - disgruntledphd2 6y ago
    There you go: https://projecteuclid.org/journals/annals-of-statistics/volume-26/issue-5/Boosting-the-margin--a-new-explanation-for-the-effectiveness/10.1214/aos/1024691352.full https://projecteuclid.org/journals/annals-of-statistics/volu... Rather like the 1957 development of cross-validation (which I remember reading, but definitely don't have a cite for), the double descent thing is part of the paper that is used on the way to something else.
    - cosmic_ape 6y ago
      Do you mean the boosting graphs in Fig. 1, 4 of that paper? It looks though that they have a double descent on the train set too, so it might not be the same phenomenon. Nevertheless, good to know, thanks for sharing! I knew both papers but never thought giving much attention to such details of the figures of the 1998 one. Is the connection between the papers well known, i.e. something people talk about?
      - disgruntledphd2 6y ago
        I wasn't aware of it. I think it's mentioned in the 2016 paper that popularised the term, and I saw the date and went to read it. Yeah, I do mean those graphs, it seems to be the same phenomenon at least.
- ta988 6y ago
  This is pretty common in many fields. Researchers don't have time to read they say. I see people that complain when they have to read one paper a week... This is disastrous in some labs where their only work is to rediscover things and add make-up so it looks new once they realize it is not.
  - disgruntledphd2 6y ago
    I read more papers than that a week, and I'm not even an academic :)

by zibzab 6y ago

This is an interesting subject and I would love to read more on it! Can anyone point me to other publications on the same issue? Preferably something with some real-world examples?

disgruntledphd2 6y ago

So, the thing that I think is most important about the article (which was wonderful, btw) is the double descent loss curve. Originally demonstrated for boosting in 1989, it seems to have made a comeback since 2016. That being said, this kind of stuff is gonna mostly be in the papers, so I suggest following the interesting references from the article, and repeating until you feel you understand :)

GregarianChild 6y ago
Have you got a reference for the 1989 discovery of double descent? I brief Googling didn't unearth it. I'm not "citation needed"-ing you. I think a lot of decision-tree and boosting work is being rediscovered by the DL community. I would love to see the original discovery.
- disgruntledphd2 6y ago
  There you go: https://projecteuclid.org/journals/annals-of-statistics/volume-26/issue-5/Boosting-the-margin--a-new-explanation-for-the-effectiveness/10.1214/aos/1024691352.full https://projecteuclid.org/journals/annals-of-statistics/volu... Rather like the 1957 development of cross-validation (which I remember reading, but definitely don't have a cite for), the double descent thing is part of the paper that is used on the way to something else.
  - cosmic_ape 6y ago
    Do you mean the boosting graphs in Fig. 1, 4 of that paper? It looks though that they have a double descent on the train set too, so it might not be the same phenomenon. Nevertheless, good to know, thanks for sharing! I knew both papers but never thought giving much attention to such details of the figures of the 1998 one. Is the connection between the papers well known, i.e. something people talk about?
    - disgruntledphd2 6y ago
      I wasn't aware of it. I think it's mentioned in the 2016 paper that popularised the term, and I saw the date and went to read it. Yeah, I do mean those graphs, it seems to be the same phenomenon at least.
ta988 6y ago
This is pretty common in many fields. Researchers don't have time to read they say. I see people that complain when they have to read one paper a week... This is disastrous in some labs where their only work is to rediscover things and add make-up so it looks new once they realize it is not.
- disgruntledphd2 6y ago
  I read more papers than that a week, and I'm not even an academic :)