3 ms·
Have you got a reference for the 1989 discovery of double descent? I brief Googling didn't unearth it. I'm not "citation needed"-ing you. I think a lot of decis
by GregarianChild 6y ago
Have you got a reference for the 1989 discovery of double descent? I brief Googling didn't unearth it. I'm not "citation needed"-ing you. I think a lot of decision-tree and boosting work is being rediscovered by the DL community. I would love to see the original discovery.
- disgruntledphd2 6y agoThere you go: https://projecteuclid.org/journals/annals-of-statistics/volume-26/issue-5/Boosting-the-margin--a-new-explanation-for-the-effectiveness/10.1214/aos/1024691352.full https://projecteuclid.org/journals/annals-of-statistics/volu... Rather like the 1957 development of cross-validation (which I remember reading, but definitely don't have a cite for), the double descent thing is part of the paper that is used on the way to something else.
- cosmic_ape 6y agoDo you mean the boosting graphs in Fig. 1, 4 of that paper? It looks though that they have a double descent on the train set too, so it might not be the same phenomenon. Nevertheless, good to know, thanks for sharing! I knew both papers but never thought giving much attention to such details of the figures of the 1998 one. Is the connection between the papers well known, i.e. something people talk about?
- disgruntledphd2 6y agoI wasn't aware of it. I think it's mentioned in the 2016 paper that popularised the term, and I saw the date and went to read it. Yeah, I do mean those graphs, it seems to be the same phenomenon at least.