5 ms·
A Survey of Deep Learning for Scientific Discovery
- throwqwerty 7y agoLooks like a good summary. Will read. But at the rate the discipline moves I feel like we need one of these every couple of months for everyone (not just "lay" scientists). Anyone know a good journal or something that produces a similar sort of survey frequently? Like once a quarter?
- ssivark 7y ago“Rate at which the discipline moves” is mostly churn, not progress. Important insights come at a slower rate — at the speed of human understanding, not at the speed of conference papers. Good papers from even decades ago are likely to still be useful — in fact, they will have the key ideas presented simply and clearly, without much jargon or hype. Yes, deep learning practice moves quite fast these days, but that’s just the veneer on top of those deeper ideas, trying out tweaks and variations. That’s not completely an indictment of deep learning, rather, any nascent field has a lot of confusing bustle.
- antipaul 7y agoIn a survey on "scientific discovery", I would have expected more examples than face and image recognition and natural language processing, which are so stale at this point. Healthcare? Physics? Chemistry? Biology? Sociology?
- p1esk 7y agoThis is just a bad title. Should have been named simply as “A Survey of Deep Learning”. This paper is an excellent and up to date overview of deep learning models, methods and best practices.
- anthony_doan 7y agoThe rest you listed require inference and causality. Deep learning does not do this. Data with less noises are what most deep learning and non statistical models does well. Meaning that image, nlp, etc.. deep learning does well. But data with lots noises/uncertainty/variance or even data that isn't large enough, such as time series, currently statistical models are still king (https://en.wikipedia.org/wiki/Makridakis_Competitions https://en.wikipedia.org/wiki/Makridakis_Competitions). Even with healthcare you're answering a question/ hypothesis. This is where statistical models strength lies because all statistical models are hypothesis tests and vice versa. There are very little opportunity in healthcare where you would use deep learning compare to statistic. I've seen NLP can be of use but the majority of work in healthcare are inference/casuality base (this is why they use propensity model so much). I'm in this space public healthcare.
- p1esk 7y agoInteresting you mentioned Makridakis competitions. There's one going on right now on Kaggle, and the current leader believes a NN will be the winning model: https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/138881 https://www.kaggle.com/c/m5-forecasting-accuracy/discussion/... More generally, it seems that time series forecasting so far has mostly attracted statisticians with little DL experience [1]. Now that there is $50k prize, this will be a good test of whether statistical methods are "still king". If I were to enter this field, I'd probably look into latest transformer based models, especially the ones used to model raw audio data, e.g. [2]. There's also a real possibility that whenever any strong forecasting method is developed (DL based or otherwise) it's not published as the developers simply use it to make money (betting, stock market, etc). [1] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889 https://journals.plos.org/plosone/article?id=10.1371/journal... [2] https://arxiv.org/abs/1904.10509 https://arxiv.org/abs/1904.10509
- anthony_doan 7y agoI'll wait to see the result at the end of the competition. This is just one of the two competitions for m5. The other one is uncertainty.
- jefft255 7y agoEric Schmidt, as in Google's ex-CEO, is the second author of this paper! I didn't know he did any scientific research.
- biomodel 7y agoAlways wonder who these kinds of reviews / surveys are for? Nobody is going to learn machine learning by reading a 50 page pdf. Meanwhile, people that have experience will have a hard time finding the info they don't already know. Opinionated & narrow >> Shallow & comprehensive
- abdullahkhalids 7y agoThey are useful for someone in a nearby field trying to learn this field. That person first reads the textbook, and a few specific papers. Then once, they have a good narrow understanding, they broaden it by reading one of these review papers. In essence, a review paper saves you the trouble of doing a literature review in a new subfield, because it identifies the important papers for you. That said, the reason review papers are usually written is for the authors to cement their own understanding of the network of research in the field.
- throwawayjava 7y agoA good review article is worth its weight in gold for both the researchers who write it and the research community. Remember that research communities are extremely transient because of the professor : phd student : practitioner ratio and the low odds that a graduated phd student a) stays in research and then b) stays in the same research area for their whole career. Therefore, most members of a given research community have approximately 1-3 years of experience in the broader academic field and approximately no experience in the area covered by the review. Therefore, a good review can simultaneously: 1. prevent a lot of wheel re-invention, and 2. push the research field in a certain direction (either accidentally or purposefully). Also, good review articles typically include some amount of synthesis. I.e., the creation of a conceptual framework and language for understanding and talking about a bunch of vaguely related stuff. This article tries to do that e.g. in Section 2.1 but the topic of the review is so incredibly broad that the categories are not super useful.
- mistrial9 7y agoI will read it, to defend my non-DeepLearning choices for supervised ML .. so many on the bandwagon for unsupervised CNN with their GPUs
- wswin 7y agofor the moment I thought it was from 2003