14 ms·
The most cited deep learning papers
- cr0sh 10y agoI can understand why it probably isn't on the list yet (not as many citations, since it is fairly new) - but NVidia's "End to End Learning for Self-Driving Cars" needs to be mentioned, I think: https://arxiv.org/abs/1604.07316 https://arxiv.org/abs/1604.07316 https://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf https://images.nvidia.com/content/tegra/automotive/images/20... I implemented a slight variation on this CNN using Keras and TensorFlow for the third project in term 1 of Udacity's Self-Driving Car Engineer nanodegree course (not special in that regard - it was a commonly used implementation, as it works). Give it a shot yourself - take this paper, install TensorFlow, Keras, and Python, download a copy of Udacity's Unity3D car simulator (it was recently released on GitHub) - and have a shot at it! Note: For training purposes, I highly recommend building a training/validation set using a steering wheel controller, and you'll want a labeled set of about 40K samples (though I have heard you can get by with much fewer, even unaugmented - my sample set actually used augmentation of about 8k real samples to boost it up to around 40k). You'll also want to use GPU and/or a generator or some other batch processing for training (otherwise, you'll run out of memory post-haste).
- sanifsdd 10y agoDo you use a paypal account... in the event if you do you can make an additional 650 a week in your revenue only working on the internet 3 hours per day. see this site... http://bit.ly/2atnA1a http://bit.ly/2atnA1a
- deleted 10y ago[deleted]
- amelius 10y agoNice. I'm wondering how often NVidia's solution makes a mistake. Also, the paper says: > More work is needed to improve the robustness of the network, to find methods to verify the robustness, and to improve visualization of the network-internal processing steps. But it doesn't hint at how this would be approached. Also, how they arrived at the particular network topology seems sort of a mystery.
- splike 10y agoThat's pretty cool. How long does it take to train something like that? And did it work?
- husky480 10y agotorchcraft is the best way to learn about machine learning. If you can sim a set of boxes, you can learn whats inside them.
- curuinor 10y agoNo PDP book? It's old and weird but interesting and has a lot of original ideas, notwithstanding the actual original backprop being from before then. Nor the original backprop stuff?
- cr0sh 10y agoHey, if you can find it, post some links here - even if not mentioned there, it could be interesting to others (hint: I'm interested!)...
- deleted 10y ago[deleted]
- cr0sh 10y agoWas this it? https://stanford.edu/~jlmcc/papers/PDP/ https://stanford.edu/~jlmcc/papers/PDP/
- curuinor 10y agoThe PDP papers are in a bunch of volumes, in addition to the handbook. Chapter 3 of the handbook, I recall, has a little bit from Hinton's original BM work which is of interest to the cognitive scientist, as well as a pretty obstruse but very good description of the entire process. Very historical, very strange by modern ANN standards.
- cr0sh 10y ago> Very historical, very strange by modern ANN standards. That's what I gathered from a brief look at things, which is why I find it very interesting - both from a historical context, as well as the idea that maybe there are some nuggets of (possibly overlooked?) wisdom hidden within, that might be useful in today's world. Then again, I am but an interested (and very amateur) hobbyist in this whole ML space, and not likely to come up with any breakthroughs - but you never know I suppose!
- pks2006 10y agoI always wanted to apply the knowledge of the deep learning to my day to day work. We build our own hardware that runs the Linux on Intel CPU and then launches a virtual machine that has our propriety code. Our code generates a lot of system logs that varies based on what is the boot sequence, environment temperature, software config etc. Now we spend a significant amount of time go over these logs when the issues are reported. Most of the time, we have 1 to 1 mapping of issue to the logs but more often, RCA'ing the issue requires the knowledge of how system works and co-relating this to the logs generated. We have tons of these logs that can be used as training set. Now any clues on how we can put all these together to make RCA'ing the issue as less human involved as possible?
- curuinor 10y agoUse dumber ML first, try some random forests. Not even because they're even that much better or worse, just because DL requires an enormous amount of knowledge and fiddliness but what you prolly want is for the bulk of the actual work to set up the data for ML, not hyperparameter fiddling and architecture fiddling.
- cr0sh 10y agoI agree with this - try the "simpler" solutions first to see if they'll model what you need. No sense in getting lost in more complex methods if a simpler solution will suffice.
- shoyer 10y agoBefore you try any sort of ML, explore your data [1, 2]. Exploratory data analysis may very well tell you that there is absolutely no point in making a fancy predictive model at all. If a few heuristics get you 90% of the way to an optimal solution, then don't even bother to start on machine learning, unless that last 10% is going to provide significant value. [1] https://en.wikipedia.org/wiki/Exploratory_data_analysis https://en.wikipedia.org/wiki/Exploratory_data_analysis [2] From one of my mentors: http://www.unofficialgoogledatascience.com/2016/10/practical-advice-for-analysis-of-large.html http://www.unofficialgoogledatascience.com/2016/10/practical...
- 10y ago
- mathoff 10y agoThe most cited deep learning papers: https://scholar.google.com/scholar?q="deep+learning" https://scholar.google.com/scholar?q="deep+learning"
- EternalData 10y agoNice. Super excited to read through and build out a few things myself.
- gravypod 10y agoThis is a really lucky find for me. I was just about to do something to try and get into machine learning. Right now I need some help getting started with writing some machine learning code. I don't know where to start. I've come up with a very simple project that I think this would work very well for. I want to buy a Raspberry Pi Zero, put it in a nice case, add to push buttons and turn it into a car music player (hook it into the USB charger and 3.5mm jack in my car). The two buttons will be "like" and "skip & dislike". I'll fill it with my music collection, write a python script that just finds a song, plays it, and waits for button clicks. I want the "like" button to be positive reinforcement and the "skip & dislike" to be negative reinforcement. Could someone point me in the right direction?
- delta1 10y agoI'm not sure your problem is well defined enough. Do you want the ML to be able to select songs similar to those you've liked? Or would simply keeping track of the number of likes on each song suffice, such that songs with more likes have a higher probability of being played?
- gravypod 10y agoPerferably (Ordered by percived difficulty): - Prefer songs from most liked genera/artist/album - Favor liked song/genera/artist/album & avoid consistantly disliked song/genera/artist/album. - If a specific song/genera/artist/album is liked more at a specific time, play it more often at that time. - If a song is being played and specific song/genera/artist/album is always liked when it follows this song then favor those. - If you are playing a song and there is a song that is always liked when it follows this song favor that song when choosing the next song. Possibly (if I decide to devote my life to making a music recommending engine): - Corelate GPS location with song choices - Corelate weather/humidity/tempature/month with song choices - Do NLP sentiment analysis to score all of the songs on an emotional-tone scale to help group them well. - Do some DSP on all of the audio to generate a intro and outro beat/rythm profile and attempt to best match the song that will lead to a good enterance/exit. edit: fixed formatting.
- 10y ago
- pizza 10y agohttp://people.idsia.ch/~juergen/deep-learning-conspiracy.html http://people.idsia.ch/~juergen/deep-learning-conspiracy.htm... oh Juergen > Machine learning is the science of credit assignment. The machine learning community itself profits from proper credit assignment to its members. The inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it). Relatively young research areas such as machine learning should adopt the honor code of mature fields such as mathematics: if you have a new theorem, but use a proof technique similar to somebody else's, you must make this very clear. If you "re-invent" something that was already known, and only later become aware of this, you must at least make it clear later.
- deleted 10y ago[deleted]
- curuinor 10y agoI mean, there was a nice 15 or so year period between the "new" post-Minsky-and-Papert Perceptron book connectionism and the current "new new" connectionism where neural nets were a definite backwater. Most of the PIs doing neural nets dealt with it by being sad but Schmidhuber seems to have dealt with it by doubling down on the weirdness.
- latently 10y agoJust because it wasn't wildly popular doesn't mean there was a gap of any sorts. I agree with Schmidhuber. http://grey.colorado.edu/emergent http://grey.colorado.edu/emergent
- gv2323 10y agoHas anyone downloaded them into their own separate folders and zipped the whole thing up?
- nojvek 10y agoSomeone needs to make a summary of the top papers and explain it in a way lay man can understand. I would pay $500 for such a book/course explaining the techniques. I've been reading a number of this papers but it's really tough to understand the nitty gritties of it.
- lomereiter 10y agoJeremy Howard's course (http://course.fast.ai/ http://course.fast.ai/) is much as you describe, the downside is it's a little too boring for people with mathematical education.
- nojvek 10y agoThanks I'll check it out.
- kriro 10y agoThis might be as good a place to ask as any. Does anyone have suggestions on the problem of annotating natural language text to get a ground truth for things that have no readily available ground truth (subjective judgments of content etc.)? I do own the book "Natural Language Annotation" which is good but not exactly what I need. The part of annotation guidelines and how the annotation was done in practice is often only brushed over in many research papers. I mean I get it at a high level it's basically have a couple of raters, calculate inter- and intrarater reliability and try to optimize that. However like I said I'm struggling a bit with details. What are actually good values to aim for, how many experts do you want, do you even want experts or crowd source, what do good annotation guidelines look like, how do you optimize them etc.? Just to play around with the idea a bit, we did a workshop with four raters and 250 tweets each (raters simply assigned one category for the entire tweet) and that was already quite a bit of work and feels like it's on the way to little side of things. I feel like I should find a lot more info on this in the sentiment analysis literature but I don't really.
- splike 10y agoHave you heard of Word2Vec? In a nutshell, its a deep learning model that given a word, predicts the other words around it, or alternatively, given some words in a sentence predicts the missing word. The idea is that similar words end up being assigned to similar vectors, all without knowing a ground truth. Now that won't exactly answer your question, but then you can keep a couple of words related to your sentiment in a list and compare the words in the tweet to that list. If they are similar enough, you can write a rule to mark that tweet as matching your sentiment.
- PeterisP 10y agoYou might want to browse archives of LREC biannual conference, they have sections focused on resource creation and some of the larger projects should have papers on annotation methodology. LDC (https://www.ldc.upenn.edu/ https://www.ldc.upenn.edu/) is probably the largest organization doing a large variety of annotation tasks, maybe they have published how they do things, I'm not sure. However, often there are no real shortcuts; in many projects the resource annotation takes much more work and more people than everything else together, it's not uncommon to see multiple man-years spent to do that properly. What you say about the high level is just about all that can be said in general, everything else will depend on your particular problem. After you've fixed the bugs in your process, interannotator agreement is not really a description of your annotators but a measure of how subjective/objective your task is - and you can't really change that without meaningful changes to how exactly you define your task. Some tasks are well suited for crowdsourcing, and some need dedicated experts. Some annotation tasks are straightforward and annotation guidelines fit on one page; for others the annotation guidelines are a literal book, and one that needs revisions as a few years after you figure out that you need changes. It depends. Shallow sentiment analysis is generally on the trivial side of annotation (but highly subjective), but you can go far enough the rabbit hole to drag the whole surrounding issues of intent, degree of belief, degree of certainty, etc - then you hit the full complexity of deep semantic annotation. Perhaps you just need to find the people who did the latest more interesting datasets in your domain and ask them directly. I don't handle sentiment, but http://alt.qcri.org/semeval2017/task5/ http://alt.qcri.org/semeval2017/task5/ is one group of people that seems to do it seriously.
- applecore 10y agoClassic papers can be worth reading but it's still useful to know what's trending. Even a simple algorithm would be effective: the number of citations for each paper decayed by the age of the paper in years.
- quinnftw 10y agoI think what you are describing here is simply "average number of citations per year", no?
- rahimnathwani 10y agoI think he values recent citations more than older ones.