10 ms·
Stanford Unsupervised Deep Learning Tutorial (2014)
- dominotw 10y agolink to tutorial http://deeplearning.stanford.edu/tutorial/ http://deeplearning.stanford.edu/tutorial/
- KerryJones 10y agoIt's missing a link. "This tutorial assumes a basic knowledge of machine learning ... go to this Machine Learning course [no link]" Am I missing something?
- metafunctor 10y agoPerhaps not exactly the right link, but Andrew Ng's Machine Learning course (also from Stanford) teaches exactly the required things in the first three weeks: https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/machine-learning
- therockhead 10y agoIt's been a while since I did any serious maths, would I be lost in course like this ?
- metafunctor 10y agoNo. The basic linear algebra and calculus concepts used on the course are rather simple, and the videos hold your hand through the math quite well. You don't need deep understanding of the math to pass the course.
- stagger87 10y agoProbably not. I'm currently about halfway through and the majority of required mathematics revolves around linear algebra (matrix multiplication almost exclusively) and basic algebra. There is a linear algebra refresher as well.
- espeed 10y agoSee this course on linear algebra: http://codingthematrix.com/ http://codingthematrix.com/ https://cs.brown.edu/video/channels/coding-matrix-fall-2014/ https://cs.brown.edu/video/channels/coding-matrix-fall-2014/ And this one on Machine Learning: http://work.caltech.edu/telecourse.html http://work.caltech.edu/telecourse.html
- jamra 10y agoI got 100% on the course and don't really do math myself. He skips over the derivations and gives the formula. The way you interact with the math is by turning a formula into code. It's actually fun and refreshing.
- aaronjg 10y agoWhen is this from? The last paper in the reference is 2010.
- deepGem 10y agoThe latest GitHub commit was done 3 years ago. So the course is probably that old. However, I don't think the age matters. I am referring this tutorial in parallel with cs231n and it's so far been good. At least for Convnets.
- Smerity 10y agoI work in the field and I'm with aaronjg - this is ancient in the scheme of deep learning and very far from modern best practices. I honestly find it confusing when I see links like this hit the top of the page with such a strong number of upvotes. There is more modern and better run material now. Even if this is being referred to historically it should involve a timestamp.
- Pyxl101 10y agoA link to an equivalent type of page that teaches modern material would be helpful.
- Smerity 10y agoSee my reply to technics256 and feel free to ask other questions :)
- technics256 10y agoWhere would you recommend one to find the more modern and better material? Thanks.
- Smerity 10y agoFor general introductory material in this style from Stanford, CS231n (fairly general but specialization in vision) and CS224d (specialization in DL for NLP) are great. The material for both of these are online for free and the video lectures (taken down due to legal challenges regarding accessibility) are available if you look hard enough ;) If you're particularly after unsupervised deep learning, I'd recommend you do one or both of the above (or equivalent) and then read relevant recent papers. http://cs231n.stanford.edu http://cs231n.stanford.edu http://cs224d.stanford.edu http://cs224d.stanford.edu
- hardmaru 10y agoI've gone through every exercise from previous version of the stanford tutorial a few year's ago, when it was called the UFLDL tutorial. The sparse autoencoder (http://ufldl.stanford.edu/wiki/index.php/Exercise:Sparse_Autoencoder http://ufldl.stanford.edu/wiki/index.php/Exercise:Sparse_Aut...) exercise has been my favourite one, and I think one that is still relevant today, but became a somewhat ignored concept.
- jypepin 10y agoThis is one of my 2017 resolution: learn more about AI/ML/DL. The field is so big I'm not sure where to start; I've started the Udacity AI class and started reading the "AI: A mordern approach book" from peter norvig. Has anyone does that before and has any recommendations on where to start, good resources etc?
- paulbaumgart 10y agoThis is a very well-written intro to Deep Learning: http://neuralnetworksanddeeplearning.com http://neuralnetworksanddeeplearning.com
- nojvek 10y agoI've been going through the online book. I've been enjoying the tutorials so far.
- paulsutter 10y agoFor Deep Learning, start with MNIST. The 1998 paper[1] describing LeNet goes into a lot more detail than more recent papers. Also there's an excellent video from Martin Gorner at Google that describes a range of neural networks for MNIST[2]. The source code used in his talk is excellent[3]. [1] http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf [2] https://www.youtube.com/watch?v=vq2nnJ4g6N0 https://www.youtube.com/watch?v=vq2nnJ4g6N0 [3] https://github.com/martin-gorner/tensorflow-mnist-tutorial https://github.com/martin-gorner/tensorflow-mnist-tutorial
- fnl 10y agoStarting with DL? That's like learning calculus before geometry... YMMV?
- paulsutter 10y agoConvolutional nets for digit recognition are certainly easier to learn than ML generally, but I wasn't suggesting to start with Deep Learning. I was suggesting that when studying Deep Learning, to start with MNIST. It was easier for me to start with the LeCun 1998 paper than to watch all the theorem-proving in the online courses, but that's just personal preference.
- e19293001 10y agoI've been following this book[0] Grokking Deep Learning. This isn't finished yet but in active development. I like the style of explanation. As of now, I've learned how to create a very simple neural network, a neural network with three input vectors and a very simple deep neural network with three input vectors - 4 hidden size - 1 output. I'm still looking forward for the next MEAP releases and my goal is to understand image deep learning for me to apply on my day job regarding image processing. [0] - https://www.manning.com/books/grokking-deep-learning https://www.manning.com/books/grokking-deep-learning
- placebo 10y agoI've also been following this book and it seems as if progress has stalled. The last sent chapter is incomplete and half baked, there is no response from the author in the book forum and errors in previous chapters remain unchanged.
- e19293001 10y agoThe author says there is a big update and is going on a review process. https://twitter.com/iamtrask/status/818114151339950081 https://twitter.com/iamtrask/status/818114151339950081
- placebo 10y agoGood to know, thanks. Guess I'm a bit of a dinosaur for not following social networks :-)
- imjustsaying 10y agoWhy did you add the word "Unsupervised" when it wasn't contained in the original text?
- vslira 10y agoThis was precisely what made me click the link.
- thomasahle 10y agoIt is in the <title> of the article. UFLDL Tutorial = Unsupervised Feature Learning & Deep Learning Tutorial.
- imjustsaying 10y agoWhoa you're right, there it is.
- TheAlchemist 10y agoGuys, please stop posting this kind of old stuff. Or at least other guys - please stop upvoting this. It's a very old tutorial, there are tons of better ones, in every aspect, today. Somebody once smartly observed that the ratio of upvotes / comments is a good indicator if something is really good. If the ratio is very high it's a red flag. This article is a perfect example - nobody has anything interesting to say about it... Maybe HN admins could take this into account maybe into the sorting algo.
- xchip 10y agoWhat are those better tutorials? I'm interested.
- option_greek 10y agoI'm currently going through this which is to the point and easily understandable: http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ My next step is https://www.tensorflow.org/tutorials/ https://www.tensorflow.org/tutorials/ as I want to move to TF with basics already in grasp.
- TheAlchemist 10y agoI really liked those ones: - CS321 from Standford (outstanding material by Andrej Karpathy): http://cs231n.github.io/ http://cs231n.github.io/ - Hugo Larochelle videos: https://www.youtube.com/channel/UCiDouKcxRmAdc5OeZdiRwAg https://www.youtube.com/channel/UCiDouKcxRmAdc5OeZdiRwAg - Michael Nielsen tutorial: http://neuralnetworksanddeeplearning.com/ http://neuralnetworksanddeeplearning.com/ - Chris Olah blog: http://colah.github.io/ http://colah.github.io/ - Keras blog: https://blog.keras.io/ https://blog.keras.io/ There is also this one (not really a tutorial though): - Goodfellow, Bengio, Courville: http://www.deeplearningbook.org/ http://www.deeplearningbook.org/ Also, there were a lot of 'Ask HN' topics - take a look.
- Dolores12 10y agothe one that was really easy to grasp and gave me good intuition about the subject is https://karpathy.github.io/neuralnets/ https://karpathy.github.io/neuralnets/