10 ms·
Tiny-dnn – A C++11 implementation of deep learning
- FraKtus 10y agoSorry if the question is not relevant but is that difficult to use that project to implement something like DeepDream (https://github.com/google/deepdream https://github.com/google/deepdream) ?
- chaosmail 10y agoThis should be possible and fairly easy to do, I guess - it's just forward and backward pass with your favorite model through the net and collect the loss for all octaves. You will have to port a few image utility functions (like roll, zoom, etc.) if not available. I ported the basic Deep Dream example to JavaScript [1] and it was not that difficult. When looking through the code I saw that BP for LRN is not implemented yet, so you cannot pick a model using LRN (or you have to implement it). [1] https://github.com/chaosmail/caffejs/blob/master/docs/assets/scripts/deepdream_worker.js https://github.com/chaosmail/caffejs/blob/master/docs/assets...
- FraKtus 10y agoThank you!
- inglor 10y ago"98.8% accuracy on MNIST in 13 minutes training" - that's really slow isn't it? I can probably write vanilla JS or Python code that does that in 10 lines or so and would finish in around 5 seconds. Why is this taking so long?
- lexy0202 10y ago> I can probably write vanilla JS or Python code that does that in 10 lines or so and would finish in around 5 seconds. Really? There are certainly frameworks that you can use to achieve this performance in that amount of code, but I would be really interested to see that in vanilla JS/Python.
- rawnlq 10y agoNot op but here's a cool example in JS: http://cs.stanford.edu/people/karpathy/convnetjs/demo/mnist.html http://cs.stanford.edu/people/karpathy/convnetjs/demo/mnist....
- inglor 10y agoHey, sure - here's 9 lines of code that implement a perceptron. I get ~95% of precision after a second with 400 samples. ```js const dotSign = (v1, v2) => v1.reduce((prev, x, i) => prev + x * v2[i], 1) > 0 ? 1 : -1 module.exports = (data, weights = Array(data[0].content.length).fill(0)) => { for(const {label, content} of data) { const delta = (label - dotSign(content, weights)) / 2; weights = weights.map((x, i) => x + delta * content[i]); } return { perceive: vector => dotSign(vector, weights), weights }; } ``` I can upload an electron app that does this with mnist if interested.
- anentropic 10y agohow long does does it take to get from 95% -> 98.8% though?
- attractivechaos 10y ago> I get ~95% of precision after a second with 400 samples. MNIST has 60k training samples and 10k test samples. Are you using only 400 of them? Is 95% the accuracy on the test samples or on the same set of training samples? I believe when we talk about MNIST accuracy, we always refer to the accuracy on the 10k test samples.
- krona 10y agoIf you're interested in C++ Frameworks that take a similar approach (e.g. a comprehensive C++ API, statically defined networks etc.) and you aren't afraid of meta-programming, then I'd also take a look at the dlib implementation: http://blog.dlib.net/2016/06/a-clean-c11-deep-learning-api.html http://blog.dlib.net/2016/06/a-clean-c11-deep-learning-api.h...
- hellofunk 10y agoI was shocked to read in that article that MS Visual Studio still does not support the full C++11 spec, which is now over 5 years old. What? I don't use MS products at all, but this really surprises me. Why are they so far behind?
- fetbaffe 10y agoWhat would that be? Visual Studio 2015 seems like fully covered except C99 preprocessor, if this table fully covers all C++11 features. https://msdn.microsoft.com/en-us/library/hh567368.aspx https://msdn.microsoft.com/en-us/library/hh567368.aspx Why they have been behind? Wild guess, Microsoft thought that C#/.NET would be the future for everything and scaled back on C++, but with Windows 10 they realized that the world still consist much of C++, so now it is a equal member again together with .NET and HTML5/JavaScript, i.e Universal Windows Platform.
- krona 10y agoVS 2015 doesn't (fully) support Expression SFINAE. Looks like VS 2017 will: https://blogs.msdn.microsoft.com/vcblog/2016/11/16/sfinae-update/ https://blogs.msdn.microsoft.com/vcblog/2016/11/16/sfinae-up...
- hellofunk 10y agoBut in 2017, there will be another significant new standard, C++17. I wonder if VS will not support it until 2022, if they remain 6 years behind on new language standards?
- fest 10y agoWhen I looked for a small ANN library with little external dependencies and which could be statically linked I settled on FANN[0]. Worked reasonably, solved my problem as well as I hoped it would. It is rather limited in features though- no training on GPU, single-threaded by design, etc. tiny-dnn appears to have a lot more choices regarding network architecture, parallelization options. Would definitely have tried tiny-dnn first if I had known about it. [0]: http://leenissen.dk/fann/wp/ http://leenissen.dk/fann/wp/
- gcp 10y agoFann doesn't do DCNN, arguably the most important right now.
- hellofunk 10y ago> arguably the most important right now. But for very specific uses, right? DCNN excel at image and video, but are DCNN also common for other tasks?
- gcp 10y agoIt's very useful for any kind of signal that can be analyzed with filters.
- mark_l_watson 10y agoSurprisingly (to me at least) convolutional neural networks (CNN) are also useful for natural language processing. (I am using CNN for NLP at work.)
- dr_zoidberg 10y agoCNNs in general excel at infering/decoding "context" from the vectors, so it makes sense to use them for language. But yes, there are cases where they don't make sense/just add complexity.
- hatsunearu 10y ago
- philliphaydon 10y agoWhen I saw "tiny-dnn" I though. Huh? Tiny Dot Net Nuke?? No such thing exists. DNN is too much bloat!
- philliphaydon 10y agoWow HN got no sense of humour today.
- hellofunk 10y agoHN has lots of humour and an appreciation for clever wit and sarcasm. I'm afraid that your effort, while noble, failed to reach this bar.
- mark_l_watson 10y agoThis looks interesting. I cloned the repository and am checking it out. The examples all use supplied with the repo binary image training data. It would be good to document data preparation better (sorry in advance if I just missed seeing that). 20+ years ago, I made my living on C++, but dropped it. Looking at the example programs using C++11 features, I am motivated to get up to speed on C++11. I put writing a NLP example for tiny-dnn and sending a pull request on my todo list.
- EvgeniyZh 10y agoContributor here. Welcome to ask question. We are working on GPU support now.
- tinco 10y agoCan anyone recommend a book, long article or tutorial on deep learning? I'm not in the field but interested in what particular advances have been made that deep learning is suddenly so much more attractive than regular neural networks etc were back in the 00's.
- chaosmail 10y agoCheck out this blog post about deep learning for computer vision applications [1]; it describes very well the historical development and the current advances. In short, not too much changed from LeCuns ConvNets in the 80s. Disclaimer, I wrote this article. [1] http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/ http://chaosmail.github.io/deeplearning/2016/10/22/intro-to-...
- sn9 10y agoGeofrey Hinton (one of the pioneers of deep learning) offers a machine learning course on Coursera: https://www.coursera.org/learn/neural-networks https://www.coursera.org/learn/neural-networks And Udacity offers a course on deep learning: https://www.udacity.com/course/deep-learning--ud730 https://www.udacity.com/course/deep-learning--ud730