benanne

7 ms·

31.
by benanne 11y ago
We sort of reverse-engineered this last week and set up a stream with live interactive "hallucinations": http://www.twitch.tv/317070 You can suggest what objects the network should dream about (combinations of two
32.
by benanne 11y ago
One of the co-authors here, we are planning to release the source code on GitHub in a few days.
33.
by benanne 11y ago
Indeed, 64 bits of accuracy is overkill for a lot of ML algorithms, where there is so much noise that the additional quantization noise due to low precision is negligible. Most deep neural nets are already being trained in single precision
34.
by benanne 11y ago
I guess... but that's so boring! ;)
35.
by benanne 11y ago
I think so.
36.
by benanne 11y ago
Unlike Tolkien, George R.R. Martin never created full-fledged languages for his books, but he did make up a bunch of words. I guess they didn't want to replace those with different words from other real-world languages. One of the chal
37.
by benanne 11y ago
This is great, thanks for sharing!
38.
by benanne 11y ago
A couple of years ago there was no book version of the LCK, but a very popular book on the zompist bulletin board was "Describing Morphosyntax: A Guide for Field Linguists", by Thomas E. Payne. It's essentially an overview of
39.
by benanne 11y ago
For those interested in constructed languages, I recommend checking out Mark Rosenfelder's website http://zompist.com/ . His Language Construction Kit in particular is a great resource: http://zompist.com&#x2
40.
by benanne 11y ago
There are some benchmarks of a neural network toolkit built on top of this: https://github.com/soumith/convnet-benchmarks compare NVIDIA's own cuDNN R2 versus NervanaSys-16 and NervanaSys-32. Pretty impressive! I&
41.
by benanne 12y ago
Thanks for the plug :) This is not quite the same thing though, we used a bunch of affine transformations for data augmentation, but we're not using any transforms with fancy invariance properties to compute the feature maps inside the
42.
by benanne 12y ago
Nice work! Since you mentioned you're looking for RNNs/LSTMs specifically: the implementation at https://github.com/skaae/nntools is an extension of Lasagne (which used to be called nntools) and will be merge
43.
by benanne 12y ago
One of the authors of Lasagne here! Lasagne is being built by a team of deep learning and music information retrieval researchers. Keras seems to share a lot of design goals with our project, but there are also some significant differences.
44.
Classifying plankton with deep neural networks(benanne.github.io)
23 points by benanne 12y ago|0 comments
45.
by benanne 12y ago
The article recommends getting a 580 as the cheapest, most cost-effective option. One thing the 580 has going against it is that the cuDNN library does not support it. Only Kepler and Maxwell cards (600, 700 and 900 series) are supported. S
46.
by benanne 12y ago
True, there is no real up to date curated reading list that I know of. Actually, there was a list of 2014 deep learning papers going around recently, but it seemed to list basically every tangentially related paper that the authors could fi
47.
by benanne 12y ago
There's some great stuff there, but that list could use an update. The most recent papers seem to be from 2011. Things move very fast in this field. This was back when pre-training was still strongly recommended and before Krizhevsky&#
48.
by benanne 12y ago
I haven't had any luck so far with rmsprop, adagrad and adadelta. SGD + Nesterov momentum has served me best.
49.
by benanne 12y ago
The discussion about this paper on r/MachineLearning is quite insightful and worth reading: http://www.reddit.com/r/MachineLearning/comments/2onzmd/deep...
50.
by benanne 12y ago
Google DeepMind chiefly uses Torch7. I presume parts of Twitter also use it now since they acquired Clement Farabet's startup MadBits. Facebook's AI research lab has contributed to the Torch7 project (which is unsurprising since i
51.
Images to Text – Toronto Deep Learning Demos(deeplearning.cs.toronto.edu)
75 points by benanne 12y ago|17 comments
52.
by benanne 12y ago
I don't really understand how it would reduce processing time, could you elaborate? The main implications seem to be for neuroscience, as far as I can tell. Backprop is considered biologically implausible because it requires either bid
53.
by benanne 12y ago
Looks like the Toronto group have been working on something very similar as well: http://lanl.arxiv.org/abs/1411.2539 Has anybody been able to find the Google paper? The article says it's on arxiv, but I can'
54.
by benanne 12y ago
Probably not. The noise in the training data itself, noise from dropout and various other sources are going to trump any quantization noise anyway, so you can use fairly inaccurate representations. This is why everyone's using gamer GP
55.
by benanne 12y ago
There are plans for Theano to build 'meta optimizers' that determine the fastest implementation separately for each layer in a network, and for each 'pass' (forward, backward w.r.t. weights, backward w.r.t. input): http
56.
by benanne 12y ago
I don't know where this figure comes from: "Nvidia's official benchmarks show a 10x speed-up when using Caffe with cuDNN.". The graph in the original announcement ( http://devblogs.nvidia.com/parallelfora
57.
by benanne 12y ago
Well, I did say "in my experience" :) I certainly didn't mean to imply that problems requiring less than 3GB of GPU RAM are laughable, or anything like that. I should have said something like like "problems that people a
58.
by benanne 12y ago
Anandtech has some compute benchmarks for the 980: http://www.anandtech.com/show/8526/nvidia-geforce-gtx-980-re... unfortunately I don't know which of these are representative of typical deep learning workloa
59.
by benanne 12y ago
I disagree with his position on memory (he mentions in the post that anything above 1.5GB should be fine). In my experience, anything below 3GB can be pretty uncomfortable these days, if you want to work on serious problems. It's not j
60.
by benanne 12y ago
That's true, there is quite some overlap with what the Echo Nest are doing, albeit using a different approach. As I mentioned in the post, a lot of different collaborative filtering techniques are currently used together, so the same c

More ›