karpathy

19 ms·

121.
by karpathy 11y ago
It's not exactly overfitting. It's more that the model is seeing an image that is out of sample, but our approach _forces_ it to say something, so it will do its best but usually fail. One simple low-hanging-fruit approach would b
122.
by karpathy 11y ago
People might be interested in this video by @kcimc , where Kyle runs the pretrained model forward in real time on his laptop while walking around the streets of Amsterdam https://vimeo.com/146492001 Something people don&#x2
123.
by karpathy 11y ago
there's an RSS feed http://karpathy.github.io/feed.xml , and also my Twitter. But to set the expectations I have no immediate plan to write a part 2 - this was a fun experiment that I consider mostly unsustainable in te
124.
by karpathy 11y ago
thank you! Except the story doesn't involve the creation of a General AI, only a cognitive step - one of several along the way. Future stories could center around the limitations still present, and what additional steps might mean. Thi
125.
by karpathy 11y ago
I screwed up on this point by the way - I had done this part of the experiment a few months ago and I incorrectly remembered the details. I went back and looked through the code and adjusted the post with more regarding this important point
126.
by karpathy 11y ago
I have no affiliation with Baidu but I'd like to defend them on this point a bit because it seems to me that they are getting quite a lot more hate than they deserve. Baidu had a without-a-doubt strong and at the very least a very near
127.
by karpathy 11y ago
supervised learning: (after millions of operations you had just performed) out of 1000 you predicted label #136, but actually the true result for this input was label #25. reinforcement learning: (after millions of operations you had just p
128.
by karpathy 11y ago
It does seem like it wasn't trained in the most optimal fashion. - The conv layer has only 20 relu units, I'm not sure if that suffices to memorize the rules. The net might be "underfitting" the rule set, and therefore m
129.
by karpathy 11y ago
I think I like it better with Elon, than if there was some random sales pitch person with a perfectly rehearsed and forcefully over-enthusiastic presentation. It's a combination of respect for the man, and a sense in which we're g
130.
by karpathy 11y ago
+1. There are several fishy statements throughout this paper. Another one in conclusion: "For satellite datasets, with inherently high variability, traditional deep learning approaches are unable to converge to a global optima even wit
131.
by karpathy 11y ago
Nicely done but wait, 15MB?? Clearly the model isn't big enough :) Are you in under or overfitting regime? As I was listening through the samples it seemed to me that it would start out quite energetic and then converge on repetitive,
132.
by karpathy 11y ago
One thing people might not realize (I'm not sure how obvious it is) is that these renders depend strongly on the statistics of the training data used for the ConvNet. In particular you're seeing a lot of dog faces because there is
133.
by karpathy 11y ago
This article would not come as a surprise to anyone who works with ConvNets. Sadly, that might not the case for those outside of the field, largely due to media's inadequate coverage of our advances (but this is common outside our fiel
134.
by karpathy 11y ago
Thanks! I'm really enjoying seeing the huge variety of domains people have applied char-rnn to. I haven't anticipated response at this scale, to both the original blog post and the code release. (I try to keep a somewhat up to dat
135.
by karpathy 11y ago
Sometimes it's hard to separate signal from noise when you're not part of a field and just hearing about projects/papers, so I wanted to quickly pitch in to say that this is a legitimately ground-breaking approach and line of
136.
by karpathy 11y ago
Thanks, I didn't realize this. Fixed!
137.
by karpathy 12y ago
The two classes that you describe: 1. Adversary has the weights and architecture and 2. Adversary can only do forward pass and observe output, are equivalent when all you're trying to do is compute the gradient on the data. In case 1 I
138.
by karpathy 12y ago
This work has led to some unfortunate misconceptions. In particular, this weakness has nothing to do with Computer Vision and also nothing to do with deep learning. They only break ConvNets on images because images are fun to look at and Co
139.
by karpathy 12y ago
I really like this line of work and I expect will grow quite substantially over the next few years. Of course, Reinforcement Learning has been around for a long time. Similarly, Q Learning (the core model in this paper) has been around a ve
140.
by karpathy 12y ago
I could make a lot of comments here but I'd just be reiterating what people much more qualified than I have already said. People who are at the forefront of AI and have actually developed state of the art AI technologies for many years
141.
by karpathy 12y ago
10x speedup in training is a huge deal since modern nets take 2-3 weeks to train on 4 GPUs. Note that the speedup is not actually in the forward-backward raw speed of the computation in the network. The speedup here is a more clever form of
142.
by karpathy 12y ago
You're referring to my google+ post from yesterday :) https://plus.google.com/+AndrejKarpathy/posts/dwDNcBuWTWf The short story is that I tried for 500 images and then got 5.1% error on the test set. Looking
143.
by karpathy 12y ago
The TLDR is that to work at scale we sacrifice some correctness in the dataset, and what's really being evaluated here is closer to some notion of human agreement. The way the data was collected was not through a 1000-way classificatio
144.
by karpathy 12y ago
Thanks! Unfortunately a lot of teaching is very relative and strongly depends on prior background. A different student gave me feedback on that section as: "Why are you expanding out all the random steps nonsense? Gradient descent take
145.
by karpathy 12y ago
Thanks! We ended up not recording the lectures this time around. I was playing with the idea of prerecording some videos MOOC style, but then ended up completely swamped even without them. Among the notes, slides, lectures, office hours, mi
146.
by karpathy 12y ago
Hi HN, the course is still in progress so the notes are not yet finished (I'm currently struggling to finish the ConvNet notes). Our syllabus is here: http://cs231n.stanford.edu/syllabus.html where you can also find le
147.
by karpathy 12y ago
I've worked on similar projects during my Master's degree. An important trick I learned was to make the controller a function of some kind of discrete feedback in the world (e.g. when the foot hits the ground). In your equation ab
148.
by karpathy 12y ago
I wasn't able to find much information about the details of the Stage 1 recovery. How long does it take for it to make it back? Will we know the outcome within a few minutes? (trying to decide if it makes sense to sleep) EDIT: Reddit s
149.
by karpathy 12y ago
This gets a little more detailed into the work, but compared to other papers that have sprung up in this area recently, our paper slightly frowns on the idea of distilling a complex image into a single short sentence description. In that se
150.
by karpathy 12y ago
I'm not sure if I fully understand, but consider that "a person with backpack" tells you extremely little about the actual positions of those two items in the 2-D coordinate system of the image: It could be an extreme closeup

More ›