colah3

14 ms·

121.
by colah3 11y ago
Not the author, but I wrote an article introducing conv nets you might find helpful: http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ I also have an article on RNNs, although it's focused on explaining
122.
by colah3 11y ago
Derek is also extremely available to his colleagues at Google. He's always friendly when I ask questions, and very thoughtful. I feel lucky to work with him, however distantly! :)
123.
by colah3 11y ago
When I was 17, I was super enthusiastic about Linux. I read a bunch of stuff about kernel development. Linus' attitude was very well known and presented as an amusing thing. Glorified even. I remember reading some of his emails with a
124.
by colah3 11y ago
We wish! We're actually in a drought where I live. I also don't think I wear a coat 75% when it is sunny. :) But I wanted to have nice numbers and it felt like a nice example.
125.
by colah3 11y ago
I feel like 90% of my motivation for writing blog posts is vicariously reliving this feeling. :)
126.
by colah3 11y ago
I drew the graphs in inkscape. It has a plugin for LaTeX equations.
127.
by colah3 11y ago
Yep. The visualization tricks in this article are for building understanding of basic ideas in probability theory and information theory. In most real situations, they wouldn't be very practical. As you note, the core trick in this ess
128.
by colah3 11y ago
I don't think this is really a case of Simpson's paradox. What we're seeing is that female students at Stanford are less concentrated on a single major than the male ones. As a result, their top pick isn't as high as the
129.
by colah3 11y ago
TreeNets are also pretty popular. Word embeddings are also a form of weight tying, and can't always be described as part of an RNN. If you want, you can think of most attentional mechanisms as involving a form of weight tying, in addit
130.
by colah3 11y ago
I think questions about representations and symmetry are really interesting. Have you looked at deep symmetry networks? (I think the right way to frame the idea is in terms of group convolutions, which I outlined here: http://col
131.
by colah3 11y ago
The problem I worry about with the neuroscience narrative is that it seems easy to justify everything. As I understand it, we don't understand the brain very well, and there's lots of different theories. A fair number of neural ne
132.
by colah3 11y ago
> Seeing different ways to look at something is always refreshing ... how do YOU think about NNs? A year or two ago, when I was really doing the wandering researcher thing, I made a point of asking lots of people at different groups how
133.
by colah3 11y ago
Very possible. It just feels like something from that general area, and I wanted to point it out. I played around with fitting this in to the Curry-Howard correspondence. For a moment, let's set aside neural networks and just talk abou
134.
by colah3 11y ago
If I didn't know that neural nets worked, and someone explained the idea, I'd strongly expect them to get stuck in local minima, instead of learning this elaborate behavior. (That's basically what happens with a normal RNN, I
135.
by colah3 11y ago
> Whats amazing to me is that if I understand correctly backprop still works. Yep! One computes the gradient with backpropagation and trains LSTMs on that. > It is very odd that SGD on the error function for some training data is conc
136.
by colah3 11y ago
All of the diagrams in this post were made in Inkscape, with the LaTeX plugin for equations.
137.
by colah3 11y ago
> hochreiter is genius! It's really impressive to me how farsighted his work on LSTMs was. In any case, I'm glad you liked the post.
138.
by colah3 11y ago
This is true. That said, I'm not sure how true it is in Indonesia. Many very attractive jobs will be outside Indonesia. Immigration without a degree can be very tricky...
139.
by colah3 11y ago
Nice! You might consider adding support for one of the MIT Places networks ( http://places.csail.mit.edu/downloadCNN.html ). That's how we got a lot of the pictures we used in the original blog post. For example, these w
140.
by colah3 12y ago
The article seems to suggest it is mostly of historical interest. The researchers seem impressed by its effectiveness for a medicine of that time. It's of further interest because it seems to have come out of a tradition of medicine th
141.
by colah3 12y ago
For the algebraic geometry side, though, *Ideals, Varieties and Algorithms" by Cox, Little and O'Shea is great.
142.
by colah3 12y ago
> but I don't know anyone who works with CAD who would likely switch their work flow from GUI to text-based, other than people in this thread, I suppose. Programming doesn't have to be writing text. :P Everyone seems to be thi
143.
by colah3 12y ago
> * this line needs to be 1/3 the length of that line > * these set of lines should start 50mm left of this line Remember that you're working with points. So, to express the length of a line, you need something like (x1-x2)^
144.
by colah3 12y ago
Sadly, I never did a serious literature review. That said, I naively tried a bunch of naive approaches and had quite a bit of success with plain old gradient descent -- it was really surprising. I'm not sure if it would scale to really
145.
by colah3 12y ago
I was really excited about handling some of the messy constraint situations better than existing tools. I'll split this into two pieces: > It happens pretty often that you'll modify a single constraint (flipping it is a common
146.
by colah3 12y ago
> I think CAD constraints would always be expressed in a linear system Typically, you want to at least be able to express the lengths of lines, and the values of angles. I don't know how you could do this without using at least quad
147.
by colah3 12y ago
Author of ImplicitCAD here! I had some neat tricks that made beveling most edges really easy! (Though, when the didn't work, it was pretty messy.) Please keep in mind that I abandoned this project 2+ years ago, and started it four year
148.
by colah3 12y ago
Author of ImplicitCAD here. I completely agree. At first, I tried to implement constraints in a Haskell DSL. Unfortunately, constraints usually only partially describe your object. Even a well-constrained object can be flipped and maintain
149.
by colah3 12y ago
Author here. Sorry, I abandoned this two or three years ago. Please look at this in the context of a project done several years ago and never completed. (I do ML research these days -- you can see my blog here: http://colah.githu
150.
by colah3 12y ago
You make some pretty strong attacks on the state of the art approach to computer vision. Could you please provide citations? As someone working on neural networks, although not really on computer vision, I do not believe several of the thin

More ›