colah3

22 ms·

61.
by colah3 9y ago
Hi Jash! I focus my work on technical research directions I believe increase the probability of advances in AI being outstanding for all of humanity. I'm fortunate that Google provides me resources to do so, with broad autonomy to work
62.
by colah3 9y ago
Hello! I'm one of the authors; we'd be happy to answer any questions. Make sure to check out our library and the colab notebooks, which allow you to reproduce our results in your browser, on a free GPU, without any setup: https:&
63.
by colah3 9y ago
It seems to me that there may be a more benign explanation: Facebook/Twitter/etc are in a large-scale, iterative, adversarial game with many opponents. It might be relatively easy for humans to catch clusters of fake followers, bu
64.
by colah3 9y ago
When you pass a differentiable function into fold -- or most higher order functions, for that matter -- you get a function that is differentiable on everything but a measure zero set. The mechanics of how you compute the derivatives are sep
65.
by colah3 9y ago
Hi! I'm the author. I was also pretty surprised by the amount of attention this article has gotten. That said, I can give you my take on how it advanced the dialogue. As you note, the main contribution of this article is the diagrams -
66.
by colah3 9y ago
Yep, neural network interpretability research involves quite a bit of staring a pretty pictures. It's a difficult job, but someone has to look at all those images. :P
67.
by colah3 9y ago
You might find it interesting to look at Jason Yosinski's "deep vis" framework: http://yosinski.com/deepvis
68.
by colah3 9y ago
Our pleasure -- very glad our work is helpful. ;)
69.
by colah3 9y ago
We're glad you enjoyed it! :D > do you think it would be possible to train a DNN to learn to visualize the "most important" neuron activations / interactions of another DNN? That sounds like a really hard problem. I&#
70.
by colah3 9y ago
I'm incredibly lucky to be working with a number of people who have an amazing intersection of design skills and scientific knowledge. Ludwig is fantastic and put an incredible amount of work into polishing this article. And my co-edit
71.
by colah3 9y ago
Capsules are cool! I spent several days hanging out with Geoff & co a few months back -- they're exploring really exciting directions. I don't feel like I have anything particularly exciting to add, though.
72.
by colah3 9y ago
Hey! I'm one of the authors, along with Alex and Ludwig. We're happy to answer any questions! :)
73.
by colah3 9y ago
Hi, I'm the author. Please keep in mind that I wrote this when I was 16. Thanks for your comments -- I agree with many of them. The only part of this that I still think is at all interesting is the section on quantifiers, and to a less
74.
by colah3 9y ago
Other content in this article makes me a bit nervous. For example, it strongly implies at several points that the dental association has a conspiracy to keep patients using less safe technology because it leads to recurring surgeries: >
75.
by colah3 9y ago
Note that there are plenty of legitimate journals, like PLOS, that charge a fee to publish. It's a common business model for open access journals. Usually they have some mechanism for waiving the fee if an author can't afford it.
76.
by colah3 10y ago
As others mentioned, this is common in other fields, it's just not done in machine learning. We've been reading through the policies of lots of journals that seem thoughtfully run and borrowing good ideas. :P
77.
by colah3 10y ago
Yep, we're still clarifying our policies. If you have questions, please email us! (Distill needs to be extra careful about a lot of this stuff because we're trying to build legitimacy for a kind of work that many people are inclin
78.
by colah3 10y ago
Please check out our journal policies page: http://distill.pub/journal/ In brief, Distill needs to see three things to publish an article: outstanding communication, advancing the dialogue, and scientific integrity. Di
79.
by colah3 10y ago
As Gabe said, we mostly expect authors to produce diagrams and us to help edit them into outstanding articles. This is one example of the editing: https://github.com/distillpub/post--momentum/issues/1 We&#x27
80.
by colah3 10y ago
Thanks! We're just lucky to have authors like Gabe (@gabrielgoh) come to us with incredible articles. :)
81.
by colah3 10y ago
Hi! This article is about momentum in the mathematical field of optimization. Acceleration also refers to a phenomenon in optimization. While there are deep connections to their physical analogues, they aren't aren't quite the sam
82.
by colah3 10y ago
Thanks for pointing that out! We fixed the diagram bug in firefox. There's still bad performance for ~30s after page load -- we're looking into why that's happening -- but after that the page seems to work well.
83.
by colah3 10y ago
I agree that when you think about individual researchers it's a negative externality -- in many cases, the researcher doesn't pay, but other members of the research community do. This is true in programming as well: often the tech
84.
by colah3 10y ago
Yep. That's why a big part of Distill, behind the scenes, is a campaign of talking to senior faculty, scientific funders, and leaders of industry research groups, to build consensus that this is important. I'm not sure how success
85.
by colah3 10y ago
Well, I also intend to persuade scientific funders. :P
86.
by colah3 10y ago
Each Distill article is a github repository, so people can make pull requests, but we mostly encourage people to use that to make corrections. If it's technically feasible to do community editing of articles, why don't we encourag
87.
by colah3 10y ago
2 pi doesn't seem like a big deal to you because you've internalized the notation. You paid the cost a long time ago (and perhaps it was a small one for you). I think it can be hard to empathize with what it's like to be a be
88.
by colah3 10y ago
And because it doesn't lead to jobs or lots of citations. Fixing that is basically the goal of Distill: build an ecosystem where this kind of work is supported and rewarded.
89.
by colah3 10y ago
In the very long run, I'd like to see Distill or Distill-like journals cover all of math/cs/science. But I think the right approach is to start with a narrow topic and do really well there. I guess startup people would say th
90.
by colah3 10y ago
You're right that reviews and textbooks do valuable distillation, but I'd claim we can do much much better. I think we can distill things massively better -- create massively better explanations and ways of thinking about ideas --

More ›