8 ms·

I've decided to move on to Distill

353 points by apsec112 10y ago

dave_sullivan 10y ago
Congrats on Distill Chris, sounds awesome. > I also have unusual experiences, like dropping out of university to support an accused terrorist, that I might like to write about. But it feels kind of like an abuse to use the attention the deep learning community has given me as a platform for these other topics. You've got the attention of the deep learning community because you write about interesting stuff; I hope you write more about your non-DL experiences and I think a lot of people (both inside and outside the DL community) would find them very interesting.
jasim 10y ago
"I began to worry that, by focusing on exposition, I was failing to do the best work I could. Often, my work felt illegitimate in some way" This could be worthy of discussion and how the author has responded to it is commendable.
- colah3 10y ago
  Thank you. Talking to others in the space, I think a number of people share my experience, and often stopped doing this kind of work. While there's extremely positive feedback from some people, it's kind of "you have a cool hobby" and it's kind of iffy how much it helps professionally. In some cases, I know people who have had well-intentioned senior researchers, or their advisor, encourage them to switch to focusing on traditional papers. In a smaller number of cases, I know people have received explicitly negative feedback. Sort of, "those who can, do research; those who can not, explain research." My career has worked out really well, but it's unclear how much of that is luck. I think we can support people better.
  - kalid 10y ago
    Hi Chris, I wanted to applaud your efforts here. The attitude of "those who can't do, teach" misses how we truly internalize and make use of ideas. Luminaries in a field (Einstein, Feynman, Tao) have "post-rigorous" understanding (https://terrytao.wordpress.com/career-advice/there%E2%80%99s-more-to-mathematics-than-rigour-and-proofs/ https://terrytao.wordpress.com/career-advice/there%E2%80%99s...). Pure research without sufficient explanation doesn't help others build this level of depth and leaves the findings as a black box.
  - joshuamorton 10y ago
    I think it is worth noting for everyone that this really is a form of impostor syndrome, and people should take note that even someone who is very highly regarded is susceptible to that kind of thing. For what its worth, having dabbled in some visualizations of concepts, its really difficult, and I didn't find that the available tools were well suited for mathematical visualization. Its really unfortunate that to be able to present something intuitively, there's a very high technical skill floor as well as a relatively high artistic floor.
    - reachtarunhere 10y ago
      You are spot on about the impostor syndrome. Colah's confession about being worried bring me some sort of relief given I hold him in really high regard. I am starting out in this area (still an undergrad) and feel really scared at times about what I am doing and at times feel I land up into good internships/projects just by sheer luck. When someone like him who is respected across the community can have these doubts I feel it is completely OK for mere mortals like me have the same issues. On similar lines, the fact that each of his posts takes 50-200 hours of work is another reality check. I always thought he was writing these on the side in 2-3 hours. This made me believe I could not with years of practice produce such great work. I am still not sure if I would be able to but at least I can now set realistic expectations for myself. Also, knowing it took so much work on his end has made me respect him even more.
  - ssivark 10y ago
    Something that's bothered me a lot through my research career is how much effort it takes to place any particular result in context (and fit all the different pieces together into a coherent whole). And how one can understand the full edifice much better once one has that context in which to interpret recent progress. Of course, that layout only becomes clear only after the area has developed a little, but IMHO the most important education that can be imparted to newcomers in a field is that "context". It pains me to see that good expository material that share insight seem to be hard to find, particularly on front-line research. Broadly, I feel that research communities ought to do a better job of writing accessible review material -- otherwise they inevitably fracture into sub-communities which start talking past each other.
  - aaachilless 10y ago
    > "those who can, do research; those who can not, explain research." I've rewritten this comment maybe 10 times because I'm struggling to capture how completely myopic this idea is. But here's a quote from Feynman's 5th messenger lecture [1] that I really enjoy, he's talking about the graph of all possible human endeavors: [The graph] is a series of concepts that we use to understand things at an ever higher level. And going on, we come to things like evil and beauty and hope. Now, which end is nearer to the ultimate creator or the ultimate or, if I make a religious metaphor, which end is nearer to God? Beauty and hope, or the fundamental laws? I think that the right way, of course, is to say that the whole structural interconnections of the thing is the thing that we have to look at. And that all the sciences and all the efforts-- not just the sciences but all the efforts of intellectual kind-- are to see the connections of the hierarchies is to connect beauty to history, to connect history to man's psychology, the man's psychology to the working of the brain, the brain to the neural impulse, the neural impulse to the chemistry, and so forth, up and down, both ways. And today, we cannot-- and there's no use making believe we can-- draw carefully a line all the way from one end of this thing to the other. In fact, we've just begun to see that there is this relative hierarchy. And so I don't think either end is nearest to God. And to stand at either end and to look out off the end of the pier only, hoping out in that direction is the complete understanding, is a mistake. And to stand with evil and beauty and hope, or the stand with the fundamental laws, hoping that way to get a deep understanding of the whole world with that aspect alone is a mistake. And it is not sensible, either, for the ones who specialize at one end and the ones who specialize at the other end to have such disregard for each other. They don't, actually. But the people say they do. Sorry. [LAUGHTER] But actually, the great mass of workers in between, connecting one step to another, are improving all the time our understanding of the world, both from working at the ends and working in the middle. And in that way, we are gradually understanding this connection, this tremendous world of interconnecting hierarchies. Thank you. [1] http://www.cornell.edu/video/richard-feynman-messenger-lecture-5-distinction-past-future http://www.cornell.edu/video/richard-feynman-messenger-lectu...
tedmiston 10y ago
> As colah.github.io has grown in prominence… Well, it’s been very rewarding, but it’s also been a bit uncomfortable. > For one thing, it’s created very high standards for my writing. Most of my articles took 50-200 hours to write. I feel like I needed to live up to that quality with every post, but that means I can’t put out thoughts without investing a huge amount of energy. I've done a bit of blogging and this echoes my experience as well. One successful posts precedes another, then you start to feel like you can't put out off-the-cuff type posts. It's tough to follow up a rockstar post with an intermezzo. So you invest more and more energy into each topic and post to ensure it'll be successful in whatever metric, and that mostly works, but then you publish few posts in very specific niches. If anyone has advice for effectively blowing one's standards out of the water and publishing more smaller general pieces, I'd love to hear that. From this perspective, I admire how Fred Wilson blogs. Edit: fixed typo
- deleted 10y ago
  [deleted]
- kalid 10y ago
  I had this issue too, and realized you can frame it by setting expectations in the title. Some posts have needed the 100-200 hours of thinking/visualization, labeled with things like "Interactive Guide to the Fourier Transform" (https://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/ https://betterexplained.com/articles/an-interactive-guide-to...) Others are smaller insights, written in a day ("Quick Insight: Easier Arithmetic With Calculus" -https://betterexplained.com/articles/calculus-arithmetic/ https://betterexplained.com/articles/calculus-arithmetic/). Others are more essays/thought pieces, learning strategies, etc. Setting the expectation up front that not every post is a big thesis-style conclusion can take away the pressure to only create those. (You can have appetizers, entrees, and desserts on the menu!) Often your big articles are built after working through ideas in smaller ones. (The Fourier Transform emerged after getting insights on e, imaginary numbers, Euler's Formula, degrees vs. radians, etc.).
- zem 10y ago
  two bloggers i admire tremendously are james hague [http://prog21.dadgum.com/ http://prog21.dadgum.com/, now retired from blogging :(] and julia evans [https://jvns.ca/ https://jvns.ca/] while pretty much everything they write is high-quality and worth reading, there are clearly different levels of effort that went into any given post, so there's a good mix of content.
dimatura 10y ago
As a grad student in the field, I 100% agree with the idea of research debt. Arxiv is great, but it has only exacerbated the problem. There's far too many papers adding some small fluorish on an already complex architecture in order to push an accuracy score a couple of percentage points higher, and not that many with insightful examination of some more basic question, even if it's considered a detail.
- unixhero 10y ago
  That is the point of traditional academic publishing. So I don't believe it's comparable to blogging platform.
russdpale 10y ago
I would argue that the dissemination of complex information to the masses is of the utmost importance. How can humans make decisions if they don't understand what they are deciding about. His contribution is extremely important in advancing these fields. We need people to do this outside of academia if the populous is going to be truly informed about such important matters. Bill Nye and Neil Tyson may be cheesy, but its hard to argue that they aren't important in getting millions of kids into science at a young age, and helping people of all ages understand the science around them.
- sdflkd 10y ago
  Neil Tyson is beyond cheesy. He's a pendant and gives the wrong idea of what acting intelligent is to youth. [1] It's a sign of immaturity and a desire to appear profound. [1] https://twitter.com/neiltyson/status/704330815321210884?lang=en https://twitter.com/neiltyson/status/704330815321210884?lang...
  - jamescostian 10y ago
    It's extremely hard to take this comment seriously when everything you fault Tyson for could easily be launched right back at you. You call Tyson "a pedant" because he tweeted saying "Leap Day" was a bad word choice. But you are also saying the post you replied to shouldn't have the word "cheesy" so doesn't that make you a pedant as well? Sure, your point is not simply that the word "cheesy" is bad - you're going a bit more in-depth and trying to make a bigger point (specifically that Tyson is harmful). But Tyson is not simply trying to feel smart by saying he doesn't like the name "Leap Day" - he's also trying to make a bigger point (specifically, he's teaching/reminding us that our concept of a year is based on our planets orbit, not an arbitrary number). Similarly, you comment that Tyson "gives the wrong idea of what acting intelligent is" but did you notice that you said "intelligent" instead of "intelligently" despite the latter being correct? Aren't you giving the wrong idea of what acting intelligently is to the youth by making gramatical errors that the fragile youth may mimic? Looking at someone under a microscope, as I have just done right back at you, is not the best way to judge them. Your typographical error will not hurt "the youth" just like Tyson starting a tweet off in a clickbait style with "The Leap Day is misnamed" won't hurt us either.
mark_l_watson 10y ago
Thanks for your past posts. Looking forward to Distill!
mathgenius 10y ago
This needs to happen in all the fields of science. Academia rewards "complicated and tricky", not anything that makes it look easy. This is a serious problem! The cult of science. Not that I should pick out science in particular, it seems that humanity in general wants to make corrupt little circles of influence out of all its endeavors. PS. Yay Colah, love your stuff! It makes me happy that people like you see the value in this work.

353 points by apsec112 10y ago

Congrats on Distill Chris, sounds awesome. > I also have unusual experiences, like dropping out of university to support an accused terrorist, that I might like to write about. But it feels kind of like an abuse to use the attention the deep learning community has given me as a platform for these other topics. You've got the attention of the deep learning community because you write about interesting stuff; I hope you write more about your non-DL experiences and I think a lot of people (both inside and outside the DL community) would find them very interesting.

jasim 10y ago

"I began to worry that, by focusing on exposition, I was failing to do the best work I could. Often, my work felt illegitimate in some way" This could be worthy of discussion and how the author has responded to it is commendable.

colah3 10y ago
Thank you. Talking to others in the space, I think a number of people share my experience, and often stopped doing this kind of work. While there's extremely positive feedback from some people, it's kind of "you have a cool hobby" and it's kind of iffy how much it helps professionally. In some cases, I know people who have had well-intentioned senior researchers, or their advisor, encourage them to switch to focusing on traditional papers. In a smaller number of cases, I know people have received explicitly negative feedback. Sort of, "those who can, do research; those who can not, explain research." My career has worked out really well, but it's unclear how much of that is luck. I think we can support people better.
- kalid 10y ago
  Hi Chris, I wanted to applaud your efforts here. The attitude of "those who can't do, teach" misses how we truly internalize and make use of ideas. Luminaries in a field (Einstein, Feynman, Tao) have "post-rigorous" understanding (https://terrytao.wordpress.com/career-advice/there%E2%80%99s-more-to-mathematics-than-rigour-and-proofs/ https://terrytao.wordpress.com/career-advice/there%E2%80%99s...). Pure research without sufficient explanation doesn't help others build this level of depth and leaves the findings as a black box.
- joshuamorton 10y ago
  I think it is worth noting for everyone that this really is a form of impostor syndrome, and people should take note that even someone who is very highly regarded is susceptible to that kind of thing. For what its worth, having dabbled in some visualizations of concepts, its really difficult, and I didn't find that the available tools were well suited for mathematical visualization. Its really unfortunate that to be able to present something intuitively, there's a very high technical skill floor as well as a relatively high artistic floor.
  - reachtarunhere 10y ago
    You are spot on about the impostor syndrome. Colah's confession about being worried bring me some sort of relief given I hold him in really high regard. I am starting out in this area (still an undergrad) and feel really scared at times about what I am doing and at times feel I land up into good internships/projects just by sheer luck. When someone like him who is respected across the community can have these doubts I feel it is completely OK for mere mortals like me have the same issues. On similar lines, the fact that each of his posts takes 50-200 hours of work is another reality check. I always thought he was writing these on the side in 2-3 hours. This made me believe I could not with years of practice produce such great work. I am still not sure if I would be able to but at least I can now set realistic expectations for myself. Also, knowing it took so much work on his end has made me respect him even more.
- ssivark 10y ago
  Something that's bothered me a lot through my research career is how much effort it takes to place any particular result in context (and fit all the different pieces together into a coherent whole). And how one can understand the full edifice much better once one has that context in which to interpret recent progress. Of course, that layout only becomes clear only after the area has developed a little, but IMHO the most important education that can be imparted to newcomers in a field is that "context". It pains me to see that good expository material that share insight seem to be hard to find, particularly on front-line research. Broadly, I feel that research communities ought to do a better job of writing accessible review material -- otherwise they inevitably fracture into sub-communities which start talking past each other.
- aaachilless 10y ago
  > "those who can, do research; those who can not, explain research." I've rewritten this comment maybe 10 times because I'm struggling to capture how completely myopic this idea is. But here's a quote from Feynman's 5th messenger lecture [1] that I really enjoy, he's talking about the graph of all possible human endeavors: [The graph] is a series of concepts that we use to understand things at an ever higher level. And going on, we come to things like evil and beauty and hope. Now, which end is nearer to the ultimate creator or the ultimate or, if I make a religious metaphor, which end is nearer to God? Beauty and hope, or the fundamental laws? I think that the right way, of course, is to say that the whole structural interconnections of the thing is the thing that we have to look at. And that all the sciences and all the efforts-- not just the sciences but all the efforts of intellectual kind-- are to see the connections of the hierarchies is to connect beauty to history, to connect history to man's psychology, the man's psychology to the working of the brain, the brain to the neural impulse, the neural impulse to the chemistry, and so forth, up and down, both ways. And today, we cannot-- and there's no use making believe we can-- draw carefully a line all the way from one end of this thing to the other. In fact, we've just begun to see that there is this relative hierarchy. And so I don't think either end is nearest to God. And to stand at either end and to look out off the end of the pier only, hoping out in that direction is the complete understanding, is a mistake. And to stand with evil and beauty and hope, or the stand with the fundamental laws, hoping that way to get a deep understanding of the whole world with that aspect alone is a mistake. And it is not sensible, either, for the ones who specialize at one end and the ones who specialize at the other end to have such disregard for each other. They don't, actually. But the people say they do. Sorry. [LAUGHTER] But actually, the great mass of workers in between, connecting one step to another, are improving all the time our understanding of the world, both from working at the ends and working in the middle. And in that way, we are gradually understanding this connection, this tremendous world of interconnecting hierarchies. Thank you. [1] http://www.cornell.edu/video/richard-feynman-messenger-lecture-5-distinction-past-future http://www.cornell.edu/video/richard-feynman-messenger-lectu...

tedmiston 10y ago

> As colah.github.io has grown in prominence… Well, it’s been very rewarding, but it’s also been a bit uncomfortable. > For one thing, it’s created very high standards for my writing. Most of my articles took 50-200 hours to write. I feel like I needed to live up to that quality with every post, but that means I can’t put out thoughts without investing a huge amount of energy. I've done a bit of blogging and this echoes my experience as well. One successful posts precedes another, then you start to feel like you can't put out off-the-cuff type posts. It's tough to follow up a rockstar post with an intermezzo. So you invest more and more energy into each topic and post to ensure it'll be successful in whatever metric, and that mostly works, but then you publish few posts in very specific niches. If anyone has advice for effectively blowing one's standards out of the water and publishing more smaller general pieces, I'd love to hear that. From this perspective, I admire how Fred Wilson blogs. Edit: fixed typo

deleted 10y ago
[deleted]
kalid 10y ago
I had this issue too, and realized you can frame it by setting expectations in the title. Some posts have needed the 100-200 hours of thinking/visualization, labeled with things like "Interactive Guide to the Fourier Transform" (https://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/ https://betterexplained.com/articles/an-interactive-guide-to...) Others are smaller insights, written in a day ("Quick Insight: Easier Arithmetic With Calculus" -https://betterexplained.com/articles/calculus-arithmetic/ https://betterexplained.com/articles/calculus-arithmetic/). Others are more essays/thought pieces, learning strategies, etc. Setting the expectation up front that not every post is a big thesis-style conclusion can take away the pressure to only create those. (You can have appetizers, entrees, and desserts on the menu!) Often your big articles are built after working through ideas in smaller ones. (The Fourier Transform emerged after getting insights on e, imaginary numbers, Euler's Formula, degrees vs. radians, etc.).
zem 10y ago
two bloggers i admire tremendously are james hague [http://prog21.dadgum.com/ http://prog21.dadgum.com/, now retired from blogging :(] and julia evans [https://jvns.ca/ https://jvns.ca/] while pretty much everything they write is high-quality and worth reading, there are clearly different levels of effort that went into any given post, so there's a good mix of content.

dimatura 10y ago

As a grad student in the field, I 100% agree with the idea of research debt. Arxiv is great, but it has only exacerbated the problem. There's far too many papers adding some small fluorish on an already complex architecture in order to push an accuracy score a couple of percentage points higher, and not that many with insightful examination of some more basic question, even if it's considered a detail.

unixhero 10y ago
That is the point of traditional academic publishing. So I don't believe it's comparable to blogging platform.

russdpale 10y ago

I would argue that the dissemination of complex information to the masses is of the utmost importance. How can humans make decisions if they don't understand what they are deciding about. His contribution is extremely important in advancing these fields. We need people to do this outside of academia if the populous is going to be truly informed about such important matters. Bill Nye and Neil Tyson may be cheesy, but its hard to argue that they aren't important in getting millions of kids into science at a young age, and helping people of all ages understand the science around them.

sdflkd 10y ago
Neil Tyson is beyond cheesy. He's a pendant and gives the wrong idea of what acting intelligent is to youth. [1] It's a sign of immaturity and a desire to appear profound. [1] https://twitter.com/neiltyson/status/704330815321210884?lang=en https://twitter.com/neiltyson/status/704330815321210884?lang...
- jamescostian 10y ago
  It's extremely hard to take this comment seriously when everything you fault Tyson for could easily be launched right back at you. You call Tyson "a pedant" because he tweeted saying "Leap Day" was a bad word choice. But you are also saying the post you replied to shouldn't have the word "cheesy" so doesn't that make you a pedant as well? Sure, your point is not simply that the word "cheesy" is bad - you're going a bit more in-depth and trying to make a bigger point (specifically that Tyson is harmful). But Tyson is not simply trying to feel smart by saying he doesn't like the name "Leap Day" - he's also trying to make a bigger point (specifically, he's teaching/reminding us that our concept of a year is based on our planets orbit, not an arbitrary number). Similarly, you comment that Tyson "gives the wrong idea of what acting intelligent is" but did you notice that you said "intelligent" instead of "intelligently" despite the latter being correct? Aren't you giving the wrong idea of what acting intelligently is to the youth by making gramatical errors that the fragile youth may mimic? Looking at someone under a microscope, as I have just done right back at you, is not the best way to judge them. Your typographical error will not hurt "the youth" just like Tyson starting a tweet off in a clickbait style with "The Leap Day is misnamed" won't hurt us either.

mark_l_watson 10y ago

Thanks for your past posts. Looking forward to Distill!

mathgenius 10y ago

This needs to happen in all the fields of science. Academia rewards "complicated and tricky", not anything that makes it look easy. This is a serious problem! The cult of science. Not that I should pick out science in particular, it seems that humanity in general wants to make corrupt little circles of influence out of all its endeavors. PS. Yay Colah, love your stuff! It makes me happy that people like you see the value in this work.