chillee

23 ms·

181.
by chillee 7y ago
Oh shit, you're right. I fixed these properly at some point, but I made some last minute modifications to the text size and such. These interactive figures are probably a bit better overall too: https://chillee.github.io&#x2
182.
by chillee 7y ago
I talk about Jax in the article. It's very cool, especially if you need higher order derivatives. However, it's not meant to be a full neural network library, and unless Google invests significantly into it, it won't take off
183.
by chillee 7y ago
Why not? Ok, admittedly, there are a couple reasons. The fact that most papers don't mention the framework they use is a big one. So if users of one framework disproportionately mentioned that framework in their paper, it would be over
184.
by chillee 7y ago
MXNet is actually pretty good. It got to the "mixing eager and graph mode" semantics before either PyTorch or TensorFlow did. On top of that, it's also blazing fast (usually the fastest of the frameworks). Admittedly, I'
185.
by chillee 7y ago
Well, you could read the article :^) As a summary, though: PyTorch has become dominant in research because of its API (both its stability + having eager mode). TF has become dominant in industry because A. it came out several years before P
186.
by chillee 7y ago
People are reluctant to use Folly for the same reason people are reluctant to use Boost - except that using Folly also implies a dependency on Boost.
187.
by chillee 7y ago
The biggest thing is using lower precision > this implementation gives 25% speedup over Nvidia's Pytorch implementation in full precision and 2.5-3x speedup when using TensorCore TensorCore is a lower precision core. Other than that
188.
by chillee 7y ago
There's a discussion here of sorts: https://www.reddit.com/r/MachineLearning/comments/cvcbu6/d_w... Perhaps Google could copy PyTorch. But Tensorflow has a lot of overhead, and for both political&#x
189.
by chillee 7y ago
I don't disagree that a gradient exists. As the other commenter commented, the gradient/subgradient will usually exist. What I'm arguing is that this gradient will not allow you to optimize anything of interest for the vast m
190.
by chillee 7y ago
It's not meaningful in the sense that it won't correspond with your intuition of "will increasing the input increase my output". Presumably the point of differentiable programming isn't just getting the derivative f
191.
by chillee 7y ago
Other people have answered what they do, but this is the big gap between people talking about 'differentiable programming' in theory, and having it actually work in practice. It's true that once you have control flow, the gra
192.
by chillee 7y ago
If statements aren't really meaningfully differentiable, regardless of how you do it. Take if x == 59: return 1000 else if x > 59: return -x else: return x How do you optimize this to maxim
193.
by chillee 7y ago
PyTorch is dominant for research, Tensorflow is dominant in industry. I'm writing an article about the general state of ML frameworks that's nearly done.
194.
by chillee 7y ago
Speaking as someone doing research in this field, I have an unbelievably hard time imagining this to be the case. The ML community is generally extremely open, and people know what the other top people are working on. If an AGI was develope
195.
by chillee 7y ago
Nitpick: I understand your point in general, but it's not actually true that you need to calculate the n-1 and n-2 values first for Fibonacci. You can use Binet's formula or matrix exponentiation to calculate it without such a dep
196.
by chillee 7y ago
I suppose the reason why PyTorch doesn't implement this boiler plate for you is due to all of these quirky edge cases :) I think Pytorch Lightning might be worth taking a look at: https://github.com/williamFalcon/p
197.
by chillee 7y ago
I'm not saying that no data science practitioners care about speed and safety, and its easy for me to come up with cases where they should/do care. My point is that even fast.ai still views S4TF as somewhat niche, and that data
198.
by chillee 7y ago
Just because fast.ai has some investment in Swift does not mean that S4TF has attracted mind share. The bulk of fast.ai is still taught on Pytorch.
199.
by chillee 7y ago
There's 3 fundamental gaps between Numpy and NN libraries (other commenters have jointly mentioned 2/3) 1. Numpy doesn't run on GPU. 2. Numpy isn't high level enough for NN building. 3. Numpy doesn't have auto diffe
200.
by chillee 7y ago
I'm not sure I would consider these an unbiased opinion on China...
201.
by chillee 7y ago
Recycling is a conspiracy? Can I get a link to that?
202.
by chillee 7y ago
A timer simply reports the current time. In this context, people are using timers to calculate how long an operation takes. Resolution is the precision that the timer reports the time. For example, it could report seconds elapsed (e.g: 8 se
203.
by chillee 8y ago
VSCode has seriously dominated these lists since its release in 2016. Overall: 2016: 7.2% 2017: N/A 2018: 34.9% 2019: 50.7% Web: 2016: Unranked 2017: 24.0% 2018: 38.7% 2019: 55.6% Mobile: 2017: N/A 2018: 36.6% 2019: 53.8% Sysadm
204.
by chillee 8y ago
I did make some slides here for my university's ICPC seminar: http://www.cs.cornell.edu/courses/cs5199/2019sp/ https://docs.google.com/presentation/d/1c1YUdhXOCLSVhH6Jn4TB... I
205.
by chillee 8y ago
What's your straightforward recursive solution? It might be what's considered Tree DP. If you want a harder problem I can give an example too :^)
206.
by chillee 8y ago
Well, A. Not exactly because there's no restrictions on when you can color nodes black, and B. Graph coloring in general is NP-complete while this problem has an O(n) solution.
207.
by chillee 8y ago
Fibonacci isn't considered Tree DP. One example of a tree DP problem is: Given a tree with N nodes, how many ways are there of coloring each node with black or white given that no two white nodes are adjacent.
208.
by chillee 8y ago
Yes, there's a ton of DP problems that require clever tricks. I've had to show these tricks to friends who believed that DP problems were too formulaic. Could you give an example of DP problems that show up in interviews that requ
209.
by chillee 8y ago
Disagree. The general concept is the same - "Where can you collapse states into one". Someone who understands DP well should find tree DP natural, and might even come up with it themselves without learning about it beforehand. I&#
210.
by chillee 8y ago
That's not really true. There's plenty of DP solutions that are >cubic or exponential.

More ›