patrickkidger

8 ms·

61.
by patrickkidger 4y ago
This is a great point I didn't cover! "Just know stuff" tends to follow naturally from "care about stuff".
62.
by patrickkidger 4y ago
Honestly, the two are now incredibly close. JAX introduced a lot of cool concepts (e.g. autobatching (vmap), autoparallel (pmap)) and supported a lot of things that PyTorch didn't (e.g. forward mode autodiff). And at least for my appli
63.
by patrickkidger 4y ago
> If you care about performance This definitely isn't true. On any benchmark I've tried, JAX and Julia basically match each other. Usually I find JAX to be a bit faster, but that might just be that I'm a bit more skilled a
64.
by patrickkidger 4y ago
At least prior to this announcement: JAX was much faster than PyTorch for differentiable physics. (Better JIT compiler; reduced Python-level overhead.) E.g for numerical ODE simulation, I've found that Diffrax ( https://githu
65.
by patrickkidger 4y ago
+1 for JAX. Basically designed to be the successor to TensorFlow, and much nicer to work with. Strangely I've not seen it discussed around HN much but it's what I do 100% of my work in these days. Whilst I'm here: shameless s
66.
by patrickkidger 4y ago
Autodifferentiable programming! Neural networks are the famous example of this, of course -- but this can be extended to all of scientific computing. ODE/SDE solvers, root-finding algorithms, LQP, molecular dynamics, ... These days I&#
67.
by patrickkidger 4y ago
There's lots of good reasons. Most notably the autodiff in Julia is unreliable -- there's been quite the litany of packages trying/failing to do this robustly and efficiently. (This issue pretty much kills its possibilities s
68.
by patrickkidger 4y ago
From personal experience: possibly Twitter. Having a presence there has given me several serious offers to interview, for jobs worth having (FAANG and similar). The folks in my field (ML/applied math/scientific computing) are all
69.
by patrickkidger 4y ago
Personally I'm a big fan of Julia's syntax here. Indentation-insensitive with the use of the word "end" as the terminator for all types of block. In practice you can lay it out like Python, and the "end"s help
70.
by patrickkidger 4y ago
I disagree. I've had a serious attempt at array typing using variadic generics and I'm not impressed. Python's type system has numerous issues... and now they just apply to any "ArrayWithNDimensions" type as well as
71.
by patrickkidger 4y ago
100% this. In a discussion on "cultural correctness issues prevents me from using Julia", it's very telling that the response is "more speed!" There's been a decent number of posts based around "Julia has
72.
by patrickkidger 4y ago
Excellent! I'm very happy to take contributions generalising the tool.
73.
by patrickkidger 4y ago
FWIW my take is not that Yuri is expressing "there are too many bugs" so much as he's expressing a problem in the culture surrounding Julia itself: > But systemic problems like this can rarely be solved from the bottom up,
74.
by patrickkidger 4y ago
You may already know of it, but if you want differential-equations-in-JAX then allow me to quickly advertise Diffrax: https://github.com/patrick-kidger/diffrax (of which I am the author, disclaimer).
75.
by patrickkidger 4y ago
Nope, this was on v0.2 of SimpleChains.jl; I did check. Anyway, something to investigate if I ever get the time.
76.
by patrickkidger 4y ago
You're discussing switching algorithms like LSODA? This is a really good question that I don't have a neat answer to. You can actually also hit similar issues when naively vectorising some stiff algorithms that detect when to reca
77.
by patrickkidger 4y ago
Not at all -- I was very happy to see some Equinox code out in the wild! I think the main slowdown was doing the model updates out of the JIT region. JAX-without-JIT is ridiculously slow.
78.
by patrickkidger 4y ago
I've just run this myself. Tidying up the JAX script, I find that it runs in ~15 seconds on my laptop CPU. Likewise, fixing the crash error in the blog post (UndefVarError: alloc_threaded_grad not defined) then I find that the Julia im
79.
by patrickkidger 4y ago
Taking a union of differential equations is a pretty bad idea for exactly the reasons you describe. But it's absolutely possible to parallelise multiple diffeq solves without needing to use the same time steps for each solve. That is,
80.
by patrickkidger 5y ago
I'm happy to hear about Equinox being used! (I'm the author.) I'm curious what your workloads are that you're seeing speedups of as much as 1e4? Greatest I've heard of before was ~1e2 on some differential equation s