mccoyb

17 ms·

211.
by mccoyb 2y ago
This is exactly why I decided to go do a PhD from industry.
212.
by mccoyb 2y ago
This website has some of the worst mobile UI I’ve ever seen — blasted by ads, and full screen subscription model — before I could even read the pitch.
213.
by mccoyb 2y ago
Why is this relevant when presenting scientific research? Or is the point of your comment to say, they are incentivized to "brand" their research in a way which is attractive to a VC audience?
214.
by mccoyb 2y ago
In the blog post, learned verifiers are mentioned. Are these learned offline using data, and is the intent to learn a scoring heuristic to help the search?
215.
by mccoyb 2y ago
Python is used here to wrap around some sort of kernel compiler (taichi). Not out of the realm of possibility that kernels which are compiled out of Python source code could be placed on device with some sort of minimal runtime (although ta
216.
by mccoyb 2y ago
Wow! thanks for the ref
217.
by mccoyb 2y ago
I think they’re likely using MLX in house now, no? (Probably not everyone, ofc - but seems likely that many will just use the native array framework designed explicitly for MX chips)
218.
by mccoyb 2y ago
Equinox has great idioms — really pioneered the Pytree perspective. Penzai is also great. JAX feels close to achieving a sort of high-level GPGPU ecosystem. Super fledgling — but I keep finding more little libraries that build on JAX (and c
219.
by mccoyb 2y ago
I can’t comment on the lowest leaf of this thread, but thanks for update! I’ll read through this section and see if my intuitions are wrong or right.
220.
by mccoyb 2y ago
Thanks! See above — I don’t think this is exactly dynamic shape support. My definition might be wrong, but I often think of full dynamic shape support as implying something dynamic about the computation graph. For instance, JAX supports a s
221.
by mccoyb 2y ago
Can you link dynamic shape support? Big if true — but I haven’t been able to find anything on it. Edit: I see — I think you mean exporting lowered StableHLO code in a shape polymorphic format —- from the docs: https://jax.readthe
222.
by mccoyb 2y ago
Can you share any code or references for this? Several people I work with are interested in this.
223.
by mccoyb 2y ago
No, I'm talking about custom `Jaxpr` interpreters which can modify programs to do things.
224.
by mccoyb 2y ago
Dex is also missing user authored composable program transformations, which is one of JAX’s hidden superpowers. So not quite “JAX without limitations” — but certainly without some of the limitations.
225.
by mccoyb 2y ago
Not quite a positive (it's ready now!) answer, but there's some interesting work on denoting problems, and constructing numerical methods for systems like the one you're describing -- I believe the design of this library (whi
226.
by mccoyb 2y ago
Agreed, JAX is specialized for GPU computation -- I'd really like similar capabilities with more permissive constructs, maybe even co-effect tagging of pieces of code (which part goes on GPU, which part goes on CPU), etc. I've tho
227.
by mccoyb 2y ago
I agree with you, I don’t think this title is helping with parsing what this is about, nor is it helping with discoverability.
228.
by mccoyb 2y ago
This is cool! Is the idea to put Kind2 on top of this in some way? I’d also love to find an example of writing a small interpreter in Bend - which runs on the GPU.
229.
by mccoyb 2y ago
That's cool -- but wouldn't it be more constructive to discuss "the ideas" in this package anyways? For instance, it would be interesting to discern if the design of PyTorch (and their modules) preclude or admit the same
230.
Penzai: JAX research toolkit for building, editing, and visualizing neural nets(github.com)
261 points by mccoyb 2y ago|51 comments
231.
by mccoyb 3y ago
Interested in the focus on comparison with PyTorch — when it seems like they should actually be comparing to JAX/XLA and IREE. Do they not know about the latter technologies?
232.
by mccoyb 3y ago
Hi Victor — exciting work! For language designers interested in this system, is there a common reference for e.g. using it (for example, from a language + lowering pipeline in Rust)? (Or perhaps that’s also what you alluded to in the next c
233.
by mccoyb 3y ago
I would start with this submission by my colleague Nishad Gothoskar to NeurIPS: https://proceedings.neurips.cc/paper/2021/hash/4fc66104f8ada... as an excellent starting point to what good inference abstractio
234.
by mccoyb 3y ago
I think you’re overgeneralizing in your control flow discussion. I also don’t understand your f example with (x, y, noise) if you fix x and the return value, you still have two unknowns with 1 equation. How is that easy to solve? Unless you
235.
by mccoyb 3y ago
I don’t think the field has converged on “the right abstractions” yet. It’s an active area of programming language research — it feels similar to where AD was at for awhile. I work on this stuff for my research — so I do believe that there
236.
by mccoyb 3y ago
One curious thing I discovered a few months ago: you can sort of hack higher-order functions into JAX by defining “Pytree closures” which introspect on normal closures, and pull out the JAX tracer data from the closure environment (and put
237.
by mccoyb 3y ago
Great point, I'll think deeper!
238.
by mccoyb 3y ago
One of my big issues with these computational artifacts is that they are not modular -- in the sense that, if I wanted a studious snake without claws -- what things do I need to do to get it to produce a version of the snake, but "fixe
239.
by mccoyb 3y ago
Why does the studious corn snake appear to have claws and whiskers?
240.
by mccoyb 4y ago
Anecdotally, I've been using GPT4 to help me get overviews of techniques in fields which I'm working adjacent to. I've been pleasantly surprised at its ability to quickly summarize not just ideas, but also write out the math.

More ›