brrrrrm

23 ms·

271.
by brrrrrm 4y ago
I disagree, the burden is almost exclusively maintaining fast implementations of primitive operators for all hardware. These ML libraries are collections of pure functions with minimal interfaces. There's very little code interdepend
272.
by brrrrrm 4y ago
A somewhat related, visual guide to discrete 2d convolutions: https://ezyang.github.io/convolution-visualizer/
273.
by brrrrrm 4y ago
that's a good point. I guess as an addendum it's not just compute efficiency but also "statistical efficiency" (if that has any meaning?)
274.
by brrrrrm 4y ago
I'm heavily involved in this area of research (getting deep learning competitive with computationally efficient statistical methods), and I'd like to note a couple things I've found: 1. Deep learning doesn't require thor
275.
by brrrrrm 4y ago
what's the deal with wasi-nn? (who is supporting it?) Seems super high level and somewhat brittle
276.
by brrrrrm 4y ago
Why do you say that? I assume the timeit magic reduces a lot of overhead
277.
by brrrrrm 4y ago
it's neat, but it's kinda slow (25x slower than native when I tested it) In a JupyterLyte notebook: import numpy as np a = np.random.randn(128,128) b = np.random.randn(128,128) %timeit a @ b 1.5 ms ± 3.3 µs p
278.
Async Good(jott.live)
1 points by brrrrrm 4y ago|0 comments
279.
by brrrrrm 4y ago
Thanks for flagging!
280.
Machine Learning with Unix Pipes(jott.live)
2 points by brrrrrm 4y ago|1 comments
281.
by brrrrrm 4y ago
it's not a matter of performance but expressibility. const y = await remote_model_a(x) // different machine const z = await remote_model_b(y) // different machine await z.backward() is trivially pipelined
282.
by brrrrrm 4y ago
Specifically distributed would be helped (multiple nodes). Right now the typical paradigm is this really clunky lock-step across nodes waiting for each other (everything is blocking). If your hardware is non-homogenous (both interconnects
283.
by brrrrrm 4y ago
All of distributed machine learning would benefit from async, but it's unfortunately a Python-only ecosystem for the time being. Probably gonna be a couple years before that has a chance of changing
284.
by brrrrrm 4y ago
what do you mean "block"? If you call an async function from a non-async context it doesn't block.
285.
by brrrrrm 4y ago
I think the issue is more like this: 1. You jump into a huge codebase and find a very useful function you want to use in your code (it uses asyncio) 2. Your code doesn't use asyncio, so you start converting functions to be async (or el
286.
by brrrrrm 4y ago
I feel the same way, it's a lot like `const` in C++. Basically it's enforcing usage for every upstream user to ensure there's no inefficiency (such as waiting for things like the network). JavaScript has an extremely nice &q
287.
by brrrrrm 4y ago
Important to note: > They are suited to non-blocking I/O with subprocesses and sockets, however, blocking I/O and CPU-bound tasks can be used in a simulated non-blocking manner using threads and processes under the covers. If y
288.
Fast Matrix Multiplication (Animated)(jott.live)
5 points by brrrrrm 4y ago|0 comments
289.
by brrrrrm 4y ago
In my opinion performance optimization should be taught from the ground up, rather than with lists of "performance rules." I found learning this stuff that a solid mental model of the compute resources available (registers, cache
290.
by brrrrrm 4y ago
Yep! This is an assumed optimization when it comes to modern linear algebra compilers. New primitives go way beyond FMAs: full matrix multiplies on nvidia/Intel and outer product accumulates on Apple silicon. It’s also expected that
291.
by brrrrrm 4y ago
avoiding writes to memory and reducing the number of loops (although not FLOPs) for j in range(10): c[j] = a[j] + b[j] for j in range(10): d[j] = c[j] * 2 becomes for j in range(10): d[j] = (a[j] + b[j])
292.
by brrrrrm 4y ago
> tread off the beaten path things get slow Yea, makes sense. I think there's something to be said for dynamic compilation solving this problem more elegantly than providing tons of hand-tuned kernels (PyTorch is 890MB lmao https:
293.
by brrrrrm 4y ago
The only thing I'd recommend is exposing "eval()" or something to let users tell you when they want you to evaluate things. It'll save a ton of time when it comes to hot-fixing performance and memory use issues. It&#x2
294.
by brrrrrm 4y ago
Doesn’t really matter for large batch/large model training on GPUs that don’t need much coordination. But Python speed is one of the main motivations for a JS/TS based ML lib I’m working on: https://github.com/face
295.
by brrrrrm 4y ago
> It compiles a custom kernel for every operation, allowing extreme shape specialization. This doesn't matter. Just look at the performance achieved by CuDNN kernels (which back PyTorch), they're dynamically shaped and hit nea
296.
by brrrrrm 4y ago
I've done some work in the past on NN representations and you actually can represent Conv and MatMul in more primitive ways. I ended up writing an IR called loop_tool that exposes this stuff: https://github.com/faceboo
297.
Ghstack: Submit stacked diffs to GitHub on the command line(github.com)
2 points by brrrrrm 4y ago|0 comments
298.
by brrrrrm 4y ago
You've presented a false dichotomy between "web" and "embedded systems". C++ fuels the engines behind Python and JavaScript. Databases, compilers, machine learning libraries: all C++ and actively maintained.
299.
by brrrrrm 4y ago
I feel like JAX usually comes up on PyTorch posts (in the same way PyTorch usually comes up in the comments on TF posts). I think the delta between JAX and PT on commodity hardware is just a little bit too small to really create much splash
300.
by brrrrrm 4y ago
most of the machine learning industry uses it almost exclusively as the driving language for GPUs

More ›