6 ms·
Compiling ML models to C for fun
- tekknolagi 3y agoThanks for posting! Happy to answer questions and curious what you all think
- loxias 3y agoI think this is no short of brilliant, and I immediately bookmarked it for further followup, because it's late. Missing piece. I don't do deep learning, but have been recently thinking about paths to optimize DAG things, user specified functions in a completely different context, and have been wondering if there's enough off the shelf code for me to glue together... I look forward to possibly standing on your shoulders to build something cool, and giving appropriate credit, should I post a "show HN" in 3mo-years. :)
- tekknolagi 3y agoOh this is far too kind, but I hope it does inspire you to build little compilers everywhere! I am curious about your idea(s); please feel free to reach out via email. EDIT: Also see the top-level comment by okaleniuk: https://news.ycombinator.com/item?id=37608605 https://news.ycombinator.com/item?id=37608605
- loxias 3y agoI'm a systems programmer, with experience writing high performance code for mathy stuff (signal processing), network stuff, filesystem stuff, and most recently, I'm learning-by-doing "database stuff". I was joking at work the other day "if I can get compilers under my belt I'll have caught them all". I see this applied to work (database) in a way I'll not elaborate until I can do it. :) But in other areas, I've done a lot of signal processing, numerical algorithm development and optimization, lots of MATLAB (for improving the underlying math, but also using approximation theory for performance. Think: "replace exp(x) with a taylor series" except .... more. :)) and C/C++ (for turning those graphs of lists of equations) into machine code. I've written dataflow style code starting from "paper of equations", to "runs on N nodes" many times. Perhaps then you see how this looks (to me) like a "missing piece". :) I can now mix between mathematical optimizations (like replacing an expensive function with a pade approximant, using a lattice of symbolic identities, etc) and _compiler_ optimizations. \o/
- tekknolagi 3y agoLooking forward to reading your query compiler :)
- choeger 3y agoNow even reading ML and compiling in the title is not a guarantee for ML the language. I hate overloading.
- tekknolagi 3y agoOne day I will write an SML compiler but today is not that day
- nerdponx 3y agoDoing ML in ML seems like an under-explored area.
- okaleniuk 3y agoI think this is brilliant! And, what's more important, practical. I did a similar thing a few years back for small linear systems: https://wordsandbuttons.online/outperforming_everything_with_anything.html https://wordsandbuttons.online/outperforming_everything_with... The idea is essentially the same - we both turn the computation process itself into code. I choose LLVM intermediate language instead of C because the whole point of my exercise was to show that you don't need C (or any other compiler) to generate efficient code. In fact, if you already have your computation figured out, C only gets in the way. For this very reason, I avoided calling the code generation process a "compiler". My thing, however, doesn't have any practical usage. This one does and I think it's beautiful!
- tekknolagi 3y agoThis is super cool! Thank you for sharing. I should probably try LLVM or some other IR like MLIR. I wanted to do MLIR originally but the Python bindings are a little unstable right now. Any other IR (not C) might allow me to express the intended semantics better than C does.
- mathisfun123 3y ago> I wanted to do MLIR originally but the Python bindings are a little unstable right now. Hmmm why do you think so? There's been one (small) breaking change in the last 6 months (and it came with a simple upgrade recommendation). (If it's not obvious, I contributedl to the python bindings but I would prefer to remain anon).
- tekknolagi 3y agoThe docs say not to use them :) I didn't try it out
- mathisfun123 3y agoreally? where does it say that? https://mlir.llvm.org/docs/Bindings/Python/ https://mlir.llvm.org/docs/Bindings/Python/ is it the "not enabled by default" that scares you off? i honestly don't even know what that's supposed to mean since there's no default anything since there's no official binary build of mlir currently being released. anyway i'm not trying to be confrontational or adversarial - i was more asking in a "how can i help you accomplish what you want" kind of tone.
- baq 3y agoMaybe you could generate Python and pass it to numba? In my very limited experiments writing numba Python is like writing C, this also includes segfaults etc. TCC compile speed to speedup ratio is bonkers BTW.
- tekknolagi 3y agoYeah, exactly! I mention this in the context of PyPy at the bottom of the post. Numba is another great suggestion here. Re: TCC: I know, right? I think it would be good to see how much we get from a) just not allocating the whole graph every time and b) doing it linearly/bytecode-style, both while still in Python-land.
- regularfry 3y agoI have a fairly strong suspicion you could generate ggml C code from python models directly. Not even for execution from python, just to give you a performant CPU model for any architecture rather than waiting for them to be manually ported. Not sure that's a job for me, though - maybe someone else has already done it and I just haven't seen it yet?
- tekknolagi 3y agoIs ggml an easy to use library? That would make generating tensor-based networks easier for sure
- JonChesterfield 3y agoSounds like triton
- montebicyclelo 3y agoThanks for this. My approach to speeding up an autodiff system like this was to write it in terms of nd-arrays rather than scalars, using numpy/cupy [1]. But it's still slower than deep learning frameworks that compile / fuse operations. Wondering how it compares to the approach in this post. (I might try to benchmark.) [1] https://github.com/sradc/SmallPebble https://github.com/sradc/SmallPebble
- deleted 3y ago[deleted]
- tekknolagi 3y agoPlease let us know the results for this network!
- tekknolagi 3y agoRe: Small Pebble: what is this Lazy thing? I've seen it crop up a couple places.
- montebicyclelo 3y agoThe underlying autodiff engine in smallpebble runs computations immediately, as they are defined, for the given inputs. But when modelling, you want to define a model, and then use it again and again, on different inputs. The laziness is a way to do that; you can define a graph of autodiff operations (using "sp.Lazy", and "sp.Placeholder"), without running it, and then pass in values and run the graph on those operations. We end up with something similar to how models are defined in tensorflow [1]. [1] https://www.tensorflow.org/js/guide/models_and_layers#the_functional_model https://www.tensorflow.org/js/guide/models_and_layers#the_fu...
- tekknolagi 3y agoAha! Okay, thank you. And how do you pass in the inputs differently?
- 3y ago
- vasilipupkin 3y agowhy not just use numpy which is already in C?
- tekknolagi 3y agoThat's covered at the bottom of the post (tensor-valued vs scalar-valued). It would make some of the math kernels faster, since numpy has a bunch of specializations built-in. But it would not speed up any of the connective "glue" bits that are still going to be slow. And it would still probably involve a bunch of allocations.