4 ms·

MLIR For Beginners: A series of articles on the MLIR framework

122 points by matt_d 3y ago

anlunx 3y ago
How does it compare with the Toy dialect tutorial?
- mathisfun123 3y ago
  kudos to jeremy for putting pen to paper but i do believe the toy tutorial is better (albeit more comprehensive). also i don't like bazel at all.
  - j2kun 3y ago
    I tried to learn from the toy tutorial and just found it to jump into the deep end too quick, without enough explanation for me to start writing code (because everything I touched in `toy` caused the build to break in mysterious ways).
    - the_svd_doctor 3y ago
      Same opinion. I have little/no exposure to compilers (yes I know, maybe I'm not the right target) and tried to follow Toy and had to resort to a lot of wikipedia/google just to understand the concepts and terms. And after finishing it I was still a bit confused about how to even get started. I think it could be made a bit more beginner friendly. I found the standalone example https://github.com/llvm/llvm-project/tree/main/mlir/examples/standalone https://github.com/llvm/llvm-project/tree/main/mlir/examples... in the MLIR repo to be very useful, on the other hand.
- amoss 3y ago
  I'm finding the Toy tutorial easier to read, and easier to use as a jumping off point into the MLIR reference docs. I thought the writing style in this tutorial was a bit dense, and Bazel seems like an unnecessary complication. Only an opinion though, YMMV.
synergy20 3y ago
two years ago MLIR was a big deal for the company I worked but then it went silent, the compiler team switched to something else for RISC-V ML design, not sure what it is though.
cratermoon 3y ago
The README could benefit from a description of what MLIR is.
- kQq9oHeAz6wLLS 3y ago
  My thought as well. Not exactly "for beginners" in a literal sense.
- cabalamat 3y ago
  This is true for most READMEs on github projects. Is it really too hard to write a paragraph giving the context?
- mathisfun123 3y ago
  i don't understand the point of this criticism - i swear people just post it because they don't know what the thing is and they feel they need to compulsively comment anyway. no one stumbles on this repo by accident (jeremy didn't post it here). the only way you find it is you google "mlir tutorial" ie you know what MLIR is and you're actively explicitly looking for a tut. there is no situation where you're ambushed by this content. like if i write a tutorial "integration by parts for beginners" but don't define integration are you gonna claim that it fails at being for beginners? does every title from now on need to explicitly have "... for beginners who know what this is and have already made the decision to start studying".
  - noman-land 3y ago
    It's funny, I agree with both of you. Considering this repo was posted to this site and prompted thousands of people to stumble on it by accident, even a link in the README on the words "MLIR framework" would go a long way. I, as someone who is both super interested in applied FHE and subscribed to this guy's blog, almost skipped this post because I didn't realize it was about FHE, and I am probably in the target audience for it.
    - mathisfun123 3y ago
      > Considering this repo was posted to this site and prompted thousands of people to stumble on it by accident but the guy himself didn't post it. how are people so entitled that they're upset about a deficiency in a thing that they sought out. no one is making you go the repo (or asking you to go). it's literally like complaining about free samples at costco - if you don't like what they have don't take one!
dmoy 3y ago
y'all should open source the Kythe indexers for that and tablegen
marssaxman 3y ago
Glad to see more work going into MLIR documentation. It's a powerful system, but it can be difficult to get a sense of what normal usage looks like.
smcleod 3y ago
Weird that it’s for beginners without saying what it actually is. Machine learning for infrared? Machine learning inference? Or Is it talking about compilers (e.g. https://en.wikipedia.org/wiki/MLIR_(software) https://en.wikipedia.org/wiki/MLIR_(software))
- Jtsummers 3y ago
  It’s pretty obviously about compilers. From the first article: > As we announced recently, my team at Google has started a new effort to build production-worthy engineering tools for Fully Homomorphic Encryption (FHE). One focal point of this, and one which I’ll be focusing on as long as Google is willing to pay me to do so, is building out a compiler toolchain for FHE in the MLIR framework (Multi-Level Intermediate Representation). The project is called Homomorphic Encryption Intermediate Representation, or HEIR And MLIR there links to the MLIR project page.
  - DonsDiscountGas 3y ago
    Right, this would be great to put in the README is opposed to behind another link.
  - kjs3 3y ago
    Your definition of obvious is quite strange. I have to go from HN to the github to another link (the right one) to figure you what the hell is going on.
    - Jtsummers 3y ago
      I mean, the github repo's README says: > This is the code repository for a series of articles on the MLIR framework. And then has links to the (currently written) articles, the first one with "(Getting Started)" in the title. Seems like an obvious place to, you know, start reading about things.
      - kjs3 3y ago
        I mean, as opposed to putting 2 lines of markdown that explain what the project is about in the github repositories instead forcing another level of indirection on the user. Seems like an obvious place to, you know, start reading about things.
ubj 3y ago
Thanks for sharing these articles. I've been interested in MLIR ever since learning that the Mojo language would integrate with it [1], but it's been hard to find reference material geared towards beginners. [1]: https://docs.modular.com/mojo/notebooks/BoolMLIR.html https://docs.modular.com/mojo/notebooks/BoolMLIR.html
armchairhacker 3y ago
Does anyone have experiences using MLIR vs. a custom IR which lowers to LLVM IR? I've worked with the latter. On one hand MLIR seems like it would be nice to work with. LLVM IR is fairly low-level and hard to debug, but has a great API; it would be great to use such an API but keep things like type-safety and invariants, and worry less about tricky undefined behavior. On the other hand, despite claims of extensibility and flexibility, it's hard to imagine it being easier to work with than a custom IR. Plus, it seems MLIR's extensibility was build to support some things in particular (custom architectures, "affine / polyhedral" optimizations).
- mathisfun123 3y ago
  i've worked on pytorch and mlir. pytorch has no less than 3 internal IRs and has tensorexprs, which lower to llvm ir. without a doubt mlir is preferrable because at the minimum it's a bigger ecosystem of users/developers etc that understand the compiler. so the code is clearer, the docs are better, there are way more features and tools. i really can't name a negative relative to pytorch, especially now that the one cute thing that pytorch had (codegen for boilerplate for the ir) has been implemented (better) in mlir. > Plus, it seems MLIR's extensibility was build to support some things in particular this isn't true in the least. just go to https://mlir.llvm.org/docs/Dialects/ https://mlir.llvm.org/docs/Dialects/ and you'll see the set of upstream dialects is very diverse. and that's just upstream.
  - subharmonicon 3y ago
    It’s not clear that all these dialects are actually helping things, or that the design of something from which it’s easy to make new dialects is that helpful. Why? I attended an MLIR online meeting months back and there was someone presenting who had a diagram that literally had a dozen different dialects and converters that were being written between different ones (but not MxN, so it was a strictly relatively arbitrary set of combinations that were being converted between, and mostly one-directional if I recall correctly). This was the “solution” to being able to make various things interoperate, but from a distance it looks a lot like a relatively obvious problem caused by the very nature of the design of MLIR.
rurban 3y ago
their tablegen still munges the trees manually. a proper tablegen would give easy to write lhs -> rhs tree rules, and doesn"t allow shooting yourself into your foot. see proper lisp/scheme matchers. even the gimple matcher hack is far better see eg. https://github.com/j2kun/mlir-tutorial/blob/92723e224c1a6e8cdffc1183dd993a1114a31b07/lib/Transform/Arith/MulToAdd.cpp#L19 https://github.com/j2kun/mlir-tutorial/blob/92723e224c1a6e8c...

122 points by matt_d 3y ago

anlunx 3y ago

How does it compare with the Toy dialect tutorial?

mathisfun123 3y ago
kudos to jeremy for putting pen to paper but i do believe the toy tutorial is better (albeit more comprehensive). also i don't like bazel at all.
- j2kun 3y ago
  I tried to learn from the toy tutorial and just found it to jump into the deep end too quick, without enough explanation for me to start writing code (because everything I touched in `toy` caused the build to break in mysterious ways).
  - the_svd_doctor 3y ago
    Same opinion. I have little/no exposure to compilers (yes I know, maybe I'm not the right target) and tried to follow Toy and had to resort to a lot of wikipedia/google just to understand the concepts and terms. And after finishing it I was still a bit confused about how to even get started. I think it could be made a bit more beginner friendly. I found the standalone example https://github.com/llvm/llvm-project/tree/main/mlir/examples/standalone https://github.com/llvm/llvm-project/tree/main/mlir/examples... in the MLIR repo to be very useful, on the other hand.
amoss 3y ago
I'm finding the Toy tutorial easier to read, and easier to use as a jumping off point into the MLIR reference docs. I thought the writing style in this tutorial was a bit dense, and Bazel seems like an unnecessary complication. Only an opinion though, YMMV.

synergy20 3y ago

two years ago MLIR was a big deal for the company I worked but then it went silent, the compiler team switched to something else for RISC-V ML design, not sure what it is though.

cratermoon 3y ago

The README could benefit from a description of what MLIR is.

kQq9oHeAz6wLLS 3y ago
My thought as well. Not exactly "for beginners" in a literal sense.
cabalamat 3y ago
This is true for most READMEs on github projects. Is it really too hard to write a paragraph giving the context?
mathisfun123 3y ago
i don't understand the point of this criticism - i swear people just post it because they don't know what the thing is and they feel they need to compulsively comment anyway. no one stumbles on this repo by accident (jeremy didn't post it here). the only way you find it is you google "mlir tutorial" ie you know what MLIR is and you're actively explicitly looking for a tut. there is no situation where you're ambushed by this content. like if i write a tutorial "integration by parts for beginners" but don't define integration are you gonna claim that it fails at being for beginners? does every title from now on need to explicitly have "... for beginners who know what this is and have already made the decision to start studying".
- noman-land 3y ago
  It's funny, I agree with both of you. Considering this repo was posted to this site and prompted thousands of people to stumble on it by accident, even a link in the README on the words "MLIR framework" would go a long way. I, as someone who is both super interested in applied FHE and subscribed to this guy's blog, almost skipped this post because I didn't realize it was about FHE, and I am probably in the target audience for it.
  - mathisfun123 3y ago
    > Considering this repo was posted to this site and prompted thousands of people to stumble on it by accident but the guy himself didn't post it. how are people so entitled that they're upset about a deficiency in a thing that they sought out. no one is making you go the repo (or asking you to go). it's literally like complaining about free samples at costco - if you don't like what they have don't take one!

dmoy 3y ago

y'all should open source the Kythe indexers for that and tablegen

marssaxman 3y ago

Glad to see more work going into MLIR documentation. It's a powerful system, but it can be difficult to get a sense of what normal usage looks like.

smcleod 3y ago

Weird that it’s for beginners without saying what it actually is. Machine learning for infrared? Machine learning inference? Or Is it talking about compilers (e.g. https://en.wikipedia.org/wiki/MLIR_(software) https://en.wikipedia.org/wiki/MLIR_(software))

Jtsummers 3y ago
It’s pretty obviously about compilers. From the first article: > As we announced recently, my team at Google has started a new effort to build production-worthy engineering tools for Fully Homomorphic Encryption (FHE). One focal point of this, and one which I’ll be focusing on as long as Google is willing to pay me to do so, is building out a compiler toolchain for FHE in the MLIR framework (Multi-Level Intermediate Representation). The project is called Homomorphic Encryption Intermediate Representation, or HEIR And MLIR there links to the MLIR project page.
- DonsDiscountGas 3y ago
  Right, this would be great to put in the README is opposed to behind another link.
- kjs3 3y ago
  Your definition of obvious is quite strange. I have to go from HN to the github to another link (the right one) to figure you what the hell is going on.
  - Jtsummers 3y ago
    I mean, the github repo's README says: > This is the code repository for a series of articles on the MLIR framework. And then has links to the (currently written) articles, the first one with "(Getting Started)" in the title. Seems like an obvious place to, you know, start reading about things.
    - kjs3 3y ago
      I mean, as opposed to putting 2 lines of markdown that explain what the project is about in the github repositories instead forcing another level of indirection on the user. Seems like an obvious place to, you know, start reading about things.

ubj 3y ago

Thanks for sharing these articles. I've been interested in MLIR ever since learning that the Mojo language would integrate with it [1], but it's been hard to find reference material geared towards beginners. [1]: https://docs.modular.com/mojo/notebooks/BoolMLIR.html https://docs.modular.com/mojo/notebooks/BoolMLIR.html

armchairhacker 3y ago

Does anyone have experiences using MLIR vs. a custom IR which lowers to LLVM IR? I've worked with the latter. On one hand MLIR seems like it would be nice to work with. LLVM IR is fairly low-level and hard to debug, but has a great API; it would be great to use such an API but keep things like type-safety and invariants, and worry less about tricky undefined behavior. On the other hand, despite claims of extensibility and flexibility, it's hard to imagine it being easier to work with than a custom IR. Plus, it seems MLIR's extensibility was build to support some things in particular (custom architectures, "affine / polyhedral" optimizations).

mathisfun123 3y ago
i've worked on pytorch and mlir. pytorch has no less than 3 internal IRs and has tensorexprs, which lower to llvm ir. without a doubt mlir is preferrable because at the minimum it's a bigger ecosystem of users/developers etc that understand the compiler. so the code is clearer, the docs are better, there are way more features and tools. i really can't name a negative relative to pytorch, especially now that the one cute thing that pytorch had (codegen for boilerplate for the ir) has been implemented (better) in mlir. > Plus, it seems MLIR's extensibility was build to support some things in particular this isn't true in the least. just go to https://mlir.llvm.org/docs/Dialects/ https://mlir.llvm.org/docs/Dialects/ and you'll see the set of upstream dialects is very diverse. and that's just upstream.
- subharmonicon 3y ago
  It’s not clear that all these dialects are actually helping things, or that the design of something from which it’s easy to make new dialects is that helpful. Why? I attended an MLIR online meeting months back and there was someone presenting who had a diagram that literally had a dozen different dialects and converters that were being written between different ones (but not MxN, so it was a strictly relatively arbitrary set of combinations that were being converted between, and mostly one-directional if I recall correctly). This was the “solution” to being able to make various things interoperate, but from a distance it looks a lot like a relatively obvious problem caused by the very nature of the design of MLIR.

rurban 3y ago

their tablegen still munges the trees manually. a proper tablegen would give easy to write lhs -> rhs tree rules, and doesn"t allow shooting yourself into your foot. see proper lisp/scheme matchers. even the gimple matcher hack is far better see eg. https://github.com/j2kun/mlir-tutorial/blob/92723e224c1a6e8cdffc1183dd993a1114a31b07/lib/Transform/Arith/MulToAdd.cpp#L19 https://github.com/j2kun/mlir-tutorial/blob/92723e224c1a6e8c...