4 ms·

"[compilation] for productive programming of hardware accelerators" But 93% of the codebase is Python lol. Whatever you think about Python, it is not a systems

"[compilation] for productive programming of hardware accelerators" But 93% of the codebase is Python lol. Whatever you think about Python, it is not a systems programming language. Takeaway: this is not a serious compiler project (and it's clearly not, it's a PhD student project). Deeper take: this is just a DSL that behind the scenes calls a (SMT) solver for tuning (what they call "scheduling"). There are a million examples of this approach for every architecture under the sun. My org is literally building out the same thing right now. Performance is directly a function of how good your model of the architecture is. At such a high-level it's very likely to produce suboptimal results because you have no control over ISA/intrinsic level decisions. Lower-level implementations are much more robustly "powerful". https://dl.acm.org/doi/10.1145/3332373 https://dl.acm.org/doi/10.1145/3332373

rscho 2y ago
Well, this is clearly an attempt at abstracting the kind of low-level stuff you describe. Perhaps it doesn't work (yet), but that shouldn't prevent people from trying ? Involving a SMT solver suggests that the solver is doing the heavy-lifting, not python. PhDs often produce inapplicable stuff, but they are also the basis for industry/application R&D, such as what your org is doing... PhDs are the slaves of science. They make stuff happen for peanuts in return and deserve our respect for that, even if what happens is oftentimes a dead-end. It's really sad seeing people shitting on PhDs.
- deleted 2y ago
  [deleted]
- pclmulqdq 2y ago
  Unfortunately, the comment you are responding to is more correct on this than I think you are. The python thing was stupid, though - a lot of high-performance code gets written in libraries like numpy (that call C 99.99% of the time) or pytorch (JIT-ed before executing) that keep Python our of the critical path. The problem with this research is that many similar ideas have been tried before in the contexts of supercomputers or CPU compilers. They ultimately all fail because they end up (1) being more work to program in, and (2) not being any faster because real life happens. Networks drop packets, clock frequencies jitter, and all sorts of non-determinism happens when you have large scale. A static scheduler forces you to stall the whole program for any of these faults. All the gain you got by painstakingly optimizing things goes away. PhD theses, in a best-case scenario, are the basis for new applications. Most of them amount to nothing. This one belongs on that pile. The sad part about that is that it isn't the student's fault that the professor sent them down a research direction that is guaranteed to amount to nothing of use. This one is on the professor.
  - fancyfredbot 2y ago
    I don't think the paper is about statically scheduled architectures. In fact they mention it's for modern accelerators. These switch between threads in a dynamic way rather than stalling. The scheduling being referred to seemed to mean the order in which instructions should be fed to a potentially dynamic scheduler to enable efficient usage of caches etc. So I'm not sure you can dismiss it as a thesis which will amount to nothing on the basis that static scheduling is a bad idea! I could easily have missed something though. It's not a particularly clear or succinct write-up and I have only read some of it. If it does say that it only works for strictly deterministic in-order architectures somewhere please can you point out where?
    - pclmulqdq 2y ago
      If you read the actual github, linked from the article, you will realize that it is a manual static scheduler. Example here: https://github.com/exo-lang/exo/blob/main/examples/avx2_matmul/README.md https://github.com/exo-lang/exo/blob/main/examples/avx2_matm... The hardware still does what it does, but this is a tool for manually ordering operations. I did, in fact, read the materials before dismissing them as useless, since this is very relevant to what I do professionally.
      - fancyfredbot 2y ago
        It's me who hadn't read all of the materials. But infact I think we both agree about what the tool does, which is scheduling an instruction stream ahead of time. I'm confused because that approach seems identical to all mainstream compilers I know of. GCC/Clang also schedule instructions statically and the right schedule will improve performance. Why won't it work here? What kind of dynamic scheduling do you think it needs in order to be useful? Like a tracing JIT? Or they need to implement the reordering in hardware and reschedule as instructions execute?
        pclmulqdq 2y ago
        The issue is that it takes manual programmer effort for no gain over what a compiler gives you. The selling point of most of these tools is that you can be smarter than the compiler, and this one (despite the example) is selling static scheduling of compute kernels, too. That would normally be up to the OS/driver with some flexibility.
QuadmasterXLII 2y ago
any sufficiently powerful compiler is going to run an interpreted language at compile time, and there’s no reason it can’t be Python instead of C++ template metaprograms or CMake
- deleted 2y ago
  [deleted]
deleted 2y ago
[deleted]
- deleted 2y ago
  [deleted]
fancyfredbot 2y ago
Your take seems to contradict the article? You say SMT solvers give "no control over ISA/intrinsic level decisions" but their design.md says "user-defined scheduling operations can encapsulate common optimization patterns and hardware-specific transformations". Are they wrong about this? Can you explain why?
saagarjha 2y ago
Maybe a takeaway that boils down to “how could anyone ever write a compiler on Python” is the wrong one to have.

rscho 2y ago

Well, this is clearly an attempt at abstracting the kind of low-level stuff you describe. Perhaps it doesn't work (yet), but that shouldn't prevent people from trying ? Involving a SMT solver suggests that the solver is doing the heavy-lifting, not python. PhDs often produce inapplicable stuff, but they are also the basis for industry/application R&D, such as what your org is doing... PhDs are the slaves of science. They make stuff happen for peanuts in return and deserve our respect for that, even if what happens is oftentimes a dead-end. It's really sad seeing people shitting on PhDs.

deleted 2y ago
[deleted]
pclmulqdq 2y ago
Unfortunately, the comment you are responding to is more correct on this than I think you are. The python thing was stupid, though - a lot of high-performance code gets written in libraries like numpy (that call C 99.99% of the time) or pytorch (JIT-ed before executing) that keep Python our of the critical path. The problem with this research is that many similar ideas have been tried before in the contexts of supercomputers or CPU compilers. They ultimately all fail because they end up (1) being more work to program in, and (2) not being any faster because real life happens. Networks drop packets, clock frequencies jitter, and all sorts of non-determinism happens when you have large scale. A static scheduler forces you to stall the whole program for any of these faults. All the gain you got by painstakingly optimizing things goes away. PhD theses, in a best-case scenario, are the basis for new applications. Most of them amount to nothing. This one belongs on that pile. The sad part about that is that it isn't the student's fault that the professor sent them down a research direction that is guaranteed to amount to nothing of use. This one is on the professor.
- fancyfredbot 2y ago
  I don't think the paper is about statically scheduled architectures. In fact they mention it's for modern accelerators. These switch between threads in a dynamic way rather than stalling. The scheduling being referred to seemed to mean the order in which instructions should be fed to a potentially dynamic scheduler to enable efficient usage of caches etc. So I'm not sure you can dismiss it as a thesis which will amount to nothing on the basis that static scheduling is a bad idea! I could easily have missed something though. It's not a particularly clear or succinct write-up and I have only read some of it. If it does say that it only works for strictly deterministic in-order architectures somewhere please can you point out where?
  - pclmulqdq 2y ago
    If you read the actual github, linked from the article, you will realize that it is a manual static scheduler. Example here: https://github.com/exo-lang/exo/blob/main/examples/avx2_matmul/README.md https://github.com/exo-lang/exo/blob/main/examples/avx2_matm... The hardware still does what it does, but this is a tool for manually ordering operations. I did, in fact, read the materials before dismissing them as useless, since this is very relevant to what I do professionally.
    - fancyfredbot 2y ago
      It's me who hadn't read all of the materials. But infact I think we both agree about what the tool does, which is scheduling an instruction stream ahead of time. I'm confused because that approach seems identical to all mainstream compilers I know of. GCC/Clang also schedule instructions statically and the right schedule will improve performance. Why won't it work here? What kind of dynamic scheduling do you think it needs in order to be useful? Like a tracing JIT? Or they need to implement the reordering in hardware and reschedule as instructions execute?
      - pclmulqdq 2y ago
        The issue is that it takes manual programmer effort for no gain over what a compiler gives you. The selling point of most of these tools is that you can be smarter than the compiler, and this one (despite the example) is selling static scheduling of compute kernels, too. That would normally be up to the OS/driver with some flexibility.

QuadmasterXLII 2y ago

any sufficiently powerful compiler is going to run an interpreted language at compile time, and there’s no reason it can’t be Python instead of C++ template metaprograms or CMake

deleted 2y ago
[deleted]

deleted 2y ago

[deleted]

deleted 2y ago
[deleted]

fancyfredbot 2y ago

Your take seems to contradict the article? You say SMT solvers give "no control over ISA/intrinsic level decisions" but their design.md says "user-defined scheduling operations can encapsulate common optimization patterns and hardware-specific transformations". Are they wrong about this? Can you explain why?

saagarjha 2y ago

Maybe a takeaway that boils down to “how could anyone ever write a compiler on Python” is the wrong one to have.