5 ms·

ONNX Runtime merges WebGPU backend

178 points by b_mc2 3y ago

CSSer 3y ago
Okay, I’ve just got to know: what’s up with the commit messages? I’ve never seen anyone just straight up use numbers before.
- taylorfinley 3y ago
  Looks like something a self-taught dev would do (ask how I know), or someone who usually works fully solo. Maybe it's a ML community thing? I might do something like this if I was in a rush and just needed a save point, especially if I was working on a branch I knew I'd squash and merge with a more meaningful commit message. I certainly wouldn't be proud of it, but I might do it.
- turnsout 3y ago
  Wow, you're not joking—there are 13 commits where the messages are just "1" or whatever the number is. There's also "w", "w2", and "w3". It has the energy of someone who has been forced into version control and deeply resents it.
  - taylorfinley 3y ago
    Definitely has big "index.old.older.bak_1.bak_2.asp" energy.
  - keyle 3y ago
    secret cabal binary codes /s
- mirekrusin 3y ago
  If you squash your prs there is no point in spending time on making individual commits nice and isolated or their messages as they'll be aggregated into one anyway. When you do large PRs you want to checkpoint your work and commits have less meaningful descriptions, it's just next step of steps that you couldn't plan ahead, they're popping up as you go.
  - flohofwoe 3y ago
    Not sure why you're downvoted, because it looks exactly like that (all changes go into PRs, and PRs are then squashed on merge).
  - mejutoco 3y ago
    It makes it easier to know what you are squashing. My approach is to combine commit messages when squashing. With that approach they are useful.
    - mirekrusin 3y ago
      The fact that you can do that indicates that they could be simply separate PRs merged to main branch independently instead. It's usually desirable as it avoids longer living PR braches that tend to create conflicts and are harder to review.
      - mejutoco 3y ago
        It is a good point. In many cases, like solo or small teams the conflicts are not a big problem, but I see what you are saying. I have also been in teams that used feature flags more heavily, and it is a possible solution, too.
- flohofwoe 3y ago
  The main branch history looks reasonably clean. Looks like they generally squash-on-merge (and probably delete the branch on merge), in that case the detailed commit messages are lost anyway. Completely valid commit strategy if you ask me (even though I bet some might beg to differ).
synergy20 3y ago
interesting, is there a helloworld example or tutorial somewhere to check out how this works in real?
minimaxir 3y ago
Some context for those who aren't in the loop: ONNX Runtime (https://onnxruntime.ai/ https://onnxruntime.ai/) is a standardization format for AI models. Nowadays, it's extremely easy to export models in the ONNX format, especially language models with tools like Hugging Face transformers which have special workflows for it. ONNX support in the browser was lacking and limited to CPU, but with a WebGPU backend it may now finally be feasible to run models in the browser on a GPU, which opens up interesting oppertunities. Although from this PR it looks like only a few operations are implemented, no browser-based GPT yet.
- UncleOxidant 3y ago
  ONNX is the ML interchange format - it's supposed to be a framework independent way to share ML models and weights. ONNX Runtime is a runtime for running those ONNX format models. And it's quite performant - probably the more performant ML runtimes currently.
  - mathisfun123 3y ago
    Gonna respond here and correct both comments >Some context for those who aren't in the loop: ONNX Runtime (https://onnxruntime.ai/ https://onnxruntime.ai/) is a standardization format for AI models. It's just an IR, one of many - every framework has its own. >Nowadays, it's extremely easy to export models in the ONNX format, especially language models with tools like Hugging Face transformers which have special workflows for it. Meh it's poorly supported by both PyTorch and TF. Why support Microsoft's IR when you have your own. >probably the most performant ML runtime at this point. Not even by a long-shot - first party compilers are generally faster because of smoother interop but even amongst third-party you have TRT and TVM. TBH I have no idea what anyone uses ONNX for these days (legacy?).
    - lairv 3y ago
      I was going to answer the same, I find the approach of machine learning compilers that directly compile models to host and device code better than having to bring a huge runtime. There are exciting projects in this area like TVM Unity [1], IREE [2], or torch.export [3] [1] https://github.com/apache/tvm/tree/unity https://github.com/apache/tvm/tree/unity [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://pytorch.org/get-started/pytorch-2.0/#inference-and-export https://pytorch.org/get-started/pytorch-2.0/#inference-and-e...
b_mc2 3y ago
A pretty cool library that uses ONNX is transformers.js [1] and they're already working to add WebGPU support.[2] [1] https://xenova.github.io/transformers.js/ https://xenova.github.io/transformers.js/ [2] https://twitter.com/xenovacom/status/1650634015060156420 https://twitter.com/xenovacom/status/1650634015060156420
WorldPeas 3y ago
Quite the uh... interesting commit strategy...
pumanoir 3y ago
A great option but there is wonnx which seems to be more complete and mature. And the bonus is that it's implemented in Rust (if you are into it). https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx
- misterdata 3y ago
  Also WONNX can both be used from native apps (through wgpu it uses Vulkan, DX or Metal) a well as on the web (using WebGPU, WONNX compiled to WebAssembly).
- wongarsu 3y ago
  That's an awesome project I didn't know about, and I'll almost certainly use it the next time I need an onnx runtime (just because I want to call it from rust). But I would guess that "more complete and mature" is only true in the narrow scope of "running common networks on webgpu"?
Culonavirus 3y ago
It would be great if, one lovely day, we could just slap together an electron app and run inference through webgpu hassle-free and cross-platform.
vrglvrglvrgl 3y ago
[dead]
01648599160 3y ago
[dead]
jarym 3y ago
As an aside, I love ONNX and the main reason I'm sticking with PyTorch. I was able to develop and train an RL model in Python and then convert it to ONNX and call it from C# production code. It still took a lot of effort but the final version is very performant and reliable.

178 points by b_mc2 3y ago

Okay, I’ve just got to know: what’s up with the commit messages? I’ve never seen anyone just straight up use numbers before.

taylorfinley 3y ago
Looks like something a self-taught dev would do (ask how I know), or someone who usually works fully solo. Maybe it's a ML community thing? I might do something like this if I was in a rush and just needed a save point, especially if I was working on a branch I knew I'd squash and merge with a more meaningful commit message. I certainly wouldn't be proud of it, but I might do it.
turnsout 3y ago
Wow, you're not joking—there are 13 commits where the messages are just "1" or whatever the number is. There's also "w", "w2", and "w3". It has the energy of someone who has been forced into version control and deeply resents it.
- taylorfinley 3y ago
  Definitely has big "index.old.older.bak_1.bak_2.asp" energy.
- keyle 3y ago
  secret cabal binary codes /s
mirekrusin 3y ago
If you squash your prs there is no point in spending time on making individual commits nice and isolated or their messages as they'll be aggregated into one anyway. When you do large PRs you want to checkpoint your work and commits have less meaningful descriptions, it's just next step of steps that you couldn't plan ahead, they're popping up as you go.
- flohofwoe 3y ago
  Not sure why you're downvoted, because it looks exactly like that (all changes go into PRs, and PRs are then squashed on merge).
- mejutoco 3y ago
  It makes it easier to know what you are squashing. My approach is to combine commit messages when squashing. With that approach they are useful.
  - mirekrusin 3y ago
    The fact that you can do that indicates that they could be simply separate PRs merged to main branch independently instead. It's usually desirable as it avoids longer living PR braches that tend to create conflicts and are harder to review.
    - mejutoco 3y ago
      It is a good point. In many cases, like solo or small teams the conflicts are not a big problem, but I see what you are saying. I have also been in teams that used feature flags more heavily, and it is a possible solution, too.
flohofwoe 3y ago
The main branch history looks reasonably clean. Looks like they generally squash-on-merge (and probably delete the branch on merge), in that case the detailed commit messages are lost anyway. Completely valid commit strategy if you ask me (even though I bet some might beg to differ).

synergy20 3y ago

interesting, is there a helloworld example or tutorial somewhere to check out how this works in real?

minimaxir 3y ago

Some context for those who aren't in the loop: ONNX Runtime (https://onnxruntime.ai/ https://onnxruntime.ai/) is a standardization format for AI models. Nowadays, it's extremely easy to export models in the ONNX format, especially language models with tools like Hugging Face transformers which have special workflows for it. ONNX support in the browser was lacking and limited to CPU, but with a WebGPU backend it may now finally be feasible to run models in the browser on a GPU, which opens up interesting oppertunities. Although from this PR it looks like only a few operations are implemented, no browser-based GPT yet.

UncleOxidant 3y ago
ONNX is the ML interchange format - it's supposed to be a framework independent way to share ML models and weights. ONNX Runtime is a runtime for running those ONNX format models. And it's quite performant - probably the more performant ML runtimes currently.
- mathisfun123 3y ago
  Gonna respond here and correct both comments >Some context for those who aren't in the loop: ONNX Runtime (https://onnxruntime.ai/ https://onnxruntime.ai/) is a standardization format for AI models. It's just an IR, one of many - every framework has its own. >Nowadays, it's extremely easy to export models in the ONNX format, especially language models with tools like Hugging Face transformers which have special workflows for it. Meh it's poorly supported by both PyTorch and TF. Why support Microsoft's IR when you have your own. >probably the most performant ML runtime at this point. Not even by a long-shot - first party compilers are generally faster because of smoother interop but even amongst third-party you have TRT and TVM. TBH I have no idea what anyone uses ONNX for these days (legacy?).
  - lairv 3y ago
    I was going to answer the same, I find the approach of machine learning compilers that directly compile models to host and device code better than having to bring a huge runtime. There are exciting projects in this area like TVM Unity [1], IREE [2], or torch.export [3] [1] https://github.com/apache/tvm/tree/unity https://github.com/apache/tvm/tree/unity [2] https://github.com/openxla/iree https://github.com/openxla/iree [3] https://pytorch.org/get-started/pytorch-2.0/#inference-and-export https://pytorch.org/get-started/pytorch-2.0/#inference-and-e...

b_mc2 3y ago

A pretty cool library that uses ONNX is transformers.js [1] and they're already working to add WebGPU support.[2] [1] https://xenova.github.io/transformers.js/ https://xenova.github.io/transformers.js/ [2] https://twitter.com/xenovacom/status/1650634015060156420 https://twitter.com/xenovacom/status/1650634015060156420

WorldPeas 3y ago

Quite the uh... interesting commit strategy...

pumanoir 3y ago

A great option but there is wonnx which seems to be more complete and mature. And the bonus is that it's implemented in Rust (if you are into it). https://github.com/webonnx/wonnx https://github.com/webonnx/wonnx

misterdata 3y ago
Also WONNX can both be used from native apps (through wgpu it uses Vulkan, DX or Metal) a well as on the web (using WebGPU, WONNX compiled to WebAssembly).
wongarsu 3y ago
That's an awesome project I didn't know about, and I'll almost certainly use it the next time I need an onnx runtime (just because I want to call it from rust). But I would guess that "more complete and mature" is only true in the narrow scope of "running common networks on webgpu"?

Culonavirus 3y ago

It would be great if, one lovely day, we could just slap together an electron app and run inference through webgpu hassle-free and cross-platform.

vrglvrglvrgl 3y ago

[dead]

01648599160 3y ago

[dead]

jarym 3y ago

As an aside, I love ONNX and the main reason I'm sticking with PyTorch. I was able to develop and train an RL model in Python and then convert it to ONNX and call it from C# production code. It still took a lot of effort but the final version is very performant and reliable.