5 ms·

Nice work! I knew it wouldn't be long before someone put this together :) I'm curious if given a different language (like Zig) with WebGPU access if you could

Nice work! I knew it wouldn't be long before someone put this together :) I'm curious if given a different language (like Zig) with WebGPU access if you could easily translate that last-mile of code to execute there or not? In specific I wonder if I can do it, and if you can give me an overview of where the code for "Universal deployment" in your diagram actually lives? I found llm_chat.js, but it seems that doesn't include the logic necessary for building WGSL shaders? Am I wrong or does that happen elsewhere like in the TVM runtime? How much is baked into llm_chat.wasm, and where is the source for that?

crowwork 4y ago
The WGSL are generated and compiled through TVM and embedded into the wasm. I think what you mean is wgpu native support. At the moment the web gpu runtime dispatches to the js webgpu environment. Once TVM runtime comes with wgpu native support (like the current ones in vulkan or metal), then it is possible to leverage any wgpu native runtime like what Zig provide. Additionally, currently tvm natively support targets like vulkan, metal directly which allows targeting these other platforms
- slimsag 4y ago
  OK that makes sense; so basically if I want to give this a shot then I would just need to read llm_chat.js and the TVM docs, and translate llm_chat.js to my language of choice effectively?
  - crowwork 4y ago
    I think instead what would be needed is a wgpu native runtime support for TVM. Like the implementations in tvm vulkan, then it will be naturally link to any runtime that provides webgpu.h Then yah the llm_chat.js would be high-level logic that targets the tvm runtime, and can be implemented in any language that tvm runtime support(that includes, js, java, c++ rust etc). Support webgpu native is an interesting direction. Feel free to open a thread in tvm discuss forum and perhaps there would be fun things to collaborate in OSS
    - summarity 4y ago
      How big is the "runtime" part? My use case would basically be: run this in a native app that links against webgpu (wgpu or dawn). Is there a reference implementation for this runtime that one could study?
      - crowwork 4y ago
        tvm runtime is pretty decent(~700k-2M level depending on dependency included), you can checkout tvm community and bring up the question there, i think there might be some common interest. There are impl of runtime for vulkan, metal that can be used as reference.

crowwork 4y ago

The WGSL are generated and compiled through TVM and embedded into the wasm. I think what you mean is wgpu native support. At the moment the web gpu runtime dispatches to the js webgpu environment. Once TVM runtime comes with wgpu native support (like the current ones in vulkan or metal), then it is possible to leverage any wgpu native runtime like what Zig provide. Additionally, currently tvm natively support targets like vulkan, metal directly which allows targeting these other platforms

slimsag 4y ago
OK that makes sense; so basically if I want to give this a shot then I would just need to read llm_chat.js and the TVM docs, and translate llm_chat.js to my language of choice effectively?
- crowwork 4y ago
  I think instead what would be needed is a wgpu native runtime support for TVM. Like the implementations in tvm vulkan, then it will be naturally link to any runtime that provides webgpu.h Then yah the llm_chat.js would be high-level logic that targets the tvm runtime, and can be implemented in any language that tvm runtime support(that includes, js, java, c++ rust etc). Support webgpu native is an interesting direction. Feel free to open a thread in tvm discuss forum and perhaps there would be fun things to collaborate in OSS
  - summarity 4y ago
    How big is the "runtime" part? My use case would basically be: run this in a native app that links against webgpu (wgpu or dawn). Is there a reference implementation for this runtime that one could study?
    - crowwork 4y ago
      tvm runtime is pretty decent(~700k-2M level depending on dependency included), you can checkout tvm community and bring up the question there, i think there might be some common interest. There are impl of runtime for vulkan, metal that can be used as reference.