9 ms·

What's performance like, compared to regular PyTorch (running on CPU)?

• sanxiyn 4y ago
  This runs on GPU with WebGL, so it will depend on what GPU you have.
  • Broge 4y ago
    Looking at the code it seems like it's only running using simd so far. I think the creator said something about the WebGL models being inaccurate when quantized or something.
    • sanxiyn 4y ago
      Indeed, I see WebGL commented out here: https://github.com/xenova/transformers.js/blob/main/src/models.js https://github.com/xenova/transformers.js/blob/main/src/mode....
      • xenova 4y ago
        Right - currently, everything runs using WASM (32-bit, with 64-bit coming soon [1,2]), and I plan to add support for WebGPU soon! (WebGPU is the successor to WebGL, which is coming out in April 2023 [3]) [1] https://github.com/WebAssembly/memory64/issues/36#issuecomment-1469800748 https://github.com/WebAssembly/memory64/issues/36#issuecomme... [2] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzPhvJCxI/m/SUhU9Z0vAgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP... [3] https://github.com/microsoft/onnxruntime/issues/11695#issuecomment-1374984530 https://github.com/microsoft/onnxruntime/issues/11695#issuec...
  • petilon 4y ago
    If I run transformer.js using Node on a machine that has GPU (such as Nvidia Jetson Nano) will it take advantage of the GPU?
    • xenova 4y ago
      Currently the library only runs on the CPU (it's only a few weeks old). WebGPU support is planned though (which releases soon [1]) [1] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzPhvJCxI/m/SUhU9Z0vAgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP...