4 ms·
This runs on GPU with WebGL, so it will depend on what GPU you have.
by sanxiyn 4y ago
This runs on GPU with WebGL, so it will depend on what GPU you have.
- Broge 4y agoLooking at the code it seems like it's only running using simd so far. I think the creator said something about the WebGL models being inaccurate when quantized or something.
- sanxiyn 4y agoIndeed, I see WebGL commented out here: https://github.com/xenova/transformers.js/blob/main/src/models.js https://github.com/xenova/transformers.js/blob/main/src/mode....
- xenova 4y agoRight - currently, everything runs using WASM (32-bit, with 64-bit coming soon [1,2]), and I plan to add support for WebGPU soon! (WebGPU is the successor to WebGL, which is coming out in April 2023 [3]) [1] https://github.com/WebAssembly/memory64/issues/36#issuecomment-1469800748 https://github.com/WebAssembly/memory64/issues/36#issuecomme... [2] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzPhvJCxI/m/SUhU9Z0vAgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP... [3] https://github.com/microsoft/onnxruntime/issues/11695#issuecomment-1374984530 https://github.com/microsoft/onnxruntime/issues/11695#issuec...
- petilon 4y agoIf I run transformer.js using Node on a machine that has GPU (such as Nvidia Jetson Nano) will it take advantage of the GPU?
- xenova 4y agoCurrently the library only runs on the CPU (it's only a few weeks old). WebGPU support is planned though (which releases soon [1]) [1] https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzPhvJCxI/m/SUhU9Z0vAgAJ https://groups.google.com/a/chromium.org/g/blink-dev/c/VomzP...