5 ms·
Mnist Demo Compiled On-the-Fly to WebAssembly in the Browser
- deleted 4y ago[deleted]
- ptspts 4y agoIs this supposed to recognize a single digit? For me it can recognize only half of the digits correctly. What is the use case for sich a low-precision image recognition?
- omnicognate 4y agoJudging by the fact the page gives no information about the model used but says it's "compiled on-the-fly to WebAssembly by loop_tool.js", I assume it's intended as a demo of loop_tool.js, not the model. It can't be a demo of MNIST itself. That's just a dataset, presumably used to train this model.
- unraveller 4y agoroll your own i-am-not-a-robot capture
- tinco 4y agoIt is trained on the mnist dataset, to have it recognize a digit you have to make sure the digit is sized roughly to fill the entire screen. It won't work if you draw a small digit in the corner. This is just a demo of the toolchain, it doesn't showcase any state of the art image recognition.
- spoiler 4y agoAh, so it's a toolchain demo. That makes more sense, and I was also initially confused; it kept saying my 5s looked like 8s most of the time lol
- dvh 4y agoThe startup was excruciatingly slow ~20s, is it mining crypto in the background?
- Bromeo 4y agoIt is "compiled on the fly", including (hardware specific?) optimizations, which can take some time.
- worldmerge 4y agoThat's interesting. It was nearly instant on my iphone.
- brrrrrm 4y agoWould you mind sharing which device you're using? I'm still working on making the first compilation faster!
- angelmm 4y agoThe code is compiled with Ecmascripten and the WASM binary is injected as part of the JS module. Since the downloading, processing and compiling is happening on the browser, it seems to be a reasonable time. However, to avoid blocking the main thread, I would run all this process in a worker. Not sure if there's any limitation there
- vmarius 4y agoThe quality is, ehm, not really good: https://imgur.com/a/uobGnpK https://imgur.com/a/uobGnpK
- steeve 4y agoAgreed https://imgur.com/a/m1DnaXP https://imgur.com/a/m1DnaXP Cool demo nontheless!
- chmod775 4y agoThe same thing happened to me: https://i.imgur.com/yN9JhGw.png https://i.imgur.com/yN9JhGw.png
- dmitrygr 4y agoQuite something indeed: https://imgur.com/a/SgqvG75 https://imgur.com/a/SgqvG75
- nonplus 4y agoSame experience: https://i.imgur.com/Z9BrQGn.png https://i.imgur.com/Z9BrQGn.png I don't hold the in browser POC at fault for the trained model (but am amused by the efficacy of the model).
- echelon 4y agoUnrelated to this demo, does anyone have pointers or recommendations for deploying 3D/WebGL + Wasm to the web? Any good libraries, tools, or resources?
- speps 4y agoYou can find a working example in my Torus Trooper project: https://github.com/speps/tt https://github.com/speps/tt In particular, the interface between WASM and WebGL: https://github.com/speps/tt/blob/master/web/tt.js#L45 https://github.com/speps/tt/blob/master/web/tt.js#L45 WASM is really quite straightforward in my opinion, export a C interface and link it with whatever you need from the browser APIs.
- echelon 4y agoThis is a fantastic reference, thank you! Cool project, too.
- natly 4y agoWow the source code is kind of blowing my mind. I had no idea this was possible (only in theory). import * as torch from "./torch.mjs"; ... const w0 = torch.tensor(weights["conv1.0.weight"]); const b0 = torch.tensor(weights["conv1.0.bias"]); const w1 = torch.tensor(weights["conv2.0.weight"]); const b1 = torch.tensor(weights["conv2.0.bias"]); const lw = torch.tensor(weights["out.weight"]); const lb = torch.tensor(weights["out.bias"]); const convs = nn.Sequential( nn.Conv2d(w0, b0, 1, 2), nn.MaxPool2d(2), nn.ReLU(), nn.Conv2d(w1, b1, 1, 2), nn.MaxPool2d(2), nn.ReLU() ); ... Looks like near identical python torch code except apparantely running in local js!
- brrrrrm 4y agothis is actually a brand new proposal! If you like it or have any comments, you can find more information here: https://dev-discuss.pytorch.org/t/proposal-torch-js-a-javascript-frontend-for-pytorch/650 https://dev-discuss.pytorch.org/t/proposal-torch-js-a-javasc...
- mountainriver 4y agoCool idea, I wonder if python wasm doesn’t make more sense though