3 ms·

Why making this a Web API, rather than a library? We have WebAssembly threads which allows high-performance multi-threaded CPU code and WebGPU for general purpo

Why making this a Web API, rather than a library? We have WebAssembly threads which allows high-performance multi-threaded CPU code and WebGPU for general purpose GPU code. Please correct me if I'm mistaken (not a ML expert): On top of these existing APIs, we can build efficient neural networks. Of course, we wouldn't be able to use specialized hardware such as Google's TPU or Graphcore's IPU, but I doubt there's a use-case for those in web. Is there any non-GPU/CPU based consumer hardware aimed at fast neural networks that I'm missing? EDIT: Seems "NPU"s have been slowly arriving for the past 2 years to iOS/Android-based devices... Maybe there's a raison d'être for the WebNN API after all.

pjmlp 6y ago
Yes, stuff like https://developer.android.com/ndk/guides/neuralnetworks https://developer.android.com/ndk/guides/neuralnetworks
- AlexAltea 6y ago
  Thank you, I didn't know that! It feels scary to see that all these NN APIs are appearing around the fact that we cannot talk to DSPs, NPUs, GPUs, CPUs under a same language. By creating such highly-specialized APIs it feels like "giving up" on ever accomplishing that. Decades ago, compiler developers solved dealing with multiple CPU architectures, by creating languages that succinctly expresses their common traits: Doing arithmetical-logical operations and accessing memory. Languages like C abstracted us from CPU registers, stacks. Optimization passes like vectorization abstracted us from different ISA extensions. Now it feels it's the same issue all over again. Is it really hard to simply express: `x' = W*x + b` and let the compiler target the CPU/GPU/NPU as needed?
  - pjmlp 6y ago
    That is what stuff like SYSCL, CUDA and Halide allow for, but I guess the Web can have only one and WebAssembly won't be exposing what they need to work with anyway.
sillysaurusx 6y ago
I kind of like the idea of having a common API for the common neural network ops, if only as an intermediate layer between pytorch and tensorflow. But that's more an expression of API design than any particular reason. Once TPUs can run JS (which may be coming sooner than it seems), it might also be reasonable to write a "webpage" which TPUs then "execute". I think neural networks in general need to learn to be more flexible. Right now it feels like you're carving a network out of marble. It should be more like clay. More practically, tensorflow.js does run in the browser; you could port this API to a tf.js backend.
tpetry 6y ago
The needed models can be quite large. Do you really want to download a large model on every website making use of the new proposed apis?
- AlexAltea 6y ago
  I don't think the WebNN API is trying to solve this problem, are they? Skimming through the specification, WebNN provides a way of describing the topology of a neural network, and compile+run it on different hardware (DSP, NPU, GPU, etc). However, the parameters of the network (which are by far the largest part) have still to be provided.

pjmlp 6y ago

Yes, stuff like https://developer.android.com/ndk/guides/neuralnetworks https://developer.android.com/ndk/guides/neuralnetworks

AlexAltea 6y ago
Thank you, I didn't know that! It feels scary to see that all these NN APIs are appearing around the fact that we cannot talk to DSPs, NPUs, GPUs, CPUs under a same language. By creating such highly-specialized APIs it feels like "giving up" on ever accomplishing that. Decades ago, compiler developers solved dealing with multiple CPU architectures, by creating languages that succinctly expresses their common traits: Doing arithmetical-logical operations and accessing memory. Languages like C abstracted us from CPU registers, stacks. Optimization passes like vectorization abstracted us from different ISA extensions. Now it feels it's the same issue all over again. Is it really hard to simply express: `x' = W*x + b` and let the compiler target the CPU/GPU/NPU as needed?
- pjmlp 6y ago
  That is what stuff like SYSCL, CUDA and Halide allow for, but I guess the Web can have only one and WebAssembly won't be exposing what they need to work with anyway.

sillysaurusx 6y ago

I kind of like the idea of having a common API for the common neural network ops, if only as an intermediate layer between pytorch and tensorflow. But that's more an expression of API design than any particular reason. Once TPUs can run JS (which may be coming sooner than it seems), it might also be reasonable to write a "webpage" which TPUs then "execute". I think neural networks in general need to learn to be more flexible. Right now it feels like you're carving a network out of marble. It should be more like clay. More practically, tensorflow.js does run in the browser; you could port this API to a tf.js backend.

tpetry 6y ago

The needed models can be quite large. Do you really want to download a large model on every website making use of the new proposed apis?

AlexAltea 6y ago
I don't think the WebNN API is trying to solve this problem, are they? Skimming through the specification, WebNN provides a way of describing the topology of a neural network, and compile+run it on different hardware (DSP, NPU, GPU, etc). However, the parameters of the network (which are by far the largest part) have still to be provided.