3 ms·

What are the runtimes?

• KenoFischer 7y ago
  About a minute or so for the batch of 64, but I haven't tuned the implementation yet (I just did enough work to get it down to a comfortable range of experimentation). The paper I linked (https://eprint.iacr.org/2018/1041.pdf https://eprint.iacr.org/2018/1041.pdf) which uses the same model, but with a more optimized implementation cites 26ms amortized per image (in a batch of 64), so I would suspect I can get down to that with a day or two of optimization work if I wanted to (or just plug in their backend - but where's the fun in that).
• deleted 7y ago
  [deleted]
• osaariki 7y ago
  I'm a researcher at Microsoft who's worked a lot on this problem (neural network inference on encrypted data). Take a look at this paper we published at PLDI [1] on our CHET compiler. The speed of the inference depends a lot on the depth of the network. A small 2 convolutional layer model takes a couple of seconds, while a 10 convolutional layer SqueezeNet takes 164 seconds. The reason is that you have to select encryption parameters according to the depth of the computation. Since this paper was published we've by the way improved these numbers ~5X. That was optimizing for latency and throughput is a something you can separately optimize for. This landmark paper (also from Microsoft) [2] showed inferencing on 2 convolutional layer network at a throughput of 58k images per hour. [1]: https://www.cs.utexas.edu/~roshan/CHET.pdf https://www.cs.utexas.edu/~roshan/CHET.pdf [2]: http://proceedings.mlr.press/v48/gilad-bachrach16.pdf http://proceedings.mlr.press/v48/gilad-bachrach16.pdf