4 ms·

This sounds like what Microsoft research did with SEAL to produce CryptoNets https://www.microsoft.com/en-us/research/publication/cryptonets-applying-neural-ne

• KenoFischer 7y ago
  Yep, same research setting, though a different network. I don't at the moment remember all the details of CryptoNets, but IIRC they were doing batch size 8192 evaluations (i.e. just using each slot as an independent value and evaluating the code as if on scalars), which allows you to get away without the fancy ciphertext encoding magic that's described in the blog post (at the cost of high latency of course).
  • osaariki 7y ago
    You're right, CryptoNets used a data layout optimized for throughput with a batch size 4096. Since then we've done a lot of work on low latency inference with our CHET compiler [1] and my colleagues with LoLa [2]. It all comes down to the data layouts you use. [1]: https://www.cs.utexas.edu/~roshan/CHET.pdf https://www.cs.utexas.edu/~roshan/CHET.pdf [2]: https://arxiv.org/pdf/1812.10659.pdf https://arxiv.org/pdf/1812.10659.pdf