markurtz

3 ms·

1. 1.▲
   by markurtz 5y ago

   Disclosure: I work for Neural Magic. Hi ml_hardware, we report results for both throughput and latency in the blog. As you noted, the throughput performance for GPUs does beat out our implementations by a bit, but we did improve the through
2. 2.▲
   by markurtz 5y ago

   Disclosure: I work for Neural Magic. Hi deepnotderp, as noted by others the speeds listed here are combining throughput for GPU from Ultralytics to latency for GPU from Neural Magic. We did also include throughput measurements, though, wher
3. 3.▲
   by markurtz 5y ago

   Disclosure: I work for Neural Magic. Hi carbocation, we'd love to see what you think of the performance using the DeepSparse engine for CPU inference: https://github.com/neuralmagic/deepsparse Take a look through
4. 4.▲
   by markurtz 5y ago

   Disclosure: I work for Neural Magic. Hi 37ef_ced3, AVX-512 has certainly helped close the gap for CPUs vs GPUs. Even then, though, most inference engines on CPUs are still very compute bound for networks. Unstructured sparsity enables us to
5. 5.▲
   Show HN: YOLOv3 – Pruning and Quantizing to Improve Object Detection Performance(docs.neuralmagic.com)

   4 points by markurtz 5y ago|0 comments