3 ms·

Addressing your last sentence, I am actually a GPU programmer by trade. GPUs are my specialty, and much of my professional work has been in Nvidia's CUDA But A

Addressing your last sentence, I am actually a GPU programmer by trade. GPUs are my specialty, and much of my professional work has been in Nvidia's CUDA But AVX-512 is a real step forward, if you look at the price/performance ratio for real-world workloads. So AVX-512 is the focus of NN-512 GPU cloud compute is almost unbelievably expensive. Even Linode charges $1000 per month, or $1.50 per hour (look at the GPU plans: https://www.linode.com/pricing/#row--compute https://www.linode.com/pricing/#row--compute). It's really hard to keep that GPU saturated, which is what you need to do to get your money's worth An AVX-512 Skylake-X cloud compute instance costs $10 per CPU-core per month, or 2 cents per hour, at Vultr (https://www.vultr.com/products/cloud-compute/ https://www.vultr.com/products/cloud-compute/). It's easy to saturate that core Think about it this way: a GPU is really just a cluster of SIMD units with fast memory. A GPU thread variable is just like a SIMD lane, and a GPU warp variable is just like a SIMD vector register Nvidia's SIMD pseudo-instructions are called PTX and they are like a 1024-bit version of AVX-512. An AVX-512 core is like a general purpose CPU with a 512-bit GPU core built in So paying for a single AVX-512 core is like paying for part of a GPU (16 threads), plus the general purpose compute you need to keep that GPU part supplied with work

bravura 6y ago
I've been interested in your project. I don't think you're realizing the potential of the parent's suggestion. You motivate the work through bottom-up construction, but I don't think you have a clear explanation for who the users are. They must be: 1) Sure they are only going to use models that are supported by NN-512. This is a big jump for me and many people who want to do transformers, LSTMs, etc. and not just what NN-512 costs. 2) They have to be price sensitive. But if you want larger companies, they will rarely just have one use-case that NN-512 supports; they will probably have a ton of use-cases that NN-512 supports. 3) They want to target AVX-512. If they are a medium or larger company they probably want to have tooling around this, to support model development and later deployment. Those are my thoughts in general. I currently need to be able to target and deploy on PC and Mac CPUs from the past 10 years. (I know, this is specific.) I think parent is saying that individual devs are more likely to try NN-512 if they don't have to learn a new API, since most of them want to have the flexibility to fool around before committing to a new tech or bringing it in-house. I guess my suggestion is really try to specifically define who the user is for NN-512 and what NN problems they are trying to solve, and let that shape the roadmap.
- 37ef_ced3 6y ago
  My original thinking was that companies needing to use transformers, LSTMs, etc., would want to fund the development after doing the simple work to evaluate NN-512's ResNeXt or DenseNet performance on their hardware Same with development of tooling (network conversion, parameter import). It just takes a lot of time/effort for me to write the code. It's easy to say that support for AVX, AVX2, etc., would be nice, but that's months of hard work (unfunded) I guess we agree that a niche tool like NN-512, that has a reduced feature set (trimmed to allow one person to complete it and release it for free), has trouble competing with the huge, free, flexible, multi-target tools funded by large companies In retrospect this is obvious!

bravura 6y ago

I've been interested in your project. I don't think you're realizing the potential of the parent's suggestion. You motivate the work through bottom-up construction, but I don't think you have a clear explanation for who the users are. They must be: 1) Sure they are only going to use models that are supported by NN-512. This is a big jump for me and many people who want to do transformers, LSTMs, etc. and not just what NN-512 costs. 2) They have to be price sensitive. But if you want larger companies, they will rarely just have one use-case that NN-512 supports; they will probably have a ton of use-cases that NN-512 supports. 3) They want to target AVX-512. If they are a medium or larger company they probably want to have tooling around this, to support model development and later deployment. Those are my thoughts in general. I currently need to be able to target and deploy on PC and Mac CPUs from the past 10 years. (I know, this is specific.) I think parent is saying that individual devs are more likely to try NN-512 if they don't have to learn a new API, since most of them want to have the flexibility to fool around before committing to a new tech or bringing it in-house. I guess my suggestion is really try to specifically define who the user is for NN-512 and what NN problems they are trying to solve, and let that shape the roadmap.

37ef_ced3 6y ago
My original thinking was that companies needing to use transformers, LSTMs, etc., would want to fund the development after doing the simple work to evaluate NN-512's ResNeXt or DenseNet performance on their hardware Same with development of tooling (network conversion, parameter import). It just takes a lot of time/effort for me to write the code. It's easy to say that support for AVX, AVX2, etc., would be nice, but that's months of hard work (unfunded) I guess we agree that a niche tool like NN-512, that has a reduced feature set (trimmed to allow one person to complete it and release it for free), has trouble competing with the huge, free, flexible, multi-target tools funded by large companies In retrospect this is obvious!