4 ms·
Since the library claims high performance I went to look into the benchmarks section https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks https://bitbucket.or
by npalli 4y ago
Since the library claims high performance I went to look into the benchmarks section
https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks https://bitbucket.org/blaze-lib/blaze/wiki/Benchmarks
Turns out all benchmarks are done circa 2016/17 and use pretty old versions of libraries and compilers. That's a long time. Any excitement I had was definitely tempered, would be good to have a recent update.
- moloch-hai 4y agoI bet it didn't get any slower. I gather Kompute.cc, which runs on GPUs via Vulkan, is the modern choice. Running floating point array kernels on CPU cores seems pointless these days. (That said, I run QubesOS, and literally everything I run does all its work on CPU cores--including all Vulkan code--because nothing can see the GPU.)
- stjohnswarts 4y agoI makes sense if you don't have a gpu, which is quite often in the embedded world.
- owlbite 4y agoOr if you have a latency-sensitive computation, and the gpu round trip time is just too long.
- bayindirh 4y agoIf you work hard enough, you can use multiple DMA channels and stream the data in and out of GPU simultaneously, however it needs a datacenter class card (Tesla, Radeon Instinct, etc.). Also in nVidia's case, you can only do this while using CUDA only, because nVidia doesn't want you to get full performance out of these cards without CUDA.
- kolbe 4y agoExplain? I didn't think there was any way around paying the pcie cost. If your data is hosted in memory/cache, and the computation takes less than a few mics, I didn't think there was any way to make GPUs competitive.
- cburdick2 4y agoPCI cost is typically in the noise for realistic applications. There are ways to reduce latency and signal flags across PCIe, but do most of the workload on GPU to keep the latency down.
- kolbe 4y agoI'm guessing the guy criticizing latency sensitivity is talking about the applications where that does matter. There are plenty of applications in say finance where they need to run a 20 parameter dot product and get the result back in nanoseconds, and I don't think GPUs can ever handle work loads like that.
- cburdick2 4y agoThat's true, but for those types of applications you're looking at a custom ASIC in most cases. high ns or very low us are FPGA territory, and low us and above are suitable on GPU.
- kolbe 4y agoThis video asserts that FPGAs do the 10ns work, and C++ does the >100ns work. I've never heard of ASICs being used, since the deployment cycle is too frequent to justify the expense of fabing. https://m.youtube.com/watch?v=8uAW5FQtcvE https://m.youtube.com/watch?v=8uAW5FQtcvE
- bayindirh 4y agoIt depends on what you’re doing. I’m strongly against moving to GPU just for the sake of it. However, in some cases, where GPU brings substantial performance benefits in a long running (hours, days, etc.) application, you can setup pinned memory locations, where they map to GPU. Then you can feed the input part, where it feeds the GPU. If you have multiple DMA engines in the GPU, you can just stream the data from in to GPU to out, and read the results from your second pinned memory location. If the speed bump is big enough, and you can eat the first startup cost, PCIe cost can become negligible. However, it’s always horses for courses, and it may not fit your case at all. In my case, Eigen is already extremely fast for what I do on CPU, and my task doesn’t run long. So adding GPU doubles my wall clock.
- VHRanger 4y agoIt makes plenty sense to run array kernels on CPU if the kernel is faster than the CPU <-> GPU data shuffling latency or bandwith hurdles
- moloch-hai 4y agoIt makes plenty of sense to do isolated computations on CPU. The crossover point where you are better off shipping the input to a GPU and results back is pretty small, presuming you have one and the program knows about and can get to it. But, as always, fast enough is fast enough.
- bayindirh 4y ago> I gather Kompute.cc, which runs on GPUs via Vulkan, is the modern choice Or you can just use Eigen via CUDA: https://eigen.tuxfamily.org/dox/TopicCUDA.html https://eigen.tuxfamily.org/dox/TopicCUDA.html > Running floating point array kernels on CPU cores seems pointless these days. CPU level vectorization can do wonders in terms of speed with right libraries (like Eigen, MKL, etc.). I can solve the same problem in seconds with Eigen where its MATLAB implementation takes hours.
- deleted 4y ago[deleted]
- moloch-hai 4y agoThat makes sense if you want to and can depend on CUDA. I gather there are efforts to provide a conversion layer from CUDA to other targets, but they seems to involve adapting the CUDA code to be more portable.
- bayindirh 4y agoI personally don't prefer CUDA (or any other form of lock-down), however they started CUDA integration ~5 years ago (or even before that), so they need to start another migration effort to support another technologies. While I use Eigen extensively on CPU, I don't follow any mailing lists of them, so I don't know the internal state and mindset.
- cyber_kinetist 4y ago> Or you can just use Eigen via CUDA: The link you’ve posted doesn’t mean you can do large dense matmul in CUDA right away, it’s just that you can use fixed sized vector/matrix operations inside kernels (which might be useful if you’re writing graphics code and need some vec3/mat3/quats.) You still have to write your own customized kernel for a large dynamic-sized gemm computation (with tiling and shared memory and all that jazz), and at that point it’ll be best to just use CUBLAS.
- bayindirh 4y ago
- adrian_b 4y ago> "Running floating point array kernels on CPU cores seems pointless" This is true only for 32-bit or lower precision floating-point numbers. For 64-bit or higher precision floating-point numbers, which are required for the bulk of the scientific computation applications, CPUs are much better than the modern GPUs. In the latest generation of both NVIDIA and AMD "consumer" GPUs, the throughput ratio of FP64 vs. FP32 has been reduced to 1:64. This ensures that these GPUs have both a performance per dollar and a performance per watt that are worse than those of CPUs. The "datacenter" GPUs have excellent performance per watt, but their immense prices ensure that these GPUs either have a performance per dollar that is worse than that of CPUs (except for those who buy hundreds or thousands, to be used 24/7, so the savings in power consumption can balance the high acquisition cost) or even when the performance is so high that the ratio performance/price is similar to CPUs, the magnitude of the price is so great that it is far outside the range that can be afforded by a small business or by an individual. More than a decade ago, NVIDIA claimed continuously that the GPUs will completely replace the traditional CPUs for all high-volume computations. Nevertheless, they themselves are those who have prevented this from happening, by segmenting the GPU market and raising the price of the 64-bit GPUs by more than an order of magnitude.
- cburdick2 4y agoI think you'd need to qualify what better means here. GPUs will give much higher performance for FP64 than CPUs will. The price of CPUs has continued to increase significantly in the last 5 years, and now high-end Xeons are more expensive than GPUs. Consumer GPUs aren't what you'd typically compare to since people doing large scientific simulations wouldn't use a desktop unless it's a very small model.
- moloch-hai 4y agoI am corrected.
- bee_rider 4y agoI’d also be curious to see how Eigen configured to dispatch to MKL where appropriate would do in these comparisons. Eigen seems to be a pretty popular library and of course MKL is vendor supplied so it should be around forever, so I’d be more confident the ability of that pairing to stay up to date…
- bayindirh 4y agoEigen's latest performance benchmarks are also much better than the Blaze page paints. Eigen has commit-based performance monitoring for some time, arewefastyet style: https://eigen.tuxfamily.org/index.php?title=Performance_monitoring https://eigen.tuxfamily.org/index.php?title=Performance_moni... I've used Eigen to play with 3K by 3K dense matrices and solve them in some cases, and it's not even blink an eye, neither in time nor in space department.
- bee_rider 4y agoThis surprised me enough to plink around in octave a bit, which lead to me to be curious about the built in blas vs mkl difference (3k isn’t huge even in dense but I didn’t expect to see “blink of an eye” associated with it). 4 seconds to .13 on that sort of matrix. With like zero effort on my part. Bit of a meandering path on my part, but thanks for kicking it off.