4 ms·
The big innovation in architectures is in the SIMD world. AVX512 (x86 512-bit), SVE (ARM 512-bit), NVidia PTX / SASS (32x32-bit), AMD RDNA (32x32-bit), AMD CDN
by dragontamer 6y ago
The big innovation in architectures is in the SIMD world.
AVX512 (x86 512-bit), SVE (ARM 512-bit), NVidia PTX / SASS (32x32-bit), AMD RDNA (32x32-bit), AMD CDNA (64x32-bit).
64-bit cores (aka: classic CPUs) are looking like a solved problem, and are becoming a commodity. SIMD compute however, remains an open question. NVidia probably leads today, but there seems to be plenty of room for smaller players.
Heck, one major company (AMD) is pushing 32x32-bit on one market (6xxx series) and 64x32-bit in another market (MI100 / Supercomputers).
- tyingq 6y agoFujitsu seems like a leader for SIMD via their A64FX. I wonder if they will ever venture outside of the supercomputing niche.
- dragontamer 6y agoSVE is looking like a general-purpose ARM instruction set in the future. I believe the Neoverse V-cores (high-performance) will have access to the SVE instructions for example. So the SVE-SIMD is not necessarily locked to Fujitsu (though Fujitsu's particular implementation is probably crazy good. HBM2 + 512-bit wide and #1 supercomputer in the world and all...)
- londons_explore 6y agoSIMD today is only really helpful with a few usecases. If you want to encode some video, decode some jpegs, or do a physics simulation quicker, it's really going to help. It won't boot Linux any quicker tho. I suspect for consumer uses, SIMD is already used for nearly all the use cases it can be.
- dragontamer 6y agoAre you sure about that? The original SIMD-papers in the 1980s show how to compile a Regex into a highly-parallel state machine and then "reduced" (aka: Scan / Prefix-operation: https://en.wikipedia.org/wiki/Prefix_sum https://en.wikipedia.org/wiki/Prefix_sum). A huge amount of operations, such as XML-whitespace removal (aka: SIMD Steam Compacting), Regular Expressions, and more, have been proven ~30 to 40 years ago to benefit from SIMD compute. Yet such libraries don't exist today yet. SIMD compute is highly niche, and clearly today's population is overly focused on deep-learning... without even seeing the easy opportunities of XML parsing or simple regex yet. Further: additional opportunities are being discovered in O(n^2) operations: such as inner-join operations on your typical database. Citations. * For Regular Expressions: Read the 1986 paper "DATA PARALLEL ALGORITHMS". Its an easy read. Hillis / Steele are great writers. They even have the "impossible Linked List" parallelism figured out in there (granted: the nodes are located in such a way that the SIMD-computer can work with the nodes. But... if you had a memory-allocator that worked with their linked-list format, you could very well implement their pointer-jumping approach to SIMD-linked list traversal) * For whitespace folding / removal, see http://www.cse.chalmers.se/~uffe/streamcompaction.pdf http://www.cse.chalmers.se/~uffe/streamcompaction.pdf. They don't cite it as XML-whitespace removal, but it seems pretty obvious to me that it could be used for parallel whitespace removal in O(lg(n)) steps. * Database SIMD: http://www.cs.columbia.edu/~kar/pubsk/simd.pdf http://www.cs.columbia.edu/~kar/pubsk/simd.pdf . Various operations have been proven to be better on SIMD, including "mass binary search" (one binary search cannot be parallelized. But if you have 5000-binary searches operating in parallel, its HIGHLY efficient to execute all 5000 in a weird parallel manner, far faster than you might originally imagine). ---------- SIMD-cuckoo hashing, SIMD-skip lists, etc. etc. There's so many data-structures that haven't really been fleshed out on SIMD yet outside of research settings. They have been proven easy to implement and simple / clean to understand. They're just not widely known yet.
- my123 6y agohttps://github.com/simdjson/simdjson https://github.com/simdjson/simdjson as an example that fits in the "outside-of-a-conventional-SIMD-workload" mold.
- skybrian 6y agoIt seems like there are a wide variety of ways to serialize and deserialize data, their performance sometimes varies by orders of magnitude, and the slow code persists because it doesn’t matter enough to optimize compared to other virtues like convenience and maintainability. The key seems to be figuring out how to get good (not the best) performance when you mostly care about other things? Machine learning itself seems like an example of throwing hardware at problems to try to improve the state of the art, to the point where it becomes so expensive that they have to think about performance more.
- dragontamer 6y agoSIMD-compute is a totally different model of compute than what most programmers are familiar with. That's the biggest problem. If you write optimal SIMD-code, no one else in your team will understand it. Since we have so much compute these days (to the point where O(n^2) scanf parsers are all over the place), its become increasingly obvious that few modern programmers care about performance at all. Nonetheless, the more and more I study SIMD-compute, the more I realize that these expert programmers have figured out a ton of good and fast solutions to a wide-variety of problems... decades ago and then somehow forgotten until recently. Seriously: that Data Parallel Algorithms paper is just WTF to me. Linked list traversal (scan-reduced sum from a linked list in SIMD-parallel), Regular Expressions and more. -------- Then I look at the GPU-graphics guys, and they're doing like BVH tree traversal in parallel so that their raytracers work. Its like "Yeah, Raytracing is clearly a parallel operation cause GPUs can do it". So I look it up and wtf? Its not easy. Someone really thought things through. Its non-obvious how they managed to get a recursive / sequential operation to operate in highly parallel SIMD operations while avoiding branch divergence issues. Really: think about it: Raytracing is effectively: If(ray hit object) recursively bounce ray. How the hell did they make that parallel? A combination of stream-compaction and very intelligent data-structures, as well as a set of new SIMD-assembly instructions to cover some obscure cases. There's some really intelligent stuff going on in the SIMD-compute world, that clearly applies beyond just the machine-learning crowd.
- m00x 6y agoThis comment seems a bit shortsighted. GPUs and TPUs are SIMD and ML models are increasingly being used in consumer hardware. Video cards are selling out so fast that there's months worth of backorders. SIMD processors are being put in self driving cars, robots with vision, doorbell cameras, drones, etc. We're only at the beginning of SIMD use-cases.
- gnufx 6y agoThe V in SVE is for vector, the S isn't for SIMD, and it's length-agnostic; I don't know how similar it is to the RISC-V vector extension. Think CDC, Cray, NEC, not AMD/Intel. I guess the recent innovation in that space is actual matrix multiplication instructions in CPUs.
- dragontamer 6y agoSome GPU assembly advancements... * Bpermute and permute. (Pshufb is like permute, a gather operation. Bpermute is the opposite, like a scatter. Bpermute Doesn't exist on x86 yet) * __shared__ memory crossbar: every simd unit can read, or write, to shared memory in parallel per clocktick. The crossbar can also broadcast 1-to-all each clocktick. * Butterfly permute: the fundamental pattern in permutations for a variety of operations. Most noticably for scan, and FFTs. Butterfly networks are closely related to pext and pdep implementation (showing how common that particular permute is). * 8+ way hyperthreads / SMT. GPUs have very bad latency, but very high SMT counteracts that problem well in practice. * PCIe Atomics: perform those compare and swap negotiations over I/O, allowing tight CPU and GPU memory integration. * Crazy RAM. 1000GBps on HBM2. 800GBps over GDDR6x thanks to 2-bits transferred per clock ticks. * Crazy networks. AMD Infinity fabric pushes over 100GB. NVlink is 600GBps. A GPU network link has more bandwidth than a typical CPU's DDR4 RAM bandwidth. * NVidia SASS has the craziest instruction set, the compiler figures out read / write hazards and publishes them in the SASS assembly itself. NVidias ISA decoder + assembler team is doing something crazy here, the likes I haven't seen in any other instruction set ever. * "Ballot" instructions. Its... really hard to explain why these are useful. They just are, lol. Just a few cool concepts I've seen in the GPU world recently. Sure, matrix multiplications get the headlines because of tensors / deep learning. But don't sleep on the obscure stuff.