3 ms·
There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocesso
by saltcured 8y ago
There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels.
Part of it is the economics at the fringe can pursue speed at any cost. And part is the heady appeal of doing things differently for researchers and advanced practitioners. But, in the long view, I think you are right that it is a bad idea. If you care about maintenance and sustainability, you usually find that these bypass solutions get abandoned as soon as the more conventional approaches can approximate their speed on newer commodity hardware. So there is huge churn in these specialist devices with specialist APIs and tooling.
There is a recurring theme in high performance networking where crazy things are tried and all sorts of fancy protocol offloading written, then eventually deprecated because it is seen as a support burden and a source of bugs. Because each of these specialized stacks has a smaller user base, they are have less economy of scale to invest in maintenance and stabilization.
- baybal2 8y ago> There is a constant dance at the fringe of high performance systems. It leads to a recurring pattern of "revolutionizing" with some kind of bypass or coprocessor architecture, then eventually reverting to traditional structures as the new performance realities reach commodity levels. Truly this is what DC hardware been about for 20+ years. Look at HPC space, and than look for "uncle Joe hosting." The moment Joe starts looking for HPC and invest heavily for some super duper RDMA powered disk, a mainstream hardware maker comes and spoils everything by releasing a mass market product doing just that if not better, thus ruining the grand scheme of gaining competitive advantage through some "secret sauce hardware" Look at video transcoding or that new "AI" neural network thing. After few years of entertaining themselves with idea of purpose made hardware, big boys simply went the route of just using a lot of mainstream hardware.