4 ms·

This was sort of the dream of KNL but today I noticed Xeon Phi CPUs support (a.k.a. Knight Landing and Knight Mill) are marked as deprecated. GCC will emit

This was sort of the dream of KNL but today I noticed Xeon Phi CPUs support (a.k.a. Knight Landing and Knight Mill) are marked as deprecated. GCC will emit a warning when using the -mavx5124fmaps, -mavx5124vnniw, -mavx512er, -mavx512pf, -mprefetchwt1, -march=knl, -march=knm, -mtune=knl or -mtune=knm compiler switches. Support will be removed in GCC 15. the issue was that coordinating across this kind of hierarchy wasted a bunch of time. If you already knew how to coordinate, mostly, you could instead get better performance you might be surprised but we're getting to the point that communicating over a super computer is on the same order of magnitude as talking across a numa node.

frognumber 3y ago
I actually wasn't so much talking from that perspective, as simply from the perspective of the design of individual pieces. There were rather clever things done in e.g. older multipliers or adders or similar which, I think, could apply to most modern parallel architectures, be that GPGPU, SP, MPE, FPGA, or whatever, in order to significantly increase density at a cost of slightly reduced serial performance. For machine learning, that's a good tradeoff. Indeed, with some of the simpler architectures, I think computation could be moved into the memory itself, as long dreamed of. (Simply sticking 32,000 SA-110 processors on a die would be very, very limited by interconnect; there's a good reason for the types of architectures we're seeing not being that)
- frozenport 3y ago
  Truth is that there is another startup called graph core that is doing exactly that, and also a really big chip
  - frognumber 3y ago
    I assume no one will read this, but good places to look for super-clever ways to reduce transistor count while maintaining good performance: - Early mainframes / room-sized computers (era of vacuum tubes and discrete transistors), especially at the upper-end , where there was enough budget to have modern pipelined and scalar architectures. - Cray X-MP and successors - DEC Alpha / StrongARM (referenced SA-110) Bad places to look are all the microcode architectures. These optimized transistor count, often sacrificing massive amounts of performance in order to save on cost. Ditto for some of the minicomputers, where the goal was to make an "affordable" computer. Something like the PDP was super-clever in cost-cutting, which made sense at the time, does much less to maintain performance. There's a ton of long-forgotten cleverness.
  - frognumber 3y ago
    They do what you were talking about, not what I was. They seem annoying. "The IPU has a unique memory architecture consisting of large amounts of In-Processor-Memory™ within the IPU made up of SRAM (organised as a set of smaller independent distributed memory units) and a set of attached DRAM chips which can transfer to the In-Processor-Memory via explicit copies within the software. The memory contained in the external DRAM chips is referred to as Streaming Memory™." There's a ™ every few words. Those seem like pretty generic terms. That's their technical documentation. The architecture is reminiscent of some ideas from circa-2000 which didn't pan out. It reminds me of Tilera (the guy who ran it was the Donald Trump of computer architectures; company was acquihired by EZchip for a fraction of the investment which was put into it, which went to Mellanox, and then to NVidia).

frognumber 3y ago

I actually wasn't so much talking from that perspective, as simply from the perspective of the design of individual pieces. There were rather clever things done in e.g. older multipliers or adders or similar which, I think, could apply to most modern parallel architectures, be that GPGPU, SP, MPE, FPGA, or whatever, in order to significantly increase density at a cost of slightly reduced serial performance. For machine learning, that's a good tradeoff. Indeed, with some of the simpler architectures, I think computation could be moved into the memory itself, as long dreamed of. (Simply sticking 32,000 SA-110 processors on a die would be very, very limited by interconnect; there's a good reason for the types of architectures we're seeing not being that)

frozenport 3y ago
Truth is that there is another startup called graph core that is doing exactly that, and also a really big chip
- frognumber 3y ago
  I assume no one will read this, but good places to look for super-clever ways to reduce transistor count while maintaining good performance: - Early mainframes / room-sized computers (era of vacuum tubes and discrete transistors), especially at the upper-end , where there was enough budget to have modern pipelined and scalar architectures. - Cray X-MP and successors - DEC Alpha / StrongARM (referenced SA-110) Bad places to look are all the microcode architectures. These optimized transistor count, often sacrificing massive amounts of performance in order to save on cost. Ditto for some of the minicomputers, where the goal was to make an "affordable" computer. Something like the PDP was super-clever in cost-cutting, which made sense at the time, does much less to maintain performance. There's a ton of long-forgotten cleverness.
- frognumber 3y ago
  They do what you were talking about, not what I was. They seem annoying. "The IPU has a unique memory architecture consisting of large amounts of In-Processor-Memory™ within the IPU made up of SRAM (organised as a set of smaller independent distributed memory units) and a set of attached DRAM chips which can transfer to the In-Processor-Memory via explicit copies within the software. The memory contained in the external DRAM chips is referred to as Streaming Memory™." There's a ™ every few words. Those seem like pretty generic terms. That's their technical documentation. The architecture is reminiscent of some ideas from circa-2000 which didn't pan out. It reminds me of Tilera (the guy who ran it was the Donald Trump of computer architectures; company was acquihired by EZchip for a fraction of the investment which was put into it, which went to Mellanox, and then to NVidia).