5 ms·

From the article -- “The true bottleneck for processor throughput is the network interface used to connect a machine to an external network, such as an Ethernet

From the article -- “The true bottleneck for processor throughput is the network interface used to connect a machine to an external network, such as an Ethernet, therefore severely limiting a processor’s data ingest capability,” said Dr. Jonathan Smith, a program manager in DARPA’s Information Innovation Office (I2O). “Today, network throughput on state-of-the-art technology is about 1014 bits per second (bps) and data is processed in aggregate at about 1014 bps. Current stacks deliver only about 1010 to 1011 bps application throughputs.” I don't disagree with this, but see the challenge not so much as the transmission speed between nodes, rather I see it as the 'semantics' for expressing addressing and placement. (and yes, I am a big fan of RDMA :-)) One of the things I helped invent/design back when I was at NetApp was some network attached memory or NAM. NetApp was among a number of people who realized that as memory sizes got larger, having them associated with a single node made less and less sense from a computational perspective. One can imagine a "node" which lives in a 64 bit address space, has say 16GB of "L3" cache shared among anywhere from 12 to 12,000 instruction execution or 'compute' elements. More general purpose compute would look more like a processor of today, more specialized compute looks like a tensor element or a GPU element. "RAM" or generally accessible memory would reside in its own unit and could be 'volatile' or 'involatile' (backed by some form of storage). With attributes like access time, volatility, redundancy, etc. That sort of architecture starts to look more like a super computer (albeit a non-uniform one) than your typical computer. With nodes "booting" by going through a self-test and then mapping their memory from the fabric and restarting from where they left off. I thought I might get a chance to build that system at Google but realized that for the first time in my career that I was aware of I was prevented by my lack of a PhD. And what was worse, given the politics at the time, no one with a PhD would sign on as a 'figurehead' (I reached out to a couple and we had some really great conversations around this) because of the way in which Google was evaluating performance at the time. (think grad school ++ but where only one author seems to get all the credit). Now that I've had some free time I've built some of it in simulation to explore the trade-offs and the ability to predict performance based on the various channels. Slow going though.

SisypheanLife 7y ago
These types of multi-tier compute systems/MITM processors are now being implemented. Microsoft publicly discusses their efforts here: https://www.microsoft.com/en-us/research/project/project-catapult/ https://www.microsoft.com/en-us/research/project/project-cat...
- ChuckMcM 7y ago
  That looks like a lot of fun. I wonder if they would let me do a 1 year residency with that team. HP Enterprise was doing something as well with "the machine" (very large memory systems) which generated some interesting papers. I feel like we do not yet have a calculus for analyzing the mix of "entanglement" of data across transactional expressions. It is a problem I've puzzled over since about 2001 when Steve Kleiman asked me to scale file systems without speeding up the processor.
  - fis 7y ago
    As I recall the solution then was "buy Spinnaker". :) . Join us at HPE storage, we'll do it right this time!
jacquesm 7y ago
It's very sad that lack of a PhD would cause such a decision to be made rather than that it would get debated on its merits.
- ChuckMcM 7y ago
  Or that the merits of a suggestion were inversely proportional to an employee number, but cultures such as those frequently emerge in engineering driven organizations with a lot of smart people. I speculate that it is an offshoot off the imposter syndrome effect, when someone is feeling like an imposter and they are being asked to make a big decision, they search for externally generated metrics that would increase their confidence in the decision. Degrees, low employee numbers, industry awards, are all signals that "someone else" thought this person was "good". And not even super geniuses understand everything about everything and they still have to make decisions. Sad or not, in my experience working around gifted engineering teams it is not uncommon.
  - pm90 7y ago
    > Sad or not, in my experience working around gifted engineering teams it is not uncommon. OK, so I had hoped that this was a problem unique to Google but I guess its not. I just don't get this kind of thinking. I understand that a PhD is a good marker for intelligence... but not for ingenuity (maybe a tiny bit). I've worked with both PhD's and non PhD's... while PhD's are amazing to talk through different ideas, what may or may not work, the people I was most productive with have universally been people who get excited about technology (not saying they're mutually exclusive). Mind, not excited about fads, but those who can't wait to code a prototype to see if it really works as expected. Those who will jump in the middle of an outage and are excited to learn about what might have gone wrong. Those are the attributes that I now look for whenever I have to make a choice between teams, between companies, whatever. The people that get excited by new designs and architectures, who love to talk through them, are the best people to work with. I guess I do understand why Google is that way... I just hope that this kind of thinking doesn't infect other workplaces.
    - 7y ago
deleted 7y ago
[deleted]
shaklee3 7y ago
This is kind of what nvswitch does, albeit in a proprietary architecture. All 16 GPUs in an NVSwitch can see the memory space of all others, and any reads/writes are transparently performed on the correct GPU. This effectively gives a 512GB address space of HBM2 memory.
- namibj 7y ago
  Better: this works between/towards POWER 9 sockets, including knowing to execute atomics right at the destination.
jiggawatts 7y ago
Wow, a NetApp employee who's actually heard of performance as a feature! I'm saying this because every time I hear that a customer has a NetApp I get giddy as a school girl, because their filer products have a literal turbo button that I get to press. Thanks to NetApp I've had some career highlights of boosting storage performance of entire enterprises by at least a factor of two, and occasionally as high as tenfold or more. I'm not saying that I'm some sort of storage performance tuning wizard! There is literally a single parameter that can be changed to at least double the performance of every NetApp filer that has been shipped in about two decades: Just increase the TCP Window size from the original 1990s era default of 17KB to something sane like 256KB and then everything gets magically faster. This is why it's so bizarre hearing you talking about performance and working at NetApp. It's like hearing that someone was on the Trabant F1 team.
- tyingq 7y ago
  The default for both CIFS and NFS is 64k since ONTAP 8.2.1, released in 2013. https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-7FBEDD68-3B0E-4553-A90A-013BFFAD95E5.html https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-7FB...
  - jiggawatts 7y ago
    That's still way too low. A window size of 64KB will be a bottleneck on any 10 Gbps Ethernet connection, except perhaps a point-to-point cable. Any real-world network with switches, routers, and firewalls will require on the order of 256-512KB window sizes for decent performance. You need megabytes for any link that leaves the building! For reference, 64KB was the default in Windows 2000-XP, and is dynamically raised up to 2MB in Vista and later, and up to 16MB on recent Windows Server versions.
    - tyingq 7y ago
      Perhaps to avoid support tickets for people with equipment that doesn't properly support window scaling. The 64k setting is as high as you can go and avoid that.
godelski 7y ago
If you want to get a PhD there's definitely a lot of research going into networks. Especially at the big computing labs (read DOE). So like LLNL, ORNL, Sandia, Argonne. I don't work in this area, but I do work on/with HPC stuff so there's a lot of talk about all this stuff. It definitely seems to be a big part of the ECP movement/funding. Data transport and memory storage are two of the biggest problems I hear being talked about in ECP.
nostrademons 7y ago
How would you get around speed-of-light delays? The speed of light is about 1 foot/nanosecond, so maintaining a 1-cycle access time @ 3 GHz gives you about 4 inches to play with. Current server designs have some headroom (100 cycles = 33 feet, while the actual physical distance is more like 1-2 feet), but as soon as you start going to another rack and want to maintain normal RAM access speeds you run up against physical limits pretty quickly.
- LargoLasskhyfv 7y ago
  I assume you mean the speed of light in copper, or silicon instead of a vacuum? If so, may i warp your assumptions by the suggestion of using silicon photonics, and [1] https://en.wikipedia.org/wiki/Photonic-crystal_fiber https://en.wikipedia.org/wiki/Photonic-crystal_fiber theryby gaining 30% more speed? Some article from 2013: [2] https://www.extremetech.com/extreme/161687-darpa-creates-hollow-core-optical-fiber-for-faster-networks-more-accurate-sensors https://www.extremetech.com/extreme/161687-darpa-creates-hol... Some company from now: [3] https://www.nktphotonics.com/lasers-fibers/product-category/photonic-crystal-fibers/ https://www.nktphotonics.com/lasers-fibers/product-category/...
gnufx 7y ago
That sort of node exists as KNL, for numbers of compute elements up to ~200 with threading on. Unfortunately the on-chip OPA (RDMA) interface didn't materialize as far as I know, and KNL is dead (unless you're Down Under Geosolutions?).

SisypheanLife 7y ago

These types of multi-tier compute systems/MITM processors are now being implemented. Microsoft publicly discusses their efforts here: https://www.microsoft.com/en-us/research/project/project-catapult/ https://www.microsoft.com/en-us/research/project/project-cat...

ChuckMcM 7y ago
That looks like a lot of fun. I wonder if they would let me do a 1 year residency with that team. HP Enterprise was doing something as well with "the machine" (very large memory systems) which generated some interesting papers. I feel like we do not yet have a calculus for analyzing the mix of "entanglement" of data across transactional expressions. It is a problem I've puzzled over since about 2001 when Steve Kleiman asked me to scale file systems without speeding up the processor.
- fis 7y ago
  As I recall the solution then was "buy Spinnaker". :) . Join us at HPE storage, we'll do it right this time!

jacquesm 7y ago

It's very sad that lack of a PhD would cause such a decision to be made rather than that it would get debated on its merits.

ChuckMcM 7y ago
Or that the merits of a suggestion were inversely proportional to an employee number, but cultures such as those frequently emerge in engineering driven organizations with a lot of smart people. I speculate that it is an offshoot off the imposter syndrome effect, when someone is feeling like an imposter and they are being asked to make a big decision, they search for externally generated metrics that would increase their confidence in the decision. Degrees, low employee numbers, industry awards, are all signals that "someone else" thought this person was "good". And not even super geniuses understand everything about everything and they still have to make decisions. Sad or not, in my experience working around gifted engineering teams it is not uncommon.
- pm90 7y ago
  > Sad or not, in my experience working around gifted engineering teams it is not uncommon. OK, so I had hoped that this was a problem unique to Google but I guess its not. I just don't get this kind of thinking. I understand that a PhD is a good marker for intelligence... but not for ingenuity (maybe a tiny bit). I've worked with both PhD's and non PhD's... while PhD's are amazing to talk through different ideas, what may or may not work, the people I was most productive with have universally been people who get excited about technology (not saying they're mutually exclusive). Mind, not excited about fads, but those who can't wait to code a prototype to see if it really works as expected. Those who will jump in the middle of an outage and are excited to learn about what might have gone wrong. Those are the attributes that I now look for whenever I have to make a choice between teams, between companies, whatever. The people that get excited by new designs and architectures, who love to talk through them, are the best people to work with. I guess I do understand why Google is that way... I just hope that this kind of thinking doesn't infect other workplaces.
  - 7y ago

deleted 7y ago

[deleted]

shaklee3 7y ago

This is kind of what nvswitch does, albeit in a proprietary architecture. All 16 GPUs in an NVSwitch can see the memory space of all others, and any reads/writes are transparently performed on the correct GPU. This effectively gives a 512GB address space of HBM2 memory.

namibj 7y ago
Better: this works between/towards POWER 9 sockets, including knowing to execute atomics right at the destination.

jiggawatts 7y ago

Wow, a NetApp employee who's actually heard of performance as a feature! I'm saying this because every time I hear that a customer has a NetApp I get giddy as a school girl, because their filer products have a literal turbo button that I get to press. Thanks to NetApp I've had some career highlights of boosting storage performance of entire enterprises by at least a factor of two, and occasionally as high as tenfold or more. I'm not saying that I'm some sort of storage performance tuning wizard! There is literally a single parameter that can be changed to at least double the performance of every NetApp filer that has been shipped in about two decades: Just increase the TCP Window size from the original 1990s era default of 17KB to something sane like 256KB and then everything gets magically faster. This is why it's so bizarre hearing you talking about performance and working at NetApp. It's like hearing that someone was on the Trabant F1 team.

tyingq 7y ago
The default for both CIFS and NFS is 64k since ONTAP 8.2.1, released in 2013. https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-7FBEDD68-3B0E-4553-A90A-013BFFAD95E5.html https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-7FB...
- jiggawatts 7y ago
  That's still way too low. A window size of 64KB will be a bottleneck on any 10 Gbps Ethernet connection, except perhaps a point-to-point cable. Any real-world network with switches, routers, and firewalls will require on the order of 256-512KB window sizes for decent performance. You need megabytes for any link that leaves the building! For reference, 64KB was the default in Windows 2000-XP, and is dynamically raised up to 2MB in Vista and later, and up to 16MB on recent Windows Server versions.
  - tyingq 7y ago
    Perhaps to avoid support tickets for people with equipment that doesn't properly support window scaling. The 64k setting is as high as you can go and avoid that.

godelski 7y ago

If you want to get a PhD there's definitely a lot of research going into networks. Especially at the big computing labs (read DOE). So like LLNL, ORNL, Sandia, Argonne. I don't work in this area, but I do work on/with HPC stuff so there's a lot of talk about all this stuff. It definitely seems to be a big part of the ECP movement/funding. Data transport and memory storage are two of the biggest problems I hear being talked about in ECP.

nostrademons 7y ago

How would you get around speed-of-light delays? The speed of light is about 1 foot/nanosecond, so maintaining a 1-cycle access time @ 3 GHz gives you about 4 inches to play with. Current server designs have some headroom (100 cycles = 33 feet, while the actual physical distance is more like 1-2 feet), but as soon as you start going to another rack and want to maintain normal RAM access speeds you run up against physical limits pretty quickly.

LargoLasskhyfv 7y ago
I assume you mean the speed of light in copper, or silicon instead of a vacuum? If so, may i warp your assumptions by the suggestion of using silicon photonics, and [1] https://en.wikipedia.org/wiki/Photonic-crystal_fiber https://en.wikipedia.org/wiki/Photonic-crystal_fiber theryby gaining 30% more speed? Some article from 2013: [2] https://www.extremetech.com/extreme/161687-darpa-creates-hollow-core-optical-fiber-for-faster-networks-more-accurate-sensors https://www.extremetech.com/extreme/161687-darpa-creates-hol... Some company from now: [3] https://www.nktphotonics.com/lasers-fibers/product-category/photonic-crystal-fibers/ https://www.nktphotonics.com/lasers-fibers/product-category/...

gnufx 7y ago

That sort of node exists as KNL, for numbers of compute elements up to ~200 with threading on. Unfortunately the on-chip OPA (RDMA) interface didn't materialize as far as I know, and KNL is dead (unless you're Down Under Geosolutions?).