3 ms·

So this is a processor where you would have 97% of the threads doing some I/O like task? But that can't be disk I/O, so that would leave networking?

by tgv 3y ago

dan-robertson 3y ago
I think generally the threads are spending a lot of time waiting on memory. It can take >100 cycles to get something from ram so you could have all your threads try to read a pointer and still have computation to spare until the first read comes back from memory. It could be that eg 97% of your threads are looking things up in big hashtables (eg computing a big join for a database query) or binary-searching big arrays, rather than ‘some I/O task’
- tgv 3y ago
  This processor is out of my league, but do you have any idea how a program would use that optimally? How do you code for that?
- AnimalMuppet 3y ago
  Let's say that I can get something from RAM in 100 cycles. But if I have 60 threads all trying to do something with RAM, I can't do 60 RAM accesses in that 100 cycles, can I? Somebody's going to have to wait, aren't they?
  - convolvatron 3y ago
    this would work really well with rambus style async memory if it every got out from under the giant pile of patents the 'plus' side here is that that condition gets handled gracefully, but yes, certainly you can end up in a situation where memory transactions per second is the bottleneck. its likely more advtangeous to have a lot of memory controllers and ddr interfaces here than a lot of banks on the same bus. but that's a real cost and pin issue. the mta 'solved' this by fully dissociating the memory from the cpu with a fabric maybe you could do the same with cxl today
  - dan-robertson 3y ago
    I’m not exactly sure what you mean. RAM allows multiple reads to be in flight at once but I guess won’t be clocked as fast as the cpu. So you’ll have to do some computation in some threads instead of reads. Peak performance will have a mix of some threads waiting on ram and others doing actual work.
yvdriess 3y ago
DRAM is the new I/O. So yes, this is designed to handle 97% of the threads doing constant bad-locality DRAM accesses.
- rbanffy 3y ago
  And with this, DRAM access becomes the new asynchronous IO.
  - pmayrgundter 3y ago
    Neat insight The protocols for HPC are so amorphous that they bubbled up into the lowest common denominator, completely software defined async global workspace
rewmie 3y ago
> But that can't be disk I/O, so that would leave networking? Networking is a huge part of cloud applications, and network connections take orders of magnitude longer to go through than disk access. There are components of any cloud architecture which are dedicated exclusively to handling networking. Reverse proxies, ingress controllers, API gateways, message broker handlers, etc etc etc. Even function-as-a-service tasks heavily favour listening and reacting to network calls. I dare say that pure horsepower servers are no longer driving demand for servers. The ability to shove as many processes and threads on a single CPU is by far the thing that cloud providers and on prem companies seek.

So this is a processor where you would have 97% of the threads doing some I/O like task? But that can't be disk I/O, so that would leave networking?

by tgv 3y ago

So this is a processor where you would have 97% of the threads doing some I/O like task? But that can't be disk I/O, so that would leave networking?

dan-robertson 3y ago

I think generally the threads are spending a lot of time waiting on memory. It can take >100 cycles to get something from ram so you could have all your threads try to read a pointer and still have computation to spare until the first read comes back from memory. It could be that eg 97% of your threads are looking things up in big hashtables (eg computing a big join for a database query) or binary-searching big arrays, rather than ‘some I/O task’

tgv 3y ago
This processor is out of my league, but do you have any idea how a program would use that optimally? How do you code for that?
AnimalMuppet 3y ago
Let's say that I can get something from RAM in 100 cycles. But if I have 60 threads all trying to do something with RAM, I can't do 60 RAM accesses in that 100 cycles, can I? Somebody's going to have to wait, aren't they?
- convolvatron 3y ago
  this would work really well with rambus style async memory if it every got out from under the giant pile of patents the 'plus' side here is that that condition gets handled gracefully, but yes, certainly you can end up in a situation where memory transactions per second is the bottleneck. its likely more advtangeous to have a lot of memory controllers and ddr interfaces here than a lot of banks on the same bus. but that's a real cost and pin issue. the mta 'solved' this by fully dissociating the memory from the cpu with a fabric maybe you could do the same with cxl today
- dan-robertson 3y ago
  I’m not exactly sure what you mean. RAM allows multiple reads to be in flight at once but I guess won’t be clocked as fast as the cpu. So you’ll have to do some computation in some threads instead of reads. Peak performance will have a mix of some threads waiting on ram and others doing actual work.

yvdriess 3y ago

DRAM is the new I/O. So yes, this is designed to handle 97% of the threads doing constant bad-locality DRAM accesses.

rbanffy 3y ago
And with this, DRAM access becomes the new asynchronous IO.
- pmayrgundter 3y ago
  Neat insight The protocols for HPC are so amorphous that they bubbled up into the lowest common denominator, completely software defined async global workspace

rewmie 3y ago

> But that can't be disk I/O, so that would leave networking? Networking is a huge part of cloud applications, and network connections take orders of magnitude longer to go through than disk access. There are components of any cloud architecture which are dedicated exclusively to handling networking. Reverse proxies, ingress controllers, API gateways, message broker handlers, etc etc etc. Even function-as-a-service tasks heavily favour listening and reacting to network calls. I dare say that pure horsepower servers are no longer driving demand for servers. The ability to shove as many processes and threads on a single CPU is by far the thing that cloud providers and on prem companies seek.