5 ms·
It will be interesting to see the SMT performance, I am expecting this would provide benefits and be further refined in future generation. With Zen5c we get 192
by ksec 2y ago
It will be interesting to see the SMT performance, I am expecting this would provide benefits and be further refined in future generation. With Zen5c we get 192 Core or 384vCPU. We should be getting 256 Core with Zen 6c next year. Which means on a Dual Socket 1U Server, that is a potential of 512 Core with 1024 vCPU.
Whatever Web App Scaling issues we had in 2014 could now fit into a single server, assuming we somehow manage to cool the thing. Even at 1 RPS per vCPU that is 1000 RPS, excluding cache hit. Even on HN front-page dont hit the server at 1000 Page View per second.
- teaearlgraycold 2y agoWeb app scaling issues are usually around database latency.
- CodesInChaos 2y agoIn my experience, scaling is limited by database throughput, not latency.
- codesnik 2y agowell, db latency in some naive unoptimized app with multiple sequential queries directly affects _app_'s throughput
- teaearlgraycold 2y agoDepends. Most apps that are slow have shitty app code with N+1 queries. If you can get past that stage and write good enough app code then eventually you’ll hit database throughout issues.
- jeltz 2y agoBut database performance is also a lot about CPU and RAM speed plus concurrent use of those resources.
- jiggawatts 2y agoOnly because the database is on the far end of a network hop, and then all too often its storage is remote from its compute as well. This is the most common scenario in enterprise or cloud deployments, where for good measure a couple of firewalls, load balancers, routers, and reverse proxies are added at every stage.
- bayindirh 2y agoServing web pages is cheap. You’ll probably hit network I/O limits before you saturate the cores. I wonder what about its HPC performance. I think cooling this won’t be big problem, but might be wet one, requiring DLC after a certain point.
- Dylan16807 2y ago> Serving web pages is cheap. You’ll probably hit network I/O limits before you saturate the cores. It's hard to be network I/O bound when serving web pages. Netflix struggles to be network I/O bound when serving video, which is so much bigger and uses so much less processing. Epyc started off with 32 cores on PCIe 3, and quickly moved to 64 cores on PCIe 4. When we hit 256 cores it's probably going to have PCIe 6, which means it's still the same I/O per core. But those numbers are crazy overkill for web serving anyway. If you wanted to allocate about a gigabit per core, with 512 cores across two CPUs, using PCIe 5 to be conservative, you'd need at total of... 16 lanes, for a single 400gb/s card you can buy today. (This is assuming you mean the I/O from the server to the network. If you're talking about I/O outside the server, then upgrade your switches. Using denser servers doesn't increase your network load, it lets you take the same load and send it to fewer racks. You're already handling the total data somewhere.)
- bayindirh 2y agoI mean, we already run NDR Infiniband, and with sometimes multiple cards per host in our data center, so the numbers are not surprising or out of the this world for me... Terabit Ethernet also is easy. Just 10 100gbit fibers. Again, not much either in port count, or in physical space needs (port count is ample, fibers are thin). What I meant is, in a conventional data center, you'll probably hit your allocated bandwidth limits before you saturate a processor like this while serving web pages, unless you're sitting on a big exchange point or the backbone router is not in the next system hall. A single socket Epyc system has 128 PCIe lanes and 12 channel memory. A complete overkill for such a job unless you serve millions, and only have a basement to put your servers with some very fat network pipes.
- 2y ago
- fulafel 2y agoSMT could use a lot more benchmarking investigations. Intuitively having more tasks working on the same problem at half speed should have a memory usage cost, are apps commonly using more memory for no speed gain when using SMT? In a lot of published benchmarks it seems most apps don't noticeably benefit in executions peed.
- crest 2y agoSMT also allows the cores to hide memory latency by executing the other thread(s). Taken to the extreme end this would give you a barrel CPU.
- fulafel 2y agoYes, this is reflected in the execution speed of the workloads (vs being an additional thing). Other things making up the potential for being more than a zero sum game (half the speed) include making better use of the ALU and branch units that would otherwise be waiting for dependencies. On the other side of the scale is how different working sets of the threads can thrash each others cache working sets.
- nullc 2y agoI think unfortunately with just 2-way smt it's pretty easy to get both of them stalled at once. The power9 4-way SMT seems to be a fair bit more effective than what I've seen on x86.
- JonChesterfield 2y agoPush the hyperthread count up higher and you have enough stuff to do while waiting on memory that branch prediction doesn't matter as much. Could call it a GPU.
- WithinReason 2y agoThe opposite might be true, the better you are at utilising the CPU pipeline the less space you have to run a 2nd thread so SMT's benefits might diminish
- ColonelPhantom 2y agoCorrect. However, Zen 5 seems to be the first major 'scaling up' that AMD has done since the original Zen, given that the core seems to be moving from dispatching 6 to 8 instructions per clock and 4 to 6 ALUs. I would expect that SMT to be excellent for low-IPC workloads to get the most out of that scaling.
- andrepd 2y agoMan I sure hope that your server does more than 1 RPS :)
- ksec 2y agoI wish I am joking but there are plenty of production instances doing less than 10 RPS per vCPU. ( I think the request part doesn't really explain well, since it could be a page view but it could also be an API request. In this case it is mostly referring to a page view )
- jiggawatts 2y agoWe're entering the era of "kilo cores" in the same way computing entered the era of kilobytes in the 1940s. If you consider a tightly-coupled rack of servers with GPUs to be one machine, then we're well into the hundreds of kilocores. I found it entertaining having a debate with someone here on HN who just couldn't grok the concept that it's possible to serve something the size of Wikipedia from a single server. That's been easy for a while now, it just isn't done for practical reasons such as availability or cost-efficiency.
- JonChesterfield 2y agoIs that actually true, or only that you can serve it from one server behind a lot of servers named CDN?
- jiggawatts 2y agoIt’s definitely true. It won’t be optimal or fast for end users, but it’s entirely doable. One of the main benefits of a CDN is that it brings the data closer to the users, reducing network latency. Obviously a single server can’t be close to every user globally! In practice one would always use a CDN for a site like WikiPedia but the point is that it isn’t necessary to handle the traffic. A single two-socket EPYC server can put out something like 400 Gbps of traffic, which is far more than all but the biggest sites.
- fulafel 2y agoGPU terminology "cores" are more like SIMD lanes, they're not comparable.