3 ms·

> FreeBSD supports up to 1024 cores on the amd64 and arm64 platforms. Sounds pretty future proofed unless I'm missing a x86 processor out there that does this

by boznz 3y ago

Koshkin 3y ago
> up to 1024 Curious where this (rather large, yet still seemingly arbitrary) limit comes from.
- throw0101a 3y ago
  > Curious where this (rather large, yet still seemingly arbitrary) limit comes from. It is Good Enough for now, while keeping various pre-allocated, statically created structures with-in reasonable size limits: > Global and allocated arrays sized by MAXCPU result in excessive bloat on systems with lower core counts. In addition, some code used u_char (8 bits) to hold a CPU index, which is not valid if MAXCPU is greater than 256. > A number of recent commits addressed these sorts of issues, including at least: […] * https://cgit.freebsd.org/src/commit/?id=9051987e40c5 https://cgit.freebsd.org/src/commit/?id=9051987e40c5 See: > The SMP system now supports up to 1024 cores on amd64 and arm64. Many kernel CPU sets are now dynamically allocated to avoid consuming excessive memory. The kernel cpuset ABI has been updated to support the higher limit. 76887e84be97[1] d1639e43c589[2] 9051987e40c5[3] e0c6e8910898[4] (Sponsored by The FreeBSD Foundation) * https://www.freebsd.org/releases/14.0R/relnotes/#kernel-general https://www.freebsd.org/releases/14.0R/relnotes/#kernel-gene...
- toast0 3y ago
  Gotta have some limit, 4x the current limit of 256 seems reasonableish. Dual socket Epyc 9654 is 96 cores * 2 threads / core * 2 sockets = 384 threads. Intel says their Xeon Platinum 8490H can live on an 8 socket board[1], if you can find one (SuperMicro has one, no price listed [2]; not sure if this is really an 8 socket system, or if it's 4x dual-socket nodes in one chassis?); 60 * 2 * 8 = 960, so that's within the limit, and 8 socket boards are pretty difficult to find. [1] https://ark.intel.com/content/www/us/en/ark/products/231747/intel-xeon-platinum-8490h-processor-112-5m-cache-1-90-ghz.html https://ark.intel.com/content/www/us/en/ark/products/231747/... [2] https://www.supermicro.com/en/products/system/mp/6u/sys-681e-tr https://www.supermicro.com/en/products/system/mp/6u/sys-681e...
  - vluft 3y ago
    9754 is 128/256 now, so 256/512 for that. That supermicro system is 8-way; it's 4 dual-socket motherboards but they're one system, hooked together by backplane boards. You can price supermicro's complete-system-only stuff (all of it now, alas) out on thinkmate or similar sites, but a minimal config (and you'd never buy that for a minimal config) hits around $60k.
- crest 3y ago
  Bitmaps for logical CPU cores and certain lock-free algorithms don't scale well to arbitrary high CPU counts e.g. reclaiming resources once in a while is O(n^2) or worse or the size of the lock structure is linear to the maximum number of cores etc. The relevant parts of the ABI have been future proofed to allow raising the kernel CPU core count limit without breaking the syscall interface for systems with less cores than the existing limit.
The_Colonel 3y ago
"supports up to" doesn't have to mean "works well/optimally with".
krylon 3y ago
If you combine multiple CPU sockets, you could get there. EPYC, IIRC, supports up to 64 cores per chip, so if you build a machine with 16 sockets, you get 1024 cores. To my knowledge, no such machine exists today, but HPE offers (or used to, anyway) a machine with 32 Xeon chips, so its core count could well reach several hundred. (I may or may not be drooling at the thought.)
- justsomehnguy 3y ago
  Intel® Xeon® Platinum 8490H Processor Total Cores 60 Total Threads 120 Max Turbo Frequency 3.50 GHz Processor Base Frequency 1.90 GHz Scalability S8S Up to 60 x 8 = 480 cores or 960 threads AMD EPYC™ 9754 # of CPU Cores 128 # of Threads 256 Max. Boost Clock Up to 3.1GHz All Core Boost Speed 3.1GHz Base Clock 2.25GHz Socket Count 1P / 2P Up to 128 x 2 = 256 cores or 512 threads
- extraduder_ire 3y ago
  It's a lot easier to build such a machine if you have something to run on it, I suppose.

> FreeBSD supports up to 1024 cores on the amd64 and arm64 platforms. Sounds pretty future proofed unless I'm missing a x86 processor out there that does this

by boznz 3y ago

> FreeBSD supports up to 1024 cores on the amd64 and arm64 platforms. Sounds pretty future proofed unless I'm missing a x86 processor out there that does this

Koshkin 3y ago

> up to 1024 Curious where this (rather large, yet still seemingly arbitrary) limit comes from.

throw0101a 3y ago
> Curious where this (rather large, yet still seemingly arbitrary) limit comes from. It is Good Enough for now, while keeping various pre-allocated, statically created structures with-in reasonable size limits: > Global and allocated arrays sized by MAXCPU result in excessive bloat on systems with lower core counts. In addition, some code used u_char (8 bits) to hold a CPU index, which is not valid if MAXCPU is greater than 256. > A number of recent commits addressed these sorts of issues, including at least: […] * https://cgit.freebsd.org/src/commit/?id=9051987e40c5 https://cgit.freebsd.org/src/commit/?id=9051987e40c5 See: > The SMP system now supports up to 1024 cores on amd64 and arm64. Many kernel CPU sets are now dynamically allocated to avoid consuming excessive memory. The kernel cpuset ABI has been updated to support the higher limit. 76887e84be97[1] d1639e43c589[2] 9051987e40c5[3] e0c6e8910898[4] (Sponsored by The FreeBSD Foundation) * https://www.freebsd.org/releases/14.0R/relnotes/#kernel-general https://www.freebsd.org/releases/14.0R/relnotes/#kernel-gene...
toast0 3y ago
Gotta have some limit, 4x the current limit of 256 seems reasonableish. Dual socket Epyc 9654 is 96 cores * 2 threads / core * 2 sockets = 384 threads. Intel says their Xeon Platinum 8490H can live on an 8 socket board[1], if you can find one (SuperMicro has one, no price listed [2]; not sure if this is really an 8 socket system, or if it's 4x dual-socket nodes in one chassis?); 60 * 2 * 8 = 960, so that's within the limit, and 8 socket boards are pretty difficult to find. [1] https://ark.intel.com/content/www/us/en/ark/products/231747/intel-xeon-platinum-8490h-processor-112-5m-cache-1-90-ghz.html https://ark.intel.com/content/www/us/en/ark/products/231747/... [2] https://www.supermicro.com/en/products/system/mp/6u/sys-681e-tr https://www.supermicro.com/en/products/system/mp/6u/sys-681e...
- vluft 3y ago
  9754 is 128/256 now, so 256/512 for that. That supermicro system is 8-way; it's 4 dual-socket motherboards but they're one system, hooked together by backplane boards. You can price supermicro's complete-system-only stuff (all of it now, alas) out on thinkmate or similar sites, but a minimal config (and you'd never buy that for a minimal config) hits around $60k.
crest 3y ago
Bitmaps for logical CPU cores and certain lock-free algorithms don't scale well to arbitrary high CPU counts e.g. reclaiming resources once in a while is O(n^2) or worse or the size of the lock structure is linear to the maximum number of cores etc. The relevant parts of the ABI have been future proofed to allow raising the kernel CPU core count limit without breaking the syscall interface for systems with less cores than the existing limit.

The_Colonel 3y ago

"supports up to" doesn't have to mean "works well/optimally with".

krylon 3y ago

If you combine multiple CPU sockets, you could get there. EPYC, IIRC, supports up to 64 cores per chip, so if you build a machine with 16 sockets, you get 1024 cores. To my knowledge, no such machine exists today, but HPE offers (or used to, anyway) a machine with 32 Xeon chips, so its core count could well reach several hundred. (I may or may not be drooling at the thought.)

justsomehnguy 3y ago
Intel® Xeon® Platinum 8490H Processor Total Cores 60 Total Threads 120 Max Turbo Frequency 3.50 GHz Processor Base Frequency 1.90 GHz Scalability S8S Up to 60 x 8 = 480 cores or 960 threads AMD EPYC™ 9754 # of CPU Cores 128 # of Threads 256 Max. Boost Clock Up to 3.1GHz All Core Boost Speed 3.1GHz Base Clock 2.25GHz Socket Count 1P / 2P Up to 128 x 2 = 256 cores or 512 threads
extraduder_ire 3y ago
It's a lot easier to build such a machine if you have something to run on it, I suppose.