4 ms·
One of my co-workers manages the SGI UV300 described at http://med.stanford.edu/gbsc/uv300.html http://med.stanford.edu/gbsc/uv300.html (new purchased get hardw
by CaliforniaKarl 8y ago
One of my co-workers manages the SGI UV300 described at http://med.stanford.edu/gbsc/uv300.html http://med.stanford.edu/gbsc/uv300.html (new purchased get hardware badged as the HPE MC990 X). It does fit most definitions of a supercomputer, as it's multiple nodes with interconnects that allow a single OS instance to run across all of them (the OS then sees & manages the separate NUMA nodes).
One thing that has been learned is, software will sometimes do interesting things if presented with so much memory. For example: https://github.com/zfsonlinux/zfs/issues/7275 https://github.com/zfsonlinux/zfs/issues/7275
- rbanffy 8y agoSpeaking of SGI, I was expecting the Cray XC family to be able to do that with the ludicrously fast interconnects they have, but I couldn't find any mention of it.
- gaius 8y agoNo, an XC40 runs a kernel on each node, the programming model is MPI, not SSI with NUMA. It runs a minimalist Linux distro called CLE. The nodes are surprisingly small, in terms of memory too, 64 or 128G. Physically they are beautiful to look at, so clean, just a CPU and some memory and nothing extraneous. Cray really should do glass cases.
- rbanffy 8y agoThey used to make beautiful computers.
- gnufx 8y agoThe salient hardware difference is probably cache coherence in the SGI interconnect.
- gaius 8y agoNo, the Ares interconnect is brutally fast but it fundamentally isn’t doing anything as complicated as that. It acts as both storage and networking between the nodes.
- gnufx 8y agoI missed this at the time, but yes -- that's the point, in comparison with NUMAlink or whatever it is now. (s/Ares/Aries/)
- gnufx 8y agoI'm fairly sure UV (or some SGI labelling) has gone up to 1024-ish cores in a single system image, but I don't have a reference. (SGI Origin 3000 was spec'ed to 512 cores long ago.)
- fanf2 8y ago1536 cores on Cambridge’s COSMOS SGI Altix UV2000 system: http://www.cosmos.damtp.cam.ac.uk http://www.cosmos.damtp.cam.ac.uk
- rootbear 8y agoThe group I'm in at NASA Goddard has two UV300s. They have 96 cores each (no hyperthreading) and 6TB of RAM. And they are loud. The SGI Altix line, which used Itanium 2, could go up to 1024 cores with a single system image, which is just insane. We had one of those at Goddard a decade or so ago, but I didn't have access to it.