6 ms·

Really cool. I remember when Sony really pushed the idea that the Cell arch would power so many different things, shame that never took off. I wonder if that u

Really cool. I remember when Sony really pushed the idea that the Cell arch would power so many different things, shame that never took off. I wonder if that university still uses their PS3 based supercomputer cluster

pavlov 5y ago
Hmm, probably not. Seems like the biggest PS3 cluster had 1,760 units and was rated at 500 teraflops (single precision float, presumably). An NVIDIA A100 GPU does about 20 teraflops. So you only need 25 of those chips to match the theoretical rating, and they have many other advantages like much higher memory per core, etc.
- teruakohatu 5y ago
  Have you got the NVIDIA numbers right? A NVIDIA DGX Station A100 (desktop/workstation sized computer with 4x A100 and draws 1.5 kW of power, is rated at 2.5 petaFLOPS, so a good 30+% more than the PS3 cluster. Also the PS3 apparently drew up to 200w, so a cluster that size would have drawn 352 kW.
  - spijdar 5y ago
    The units being used here are likely different. In most press releases Nvidia uses their "tensor core" performance, usually with either sparsity or 16 bit data. A single A100 is said to have 320 teraflops of "tensor float" performance but only 19 teraflops of "normal" full FP32 performance. This is way out of my field so I don't know the whole implications, but my understanding is Nvidia cards cam only reach these speeds at the loss of precision or full functionality, so it's an apples to oranges comparison versus non-nvidia chips.
    - deleted 5y ago
      [deleted]
pcwalton 5y ago
I mean, the Cell SPU design is everywhere, it's just now part of every GPU and called compute shader. Vector processors are a good idea, but they have to be accessible to programmers through standard, easy-to-use, ideally-cross-platform APIs or invariably they won't get used to their full potential.
urthor 5y ago
People forget that this was 2006. CUDA was not invented yet. And arguably didn't take off until Alexnet. NVIDIA had not even released the API for writing vectorised C code yet. Hence all those 2007 supercomputer stories. They actually had a genuine use, because the Cell was fully implemented into the Linux kernel by IBM. These guys did incredibly well for their time, and they were entirely right about vectorised code. They were just superseded by the longer term trend of tying together multiple silicon dies and ASICs for HPC.
- bcatanzaro 5y ago
  CUDA was released in beta in November 2006, so it was invented by this point, even if it wasn’t well known.
jabl 5y ago
> I wonder if that university still uses their PS3 based supercomputer cluster Almost certainly not. A typical lifetime of a supercomputer is around 5 years, give or take. After that the electricity they consume makes it not worth continuing to run them vs. buying a new one. See e.g. the Cell-based Roadrunner, in use 2008-2013: https://en.wikipedia.org/wiki/Roadrunner_(supercomputer) https://en.wikipedia.org/wiki/Roadrunner_(supercomputer)
- dragontamer 5y ago
  5 is a bit young. ORNL's Titan lasted about 7 years: 2012 through 2019. Its predecessor, Jaguar, was 2005 to 2012. Also 7 years.
  - freemint 5y ago
    Notice the N in National Labs, entirely different order of magnitude.
  - jabl 5y ago
    AFAIK both Titan and Jaguar received significant mid-life upgrades. So in such a scenario 7 years sounds reasonable. (At a previous job, we had a cluster that was about 15 years old. Of course, it had been expanded and upgraded over the years, so I'm not sure anything was left of the original. Maybe some racks and power cables.. :) )
jgtrosh 5y ago
Afaik these university projects were done to test out how Cell-based supercomputers would behave. They're not competitive with up-to-date Supercomputers. In my lab I found a couple of PS3s lying around several years ago, that hadn't been used in quite a while. (One of them may or may not have been adopted for less scientific purposes …)
flipacholas 5y ago
If you are referring to the Barcelona Supercomputing Center, I think it's been decommissioned (their project is now archived https://web.archive.org/web/20090426190617/https://www.bsc.es/projects/deepcomputing/linuxoncell/ https://web.archive.org/web/20090426190617/https://www.bsc.e...). Though this is expected in my opinion (Buy equipment -> Produce research -> Move to the next thing). IIRC their big thing is now the MareNostrum 4 supercomputer (https://en.wikipedia.org/wiki/MareNostrum https://en.wikipedia.org/wiki/MareNostrum).
shaggie76 5y ago
I worked on PS3 titles that used SPUs and I can't tell you how relieved I was to find that the PS4 did away with them. The fear that the next PS would have "64 SPUs and a marginally faster PPU" was real. The analogy I gave to my friends at the time was working in a restaurant kitchen with a tiny stovetop and 2-dozen microwave ovens; does some things really fast, but only if you can cut them up into small pieces that are microwave-friendly.
- jiggawatts 5y ago
  The SPU approach is pretty much what a GPU does, but instead of 64 they have hundreds if not thousands. I think the main issue with the Cell design was that it was too "middle road" and wasn't specialised enough in either direction.
  - pcwalton 5y ago
    > The SPU approach is pretty much what a GPU does, but instead of 64 they have hundreds if not thousands. Eh, only if you count every SIMD lane as a separate "core" like GPU manufacturer marketing does. More realistically, you should count what NVIDIA calls SMs, where the numbers are more comparable (GeForce RTX 3080 has 80, for example).
    - pandaman 5y ago
      PS3 6 SPUs(1 is system reserved) 4-way SIMD PS4 - 18 GCN CUs, each has four 16-ways SIMDs for 72 SIMDs but each is 4 times wider so PS4 has the same number of ways as in 288 4-ways SPUs.
  - unsigner 5y ago
    It really isn’t - the individual elements in a GPU can access RAM directly, while you needed to wheelbarrow data (and code!) to and from SPUs manually. This was the major pain point, not the extreme (for that time) parallelism or the weird-ish instruction set.
    - 5y ago
em500 5y ago
The hype was probably mostly from "Crazy" Ken Kutaragi, who had a knack for dialing the Playstation hype beyond 11. The PS2 was already supposed to replace your home PC and revolutionize ecommerce and online gaming and plug yourself into the Matrix [1]. In the past he was compared to Steve Jobs by some, but in hindsight he seems more like Sony's Baghdad Bob. [1] https://www.newsweek.com/here-comes-playstation-2-156589 https://www.newsweek.com/here-comes-playstation-2-156589
- djmips 5y ago
  Being forced kicking and screaming into the multicore on the PS3 improved our PC codebase by quite a good margin. So what I'm trying to say is that the hype was real, we have gone multicore, just not with the Cell architecture.
  - Dylan16807 5y ago
    Did the 360 not drag things into multicore very similarly? Pretending its CPU has one core is a pretty similar experience to pretending a PS3 has no SPEs.
- monocasa 5y ago
  At least the home computer part of that was a shtick to avoid some european import taxes. https://www.theguardian.com/technology/2003/oct/01/business.games https://www.theguardian.com/technology/2003/oct/01/business.... I've heard on the grapevine that the PS3's OtherOS facility was internally thought of as another go at the same idea. "Look, judge, it's a general purpose computer for reals this time. Your own universities are using it in super computing clusters, without ever launching a game".
Rodeoclash 5y ago
I remember speculation at the time that the PS3 would be able to borrow from other cell powered appliances in your house when it was running, i.e. your toaster, to enhance its power. Actually, I just did a quick Google on this and it was "Sony" themselves that appeared to mention this! [1] [1] https://www.tomshardware.com/news/cell-broadband-engine-ps3-wii,11442.html https://www.tomshardware.com/news/cell-broadband-engine-ps3-...
- bitwize 5y ago
  There was even a Penny Arcade comic about that: https://www.penny-arcade.com/comic/2002/08/07 https://www.penny-arcade.com/comic/2002/08/07
mustacheemperor 5y ago
In 2012 the Air Force Research Laboratory built a supercomputer cluster from 1760 PS3s that I think was in practical use for a while[0]. I recall reading online that the Air Force struck a special deal with Sony to buy some of the last remaining PS3s that had not been updated to no longer support Linux installation during manufacturing, but that isn't mentioned in this source. >The Condor Cluster project began four years ago, when PlayStation consoles cost about $400 each. At the same time, comparable technology would have cost about $10,000 per unit. Overall, the PS3s for the supercomputer's core cost about $2 million. According to AFRL Director of High Power Computing Mark Barnell, that cost is about 5-10% of the cost of an equivalent system built with off-the-shelf computer parts. >Another advantage of the PS3-based supercomputer is its energy efficiency: it consumes just 10% of the power of comparable supercomputers. I wonder how significant the cost and energy savings were by the time the project was finished, and how long the cluster was actually used. [0]https://phys.org/news/2010-12-air-playstation-3s-supercomputer.html https://phys.org/news/2010-12-air-playstation-3s-supercomput...
- Nbox9 5y ago
  PS3 clusters are limited in supercomputing tasks largely by RAM, each PS3 having 256MB which is anemic.
  - pjmlp 5y ago
    Still, https://en.wikipedia.org/wiki/PlayStation_3_cluster https://en.wikipedia.org/wiki/PlayStation_3_cluster
  - flatiron 5y ago
    I can think of a ton of number crunching you can do where each node having 256 wouldn’t be awful. Like maybe brute forcing stuff where they can just segment the use case to each node. But it would certainly be a specialized case.

pavlov 5y ago

Hmm, probably not. Seems like the biggest PS3 cluster had 1,760 units and was rated at 500 teraflops (single precision float, presumably). An NVIDIA A100 GPU does about 20 teraflops. So you only need 25 of those chips to match the theoretical rating, and they have many other advantages like much higher memory per core, etc.

teruakohatu 5y ago
Have you got the NVIDIA numbers right? A NVIDIA DGX Station A100 (desktop/workstation sized computer with 4x A100 and draws 1.5 kW of power, is rated at 2.5 petaFLOPS, so a good 30+% more than the PS3 cluster. Also the PS3 apparently drew up to 200w, so a cluster that size would have drawn 352 kW.
- spijdar 5y ago
  The units being used here are likely different. In most press releases Nvidia uses their "tensor core" performance, usually with either sparsity or 16 bit data. A single A100 is said to have 320 teraflops of "tensor float" performance but only 19 teraflops of "normal" full FP32 performance. This is way out of my field so I don't know the whole implications, but my understanding is Nvidia cards cam only reach these speeds at the loss of precision or full functionality, so it's an apples to oranges comparison versus non-nvidia chips.
  - deleted 5y ago
    [deleted]

pcwalton 5y ago

I mean, the Cell SPU design is everywhere, it's just now part of every GPU and called compute shader. Vector processors are a good idea, but they have to be accessible to programmers through standard, easy-to-use, ideally-cross-platform APIs or invariably they won't get used to their full potential.

urthor 5y ago

People forget that this was 2006. CUDA was not invented yet. And arguably didn't take off until Alexnet. NVIDIA had not even released the API for writing vectorised C code yet. Hence all those 2007 supercomputer stories. They actually had a genuine use, because the Cell was fully implemented into the Linux kernel by IBM. These guys did incredibly well for their time, and they were entirely right about vectorised code. They were just superseded by the longer term trend of tying together multiple silicon dies and ASICs for HPC.

bcatanzaro 5y ago
CUDA was released in beta in November 2006, so it was invented by this point, even if it wasn’t well known.

jabl 5y ago

> I wonder if that university still uses their PS3 based supercomputer cluster Almost certainly not. A typical lifetime of a supercomputer is around 5 years, give or take. After that the electricity they consume makes it not worth continuing to run them vs. buying a new one. See e.g. the Cell-based Roadrunner, in use 2008-2013: https://en.wikipedia.org/wiki/Roadrunner_(supercomputer) https://en.wikipedia.org/wiki/Roadrunner_(supercomputer)

dragontamer 5y ago
5 is a bit young. ORNL's Titan lasted about 7 years: 2012 through 2019. Its predecessor, Jaguar, was 2005 to 2012. Also 7 years.
- freemint 5y ago
  Notice the N in National Labs, entirely different order of magnitude.
- jabl 5y ago
  AFAIK both Titan and Jaguar received significant mid-life upgrades. So in such a scenario 7 years sounds reasonable. (At a previous job, we had a cluster that was about 15 years old. Of course, it had been expanded and upgraded over the years, so I'm not sure anything was left of the original. Maybe some racks and power cables.. :) )

jgtrosh 5y ago

Afaik these university projects were done to test out how Cell-based supercomputers would behave. They're not competitive with up-to-date Supercomputers. In my lab I found a couple of PS3s lying around several years ago, that hadn't been used in quite a while. (One of them may or may not have been adopted for less scientific purposes …)

flipacholas 5y ago

If you are referring to the Barcelona Supercomputing Center, I think it's been decommissioned (their project is now archived https://web.archive.org/web/20090426190617/https://www.bsc.es/projects/deepcomputing/linuxoncell/ https://web.archive.org/web/20090426190617/https://www.bsc.e...). Though this is expected in my opinion (Buy equipment -> Produce research -> Move to the next thing). IIRC their big thing is now the MareNostrum 4 supercomputer (https://en.wikipedia.org/wiki/MareNostrum https://en.wikipedia.org/wiki/MareNostrum).

shaggie76 5y ago

I worked on PS3 titles that used SPUs and I can't tell you how relieved I was to find that the PS4 did away with them. The fear that the next PS would have "64 SPUs and a marginally faster PPU" was real. The analogy I gave to my friends at the time was working in a restaurant kitchen with a tiny stovetop and 2-dozen microwave ovens; does some things really fast, but only if you can cut them up into small pieces that are microwave-friendly.

jiggawatts 5y ago
The SPU approach is pretty much what a GPU does, but instead of 64 they have hundreds if not thousands. I think the main issue with the Cell design was that it was too "middle road" and wasn't specialised enough in either direction.
- pcwalton 5y ago
  > The SPU approach is pretty much what a GPU does, but instead of 64 they have hundreds if not thousands. Eh, only if you count every SIMD lane as a separate "core" like GPU manufacturer marketing does. More realistically, you should count what NVIDIA calls SMs, where the numbers are more comparable (GeForce RTX 3080 has 80, for example).
  - pandaman 5y ago
    PS3 6 SPUs(1 is system reserved) 4-way SIMD PS4 - 18 GCN CUs, each has four 16-ways SIMDs for 72 SIMDs but each is 4 times wider so PS4 has the same number of ways as in 288 4-ways SPUs.
- unsigner 5y ago
  It really isn’t - the individual elements in a GPU can access RAM directly, while you needed to wheelbarrow data (and code!) to and from SPUs manually. This was the major pain point, not the extreme (for that time) parallelism or the weird-ish instruction set.
  - 5y ago

em500 5y ago

The hype was probably mostly from "Crazy" Ken Kutaragi, who had a knack for dialing the Playstation hype beyond 11. The PS2 was already supposed to replace your home PC and revolutionize ecommerce and online gaming and plug yourself into the Matrix [1]. In the past he was compared to Steve Jobs by some, but in hindsight he seems more like Sony's Baghdad Bob. [1] https://www.newsweek.com/here-comes-playstation-2-156589 https://www.newsweek.com/here-comes-playstation-2-156589

djmips 5y ago
Being forced kicking and screaming into the multicore on the PS3 improved our PC codebase by quite a good margin. So what I'm trying to say is that the hype was real, we have gone multicore, just not with the Cell architecture.
- Dylan16807 5y ago
  Did the 360 not drag things into multicore very similarly? Pretending its CPU has one core is a pretty similar experience to pretending a PS3 has no SPEs.
monocasa 5y ago
At least the home computer part of that was a shtick to avoid some european import taxes. https://www.theguardian.com/technology/2003/oct/01/business.games https://www.theguardian.com/technology/2003/oct/01/business.... I've heard on the grapevine that the PS3's OtherOS facility was internally thought of as another go at the same idea. "Look, judge, it's a general purpose computer for reals this time. Your own universities are using it in super computing clusters, without ever launching a game".

Rodeoclash 5y ago

I remember speculation at the time that the PS3 would be able to borrow from other cell powered appliances in your house when it was running, i.e. your toaster, to enhance its power. Actually, I just did a quick Google on this and it was "Sony" themselves that appeared to mention this! [1] [1] https://www.tomshardware.com/news/cell-broadband-engine-ps3-wii,11442.html https://www.tomshardware.com/news/cell-broadband-engine-ps3-...

bitwize 5y ago
There was even a Penny Arcade comic about that: https://www.penny-arcade.com/comic/2002/08/07 https://www.penny-arcade.com/comic/2002/08/07

mustacheemperor 5y ago

In 2012 the Air Force Research Laboratory built a supercomputer cluster from 1760 PS3s that I think was in practical use for a while[0]. I recall reading online that the Air Force struck a special deal with Sony to buy some of the last remaining PS3s that had not been updated to no longer support Linux installation during manufacturing, but that isn't mentioned in this source. >The Condor Cluster project began four years ago, when PlayStation consoles cost about $400 each. At the same time, comparable technology would have cost about $10,000 per unit. Overall, the PS3s for the supercomputer's core cost about $2 million. According to AFRL Director of High Power Computing Mark Barnell, that cost is about 5-10% of the cost of an equivalent system built with off-the-shelf computer parts. >Another advantage of the PS3-based supercomputer is its energy efficiency: it consumes just 10% of the power of comparable supercomputers. I wonder how significant the cost and energy savings were by the time the project was finished, and how long the cluster was actually used. [0]https://phys.org/news/2010-12-air-playstation-3s-supercomputer.html https://phys.org/news/2010-12-air-playstation-3s-supercomput...

Nbox9 5y ago
PS3 clusters are limited in supercomputing tasks largely by RAM, each PS3 having 256MB which is anemic.
- pjmlp 5y ago
  Still, https://en.wikipedia.org/wiki/PlayStation_3_cluster https://en.wikipedia.org/wiki/PlayStation_3_cluster
- flatiron 5y ago
  I can think of a ton of number crunching you can do where each node having 256 wouldn’t be awful. Like maybe brute forcing stuff where they can just segment the use case to each node. But it would certainly be a specialized case.