7 ms·

When I left Sun in 1995 their "biggest" Iron was the Enterprise 10K (which internally was called "Dragon" because of the Xerox bus) A system with 64 cores and 2

When I left Sun in 1995 their "biggest" Iron was the Enterprise 10K (which internally was called "Dragon" because of the Xerox bus) A system with 64 cores and 256GB of RAM was just under 2.5 million dollars list. It needed over 10kW of power provided by a 60A 240V circuit. The power cord weighed in at like 20 lbs. I put together a new desktop with the TR3960 and 128GB of ECC ram, that motherboard will take the 3990 and 256GB of RAM if I chose to upgrade it. It really boggles my mind what you can fit under your desk these days with a single 120V outlet.

chx 7y ago
When the univ I went to at Budapest got a second hand VAX cluster (also around '95) for the bargain basement price of only 50 000 CHF they needed to dig up the street to the nearest substation and have a new powerline installed. http://hampage.hu/oldiron/vaxen/9000_4.jpg http://hampage.hu/oldiron/vaxen/9000_4.jpg the furthest away cabinet is the power supply. This photo is not even half of the cluster. A few years before that, at another univ, they put an ancient IBM mainframe in place with a crane, temporarily removing the roof of the building.
jacquesm 7y ago
The thing that boggles my mind is what people casually put in their pockets.
- solotronics 7y ago
  I think we already live in the cyberpunk/sci-fi realm in this regard. Soon we will have swarms of tiny AI powered robots...
  - andy_ppp 7y ago
    I was thinking about this for house building, you go to a panel and the bots that make up the house reconfigure themselves to add a swimming pool, or an extra guest room, etc. Could be pretty awesome. Let’s hope they are not used for evil :-/
    - jotm 7y ago
      Until you try to open your door once and it goes "intruder alert"... Wasn't that a Doctor Who episode? :D
jonas21 7y ago
In 2001, the fastest supercomputer in the world was ASCI White. It cost $110M, weighed 106 tons, consumed 3MW of power (plus 3MW for cooling), and had a peak speed of 12.3 TFLOPS. Right now, sitting under my desk is a RTX 2080 Ti GPU which cost around $1000, weighs 3 pounds, draws a maximum of 250 watts, and has a peak speed of 13.4 TFLOPS [1]. We truly live in amazing times. [1] Not quite a fair comparison: the GPU is using 32-bit floating-point, while ASCI White used 64-bit. But for many applications, the precision difference doesn't matter.
- nullc 7y ago
  > while ASCI White used 64-bit. x87 is 80-bit. :)
  - alkonaut 7y ago
    ASCI White wasn't Intel (it was IBM Power3)
    - nullc 7y ago
      oh, I was thinking ASCI red! fair enough!
- sillysaurusx 7y ago
  It's not fast enough. After having access to 160 TPUs, it's physically painful to use anything else. I hope in 20 years I'll have the equivalent of 160 TPUs under my desk. Hopefully less. The reason it's not fast enough is that ... there's so much you can do! People don't know. You can't really know until you have access to such a vast amount of horsepower, and can apply it to whatever you want. You might think "What could I possibly use it for?" but there are so many things. The most important thing you can use it for is fun, and intellectual gratification. You can train ML models just to see what they do. And as AI Dungeon shows, sometimes you win the lottery. I can't wait for the future. It's going to be so cool.
  - fluffything 7y ago
boris 7y ago
Could you share the specs for your build, specifically the motherboard and RAM models. Also have you verified that ECC actually works? I am tempted to build something similar but AMD's wishy-washy ECC guarantees as well as Linux-specific issues make me unsure.
- ChuckMcM 7y ago
  TR3960X Threadripper (24 core, 48 thread) ASUS PRIME TRX40-Pro motherboard 8 sticks Kingston KSM26ED8/16ME (16GB, 2666Mhz DDR4, ECC) Dual boot Windows 10 / Ubuntu 18.04 2x Samsung EVO 970 NvME M.2 1TB SSDs in RAID0 configuration. Running in a Coolermaster Cosmos case with a stupidly big air cooler at the moment, to be replaced with a decent liquid cooler (it works, the Cosmos is a huge case because it had to be to hold a hacked Supermicro dual Xeon server board in it before (I really wanted ECC for my workstation)) nVidia 1080Ti+ GPU. The ECC is detected and claims it is working although I've yet to see it correct an SBE. I haven't been running non-stop memory tests either though so. I may end up removing the Linux partition since WSL2 works so well on this box.
  - voltagex_ 7y ago
    What are you going to be using for the liquid cooler?
  - glandium 7y ago
    I have a 3970X with ECC RAM and got error correction notifications in my Linux logs when I tweaked RAM timings too tight. Note that memtest86 doesn't know about ECC on Ryzen, so you may get unnotified error corrections happening if you use that.
    - boris 7y ago
      Thanks for the info. Is this the same motherboard as OP?
      - glandium 7y ago
fulafel 7y ago
OTOH we'd rather have seen the single-core perf keep improving. What's the average performance speedup (vs 1 core) that the Sun customers got, or the AMD users get, on the average software they use? The progress in programming language technology hasn't been very kind to multiprocessing[1]. [1] GPUs are another kettle of fish of course, but have their own wellknown problems that prevent widespread use outside graphics
mlang23 7y ago
I recently "upgraded" from a desktop to a laptop. This marks the end of an era for me. I always had a relatively powerful desktop at home, mostly running 24/7 since I couldn't be bothered to wait for it to boot. This ThinkPad X1 is the first laptop I own which is apparently powerful enough to host all my work in a 1.09kg package that easily fits in my backpack... OK, I am a text mode user, so gfx isn't what keeps my computers busy. Low latency realtime audio synthesis much more so. And I still remember when 33.6kbps were an exciting thing to have :-) Nice to see that tech moves ahead.
jankotek 7y ago
7 years ago it was doable to build similar desktop at similar price from refurbished server (4x6core Opteron CPU, 128GB DDR2 ECC RAM). It took about 1KW and was loud, but great complement for performance testing. BTW I seen 64GB DDR4 sticks on amazon...
- tracker1 7y ago
  The memory access performance on ThreadRipper (and current Ryzen/Epyc) is much better than anything prior to this generation for workstation loads. Not that the shared I/O controller is without cost, only that it tends to average out better in most cases where multiple cores across chips are in use together. Just got my 3950X w/ 64gb ram, not sure that I'd be able to practically use any more compute than this for what I play with, which is mostly multiple back-ends and some container orchestration for local dev and occasionally video re-encodes (BR-Ripping for NAS). Some think $4k for this CPU is too much... considering the shear performance that you can get these days for under $10K there's never been a better time to build or buy a computer. My only regret is wasting time and money on aRGB that I cannot configure in Linux.
agumonkey 7y ago
sorry for the buzz kill but I feel super weird about how something that was once a corporation class tool is now a casual consumer thingie. I do wish for exascale in the medical field though. I even-more-wish that the energy world can see ~similar improvements in efficiency.
- cr0sh 7y ago
  The real funny thing is that for most people (not OP) it would still be used mainly for word processing, email, and occasionally casual gaming - and it would still be slow to run and boot. The amount of processing power we each carry in our pockets (even the cheapest throw-away smart phones) would have been almost unthinkable 30 years ago; it's akin to the difference of an Altair of the 1970s vs what was available just 10-20 years prior. What took up a room now sat on a desk and could be purchased for the price of a car. Now, what took up a room now sits in your pocket, and almost could be given away in a box of cereal its so cheap. Heck - think about what's available in the embedded computing realm for pennies (or just a few dollars in single quantities) - it's mind boggling to an extent.
  - agumonkey 7y ago
    I concur but it depresses me somehow. Some say that it's worth it, to me it's just the same cycle of marketing trying to disguise the things as progress.
lokedhs 7y ago
Are you sure it was in 1995? I joined Sun in 1997, and the E10k was launched a bit after that. According to Wikipedia it was launched in 1997 so it does line up. If I remember correctly, the system was bought from Cray after SGI bought the rest, so I didn't know they even had it at Sun in 1995. Also, the original model of the E10k supported 64 GB RAM.
- tverbeure 7y ago
  My company back then bought one of those around that time. It was used to run large EDA software jobs. On one beautiful day during the weekend, only a two weeks after delivery, our system admin noticed that the machine went offline. Logging in remotely didn’t work at all. No ping either. He drove to work and ... the machine was gone. Thieves had used a crane to lift it out of the building through a window onto a truck. Sun told him that this wasn’t the first time such a thing had happened and that somewhere in the chain from order to delivery, an insider tipped off the thieves about where to find the latest.
  - selectodude 7y ago
    I'm curious where the black market for something like that would even come from?
    - cr0sh 7y ago
      Back then, smaller nation-states wanting to do nuclear device simulation and the like would be my guess. Basically, countries that were restricted in some manner on gaining large amounts of parallel processing compute power for such simulations.
      - tverbeure 7y ago
        My money is on breaking it up and sell it for components. The 256 GB in server grade RAM alone cost a fortune.

chx 7y ago

When the univ I went to at Budapest got a second hand VAX cluster (also around '95) for the bargain basement price of only 50 000 CHF they needed to dig up the street to the nearest substation and have a new powerline installed. http://hampage.hu/oldiron/vaxen/9000_4.jpg http://hampage.hu/oldiron/vaxen/9000_4.jpg the furthest away cabinet is the power supply. This photo is not even half of the cluster. A few years before that, at another univ, they put an ancient IBM mainframe in place with a crane, temporarily removing the roof of the building.

jacquesm 7y ago

The thing that boggles my mind is what people casually put in their pockets.

solotronics 7y ago
I think we already live in the cyberpunk/sci-fi realm in this regard. Soon we will have swarms of tiny AI powered robots...
- andy_ppp 7y ago
  I was thinking about this for house building, you go to a panel and the bots that make up the house reconfigure themselves to add a swimming pool, or an extra guest room, etc. Could be pretty awesome. Let’s hope they are not used for evil :-/
  - jotm 7y ago
    Until you try to open your door once and it goes "intruder alert"... Wasn't that a Doctor Who episode? :D

jonas21 7y ago

In 2001, the fastest supercomputer in the world was ASCI White. It cost $110M, weighed 106 tons, consumed 3MW of power (plus 3MW for cooling), and had a peak speed of 12.3 TFLOPS. Right now, sitting under my desk is a RTX 2080 Ti GPU which cost around $1000, weighs 3 pounds, draws a maximum of 250 watts, and has a peak speed of 13.4 TFLOPS [1]. We truly live in amazing times. [1] Not quite a fair comparison: the GPU is using 32-bit floating-point, while ASCI White used 64-bit. But for many applications, the precision difference doesn't matter.

nullc 7y ago
> while ASCI White used 64-bit. x87 is 80-bit. :)
- alkonaut 7y ago
  ASCI White wasn't Intel (it was IBM Power3)
  - nullc 7y ago
    oh, I was thinking ASCI red! fair enough!
sillysaurusx 7y ago
It's not fast enough. After having access to 160 TPUs, it's physically painful to use anything else. I hope in 20 years I'll have the equivalent of 160 TPUs under my desk. Hopefully less. The reason it's not fast enough is that ... there's so much you can do! People don't know. You can't really know until you have access to such a vast amount of horsepower, and can apply it to whatever you want. You might think "What could I possibly use it for?" but there are so many things. The most important thing you can use it for is fun, and intellectual gratification. You can train ML models just to see what they do. And as AI Dungeon shows, sometimes you win the lottery. I can't wait for the future. It's going to be so cool.
- fluffything 7y ago

boris 7y ago

Could you share the specs for your build, specifically the motherboard and RAM models. Also have you verified that ECC actually works? I am tempted to build something similar but AMD's wishy-washy ECC guarantees as well as Linux-specific issues make me unsure.

ChuckMcM 7y ago
TR3960X Threadripper (24 core, 48 thread) ASUS PRIME TRX40-Pro motherboard 8 sticks Kingston KSM26ED8/16ME (16GB, 2666Mhz DDR4, ECC) Dual boot Windows 10 / Ubuntu 18.04 2x Samsung EVO 970 NvME M.2 1TB SSDs in RAID0 configuration. Running in a Coolermaster Cosmos case with a stupidly big air cooler at the moment, to be replaced with a decent liquid cooler (it works, the Cosmos is a huge case because it had to be to hold a hacked Supermicro dual Xeon server board in it before (I really wanted ECC for my workstation)) nVidia 1080Ti+ GPU. The ECC is detected and claims it is working although I've yet to see it correct an SBE. I haven't been running non-stop memory tests either though so. I may end up removing the Linux partition since WSL2 works so well on this box.
- voltagex_ 7y ago
  What are you going to be using for the liquid cooler?
- glandium 7y ago
  I have a 3970X with ECC RAM and got error correction notifications in my Linux logs when I tweaked RAM timings too tight. Note that memtest86 doesn't know about ECC on Ryzen, so you may get unnotified error corrections happening if you use that.
  - boris 7y ago
    Thanks for the info. Is this the same motherboard as OP?
    - glandium 7y ago

fulafel 7y ago

OTOH we'd rather have seen the single-core perf keep improving. What's the average performance speedup (vs 1 core) that the Sun customers got, or the AMD users get, on the average software they use? The progress in programming language technology hasn't been very kind to multiprocessing[1]. [1] GPUs are another kettle of fish of course, but have their own wellknown problems that prevent widespread use outside graphics

mlang23 7y ago

I recently "upgraded" from a desktop to a laptop. This marks the end of an era for me. I always had a relatively powerful desktop at home, mostly running 24/7 since I couldn't be bothered to wait for it to boot. This ThinkPad X1 is the first laptop I own which is apparently powerful enough to host all my work in a 1.09kg package that easily fits in my backpack... OK, I am a text mode user, so gfx isn't what keeps my computers busy. Low latency realtime audio synthesis much more so. And I still remember when 33.6kbps were an exciting thing to have :-) Nice to see that tech moves ahead.

jankotek 7y ago

7 years ago it was doable to build similar desktop at similar price from refurbished server (4x6core Opteron CPU, 128GB DDR2 ECC RAM). It took about 1KW and was loud, but great complement for performance testing. BTW I seen 64GB DDR4 sticks on amazon...

tracker1 7y ago
The memory access performance on ThreadRipper (and current Ryzen/Epyc) is much better than anything prior to this generation for workstation loads. Not that the shared I/O controller is without cost, only that it tends to average out better in most cases where multiple cores across chips are in use together. Just got my 3950X w/ 64gb ram, not sure that I'd be able to practically use any more compute than this for what I play with, which is mostly multiple back-ends and some container orchestration for local dev and occasionally video re-encodes (BR-Ripping for NAS). Some think $4k for this CPU is too much... considering the shear performance that you can get these days for under $10K there's never been a better time to build or buy a computer. My only regret is wasting time and money on aRGB that I cannot configure in Linux.

agumonkey 7y ago

sorry for the buzz kill but I feel super weird about how something that was once a corporation class tool is now a casual consumer thingie. I do wish for exascale in the medical field though. I even-more-wish that the energy world can see ~similar improvements in efficiency.

cr0sh 7y ago
The real funny thing is that for most people (not OP) it would still be used mainly for word processing, email, and occasionally casual gaming - and it would still be slow to run and boot. The amount of processing power we each carry in our pockets (even the cheapest throw-away smart phones) would have been almost unthinkable 30 years ago; it's akin to the difference of an Altair of the 1970s vs what was available just 10-20 years prior. What took up a room now sat on a desk and could be purchased for the price of a car. Now, what took up a room now sits in your pocket, and almost could be given away in a box of cereal its so cheap. Heck - think about what's available in the embedded computing realm for pennies (or just a few dollars in single quantities) - it's mind boggling to an extent.
- agumonkey 7y ago
  I concur but it depresses me somehow. Some say that it's worth it, to me it's just the same cycle of marketing trying to disguise the things as progress.

lokedhs 7y ago

Are you sure it was in 1995? I joined Sun in 1997, and the E10k was launched a bit after that. According to Wikipedia it was launched in 1997 so it does line up. If I remember correctly, the system was bought from Cray after SGI bought the rest, so I didn't know they even had it at Sun in 1995. Also, the original model of the E10k supported 64 GB RAM.

tverbeure 7y ago
My company back then bought one of those around that time. It was used to run large EDA software jobs. On one beautiful day during the weekend, only a two weeks after delivery, our system admin noticed that the machine went offline. Logging in remotely didn’t work at all. No ping either. He drove to work and ... the machine was gone. Thieves had used a crane to lift it out of the building through a window onto a truck. Sun told him that this wasn’t the first time such a thing had happened and that somewhere in the chain from order to delivery, an insider tipped off the thieves about where to find the latest.
- selectodude 7y ago
  I'm curious where the black market for something like that would even come from?
  - cr0sh 7y ago
    Back then, smaller nation-states wanting to do nuclear device simulation and the like would be my guess. Basically, countries that were restricted in some manner on gaining large amounts of parallel processing compute power for such simulations.
    - tverbeure 7y ago
      My money is on breaking it up and sell it for components. The 256 GB in server grade RAM alone cost a fortune.