6 ms·

Winning team's approach to EY's SHA-1 contest (Fast GPU cluster, not the cloud)

43 points by olivers 17y ago

olivers 17y ago
Of note: DJB personally helped this team by writing key pieces of the cluster software.
- tsally 17y ago
  ...who has a grant from the NSF in high speed crypto.
  - jcl 17y ago
    I called it: http://news.ycombinator.com/item?id=704458 http://news.ycombinator.com/item?id=704458 :) (And I was amused to see the winning solution is also hosted on win.tue.nl, although it doesn't seem to be the same people.)
    - profquail 17y ago
      Their page says that Bernstein is currently visiting TU/e. It looks like some people from other universities committed their computing resources for the contest though.
Oompa 17y ago
I'm not really surprised. This is exactly why I thought the contest was kinda lame.
- profquail 17y ago
  What weren't you really surprised about? That the team who won had an expert in cryptography on their team? The guys from the CUDA forum (one of whom got TWO scores of 31, vs. this team's 30) wrote their software in a couple of days, and are not experts in cryptography. If anything, I was happy to see that CUDA (and a few ATI Stream entries) made such a big impact in this contest. Short of using a massive CPU cluster, no one really had much of a chance without it (expert or not).
  - eugenejen 17y ago
    Not only expert in Cryptography but also math progidy, Bo-Yin Yang was a mathematics prodigy in Taiwan. The competition is just best for people who already good at problem solving in the domain.
    - brown9-2 17y ago
      Isn't that true of most competitions?
    - Periodic 17y ago
      I'm not really sure what you mean about best at solving problems in this domain. This was a brute force problem, so the only real skill was in making it as fast as possible. I don't think it was any secret techniques, but rather that they happened to be a group sitting on the right tools at the right time and with a little bit of luck. You can't infer after the fact that it was their genius and domain knowledge that gave them the win, for I'm sure there were many other groups with geniuses and domain knowledge that didn't win, as well as some without the domain knowledge that just didn't get as lucky.
kingkongrevenge 17y ago
What are the "cloud" options for cpu intensive tasks? What is the billing? I actually have a couple simulation problems.
- imp 17y ago
  Amazon has "high-cpu" instances: http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/
- markbao 17y ago
  Notable that the GPU seemed to run far faster than the CPU for these calculation instances. I think many of the low Hamming achievers used Nvidia GPUs using CUDA which ran up to 700 million hashes per second (seeing the tesla machine in the article). So you probably want to go for GPU than CPU. There's a service called Hoopoe that does GPU cloud computing that uses CUDA: http://www.gass-ltd.co.il/hoopoe/ http://www.gass-ltd.co.il/hoopoe/
mrduncan 17y ago
What I find pretty fascinating is that according to their numbers the 14 GPU cores were performing about 1.5 times as many hashes per second as the 115 CPU cores were.
- ars 17y ago
  Graph: http://www.flickr.com/photos/hoyhoy/3743710781/ http://www.flickr.com/photos/hoyhoy/3743710781/
- pmorici 17y ago
  Those 14 GPUs have a lot more than 14 cores though. A GTX 295 graphics card for example has 240 cores per chip x 2 chips. http://en.wikipedia.org/wiki/GeForce_300_Series#Technical_summary http://en.wikipedia.org/wiki/GeForce_300_Series#Technical_su...
jrockway 17y ago
So what exactly did engine yard get out of this? The contest showed that neither Ruby nor "The Cloud" were useful in solving this (useless, admittedly) problem. As a company that sells Ruby cloud services and consulting, that doesn't exactly sound like great marketing.
basugasubaku 17y ago
Interesting that CodingCrypto and hashbreaker (who tied for 2nd place but lost the coin flip) were the same team. This wasn't mentioned on the Engine Yard blog. Also, this story is cute: Paul noticed the contest and brought Dan and Tanja in - just one condition: he would get the iPhone. Sure - sufficiently unlikely to work anyway. To make it more likely and to have more fun with GPUs, Dan and Tanja brought in Bo-Yin and Doug - who might now figure out how to program in Ruby and were very busy getting all machines to work and running the code which included reconfiguring the machines Not sure why they mention Ruby because it doesn't seem to be actually used.
- nkurz 17y ago
  Ruby was mentioned because once one strips the iPhone from the prize, the part remaining is $2000 worth of time on a cluster of machines that specializes in supporting Ruby.

43 points by olivers 17y ago

olivers 17y ago

Of note: DJB personally helped this team by writing key pieces of the cluster software.

tsally 17y ago
...who has a grant from the NSF in high speed crypto.
- jcl 17y ago
  I called it: http://news.ycombinator.com/item?id=704458 http://news.ycombinator.com/item?id=704458 :) (And I was amused to see the winning solution is also hosted on win.tue.nl, although it doesn't seem to be the same people.)
  - profquail 17y ago
    Their page says that Bernstein is currently visiting TU/e. It looks like some people from other universities committed their computing resources for the contest though.

Oompa 17y ago

I'm not really surprised. This is exactly why I thought the contest was kinda lame.

profquail 17y ago
What weren't you really surprised about? That the team who won had an expert in cryptography on their team? The guys from the CUDA forum (one of whom got TWO scores of 31, vs. this team's 30) wrote their software in a couple of days, and are not experts in cryptography. If anything, I was happy to see that CUDA (and a few ATI Stream entries) made such a big impact in this contest. Short of using a massive CPU cluster, no one really had much of a chance without it (expert or not).
- eugenejen 17y ago
  Not only expert in Cryptography but also math progidy, Bo-Yin Yang was a mathematics prodigy in Taiwan. The competition is just best for people who already good at problem solving in the domain.
  - brown9-2 17y ago
    Isn't that true of most competitions?
  - Periodic 17y ago
    I'm not really sure what you mean about best at solving problems in this domain. This was a brute force problem, so the only real skill was in making it as fast as possible. I don't think it was any secret techniques, but rather that they happened to be a group sitting on the right tools at the right time and with a little bit of luck. You can't infer after the fact that it was their genius and domain knowledge that gave them the win, for I'm sure there were many other groups with geniuses and domain knowledge that didn't win, as well as some without the domain knowledge that just didn't get as lucky.

kingkongrevenge 17y ago

What are the "cloud" options for cpu intensive tasks? What is the billing? I actually have a couple simulation problems.

imp 17y ago
Amazon has "high-cpu" instances: http://aws.amazon.com/ec2/instance-types/ http://aws.amazon.com/ec2/instance-types/
markbao 17y ago
Notable that the GPU seemed to run far faster than the CPU for these calculation instances. I think many of the low Hamming achievers used Nvidia GPUs using CUDA which ran up to 700 million hashes per second (seeing the tesla machine in the article). So you probably want to go for GPU than CPU. There's a service called Hoopoe that does GPU cloud computing that uses CUDA: http://www.gass-ltd.co.il/hoopoe/ http://www.gass-ltd.co.il/hoopoe/

mrduncan 17y ago

What I find pretty fascinating is that according to their numbers the 14 GPU cores were performing about 1.5 times as many hashes per second as the 115 CPU cores were.

ars 17y ago
Graph: http://www.flickr.com/photos/hoyhoy/3743710781/ http://www.flickr.com/photos/hoyhoy/3743710781/
pmorici 17y ago
Those 14 GPUs have a lot more than 14 cores though. A GTX 295 graphics card for example has 240 cores per chip x 2 chips. http://en.wikipedia.org/wiki/GeForce_300_Series#Technical_summary http://en.wikipedia.org/wiki/GeForce_300_Series#Technical_su...

jrockway 17y ago

So what exactly did engine yard get out of this? The contest showed that neither Ruby nor "The Cloud" were useful in solving this (useless, admittedly) problem. As a company that sells Ruby cloud services and consulting, that doesn't exactly sound like great marketing.

basugasubaku 17y ago

Interesting that CodingCrypto and hashbreaker (who tied for 2nd place but lost the coin flip) were the same team. This wasn't mentioned on the Engine Yard blog. Also, this story is cute: Paul noticed the contest and brought Dan and Tanja in - just one condition: he would get the iPhone. Sure - sufficiently unlikely to work anyway. To make it more likely and to have more fun with GPUs, Dan and Tanja brought in Bo-Yin and Doug - who might now figure out how to program in Ruby and were very busy getting all machines to work and running the code which included reconfiguring the machines Not sure why they mention Ruby because it doesn't seem to be actually used.

nkurz 17y ago
Ruby was mentioned because once one strips the iPhone from the prize, the part remaining is $2000 worth of time on a cluster of machines that specializes in supporting Ruby.