3 ms·

A technical description and how they got it running together with more pictures: http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomputer_southampton.htm

by fpp 14y ago

A technical description and how they got it running together with more pictures: http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomputer_southampton.htm http://www.southampton.ac.uk/~sjc/raspberrypi/pi_supercomput... As you can see from the pictures each of the devices uses its own power supply, so there should be quite some possibility to improve overall power consumption. Anybody with more experience of using MPI ?

bravura 14y ago
I programmed MPI about 15 years ago, during a summer at the MIT AI Lab. I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence"). Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.
- apawloski 14y ago
  What do you use now instead of MPI?
  - pm90 14y ago
    He talked about higher latency so I'm guessing just sockets
    - bravura 14y ago
      Incorrect. With the sort of work I do nowadays (large-scale ML and NLP), I generally need very little synchronization, i.e. my tasks are usually embarrassingly parallel. I typically save final results in a centralized store (DB or NFS) and look at it there. I also use Hadoop where appropriate.
apawloski 14y ago
It is almost always the case that programs written for clusters -- including those which use MPI -- are communication bound. Right now the limiting factor is not these little processors, but rather the interconnect speed. If I remember correctly, Raspberry Pi has 10/100 megabit ethernet. [Edit: just checked, this is the case] So while this looks like a lot of fun, it's not very useful for anything meaningful yet. Of course it's not fair to compare this to an infiniband cluster (that's not the point of this exercise), but I'd really be interested to see a cluster built on $0.50 ARM chips with at least a gigabit ethernet interconnect. A couple of years from now -- given the low entry cost and lower infrastructure costs (cooling/power consumption/etc) -- that could be a game changer.
- jacques_chester 14y ago
  Before the GFC killed them, SiCortex had a design where MIPS cores were grouped together on a single die with MPI-specific fabric logic.
  - adestefan 14y ago
    This is the reason why Cray is still relevant. Their proprietary interconnects are what you're paying for and not the CPUs.
    - wickberg 14y ago
      Oddly enough, Cray recently sold their interconnect tech to Intel [0]. Intel seems to be planning to integrate it on-chip down the road [1], which seems to leave Cray serving as a somewhat quirky system integrator longer-term. [0] http://newsroom.intel.com/community/intel_newsroom/blog/2012/04/24/intel-acquires-industry-leading-high-performance-computing-interconnect-technology-and-expertise http://newsroom.intel.com/community/intel_newsroom/blog/2012... [1] http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strategy_to_put_interconnect_fabrics_on_chip.html http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strat...
- adestefan 14y ago
  It's even worse because the Ethernet is connected via USB. What I would love to see are 16 or 32 ARM cores on a single card connected via high speed bus such as Infiniband and pack 4 or 8 of these cards into a chassis.
- wickberg 14y ago
  There are a few different companies that have ARM + custom interconnect systems out there or in development. They're not necessarily cost-competitive yet, but they're an interesting start. Dell's "Project Copper" - http://content.dell.com/us/en/enterprise/d/campaigns/project-copper http://content.dell.com/us/en/enterprise/d/campaigns/project... Boston Viridis - http://www.boston.co.uk/solutions/viridis/default.aspx http://www.boston.co.uk/solutions/viridis/default.aspx
ChuckMcM 14y ago
I found it amusing because at Blekko I was playing with one and talked about building a cluster with them. I think it would be tremendously valuable as a teaching tool to build smallish (24 - 96 machine) clusters and teach folks to write distributed algorithms. Its a stretch to call it a 'super computer' but it is quite educational. One of my favorite systems questions is to have someone walk through the design and implementation of a system where all the machines in the system respond to a query Q based on the contents of a linked list L. The system has an API which consists of L <- M(op) (mutate list), R <- Q(id) respond to a query based on the contents of the list, and R <- S() report on the stability of the list. Start with M(op) being idempotent, then non-idempotent, Etc. Folks who've had a good introduction to state machines will immediately recognize and a number of problems that arise as you control correctness. If folks get through the whole sequence we're taking about a function f(C) which takes a correctness coefficent from 1.0 (fully correct) to 0.0 (unspecified) and look at the performance of the system across that range. That kind of stuff you could easily do on a 48 node Pi Cluster.
Xcelerate 14y ago
MPI is essential for molecular dynamics simulations. You split the "box" of atoms/molecules up into different domains -- one on each processor. Occasionally you'll have particles wander into the next box. The information of these ghost particles must be passed around and MPI facilitates this.

by fpp 14y ago

bravura 14y ago

I programmed MPI about 15 years ago, during a summer at the MIT AI Lab. I implemented neural network feature creation for a backgammon agent ("Automated feature selection to maximize learning in artificial intelligence"). Nowadays, I mainly do parallel machine learning on machines with higher network latency. I haven't used MPI since.

apawloski 14y ago
What do you use now instead of MPI?
- pm90 14y ago
  He talked about higher latency so I'm guessing just sockets
  - bravura 14y ago
    Incorrect. With the sort of work I do nowadays (large-scale ML and NLP), I generally need very little synchronization, i.e. my tasks are usually embarrassingly parallel. I typically save final results in a centralized store (DB or NFS) and look at it there. I also use Hadoop where appropriate.

apawloski 14y ago

It is almost always the case that programs written for clusters -- including those which use MPI -- are communication bound. Right now the limiting factor is not these little processors, but rather the interconnect speed. If I remember correctly, Raspberry Pi has 10/100 megabit ethernet. [Edit: just checked, this is the case] So while this looks like a lot of fun, it's not very useful for anything meaningful yet. Of course it's not fair to compare this to an infiniband cluster (that's not the point of this exercise), but I'd really be interested to see a cluster built on $0.50 ARM chips with at least a gigabit ethernet interconnect. A couple of years from now -- given the low entry cost and lower infrastructure costs (cooling/power consumption/etc) -- that could be a game changer.

jacques_chester 14y ago
Before the GFC killed them, SiCortex had a design where MIPS cores were grouped together on a single die with MPI-specific fabric logic.
- adestefan 14y ago
  This is the reason why Cray is still relevant. Their proprietary interconnects are what you're paying for and not the CPUs.
  - wickberg 14y ago
    Oddly enough, Cray recently sold their interconnect tech to Intel [0]. Intel seems to be planning to integrate it on-chip down the road [1], which seems to leave Cray serving as a somewhat quirky system integrator longer-term. [0] http://newsroom.intel.com/community/intel_newsroom/blog/2012/04/24/intel-acquires-industry-leading-high-performance-computing-interconnect-technology-and-expertise http://newsroom.intel.com/community/intel_newsroom/blog/2012... [1] http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strategy_to_put_interconnect_fabrics_on_chip.html http://www.hpcwire.com/hpcwire/2012-09-10/intel_weaves_strat...
adestefan 14y ago
It's even worse because the Ethernet is connected via USB. What I would love to see are 16 or 32 ARM cores on a single card connected via high speed bus such as Infiniband and pack 4 or 8 of these cards into a chassis.
wickberg 14y ago
There are a few different companies that have ARM + custom interconnect systems out there or in development. They're not necessarily cost-competitive yet, but they're an interesting start. Dell's "Project Copper" - http://content.dell.com/us/en/enterprise/d/campaigns/project-copper http://content.dell.com/us/en/enterprise/d/campaigns/project... Boston Viridis - http://www.boston.co.uk/solutions/viridis/default.aspx http://www.boston.co.uk/solutions/viridis/default.aspx

ChuckMcM 14y ago

I found it amusing because at Blekko I was playing with one and talked about building a cluster with them. I think it would be tremendously valuable as a teaching tool to build smallish (24 - 96 machine) clusters and teach folks to write distributed algorithms. Its a stretch to call it a 'super computer' but it is quite educational. One of my favorite systems questions is to have someone walk through the design and implementation of a system where all the machines in the system respond to a query Q based on the contents of a linked list L. The system has an API which consists of L <- M(op) (mutate list), R <- Q(id) respond to a query based on the contents of the list, and R <- S() report on the stability of the list. Start with M(op) being idempotent, then non-idempotent, Etc. Folks who've had a good introduction to state machines will immediately recognize and a number of problems that arise as you control correctness. If folks get through the whole sequence we're taking about a function f(C) which takes a correctness coefficent from 1.0 (fully correct) to 0.0 (unspecified) and look at the performance of the system across that range. That kind of stuff you could easily do on a 48 node Pi Cluster.

Xcelerate 14y ago

MPI is essential for molecular dynamics simulations. You split the "box" of atoms/molecules up into different domains -- one on each processor. Occasionally you'll have particles wander into the next box. The information of these ghost particles must be passed around and MPI facilitates this.