4 ms·

I do think this is sort of fool's gold in terms of actual performance. Even though the core count and RAM size is impressive, those cores are talking over ether

I do think this is sort of fool's gold in terms of actual performance. Even though the core count and RAM size is impressive, those cores are talking over ethernet rather than system bus. Latency and bandwidth is atrocious in comparison, and you're going to run into problems like no individual memory allocation being able to exceed 8 Gb. Like for running a hundred truly independent jobs then sure, maybe you'll get equivalent performance, but that's a very unique scenario that is rare in the real world.

varispeed 4y ago
I built such a toy cluster once to see for my self and gave up. It is too slow to do anything serious. You can be much better off by just buying older post lease server. Sure it will consume more power, but conversely you will finish more tasks in shorter time, so advantage of using ARM in that case may be negligible. If it was Apple's M1 or M2, that would have been a different story though. RPi4 and clones are not there yet.
- marginalia_nu 4y ago
  I overall think people tend to underestimate the overhead of clustering. It's always significantly faster to run a computation on one machine than spread over N machines with hardware of (1/N) power. That's not always a viable option because of hardware costs, and sometimes you want redundancy, but those concerns are on an orthogonal axis to performance.
  - jbverschoor 4y ago
    Well, a complete M1 board, which is basically about as large as half an iPhone mini, is fast enough. It's also super efficient. So I'm still waiting for Apple to announce their cloud. They're currently putting Mx chips in every device they have, even the monitors. It'll be the base system for any electric device. I'm sure we'll see more specialized devices for different applications, because at this point, the hardware is compact, fast, and secure enough for anything, as well as the software stack. Hello Apple Fridge
    - marginalia_nu 4y ago
      Fast enough for what?
  - convolvatron 4y ago
    Lines gets blurred when you are on a supercomputer interconnect and a global address space or even rdma
    - dekhn 4y ago
      the fastest practical interconnects are roughly 1/10th the speed of local RAM. Because of that, if you use interconnect, you don't use it for remote RAM (through virtual memory). I don't think anybody in the HPC business really pursued mega-SMP after SGI because it was not cost-effective for the gains.
      - convolvatron 4y ago
        Cray did some vector machines that were globally addressed but not coherent. That’s an interesting direction. So is latency hiding. The really important thing is that the big ‘single machine’ you’re talking about already has numa latency problems. Sharing a chassis doesn’t actually save you from needing to tackle latency at scale.
      - p_l 4y ago
        Both Single System Image and giant NUMA machines were and are still pursued because not everything scales in shared-nothing message passing well (some stuff straddles it by doing distributed shared memory over MPI but using it mostly for synchronisation). It's just that there's a range of very well paying problems that scale quite well in message passing systems, and this means that even if your problem scales very badly on them, you might have easier time brute forcing the task on larger but inefficient supercomputer rather than getting funding for smaller more efficient one that fits your problems better.
DrBazza 4y ago
It probably lends itself to tasks where CPU time is much greater than network round trip. Maybe scientific problems that massively parallel. Way back in the 90s I worked with plasma physics guys that used a parallel system on "slow" Sun boxes. I can't remember the name of the software though.
PragmaticPulp 4y ago
>I do think this is sort of fool's gold in terms of actual performance. It’s a fun toy for learning (and clicks, let’s be honest). It’s not a serious attempt at a high performance cluster or an exercise in building an optimal computing platform. Enjoy the experiment and the uniqueness of it. Nobody is going to be choosing this as their serious compute platform.
- analognoise 4y ago
  In TFA, isn't Jetbrains using it as a CI system?
  - bee_rider 4y ago
    Tangential, but it is so funny to me that “TFA” has become a totally polite and normal way to refer to the linked article on this site. Expanding that acronym would really change the tone!
    - OJFord 4y ago
      I'm not sure it is 'totally polite'? I usually read it as having a 'did you even open it' implication that 'OP' or 'the submission' doesn't. Maybe that's just me.
      - bee_rider 4y ago
        Maybe it isn’t totally polite, but it IMO it reads in this case more like slight correction than “In the fucking article,” which would be pretty aggressive, haha.
      - cyberpunk 4y ago
        I always thought it meant “the featured article”; I must be a lot more wholesome than previously indicated!
  - lmz 4y ago
    Unless they need something Pi specific I don't understand why this would be preferable versus just virtualizing instances on a "big ARM" server. I'm sure those exist.

varispeed 4y ago

I built such a toy cluster once to see for my self and gave up. It is too slow to do anything serious. You can be much better off by just buying older post lease server. Sure it will consume more power, but conversely you will finish more tasks in shorter time, so advantage of using ARM in that case may be negligible. If it was Apple's M1 or M2, that would have been a different story though. RPi4 and clones are not there yet.

marginalia_nu 4y ago
I overall think people tend to underestimate the overhead of clustering. It's always significantly faster to run a computation on one machine than spread over N machines with hardware of (1/N) power. That's not always a viable option because of hardware costs, and sometimes you want redundancy, but those concerns are on an orthogonal axis to performance.
- jbverschoor 4y ago
  Well, a complete M1 board, which is basically about as large as half an iPhone mini, is fast enough. It's also super efficient. So I'm still waiting for Apple to announce their cloud. They're currently putting Mx chips in every device they have, even the monitors. It'll be the base system for any electric device. I'm sure we'll see more specialized devices for different applications, because at this point, the hardware is compact, fast, and secure enough for anything, as well as the software stack. Hello Apple Fridge
  - marginalia_nu 4y ago
    Fast enough for what?
- convolvatron 4y ago
  Lines gets blurred when you are on a supercomputer interconnect and a global address space or even rdma
  - dekhn 4y ago
    the fastest practical interconnects are roughly 1/10th the speed of local RAM. Because of that, if you use interconnect, you don't use it for remote RAM (through virtual memory). I don't think anybody in the HPC business really pursued mega-SMP after SGI because it was not cost-effective for the gains.
    - convolvatron 4y ago
      Cray did some vector machines that were globally addressed but not coherent. That’s an interesting direction. So is latency hiding. The really important thing is that the big ‘single machine’ you’re talking about already has numa latency problems. Sharing a chassis doesn’t actually save you from needing to tackle latency at scale.
    - p_l 4y ago
      Both Single System Image and giant NUMA machines were and are still pursued because not everything scales in shared-nothing message passing well (some stuff straddles it by doing distributed shared memory over MPI but using it mostly for synchronisation). It's just that there's a range of very well paying problems that scale quite well in message passing systems, and this means that even if your problem scales very badly on them, you might have easier time brute forcing the task on larger but inefficient supercomputer rather than getting funding for smaller more efficient one that fits your problems better.

DrBazza 4y ago

It probably lends itself to tasks where CPU time is much greater than network round trip. Maybe scientific problems that massively parallel. Way back in the 90s I worked with plasma physics guys that used a parallel system on "slow" Sun boxes. I can't remember the name of the software though.

PragmaticPulp 4y ago

>I do think this is sort of fool's gold in terms of actual performance. It’s a fun toy for learning (and clicks, let’s be honest). It’s not a serious attempt at a high performance cluster or an exercise in building an optimal computing platform. Enjoy the experiment and the uniqueness of it. Nobody is going to be choosing this as their serious compute platform.

analognoise 4y ago
In TFA, isn't Jetbrains using it as a CI system?
- bee_rider 4y ago
  Tangential, but it is so funny to me that “TFA” has become a totally polite and normal way to refer to the linked article on this site. Expanding that acronym would really change the tone!
  - OJFord 4y ago
    I'm not sure it is 'totally polite'? I usually read it as having a 'did you even open it' implication that 'OP' or 'the submission' doesn't. Maybe that's just me.
    - bee_rider 4y ago
      Maybe it isn’t totally polite, but it IMO it reads in this case more like slight correction than “In the fucking article,” which would be pretty aggressive, haha.
    - cyberpunk 4y ago
      I always thought it meant “the featured article”; I must be a lot more wholesome than previously indicated!
- lmz 4y ago
  Unless they need something Pi specific I don't understand why this would be preferable versus just virtualizing instances on a "big ARM" server. I'm sure those exist.