10 ms·
Can My Water Cooled Raspberry Pi Cluster Beat My MacBook?
- chetangoti 6y agoSite got HOD. Here is https://archive.is/jhbQV https://archive.is/jhbQV
- neatze 6y agoFrom quick overview seems like speed up is achieved because there is comparison of 4 local processes, versus 16 processes, since it is only single gather operation does not seem like latency would make much of difference. I wonder what performance would look like on 5 years old 16 core CPU from ebay for like $20, compared to py cluster.
- SethTro 6y agoI have a 2 socket E5-2650 v2 for 16 cores which is as fast as a modern Ryzen above (but 50% slower on real world tasks) 0.8 seconds for 10,000 28.16 seconds for 100,000
- neatze 6y agoThis would be interesting comparison against pi cluster on more accurate tasks, I have suspicion dual e5 will out perform pi cluster.
- nielsbot 6y agoI assumed it was the M1. (It's an Intel MacBook.)
- tumblewit 6y agosame. the m1 would be much faster. also the water cooling is going to add cost which can be reused to add more Pi’s instead and I know i can run mine at 1800Mhz with a tiny heatsink via thermal paste and a 40mm fan with it not going above 65C so the cluster with a large fan would be very easy to cool
- m463 6y agoI tried FindingPrimesMulti.py on a 3970x: Find all primes up to: 10000 using 256 processes. Time elasped: 0.51 seconds Number of primes found 1229 Find all primes up to: 100000 using 256 processes. Time elasped: 36.71 seconds Number of primes found 9592 Find all primes up to: 200000 using 256 processes. Time elasped: 149.55 seconds Number of primes found 17984 EDIT: ran it again: Find all primes up to: 200000 using 256 processes. Time elasped: 145.76 seconds Number of primes found 17984
- neatze 6y agothis is with a fixed multiprocessing script ?
- m463 6y agois this what you mean? FindingPrimesMulti-1.zip
- neatze 6y agoIf you have mpi4py then I would try this one: https://github.com/joshjerred/mpi4py-with-multiprocessing-Check-for-primes/blob/main/primeMP.py https://github.com/joshjerred/mpi4py-with-multiprocessing-Ch... To me this would be more fair comparison between single cpu and pi cluster.
- m463 6y agoapt-get mpi4py3 and using python3 primeMP.py is lots faster Find all primes up to: 10000 Nodes: 1 Time elasped: 0.12 seconds [1229] Primes discovered: 1229 Find all primes up to: 100000 Nodes: 1 Time elasped: 4.52 seconds [9592] Primes discovered: 9592 Find all primes up to: 200000 Nodes: 1 Time elasped: 17.43 seconds [17984] Primes discovered: 17984 I even tried: Find all primes up to: 1000000 Nodes: 1 Time elasped: 383.2 seconds [78498] Primes discovered: 78498
- 6y ago
- toyota86 6y agoThe test script didn't share the load across processors so the author is rerunning the tests. It seems the graphs therefore are misleading (for now). Nevertheless this is not as interesting as testing the M1 chip on the latest MacBook offering. I feel a bit misled but perhaps it was just my fondness for the M1 causing this bias.
- albanread 6y agoI can confirm for you; that there is no rational reason to assume the authors own MacBook is a new M1 system.
- hutrdvnj 6y agoPlease redo the benchmark against the M1.
- wil421 6y agoSee my comment above.
- hutrdvnj 6y agoDon't know why I'm getting down voted, but thanks, that's exactly what I was looking for.
- waterhouse 6y agoI suspect because the author of the blogpost probably doesn't own an M1 Mac (he mentions benchmarking several machines and it seems unlikely he would have forgotten the M1 if he had one), so your comment (interpreted as a request to the author) amounts to "Please spend about $1000 on another computer because I want another benchmark result", not to mention shipping time would make it days before he could report back. In other words, it seems to be an unreasonable request. If you meant to ask the other commenters here, then something like "Does anyone have an M1 Mac? If so, would you run his benchmarks and post the results here?" would have been clearer.
- hutrdvnj 6y agoAh okay, I didn't want the author to spend $1000, I just wanted someone with a M1 like wil421 to redo the benchmark.
- artiscode 6y agoI would really like to see how a raspi cluster fares with "real world" loads, like running an app or a distributed system, instead of calculating pi. I'm genuinely curious whether there would be any gains by running Docker containers, each on a separate Raspberry Pi. I personally am a web developer, therefore my technology stack is almost always the same - an app server, a background worker, a queue and a database. Often the app server and the background worker is the same process, therefore a cluster of 3-4 Pi's would be sufficent for such workloads. Theoretically the combined horsepower of all these Pi's should stack up and deliver better performance than writing code on my M1. Or perhaps I'm trying to solve a problem that doesn't exist.
- b3lvedere 6y agoI think that's a great idea! Personally i'd like a nice failover test using a Pi Cluster. So if Pi's when failing and getting replaced on the fly if processes would still be up and running flawlessly. Probably a pretty useless and expensive setup, but i think it'd be awesome to watch.
- thraway123412 6y ago> Theoretically the combined horsepower of all these [4] Pi's should stack up and deliver better performance than writing code on my M1. If that is the case, then M1 should be the slowest CPU Apple has used in the past 10 years.
- jvanderbot 6y agoM1 is good at ops/watt and also ops/second. M1 probably will still beat the Pi cluster by having a tighter Pareto curve.
- ThrowawayR2 6y agoThere's the Geekbench 5 measurements: Apple M1: https://browser.geekbench.com/v5/cpu/search?utf8=%E2%9C%93&q=Apple+M1 https://browser.geekbench.com/v5/cpu/search?utf8=%E2%9C%93&q... Raspberry Pi 4: https://browser.geekbench.com/v5/cpu/search?utf8=%E2%9C%93&q=Raspberry+pi+4 https://browser.geekbench.com/v5/cpu/search?utf8=%E2%9C%93&q...
- danbruc 6y agoThere are only 1229 primes up to 10,000, not 1230 as the article says. Not sure whether this is a bug in the code or a typo in the article. I still remember this because 25 years ago as a teenager I spend quite some time making primality testing as fast as I could. My implementation was certainly not as naive as the one from the article - only testing up to the square root of n, only testing against primes I had found before - but not sophisticated in any way, for that I lacked the mathematical knowledge. I can not exactly remember how fast I got it but I am pretty sure it was sub-one-second, like 0.2 or 0.3 seconds maybe for the range up to 10,000. On a 50 MHz i486.
- willis936 6y ago1 is considered prime by some.
- WJW 6y agoThere are also people who consider the world to be flat though. There is a very thorough mathematical definition of prime numbers and 1 is not prime according to that definition.
- willis936 6y agoThe definition a prime number is having two factors. As the multiplicative identity, one only has one factor. Some might see primes as being numbers with >=2 factors. This interpretation is less useful. What is completely useless is a definition that makes exclusions of specific numbers as dogma. What does that say about anything? It’s totally arbitrary and unexplanatory.
- spekcular 6y agoNo, that is not the correct definition of a prime. You can find the right one on Wikipedia. The reason that 1 is not a prime is to preserve unique factorization. A basic fact from number theory is that every integer uniquely factors as a product of primes, say 21 = 7 times 3. If 1 were a prime, we'd also have 21 = 7 times 3 times 1. That's bad. In more general number rings, other units (e.g. i) are also not considered primes for the same reason. The exclusion is not arbitrary.
- shaggie76 6y agoI'd be curious the relative power draw -- is the performance/watt actually better?
- thraway123412 6y ago2GB Pi is $55 on amazon. 4GB version is $62. That'd be $450+ for the eight pies. Ryzen 2700 launched at $300 (and tapered down to ~$200) and would pretty much run circles around such a Pi cluster. Just saying. These Pi clusters can be a cool and fun thing to build but if you're looking for compute power, you'd be better served by a mid-range desktop.
- willis936 6y agoIn terms of computer per dollar yes. In terms of compute per joule, no.
- imtringued 6y agoYou can just undervolt the Ryzen and it will still run cycles around the Raspberry Pi while consuming less power if you are into that sort of thing.
- willis936 6y agoThat is not true. The energy efficiency will go down if you downclock because the static to dynamic power ratio will increase.
- rcxdude 6y agostatic to dynamic power ratio isn't efficiency though (and static power in modern desktop chips is tiny compared to the dynamic power). The dynamic power is not linearly related to processing speed (in fact it's much worse). If you downclock and undervolt a ryzen processor it will use much less power than the decrease in speed (e.g. from stock, if you drop performance by ~20% you might get a ~40% power decrease). Obviously at a certain point you will start to get worse again but most chips are not at peak performance/watt at their stock settings because raw performance and performance/cost also matters.
- willis936 6y ago
- smallpipe 6y agoThis is fun and all, but the benchmarks here aren't really what the author seems to think they are. This isn't "computationally expensive", the script is basically only control flow, where the CPU spend more time doing variable lookups than the actual computation. This means most of the pipeline width sits completely unused, whcih is a pretty large disservice to the M1 and the i5. There's also no control for the thermal throttling of the M1, which is probably why the 100,000 example is performing worse.
- kortex 6y agoUsing Ray distributed would be a better stress test. Computing primes this way probably isn't the best way to saturate cores. You are spending a lot of time doing python vm operations vs pure number crunching. Using numeric arrays chunked into blocks of number ranges would be more efficient (and therefore "crunchier") https://ray.io/ https://ray.io/
- cashewchoo 6y agoIf they're thinking pure CPU benchmarking, prime95/mprime has several options, such as: [this is from my recollection] * completely in-register only, no caches. they warn an organic workload will never ever do this and not to do it if you're not confident in your cooling * somewhat more normal math-heavy workloads that don't do any IO but also do access the caches like a normal person * etc The python benchmark might be fair, though? I wouldn't be surprised if the ARM chip on the pi is like, fast, but then when it came to doing something more holistic - like a bunch of python vm operations - something about the mac is more robust than the pi in a significant way.
- wil421 6y agoM1 Air Results: 10,000: 0.52 seconds. 100,000: 41.48 seconds. 200,000: 157.68 seconds.
- 908B64B197 6y agoHis workload is embarrassingly parallel[0] and the message-passing between the Pi is basically free since the results are so small to send-back (and there's nothing to send to each node on startup). He's effectively doing a trivial map-reduce [1]. He could probably get even better than the Pi cluster by using a (single) GPU. [0] https://en.wikipedia.org/wiki/Embarrassingly_parallel https://en.wikipedia.org/wiki/Embarrassingly_parallel [1] https://en.wikipedia.org/wiki/MapReduce https://en.wikipedia.org/wiki/MapReduce