19 ms·
Oracle built a 1,060 node Raspberry Pi cluster
- axegon_ 7y agoIt's incredible how powerful these small things are. I have a couple and I use them for just about anything, from access servers to access my home network, to running various applications inside containers on them. I did plan on making a cluster some time ago but for various reasons I've put that plan at the back of the line. The real question here is did they use it to run some heavy applications on the cluster and how did it perform? Basically do they have a purpose for it or was it simply a "because we can" type of thing?
- shervinafshar 7y agoI think an objective might have been demonstrating the benefits of Oracle Autonomous Linux regarding automatic patching and tuning; lower TCO being the advertised selling point.
- nrki 7y agoNo benchmarks. Cool but just seems like an ad for Oracle.
- rhinoceraptor 7y agoOf course, you're not allowed to benchmark Oracle publicly, it's in the EULA :)
- ineedasername 7y agoSeriously? As someone with purchasing power, I could only assume they were slower than alternatives in that case. Anything they quoted me would have to be considered marketing fluff without independent numbers.
- rhinoceraptor 7y agoI can almost understand why they do it. Benchmarking is incredibly hard to do correctly, and it requires a very deep understanding of software and hardware. For example, I remember hearing about Joyent's port of KVM to Illumos. They were testing the Illumos KVM against Linux KVM, and the Illumos KVM was faster. After a lot of debugging, they realized the Illumos machine was turboboosting better than the Linux machine, since it was in a colder part of the data center.
- _bxg1 7y agoI wonder what the cost-effectiveness is in terms of compute power. I'd assume not great, but who knows? Raspberry PIs pack a lot of punch for their size and price.
- chapium 7y agoDefinitely not great but thats one of the tradeoffs.
- jandrese 7y agoIf they can leverage the GPU on the Pis it could be quite a bit of compute power spread across that array. Even if they're purely CPU bound that is 4240 cores which is not nothing. Power wise each Pi going flat out burns around 5W, so all told you're burning ~6kW, which is quite modest for a big computing setup.
- willis936 7y agoIt looks like you can get 400 single precision MFLOPS for those 5W. So 80 MFLOPS/W +/- 20%. It isn’t the worst and it isn’t the best, but it is respectable. It would be a big hassle to make and maintain a cluster of thousands of these. https://www.arnabkumardas.com/platforms/nvidia/nvidia-jetson-nano-review-and-benchmark/ https://www.arnabkumardas.com/platforms/nvidia/nvidia-jetson... https://raspi.tv/2019/how-much-power-does-the-pi4b-use-power-measurements https://raspi.tv/2019/how-much-power-does-the-pi4b-use-power... https://en.wikipedia.org/wiki/Performance_per_watt https://en.wikipedia.org/wiki/Performance_per_watt
- ramses0 7y ago...but you could experiment with "disposable computing" (ie: similar to original google datacenter racks: never replace a node, only replace a rack at a time if 51% is failing).
- jacquesm 7y agoThat and interesting configurations (systolic arrays, message passing architectures, Erlang on a large number of nodes and so on). Lots of fun with stuff like this.
- tra3 7y agoI've been curious about RPis for a while. I have a file server running AMD Phenom II 955 that I'm thinking of replacing with a RPi. Is there a practical way to have RPis manage a software RAID? Performance is not a consideration. Is it even worth looking at RPi in this scenario? The advantages I can think of is small footprint, lower electricity cost.
- whalesalad 7y ago> Is there a practical way to have RPis manage a software RAID? I can't think of any good reason to do use software RAID, let alone doing so on a Raspberry Pi. I would recommend hardware RAID, ZFS, or a Synology appliance. If performance is truly not a concern, yes it is possible to do this with a Pi but I don't think you are going to be super impressed.
- ComputerGuru 7y agoZFS is 100% software raid.
- whalesalad 7y agoSorta. At the end of the day, yes, the disks are being managed by your CPU and not a dedicated raid controller... but it's a completely different ballgame than something like mdadm.
- ownagefool 7y agoNot "sorta", it's software raid and thus, like software raid, it's portable to any system that'll run the software and JBOD the drives. Most would argue RAIDZ is better, but many of these reasons are functionality based over and about just redundancy, i.e. Copy on Write, et all.
- whalesalad 7y agoI say sorta because in most circles when one is speaking of 'software RAID' they are referring to a very specific configuration and are explicitly not referring to ZFS.
- Yuioup 7y agoThat's what Azure feels like sometimes, running like a cluster of Pis.
- partisan 7y agoI really do wonder why Azure is so slow. You have to optimize somewhat aggressively to squeeze average performance out of Azure SQL, for instance.
- vorticalbox 7y agoMaybe because Windows updates and xbox love run through the network?
- weberc2 7y agoPretty sure all of Netflix runs on AWS and I've got to think that's several orders of magnitude more traffic than Windows updates or xbl.
- jlgaddis 7y agoFWIW, the video streaming part of Netflix does not run in AWS (but, AIUI, pretty much everything else does).
- weberc2 7y agoOh, TIL I guess. Thanks for the correction.
- ofrzeta 7y agoNetflix hosts video content at ISPs' sites: https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/
- slenk 7y agoEdge content yes, but everything else is in AWS
- tobyhinloopen 7y agoWhy...
- rvz 7y agoExactly, I just feel that Oracle just needed to do a huge flex to the world to show that they are relevant. Weird flex but okay, I'll just carry on with my ARM64 Cavium kubernetes cluster that blows this toy out of the ocean.
- walshemj 7y agoBut they probably built it as test cluster to do initial work on before going for a big shot on a big boy cluster.
- faizshah 7y agoI’ve seen a few of these RPi clusters (obviously none this big), I think the main reason people make them is to demo distributed systems or build a homelab cluster(sometimes they will argue its cause of the power efficiency). Also they are usually a nerd magnet so it makes for a great conversation piece as this probably was for their convention.
- yellowapple 7y agoWhy not?
- et2o 7y agoBurning a few interns worth of money?
- tr33house 7y agoThis seems like Oracle's way of marketing to devs that they're a cool place to work... And it might have actually just worked
- djaque 7y agoI think this is cool, but engineering for engineering's sake aside I've seriously looked at building a raspi cluster a couple of times and have always concluded that it isn't worth it. The real selling point for me is energy efficiency and price, but the reason I could never make it happen is because most of the applications I want to use are x86 only and I don't have the resources to be a trailblazer in that area. In addition sourcing becomes an issue with raspberry pis when you aren't a big company like Oracle. It's hard to buy more than a handful as an individual.
- jacquesm 7y agoThe real selling point for me would be physical isolation. If you run all of them single-core you wouldn't have to worry at all about snooping using the various side channel attacks.
- Skunkleton 7y agoYes, but how many RPis do you need to match the workload of a single x86 core? How much does it cost to cluster and maintain these systems? IMO you would probably have a better story all around if you based your cluster on low end AMD chips, or perhaps even old refurbished systems. That's not to say this isn't cool engineering for engineering's sake. Just that if you are considering practical applications for this, maybe think twice.
- jacquesm 7y agoWhen you need that guarantee of isolation you are picking your solution for different reasons than when you need a particular workload. So there is no point comparing the workload if that wasn't the deciding factor in the first place.
- Skunkleton 7y agoMy point is that you can probably get physical isolation for less money on x86.
- NohatCoder 7y agoNow load any piece of Oracle software onto it and get charged for running 1060 CPUs. Whoops.
- rowanG077 7y agoOracle doesn't charge per CPU it charges per core. Enjoy paying for 4240 cores!
- malux85 7y agoI don’t know why you’re getting downvoted, you’re correct!
- chrisseaton 7y agoI believe there's a scaling factor, so lower powered cores like this are charged as a fraction of a Xeon you'd normally use.
- tyingq 7y agoARM isn't listed on their table, so it slots into "all other multicore CPUs". Scaling factor is 1. So no help there.
- MR4D 7y ago“The only thing that scales better than an Oracle database is their licensing cost! “ - heard years ago from a friend. Not sure of the origination of the quote.
- zepearl 7y agoAha - I stubled upon this https://jira.mariadb.org/browse/MDEV-10872 https://jira.mariadb.org/browse/MDEV-10872 yesterday and thought "why would they want to achieve compatibility with Oracle DBs?" => now (I think that) I do understand that it would be to offer an alternative that would provide the same capabilities (but maybe with a different performance/reliability/whatever) for a different price, for any software that Oracle sells which wants to run on an Oracle DB, right? Is the process "Google vs. Oracle about Java" (I understood that it's about the functions/methods declarations and not about the code itself) similar to a theoretical similar situation where DB XXXX would provide the same functions as the ones provided by the Oracle DB?
- tzs 7y ago> At Oracle OpenWorld 2019, tucked away in the Oracle Code One area [...] As someone not familiar with Oracle events, I'm rather confused about Oracle Code One. I found a FAQ [1] which says: > Oracle Code One is the ultimate developer event where we will cover the technologies and programming languages developers have come to love over the years. Expect talks on Go, Rust, Python, JavaScript, and R and SQL along with more of the latest Java technical content. but looking at the photos, I see in the background quite a few arcade video games. Is this some sort of break area for people to hang out in between talks, or something like that? [1] https://www.oracle.com/code-one/faq.html https://www.oracle.com/code-one/faq.html
- deleted 7y ago[deleted]
- SanderMak 7y agoOracle Code One is the successor of JavaOne. Since Oracle acquired Sun, the JavaOne conference was co-hosted with the much larger Oracle OpenWorld conference. Recently, JavaOne was rebranded into Oracle Code One, morphing it into a cross-technology conference rather than a Java-focused one.
- jaaron 7y agoI remember when JavaOne was definitely worth going to. I've not heard anything like that for years and just assume it's dead as "Code One." Anyone want to confirm?
- jlgaddis 7y agoTFA says this cluster was part of "developor relations" and runs OAL and Java. Perhaps the arcade games are also running on Java and Oracle is simply trying to "show off" a variety of uses of Java or something.
- jonplackett 7y agoNot much info on what they can actually do with it
- jimmcslim 7y agoMore info on the USB power supplies they were using would be interesting, seems likely it would have been rack mounted and probably custom?
- rasz 7y agocouple of those will do the job https://www.digikey.com/product-detail/en/tdk-lambda-americas-inc/HWS1000-5-HD/HWS1000-5-HD-ND/3984323 https://www.digikey.com/product-detail/en/tdk-lambda-america...
- Havoc 7y ago>an array of USB power supplies. Surprised they didn't just bridge 5V to GPIO pins.
- SEJeff 7y agoThis explains so much about Ellison's vision for Oracle Cloud.
- salvagedcircuit 7y agoTalk about a thermal design nightmare. Definitely just an "look-at-me" exhibit