6 ms·
BitScope: 3000-core Raspberry Pi cluster computer
- jacquesm 9y agoNow, to port lisp and C to it. The interconnect will be the most interesting part of this cluster.
- pjmlp 9y agoYou forgot the star reference. :)
- ataturk 9y agoI'm not so sure about the claim "I’ve long thought it the ideal candidate to build low-cost cloud and cluster computing solutions for research and education." Well, you thoguht wrong--a better solution would be any typical rack mount computer running a bunch of VMs or just buy cloud services from AWS, Google, or Cloud Foundry and call it a day. I am more of the mind that these articles are to sell more product and that big RPi clusters aren't remotely powerful enough to do anything really real because they lack RAM and processing horsepower but have all the same issues as every other big cluster of computers you have to maintain. It's clear that cloud services have not gone in the "tinier is better" direction for a multitude of reasons and that virtualization (e.g. a lot of compute nodes running as VMs or containers like CoreOS on better underlying hardware) is a far better solution overall. RPi has a coolness factor, but every time I think I want to buy one I realize my needs are better served some other way. I'm sure one day I'll find a killer reason to buy one.
- bradfa 9y agoIt's a neat way to teach people how to program distributed systems at this scale but I don't understand why they wouldn't use the Raspberry Pi Compute Moudle instead of normal Raspberry Pis. Using the Compute Modules would, I'd think, make for a much easier to integrate system, even possibly allowing higher density of processors per unit volume.
- danielvf 9y agoRaspberry Pi Compute Modules don't have ethernet built in - that would require quite a non-standard networking system, wouldn't it?
- srcmap 9y agoAny comments on using Pi Zero as base line module? Supposedly only $5 per PCB from https://www.adafruit.com/product/2885 Power and network can go thru usb. (usbnet)
- stan_rogers 9y agoYou'd need to buy them one at a time.
- squeaky-clean 9y agoLike stan_rogers says, you're limited to buying them one at a time from pretty much any retailer. And it's been this way for years, so it doesn't seem like it will change anytime soon. You can find more on eBay, but then they're usually priced $15-20 each and it would still be difficult to buy them in the hundreds.
- bradfa 9y agoThe Broadcom SoC doesn't have Ethernet built in, it's a USB connected Ethernet on the normal Pis. The USB bus from the SoC is brought out onto the SODIMM connector on the Compute Module so one could easily add Ethernet on the carrier board if needed. The Ethernet used on the normal Raspberry Pi 3 is a Microchip/SMSC LAN9514 USB hub/Ethernet part: http://www.microchip.com/wwwproducts/en/LAN9514 http://www.microchip.com/wwwproducts/en/LAN9514 There are likely other methods of networking, if one wanted to avoid paying for a USB/Ethernet adapter chip for each Compute Module. Such as running a USB gadget on each Compute Module within Linux and have all the Compute Modules connected as USB devices to a USB SS host. If you used USB SuperSpeed hub chips and did the network switching on a non-Raspi SuperSpeed host within Linux (software defined networking!), this could likely easily get high-ish speed (ie: 100+ Mbps) networking working cost effectively, as the SuperSpeed hubs should allow upstream to run at USB SS rates while downstreams all run at USB HS rates, 7 port USB SS hubs aren't that expensive of chips. Microchip sell the USB5807 7-port USB SS hub chip for about $5, which is only slightly more than the LAN9514, as then you'd only need one hub for every 7 Compute Modules instead of one LAN9514 (or similar) per Compute Module. http://www.microchip.com/wwwproducts/en/USB5807 http://www.microchip.com/wwwproducts/en/USB5807
- logronoide 9y ago35U? That’s a full rack. The cluster looks like a 8U server case.
- pero 9y agowaste of (taxpayer) money using the (overpriced) rpi's gimped (no aes, usb3, gbe) soc the rpi value prop is the community support but that would mean that this cluster is running a 32bit os - so what really is the point of using these instead of something smaller and cheaper or same size and more powerful for the same money
- taylorexpander 9y agoGood that thing national laboratories and research in general don’t have to show profitability then. Because otherwise we’d get nothing done and funded due to the likes of people like you.
- pero 9y agoi don't even know where to start understanding this remark i am questioning why they are using immensely popular and commensurately overpriced but yet woefully underspecced components instead of something better and/or cheaper is it just because 'raspberry pi' makes for a hot title in a press release?
- phkahler 9y agoYour question might be answered if you offered an example (or two) of something more capable and/or cheaper. I'm really interested in what you think would be cheaper. Keep in mind the goal seems to be to build something with a high node-count, rather than just core count so small size is important.
- berbec 9y agoA Nanopi Neo2 is 1/3 the size, similar CPU, has GbE and is 1/2 the cost @$18. [1] I use one for a seedbox/VPN and it is a very nice piece of kit. 1. http://www.friendlyarm.com/index.php?route=product/product&product_id=180 http://www.friendlyarm.com/index.php?route=product/product&p...
- _wmd 9y ago
- ballen 9y agoAnother neat use-case for this hardware is scale testing of systems management tooling, eg. provisioning, configuration management. HPC centers are looking at the possibility of having to manage 100k+ nodes in a single cluster. Additional links: http://cluster.bitscope.com http://cluster.bitscope.com http://www.lanl.gov/discover/news-release-archive/2017/November/1113-raspberry-pi.php http://www.lanl.gov/discover/news-release-archive/2017/Novem... My favorite form factor to go build a Pi-like Cluster is something like this: http://www.friendlyarm.com/index.php?route=product/product&path=69&product_id=211 http://www.friendlyarm.com/index.php?route=product/product&p.... - Gigabit Ethernet MAC on-board, Pine64's SOPINE and Raspberry Pi Compute Module both require per node networking components to be on the carrier board. - Basic headers used for connecting to a carrier board - Carrier board "only" needs an embedded switch and 5v power. However, I've not yet come across an embedded switch yet that has a non-blocking ratio of 1GbE ports to 10/25GbE uplinks. - Carrier board should probably be mini-ITX or other standard form factor to fit in existing chassis. Form factor and embedded switch options are going to limit the number of nodes per carrier board.
- anfractuosity 9y agoI'm just looking at the friendly arm website at some of the other NanoPi's, which I'd not heard of before, I'm quite tempted by the Gbps ethernet. How do you find the software/OS support, are they pretty reliable, with longish uptimes?
- pero 9y agoi have some nanopi neo(1)s and they're rock solid - my rule of thumb with arm boards is that support is good enough if it has an armbian _debian_ (vs just armbian ubuntu) release
- anfractuosity 9y agoCheers I might look at getting one then, as I had problems with my Pi's network connectivity being slow compared to a laptop's using the same ethernet over power line adapter.
- xori 9y agoI bet the IO throughput sucks. Is the ethernet bus still shared with the usb/sd card bus in the rpi3?
- pero 9y agoyes