9 ms·
Show HN: Turing Pi 2 – a compact ARM cluster with 32 GB RAM
- nsky-world 6y agoToday we are thrilled to announce the Turing Pi V2. The Turing Pi V2 is s compact cluster in a mini ITX form factor with 4 x cluster nodes, 2x mini PCIe (Gen 2) ports, 2x SATA (Gen 3) ports, and new Raspberry Pi compute modules 4 support. The Turing Pi 2 cluster is architecturally similar to the AWS Graviton clusters. The CM4 processor uses the ARMv8 architecture. You can build images and applications for AWS instances of Graviton 1 and 2, host cloud native apps locally or at the edge and more.
- geerlingguy 6y agoFor the mini PCIe, do you mean they will be PCIe 1x slots, or something like an M.2 connector? And will it be set up where each board gets one of those slots (e.g. 2 boards get single PCIe, 2 boards get single SATA)?
- nsky-world 6y agom2 connector is something we are still experimenting with. The first two nodes are exposed to mini PCI express port, the third node is exposed to 2-ports 6 Gbps SATA controller. You can use the network file system within the cluster to access hard drives from other nodes.
- geerlingguy 6y agoI guess it's possible to adapt the mini PCI express port to M.2 pretty easily too. Having it PCI express lets you do other interesting things though too besides just storage.
- ant6n 6y agoWould it be possible to install a modern graphics card on this (e.g. some ati Radeon)? That would require some physical adapter on the one hand, and driver support for armv8 on the other.
- rektide 6y ago> The Turing Pi 2 cluster is architecturally identical to the AWS Graviton clusters. Graviton2 uses Armv8.2-A, as opposed to the Armv8.0-A on the CM4. Graviton2's cores are "Neoverse" vs CM4's Cortex A73, which is also a significant difference. A lot of code will be portable, but these are significantly different cores.
- geerlingguy 6y agoProbably more accurate to put "architecturally similar".
- nsky-world 6y agoNoted!
- iron2disulfide 6y agoI'd like to know more about the PCIe Gen2. From your comment it sounds like the root port is capable of Gen2x2, or is it 2x Gen2x1 ports? If I wanted to attach a PCIe accelerator card with a standard PCIe connector, is there some kind of adapter cable I would need to buy?
- nsky-world 6y ago2x Gen2x1
- russell_h 6y agoThis is very neat. I’m sorry if I missed it, but have you announced pricing?
- nsky-world 6y agoWe are expecting the price to be cheaper than the V1, but the final price will be announced later this year.
- k1e 6y agoExcuse me, whats the price of V1? Cant find it on your website.
- nsky-world 6y agoV1 is $189
- mobilio 6y agoIT's amazing device! For now i'm running a few RPi devices and i was dream about such device to put them in for more processing power.
- deleted 6y ago[deleted]
- boris 6y agoOn the page you mention CI/CD applications and we are considering using Raspberry Pi for our CI infra (for the lack of a more powerful option). While there are various possible setups, one fairly common arrangement (especially in a more public-facing services) is to start a throw-away VM for each CI request. This is how our CI service works and you can see it here[1] in action. And while it would have been nice to use Turing Pi, it does not appear to fit our requirements very well because we need to have high-performance storage attached to each node (where we store/run the VMs). An ideal board for us would be a network switch plus for each module a PCIe (or better yet, NVMe) slot and USB 2.0 ports (at least 2x). [1] https://ci.cppget.org https://ci.cppget.org
- theandrewbailey 6y agoAre those USB 3 ports? or USB 2? Do they belong to any particular node?
- geerlingguy 6y agoI spent a bit of time testing the v1 board, and while it isn't without fault, it's the best SBC-based cluster project I've used (haven't personally tested the SOPINE Clusterboard though). Since the CM4 is 2x-3x faster than the CM3+ in pretty much every aspect[1], I'd be very excited to be able to upgrade my Pi Dramble[2] 4x Raspberry Pi 4 model B cluster to a Turing Pi V2! [1] https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-module-4-review https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-... [2] http://www.pidramble.com http://www.pidramble.com
- nsky-world 6y agoThank you. As always, your review of the new CMs is awesome!
- rektide 6y agoPCIe switching would have been really interesting to see, even in a non-concurrent mode. One also desperately wishes that multi-host network adapters had a low-end market. A 1 x 25Gbe connection shared between 6 hosts would be epic. Or if sharing the connection is too much, just a 4 x 2.5Gbe NIC chip would be acceptably interesting & useful to have. I forget, maybe the gbe is free on the Pi4's chip & this is more of a non-issue.
- nsky-world 6y agoPCIe switching would be awesome, for sure, but so far too costly. We want the Turing Pi 2 be cost friendly.
- jackyinger 6y agoPCIe switching sounds nice, but let me tell you that it is a terrible nightmare on every level. Hardware: these PCIe switches are finicky as all get out; will it link up to your device reliably? Who knows, depends on how the switch and device manufacturers interpreted the PCIe specs. Software: there is no nice easy standard for connecting host-to-host (which I’m assuming is what one would want on a multinode system Like the one linked). There exists the idea of a PCIe non-transparent bridge, which if I recall correctly, allows CPUs on either side of it to do blind memory reads and writes to each other (hope your IOMMU is set up correctly to protect you from ugliness that could occur here). Long story short there’s a reason PCIe is a local bus; it was never designed to do anything else. Anything else is largely a proprietary bolt on.
- nsky-world 6y agoTotally agree
- rektide 6y agoI was definitely thinking of NTB, for host-to-host connectivity. As you say, it doesn't provide a whole lot of capabilities. However, there's been a pretty competent ethernet driver available on Linux since Linux 3.9 released April 2013. There's out of kernel NTRDMA if you want ibverbs like communication that will make really good use of the PCIe bus too. In addition to mapped memory, there are some other modest capabilities: support for ringing the peer's doorbell (one way), & scratchpad configuration. There's a pretty good 2016 presentation on NTB in Linux here: https://events.static.linuxfound.org/sites/events/files/slides/Linux%20NTB_0.pdf https://events.static.linuxfound.org/sites/events/files/slid... Interesting to note, Nvidia has docs for settings up NTB on their Pegasus & Xavier systems: https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/System%20Programming/sys_components_non_transparent_bridging.html https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/i... I for one think more people should be investigating & harnessing the open-source Linux NTB tools. Switches are a bit expensive, but compared to ethernet switches their cost/throughput is great. Alas, yes, the whole ecosystem is woefully undersupported, under-developed. Isn't PCI part of the whole PC thing, this great revolution in computing that allowed interoperability to rise? Such a pity that all the PCI SIG's seemed to fizzle out, that not a lot of people participated. Sure we got some SR-IOV but that feels like as far as we got.
- hexman 6y agoAre there more detailed characteristics of the ethernet switch?
- nsky-world 6y agoWe will post switch datasheet later
- billsmithaustin 6y agoI don’t understand the claim about 32GB RAM. The diagram shows slots for four RPi compute modules, each of which has 1GB RAM.
- simlevesque 6y agothere are thirty two flavors of CM4, some contain up 8gb of ram.
- nsky-world 6y agoThe new compute modules comes with up to 8 Gb of RAM. Here is a detailed review of new CMs https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-module-4-review https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-...
- Someone 6y agoReading https://turingpi.com/turing-pi-2-announcement/ https://turingpi.com/turing-pi-2-announcement/, they have designed a “Raspberry Pi 4 Compute Module (CM4)” with 8GB RAM. https://www.raspberrypi.org/products/compute-module-4/?variant=raspberry-pi-cm4001000 https://www.raspberrypi.org/products/compute-module-4/?varia... also lists it as for sale.
- dogma1138 6y agoBesides academic uses to perhaps teach distributed systems on these are there any real world use cases for these clusters?
- pella 6y ago- cloud software developers .. (testing/developing with/without internet connections)
- nsky-world 6y agoThis is a great case study of how the Turing Pi helps with CI/CD for ARM infrastructure https://synyx.de/blog/turing-pi-gitlab-ci-ansible/ https://synyx.de/blog/turing-pi-gitlab-ci-ansible/ Besides that the main direction here is edge computing.
- Voloskaya 6y agoPretty anecdotal use case, but when trying to do some type of neural architecture search for ML models that are ultimately going to be run on the edge, running some performance evaluation on a raspberry pi is a decent proxy. I have seen teams build their own type a raspberry clusters for this to be able to evaluate many models in parallel. It's basically a testing proxy for ARM architecture. But this thing only has 4 RPI per box so I don't quite understand which problem they are targeting either.
- geerlingguy 6y agoI use mine to host some non-revenue-generating websites (they are super cheap for that purpose), and also for some ARM image builds that I integrate into my Docker/Kubernetes pipeline. I haven't yet moved to ARM CPUs on AWS for some of my projects, but may be able to more quickly if I can have a local cluster like this for testing that isn't as slow as the CM3+-based one the Turing Pi V1 supported.
- deleted 6y ago[deleted]
- tuananh 6y ago
- pella 6y agoLong term request for the next Turing ( mixed CPU cards together - in a Box! ) + ARM cards + X86 cards + RISC-V cards
- silasb 6y agoRack mount case would be nice.
- dragontamer 6y agoI assume DIN-mounts, more so than standard 19-inch mounts, given the size of the Turing Pi.
- nsky-world 6y agoThe Turing Pi comes in a standard mini ITX form factor
- draklor40 6y agoOutside of doing it for the heck of it (which is a valid enough use case , IMHO), what practical use do you see out of this ? To me, it looks like I can do all of the same things mentions (perhaps even waay faster) on a x86 processor in the same size factor.
- nsky-world 6y agoSo your question what's the advantage of cluster comparing to one processor unit?
- ianhowson 6y agoYou could get an off-the-shelf x86 machine in the same form factor and run the cluster nodes in VMs, for instance. Maybe you could argue that four RPis on a network are more reliable than a single x86 with VMs, but that doesn't seem to be the point of this machine.
- nsky-world 6y agoThat's one of the point too. We have industrial customers who deploy apps in the places with a limited access to the internet, so increased reliability something important to them. Another thing is that they can host apps with different purposes on different nodes in containers. Besides that all the infrastructure could be managed with a tool like Kubernetes and organically work with existing cloud infrastructure. And the last but not least in a cluster like that you can combine general purpose modules with accelerated compute modules.
- capt_hotpants 6y agoThat's really nice. There are high-end products out there with redundant backplanes (dual or triple) and redundant PSUs (2 or more). To match that level of redundancy it seems you would need 4x Turing Pi, to be able to survive a simultaneous failure of 1x arbitrary PSU and failure of 1x arbitrary backplane (ie. the integrated RTL8370 GigE switch). The 2x external GigE ports on the Turing Pi helps a lot, in that each cluster can connect simultaneously to 2 external GigE switches, in case one of those things fails. Could be an interesting exercise to set up and get it all working, in particular K3s + dynamic routing might be a fun challenge.
- tra3 6y agoForgive me if this is a stupid question; but I see that it has SATA support. Does that mean this can be shoehorned into NAS service? I have an irrational desire to create a cluster, but I would like to use it for something practical like a NAS. Bad idea?
- nsky-world 6y agoYou can. Why not?
- InvaderFizz 6y agoYes, but I would opt for the newly released HC4 if two drives were fine. If you are looking for four to five drives, I'm sure someone will make a PCB at some point that uses A PCIe connected SATA controller. Price becomes a real problem though. The Pi foundation carrier board is dirt cheap, everyone else's boards seem to be a lot of money. Probably due to low volume.
- tra3 6y agoYeah that's what I kinda figured. It's interesting but doesn't make sense from a financial standpoint.
- MichaelBurge 6y agoWhat's the price?
- tuananh 6y agonot announced yet. > The Turing Pi 2 is more functional than V1 and we also expect it to be cheaper to manufacture.
- hinkley 6y agoLots of cool things people were asking for, but I’m a bit sad that they decreased it to 4 slots.
- anaisbetts 6y agoStupid question, how do these 4 SoCs share the 2x SATA ports? Do you assign them to a specific processor somehow? Are the other machines booting off of PXE boot?
- mariusmarais 6y agoThe blog post[0] answers that: peripherals spread between nodes. [0]https://turingpi.com/turing-pi-2-announcement/ https://turingpi.com/turing-pi-2-announcement/
- anaisbetts 6y agoBut like, how? Mounting a volume in read/write mode on more than one machine is obviously going to corrupt it
- tuananh 6y agoRack mount form would be nice. Please do it!
- capt_hotpants 6y agoOut of curiosity, does the firmware in the Compute Module allow for PXE boot? Would be a neat way to handle corruptions of the SD / eMMC. Flick the power off through the Turing Pi's I2C, have one of the other nodes act as a PXE server, flick the power on again to the affected node, and have it pull in a tiny kernel and init-script through PXE that flashes the SD / eMMC with an image from the PXE node.
- capt_hotpants 6y agoAny thoughts on keeping accurate RTC time on these things at sites with poor internet connectivity, weird latency or restrictions on outbound NTP? I'm thinking a nice addition would be a small module to plug on top of the 40-pin GPIO connector with a GPS receiver in it and an antenna attached. The attached node could run NTP and pull accurate time from satellites. Throw in 3 of the modules, and NTP can use quorum to keep the time accurate even if one GPS module fails.
- jmnicolas 6y agoI can't think of anything better than a GPS receiver. About 10 years ago we had a legacy system that was using a weird shaped (think UFO spaceship) FM radio receiver to get time. I never understood why our provider chose that over a simple USB GPS receiver.
- webmaven 6y agoBrings to mind the Slashdot-era refrain "Imagine a Beowulf cluster of these!"