14 ms·

Raspberry Pi Colocation

205 points by 3xa 5y ago

sgtnoodle 5y ago
I wonder how they mirror the raspberry pis without destroying them. (Look at their cad drawing of 12 pis in a rack.)
PragmaticPulp 5y ago
I like the idea in theory, but I can’t entirely agree with the “Green” designation. Putting 12 Raspberry Pis, 12 USB SSDs, 12 switch ports, and cabling and power supplies for all of the above adds up quickly. From a pure compute-per-watt perspective using typical cloud workloads, I’d still expect a run of the mill shared cloud server to be more efficient. It would also allow for more burst overhead for individual workloads. This is an interesting option for people who need a specific Raspberry Pi hosted somewhere.
- 3xa 5y ago
  Hi, "Green", now removed from the title, but still on the landingpage refers to the power used by our data center. Our data centers operate inside wind & solar parks, therefor we only use green energy. If there is no wind or sun we directly purchase green energy. Of course every DC is connected to the public grid for continuous operation.
- piaste 5y ago
  > This is an interesting option for people who need a specific Raspberry Pi hosted somewhere. What is that use case though? The page says that they only host regular Pis and optionally a USB SSD. So they can't do anything that a regular cloud server can't do - no custom hats, etc. I have a Pi 4 home server, and the biggest issue right now is that my home upload is a bit weak for remote video streaming. So this product could interest me, in theory - saves me from having to migrate all my data & configuration to a cloud server. But I would rather pay Hetzner a very similar amount of money to get a VPS that's about as powerful as a Pi (probably more) and still have the physical Pi here at home as a fallback. Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient?
  - zamadatix 5y ago
    Edit: This is in response to the "fleet of Pi's" question, obviously a Mac Mini is not going to be cheaper to rent than a single Pi! The aforementioned VPS route is the better way to go for that case. Scaleway will give you an 8 core 16GB RAM 256GB SSD M1 Mac Mini for €0.1/h. It may not sound like much of an increase from core count but it is ~10x faster for multicore which means it probably comes out on top for perf/€, perf/Watt, and total perf compared to a rack of Pi 4's for any such distributed ARM use case. For pure traditional cloud a Graviton2 instance on AWS is probably more green, albeit probably less cost efficient to the user.
    - selectodude 5y ago
      That’s €72/mo. Different price class there.
      - zamadatix 5y ago
        To be clear this isn't an alternative to hosting a single Pi it was in response to the distributed case: > Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient? Assuming the task really requires ARM, is perfectly scalable among multiple systems, doesn't require more than 10mbps between the nodes, and doesn't require dedicated control/scheduling nodes (i.e. best case for the Pi's) a fleet in multiples of 10 Pi's per would be $59.90 month each plus the up front cost of the Pi's, power adapters, SSDs, and shipping. And even if you wrote off the up front hardware as on hand it would still be significantly less green to run.
  - jo909 5y ago
    > I have a Pi 4 home server, and the biggest issue right now is that my home upload is a bit weak for remote video streaming. "To ensure that every Pi at our decentralized locations always has enough network throughput, the uplink and downlink is fixed at 10 Mbps." Not sure about your use case, but for me that is way less bandwidth than I have at home.
- AshamedCaptain 5y ago
  Raspberry Pis may be "green" in that they are cheap, but power efficient they aren't. They have barely any power management support, making their idle power usage higher than even some x86 chips.
  - Sebb767 5y ago
    > making their idle power usage higher than even some x86 chips What x86 chip can idle on 4W when including RAM and the mainboard? I have some very low power J1900 boards, but even they idle on ~10W.
    - AshamedCaptain 5y ago
      It's not hard for x86 laptops to idle below 4W (see Surface https://www.notebookcheck.net/Microsoft-Surface-Go-Pentium-64GB-eMMC-Tablet-Review.336420.0.html#toc-7 https://www.notebookcheck.net/Microsoft-Surface-Go-Pentium-6... , even Pro ones with screen on they idle well below 10W). With 10W just for the SoC you get into desktop or gaming laptop territory. I have a full x86 system that idles at 1.7W _at the wall_ https://news.ycombinator.com/item?id=26639929 https://news.ycombinator.com/item?id=26639929 . This is an off-the-shelf ASUS PN40 mini-desktop, running an N4000, and includes 8GB RAM, a SATA SSD, and Gigabit ethernet, all running and accepting requests.
      - KennyBlanken 5y ago
        > I have a full x86 system that idles at 1.7W _at the wall_ A single stick of DDR1/2/3 ram uses more than that alone. SATA SSDs are typically 0.5-1W idling. The mobo chipset, figure at least a few watts That doesn't even account for CPU idle power consumption. Whatever you're using for power monitoring isn't accurate. Probably doesn't fully integrate, instead using sampling, and the sampling rate isn't high enough to show true power consumption from your PC's switching PSU.
        AshamedCaptain 5y ago
        It's DDR3L, not DDR3. Even the active power use of the stick is not going to be > 0.5W. And RAPL/ACPI can show the CPU/chipset (respectively) consumption, and it averages less than 1W. The official ASUS measurement is at ~3.7W https://csr.asus.com/english/file/ErP_PN40.pdf https://csr.asus.com/english/file/ErP_PN40.pdf , and that is with Windows, 8 GB stick and 2 SSDs (personally I couldn't get it to idle at less than 6-7W with Windows). Again: there are laptops that idle at these values, and this is even worse than a CPU laptop -- it's an Atom. Think S0i3... which keeps the RAM (in SR), chipset, and a shitton of things awake and yet consumes way less than 1W. > Whatever you're using for power monitoring isn't accurate. Sure (as in: one can never disagree with that), but the point is: it's still at least half the consumption of the Raspberry Pi 4, and lower than the consumption of a Raspberry Pi 3 where I disabled almost everything. For which apparently the measurements match to the 2 digits what I can find online, which I can compare since it's much easier to use the same software-hardware combo. This is not ARM vs x86. It's just the RPI that is crap regarding power management, since by design _they are not even trying_. They just wanted to be cheap. There are many (ARM and other) CPUs that idle even lower than that. If they wanted to be efficient, they would beat the many year old Atom in an instant. Probably already do in (peak) performance/watt.
sokoloff 5y ago
I know that co-location means “customer owned hardware”, but in this case, I think I’d way rather rent data center owned RPis and just pay them money rather than sending in hardware, having to cycle out hardware if/when it fails, etc. It also means the colo is running whatever random power supply I send them, which seems like something they’d want to avoid and means that there’s all the inefficiency of 12 supplies per U rather than one beefy +5.1V supply (with battery backing) feeding the Pis via the GPIO pins.
- 3xa 5y ago
  Thanks for your comment about the power supplies. Not every power supply will work. a) As stated in the FAQs and again when successfully booked: " Note that the power supply should have at least 3.0A / 5.1V power. It must be CE certified and support DC power." b) The DCs are located in Germany: 230 V - DC. I have also read all the other comments on this subject: Yes, currently each Pi runs with its own power supply. Currently, our team is developing a dedicated power supply for all Pi slots within the rack. Then it will also be possible for users to turn the power on and off via the colocation dashboard. However, the development will still take a little time. The reason for this is that some components are not immediately available and especially not in large quantities. See analogously the availability of Pis.
- deleted 5y ago
  [deleted]
- geerlingguy 5y ago
  It seems like either feeding 5v via custom power supply on GPIO or requiring PoE HATs (though those are slightly less efficient...) would be a better scalable option. One massive thing that seems to be missing here (unless I've missed it) is any kind of remote ability to manage the server, eg at a minimum remote power cycling, if the Pi locks up. It would also be nice to get remote console but that would require even more effort and potentially slight customization on the Pis' boot config (to enable UART).
  - sokoloff 5y ago
    Agree, a central supply with a 3A MOSFET for control and a PTC for basic protection would give a lot more functionality and reduce customer downtime and smart-hands touches.
bennyp101 5y ago
From https://examesh.de/en/docs/colocation/accessing-the-pi/ https://examesh.de/en/docs/colocation/accessing-the-pi/ : "Instead of using a public IP the Pi is accessed by combining a public hostname with dedicated TCP ports. The hostname points to one of the ExaMesh gateways and is assigned to the colocation along with the available TCP ports in the booking process." So maybe useful for an extra node for redundency, but maybe not as useful as having an actual address. Perhaps an extra encrypted Syncthing node or something
- 3xa 5y ago
  Thanks for mentioning it. Yes, there are quite a few use cases as these are already implemented by customers: Backup, Sync, Nodes (e.g. BTC Full Nodes), Relay, ...
joosters 5y ago
Somewhat ironically, I'd guess that putting a server inside a wind turbine makes it less likely that you are utilising green energy. The power and comms connections to that location are there primarily to monitor the turbine, and they want that to work all the time, and especially when the blades aren't turning. So you don't go powering it with the wind farm itself. Installing the server anywhere else means there is a chance that its power is being generated by that wind turbine!
- sgtnoodle 5y ago
  One could power the equipment off the turbine when it's operating, and off an alternative supply otherwise. It seems a little silly to worry about where the specific electrons came from to power the equipment, though. If powering that equipment enables a wind turbine to produce more power than it would have without that equipment, then it seems like the existence of that equipment is "green" whether or not its power came from dirtier sources.
  - mike_d 5y ago
    Wind turbines generate 690v three phase. It isn't stepped down until it gets to a substation near the consumer. The power needed onsite (lights, control systems, energy to start the blades spinning, etc) usually comes from a natural gas generator or a direct feed from a fossil fuel plant if one is nearby. Due to circular dependencies, you can't power them off the energy they generate.
    - sgtnoodle 5y ago
      I don't know anything about the domain, but I'm pretty sure it wouldn't be a big deal to tap off a few kilowatts or so using a secondary winding. Something similar to the alternator on a car perhaps. It doesn't need to be the same circuit and efficiency as the main power generation path. What do you mean circular dependencies? You mean that you need an independent source of power for all the infrastructure to turn the turbines on when they're not currently generating? Sure, that makes sense. It seems like there's a ton of ways to implement that. Grid power, generators, batteries and solar panels, etc.
RL_Quine 5y ago
The description of "decentralized" seems to be a little weak here.
mr_sturd 5y ago
EDIS offered a colo service for free, back in the early RPi days. I had two gen 1.5 machines hosted with them; one with OwnCloud, and another hosted my music via SFTP.
- bullen 5y ago
  gen 1.5? Are they still running those for free? I had "free" colocations in Sweden and Holland that then turned not free then got cancelled altogether. Pi clusters are best for home hosting on your own fiber. Also those Pi 4 need heatsinks like so: http://move.rupy.se/file/final_pi_2_4_hybrid.png http://move.rupy.se/file/final_pi_2_4_hybrid.png
  - 3xa 5y ago
    There were more than a handful of free Raspberry Pi colocations. However, since there is real manual work and infrastructure behind it, there are no such offers anymore, as far as we could check.
  - mr_sturd 5y ago
    They were a revision of the first gen Raspberry Pi. A bit more stable and seemed to not corrupt the root filesystem after a few hours, which the initial one seemed to invariably do for me. They're not offering it any more, no. I moved to self-hosting after that. Even got a static IP address for it.
    - bullen 5y ago
      I find static IP a bit overpriced when you can easily build your own dynamic IP solver with a few instances with static IPs that you need anyway to get low latency around the globe, also most home IPSes block ports 25 and 53 so if you want mail and dns you still need additional boxes. I'm using google (bad I know but they are the best currently) as my main 3 continent provider (my own HTTP with JSON db, SMTP and DNS implemented from scratch) and I have redundancy in asia on AWS and in central US with IONOS, that way I have the cheapest double redundant cloud you can get with the most customers in low latency reach!
mysterydip 5y ago
"12 pis in 1U" where 1U is defined as the height of a pi on its side plus shelf, rather than the definition of 1U in every other 19" rackmount data center
- 3xa 5y ago
  Cool spelling mistake found. It's already on our ToDo list and will be corrected soon.
- zamadatix 5y ago
  From their rack diagram it looks to really be 2.67U + 1U 24 port switch per 12.
- aae42 5y ago
  it seemed like they didn't quite understand the concept of rack units to me either
projektfu 5y ago
How do you fit a raspberry pi edge-wise (56.5mm) into a rack unit (44.5mm)?
- kingcharles 5y ago
  Very carefully.
- evan_ 5y ago
  It looks like they have a rack that’s 1U tall, but they only fill every few units to leave space for the pis. So in other words, it’s 3U...
- mbalyuzi 5y ago
  This https://twitter.com/Merocle/status/1407684311344730117 https://twitter.com/Merocle/status/1407684311344730117 is quite a nice approach, albeit using a CM4.
whalesalad 5y ago
This is neat but from a scaling perspective it doesn’t make sense. A single server grade Xeon chip can expose the same compute power as a cluster of these devices, with better performance across the board (memory access, peripherals, etc) Just trying to grok a legit use case?
- flatiron 5y ago
  Nervous people about spectre? Only thing I can think of besides renting a VM.
- KennyBlanken 5y ago
  The use case is greenwashing and separating people from their money because everyone thinks Pis are just the bee's knees. The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400...so in theory a 5600x is twice as energy efficient as a Pi4 if fully loaded. Sure this doesn't account for system fans and the motherboard, but they don't use that much compared to the CPU. The whole point of virtualization is that most systems are idle a lot of the time. At datacenter scale virtualization, you can dramatically over-provision and shut down/sleep unnecessary nodes, firing them up when you need to. You can get near 100% utilization on your hardware, making the very most of every watt that doesn't go to actually computing. Here they're going to have a zillion Pi4's, most of them sitting idle, but still using a couple watts. They're not even bothering to use any sort of shared power to improve PSU efficiency. They're not even bothering to use Pi4 compute modules. Now, the interesting bit is that now there's the Pi Zero Wireless 2. It has nearly the compute power of the 3B+, but the highest energy efficiency per watt of any Pi board so far...
  - selfhoster11 5y ago
    > The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400... That's more energy efficient, sure. But it sets a lower boundary on the power draw much higher than a normal Pi. A Pi plus a single external HDD draws 12W at the socket, according to my measurements. A PC CPU draws 4-5x that, just by itself. The other components on the motherboard need power too, even if you use integrated graphics or no graphics at all. A PC only becomes more power-efficient if your load can't fit in three or more Pis. For plenty of uses, more than two Pis are an overkill.
buildbuildbuild 5y ago
I think they'll need to iterate a bit to find product market fit. The 10mbit bandwidth limit, calling it "Decentralized", no public IP downsides are off-putting even at this price.
john61 5y ago
In Vienna/Austria there is a rpi or similar housing with a real ipv4 address: https://www.easyserver.at/serverhousing https://www.easyserver.at/serverhousing
solarkraft 5y ago
I'm gonna ask the dumb/obvious question: Why would I want this? It's certainly not for the compute. Isn't the point of a Raspberry Pi controlling periphery on the edge? But that's not possible here? ??? It's not even needing ARM cores, as those are now cheaply offered by all the cloud computing companies. Is it just for some cheap fun? But if I'm going to host something on cheap amateur grade hardware, why would I not also just use my home connection? Is this for the experience and education? ... I really don't see what it's good for (explanations welcome).
- my123 5y ago
  Small unit of dedicated hardware, without any other tenants on that same host.
  - tata71 5y ago
    Hugely underrated comment.
  - contravariant 5y ago
    I mean sure, but if you've already bough a raspberry pi then you're most of the way there surely?
- deleted 5y ago
  [deleted]
- wpietri 5y ago
  For many years I was part of a bandwidth cooperative. We had a cabinet and a fat pipe and a bunch of sysadmins who wanted a place to keep their stuff. Early on it was all 1U or 2U systems. But later there was enough demand for Mac Minis that we dedicated a shelf to them. It didn't make much sense from a professional syadmin's perspective. But for a Mac user who already had their little project on a Mini and wanted to get it off their home bandwidth, it made sense to them in that it was one simple, incremental change. I imagine the market here is similar.
  - 5y ago
frankjr 5y ago
While I completely understand the allure of running on your own hardware, if you just want a cheap server to host a personal page or similar, you cannot beat Scaleway's Stardust VM instances. For less than 2 EUR a month you get 1 vCPU, 1 GB of RAM, 1 IPv4+IPv6 address, 10GB of storage and unlimited traffic. They claim up-to 100Mbps bandwidth but I regularly get much more than that. This sounds like a commercial but I'm just really happy with the service.
- ringworld 5y ago
  https://www.scaleway.com/en/stardust-instances/ https://www.scaleway.com/en/stardust-instances/ I am not a fan of the lottery approach and being told you're lucky to do business with them.
  - frankjr 5y ago
    I understand it more like a struggle to keep up it the demand for these instances and this was their attempt to "gamify" it.
  - randomluck040 5y ago
    The lottery approach pissed me off so bad, I decided to leave Scaleway altogether. They also aren’t upfront about their contingents. I wanted to try out an M1 instance and before registering and putting my credit card info in, it seemed like it’d be all fine, I just have to put in cc info. I did, they told me there are no instances available. The fair way would be to be upfront about it in my opinion.
    - xuki 5y ago
      Hetzner might have M1 mac available: https://www.hetzner.com/dedicated-rootserver/mac-mini-m1 https://www.hetzner.com/dedicated-rootserver/mac-mini-m1
  - mike_d 5y ago
    It isn't a lottery as much as an availability constraint. First you have a limit of 1 stardust per datacenter per customer. Second, they only spin up a fixed number of new stardust servers per day. It is a loss leader just like in any other business. They are losing money on one instance to get you in the door and you realize their other services are awesome.
anyfactor 5y ago
I bet everyone who has a raspberry pi had this idea. Throwing a raspberry pi with a solar panel and a sim card to a random place. It could be for backup, vpn or to access some private network. But having it be a rackmounted VM in a fixed location doesn't sound that fun to me.
1MachineElf 5y ago
At least this offering eliminates the threat vector a of compromised hypervisor.
paulcole 5y ago
How exactly is it exclusive?
ushakov 5y ago
there is a similiar, anonymous option from The Pirate Bay founder https://prq.se/?p=rpi https://prq.se/?p=rpi
ZiiS 5y ago
Realy needs a secure boot option.
aofeisheng 5y ago
> What is the traffic limit? > It's 2021. We don't have a traffic limit for a Raspberry Pi. > What is the data transfer rate? > The data rate is synchronously set to 10 Mbit/s per Raspberry. It's 2021, and you think 10 Mbit/s is enough.
- bogantech 5y ago
  It's because they don't know how to do any kind of QoS
- ChuckNorris89 5y ago
  >It's 2021, and you think 10 Mbit/s is enough. I mean, not to disagree here, but that's pretty much the average internet speed in some third world countries, like Austria for example. :)
  - tiagod 5y ago
    So, a single person in such country would be enough to saturate the server...
- deleted 5y ago
  [deleted]
walrus01 5y ago
I think I'd rather have a decently specced KVM VM on a x86-64 hypervisor somewhere, I can run mainline debian on, for $6/mo than a raspberry pi. For that money if you look you can get something with 2 pseudo cores, 2 gigs of ram, and probably 40GB of storage. At least I can have more confidence that the storage won't spontaneously fail, and network throughput greater than 10 Mbps. This seems like a cool idea and all and it's certainly cheap for hobby projects. But I wonder how viable it really is as a business model. Doing the math on person-hour costs if just one pi requires 15 minutes of support/human attention from a person at the ISP, once, you're losing money on that customer forever.
ruined 5y ago
ok but does anyone offer pi VMs that can boot my sdcard image
roughly 5y ago
I totally understand all of the drawbacks here, I agree that it’s hard to think of an actual use case, and all that aside, there’s something aesthetically pleasing here in an “I’d read about this in a William Gibson novel” kind of way. “My compute fleet is distributed across a field of windmills in Europe” just _sounds_ cool.
erulabs 5y ago
Hey this is sort of the mirror opposite of my startup (we try to bring the internet to your home-pi, rather than ship your home-pi to a datacenter!). Neat tho! I'm not entirely sure it's that power efficient versus a carved up hypervisor tho...
- 3xa 5y ago
  Very cool commenting on our product. Saw your PiBox on kickstarter and PH. Love it!
- alexatalktome 5y ago
  Oh my god you run KubeSail! Neat! I saw this and thought “can I use kube sail and host stuff in a mini cloud?”
jagger27 5y ago
I can’t imagine having any hardware colocated without proper out-of-band KVM access. Who is going to drive out to the wind turbine and flash a new disk image to my Pi? 10Mbps is also excruciatingly slow. I was ready to see a 100Mbps cap.
smarx007 5y ago
> To ensure that every Pi at our decentralized locations always has enough network throughput, the uplink and downlink is fixed at 10 Mbps. Ok, thx, I have a 100/10 Mpbs link at home. The only reason I'd place my Pi in a colo is to get 100/100 Mpbs or 1 Gbit network. Edit: https://contabo.com/en/vps/ https://contabo.com/en/vps/ (200Mpbs in the cheapest plan) or https://www.seedhost.eu/ https://www.seedhost.eu/ (1/10G) is not too far from the €6,- price mark and I don't have to own the hardware.
daneel_w 5y ago
The main gotchas: 1) no public IP of your own but instead you get a few dedicated ports NATd to you on a shared IP, 2) bandwidth is fixed at 10/10 Mbit/s. Pi co-lo for just four bucks a month sounded great until the fine print was revealed...
Elyott 5y ago
10mbps? Seems pretty low and a bit pointless unless you preload the SSD, even then, why?
daneel_w 5y ago
A question to ExaMesh, in case someone from there happens to be reading: could you publically offer a guarantee and promise that not a single byte will be read off of (or written to) customers' SD cards or SSD drives before being installed, or after being uninstalled from your racks?
- 3xa 5y ago
  I personally vouch for our team, but think about it: your Pi is connected to the Internet. You are responsible for the security of the Pi (system security, updates, encryption ...). You send us the hardware in the mail. How many hands does the package go through? Of course, no one gets into the data center without access authorization, etc. pp. Encryption of your data is the key here.
  - daneel_w 5y ago
    Thanks for the assurance. The concern with "mail order co-lo" is of course that the customers themselves don't travel to the DC to install and power up the server.
- KennyBlanken 5y ago
  Not ExaMesh, but there are a variety of ways to encrypt the card and remotely unlock it. You can use dropbear to ssh into the Pi and provide the key during boot. You can have the Pi connect to a remote system to retrieve the key. Some methods are obviously not perfect, but it'll definitely make it more complicated than just "copy the card." Remote key retrieval would let you audit when the system was booted and so on. There's a more complex purpose-built open source software package specifically for handling remote disk unlocks but I'm blanking on the name and my google searches aren't turning it up. I vaguely recall it had fairly high levels of paranoia in its design.
mkj 5y ago
If it's BYO power brick you could also colo a second computer inside the power brick, using the rpi as a "modem" for it. Put a tamper resistant device inside that case.
systemvoltage 5y ago
EU Privacy means nothing, it's just marketing: https://news.ycombinator.com/item?id=29051305 https://news.ycombinator.com/item?id=29051305
drcongo 5y ago
Can't believe they didn't call it Picolo.
- 3xa 5y ago
  Most domain extensions are already registered. And since we are from Germany, the term "Piccolo" sound very similar to "Picolo" but means something very different: https://de.wikipedia.org/wiki/Piccolo_(Sekt) https://de.wikipedia.org/wiki/Piccolo_(Sekt) However, we would have definitely pursued an overlap/commonality of the word with, say, beer!

205 points by 3xa 5y ago

I wonder how they mirror the raspberry pis without destroying them. (Look at their cad drawing of 12 pis in a rack.)

I like the idea in theory, but I can’t entirely agree with the “Green” designation. Putting 12 Raspberry Pis, 12 USB SSDs, 12 switch ports, and cabling and power supplies for all of the above adds up quickly. From a pure compute-per-watt perspective using typical cloud workloads, I’d still expect a run of the mill shared cloud server to be more efficient. It would also allow for more burst overhead for individual workloads. This is an interesting option for people who need a specific Raspberry Pi hosted somewhere.

3xa 5y ago
Hi, "Green", now removed from the title, but still on the landingpage refers to the power used by our data center. Our data centers operate inside wind & solar parks, therefor we only use green energy. If there is no wind or sun we directly purchase green energy. Of course every DC is connected to the public grid for continuous operation.
piaste 5y ago
> This is an interesting option for people who need a specific Raspberry Pi hosted somewhere. What is that use case though? The page says that they only host regular Pis and optionally a USB SSD. So they can't do anything that a regular cloud server can't do - no custom hats, etc. I have a Pi 4 home server, and the biggest issue right now is that my home upload is a bit weak for remote video streaming. So this product could interest me, in theory - saves me from having to migrate all my data & configuration to a cloud server. But I would rather pay Hetzner a very similar amount of money to get a VPS that's about as powerful as a Pi (probably more) and still have the physical Pi here at home as a fallback. Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient?
- zamadatix 5y ago
  Edit: This is in response to the "fleet of Pi's" question, obviously a Mac Mini is not going to be cheaper to rent than a single Pi! The aforementioned VPS route is the better way to go for that case. Scaleway will give you an 8 core 16GB RAM 256GB SSD M1 Mac Mini for €0.1/h. It may not sound like much of an increase from core count but it is ~10x faster for multicore which means it probably comes out on top for perf/€, perf/Watt, and total perf compared to a rack of Pi 4's for any such distributed ARM use case. For pure traditional cloud a Graviton2 instance on AWS is probably more green, albeit probably less cost efficient to the user.
  - selectodude 5y ago
    That’s €72/mo. Different price class there.
    - zamadatix 5y ago
      To be clear this isn't an alternative to hosting a single Pi it was in response to the distributed case: > Maybe there are ARM-specialized, highly distributed tasks for which a fleet of Pis is particularly efficient? Assuming the task really requires ARM, is perfectly scalable among multiple systems, doesn't require more than 10mbps between the nodes, and doesn't require dedicated control/scheduling nodes (i.e. best case for the Pi's) a fleet in multiples of 10 Pi's per would be $59.90 month each plus the up front cost of the Pi's, power adapters, SSDs, and shipping. And even if you wrote off the up front hardware as on hand it would still be significantly less green to run.
- jo909 5y ago
  > I have a Pi 4 home server, and the biggest issue right now is that my home upload is a bit weak for remote video streaming. "To ensure that every Pi at our decentralized locations always has enough network throughput, the uplink and downlink is fixed at 10 Mbps." Not sure about your use case, but for me that is way less bandwidth than I have at home.
AshamedCaptain 5y ago
Raspberry Pis may be "green" in that they are cheap, but power efficient they aren't. They have barely any power management support, making their idle power usage higher than even some x86 chips.
- Sebb767 5y ago
  > making their idle power usage higher than even some x86 chips What x86 chip can idle on 4W when including RAM and the mainboard? I have some very low power J1900 boards, but even they idle on ~10W.
  - AshamedCaptain 5y ago
    It's not hard for x86 laptops to idle below 4W (see Surface https://www.notebookcheck.net/Microsoft-Surface-Go-Pentium-64GB-eMMC-Tablet-Review.336420.0.html#toc-7 https://www.notebookcheck.net/Microsoft-Surface-Go-Pentium-6... , even Pro ones with screen on they idle well below 10W). With 10W just for the SoC you get into desktop or gaming laptop territory. I have a full x86 system that idles at 1.7W _at the wall_ https://news.ycombinator.com/item?id=26639929 https://news.ycombinator.com/item?id=26639929 . This is an off-the-shelf ASUS PN40 mini-desktop, running an N4000, and includes 8GB RAM, a SATA SSD, and Gigabit ethernet, all running and accepting requests.
    - KennyBlanken 5y ago
      > I have a full x86 system that idles at 1.7W _at the wall_ A single stick of DDR1/2/3 ram uses more than that alone. SATA SSDs are typically 0.5-1W idling. The mobo chipset, figure at least a few watts That doesn't even account for CPU idle power consumption. Whatever you're using for power monitoring isn't accurate. Probably doesn't fully integrate, instead using sampling, and the sampling rate isn't high enough to show true power consumption from your PC's switching PSU.
      - AshamedCaptain 5y ago
        It's DDR3L, not DDR3. Even the active power use of the stick is not going to be > 0.5W. And RAPL/ACPI can show the CPU/chipset (respectively) consumption, and it averages less than 1W. The official ASUS measurement is at ~3.7W https://csr.asus.com/english/file/ErP_PN40.pdf https://csr.asus.com/english/file/ErP_PN40.pdf , and that is with Windows, 8 GB stick and 2 SSDs (personally I couldn't get it to idle at less than 6-7W with Windows). Again: there are laptops that idle at these values, and this is even worse than a CPU laptop -- it's an Atom. Think S0i3... which keeps the RAM (in SR), chipset, and a shitton of things awake and yet consumes way less than 1W. > Whatever you're using for power monitoring isn't accurate. Sure (as in: one can never disagree with that), but the point is: it's still at least half the consumption of the Raspberry Pi 4, and lower than the consumption of a Raspberry Pi 3 where I disabled almost everything. For which apparently the measurements match to the 2 digits what I can find online, which I can compare since it's much easier to use the same software-hardware combo. This is not ARM vs x86. It's just the RPI that is crap regarding power management, since by design _they are not even trying_. They just wanted to be cheap. There are many (ARM and other) CPUs that idle even lower than that. If they wanted to be efficient, they would beat the many year old Atom in an instant. Probably already do in (peak) performance/watt.

sokoloff 5y ago

I know that co-location means “customer owned hardware”, but in this case, I think I’d way rather rent data center owned RPis and just pay them money rather than sending in hardware, having to cycle out hardware if/when it fails, etc. It also means the colo is running whatever random power supply I send them, which seems like something they’d want to avoid and means that there’s all the inefficiency of 12 supplies per U rather than one beefy +5.1V supply (with battery backing) feeding the Pis via the GPIO pins.

3xa 5y ago
Thanks for your comment about the power supplies. Not every power supply will work. a) As stated in the FAQs and again when successfully booked: " Note that the power supply should have at least 3.0A / 5.1V power. It must be CE certified and support DC power." b) The DCs are located in Germany: 230 V - DC. I have also read all the other comments on this subject: Yes, currently each Pi runs with its own power supply. Currently, our team is developing a dedicated power supply for all Pi slots within the rack. Then it will also be possible for users to turn the power on and off via the colocation dashboard. However, the development will still take a little time. The reason for this is that some components are not immediately available and especially not in large quantities. See analogously the availability of Pis.
deleted 5y ago
[deleted]
geerlingguy 5y ago
It seems like either feeding 5v via custom power supply on GPIO or requiring PoE HATs (though those are slightly less efficient...) would be a better scalable option. One massive thing that seems to be missing here (unless I've missed it) is any kind of remote ability to manage the server, eg at a minimum remote power cycling, if the Pi locks up. It would also be nice to get remote console but that would require even more effort and potentially slight customization on the Pis' boot config (to enable UART).
- sokoloff 5y ago
  Agree, a central supply with a 3A MOSFET for control and a PTC for basic protection would give a lot more functionality and reduce customer downtime and smart-hands touches.

bennyp101 5y ago

From https://examesh.de/en/docs/colocation/accessing-the-pi/ https://examesh.de/en/docs/colocation/accessing-the-pi/ : "Instead of using a public IP the Pi is accessed by combining a public hostname with dedicated TCP ports. The hostname points to one of the ExaMesh gateways and is assigned to the colocation along with the available TCP ports in the booking process." So maybe useful for an extra node for redundency, but maybe not as useful as having an actual address. Perhaps an extra encrypted Syncthing node or something

3xa 5y ago
Thanks for mentioning it. Yes, there are quite a few use cases as these are already implemented by customers: Backup, Sync, Nodes (e.g. BTC Full Nodes), Relay, ...

joosters 5y ago

Somewhat ironically, I'd guess that putting a server inside a wind turbine makes it less likely that you are utilising green energy. The power and comms connections to that location are there primarily to monitor the turbine, and they want that to work all the time, and especially when the blades aren't turning. So you don't go powering it with the wind farm itself. Installing the server anywhere else means there is a chance that its power is being generated by that wind turbine!

sgtnoodle 5y ago
One could power the equipment off the turbine when it's operating, and off an alternative supply otherwise. It seems a little silly to worry about where the specific electrons came from to power the equipment, though. If powering that equipment enables a wind turbine to produce more power than it would have without that equipment, then it seems like the existence of that equipment is "green" whether or not its power came from dirtier sources.
- mike_d 5y ago
  Wind turbines generate 690v three phase. It isn't stepped down until it gets to a substation near the consumer. The power needed onsite (lights, control systems, energy to start the blades spinning, etc) usually comes from a natural gas generator or a direct feed from a fossil fuel plant if one is nearby. Due to circular dependencies, you can't power them off the energy they generate.
  - sgtnoodle 5y ago
    I don't know anything about the domain, but I'm pretty sure it wouldn't be a big deal to tap off a few kilowatts or so using a secondary winding. Something similar to the alternator on a car perhaps. It doesn't need to be the same circuit and efficiency as the main power generation path. What do you mean circular dependencies? You mean that you need an independent source of power for all the infrastructure to turn the turbines on when they're not currently generating? Sure, that makes sense. It seems like there's a ton of ways to implement that. Grid power, generators, batteries and solar panels, etc.

RL_Quine 5y ago

The description of "decentralized" seems to be a little weak here.

mr_sturd 5y ago

EDIS offered a colo service for free, back in the early RPi days. I had two gen 1.5 machines hosted with them; one with OwnCloud, and another hosted my music via SFTP.

bullen 5y ago
gen 1.5? Are they still running those for free? I had "free" colocations in Sweden and Holland that then turned not free then got cancelled altogether. Pi clusters are best for home hosting on your own fiber. Also those Pi 4 need heatsinks like so: http://move.rupy.se/file/final_pi_2_4_hybrid.png http://move.rupy.se/file/final_pi_2_4_hybrid.png
- 3xa 5y ago
  There were more than a handful of free Raspberry Pi colocations. However, since there is real manual work and infrastructure behind it, there are no such offers anymore, as far as we could check.
- mr_sturd 5y ago
  They were a revision of the first gen Raspberry Pi. A bit more stable and seemed to not corrupt the root filesystem after a few hours, which the initial one seemed to invariably do for me. They're not offering it any more, no. I moved to self-hosting after that. Even got a static IP address for it.
  - bullen 5y ago
    I find static IP a bit overpriced when you can easily build your own dynamic IP solver with a few instances with static IPs that you need anyway to get low latency around the globe, also most home IPSes block ports 25 and 53 so if you want mail and dns you still need additional boxes. I'm using google (bad I know but they are the best currently) as my main 3 continent provider (my own HTTP with JSON db, SMTP and DNS implemented from scratch) and I have redundancy in asia on AWS and in central US with IONOS, that way I have the cheapest double redundant cloud you can get with the most customers in low latency reach!

mysterydip 5y ago

"12 pis in 1U" where 1U is defined as the height of a pi on its side plus shelf, rather than the definition of 1U in every other 19" rackmount data center

3xa 5y ago
Cool spelling mistake found. It's already on our ToDo list and will be corrected soon.
zamadatix 5y ago
From their rack diagram it looks to really be 2.67U + 1U 24 port switch per 12.
aae42 5y ago
it seemed like they didn't quite understand the concept of rack units to me either

projektfu 5y ago

How do you fit a raspberry pi edge-wise (56.5mm) into a rack unit (44.5mm)?

kingcharles 5y ago
Very carefully.
evan_ 5y ago
It looks like they have a rack that’s 1U tall, but they only fill every few units to leave space for the pis. So in other words, it’s 3U...
mbalyuzi 5y ago
This https://twitter.com/Merocle/status/1407684311344730117 https://twitter.com/Merocle/status/1407684311344730117 is quite a nice approach, albeit using a CM4.

whalesalad 5y ago

This is neat but from a scaling perspective it doesn’t make sense. A single server grade Xeon chip can expose the same compute power as a cluster of these devices, with better performance across the board (memory access, peripherals, etc) Just trying to grok a legit use case?

flatiron 5y ago
Nervous people about spectre? Only thing I can think of besides renting a VM.
KennyBlanken 5y ago
The use case is greenwashing and separating people from their money because everyone thinks Pis are just the bee's knees. The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400...so in theory a 5600x is twice as energy efficient as a Pi4 if fully loaded. Sure this doesn't account for system fans and the motherboard, but they don't use that much compared to the CPU. The whole point of virtualization is that most systems are idle a lot of the time. At datacenter scale virtualization, you can dramatically over-provision and shut down/sleep unnecessary nodes, firing them up when you need to. You can get near 100% utilization on your hardware, making the very most of every watt that doesn't go to actually computing. Here they're going to have a zillion Pi4's, most of them sitting idle, but still using a couple watts. They're not even bothering to use any sort of shared power to improve PSU efficiency. They're not even bothering to use Pi4 compute modules. Now, the interesting bit is that now there's the Pi Zero Wireless 2. It has nearly the compute power of the 3B+, but the highest energy efficiency per watt of any Pi board so far...
- selfhoster11 5y ago
  > The Pentium G6400 outperforms a Pi4 4-5x, and has a 54W TDP (onboard GPU so at least part of that is for the GPU, so CPU-only workloads will be less.) The Ryzen 5600x is 65W and is twice as fast (at least) as the G6400... That's more energy efficient, sure. But it sets a lower boundary on the power draw much higher than a normal Pi. A Pi plus a single external HDD draws 12W at the socket, according to my measurements. A PC CPU draws 4-5x that, just by itself. The other components on the motherboard need power too, even if you use integrated graphics or no graphics at all. A PC only becomes more power-efficient if your load can't fit in three or more Pis. For plenty of uses, more than two Pis are an overkill.

buildbuildbuild 5y ago

I think they'll need to iterate a bit to find product market fit. The 10mbit bandwidth limit, calling it "Decentralized", no public IP downsides are off-putting even at this price.

john61 5y ago

In Vienna/Austria there is a rpi or similar housing with a real ipv4 address: https://www.easyserver.at/serverhousing https://www.easyserver.at/serverhousing

solarkraft 5y ago

I'm gonna ask the dumb/obvious question: Why would I want this? It's certainly not for the compute. Isn't the point of a Raspberry Pi controlling periphery on the edge? But that's not possible here? ??? It's not even needing ARM cores, as those are now cheaply offered by all the cloud computing companies. Is it just for some cheap fun? But if I'm going to host something on cheap amateur grade hardware, why would I not also just use my home connection? Is this for the experience and education? ... I really don't see what it's good for (explanations welcome).

my123 5y ago
Small unit of dedicated hardware, without any other tenants on that same host.
- tata71 5y ago
  Hugely underrated comment.
- contravariant 5y ago
  I mean sure, but if you've already bough a raspberry pi then you're most of the way there surely?
deleted 5y ago
[deleted]
wpietri 5y ago
For many years I was part of a bandwidth cooperative. We had a cabinet and a fat pipe and a bunch of sysadmins who wanted a place to keep their stuff. Early on it was all 1U or 2U systems. But later there was enough demand for Mac Minis that we dedicated a shelf to them. It didn't make much sense from a professional syadmin's perspective. But for a Mac user who already had their little project on a Mini and wanted to get it off their home bandwidth, it made sense to them in that it was one simple, incremental change. I imagine the market here is similar.
- 5y ago

frankjr 5y ago

While I completely understand the allure of running on your own hardware, if you just want a cheap server to host a personal page or similar, you cannot beat Scaleway's Stardust VM instances. For less than 2 EUR a month you get 1 vCPU, 1 GB of RAM, 1 IPv4+IPv6 address, 10GB of storage and unlimited traffic. They claim up-to 100Mbps bandwidth but I regularly get much more than that. This sounds like a commercial but I'm just really happy with the service.

ringworld 5y ago
https://www.scaleway.com/en/stardust-instances/ https://www.scaleway.com/en/stardust-instances/ I am not a fan of the lottery approach and being told you're lucky to do business with them.
- frankjr 5y ago
  I understand it more like a struggle to keep up it the demand for these instances and this was their attempt to "gamify" it.
- randomluck040 5y ago
  The lottery approach pissed me off so bad, I decided to leave Scaleway altogether. They also aren’t upfront about their contingents. I wanted to try out an M1 instance and before registering and putting my credit card info in, it seemed like it’d be all fine, I just have to put in cc info. I did, they told me there are no instances available. The fair way would be to be upfront about it in my opinion.
  - xuki 5y ago
    Hetzner might have M1 mac available: https://www.hetzner.com/dedicated-rootserver/mac-mini-m1 https://www.hetzner.com/dedicated-rootserver/mac-mini-m1
- mike_d 5y ago
  It isn't a lottery as much as an availability constraint. First you have a limit of 1 stardust per datacenter per customer. Second, they only spin up a fixed number of new stardust servers per day. It is a loss leader just like in any other business. They are losing money on one instance to get you in the door and you realize their other services are awesome.

anyfactor 5y ago

I bet everyone who has a raspberry pi had this idea. Throwing a raspberry pi with a solar panel and a sim card to a random place. It could be for backup, vpn or to access some private network. But having it be a rackmounted VM in a fixed location doesn't sound that fun to me.

1MachineElf 5y ago

At least this offering eliminates the threat vector a of compromised hypervisor.

paulcole 5y ago

How exactly is it exclusive?

ushakov 5y ago

there is a similiar, anonymous option from The Pirate Bay founder https://prq.se/?p=rpi https://prq.se/?p=rpi

ZiiS 5y ago

Realy needs a secure boot option.

aofeisheng 5y ago

> What is the traffic limit? > It's 2021. We don't have a traffic limit for a Raspberry Pi. > What is the data transfer rate? > The data rate is synchronously set to 10 Mbit/s per Raspberry. It's 2021, and you think 10 Mbit/s is enough.

bogantech 5y ago
It's because they don't know how to do any kind of QoS
ChuckNorris89 5y ago
>It's 2021, and you think 10 Mbit/s is enough. I mean, not to disagree here, but that's pretty much the average internet speed in some third world countries, like Austria for example. :)
- tiagod 5y ago
  So, a single person in such country would be enough to saturate the server...
deleted 5y ago
[deleted]

walrus01 5y ago

I think I'd rather have a decently specced KVM VM on a x86-64 hypervisor somewhere, I can run mainline debian on, for $6/mo than a raspberry pi. For that money if you look you can get something with 2 pseudo cores, 2 gigs of ram, and probably 40GB of storage. At least I can have more confidence that the storage won't spontaneously fail, and network throughput greater than 10 Mbps. This seems like a cool idea and all and it's certainly cheap for hobby projects. But I wonder how viable it really is as a business model. Doing the math on person-hour costs if just one pi requires 15 minutes of support/human attention from a person at the ISP, once, you're losing money on that customer forever.

ruined 5y ago

ok but does anyone offer pi VMs that can boot my sdcard image

roughly 5y ago

I totally understand all of the drawbacks here, I agree that it’s hard to think of an actual use case, and all that aside, there’s something aesthetically pleasing here in an “I’d read about this in a William Gibson novel” kind of way. “My compute fleet is distributed across a field of windmills in Europe” just _sounds_ cool.

erulabs 5y ago

Hey this is sort of the mirror opposite of my startup (we try to bring the internet to your home-pi, rather than ship your home-pi to a datacenter!). Neat tho! I'm not entirely sure it's that power efficient versus a carved up hypervisor tho...

3xa 5y ago
Very cool commenting on our product. Saw your PiBox on kickstarter and PH. Love it!
alexatalktome 5y ago
Oh my god you run KubeSail! Neat! I saw this and thought “can I use kube sail and host stuff in a mini cloud?”

jagger27 5y ago

I can’t imagine having any hardware colocated without proper out-of-band KVM access. Who is going to drive out to the wind turbine and flash a new disk image to my Pi? 10Mbps is also excruciatingly slow. I was ready to see a 100Mbps cap.

smarx007 5y ago

> To ensure that every Pi at our decentralized locations always has enough network throughput, the uplink and downlink is fixed at 10 Mbps. Ok, thx, I have a 100/10 Mpbs link at home. The only reason I'd place my Pi in a colo is to get 100/100 Mpbs or 1 Gbit network. Edit: https://contabo.com/en/vps/ https://contabo.com/en/vps/ (200Mpbs in the cheapest plan) or https://www.seedhost.eu/ https://www.seedhost.eu/ (1/10G) is not too far from the €6,- price mark and I don't have to own the hardware.

daneel_w 5y ago

The main gotchas: 1) no public IP of your own but instead you get a few dedicated ports NATd to you on a shared IP, 2) bandwidth is fixed at 10/10 Mbit/s. Pi co-lo for just four bucks a month sounded great until the fine print was revealed...

Elyott 5y ago

10mbps? Seems pretty low and a bit pointless unless you preload the SSD, even then, why?

daneel_w 5y ago

A question to ExaMesh, in case someone from there happens to be reading: could you publically offer a guarantee and promise that not a single byte will be read off of (or written to) customers' SD cards or SSD drives before being installed, or after being uninstalled from your racks?

3xa 5y ago
I personally vouch for our team, but think about it: your Pi is connected to the Internet. You are responsible for the security of the Pi (system security, updates, encryption ...). You send us the hardware in the mail. How many hands does the package go through? Of course, no one gets into the data center without access authorization, etc. pp. Encryption of your data is the key here.
- daneel_w 5y ago
  Thanks for the assurance. The concern with "mail order co-lo" is of course that the customers themselves don't travel to the DC to install and power up the server.
KennyBlanken 5y ago
Not ExaMesh, but there are a variety of ways to encrypt the card and remotely unlock it. You can use dropbear to ssh into the Pi and provide the key during boot. You can have the Pi connect to a remote system to retrieve the key. Some methods are obviously not perfect, but it'll definitely make it more complicated than just "copy the card." Remote key retrieval would let you audit when the system was booted and so on. There's a more complex purpose-built open source software package specifically for handling remote disk unlocks but I'm blanking on the name and my google searches aren't turning it up. I vaguely recall it had fairly high levels of paranoia in its design.

mkj 5y ago

If it's BYO power brick you could also colo a second computer inside the power brick, using the rpi as a "modem" for it. Put a tamper resistant device inside that case.

systemvoltage 5y ago

EU Privacy means nothing, it's just marketing: https://news.ycombinator.com/item?id=29051305 https://news.ycombinator.com/item?id=29051305

drcongo 5y ago

Can't believe they didn't call it Picolo.

3xa 5y ago
Most domain extensions are already registered. And since we are from Germany, the term "Piccolo" sound very similar to "Picolo" but means something very different: https://de.wikipedia.org/wiki/Piccolo_(Sekt) https://de.wikipedia.org/wiki/Piccolo_(Sekt) However, we would have definitely pursued an overlap/commonality of the word with, say, beer!