5 ms·
Is it optimal for Codeberg to host on their own hardware? They write that it puts immense pressure on them: We transitioned from a basic stack (ext4 filesyst
by nequo 4y ago
Is it optimal for Codeberg to host on their own hardware? They write that it puts immense pressure on them:
We transitioned from a basic stack (ext4 filesystem, Postfix, not much more) to one including so many shiny new things: Starting with the duty of maintaining own hardware (which is a journey on its own), we introduced LXC containers, Ansible, BTRFS, ZFS, Ceph and more.
In the past months, I found myself reading a lot of documentation … Still, there was no time left to dig deep into Ansible and ZFS, and by now both tools have even been mostly dropped, to reduce the stress on our team members.
They write that self-hosting also leads head-on to the trade-off of what to spend scarce donor money on:
For donation-powered non-profits, this uses donations that could have served better, and leads to burn-out in the team.
The argument against cloud hosting is that the data then resides with a big hosting company:
No matter which instance you join or if you host yourself, the data will reside on one of the few big cloud provider's systems.
But Codeberg’s mission is to host public repositories of open source projects. Why is it a problem if the data sits on someone else’s hardware? Wouldn’t the mission be better served by focusing on “Codeberg the software” instead of “Codeberg the hardware?”
- jacooper 4y ago100%, you can't always reach your goal in the ideal way, so don't put up with unnecessary things that hinder your progress towards you end goal.
- latchkey 4y agoWe run petabytes of storage, and growing. We dropped Ceph (which is too complex with a minefield of issues, and few people have deep experience in it) and just use ZFS (which tons of people have experience in and performs flawlessly). We also use Ansible since it automates and codifies our deployments. I can't imagine dropping it... in favor of what? Manually typing install commands on servers? It is good they have lifted the covers on their service in this blog post. It screams "stay away".
- arter4 4y agoOut of curiosity, are you running ZFS across datacenters or do you have a single DC? My understanding is, ZFS doesn't inherently provide clustering, so each server has its own pool. If you have multiple DCs, how do you handle disaster recovery? Do you use async replication?
- latchkey 4y agoWe have multiple DC's although the data isn't replicated across DC's. We are a Filecoin storage provider. Users of that network choose where they want to store things and are effectively responsible for geo data replication, then using the protocol we can prove to them that their data is on our machines.
- phphphphp 4y ago> We run petabytes of storage, and growing. That can't be right -- can it? They have less than 50,000 users. For petabytes of storage to be necessary, their average per-user storage would need to be dozens of gigabytes. I've been using GitHub for more than a decade and the sum total of every repository I've published is less than that. I found a few repositories on Codeberg that are using gigabytes of storage but the majority are normal sized Git repositories.
- latchkey 4y agoI'm talking about my own business.
- scraptor 4y agoIf codeberg's mission only to host public repos with no further qualifiers they could just use github. The whole point is to not use services provided by untrustworthy and ideologically misaligned megacorps.
- nequo 4y agoI sympathize with that. I do see a point in-between those two extremes though. I don't want to give GitHub free advertising by hosting my code with them nor do I want to feed their ML models. The latter is not completely avoidable since someone could mirror my repo on GitHub or they could just scrape it from Codeberg. But the former is successfully avoided by using Codeberg, even if Codeberg is hosted in the cloud.