7 ms·
Github is becoming too important. A routine backup system is needed, something like archive.org, that backs up important Github projects.
by Animats 3y ago
Github is becoming too important. A routine backup system is needed, something like archive.org, that backs up important Github projects.
- shepherdjerred 3y agoI assume that there are some archives of public GitHub repositories somewhere.
- rizky05 3y ago[dead]
- hipadev23 3y agoThat exists on every dev’s machine.
- Animats 3y agoIn theory, the code is there, but putting a project back together in a hurry after trouble at Github is non-trivial. Especially if the build process uses proprietary stuff such as "GitHub actions". The issues and discussions are all Github-only, too.
- jrockway 3y agoThe source code of the actions is in your repo. This is more of the problem of relying on proprietary software; if the company that makes it dies, you die too!
- misnome 3y agoIt’s crazy that there is no common CI spec. Every single platform is completely different. Last time I tried Act it had some pretty severe limitations. Perhaps I should take it for another spin.
- hotnfresh 3y agoThe common CI spec is sh & make, but everybody hates those I guess. Git hook management's really awkward. Even with tools to synchronize them (... all of them? You may want some that are just for your own use) it's a pain. "I want these hooks to run, in order, but only when a merge commit happens on a machine with such-and-such designation, and I want it to run the task on a different machine, but we need to make sure that runner's a Windows box because..." that just sucks to self-manage, and yeah, there's no standard for expressing that, you're bound to incompatible solutions. Secret management's a hellscape and everyone's always glad when someone else solves the problem for you. That alone is like 50% of the value of Github Actions.
- crabbone 3y ago> sh & make Are we talking GNU Make, nmake, BSD Make? (Isn't this why autotools exist in the first place -- to make something allegedly cross-platform for different flavors of Make)? I get bitten repeatedly by sh being a symlink to different Shells, even though I've been using it for many years. The most recent piece of insanity being "cd foo bar" resulting in an error, but changing to foo directory while in some other version simply resulting in an error. Also, error reporting. It's way too broken to consider sh a reliable tool. I wish things could be done with very simple tools and all this complexity around them was unnecessary. Unfortunately, here, this complexity while isn't unavoidable is indeed warranted due to abysmal quality of the simple tools.
- misnome 3y ago> The common CI spec is sh & make, but everybody hates those I guess. These aren't very good at some of the things you actually want to use a CI for, other than the literal "build" step (which probably is using them anyway, or the per-language equivalent). Coordinating _and visualising_ multiple builds where some parts run in parallel (e.g. multiplatform), or parts that might be different or skipped depending on how the flow was triggered. Managing caches and artifacts between stages and between runs. Managing running in different contexts, including generic shell and across multiple machine types. A consistent way to manage and inject secrets. Reusing common parts across different CI workflows. I suppose you could have a parent job taking up a full slot to run a Makefile that launches and manages the running of jobs on other nodes, but imagine you'd have to step into some toolset that abstracts some of it, and hope that is shared development or you end up with an in-house convoluted nightmare of shell scripts. "Something DAG and yaml shaped" is about the closest convergence we have gotten, and the closest that it looks like we'll get.
- antoineMoPa 3y agoIf only there was a decentralized system to allow version management of code.
- fragmede 3y agoYou'd still need some sort of centralized... hub to communicate and manage things.
- rglullis 3y agoLike the internet itself?
- uncomputation 3y agoThe internet is not centralized. It is literally the largest decentralized network in existence. A network of networks with no central hub.
- agilob 3y agogit on ipfs and issues stored in a literally plaintext file
- asdff 3y agoWhy do you need a centralized hub? Most repos on github are entirely unrelated to eachother. Repos can be self hosted. Issues can be tracked on your own website. You don't need a megacorp to run a website. If you are writing open source tooling you are more than qualified to roll your own here.
- paulddraper 3y agoMy hard drive.
- bogota 3y agoHost a backup of your own code? It’s easy and can be done on a rpi. I wrote a go program in 1000 lines that automatically does this for me. And then I actually started using that as the main source and pushing the backup to github. It also pulls down anything i star into a different folder which get a sync one a day. The rest get a sync every hour.
- 369548684892826 3y agoThere's also a tool called myrepos[0] that can be cronjobbed to keep local repos up to date 0: https://myrepos.branchable.com/ https://myrepos.branchable.com/
- bogota 3y agoI like mine better because i wrote it :)
- swatcoder 3y agoI know this a greybeard's fantasy and that most people working today were trained not to bother, but: important things should not have GitHub as a failure point. Hobby projects and today's work? Sure. Point straight at GitHub and hack away. And when it goes down, get yourself a coffee. But everything that's anywhere near production should have already pointed those github requests to a mirror or other tool in your own controlled ecosystem. The status of GitHub should have nothing to do with whether you're customers are getting what they're paying you for. Same goes for Docker containers and every other kind of remotely distributed dependency.
- willsmith72 3y agoIt's not that black and white. Where do you draw the line on what can and can't be a failure point? My cloud provider is probably an acceptable point. If every AWS region goes down I'm not going to have a spare cloud provider. What about an auth provider? Do I need a backup there? What about CI, do I need multiple CI systems? 3rd party search services, realtime messaging services, the list goes on. For 1% of systems, you need backups for all of these (or to not use anything external). The other 99%, building backups for every one of these systems is a losing business strategy. Some of them sure, but which those are will vary based on the context. It's not as simple as having a backup for "every other kind of remotely distributed dependency."
- GrinningFool 3y agoThat's all besides the point isn't it? Everything you mentioned can be replaced. Your source code can't.
- wongarsu 3y agoIt depends on the Recovery Time Objective we are talking about. For example at our company there are daily backups of both our dependencies and those used in the build process. If github had a prolonged outage or accidentally deleted all their data we could set up a gitea with all our code within hours, and get a replacement for Github Actions working within days. But that takes longer than the expected duration of this outage, and is a lot of work. It's not like we have a standby gitea we can just seamlessly switch to, so we are still hit by this outage. On the other hand for build dependencies we do have a standby mirror.
- ajsfoux234 3y agohttps://archive.softwareheritage.org/browse/ https://archive.softwareheritage.org/browse/ has some repositories archived.
- lmarcos 3y agoIn the context of companies: if you don't have your own cache of github,npm, etc., you're doing it wrong.