9 ms·
Cgit, Nginx and Gitolite: A Personal Git Server
- tgragnato 6y agoIf you do not want to mange a full featured web server, you can also replace nginx with haproxy. global master-worker program fcgiwrap command /usr/sbin/fcgiwrap -c <preforks> -f -s tcp:127.0.0.1:8080 user www-data group www-data frontend ... backend cgit use-fcgi-app cgit server fcgiwrap 127.0.0.1:8080 proto fcgi check fcgi-app cgit log-stderr global option keep-conn docroot <path> set-param SCRIPT_FILENAME /usr/lib/cgit/cgit.cgi set-param PATH_INFO %[path] set-param QUERY_STRING %[query] set-param HTTP_HOST %[hdr(host)]
- jrwr 6y agoHow is HAProxy any easier then nginx when it comes to proxying fcgi work?
- tgragnato 6y agoI'd say that they have a different set of features: use the software that suits best your needs. A default installation of NginX in Ubuntu does many things, which you need to administer and maintain. Haproxy is simpler because it's not a webserver and does fewer things.
- iveqy 6y agoThe big thing missing here is a good way to do code reviews and CI/CD. This might not be important for a single user but if you're collaborating with someone it can. For CI/CD you can get quite a long way with git hooks and for code review I would look into git appraise. that's the best one I've found but I would really like to hear if someone else has a better idea here!
- krageon 6y agoI wouldn't enjoy writing personal code without CI/CD if it grew beyond a few hundred lines. It's a lot more convenient to have a setup like that, to the point where I'd say it's essential.
- tgragnato 6y agoThe old school way is to notify the ci/cd server using a post-receive hook. You can do it, but it’s not integrated in cgit, it must be set from the command line when you initialize the repo
- Wilya 6y agoI used the exact same stack (gitolite+cgit) in the early stages of a previous startup, and code reviews were the big missing part that made us move to something more full featured (for us, gitlab). It's pretty easy to trigger ci runs via git hooks, and once you're used to it, checking their results in jenkins instead of in the git repository UI makes no difference. But code reviews really need a dedicated interface.
- arminiusreturns 6y agoI can second git hooks as a vastly underrated approach to automation. Lots of people who only use github don't know about them. Copy-pasta from an old comment of mine: "So, git hooks are basically executable scripts (of whatever language you have available, I use bash) placed in the .git/hooks dir, whcih are then executed at whichever event is designated (by the name of the script itself) For me, it's post-recieve. After initial setup, a push automatically triggers the post-recieve hook that does something like the following from my git dir: GIT_WORK_TREE=..../example.com git checkout -f and voila, pushes are insta-live, but commits can be pulled and worked on in the meantime. Learned from a few sources (shout out to Dreamhost for my original intro to this idea), but here are some relevant readings: https://www.ibm.com/developerworks/library/wa-git/ https://www.ibm.com/developerworks/library/wa-git/ https://opensource.com/life/16/8/how-construct-your-own-git- https://opensource.com/life/16/8/how-construct-your-own-git-... "
- ckpowell 6y agoIt is funny that the author mentions the security dilemma. I fall into the category of people who never get anything done because they worry about exploits all the time. If you also fall into that category, it is possible to provide read only ("dumb" in the official terminology) git access via http(s). I think that it is important that the primary git repository of projects is not on GitHub, so that may be a first step for projects where the main development happens on mailing lists. For CI, a GitHub mirror could be used that is updated with a cron job.
- deleted 6y ago[deleted]
- a012 6y agoI use Gitea for personal git server, they have a curated list of things can do with it: https://gitea.com/gitea/awesome-gitea https://gitea.com/gitea/awesome-gitea What I like about it is the old UI of Github, mean time to start from scratch is very fast and low maintenance.
- h_anna_h 6y agoPlease avoid gitea. I can't say anything about the other software but gitea is known for having accessibility issues, see https://github.com/go-gitea/gitea/issues/7057 https://github.com/go-gitea/gitea/issues/7057 Edit: the reason that I posted this was not due to some hypothetical blind people but rather because I have been in a situation where I want to show a cool project to a friend with vision issues but I can't because the project is hosted on a gitea instance. I posted this because I was unaware of this issue until recently and presumed that there would be others here who are also not aware of this.
- stonesweep 6y agoI tried out Gitea hosted via codeberg, and within a week disabled my account - Gitea allows usernames to be reused, there were bugs allowing people to push repos into other people's accounts (someone ran rampant on codeberg), you can view private repos via the Pages system and whatnot. In many of the issues, the current state is "it's a problem with Gitea" such as: https://codeberg.org/Codeberg/Community/issues/356 https://codeberg.org/Codeberg/Community/issues/356 HN readers are downvoting you, but Gitea's problems are real and they exist. Enough of them are listed in codeberg's issue pages that I felt the software/platform is very buggy and has a lot of security issues as just a generic user with one week of experience, much less an admin.
- kenniskrag 6y ago> there were bugs allowing people to push repos into other people's accounts (someone ran rampant on codeberg), you can view private repos via the Pages system and whatnot. Do you have a bug report? I'm a gitea user and want to reproduce it.
- Simplicitas 6y agoGood article. But yeah, I too opted for a Gitea's Dockerized setup (https://github.com/lencap/lux https://github.com/lencap/lux). Looking to secure it better.
- aigoota 6y agoI used to use a personal git server but the possible security issues and/or making the entire server vulnerable scared me away from it. Instead, I simply use bare git repos on the server and the server is only accessible via ssh/private key. I can simply fetch/push my repos directly. There's really no need for a frontend...
- moehm 6y agoYou interact with Gitolite via ssh. The things Gitolite makes easier for you is it generate those repos for you (no need to login, just push a config file) and it handles access via ssh keys. I for instance have an own key for my phone, so I can access the repo with my notes etc, but if you steal my phone you cannot push any code.
- brobdingnagians 6y agoI do the same. There are plenty of tools for local analysis and visualization of git repos; I finder it better to use those as appropriate and just use bare repos cloned from the server.
- alexhutcheson 6y agoAnother option is to run a web GUI like cgit, gitweb, Gitea, or Gogs without exposing it to the internet (so it's only accessible at http://localhost:8888 http://localhost:8888 or similar). When you want to view it from another machine you can just use SSH port forwarding, like: ssh -L 8888:localhost:8888 user@hostname
- deleted 6y ago[deleted]
- mdtrooper 6y agoI think is more easy https://fossil-scm.org https://fossil-scm.org and a cloud such as nextcloud or google drive. Yes, it is not git, but it is not bad.
- diego_moita 6y agoPersonally, I think fossil is far better than git. It offers you a lot more than git: wiki, ticket system, chat, ...
- moehm 6y agoI just want to note I once run this stack on an old Raspberry Pi 2, and it was still really, really fast. Also it's very lightweight. My current setup runs on a box with 256 mb RAM and for a lot of small git repos it's fine. (Larger ones get OOM reaped.)
- LeSaucy 6y agoThis is great to use old-school tools. What really rubs me the wrong way is how manual it is to setup/configure/deploy. Things like this run great for years until they don't, and then you are left walking through the entire setup process again to debug things.
- flowerlad 6y agoMy personal git server is just a shared folder on a Raspberry Pi running Ubuntu. This is on my intranet, so I don't worry about someone else accessing or checking in code. The server runs Samba, that's it! Git does not require a server for simple scenarios such as mine. The "remote" can be a folder, and the folder can be on be on your local box, or it can be a mounted remote folder. I am even able to kick off Jenkins job on the Raspberry Pi when I push.
- stonesweep 6y agoPlease do not follow this blog's advice of compiling your own cgit with a git pull off master; the blog author is now (a) using some random commit point (untaggged release) which contains who knows what, and (b) has to now manually upgrade cgit for the life of his server. At this exact moment there are 13 commits on master ahead of the latest tag, who knows what that might be. https://git.zx2c4.com/cgit/log/ https://git.zx2c4.com/cgit/log/ Find a repo (apt/PPA, dnf/EPEL, AUR, whatever works for you) and use a repo to manage your server software unless you have no other choice than to hand-compile it. Hand-compiling server daemons like this means they never get upgraded with bugfixes and CVE issues regularly - this isn't 1995, we did this back then because we had to do it. It's not a good idea in 2021 to be doing this unless you're going to hand-manage that cgit binary every day of your life going forward, this is an internet facing endpoint which will get people probing it to attack (assuming you used a cloud server). If nothing else, clone off the last tagged release by the upstream authors not HEAD.
- mbreese 6y agoI’m not going to disagree with you on the usage of package managers. No only is it easier to install, but easier to maintain. But, isn’t cgit basically read-only? If so, that doesn’t seem to be particularly risky. All of the main interactions with gitolite are through SSH, which I’d expect to be the main area of attack.
- drdec 6y ago> But, isn’t cgit basically read-only? So are SELECT statements until little Bobby Tables shows up. Which is to say that SQL injection can be used to turn read-only functionality into write functionality. Secondly, if you care about who has access to the data, even read-only vulnerabilities can be damaging.
- mbreese 6y agoTrue, but exploiting cgit would be significantly more difficult than a SQL injection. I could only find a handful of CVEs for cgit, and the last one was in 2018 (v 1.2.1)[1]. It involved reading files outside the git tree. That said, it's not the common software to see publicly, so probably hasn't been thoroughly vetted. But I think the point still stands that installing from source isn't going to be a horrible security risk. Similarly though -- It is a mature project without a lot of new features. I don't think there are killer features in master that would make me want to install from source over `apt install cgit` [1] https://www.cvedetails.com/vulnerability-list/vendor_id-15775/product_id-33075/Cgit-Project-Cgit.html https://www.cvedetails.com/vulnerability-list/vendor_id-1577...
- kenniskrag 6y agoIs cgit read only? Or do you have to restrict access (e.g. push)
- nirui 6y agoSince we are recommending personal Git servers at this point ... if you want a really really lightweight Git repo server without web interface support, maybe checkout gitdir (https://github.com/belak/gitdir https://github.com/belak/gitdir) There is another project called Sorcia (https://sorcia.org https://sorcia.org, https://news.ycombinator.com/item?id=22685914 https://news.ycombinator.com/item?id=22685914) which is also very lightweight and includes web interface. On the security side, you can put them in Docker and apply all necessary measures there. PS: You can also run SSHd and git in a Docker container.
- codetrotter 6y agoSorcia website doesn’t load for me
- nirui 6y agoYeah that website is kind flicking sometimes, maybe try again later ;)
- xaduha 6y agoAll you need to run a 'personal git server' is sshd and a user. If you plan to have many others to use it, then it's not really personal, is it? > Even though we've turned off password based authentication in a previous section, we will still receive a significant amount of bots wasting our compute cycles trying to login. I think that's proven to be false.
- nanidin 6y agoI also came to chime in with this. I have been running a "personal git server" this way for almost a decade. On the server: cd ~ mkdir directoryname.git cd directoryname.git git init --bare On the client(s): git clone username@server:directoryname.git That's all there is to it!
- shuntress 6y ago>All you need to run a 'personal git server' is sshd To expand on this slightly: making your "personal" git server accessible to collaborators (without needing to manage user accounts user accounts) is also very simple. You can statically serve your repository using any web server. Collaboration is then based on each contributor pushing (ssh) to their personal server from which everyone else may pull (http).
- pokstad 6y agorsync.net is great for this: https://blog.kozubik.com/john_kozubik/2010/02/git-and-subversion-support-at-rsyncnet.html https://blog.kozubik.com/john_kozubik/2010/02/git-and-subver...
- anderspitman 6y agoI dream of the day when the internet is decentralized again (either by ipv6 or tunnel proxies[0]) and we can just push/pull directly to each other. [0]: https://github.com/anderspitman/awesome-tunneling https://github.com/anderspitman/awesome-tunneling
- warmwaffles 6y agoOr host your own sourcehut https://sourcehut.org/ https://sourcehut.org/
- waffletower 6y agoKeeping running fail2ban, but I also recommend using a non-standard SSH port. When port 22 is closed, bots are stopped at the firewall. git urls can include custom ports and you can also use bash aliases to manage the difference.
- tyingq 6y agoFor anyone that has issues with fail2ban, I'm a fan of pam_shield. It's a pam module, so it's intercepting live requests rather than tailing logs, which seems cleaner to me. For debian, it was a relatively simple apt-get type install (libpam-shield). But, yeah, changing the default port surely kills off the noise.
- ed25519FUUU 6y agoGogs is probably the most painless way to do this, especially if you’re setting up git on an existing host and don’t want to change or install a bunch of stuff. https://gogs.io/ https://gogs.io/
- a20eac1d 6y agoThe one thing that gives me hypertension when self hosting a code repository is backups. The hard drive of your server can fail at any time and when self hosting you are responsible for your backups. This is giving me night terrors, especially when its on a cloud server and I don't have access to the hardware. Currently, I'm running a cron task once per day executing a simple backup script that does the following: Stop the Gitea container, copy the entire Gitea directory (including the docker-compose.yml and the data directory) to a backup folder, restart the container, sync that folder to a Backblaze bucket, delete the backup folder. Restoring the backup is (should) be as easy as downloading the bucket from Backblaze and simply docker-composing it up. I'm looking for other ideas advice that will help me sleep at night. Thanks!
- hellcow 6y agoDo you not have all your code locally as well? It seems like if your drive failed for a personal git host, you could just re-push everything back up to it.
- LinuxBender 6y agoFor what its worth, another thing you might consider looking into is using RSnapshot [0]. RSnapshot helps me sleep at night. This will create multiple directories that are hard links to files that did not change, thus saving disk space and giving you multiple days or weeks of snapshots, in the even something was corrupted and you want to roll back at the filesystem level. Most questions one could come up about rsnapshot with are answered on Serverfault [1] There are many how-to sites [2] with usage examples. You can create snapshots locally or remote. On mac you can brew install rsnapshot. [0] - https://rsnapshot.org/ https://rsnapshot.org/ [1] - https://serverfault.com https://serverfault.com [2] - https://linuxconfig.org/guide-to-rsnapshot-and-incremental-backups-on-linux https://linuxconfig.org/guide-to-rsnapshot-and-incremental-b...
- hnlmorg 6y agoSnapshots aren't a backup. However they do obviously have their merits too. Personally I'd recommend ZFS over RSnapshot and then make use of raidz so you have redundancy at the hardware level as well as at the snapshots.
- alexhutcheson 6y agoIf you just want to browse some repos via a read-only web interface, you might also want to check out gitweb[1] and the wrapper script git-instaweb[2], which are bundled with git. [1] https://git-scm.com/docs/gitweb https://git-scm.com/docs/gitweb [2] https://git-scm.com/docs/git-instaweb https://git-scm.com/docs/git-instaweb