6 ms·

How Git servers work, and how to keep yours secure

167 points by danielg0 6y ago

amelius 6y ago
You can also install git-shell, which is a login-shell which can be accessed through SSH and provides access restricted to Git commands.
lobo_tuerto 6y ago
If interested in self hosting your git repos have a look at gitolite: https://gitolite.com/gitolite/index.html https://gitolite.com/gitolite/index.html
- fanf2 6y ago
  Gitolite is good if you want a locked-down git server with sophisticated access control Gitolite provides a command-line UI only; you can use it with gitweb or cgit to allow people to view repositories in a web browser. No issues or pull requests or fancy stuff like that!
  - 1337shadow 6y ago
    gitolite is pretty fancy but depends on a text based configuration, and it's easy to make your own git-shell which is basically a shell that will allow certain commands, and assign each ssh key to a git-shell with arguments such as a username if you want to bind to a db or something
    - mbreese 6y ago
      > basically a shell that will allow certain commands, and assign each ssh key to a git-shell with arguments such as a username Well, to be fair, you don’t need gitolite for that. You can just do that with any account and the right .ssh/authorized_keys file. prgmr.com uses/used(?) this for out of band access to VMs for example.
- yuters 6y ago
  Oh man, I remember 10 years ago when Gitlab was a very modest UI on top of this.
urda 6y ago
Does anyone host something like "gogs" in the cloud for themselves ?
- Sebb767 6y ago
  We use Gitea (a Gogs fork) for our company and it works great :)
  - 1337shadow 6y ago
    What CI runner ?
    - gnabgib 6y ago
      Not OP, but we use Drone[https://www.drone.io/ https://www.drone.io/] with Gitea[https://gitea.io/en-us/ https://gitea.io/en-us/]
      - Sebb767 6y ago
        We actually use drone, too! It has its quirks (for example, it would be nice to be able to start jobs via the web interface), but it works well for us.
chungy 6y ago
The article makes the presumptions that one would be running Arch on a server and that you must be using nginx as a web server. The latter half of that is particularly striking to me, given that he immediately dives into a shortcoming of nginx... rather than reaching for Apache, he works around nginx's shortcoming.
- moistbar 6y ago
  Arch also doesn't make for a particularly good server OS.
  - megous 6y ago
    Why? (Asking as someone running Arch on many servers for the last 5 years, and debian server for much longer) I can think of a few things I dislike, but all of the failures so far were self-inflicted. Like forgetting to update packages that I self compiled outside of supported repository. It's not completely autonomous when it comes to upgrades, you have to think for a while before hitting "yes" after seeing the list of packages to update, but so is not Debian in the long run... (some of the servers I have to manage are 13 years old or so, and going through major dist-upgrades is never that pleasant either. It's a bit more hassle, because I don't trust the major version upgrade so I have to run them on a backup VM first just to see whether some issues will crop up). But having the latest versions of the programs is great, I don't have to second guess myself when writing new programs (will it be compatible?), can use the latest kernel APIs, etc.
    - moistbar 6y ago
      This is all personal experience, so obvious salt is required with it. I've had bizarre networking bugs pop up on arch that I've not had elsewhere. I also just don't like rolling releases as much as I used to. Since the majority of what I do can be containerized, I prefer a much slower release cadence for my hosts, and anything that requires more up to date packages just gets thrown in a container. Basically, I think my server workflow just doesn't line up with how arch works. On desktops it's great, since I'll always have up to date video drivers, desktop environments, etc, but on a server I don't usually use things that require the latest and greatest software. I figure as long as it's still getting bug fixes, I'm probably fine.
neilv 6y ago
Is there a rock-solid git server that I can use on a home server for versioned immutable backups of misc. files on personal devices (e.g., account config), as well as private software development git repos? (I've done a cheaper version of this -- except for the immutable part, and the separation of accounts between devices -- in the past using SSH+SVN to a home server, and it was great.) I was thinking immutable from the perspective of a device. A given device can pull branches of certain repos, and make commits to the branches. But a device's user account on the git server doesn't have permission to affect past commits. So, for example, if my dodgy Linux smartphone is compromised, a hypothetical person who isn't being nice can't do anything to my backups, other than make bogus additional commits. Maybe each device has its own branch (e.g., `big-laptop`, `little-laptop`, `smartphone`, `media-server`), where they can commit their changes, and maybethey can pull from main/trunk. And then the physical console for the git server lets me inspect and merge changes from the different devices, so that other devices can pick up those changes. I thought about starting with Gitlab CE, but that's pretty big, so, even if the features could be made to do what I want, I don't know whether I'd always be running too many vulnerabilities that defeat some of my purposes.
- ghostbrainalpha 6y ago
  Is there any reason your example would ever happen? That seems a little far fetched of a security concern to me.
  - neilv 6y ago
    It absolutely happens for some organizations. There's industries around it. And anyway, seems like good practice, and shouldn't be hard to do, and should fit with workflows already familiar from work. What would be bad practice is to give less-trusted devices (e.g., Linux development phones, or some disposable PC on which I had to install some sketchy software) access to all my files and backups. Using git this way might be a simple (given we have to know git anyway) way to give the goodness of backups and selectively syncing various kinds of files both ways with the less-trusted devices.
    - 6y ago
elevation 6y ago
I like self hosting git but these tutorials set you up with only a one-machine solution. I'd like to be able to self-host a git service that's robust in the face of network/hardware/OS maintenance. I know git is distributed by design. So if I want to push code to a pair of servers for better availability, I can do it explicitly: git push <remote1> <branch> git push <remote2> <branch> But what if I wanted to make this transparent but still highly available, such that the remote URL in git push <remote> <branch> is actually backed by a HA cluster? Some of the software and ops to make this happen is Github's secret sauce. I'm not looking to compete with them, but would love an open source solution that had a better uptime than a single digital ocean droplet running debian. Ideally, I could get there without green-fielding raft consensus shims into a modified git binary.
- Ajedi32 6y ago
  git remote set-url --add origin $second_url Not quite HA cluster levels of redundancy, but it's also way simpler to set up.
  - lucb1e 6y ago
    I actually have this at work for a repo where we want to both push to the server where it'll be deployed (so this server doesn't need some sort of access to another system, trust relations going only one way) as well as to the source repo where we actually collaborate and have an issue tracker etc. Works great for us. Yes, of course you can setup some limited access for this server to pull from the original source but... small team with high security requirements, so this low-tech solution is quite perfect.
- 3np 6y ago
  Plenty of alternative approaches that can get you there. Here’s one: * distributed file system like gluster or ceph for repos * clustered db (eg replicated Postgres) * redundant instances of gitea * load balancing Am I missing something?
  - elevation 6y ago
epja 6y ago
What would be the best practice way of allowing git pushes similar to how heroku handles them? I'm wanting to implement a feature that allows users to push to deploy new themes.
- rovr138 6y ago
  Push to an intermediary location. Use post-receive hook to trigger whatever actions you need. At the end, push that to the server and reload whatever you need. https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_post_receive https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_po...
jeffmk 6y ago
Gitea[0] is a pleasant, easy-to-set-up 'GitHub lite', if you are looking for something more turnkey and less custom: - users/organizations - issues - PRs - milestones - releases - wikis - activity/contrib graphs I have it running on a $5/mo Linode instance for some of my personal projects. [0] https://gitea.io/ https://gitea.io/
- acidburnNSA 6y ago
  Is the code review process roughly on par with the github experience? I can't find many examples of a PR with inline comments, etc.
- redwall_hp 6y ago
  It supports LFS too. I use it for my Unity hobby projects, because I'm already paying for my VPS (which has tens of gigabytes of space), so paying GitHub for extra space would be silly.
mnutt 6y ago
I enjoyed reading the source for https://github.com/c4milo/gitd https://github.com/c4milo/gitd, also in go, as a way to simply wrap the go utilities, add your own auth, and get heroku-like deployments.
stevenhuang 6y ago
For those trying to host a git server on your home network like I have and encountered network issues even after port forwarding, you might be behind carrier grade NAT. You'll need some forwarding solution, or use an SSH reverse tunnel to punch through CGNat. Use something like ngrok or localhost.run. For example if you're using gitea, host it on localhost:8080 then run this: ssh -R 80:localhost:8080 localhost.run This will forward localhost:8080 to <subdomain>.localhost.run Now you just pass <subdomain>.localhost.run to colleagues and they'd be able to connect to your self-hosted git instance. Now do the same for ssh port. Caveat that your traffic would route through localhost.run, so it's best to not use this for anything serious, or alternatively host your own reverse SSH tunnel on a VPS somewhere.
- xomodo 6y ago
  How could one selfhost localhost.run setup on own wildcard domaine?
  - rovr138 6y ago
    If you're talking about, > alternatively host your own reverse SSH tunnel on a VPS somewhere. To make a quick version, on a VPS or somewhere, install OpenSSH server. Modify your sshd.conf file adding, GatewayPorts yes Then you can use something like this, ssh -R 8080:localhost:22 user@server.example.com After that, you can use, ssh user@server.example.com -p 8080 from any other computer and it will connect you to the machine you ran the ssh -R from.
- cookiengineer 6y ago
  In addition to this: Carrier grade NAT ISPs sometimes have global scope ipv6s assigned, and if the other endpoint has ipv6 support, too, you can breakout easily using the assigned ipv6. Rather than that I would recommend reading up on DNS exfiltration techniques [1] and things like pwnat [2] that use faked SNMP reply packets that make routers think they forgot to let a data packet through for hop traces. And if you have the time, I'd recommend to use websockets as a tunneling protocol because it's very flexible in its payload size and allows compressions via websocket extensions and the srv flags. I wrote a detailed article that explains the WS13 protocol and all its quirks [3] Additionally to that it's good to know the limitations of a SOCKS proxy, hence that's what most "easy to use" implementations provide. Spoiler: forget ipv6 via socks5 proxies. I also wrote a detailed article about its quirks [4] I'm currently experimenting with the idea of a DNS protocol implementation that uses multicast DNS service discovery to find local peers and that uses DNS exfiltration techniques to breakout of a CGNAT, but I'm not there yet to write a detailed article about it. It's current research for my stealth browser project. [1] https://blogs.akamai.com/2017/09/introduction-to-dns-data-exfiltration.html https://blogs.akamai.com/2017/09/introduction-to-dns-data-ex... [2] https://github.com/samyk/pwnat https://github.com/samyk/pwnat [3] https://cookie.engineer/weblog/articles/implementers-guide-to-websockets.html https://cookie.engineer/weblog/articles/implementers-guide-t... [4] https://cookie.engineer/weblog/articles/implementers-guide-to-socks.html https://cookie.engineer/weblog/articles/implementers-guide-t...

167 points by danielg0 6y ago

You can also install git-shell, which is a login-shell which can be accessed through SSH and provides access restricted to Git commands.

lobo_tuerto 6y ago

If interested in self hosting your git repos have a look at gitolite: https://gitolite.com/gitolite/index.html https://gitolite.com/gitolite/index.html

fanf2 6y ago
Gitolite is good if you want a locked-down git server with sophisticated access control Gitolite provides a command-line UI only; you can use it with gitweb or cgit to allow people to view repositories in a web browser. No issues or pull requests or fancy stuff like that!
- 1337shadow 6y ago
  gitolite is pretty fancy but depends on a text based configuration, and it's easy to make your own git-shell which is basically a shell that will allow certain commands, and assign each ssh key to a git-shell with arguments such as a username if you want to bind to a db or something
  - mbreese 6y ago
    > basically a shell that will allow certain commands, and assign each ssh key to a git-shell with arguments such as a username Well, to be fair, you don’t need gitolite for that. You can just do that with any account and the right .ssh/authorized_keys file. prgmr.com uses/used(?) this for out of band access to VMs for example.
yuters 6y ago
Oh man, I remember 10 years ago when Gitlab was a very modest UI on top of this.

urda 6y ago

Does anyone host something like "gogs" in the cloud for themselves ?

Sebb767 6y ago
We use Gitea (a Gogs fork) for our company and it works great :)
- 1337shadow 6y ago
  What CI runner ?
  - gnabgib 6y ago
    Not OP, but we use Drone[https://www.drone.io/ https://www.drone.io/] with Gitea[https://gitea.io/en-us/ https://gitea.io/en-us/]
    - Sebb767 6y ago
      We actually use drone, too! It has its quirks (for example, it would be nice to be able to start jobs via the web interface), but it works well for us.

chungy 6y ago

The article makes the presumptions that one would be running Arch on a server and that you must be using nginx as a web server. The latter half of that is particularly striking to me, given that he immediately dives into a shortcoming of nginx... rather than reaching for Apache, he works around nginx's shortcoming.

moistbar 6y ago
Arch also doesn't make for a particularly good server OS.
- megous 6y ago
  Why? (Asking as someone running Arch on many servers for the last 5 years, and debian server for much longer) I can think of a few things I dislike, but all of the failures so far were self-inflicted. Like forgetting to update packages that I self compiled outside of supported repository. It's not completely autonomous when it comes to upgrades, you have to think for a while before hitting "yes" after seeing the list of packages to update, but so is not Debian in the long run... (some of the servers I have to manage are 13 years old or so, and going through major dist-upgrades is never that pleasant either. It's a bit more hassle, because I don't trust the major version upgrade so I have to run them on a backup VM first just to see whether some issues will crop up). But having the latest versions of the programs is great, I don't have to second guess myself when writing new programs (will it be compatible?), can use the latest kernel APIs, etc.
  - moistbar 6y ago
    This is all personal experience, so obvious salt is required with it. I've had bizarre networking bugs pop up on arch that I've not had elsewhere. I also just don't like rolling releases as much as I used to. Since the majority of what I do can be containerized, I prefer a much slower release cadence for my hosts, and anything that requires more up to date packages just gets thrown in a container. Basically, I think my server workflow just doesn't line up with how arch works. On desktops it's great, since I'll always have up to date video drivers, desktop environments, etc, but on a server I don't usually use things that require the latest and greatest software. I figure as long as it's still getting bug fixes, I'm probably fine.

neilv 6y ago

Is there a rock-solid git server that I can use on a home server for versioned immutable backups of misc. files on personal devices (e.g., account config), as well as private software development git repos? (I've done a cheaper version of this -- except for the immutable part, and the separation of accounts between devices -- in the past using SSH+SVN to a home server, and it was great.) I was thinking immutable from the perspective of a device. A given device can pull branches of certain repos, and make commits to the branches. But a device's user account on the git server doesn't have permission to affect past commits. So, for example, if my dodgy Linux smartphone is compromised, a hypothetical person who isn't being nice can't do anything to my backups, other than make bogus additional commits. Maybe each device has its own branch (e.g., `big-laptop`, `little-laptop`, `smartphone`, `media-server`), where they can commit their changes, and maybethey can pull from main/trunk. And then the physical console for the git server lets me inspect and merge changes from the different devices, so that other devices can pick up those changes. I thought about starting with Gitlab CE, but that's pretty big, so, even if the features could be made to do what I want, I don't know whether I'd always be running too many vulnerabilities that defeat some of my purposes.

ghostbrainalpha 6y ago
Is there any reason your example would ever happen? That seems a little far fetched of a security concern to me.
- neilv 6y ago
  It absolutely happens for some organizations. There's industries around it. And anyway, seems like good practice, and shouldn't be hard to do, and should fit with workflows already familiar from work. What would be bad practice is to give less-trusted devices (e.g., Linux development phones, or some disposable PC on which I had to install some sketchy software) access to all my files and backups. Using git this way might be a simple (given we have to know git anyway) way to give the goodness of backups and selectively syncing various kinds of files both ways with the less-trusted devices.
  - 6y ago

elevation 6y ago

I like self hosting git but these tutorials set you up with only a one-machine solution. I'd like to be able to self-host a git service that's robust in the face of network/hardware/OS maintenance. I know git is distributed by design. So if I want to push code to a pair of servers for better availability, I can do it explicitly: git push <remote1> <branch> git push <remote2> <branch> But what if I wanted to make this transparent but still highly available, such that the remote URL in git push <remote> <branch> is actually backed by a HA cluster? Some of the software and ops to make this happen is Github's secret sauce. I'm not looking to compete with them, but would love an open source solution that had a better uptime than a single digital ocean droplet running debian. Ideally, I could get there without green-fielding raft consensus shims into a modified git binary.

Ajedi32 6y ago
git remote set-url --add origin $second_url Not quite HA cluster levels of redundancy, but it's also way simpler to set up.
- lucb1e 6y ago
  I actually have this at work for a repo where we want to both push to the server where it'll be deployed (so this server doesn't need some sort of access to another system, trust relations going only one way) as well as to the source repo where we actually collaborate and have an issue tracker etc. Works great for us. Yes, of course you can setup some limited access for this server to pull from the original source but... small team with high security requirements, so this low-tech solution is quite perfect.
3np 6y ago
Plenty of alternative approaches that can get you there. Here’s one: * distributed file system like gluster or ceph for repos * clustered db (eg replicated Postgres) * redundant instances of gitea * load balancing Am I missing something?
- elevation 6y ago

epja 6y ago

What would be the best practice way of allowing git pushes similar to how heroku handles them? I'm wanting to implement a feature that allows users to push to deploy new themes.

rovr138 6y ago
Push to an intermediary location. Use post-receive hook to trigger whatever actions you need. At the end, push that to the server and reload whatever you need. https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_post_receive https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_po...

jeffmk 6y ago

Gitea[0] is a pleasant, easy-to-set-up 'GitHub lite', if you are looking for something more turnkey and less custom: - users/organizations - issues - PRs - milestones - releases - wikis - activity/contrib graphs I have it running on a $5/mo Linode instance for some of my personal projects. [0] https://gitea.io/ https://gitea.io/

acidburnNSA 6y ago
Is the code review process roughly on par with the github experience? I can't find many examples of a PR with inline comments, etc.
redwall_hp 6y ago
It supports LFS too. I use it for my Unity hobby projects, because I'm already paying for my VPS (which has tens of gigabytes of space), so paying GitHub for extra space would be silly.

mnutt 6y ago

I enjoyed reading the source for https://github.com/c4milo/gitd https://github.com/c4milo/gitd, also in go, as a way to simply wrap the go utilities, add your own auth, and get heroku-like deployments.

stevenhuang 6y ago

For those trying to host a git server on your home network like I have and encountered network issues even after port forwarding, you might be behind carrier grade NAT. You'll need some forwarding solution, or use an SSH reverse tunnel to punch through CGNat. Use something like ngrok or localhost.run. For example if you're using gitea, host it on localhost:8080 then run this: ssh -R 80:localhost:8080 localhost.run This will forward localhost:8080 to <subdomain>.localhost.run Now you just pass <subdomain>.localhost.run to colleagues and they'd be able to connect to your self-hosted git instance. Now do the same for ssh port. Caveat that your traffic would route through localhost.run, so it's best to not use this for anything serious, or alternatively host your own reverse SSH tunnel on a VPS somewhere.

xomodo 6y ago
How could one selfhost localhost.run setup on own wildcard domaine?
- rovr138 6y ago
  If you're talking about, > alternatively host your own reverse SSH tunnel on a VPS somewhere. To make a quick version, on a VPS or somewhere, install OpenSSH server. Modify your sshd.conf file adding, GatewayPorts yes Then you can use something like this, ssh -R 8080:localhost:22 user@server.example.com After that, you can use, ssh user@server.example.com -p 8080 from any other computer and it will connect you to the machine you ran the ssh -R from.
cookiengineer 6y ago
In addition to this: Carrier grade NAT ISPs sometimes have global scope ipv6s assigned, and if the other endpoint has ipv6 support, too, you can breakout easily using the assigned ipv6. Rather than that I would recommend reading up on DNS exfiltration techniques [1] and things like pwnat [2] that use faked SNMP reply packets that make routers think they forgot to let a data packet through for hop traces. And if you have the time, I'd recommend to use websockets as a tunneling protocol because it's very flexible in its payload size and allows compressions via websocket extensions and the srv flags. I wrote a detailed article that explains the WS13 protocol and all its quirks [3] Additionally to that it's good to know the limitations of a SOCKS proxy, hence that's what most "easy to use" implementations provide. Spoiler: forget ipv6 via socks5 proxies. I also wrote a detailed article about its quirks [4] I'm currently experimenting with the idea of a DNS protocol implementation that uses multicast DNS service discovery to find local peers and that uses DNS exfiltration techniques to breakout of a CGNAT, but I'm not there yet to write a detailed article about it. It's current research for my stealth browser project. [1] https://blogs.akamai.com/2017/09/introduction-to-dns-data-exfiltration.html https://blogs.akamai.com/2017/09/introduction-to-dns-data-ex... [2] https://github.com/samyk/pwnat https://github.com/samyk/pwnat [3] https://cookie.engineer/weblog/articles/implementers-guide-to-websockets.html https://cookie.engineer/weblog/articles/implementers-guide-t... [4] https://cookie.engineer/weblog/articles/implementers-guide-to-socks.html https://cookie.engineer/weblog/articles/implementers-guide-t...