4 ms·

- We would continue to use our existing cloud hosted Chef server for now. - We want to use Kubernetes instead of virtualization. - We provisioned 2 spare data

by sytse 10y ago

- We would continue to use our existing cloud hosted Chef server for now. - We want to use Kubernetes instead of virtualization. - We provisioned 2 spare database servers for that reasons. - Git already compresses files with zlib so I'm not sure we can compress it much further https://git-scm.com/book/uz/v2/Git-Internals-Packfiles https://git-scm.com/book/uz/v2/Git-Internals-Packfiles - The servers have a separate management port and "Apart from those routers we'll have a separate router for a 1Gbps management network." - I agree we'll probably need more spare servers.

viraptor 10y ago
> The servers have a separate management port and "Apart from those routers we'll have a separate router for a 1Gbps management network." It was followed with "For example to make STONITH reliable when there is a lot of traffic on the normal network". That sounded like you want the heartbeats on it, not out of band management.
- sytse 10y ago
  I probably made a mistake here. I assumed that since the management network was never congested we could run the heartbeats there. But it might be that the server can't address the management port from the operating system.
lykron 10y ago
I hope to god you are not using Kubernetes for your databases.
- campervan 10y ago
  Why not? Google Cloud SQL runs on Kubernetes.
  - lykron 10y ago
    Docker, and Kubernetes is still a rapidly evolving technology and there are too many horror stories of it crashing and burning in production for it to be worth it, in my view. Databases are not immutable, and even though Docker has persistent volumes it is still new and evolving. If the database breaks in some way, I don't want to have to worry about the new container having a slightly different patch level and that messing up the database. I don't want to have to worry about rebuilding my database cluster to upgrade it. Google might do some really cool shit with Kubernetes, but they are Google. I don't think using them as a good example for anything but infrastructure on a massive scale is correct. They are years ahead of everyone else, and if they are doing something in production, they have been testing it for years. If shit hits the fan, they have thousands of employees to throw at the problem. GitLab is building a team, and therefore does not have the experience to know these systems inside and out. In my view, using Docker/Kubernetes is adding unnecessary complexity to the database fabric for minimal tradeoffs. And considering that they are speaking out dedicated database servers, it makes no sense to add a unneeded layer of abstraction when in all likelihood the container will be bound to a node.

by sytse 10y ago

viraptor 10y ago

> The servers have a separate management port and "Apart from those routers we'll have a separate router for a 1Gbps management network." It was followed with "For example to make STONITH reliable when there is a lot of traffic on the normal network". That sounded like you want the heartbeats on it, not out of band management.

sytse 10y ago
I probably made a mistake here. I assumed that since the management network was never congested we could run the heartbeats there. But it might be that the server can't address the management port from the operating system.

lykron 10y ago

I hope to god you are not using Kubernetes for your databases.

campervan 10y ago
Why not? Google Cloud SQL runs on Kubernetes.
- lykron 10y ago
  Docker, and Kubernetes is still a rapidly evolving technology and there are too many horror stories of it crashing and burning in production for it to be worth it, in my view. Databases are not immutable, and even though Docker has persistent volumes it is still new and evolving. If the database breaks in some way, I don't want to have to worry about the new container having a slightly different patch level and that messing up the database. I don't want to have to worry about rebuilding my database cluster to upgrade it. Google might do some really cool shit with Kubernetes, but they are Google. I don't think using them as a good example for anything but infrastructure on a massive scale is correct. They are years ahead of everyone else, and if they are doing something in production, they have been testing it for years. If shit hits the fan, they have thousands of employees to throw at the problem. GitLab is building a team, and therefore does not have the experience to know these systems inside and out. In my view, using Docker/Kubernetes is adding unnecessary complexity to the database fabric for minimal tradeoffs. And considering that they are speaking out dedicated database servers, it makes no sense to add a unneeded layer of abstraction when in all likelihood the container will be bound to a node.