3 ms·

How would you run Apache + PHP + MySQL on Borg? Hint: you can't. Not without HUGE difficulty, anyway. Nobody does it. Part of Kubernetes "meet people where t

How would you run Apache + PHP + MySQL on Borg? Hint: you can't. Not without HUGE difficulty, anyway. Nobody does it. Part of Kubernetes "meet people where they are" mindset is that we simply can not ask people to rewrite their apps. Truth is, a LOT of people don't write code. They write content and use pre-built code (think WordPress). Borg simply can not accomodate that very well. It's simple as long as you control things from soup to nuts. Yes, some "organic growth" was useful features. But a lot of it was useless features, or features that are now obsolete but can't be removed because someone somewhere is using them, and probably doesn't have enough time to re-test without the feature (true story).

mike_hearn 11y ago
Put together the packages, request a fixed port, disable the health checks (unless I had a file in the Apache root with the right name), start it up? I don't think Borg imposes all that many requirements on jobs, really, and the few it does can be disabled. Or at least could. But I guess we're probably wandering out of the area covered by the papers now.
thockingoog 11y ago
I can not reply to your last comment, but "port 80 request denied". And where do you store your MySQL data? The point being nothing is impossible, just prohibitively hard.
- dekhn 11y ago
  There are any number of places you can store your MySQL data in a container world. The first and best is to make all your MySQL IO go to an external cluster filesystem or other remote IO system. Because MySQL supports pluggable storage, you could write an Hadoop FS storage manager. This has the advantage that if a single MySQL instance is blown away, all the committed data is available for a new replica to start reading. I don't know if Docker or other container systems support automagically turning local IO calls into remote IO calls (or whether that really makes sense in a MySQL environment), but that's a similar approach. Condor supported this through their remote libc interface. The second is to use some sort of per-task persistent local storage. In Docker world, this would be a mounted volume- the docker host would manage the storage, and new containers would remount that storage. You could have a process that restored the local storage from a backup, and the use replication from a master to catch up. The third would be to have some sort of per-container persistent storage (the Borg paper calls this an "alloc"). For the server, most people wouldn't have Apache bind port 80 inside the container- you'd bind another port, and use some sort of other mechanism such as load balancing to expose the web server on a standard port
  - thockingoog 11y ago
    The question had an implied "... in Borg" suffix. The point was to demonstrate that Borg does not have "legacy" affordances like durable storage (well, not in the same way as MySQL would need).
    - dekhn 11y ago
      We both are Google employees. I used to be MySQL sre, with experience in this.
      - thockingoog 11y ago
        You're still making my point. Using something like MySQL on Borg is not trivial.

mike_hearn 11y ago

Put together the packages, request a fixed port, disable the health checks (unless I had a file in the Apache root with the right name), start it up? I don't think Borg imposes all that many requirements on jobs, really, and the few it does can be disabled. Or at least could. But I guess we're probably wandering out of the area covered by the papers now.

thockingoog 11y ago

I can not reply to your last comment, but "port 80 request denied". And where do you store your MySQL data? The point being nothing is impossible, just prohibitively hard.

dekhn 11y ago
There are any number of places you can store your MySQL data in a container world. The first and best is to make all your MySQL IO go to an external cluster filesystem or other remote IO system. Because MySQL supports pluggable storage, you could write an Hadoop FS storage manager. This has the advantage that if a single MySQL instance is blown away, all the committed data is available for a new replica to start reading. I don't know if Docker or other container systems support automagically turning local IO calls into remote IO calls (or whether that really makes sense in a MySQL environment), but that's a similar approach. Condor supported this through their remote libc interface. The second is to use some sort of per-task persistent local storage. In Docker world, this would be a mounted volume- the docker host would manage the storage, and new containers would remount that storage. You could have a process that restored the local storage from a backup, and the use replication from a master to catch up. The third would be to have some sort of per-container persistent storage (the Borg paper calls this an "alloc"). For the server, most people wouldn't have Apache bind port 80 inside the container- you'd bind another port, and use some sort of other mechanism such as load balancing to expose the web server on a standard port
- thockingoog 11y ago
  The question had an implied "... in Borg" suffix. The point was to demonstrate that Borg does not have "legacy" affordances like durable storage (well, not in the same way as MySQL would need).
  - dekhn 11y ago
    We both are Google employees. I used to be MySQL sre, with experience in this.
    - thockingoog 11y ago
      You're still making my point. Using something like MySQL on Borg is not trivial.