3 ms·

I've been using docker in production with Elasticsearch and MySQL for 3 years in the PB scale and have never had data corruption issues occur. Corruption occur

by k2xl 10y ago

I've been using docker in production with Elasticsearch and MySQL for 3 years in the PB scale and have never had data corruption issues occur. Corruption occurs on data drives even without docker - you still have to plan for it. This is why you enable replication. This is why you snapshot/backup your data daily and have disaster recovery plans. There are some major reasons why I actually think running databases in docker containers, even if you are mounting a volume for the data. 1) Development environments can be similar to production. Ensures everyone runs the same version that is running in prod. 2) You don't have to worry as much about what is installed on the host machine. 3) In a clustered setup, it's easier to ensure each node is running the same configuration, version, etc... One of my issues with all the gripes about docker are the assertions that it causes issues. In all of my time of using docker, 99% of the time when there is an issue it has nothing to do with docker itself. Everyone loves to blame it when things go wrong though. This article doesn't really back up any of the claims about any of its issues. It just makes blanket statements without backing them up. Don't like docker's networking? Use host networking then. What people don't think about is the countless issues that will never come up when using containerization. I never have to worry about whether or not python 2.7 is installed on a server that I'm going to deploy a python 3 app on. I also have MUCH higher confidence that if things work on my local development env (which runs the same containers), then there is a high chance it will work in production. YMMV

Bombthecat 10y ago
Wow, three years ago? Balls of steel!
- tinco 10y ago
  People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. We have also been running databases in Docker (on the tb scale though) for around 3 years, we had the odd issue here and there, but nothing terrible and certainly nothing fundamental or resulting in data loss. If your data is corrupted by a single process dying in an unclean fashion then you have other operational problems.
  - jcahill84 10y ago
    This is one of the biggest problems with people hating on docker... They don't understand what it is and actually does. Great comment!
    - Bombthecat 10y ago
      Oh hell no, not hating docker! I love docker ( I'm the multiplier in my company and teaching docker,and doing presentations) But putting pb of data in docker three years ago is just insane. There are still things not clear right now,which are not stable ( not in unstable destroying data) but unstable in: if something goes wrong with your data you need to dig deep and find out what's going on. If stuff changes every few month. You will have a hard time. Also, enterprise concerns like: Will there be a docker standard from Google and Facebook? Will AUfs be Version 4 or scrapped again? Will the composer format stay? How to manage runtime upgrades? Ie syncronize 1000 dockers with one version update than another 1000 with another software. So that one update depends on another. Etc etc etc. Three years ago it just wasn't there. Two years ago I would have said: "good enough" to play around. One year ago it started to get really interesting for enterprise. That's why I say: you handle pb of data,with docker,three years ago? Balls of steel.
      - k2xl 10y ago
        The data wasn't petabytes 3 years ago, it grew to that amount over time. But with that said, if you have replication and operational workflows set up, you minimize the chance of issues. If you have mission critical data, you should set up your database to handle failovers of any type, let alone docker.
      - deleted 10y ago
        [deleted]
      - jcahill84 10y ago
        Didn't mean you Bombthecat :-) I was referring the article.
  - eeZah7Ux 10y ago

by k2xl 10y ago

Bombthecat 10y ago

Wow, three years ago? Balls of steel!

tinco 10y ago
People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. We have also been running databases in Docker (on the tb scale though) for around 3 years, we had the odd issue here and there, but nothing terrible and certainly nothing fundamental or resulting in data loss. If your data is corrupted by a single process dying in an unclean fashion then you have other operational problems.
- jcahill84 10y ago
  This is one of the biggest problems with people hating on docker... They don't understand what it is and actually does. Great comment!
  - Bombthecat 10y ago
    Oh hell no, not hating docker! I love docker ( I'm the multiplier in my company and teaching docker,and doing presentations) But putting pb of data in docker three years ago is just insane. There are still things not clear right now,which are not stable ( not in unstable destroying data) but unstable in: if something goes wrong with your data you need to dig deep and find out what's going on. If stuff changes every few month. You will have a hard time. Also, enterprise concerns like: Will there be a docker standard from Google and Facebook? Will AUfs be Version 4 or scrapped again? Will the composer format stay? How to manage runtime upgrades? Ie syncronize 1000 dockers with one version update than another 1000 with another software. So that one update depends on another. Etc etc etc. Three years ago it just wasn't there. Two years ago I would have said: "good enough" to play around. One year ago it started to get really interesting for enterprise. That's why I say: you handle pb of data,with docker,three years ago? Balls of steel.
    - k2xl 10y ago
      The data wasn't petabytes 3 years ago, it grew to that amount over time. But with that said, if you have replication and operational workflows set up, you minimize the chance of issues. If you have mission critical data, you should set up your database to handle failovers of any type, let alone docker.
    - deleted 10y ago
      [deleted]
    - jcahill84 10y ago
      Didn't mean you Bombthecat :-) I was referring the article.
- eeZah7Ux 10y ago