5 ms·

Storage with containers is something that we need to solve properly ...Or you could just use SmartOS zones, and then all these artificial problems go away, as

by Annatar 10y ago

Storage with containers is something that we need to solve properly ...Or you could just use SmartOS zones, and then all these artificial problems go away, as zones natively reside on the ZFS filesystem inside of the zpool underneath them. Why would one want to waste one's time on technology which is clearly not finished and doesn't offer any advantages over zones? As a logical being, I'm genuinely perplexed by the overall insistence on Docker. What is the cause of it?

cyphar 10y ago
Here we go again. As we've discussed before, there are problems that are inherent to designing distributed systems that are not solved by SmartOS's magic sauce. Please stop pretending that every possible problem that is faced by GNU/Linux technologies has already been solved by SmartOS, it's getting quite tiresome. "Storage with containers" refers to correctly decoupling the state from your application and storing it a way that allows for horizontal scaling. It's not as simple as proclaiming "we have a magical filesystem, therefore all problems are solved and you should use SmartOS kthxbai". There are harder problems here, and no amount of SmartOS shilling will get around that.
- Annatar 10y ago
  You completely ignored my question: what is the cause of insistence on Docker? Docker storage does not solve the scaling problem either, indeed, no known technology in existence solves it: this is one of the unsolved problems in computer science, still largely terra incognita. The only way to design for horizontal scalability is in the application, where the application keeps track of the global state across all nodes it runs on. Even if one were to use distributed storage engine like Oracle RAC with the automatic storage manager and synchronous multimaster replication, one's application would still have to contain logic about ejecting/re-integrating failed RAC nodes, as well as re-trying the transaction on a different node, and that is in addition to internal loadbalancing it would have to perform, especially if the goal is high availability with exactly zero downtime or zero loss of service at all times. As far as my "pretending", show me one problem which GNU/Linux has solved that SmartOS already hasn't. Just one. In closing, I mentioned SmartOS on purpose: perhaps someone reading this will look it up, and try it out. And perhaps they'll like it (Linux became popular the same way). What I get out of it eventually is a job market with opportunities, but most importantly, I get to sleep through the night without having to deal with idiotic problems in GNU/Linux solved anywhere from 11 to 25 years ago in SmartOS (depending on the problem). I really do not want to waste any more time on Linux, and certainly not on Docker. SmartOS can do Docker, just so you know, although it doesn't need it to provide containerization, at all, but one does have that choice. So if you really want to run Docker at all costs, which advantages does GNU/Linux offer you over SmartOS? Let's talk technology.
  - icebraining 10y ago
    If you want to argue the merits of SmartOS over Docker on any other topic, you have plenty of avenues. This thread is about storage, and while I feel sympathetic to your position, frankly you're off-topic (and off-putting).
    - Annatar 10y ago
      Okay, then answer me why it makes sense to waste one's time on storage contortions in Docker on Linux when a different technology offers an enterprise tested solution? In SmartOS all I have to do is instantiate a docker branded zone, and the storage it uses is automatically permanent, no extra steps needed like on Linux.
      - cyphar 10y ago
        > and the storage it uses is automatically permanent Is it also automatically replicated to a distributed storage pool so that it is mirrored to all of your other containers? Because that's what people are talking about when they discuss "container storage" -- they're discussing creating storage for a distributed system. You can create persistence incredibly easily with runC (the core executor for Docker). In fact, runC by default just allows you to store all of your state in your container's rootfs (you need to manually add mounts to make your state run in a tmpfs). It's just not very useful to have all of your state on one of your compute nodes. I really don't understand why you keep pretending that we're focusing on the simple things. If I didn't know better, I would think that you're trying to distort the problems we're discussing (which are problems for every system) into very easy problems that SmartOS has solved (but also every other operating system has solved as well). As someone who thinks SmartOS has some very powerful technology, I'm surprised that an advocate of SmartOS is resorting to strawmanning the problems that we're discussing. Surely the technology should stand for itself.
        Annatar 10y ago
        Is it also automatically replicated to a distributed storage pool so that it is mirrored to all of your other containers? Even better, it uses a distributed object store backed by ZFS called "Manta". Do you have any more questions? Meanwhile, you are very persistent at stonewalling my original question: why the insistence on using Docker?

by Annatar 10y ago

cyphar 10y ago

Here we go again. As we've discussed before, there are problems that are inherent to designing distributed systems that are not solved by SmartOS's magic sauce. Please stop pretending that every possible problem that is faced by GNU/Linux technologies has already been solved by SmartOS, it's getting quite tiresome. "Storage with containers" refers to correctly decoupling the state from your application and storing it a way that allows for horizontal scaling. It's not as simple as proclaiming "we have a magical filesystem, therefore all problems are solved and you should use SmartOS kthxbai". There are harder problems here, and no amount of SmartOS shilling will get around that.

Annatar 10y ago
You completely ignored my question: what is the cause of insistence on Docker? Docker storage does not solve the scaling problem either, indeed, no known technology in existence solves it: this is one of the unsolved problems in computer science, still largely terra incognita. The only way to design for horizontal scalability is in the application, where the application keeps track of the global state across all nodes it runs on. Even if one were to use distributed storage engine like Oracle RAC with the automatic storage manager and synchronous multimaster replication, one's application would still have to contain logic about ejecting/re-integrating failed RAC nodes, as well as re-trying the transaction on a different node, and that is in addition to internal loadbalancing it would have to perform, especially if the goal is high availability with exactly zero downtime or zero loss of service at all times. As far as my "pretending", show me one problem which GNU/Linux has solved that SmartOS already hasn't. Just one. In closing, I mentioned SmartOS on purpose: perhaps someone reading this will look it up, and try it out. And perhaps they'll like it (Linux became popular the same way). What I get out of it eventually is a job market with opportunities, but most importantly, I get to sleep through the night without having to deal with idiotic problems in GNU/Linux solved anywhere from 11 to 25 years ago in SmartOS (depending on the problem). I really do not want to waste any more time on Linux, and certainly not on Docker. SmartOS can do Docker, just so you know, although it doesn't need it to provide containerization, at all, but one does have that choice. So if you really want to run Docker at all costs, which advantages does GNU/Linux offer you over SmartOS? Let's talk technology.
- icebraining 10y ago
  If you want to argue the merits of SmartOS over Docker on any other topic, you have plenty of avenues. This thread is about storage, and while I feel sympathetic to your position, frankly you're off-topic (and off-putting).
  - Annatar 10y ago
    Okay, then answer me why it makes sense to waste one's time on storage contortions in Docker on Linux when a different technology offers an enterprise tested solution? In SmartOS all I have to do is instantiate a docker branded zone, and the storage it uses is automatically permanent, no extra steps needed like on Linux.
    - cyphar 10y ago
      > and the storage it uses is automatically permanent Is it also automatically replicated to a distributed storage pool so that it is mirrored to all of your other containers? Because that's what people are talking about when they discuss "container storage" -- they're discussing creating storage for a distributed system. You can create persistence incredibly easily with runC (the core executor for Docker). In fact, runC by default just allows you to store all of your state in your container's rootfs (you need to manually add mounts to make your state run in a tmpfs). It's just not very useful to have all of your state on one of your compute nodes. I really don't understand why you keep pretending that we're focusing on the simple things. If I didn't know better, I would think that you're trying to distort the problems we're discussing (which are problems for every system) into very easy problems that SmartOS has solved (but also every other operating system has solved as well). As someone who thinks SmartOS has some very powerful technology, I'm surprised that an advocate of SmartOS is resorting to strawmanning the problems that we're discussing. Surely the technology should stand for itself.
      - Annatar 10y ago
        Is it also automatically replicated to a distributed storage pool so that it is mirrored to all of your other containers? Even better, it uses a distributed object store backed by ZFS called "Manta". Do you have any more questions? Meanwhile, you are very persistent at stonewalling my original question: why the insistence on using Docker?