4 ms·

Of course - following the first law of technical documentation, the amount written about it is inversely proportional to the complexity of the problem. Serious

Of course - following the first law of technical documentation, the amount written about it is inversely proportional to the complexity of the problem. Seriously though, it is getting a bit silly how almost every single kubernetes demo involving persistent data just uses emptyDir or hostPath and we are all supposed to pretend that those are a production-ready solution that won't lose data. AFAIK the PersistentVolumeClaim and StorageClass stuff will be the solution for this?

cies 10y ago
To me it seems that there are many approaches. A storage cluster seems to be a necessity for anything serious. But then the questions is: which one? Open source solutions are not too many (Gluster, Ceph, FreeNAS come to mind), and from the closed source camp ScaleIO looks really promising. But non of these are part of your K8s tutorial :)
- creshal 10y ago
  And gluster at least has horrible performance and stability issues. And with containers, kernel panics kill an entire host…
  - strikerz 10y ago
    Do you have more details on performance and stability issues? The recent releases of gluster have been working very well. Gluster is completely in userspace. How would stability issues, if any, in userspace cause kernel panics? Can you comment more about that?
    - creshal 10y ago
      > Do you have more details on performance and stability issues? Tried using it for a while (last November until ~April) on Debian Jessie, with latest versions. Lots of bugs with SSL mode, lots of desynchronization issues, crippling performance issues, … And no matter what I ran into, the bugs were already known and filed for months at that point, with zero developer reaction. > Gluster is completely in userspace. Unless you try using its NFS server to get not entirely disastrous performance.
      - strikerz 10y ago
        What workloads were you using Gluster for? How did you reach out to the developers? Usually the mailing lists are very responsive and most bugs brought up on the lists are addressed. > Unless you try using its NFS server to get not entirely disastrous performance. Wrong again, Gluster's NFS server is in userspace too. Not sure how that can cause a kernel panic.
        creshal 10y ago
        > What workloads were you using Gluster for? Sharing config files. Wordpress sites. Django sites. The performance was too shitty for everything. > How did you reach out to the developers? Usually the mailing lists are very responsive and most bugs brought up on the lists are addressed. IRC. I ended up finding all my errors unresolved by it in the mailing list archives with zero developer attention, and since I didn't have the time to help RedHat make their product actually usable, I dropped Gluster. > Not sure how that can cause a kernel panic. I guess it's technically not a kernel panic if I/O just stalls so hard that you can neither mount nor unmount nor otherwise access any of your NFS targets… but you have to hard reset the whole machine anyway.
  - lazylizard 10y ago
    we used to run 3 gluster nodes, with a few hundred gb of shared storage..mostly used to keep logs and as war file directory for jboss/tomcat. from what i can tell gluster itself seems ok..but someone also enabled ctdb..which seems to hang every half a year or so..
- tw04 10y ago
  Probably depends on the scale/scope of your deployment. I know NetApp has a docker plugin for their storage platforms, I would imagine other vendors do as well: https://github.com/NetApp/netappdvp https://github.com/NetApp/netappdvp
- user5994461 10y ago
  These storage systems are meant to be SAN devices. Meaning either Amazon EBS volume or Google volumes, on their respective clouds. I recall reading that EBS is being [partially] implemented and tested. (Read: The documentation is full of "known bug: ..." and "please don't try that in production").
e12e 10y ago
Lol, this is how I've been feeling about every virutalization and container/jail solution since the dawn of this century. The early jail-type solutions were somewhat excused, because the answer was either "local filesytem", "nfs" (later nfs and/or iSCSI) - or for the adventurous some kind of cluster filesytem. On that note, are anyone using encrypted NFSv4 in production? On paper it ticks most of the right boxes, but I'm not convinced it actually works...
- zwischenzug 10y ago
  It's mandated by our security team, but no-one has ever got it working...
deleted 10y ago
[deleted]

cies 10y ago

To me it seems that there are many approaches. A storage cluster seems to be a necessity for anything serious. But then the questions is: which one? Open source solutions are not too many (Gluster, Ceph, FreeNAS come to mind), and from the closed source camp ScaleIO looks really promising. But non of these are part of your K8s tutorial :)

creshal 10y ago
And gluster at least has horrible performance and stability issues. And with containers, kernel panics kill an entire host…
- strikerz 10y ago
  Do you have more details on performance and stability issues? The recent releases of gluster have been working very well. Gluster is completely in userspace. How would stability issues, if any, in userspace cause kernel panics? Can you comment more about that?
  - creshal 10y ago
    > Do you have more details on performance and stability issues? Tried using it for a while (last November until ~April) on Debian Jessie, with latest versions. Lots of bugs with SSL mode, lots of desynchronization issues, crippling performance issues, … And no matter what I ran into, the bugs were already known and filed for months at that point, with zero developer reaction. > Gluster is completely in userspace. Unless you try using its NFS server to get not entirely disastrous performance.
    - strikerz 10y ago
      What workloads were you using Gluster for? How did you reach out to the developers? Usually the mailing lists are very responsive and most bugs brought up on the lists are addressed. > Unless you try using its NFS server to get not entirely disastrous performance. Wrong again, Gluster's NFS server is in userspace too. Not sure how that can cause a kernel panic.
      - creshal 10y ago
        > What workloads were you using Gluster for? Sharing config files. Wordpress sites. Django sites. The performance was too shitty for everything. > How did you reach out to the developers? Usually the mailing lists are very responsive and most bugs brought up on the lists are addressed. IRC. I ended up finding all my errors unresolved by it in the mailing list archives with zero developer attention, and since I didn't have the time to help RedHat make their product actually usable, I dropped Gluster. > Not sure how that can cause a kernel panic. I guess it's technically not a kernel panic if I/O just stalls so hard that you can neither mount nor unmount nor otherwise access any of your NFS targets… but you have to hard reset the whole machine anyway.
- lazylizard 10y ago
  we used to run 3 gluster nodes, with a few hundred gb of shared storage..mostly used to keep logs and as war file directory for jboss/tomcat. from what i can tell gluster itself seems ok..but someone also enabled ctdb..which seems to hang every half a year or so..
tw04 10y ago
Probably depends on the scale/scope of your deployment. I know NetApp has a docker plugin for their storage platforms, I would imagine other vendors do as well: https://github.com/NetApp/netappdvp https://github.com/NetApp/netappdvp
user5994461 10y ago
These storage systems are meant to be SAN devices. Meaning either Amazon EBS volume or Google volumes, on their respective clouds. I recall reading that EBS is being [partially] implemented and tested. (Read: The documentation is full of "known bug: ..." and "please don't try that in production").

e12e 10y ago

Lol, this is how I've been feeling about every virutalization and container/jail solution since the dawn of this century. The early jail-type solutions were somewhat excused, because the answer was either "local filesytem", "nfs" (later nfs and/or iSCSI) - or for the adventurous some kind of cluster filesytem. On that note, are anyone using encrypted NFSv4 in production? On paper it ticks most of the right boxes, but I'm not convinced it actually works...

zwischenzug 10y ago
It's mandated by our security team, but no-one has ever got it working...

deleted 10y ago

[deleted]