4 ms·

Thank goodness. btrfs has bitten me way too often on CoreOS. I eventually succumbed to wiping the /var/lib/docker mounted directory on every reboot to minimize

Thank goodness. btrfs has bitten me way too often on CoreOS. I eventually succumbed to wiping the /var/lib/docker mounted directory on every reboot to minimize the occurrence of the low disk space issues, but nevertheless, issues still exist. Even today, I still have some cases where docker will complain about not being able to create the container/file, due to some inane error. I've had to reboot machines many times just to try to get it to cooperate. Furthermore, that solution of wiping the partition means that my machines need to redownload their docker images on every reboot, which means they come up more slowly than absolutely necessary.

Gigablah 12y ago
Yep. It's fun when your entire etcd cluster dies at the same time (because you started a global unit that pulls a huge docker container). You'd think they have safeguards to automatically rebalance on certain conditions (metadata filling up, etc) but no, you have to babysit btrfs.
- saryant 12y ago
  I'm glad I'm not the only one with these problems. I was starting to think I was going crazy with my etcd+btrfs issues.
- 23david 12y ago
  hrm... what's the failure scenario here? Are there any good btrfs management utils that can help with running btrfs in prod?
  - throwaway_nsa2 12y ago
    Stop it with the anti-American attitude. We are keeping tabs on you.
  - throwaway_nsa3 12y ago
    Stop it with the anti-American attitude. We are keeping tabs on you.
sureshv 12y ago
I've seen lots devicemapper failures running docker build on ext4 (AWS SSD EBS). Really painful to have to rebuild on failures since these issue seem to happen quite often.
Shish2k 12y ago
> Even today, I still have some cases where docker will complain about not being able to create the container/file, due to some inane error. I've had to reboot machines many times just to try to get it to cooperate. I get those, and am not using btrfs...
- peteridah 12y ago
  yes this, I get those too and no btrfs
  - Gigablah 12y ago
    Is it this one? https://github.com/docker/docker/issues/5684 https://github.com/docker/docker/issues/5684 Or this? https://github.com/docker/docker/issues/4036 https://github.com/docker/docker/issues/4036
23david 12y ago
This is ridiculous. CoreOS is managing their own distro, so why not just package in ZFS and get it over with. It's been how many years now waiting for the next great ZFS competitor? If nobody is able to improve on ZFS, how about we just all jump on the bandwagon and move on with our lives? If nothing else, having more people using ZFS may inspire someone to actually improve on it. BTRFS has such a limited feature set in comparison to ZFS that with the advent of ZFS on Linux, it's really hard to understand the community's continued backing of BTRFS as the Linux community's CoW filesystem competitor to ZFS. Spend a few weeks using ZFS on Linux on a machine with a few SSDs and a few HDDs, or just enable it on a linux laptop with a SSD drive, and it's hard to go back to anything else.
- tacticus 12y ago
  Because sun decided to sabotage it when they released it and oracle haven't stepped back yet.
  - throwaway90446 12y ago
    Sabotage? How, exactly? I have heard of no major corruption or performance bugs in ZFS, what am I missing?
    - kbar13 12y ago
      licensing
    - ecnahc515 12y ago
      From http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue "ZFS is licensed under the Common Development and Distribution License (CDDL), and the Linux kernel is licensed under the GNU General Public License Version 2 (GPLv2). While both are free open source licenses they are restrictive licenses. The combination of them causes problems because it prevents using pieces of code exclusively available under one license with pieces of code exclusively available under the other in the same binary. "
    - 12y ago
jaytaylor 12y ago
I've been bitten badly by btrfs before as well. I wish it were feasible to get CoreOS (and Docker, for that matter) to add support for ZFS. I use LXC + ZFS every day and it is stable, reasonably fast, and all around wonderful!!! sigh.

Gigablah 12y ago

Yep. It's fun when your entire etcd cluster dies at the same time (because you started a global unit that pulls a huge docker container). You'd think they have safeguards to automatically rebalance on certain conditions (metadata filling up, etc) but no, you have to babysit btrfs.

saryant 12y ago
I'm glad I'm not the only one with these problems. I was starting to think I was going crazy with my etcd+btrfs issues.
23david 12y ago
hrm... what's the failure scenario here? Are there any good btrfs management utils that can help with running btrfs in prod?
- throwaway_nsa2 12y ago
  Stop it with the anti-American attitude. We are keeping tabs on you.
- throwaway_nsa3 12y ago
  Stop it with the anti-American attitude. We are keeping tabs on you.

sureshv 12y ago

I've seen lots devicemapper failures running docker build on ext4 (AWS SSD EBS). Really painful to have to rebuild on failures since these issue seem to happen quite often.

Shish2k 12y ago

> Even today, I still have some cases where docker will complain about not being able to create the container/file, due to some inane error. I've had to reboot machines many times just to try to get it to cooperate. I get those, and am not using btrfs...

peteridah 12y ago
yes this, I get those too and no btrfs
- Gigablah 12y ago
  Is it this one? https://github.com/docker/docker/issues/5684 https://github.com/docker/docker/issues/5684 Or this? https://github.com/docker/docker/issues/4036 https://github.com/docker/docker/issues/4036

23david 12y ago

This is ridiculous. CoreOS is managing their own distro, so why not just package in ZFS and get it over with. It's been how many years now waiting for the next great ZFS competitor? If nobody is able to improve on ZFS, how about we just all jump on the bandwagon and move on with our lives? If nothing else, having more people using ZFS may inspire someone to actually improve on it. BTRFS has such a limited feature set in comparison to ZFS that with the advent of ZFS on Linux, it's really hard to understand the community's continued backing of BTRFS as the Linux community's CoW filesystem competitor to ZFS. Spend a few weeks using ZFS on Linux on a machine with a few SSDs and a few HDDs, or just enable it on a linux laptop with a SSD drive, and it's hard to go back to anything else.

tacticus 12y ago
Because sun decided to sabotage it when they released it and oracle haven't stepped back yet.
- throwaway90446 12y ago
  Sabotage? How, exactly? I have heard of no major corruption or performance bugs in ZFS, what am I missing?
  - kbar13 12y ago
    licensing
  - ecnahc515 12y ago
    From http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue "ZFS is licensed under the Common Development and Distribution License (CDDL), and the Linux kernel is licensed under the GNU General Public License Version 2 (GPLv2). While both are free open source licenses they are restrictive licenses. The combination of them causes problems because it prevents using pieces of code exclusively available under one license with pieces of code exclusively available under the other in the same binary. "
  - 12y ago

jaytaylor 12y ago

I've been bitten badly by btrfs before as well. I wish it were feasible to get CoreOS (and Docker, for that matter) to add support for ZFS. I use LXC + ZFS every day and it is stable, reasonably fast, and all around wonderful!!! sigh.