3 ms·

Tried using ZFS in the earnest and got spooked, felt it was not production ready. Wanted to use ZFS for MongoDB on Amazon Linux (primarily for compression, but

Tried using ZFS in the earnest and got spooked, felt it was not production ready. Wanted to use ZFS for MongoDB on Amazon Linux (primarily for compression, but also for snapshot functionality for backups). Tried 0.6.2. Ended up running into a situation where a snapshot delete hung and none of my ZFS commands were returning. The snapshot delete was not killable with kill -9. https://github.com/zfsonlinux/zfs/issues/1283 https://github.com/zfsonlinux/zfs/issues/1283 Also, under load encountered a kernel panic or a hang (I forget), turns out it's because the Amazon Linux kernel comes compiled with no preemption. It seems that "voluntary preemption" is the only setting that's reliable. https://github.com/zfsonlinux/zfs/issues/1620 https://github.com/zfsonlinux/zfs/issues/1620 That left a bad taste in my mouth. Might be worth trying out 0.6.3 again. I am still leafing through the issues closed in 0.6.3, but based on what I see, 0.6.2 did not seem production-ready-enough for me: https://github.com/zfsonlinux/zfs/issues?page=2&q=is%3Aissue+milestone%3A0.6.3 https://github.com/zfsonlinux/zfs/issues?page=2&q=is%3Aissue...

ryao 12y ago
Your deadlock was likely caused by the sole regression to get by us in the 0.6.2 release: https://github.com/zfsonlinux/zfs/commit/a117a6d66e5cf1e9d4f173bccc786a169e9a8e04 https://github.com/zfsonlinux/zfs/commit/a117a6d66e5cf1e9d4f... This occurred because it was rare enough that neither us nor the buildbots caught it back in Feburary. George Wilson wrote a fix for it in Illumos rather promptly. However, Illumos and ZoL projects had different formats for the commit titles of regression fixes. In specific, the Illumos developers would reuse the same exact title while the ZoL developers would generally expect a different title, so we missed it when merging work done in Illumos. I caught it in November when I was certain that George had made a mistake and noticed that our code and the Illumos code was different. It is fixed in 0.6.3. The fix was backported to a few distribution repositories, but not to all of them. The 0.6.3 release was notable for having a very long development cycle. As I described in the blog post, the project will begin doing official bug fix releases when 1.0 is tagged. That should ensure that these fixes become available to all distributions much sooner. In the mean time, future releases are planned to have much shorter development cycles than 0.6.3 had, so fixes like this will become available more quickly. That being said, I was at the MongoDB office in NYC earlier this year to troubleshoot poor performance on MongoDB. I will refrain from naming the MongoDB developer with whom I worked lest he become flooded with emails, but my general understanding is that 0.6.3 resolved the performance issues that the MongoDB had observed. Future releases should further increase performance.
- astral303 12y ago
  Thank you so much for the information! This is very encouraging. I will definitely give 0.6.3 a whirl!

ryao 12y ago

Your deadlock was likely caused by the sole regression to get by us in the 0.6.2 release: https://github.com/zfsonlinux/zfs/commit/a117a6d66e5cf1e9d4f173bccc786a169e9a8e04 https://github.com/zfsonlinux/zfs/commit/a117a6d66e5cf1e9d4f... This occurred because it was rare enough that neither us nor the buildbots caught it back in Feburary. George Wilson wrote a fix for it in Illumos rather promptly. However, Illumos and ZoL projects had different formats for the commit titles of regression fixes. In specific, the Illumos developers would reuse the same exact title while the ZoL developers would generally expect a different title, so we missed it when merging work done in Illumos. I caught it in November when I was certain that George had made a mistake and noticed that our code and the Illumos code was different. It is fixed in 0.6.3. The fix was backported to a few distribution repositories, but not to all of them. The 0.6.3 release was notable for having a very long development cycle. As I described in the blog post, the project will begin doing official bug fix releases when 1.0 is tagged. That should ensure that these fixes become available to all distributions much sooner. In the mean time, future releases are planned to have much shorter development cycles than 0.6.3 had, so fixes like this will become available more quickly. That being said, I was at the MongoDB office in NYC earlier this year to troubleshoot poor performance on MongoDB. I will refrain from naming the MongoDB developer with whom I worked lest he become flooded with emails, but my general understanding is that 0.6.3 resolved the performance issues that the MongoDB had observed. Future releases should further increase performance.

astral303 12y ago
Thank you so much for the information! This is very encouraging. I will definitely give 0.6.3 a whirl!