3 ms·
A few things jump out to me after skimming the article: 1. It appears that Ceph/XFS is pretty complex and complicated, and that it doesn't have the best disc
by KronisLV 6y ago
A few things jump out to me after skimming the article:
1. It appears that Ceph/XFS is pretty complex and complicated, and that it doesn't have the best discoverability.
2. Distributed storage in general feels like it could cause a lot of problems unless you have someone who has lots of knowledge about it.
The dashboard seems pretty handy to get an overview of the overall state of the cluster, but the error messages in the logs do feel really low level and not that actionable on their own (unless you are really familiar with XFS and Ceph). Without such familiarity, i'd probably be stuck for a long time.
On an unrelated note, the command at the end of the article would cause me problems because i wouldn't tell what it does at a glance (though maybe that's just because i'd prefer longer scripts instead of long one liners). Is there a reason why people don't store comments with every command, so that they remember what it does after a few months?
For example:
# Check whether XFS mount points are affected by a problem in the 4.18-rc1 kernel version: https://bugzilla.kernel.org/show_bug.cgi?id=202127
# Will feed the current mounts into xfs_info to figure out if sunit and swidth values are incorrect.
awk '$3 == "xfs"{print $2}' /proc/self/mounts | while read mount ; do echo -n "$mount " ; xfs_info $mount | awk '$0 ~ "swidth"{gsub(/.*=/,"",$2); gsub(/.*=/,"",$3); print $2,$3}' | awk '{ if ($1 > $2) print "impacted"; else print "OK"}' ; done
Or do you just get used to "decoding" what the awk and gsub invocations do eventually?
Also, i bet that Ceph is good in an enterprise setting where you really need to store a lot of data, but what about smaller and simpler distributed file systems? I feel like maybe something is missing between just having some servers with local storage or maybe NFS, and full blown solutions like Ceph. I have heard about GlusterFS, LizardFS, MooseFS, SeaweedFS and others, yet all of them seem noticeably more complicated than setting up a Docker Swarm cluster would be (essentially just telling a bunch of nodes to communicate amongst themselves with 1 CLI command per node, and letting them sort the rest out themselves).
Plus, i've heard that even some file systems can have a noticeable impact on the resource consumption on the server (though i just searched and can't find anything concrete in DuckDuckGo, even the Wikipedia page i used doesn't seem to have any good recommendations for CPU/RAM resources: https://en.wikipedia.org/wiki/Comparison_of_distributed_file_systems https://en.wikipedia.org/wiki/Comparison_of_distributed_file... ), so setting clusters like that up doesn't feel like something that a person would do in their small homelab with a few old Athlon processors. :(
Of course, it may be that distributed file storage is just inherently more complex and demanding than making a few servers talk amongst themselves and achieve consensus (the Docker Swarm example above). Any suggestions/opinions?