3 ms·
You should be able to split the object store into 2 systems: 1 metadata (think rdbms/nosql/etc) and a blob-data service, keeping large files, think 10KB+. Both
by ddorian43 6y ago
You should be able to split the object store into 2 systems: 1 metadata (think rdbms/nosql/etc) and a blob-data service, keeping large files, think 10KB+. Both systems should be able to be more efficient than the current method.
Example: you can add erasure coding to the blob-data service for better efficiency. You can add fancy indexing to your metadata store. etc etc.
But somebody has to create it, that's the issue.
- lumost 6y agoThat's exactly how distributed filesystems are built. Systems such as HDFS use the NameNode for this task, but depending on the exact characteristics of the fileSystem a multi-master setup is often used. I know of at least one NFS implementation which uses postgres as its metadata layer.