4 ms·

How about a 100TB database with 95TB being pdf's? It seems there are better ways of managing lots of immutable small files (including for backups/replication)

• coldtea 9y ago
  >How about a 100TB database with 95TB being pdf's? How about it? >It seems there are better ways of managing lots of immutable small files (including for backups/replication) than shoving them all into a database. Depends on the business case. A database gives certain guarantees you'll have to replicate (usually badly) in any other way. Conceptually, it's absolutely cleaner. As for from a scientific or engineering standpoint, there are again no laws that dictate saying whether this is bad or good. Even the performance characteristics depend on the implementation of the particular DB storage engine. They might be totally on par with storing in the filesystem (or close enough not to matter). Not to mention that some filesystems might even have worse overhead depending on the type, size, etc. of files. In fact this very FA speaks of "SQLite small blob storage" being "35% Faster Than the Filesystem". Plus filesystem storage and DBs are not that different in most cases -- they share algorithms for storage and indexing (logstorage, btrees, etc), and the main difference is the access layer. The FreeBSD filesystem, for one, was more like a DB storage layer than a 70s style Unix filesystem.