3 ms·

How old is this benchmark? HDD or SSD (I think the former given the 2011 reference. A filesystem is the canonical means for storing files - just use it - it's

How old is this benchmark? HDD or SSD (I think the former given the 2011 reference. A filesystem is the canonical means for storing files - just use it - it's incredibly handy to have regular tools at your disposal for managing those files outside of the database. The only case I can think where maybe files in the db makes sense is if you have millions of really tiny files (100-200 bytes). Having files in the db maybe avoids an extra seek - but in any real system high throughput system, you're probably leaning towards SSDs where you get many more seeks/s than you do on a spinny disk. If your application needs something more like a distributed filesystem, there are better things (gluster, moosefs, even nfs, etc).

jpgvm 12y ago
If consistency and reliability matters I would probably avoid GlusterFS like the plague. (things may have changed now but I worked with it in a petabyte scale bioinformatics cluster and it was not fun) Look at Ceph if you want to store files in a consistent and distributed fashion. Not the filesystem though, the object store (RADOS).
- mattbillenstein 12y ago
  I had trouble with Gluster - and the Ceph filesystem as well. But MooseFS was solid.
gilgoomesh 12y ago
> A filesystem is the canonical means for storing files - just use it Sure but what about the many use cases where you're not really storing files but metadata or other generated data? A simple example might be photo libraries. The photos themselves are files but are the 5kB small and 50kB medium thumbnails generated from the photos worth storing as files? Or should they be stored as blobs in the database along with their other metadata? It's good to know where the cutoff should be.
- mattbillenstein 12y ago
  Metadata makes sense in the db - it's structured non-binary data. But I think the binary thumbnails and so forth should almost always be files on the filesystem.
Bill_Dimm 12y ago
> How old is this benchmark? It says they used SQLite version 3.7.8. Since 3.7.8 came out on September 19, 2011 and 3.7.9 came out on November 1, 2011, it was probably in that range.

jpgvm 12y ago

If consistency and reliability matters I would probably avoid GlusterFS like the plague. (things may have changed now but I worked with it in a petabyte scale bioinformatics cluster and it was not fun) Look at Ceph if you want to store files in a consistent and distributed fashion. Not the filesystem though, the object store (RADOS).

mattbillenstein 12y ago
I had trouble with Gluster - and the Ceph filesystem as well. But MooseFS was solid.

gilgoomesh 12y ago

> A filesystem is the canonical means for storing files - just use it Sure but what about the many use cases where you're not really storing files but metadata or other generated data? A simple example might be photo libraries. The photos themselves are files but are the 5kB small and 50kB medium thumbnails generated from the photos worth storing as files? Or should they be stored as blobs in the database along with their other metadata? It's good to know where the cutoff should be.

mattbillenstein 12y ago
Metadata makes sense in the db - it's structured non-binary data. But I think the binary thumbnails and so forth should almost always be files on the filesystem.

Bill_Dimm 12y ago

> How old is this benchmark? It says they used SQLite version 3.7.8. Since 3.7.8 came out on September 19, 2011 and 3.7.9 came out on November 1, 2011, it was probably in that range.