4 ms·
One issue I have with this article: "Images, files, and binary data: Great now you can't use nginx or another lightweight web server to serve them up." That's
by mmastrac 15y ago
One issue I have with this article:
"Images, files, and binary data: Great now you can't use nginx or another lightweight web server to serve them up."
That's not true - you can create a private HTTP endpoint in your web app that your lightweight caching frontend can serve and cache from.
(I'm not saying it's not a better idea to store things in S3 or another big-blob service, but the reasoning in this article doesn't hold water)
- duskwuff 15y agoEven so, serving them up will still be less efficient (and more complex!) than if they were files.
- mmastrac 15y ago(note: I'm playing devil's advocate here for storing images/files in the database - this is not something I'd recommend doing without careful thought) In terms of efficiency, you're only paying a small price on the first hit for an image before the cache comes into play. It won't be as efficient for you as directly serving from S3, but with a properly configured cache, the effect would be negligible. In terms of complexity, you're looking at a single nginx configuration endpoint of a handful of lines to cache the images on-disk or in memcache for fast serving. In the case where you've got CloudFlare in front of your web app, it's free. Overall it can be less complex, because you can re-use your existing code for talking to the database, rather than adding code to talk to another service like S3. The real argument against storing big binary data in the database should be based on the argument that databases aren't necessarily always tuned properly for storing things that aren't relational (which the article briefly addresses), and that storing your binary data alongside your relational data could have deleterious effects on the performance of your relational queries.
- d503 15y agoIncidentally, SQL Server 2008 has a FILESTREAM type which integrates with NTFS to give you streaming access to BLOBs (you can even work with them via Windows file handles). MSDN has a pretty thorough (and I thought interesting) technical article on them. http://msdn.microsoft.com/en-us/library/cc949109(v=sql.100).aspx http://msdn.microsoft.com/en-us/library/cc949109(v=sql.100)....
- junto 15y agoInteresting. Thanks for linking to this. Notably the article states a recommendation / constraint: The recommendation for using FILESTREAM is that the individual units of BLOB data be 1 MB in size or higher. Though, the benefits when that is the case, look mighty impressive.
- shazow 15y agoAdditionally, there may be an argument for locality here. If every time you're serving an image, you need to look up a bunch of metadata for it, you can benefit from storing it in the same place. Then it's a matter of whether you rather encode the metadata in the image and store it as a file, or encode the image in the metadata and store it as a database entry. Now if you throw in having to do a little bit of post-processing based on various conditions, then the scales start to tip.
- arethuza 15y agoThe component to serve the files up may be slightly more complex, but that has to be balanced against the need to have separate high-availability, scalability and backup solutions for your data that is now in two kinds of places: a DB and a file-system. Of course, it depends on what you are doing, but many databases make working with BLOBs pretty straightforward and I wouldn't rule out the benefits just to follow some over-general rules.
- gaius 15y agoSingTel do their streaming VOD service out of an RDBMS, seems to work OK.
- thisishugo 15y agoAnd not forgetting, for example, Riak (Luwak), MongoDB (GridFS) or CouchDB (attachments), all of which are intended to hold BLOBs.