4 ms·

That didn't really say much. Don't store file content in your databases, otherwise your backups will balloon to massive size.

mcny 8y ago
Personally, I think the answer is it depends. If all we are storing are profile pictures for a small app for a small user base, I think it is better to store things in the database rather than adding complexity of S3. I mean if you are making an image board like https://github.com/DangerOnTheRanger/maniwani https://github.com/DangerOnTheRanger/maniwani you should not listen to me and use something like S3 because the images are the bread and butter of your application but often user generated images only used for profile pictures might be okay to store in a database. Why make things more complicated than they have to be?
- scarface74 8y ago
  What complexity? If you Store your files in an S3 you get redundancy for free, easy cross region replication, you can put them behind CloudFront as a CDN.
  - blondin 8y ago
    S3 is the complexity. anything added to a system doubles the complexity of said system...
    - scarface74 8y ago
      So instead of aphorisms, care to explain the complexity of issuing one command to copy a file and storing the URL in the database and letting AWS taking care of copying?
      - BurningFrog 8y ago
        Doing something is always more complexity than doing nothing. Some seem to think "complexity" only happens when something is really hard. That's not how to think of it at all. Systems usually become overly complex by adding many little simple things. Each of those may be trivial, but together they can make your system impossible to understand and manage.
        scarface74 8y ago
        You still haven’t said how in this particular case it’s more complex. In fact, you are offloading sending images to AWS instead of you doing it yourself.
        0xFACEFEED 8y ago
        Deletions for one. You'll need a background job with retry logic.
        scarface74 8y ago
        Why a background job? Just delete the object when you delete the record.
      - bencoder 8y ago
        You've skipped over quite a lot. - learning about and understanding S3 in the first place. - Billing and AWS account management (possibly requiring agreement/permission from others) - maintaining credentials securely for your app to access S3. - bug fixing (for e.g. it's easy to get the permissions wrong when you upload so that it's not visible) Since you're already storing the URL in your dB, why not just store the binary data you have? I don't think you're wrong about using something like S3 being a better solution, but it's not quite as simple as "issuing one command"
        scarface74 8y ago
        - there is literally one command and a quick Google search. - he brought up S3, I am assuming he’s already on AWS - you don’t have to. The role attached to your EC2 instance will have permission. If he is already on AWS, hopefully he knows that. - From the requirements it seems like the images should be publicly accessible. If not, when you create a bucket, it’s by default private. Yes if he wants to limit access he would have to either stream from S3 to the client - I had to do that it was a quick Google search - or generate presigned URL. Why not store binary data in the DB? It makes backups larger. That being said, SQL Server has the filestream type that does in fact store the binary data as a separate file on disk, but gets treated as if it is part of the database for backup and restore purposes.
- basetop 8y ago
  If you don't have to worry about scale and you are sure your user base is going to remain small and stable, then you are fine. If you expect it to grow, then you should look into dumping your files on a file server. This way you can scale, backups of your database and file server can be done quickly, independently or in parallel and it allows for better use of network resources ( your app/server/etc can distribute requests to both db and file server instead of having one large request from the db). At the end of the day, if it works and you are aware of the business needs, that's all that matters.
- manigandham 8y ago
  That means you also need to serve those images yourself and even the smallest HTTP server/route is more work to maintain instead of just serving directly from the URL you get with S3.

mcny 8y ago

Personally, I think the answer is it depends. If all we are storing are profile pictures for a small app for a small user base, I think it is better to store things in the database rather than adding complexity of S3. I mean if you are making an image board like https://github.com/DangerOnTheRanger/maniwani https://github.com/DangerOnTheRanger/maniwani you should not listen to me and use something like S3 because the images are the bread and butter of your application but often user generated images only used for profile pictures might be okay to store in a database. Why make things more complicated than they have to be?

scarface74 8y ago
What complexity? If you Store your files in an S3 you get redundancy for free, easy cross region replication, you can put them behind CloudFront as a CDN.
- blondin 8y ago
  S3 is the complexity. anything added to a system doubles the complexity of said system...
  - scarface74 8y ago
    So instead of aphorisms, care to explain the complexity of issuing one command to copy a file and storing the URL in the database and letting AWS taking care of copying?
    - BurningFrog 8y ago
      Doing something is always more complexity than doing nothing. Some seem to think "complexity" only happens when something is really hard. That's not how to think of it at all. Systems usually become overly complex by adding many little simple things. Each of those may be trivial, but together they can make your system impossible to understand and manage.
      - scarface74 8y ago
        You still haven’t said how in this particular case it’s more complex. In fact, you are offloading sending images to AWS instead of you doing it yourself.
        0xFACEFEED 8y ago
        Deletions for one. You'll need a background job with retry logic.
        scarface74 8y ago
        Why a background job? Just delete the object when you delete the record.
    - bencoder 8y ago
      You've skipped over quite a lot. - learning about and understanding S3 in the first place. - Billing and AWS account management (possibly requiring agreement/permission from others) - maintaining credentials securely for your app to access S3. - bug fixing (for e.g. it's easy to get the permissions wrong when you upload so that it's not visible) Since you're already storing the URL in your dB, why not just store the binary data you have? I don't think you're wrong about using something like S3 being a better solution, but it's not quite as simple as "issuing one command"
      - scarface74 8y ago
        - there is literally one command and a quick Google search. - he brought up S3, I am assuming he’s already on AWS - you don’t have to. The role attached to your EC2 instance will have permission. If he is already on AWS, hopefully he knows that. - From the requirements it seems like the images should be publicly accessible. If not, when you create a bucket, it’s by default private. Yes if he wants to limit access he would have to either stream from S3 to the client - I had to do that it was a quick Google search - or generate presigned URL. Why not store binary data in the DB? It makes backups larger. That being said, SQL Server has the filestream type that does in fact store the binary data as a separate file on disk, but gets treated as if it is part of the database for backup and restore purposes.
basetop 8y ago
If you don't have to worry about scale and you are sure your user base is going to remain small and stable, then you are fine. If you expect it to grow, then you should look into dumping your files on a file server. This way you can scale, backups of your database and file server can be done quickly, independently or in parallel and it allows for better use of network resources ( your app/server/etc can distribute requests to both db and file server instead of having one large request from the db). At the end of the day, if it works and you are aware of the business needs, that's all that matters.
manigandham 8y ago
That means you also need to serve those images yourself and even the smallest HTTP server/route is more work to maintain instead of just serving directly from the URL you get with S3.