3 ms·

Why did we need the databases in the first place?

by Ace17 9y ago

peterwwillis 9y ago
So you have a post on a forum. You can just save that to disk by itself and access it later. And here come the problems. How many posts are there? Filesystems commonly have problems dealing with too many files, like inode limits, slowness reading large directories or in deep paths, slowness traversing large trees, path name max size, etc. Now you have to engineer a bunch of hacks around the filesystem, or try to find a perfect filesystem and be stuck with it. What if you want to search posts? Well, you have to open and read all the posts. But what if you want to speed that up, like with a word index? Congrats, you're about to write a database/search engine. What if dynamic processing is too resource-intense and you want to serve static content? Once you apply transforms to the user content and create your static resource, you get to decide if you keep the original content, because it's not useful to you just sitting on disk. What if you want to merge, upgrade, convert, export, etc parts of the data? You have a big job ahead of you if you didn't keep the original content, and if you did, I hope you kept metadata to make this task easier. What if you want to spread the load, store records efficiently, control I/O methods and cache, support strong locking, platform independence, and storage backend independence? Databases are a powerful and simple tool to add features to an application and make it easier and more efficient to process data. If you don't want app features, data processing features, platform independence, etc then don't use a database.
- harperlee 9y ago
  What about the converse question? Why does the filesystem not provide indexing, tabulation, relationships among document, archival tools, tools for merging two diverged versions of a folder tree, etc.? (Is there one that does it? I don't know!). E.g. Why can't I store a slidedeck by slides and show a "document" that is composed of transcluded bits and pieces? I think pondering alternate systems from time to time is valuable.

Why did we need the databases in the first place?

by Ace17 9y ago

Why did we need the databases in the first place?

So you have a post on a forum. You can just save that to disk by itself and access it later. And here come the problems. How many posts are there? Filesystems commonly have problems dealing with too many files, like inode limits, slowness reading large directories or in deep paths, slowness traversing large trees, path name max size, etc. Now you have to engineer a bunch of hacks around the filesystem, or try to find a perfect filesystem and be stuck with it. What if you want to search posts? Well, you have to open and read all the posts. But what if you want to speed that up, like with a word index? Congrats, you're about to write a database/search engine. What if dynamic processing is too resource-intense and you want to serve static content? Once you apply transforms to the user content and create your static resource, you get to decide if you keep the original content, because it's not useful to you just sitting on disk. What if you want to merge, upgrade, convert, export, etc parts of the data? You have a big job ahead of you if you didn't keep the original content, and if you did, I hope you kept metadata to make this task easier. What if you want to spread the load, store records efficiently, control I/O methods and cache, support strong locking, platform independence, and storage backend independence? Databases are a powerful and simple tool to add features to an application and make it easier and more efficient to process data. If you don't want app features, data processing features, platform independence, etc then don't use a database.

harperlee 9y ago
What about the converse question? Why does the filesystem not provide indexing, tabulation, relationships among document, archival tools, tools for merging two diverged versions of a folder tree, etc.? (Is there one that does it? I don't know!). E.g. Why can't I store a slidedeck by slides and show a "document" that is composed of transcluded bits and pieces? I think pondering alternate systems from time to time is valuable.