6 ms·

Yet for some reason I never hear about people complaining that some idiot chose a file system to be the database for their application . I do hear quite often,

Yet for some reason I never hear about people complaining that some idiot chose a file system to be the database for their application . I do hear quite often, disdain for the choice of MongoDB.

madsbuch 4y ago
That's on you, it happens quite often :)
- dragontamer 4y ago
  I think most people know that filesystems make for poor databases. However, most people don't seem to realize that Excel spreadsheets are rather poor databases... they're the _real_ issue.
  - dsr_ 4y ago
    There are lots of situations where filesystems are good databases. The problem is that relational databases are flexible in ways that filesystems are not, so when your requirements change, a relational database is likely to be able to support the change whereas the filesystem may not. Excel spreadsheets are terrible as databases and are also terrible as a file format for sending database records.
    - dragontamer 4y ago
      Filesystems already fail at step#1 of a database, which is concurrent read / concurrent writes. Its pretty obscure and arcane for the rules of read/write ordering on Linux/Windows systems and filesystems. You've got FILE* fwrites interacting with mmap regions, and locks don't exactly do what you think in Linux. In both Linux / Windows, you end up needing to create a Filesystem mutex of some kind (Linux flock, Windows Mutex). Then, and only then, is it really safe to read or write to a file.
      - whatshisface 4y ago
        That complexity only exists if you create it. The regular read and write calls are serialized by the operating system.
        dragontamer 4y ago
        I'm sure you're familiar with the ATM example? Lets say Bob's money in an ATM is being held in your "database" (be it a file, or SQL database, or whatever). Thread#1: Depositing money -------------- int bob_balance = readfile(); bob_balance += 100; // Add 100 to Bob's account. writefile(bob_balance); Thread#2: Withdrawing money ------------- int bob_balance = readfile(); bob_balance -= 100; // Add 100 to Bob's account. writefile(bob_balance); ------------ Lets say Bob started with X-dollars. What is Bob's value of his account after Thread#1 and Thread#2 run concurrently? More than likely, Bob's account is either X, X-100, or X+100. But under the worst case scenario, a full corruption of Bob's account value could happen. If Bob's account straddles across a filesystem boundary, like across 4096 hard drive cluster or something, then the "writefile" itself could be corrupted and interleaved between the two threads.
        whatshisface 4y ago
        Most of Linux's userspace uses files as databases. There are a lot of cases where you won't ever have two of the same process running at once. One example is those that bind a specific port.
        dragontamer 4y ago
        People are talking about MongoDB, PostgreSQL, MySQL and the like in this thread of discussion. And suddenly you're talking about local userspace programs on Linux? There's a web-bias in the discussion. Database here is innately concurrent, such as when you and I submit a comment to this webpage (Hacker News), the ordering of our submissions are ill specified and effectively random. A proper database organizes the data (ex: PostgreSQL) so that its in a form that's well defined. This is true for even the smallest of web servers. Concurrency and concurrent access is fundamental to web applications today. I know PHP has access to raw files if you ever wanted them, but its definitely a dangerous proposition to be doing raw file access on web apps. There's just too much risk of data corruption.
dathinab 4y ago
> I do hear quite often, disdain for the choice of MongoDB. I guess a lot of people have been bitten really painfully by MongoDB misbehaving int the past. At the same time people generally understand that for most use-cases the fs isn't a good database and if they use it anyway they put the blame on themself as they did knew better. Through I mean I have seen projects use the filesystem as a database successfully. Mainly in context of some initially ad-hoc versions of special purpose tooling written in bash or similar which worked surprisingly well for initial usage and long term anyway had to be rewritten (then normally not using bash and often using sqlite in rare cases postgres).

Yet for some reason I never hear about people complaining that some idiot chose a file system to be the database for their application . I do hear quite often, disdain for the choice of MongoDB.

madsbuch 4y ago

That's on you, it happens quite often :)

dragontamer 4y ago
I think most people know that filesystems make for poor databases. However, most people don't seem to realize that Excel spreadsheets are rather poor databases... they're the _real_ issue.
- dsr_ 4y ago
  There are lots of situations where filesystems are good databases. The problem is that relational databases are flexible in ways that filesystems are not, so when your requirements change, a relational database is likely to be able to support the change whereas the filesystem may not. Excel spreadsheets are terrible as databases and are also terrible as a file format for sending database records.
  - dragontamer 4y ago
    Filesystems already fail at step#1 of a database, which is concurrent read / concurrent writes. Its pretty obscure and arcane for the rules of read/write ordering on Linux/Windows systems and filesystems. You've got FILE* fwrites interacting with mmap regions, and locks don't exactly do what you think in Linux. In both Linux / Windows, you end up needing to create a Filesystem mutex of some kind (Linux flock, Windows Mutex). Then, and only then, is it really safe to read or write to a file.
    - whatshisface 4y ago
      That complexity only exists if you create it. The regular read and write calls are serialized by the operating system.
      - dragontamer 4y ago
        I'm sure you're familiar with the ATM example? Lets say Bob's money in an ATM is being held in your "database" (be it a file, or SQL database, or whatever). Thread#1: Depositing money -------------- int bob_balance = readfile(); bob_balance += 100; // Add 100 to Bob's account. writefile(bob_balance); Thread#2: Withdrawing money ------------- int bob_balance = readfile(); bob_balance -= 100; // Add 100 to Bob's account. writefile(bob_balance); ------------ Lets say Bob started with X-dollars. What is Bob's value of his account after Thread#1 and Thread#2 run concurrently? More than likely, Bob's account is either X, X-100, or X+100. But under the worst case scenario, a full corruption of Bob's account value could happen. If Bob's account straddles across a filesystem boundary, like across 4096 hard drive cluster or something, then the "writefile" itself could be corrupted and interleaved between the two threads.
        whatshisface 4y ago
        Most of Linux's userspace uses files as databases. There are a lot of cases where you won't ever have two of the same process running at once. One example is those that bind a specific port.
        dragontamer 4y ago
        People are talking about MongoDB, PostgreSQL, MySQL and the like in this thread of discussion. And suddenly you're talking about local userspace programs on Linux? There's a web-bias in the discussion. Database here is innately concurrent, such as when you and I submit a comment to this webpage (Hacker News), the ordering of our submissions are ill specified and effectively random. A proper database organizes the data (ex: PostgreSQL) so that its in a form that's well defined. This is true for even the smallest of web servers. Concurrency and concurrent access is fundamental to web applications today. I know PHP has access to raw files if you ever wanted them, but its definitely a dangerous proposition to be doing raw file access on web apps. There's just too much risk of data corruption.

dathinab 4y ago

> I do hear quite often, disdain for the choice of MongoDB. I guess a lot of people have been bitten really painfully by MongoDB misbehaving int the past. At the same time people generally understand that for most use-cases the fs isn't a good database and if they use it anyway they put the blame on themself as they did knew better. Through I mean I have seen projects use the filesystem as a database successfully. Mainly in context of some initially ad-hoc versions of special purpose tooling written in bash or similar which worked surprisingly well for initial usage and long term anyway had to be rewritten (then normally not using bash and often using sqlite in rare cases postgres).