3 ms·

Deleted.

actionfromafar 3y ago
I don’t understand how any of that follows. Databases are as much about locking and data structures as about persistence.
- austin-cheney 3y ago
  Deleted.
  - rewmie 3y ago
    > Databases are a means to query and index data with greater performance than without. I don't think you understand your own point. The querying, indexing, and performance bit are tied to the data structures used internally by the database, not the technology used to persist data.
  - c0balt 3y ago
    Well, that's one hell of an undersell of what a database like PostgreSQL, MSSQL or MongoDB does. It's not just that people "can't write priginal applications" but that in fact people shouldn't always write their own bespoke single-purpose databases for each application. Getting ACID, MVCC, efficient storage, indexing and backups etc. at the same time is hard, really damn hard. You might get over some of them, e.g., efficient storage, with hardware but there's no free lunch on those topics. A database is like using a library: You can always write it from scratch (and sometimes you should even) but in 99% percent of cases you should rely on the tried, battle tested existing solution.
- nivertech 3y ago
  Locking and data structures are more or less a solved problem. Persistence is not yet solved. Our programming models are currently heavily influenced by the way we store and query data and the underlying registers/memory/cache/storage HW. With PRAM, we can simplify programming models using Persistence Ignorance (PI). https://deviq.com/principles/persistence-ignorance https://deviq.com/principles/persistence-ignorance
rewmie 3y ago
> Databases are automatically obsolete. A file system is enough and performance is improved by not using a database. This makes no sense at all. Databases are much more about what data structures are used internally and the high-level interfaces provided to access said data than the underlying technology used to persist data. There is also the matter of how much data can/needs to be persisted, which is not addressed at all.
- austin-cheney 3y ago
  Deleted.
  - deleted 3y ago
    [deleted]
c03 3y ago
I believe the author means database as disk persistence. But we could all host sqlite db's in /dev/shm today if we wanted.. I guess it's just excitement about persistence at speed :)
- sfink 3y ago
  The original post is deleted, but in a proper ACID database, persistence and speed are quite highly related. The DB can't truly move on until it has confirmed that a set of writes has truly hit the persistent storage, and the time required for that is vastly larger than the time required to process a transaction in volatile memory. We cheat in every possible way, notably with battery-backed NVRAM, so in actual practice the cost isn't always visible. But if you need truly persistent transactions, then fast persistent memory is a godsend. (Especially if it's big enough to put the WAL into. Though even a couple of bytes to store the last committed transaction ID can be useful.)
kreetx 3y ago
What exactly will become obsolete? (IMO) file systems are a degenerate form of databases already, with graph structure, large enough directory entries becoming b-trees etc. But even with fast storage you'd still need indexes for any access pattern different for what's encoded in the directory tree.

Deleted.

I don’t understand how any of that follows. Databases are as much about locking and data structures as about persistence.

austin-cheney 3y ago
Deleted.
- rewmie 3y ago
  > Databases are a means to query and index data with greater performance than without. I don't think you understand your own point. The querying, indexing, and performance bit are tied to the data structures used internally by the database, not the technology used to persist data.
- c0balt 3y ago
  Well, that's one hell of an undersell of what a database like PostgreSQL, MSSQL or MongoDB does. It's not just that people "can't write priginal applications" but that in fact people shouldn't always write their own bespoke single-purpose databases for each application. Getting ACID, MVCC, efficient storage, indexing and backups etc. at the same time is hard, really damn hard. You might get over some of them, e.g., efficient storage, with hardware but there's no free lunch on those topics. A database is like using a library: You can always write it from scratch (and sometimes you should even) but in 99% percent of cases you should rely on the tried, battle tested existing solution.
nivertech 3y ago
Locking and data structures are more or less a solved problem. Persistence is not yet solved. Our programming models are currently heavily influenced by the way we store and query data and the underlying registers/memory/cache/storage HW. With PRAM, we can simplify programming models using Persistence Ignorance (PI). https://deviq.com/principles/persistence-ignorance https://deviq.com/principles/persistence-ignorance

rewmie 3y ago

> Databases are automatically obsolete. A file system is enough and performance is improved by not using a database. This makes no sense at all. Databases are much more about what data structures are used internally and the high-level interfaces provided to access said data than the underlying technology used to persist data. There is also the matter of how much data can/needs to be persisted, which is not addressed at all.

austin-cheney 3y ago
Deleted.
- deleted 3y ago
  [deleted]

c03 3y ago

I believe the author means database as disk persistence. But we could all host sqlite db's in /dev/shm today if we wanted.. I guess it's just excitement about persistence at speed :)

sfink 3y ago
The original post is deleted, but in a proper ACID database, persistence and speed are quite highly related. The DB can't truly move on until it has confirmed that a set of writes has truly hit the persistent storage, and the time required for that is vastly larger than the time required to process a transaction in volatile memory. We cheat in every possible way, notably with battery-backed NVRAM, so in actual practice the cost isn't always visible. But if you need truly persistent transactions, then fast persistent memory is a godsend. (Especially if it's big enough to put the WAL into. Though even a couple of bytes to store the last committed transaction ID can be useful.)

kreetx 3y ago

What exactly will become obsolete? (IMO) file systems are a degenerate form of databases already, with graph structure, large enough directory entries becoming b-trees etc. But even with fast storage you'd still need indexes for any access pattern different for what's encoded in the directory tree.