4 ms·

> I suggested to a client that they use Sqlite3 for their company database rather than a disk based database. Sqlite is a disk based database...

capableweb 6y ago
Well, maybe not "disk based" per se. It can be stored on disk, yeah. But it can also run in memory. I think in this case it's clear that ChuckMcM meant a machine with 64GB of RAM (not disk space), therefore they are talking about having SQLite running in memory, not on disk.
- sillysaurusx 6y ago
  Even if a DB is disk-based, you can throw it in a ramdisk to get the same speedups, often. I would be curious to know if there are any drawbacks or dangers from doing that, apart from the obvious “if your box powers off, the ramdisk is gone” and “each time the box restarts, you have to copy the db to the ramdisk.” The main problem I was uneasy about was, I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop, which seemed to ensure safety in the event of a poweroff. It never broke, but I wouldn’t say it was a good idea, necessarily. Sure speeds up most things though. The benefits are pretty crazy for I/O bound programs.
  - AI_WAIFU 6y ago
    Does anyone know the risk of radiation bit flips in this case? I don't think it's an issue with ecc ram, but if it's not that could be a problem for certain applications.
    - sillysaurusx 6y ago
      Exactly! I was wondering about that. It seems like it's so rare as not to matter, but that's no guarantee it won't matter.
    - count 6y ago
      A friend of mine wrote a 'chaos engineering' tester for this specific issue: http://github.com/enferex/cme http://github.com/enferex/cme Randomly inject bit flips into a target process memory space, so you can see/test how your software handles it (if at all).
    - fomine3 6y ago
      Bit flip is always a risk even if you don't use ram disk.
  - MertsA 6y ago
    >I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop That's probably a bad idea. rsync isn't going to copy over writes in the same order they were written by the DB. Most DBs assume that when there's some crash that at least the disk would have been in a logically consistent state with respect to the order of certain writes separated by e.g. fsync and barriers.
- dmh2000 6y ago
  'The most common way to force an SQLite database to exist purely in memory is to open the database using the special filename ":memory:".'
deleted 6y ago
[deleted]

capableweb 6y ago

Well, maybe not "disk based" per se. It can be stored on disk, yeah. But it can also run in memory. I think in this case it's clear that ChuckMcM meant a machine with 64GB of RAM (not disk space), therefore they are talking about having SQLite running in memory, not on disk.

sillysaurusx 6y ago
Even if a DB is disk-based, you can throw it in a ramdisk to get the same speedups, often. I would be curious to know if there are any drawbacks or dangers from doing that, apart from the obvious “if your box powers off, the ramdisk is gone” and “each time the box restarts, you have to copy the db to the ramdisk.” The main problem I was uneasy about was, I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop, which seemed to ensure safety in the event of a poweroff. It never broke, but I wouldn’t say it was a good idea, necessarily. Sure speeds up most things though. The benefits are pretty crazy for I/O bound programs.
- AI_WAIFU 6y ago
  Does anyone know the risk of radiation bit flips in this case? I don't think it's an issue with ecc ram, but if it's not that could be a problem for certain applications.
  - sillysaurusx 6y ago
    Exactly! I was wondering about that. It seems like it's so rare as not to matter, but that's no guarantee it won't matter.
  - count 6y ago
    A friend of mine wrote a 'chaos engineering' tester for this specific issue: http://github.com/enferex/cme http://github.com/enferex/cme Randomly inject bit flips into a target process memory space, so you can see/test how your software handles it (if at all).
  - fomine3 6y ago
    Bit flip is always a risk even if you don't use ram disk.
- MertsA 6y ago
  >I would simply use the equivalent of rsync -Pa /ramdisk /disk in a loop That's probably a bad idea. rsync isn't going to copy over writes in the same order they were written by the DB. Most DBs assume that when there's some crash that at least the disk would have been in a logically consistent state with respect to the order of certain writes separated by e.g. fsync and barriers.
dmh2000 6y ago
'The most common way to force an SQLite database to exist purely in memory is to open the database using the special filename ":memory:".'

deleted 6y ago

[deleted]