5 ms·

I tried and failed to read up on what pragma queries are. Would you like to elaborate? This wasn't able to inform me https://www.sqlite.org/pragma.html https://

I tried and failed to read up on what pragma queries are. Would you like to elaborate? This wasn't able to inform me https://www.sqlite.org/pragma.html https://www.sqlite.org/pragma.html

carterschonwald 4y ago
I believe he means build flag cpp pragmas
- warmwaffles 4y ago
  No there are some PRAGMA calls that people typically don't set or bother to adjust from the defaults that really improve performance. Some are setting the `journal_mode`, `cache_spill`, and `cache_size`.
bob1029 4y ago
These are the specific pragmas I am referring to: https://www.sqlite.org/pragma.html#pragma_synchronous https://www.sqlite.org/pragma.html#pragma_synchronous https://www.sqlite.org/pragma.html#pragma_journal_mode https://www.sqlite.org/pragma.html#pragma_journal_mode I set them to "NORMAL" and "WAL" respectively.
- gpderetta 4y ago
  > A transaction committed in WAL mode with synchronous=NORMAL might roll back following a power loss or system crash That's a pretty big failure to preserve durability.
  - zepolen 4y ago
    Yes and the speedup is not that great, I hear piping changes to /dev/null is even faster.
    - Filligree 4y ago
      My computer rolling back a couple seconds on crashes is, actually, perfectly fine! Furthermore, I think most personal computer owners would agree. The near impossibility of achieving ACI semantics on modern filesystems (other than ZFS) is one of my biggest pet peeves with how data is being handled. Servers may need ACID, but practically no home computers do.
  - pantsforbirds 4y ago
    I mean if you read the full quote: > When synchronous is NORMAL (1), the SQLite database engine will still sync at the most critical moments, but less often than in FULL mode. There is a very small (though non-zero) chance that a power failure at just the wrong time could corrupt the database in journal_mode=DELETE on an older filesystem. WAL mode is safe from corruption with synchronous=NORMAL, and probably DELETE mode is safe too on modern filesystems. WAL mode is always consistent with synchronous=NORMAL, but WAL mode does lose durability. A transaction committed in WAL mode with synchronous=NORMAL might roll back following a power loss or system crash. Transactions are durable across application crashes regardless of the synchronous setting or journal mode. The synchronous=NORMAL setting is a good choice for most applications running in WAL mode. It seems extremely reasonable to use WAL with synchronous=NORMAL
    - gpderetta 4y ago
      You might roll back committed transactions in case of a system crash. It might be acceptable for some applications, but I wouldn't certainly call it extremely reasonable. If you do not care about system crashes, why sync at all?
      - wongarsu 4y ago
        I've written plenty systems that would be fine with a valid, consistent but maybe slightly older state (for example when the db describes both work to be done and work results, or the db otherwise implicitly records where to resume work). On the other hand there are obvious examples where this is unacceptable, say recording the transactions of a cash register.
        gpderetta 4y ago
        I have done as well! My concern is the assertion that this behavior is reasonable for the vast majority of applications.
      - JohnBooty 4y ago
        If you do not care about system crashes, why sync at all? Well, you have to sync sometime if you care about persist your data at all, right? And there are scads of use cases where you want to: - persist data - enforce some sort of schema (even though SQLite is lax here) - query/join it in SQL-y ways - are willing to trade ACID-compliant journaled durability in exchange for performance Cache stores are maybe the most obvious use case. This is roughly how Redis is configured out of the box after all (sans the SQL) IIRC. Not ideal if your cache is trashed but also, not a huge deal. There are also a lot of logging/timeseries type applications where it's just not a big deal if you lose, say, a few minutes of data per year.
  - chasil 4y ago
    WAL mode also cannot preserve transactions across multiple ATTACHed database files. Transactions on WAL mode databases must use only a single database file to maintain ACID.
- password4321 4y ago
  A bit more discussion yesterday: https://news.ycombinator.com/item?id=34247738#34250043 https://news.ycombinator.com/item?id=34247738#34250043

I tried and failed to read up on what pragma queries are. Would you like to elaborate? This wasn't able to inform me https://www.sqlite.org/pragma.html https://www.sqlite.org/pragma.html

carterschonwald 4y ago

I believe he means build flag cpp pragmas

warmwaffles 4y ago
No there are some PRAGMA calls that people typically don't set or bother to adjust from the defaults that really improve performance. Some are setting the `journal_mode`, `cache_spill`, and `cache_size`.

bob1029 4y ago

These are the specific pragmas I am referring to: https://www.sqlite.org/pragma.html#pragma_synchronous https://www.sqlite.org/pragma.html#pragma_synchronous https://www.sqlite.org/pragma.html#pragma_journal_mode https://www.sqlite.org/pragma.html#pragma_journal_mode I set them to "NORMAL" and "WAL" respectively.

gpderetta 4y ago
> A transaction committed in WAL mode with synchronous=NORMAL might roll back following a power loss or system crash That's a pretty big failure to preserve durability.
- zepolen 4y ago
  Yes and the speedup is not that great, I hear piping changes to /dev/null is even faster.
  - Filligree 4y ago
    My computer rolling back a couple seconds on crashes is, actually, perfectly fine! Furthermore, I think most personal computer owners would agree. The near impossibility of achieving ACI semantics on modern filesystems (other than ZFS) is one of my biggest pet peeves with how data is being handled. Servers may need ACID, but practically no home computers do.
- pantsforbirds 4y ago
  I mean if you read the full quote: > When synchronous is NORMAL (1), the SQLite database engine will still sync at the most critical moments, but less often than in FULL mode. There is a very small (though non-zero) chance that a power failure at just the wrong time could corrupt the database in journal_mode=DELETE on an older filesystem. WAL mode is safe from corruption with synchronous=NORMAL, and probably DELETE mode is safe too on modern filesystems. WAL mode is always consistent with synchronous=NORMAL, but WAL mode does lose durability. A transaction committed in WAL mode with synchronous=NORMAL might roll back following a power loss or system crash. Transactions are durable across application crashes regardless of the synchronous setting or journal mode. The synchronous=NORMAL setting is a good choice for most applications running in WAL mode. It seems extremely reasonable to use WAL with synchronous=NORMAL
  - gpderetta 4y ago
    You might roll back committed transactions in case of a system crash. It might be acceptable for some applications, but I wouldn't certainly call it extremely reasonable. If you do not care about system crashes, why sync at all?
    - wongarsu 4y ago
      I've written plenty systems that would be fine with a valid, consistent but maybe slightly older state (for example when the db describes both work to be done and work results, or the db otherwise implicitly records where to resume work). On the other hand there are obvious examples where this is unacceptable, say recording the transactions of a cash register.
      - gpderetta 4y ago
        I have done as well! My concern is the assertion that this behavior is reasonable for the vast majority of applications.
    - JohnBooty 4y ago
      If you do not care about system crashes, why sync at all? Well, you have to sync sometime if you care about persist your data at all, right? And there are scads of use cases where you want to: - persist data - enforce some sort of schema (even though SQLite is lax here) - query/join it in SQL-y ways - are willing to trade ACID-compliant journaled durability in exchange for performance Cache stores are maybe the most obvious use case. This is roughly how Redis is configured out of the box after all (sans the SQL) IIRC. Not ideal if your cache is trashed but also, not a huge deal. There are also a lot of logging/timeseries type applications where it's just not a big deal if you lose, say, a few minutes of data per year.
- chasil 4y ago
  WAL mode also cannot preserve transactions across multiple ATTACHed database files. Transactions on WAL mode databases must use only a single database file to maintain ACID.
password4321 4y ago
A bit more discussion yesterday: https://news.ycombinator.com/item?id=34247738#34250043 https://news.ycombinator.com/item?id=34247738#34250043