5 ms·

Firefox uses SQLite for localStorage. When Firefox is force quit, the database becomes corrupted. https://bugzilla.mozilla.org/show_bug.cgi?id=1341070 https://b

Firefox uses SQLite for localStorage. When Firefox is force quit, the database becomes corrupted. https://bugzilla.mozilla.org/show_bug.cgi?id=1341070 https://bugzilla.mozilla.org/show_bug.cgi?id=1341070 This bug has occurred to me personally multiple times. Does it really have advantages over a flat file?

edoceo 5y ago
if you're corrupting sqlite files you're for sure going to corrupt other formats in a force quit situation.
astrobe_ 5y ago
This is quite surprising. SQLite is also used in embedded where you don't have comparatively civil "forced quits" but also straight power offs. SQLite seems to do everything it can to handle this gracefully.
throwawayboise 5y ago
I use Firefox almost exclusively and have never encountered this. Whatever capabilities SQLLite has to recover corrupted or inconsistent storage, it certainly exceeds what you get with flat files (none, unless you implement it yourself).
- btilly 5y ago
  Per that bug report, you are likely to encounter it at https://codesandbox.io/s/sentry-js-sdk-unhandled-promise-fuq2h?file=/src/index.js https://codesandbox.io/s/sentry-js-sdk-unhandled-promise-fuq.... Which suggests an application error to me. I'd give decent odds to the real problem being some bounds check or pointer writing to the wrong place. To prove it, I'd instrument SQLite to log every single query that it was asked to do, then try to run those queries outside of the application. If the queries work fine, then the copy of SQLite in the application is being corrupted somehow. In which case it really isn't fair to blame SQLite.
  - mcdevilkiller 5y ago
    What's getting corrupted is Firefox's own SQLite DB.
    - btilly 5y ago
      I'm perfectly aware of that. The question is how it is getting corrupted. Is it that SQLite has a bug? Or the copy of SQLite in Firefox is getting corrupted first? Given the complexity and relative levels of testing of those two products, my bet is that the problem is Firefox.
NicoJuicy 5y ago
Sqlite is certified for use in planes. This requires a high level of tests. I think the odds are likely that Firefox is doing something unsuspected. https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html - Four independently developed test harnesses - 100% branch test coverage in an as-deployed configuration - Millions and millions of test cases - Out-of-memory tests - I/O error tests - Crash and power loss tests - Fuzz tests - Boundary value tests - Disabled optimization tests - Regression tests - Malformed database tests - Extensive use of assert() and run-time checks - Valgrind analysis - Undefined behavior checks - Checklists
- easton 5y ago
  "Valgrind is perhaps the most amazing and useful developer tool in the world." It seems that about once a month I read something that makes me like SQLite even more. I guess that they like Valgrind as much as I do is the reason this month.
  - unmole 5y ago
    Valgrind is fine if you can live with the false positives, limited stack validation and execution slowdown. In a development environment, AddressSanitizer is a far better choice.
- throwaway2037 5y ago
  I am trolling on the person you to whom you replied... <<This bug has occurred to me personally multiple times.>> Hello, I am AirBus engineer (not really, only pretend). Does my plane crash? No, it does not! Dear Random Internet Person who complains that open source software is broken. Are you joking? We are talking about SQLite!!?? SQLite has whole pages dedicated to "why you should NOT use SQLite". How many open source projects are /so/ good they can do this? Incredible! The test coverage feels like NASA.
ravenstine 5y ago
Why would that not happen to any other form of flat file storage? When you force quit a process, unless there's some nuance I'm unaware of, you're not only bypassing any graceful cleanup but there could be an FS operation in progress. Although I'm certain there are mitigation techniques, I would never expect a process to guarantee no file corruption when it's forcefully quit. Interestingly enough, I don't think I've ever had this issue. I've force quit Firefox many times in the past (not often, but I'm sure I've done it countless times in the 10+ years I've been using it) and I still have localStorage data intact. There's old crap in StackEdit.io, which uses localStorage, that I haven't touched in years. (no, I have neither that service or Firefox itself set up to sync with anything) EDIT: Maybe it's OS specific? I've been on macOS, and maybe there's something about other file systems where corruption is more likely to happen with a force-quit? Just throwing darts here.
- asimpletune 5y ago
  I believe for a proper implementation it would be done transactionally. So if the program crashed then that record wouldn’t be written but the DB wouldn’t be corrupted.
nix0n 5y ago
I force-quit Firefox regularly (easiest way to get my tabs to stay open), on both Linux and Windows. I've never seen any error messages from Firefox suggesting that anything has been corrupted. Are there other symptoms that I would expect to see, if this corruption is happening?
- crazypython 5y ago
  Some sites break due to not expecting the exception being thrown– deviantart.com won't work, for instance.

edoceo 5y ago

if you're corrupting sqlite files you're for sure going to corrupt other formats in a force quit situation.

astrobe_ 5y ago

This is quite surprising. SQLite is also used in embedded where you don't have comparatively civil "forced quits" but also straight power offs. SQLite seems to do everything it can to handle this gracefully.

throwawayboise 5y ago

I use Firefox almost exclusively and have never encountered this. Whatever capabilities SQLLite has to recover corrupted or inconsistent storage, it certainly exceeds what you get with flat files (none, unless you implement it yourself).

btilly 5y ago
Per that bug report, you are likely to encounter it at https://codesandbox.io/s/sentry-js-sdk-unhandled-promise-fuq2h?file=/src/index.js https://codesandbox.io/s/sentry-js-sdk-unhandled-promise-fuq.... Which suggests an application error to me. I'd give decent odds to the real problem being some bounds check or pointer writing to the wrong place. To prove it, I'd instrument SQLite to log every single query that it was asked to do, then try to run those queries outside of the application. If the queries work fine, then the copy of SQLite in the application is being corrupted somehow. In which case it really isn't fair to blame SQLite.
- mcdevilkiller 5y ago
  What's getting corrupted is Firefox's own SQLite DB.
  - btilly 5y ago
    I'm perfectly aware of that. The question is how it is getting corrupted. Is it that SQLite has a bug? Or the copy of SQLite in Firefox is getting corrupted first? Given the complexity and relative levels of testing of those two products, my bet is that the problem is Firefox.

NicoJuicy 5y ago

Sqlite is certified for use in planes. This requires a high level of tests. I think the odds are likely that Firefox is doing something unsuspected. https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html - Four independently developed test harnesses - 100% branch test coverage in an as-deployed configuration - Millions and millions of test cases - Out-of-memory tests - I/O error tests - Crash and power loss tests - Fuzz tests - Boundary value tests - Disabled optimization tests - Regression tests - Malformed database tests - Extensive use of assert() and run-time checks - Valgrind analysis - Undefined behavior checks - Checklists

easton 5y ago
"Valgrind is perhaps the most amazing and useful developer tool in the world." It seems that about once a month I read something that makes me like SQLite even more. I guess that they like Valgrind as much as I do is the reason this month.
- unmole 5y ago
  Valgrind is fine if you can live with the false positives, limited stack validation and execution slowdown. In a development environment, AddressSanitizer is a far better choice.
throwaway2037 5y ago
I am trolling on the person you to whom you replied... <<This bug has occurred to me personally multiple times.>> Hello, I am AirBus engineer (not really, only pretend). Does my plane crash? No, it does not! Dear Random Internet Person who complains that open source software is broken. Are you joking? We are talking about SQLite!!?? SQLite has whole pages dedicated to "why you should NOT use SQLite". How many open source projects are /so/ good they can do this? Incredible! The test coverage feels like NASA.

ravenstine 5y ago

Why would that not happen to any other form of flat file storage? When you force quit a process, unless there's some nuance I'm unaware of, you're not only bypassing any graceful cleanup but there could be an FS operation in progress. Although I'm certain there are mitigation techniques, I would never expect a process to guarantee no file corruption when it's forcefully quit. Interestingly enough, I don't think I've ever had this issue. I've force quit Firefox many times in the past (not often, but I'm sure I've done it countless times in the 10+ years I've been using it) and I still have localStorage data intact. There's old crap in StackEdit.io, which uses localStorage, that I haven't touched in years. (no, I have neither that service or Firefox itself set up to sync with anything) EDIT: Maybe it's OS specific? I've been on macOS, and maybe there's something about other file systems where corruption is more likely to happen with a force-quit? Just throwing darts here.

asimpletune 5y ago
I believe for a proper implementation it would be done transactionally. So if the program crashed then that record wouldn’t be written but the DB wouldn’t be corrupted.

nix0n 5y ago

I force-quit Firefox regularly (easiest way to get my tabs to stay open), on both Linux and Windows. I've never seen any error messages from Firefox suggesting that anything has been corrupted. Are there other symptoms that I would expect to see, if this corruption is happening?

crazypython 5y ago
Some sites break due to not expecting the exception being thrown– deviantart.com won't work, for instance.