4 ms·

Am I missing something or does this not address the main issue the original article raised: The documentation is simply incorrect. It claims to support SNAPSHOT

Am I missing something or does this not address the main issue the original article raised: The documentation is simply incorrect. It claims to support SNAPSHOT ISOLATION but does not. The company knows this and even this article says the behaviour "is totally expected". Seems like the first response should be to fix the docs and not claim capabilities beyond what's implemented. (Also it was pretty clear from the original article that corrupted data meant the balances were incorrect, not that the file was corrupted like a bad checksum.)

numbsafari 11y ago
Yeah, it's pretty clear that OP missed the point, especially in that his recommended solution is to use lock hints. They really should just update their documentation.
fipar 11y ago
Disclaimer: I work for Percona. The docs are on the galeracluster.com page, which is owned and maintained by another company, so there's no way we could fix those. A staff member from this company (And one of the Galera authors) replied on the original 'Call me maybe' post indicating they would fix the docs, though. I think the 'corruption vs inconsistency' debate could seem as nitpicking, but anybody who has been working long enough on databases has a very specific concept for each word, and, given transaction processing (distributed or otherwise) is such a complex topic, it does not help to use the wrong terminology.
- MichaelGG 11y ago
  Perhaps update the article to make a note that you don't control that site's docs? It wasn't immediately apparent that there's yet another party involved. I'd suggest that the tone of the article be very clear that the problem described is accurate and isn't trying to be downplayed. Suggesting that Aphyr write another "FUD" article comes across as dismissive of the issue. And that'll make people read more into the corruption/inconsistent comment. (Unfair as it may be.)
- sergiosgc 11y ago
  Corruption is read by the database people as: "Your data is unreadable"; Inconsistency as: "Your data is wrong". The article implies that inconsistent is better than corrupt, and that mixing up the terms paints a worse picture. The assumption is flawed. Inconsistent is as bad or worse than corrupted. It is a silent failure, and silent failures are worse than visible ones.
  - bsg75 11y ago
    What database people? Maybe those dealing with low value, fluff data in their attempt to be the next social media breakout, but for those of us who work with data containing anything with real business value, inconsistent == corrupt. No other definition is sufficient.
    - sergiosgc 11y ago
      Did you actually read my comment all the way through?
      - bsg75 11y ago
        It appears I misread it.
  - bakhy 11y ago
    i'm kinda surprised by the prevalence of that attitude. in theory, yes, you are right. but that's really a bit of programmer purism. in practice, a DB that stops working can mean a sudden stop in doing business, which is a huge catastrophe. money lost every second. inconsistency is horrible, and theoretically the same thing, you are right! but it's something you might be able to fix, without the whole system failing, with maybe just a limited number of people knowing what happened and not the whole company and/or all customers. this is kinda fucked up, but i think it counts. (and PS, accountants find these errors. it's their job, and they have experience enough not to trust any proof of correctness, and check everything twice.)
    - tptacek 11y ago
- pgaddict 11y ago
  So where are the definitions of those terms, actually? The article merely complains that Tkachenko's personal definitions of those two terms do not match Aphyr's, but that's merely a personal opinion. I'd argue that "corrupted" is usually used in cases when the data get messed up due to external circumstances (say, disk error corrupting a page of data) or because of a clear bug (say, writing to incorrect place), while "consistency" is usually used for data violating some invariants (foreign keys, check constraints, unique constraints, or global constraints like "total sum of money is X"). But in the end this is utterly irrelevant because the database failed to provide correct data (to the extend implied by the advertised guarantees like isolation level, for example). For the user the end result is pretty much the same. And actually, corrupted data tend to be detected much sooner, because when you can't read the disk pace you know there's an error and can address it. The inconsistency is often creepy and takes very long until someone spots that.
  - fipar 11y ago
    I don't know if there is some official definition, but my experience talking to others working with databases daily tells me they have the same concepts as I do, and a 5 second google search on "data corruption" followed by "data consistency" yield wikipedia articles in the top 3 results for both queries, showing concepts that, also, match those that I have.
    - dsp1234 11y ago
      If I sat down at one of our database consoles and fired off a few queries and saw that hundreds of thousands of dollars had appeared out of nowhere, I'd tell my boss that the database was corrupt. However, for the purposes of Aphyr's original article, it's a moot point specifically because Aphyr replaced the symbol with the substance[0]. He specifically states the conclusion, and in detail explains why it happened and what the ramifications are. Any quibbling about the word corruption is a distraction, because the underlying fault of the system is still present. The system claims to support snapshot isolation, but does not, and the result of that fact is that data will not be correct for users expecting snapshot isolation. [0] - http://lesswrong.com/lw/nv/replace_the_symbol_with_the_substance/ http://lesswrong.com/lw/nv/replace_the_symbol_with_the_subst...
fweespeech 11y ago
Galera documentation: > It is recommended that you avoid using SERIALIZABLE in Galera Cluster. Aphyr: > SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE Eh.

numbsafari 11y ago

Yeah, it's pretty clear that OP missed the point, especially in that his recommended solution is to use lock hints. They really should just update their documentation.

fipar 11y ago

Disclaimer: I work for Percona. The docs are on the galeracluster.com page, which is owned and maintained by another company, so there's no way we could fix those. A staff member from this company (And one of the Galera authors) replied on the original 'Call me maybe' post indicating they would fix the docs, though. I think the 'corruption vs inconsistency' debate could seem as nitpicking, but anybody who has been working long enough on databases has a very specific concept for each word, and, given transaction processing (distributed or otherwise) is such a complex topic, it does not help to use the wrong terminology.

MichaelGG 11y ago
Perhaps update the article to make a note that you don't control that site's docs? It wasn't immediately apparent that there's yet another party involved. I'd suggest that the tone of the article be very clear that the problem described is accurate and isn't trying to be downplayed. Suggesting that Aphyr write another "FUD" article comes across as dismissive of the issue. And that'll make people read more into the corruption/inconsistent comment. (Unfair as it may be.)
sergiosgc 11y ago
Corruption is read by the database people as: "Your data is unreadable"; Inconsistency as: "Your data is wrong". The article implies that inconsistent is better than corrupt, and that mixing up the terms paints a worse picture. The assumption is flawed. Inconsistent is as bad or worse than corrupted. It is a silent failure, and silent failures are worse than visible ones.
- bsg75 11y ago
  What database people? Maybe those dealing with low value, fluff data in their attempt to be the next social media breakout, but for those of us who work with data containing anything with real business value, inconsistent == corrupt. No other definition is sufficient.
  - sergiosgc 11y ago
    Did you actually read my comment all the way through?
    - bsg75 11y ago
      It appears I misread it.
- bakhy 11y ago
  i'm kinda surprised by the prevalence of that attitude. in theory, yes, you are right. but that's really a bit of programmer purism. in practice, a DB that stops working can mean a sudden stop in doing business, which is a huge catastrophe. money lost every second. inconsistency is horrible, and theoretically the same thing, you are right! but it's something you might be able to fix, without the whole system failing, with maybe just a limited number of people knowing what happened and not the whole company and/or all customers. this is kinda fucked up, but i think it counts. (and PS, accountants find these errors. it's their job, and they have experience enough not to trust any proof of correctness, and check everything twice.)
  - tptacek 11y ago
pgaddict 11y ago
So where are the definitions of those terms, actually? The article merely complains that Tkachenko's personal definitions of those two terms do not match Aphyr's, but that's merely a personal opinion. I'd argue that "corrupted" is usually used in cases when the data get messed up due to external circumstances (say, disk error corrupting a page of data) or because of a clear bug (say, writing to incorrect place), while "consistency" is usually used for data violating some invariants (foreign keys, check constraints, unique constraints, or global constraints like "total sum of money is X"). But in the end this is utterly irrelevant because the database failed to provide correct data (to the extend implied by the advertised guarantees like isolation level, for example). For the user the end result is pretty much the same. And actually, corrupted data tend to be detected much sooner, because when you can't read the disk pace you know there's an error and can address it. The inconsistency is often creepy and takes very long until someone spots that.
- fipar 11y ago
  I don't know if there is some official definition, but my experience talking to others working with databases daily tells me they have the same concepts as I do, and a 5 second google search on "data corruption" followed by "data consistency" yield wikipedia articles in the top 3 results for both queries, showing concepts that, also, match those that I have.
  - dsp1234 11y ago
    If I sat down at one of our database consoles and fired off a few queries and saw that hundreds of thousands of dollars had appeared out of nowhere, I'd tell my boss that the database was corrupt. However, for the purposes of Aphyr's original article, it's a moot point specifically because Aphyr replaced the symbol with the substance[0]. He specifically states the conclusion, and in detail explains why it happened and what the ramifications are. Any quibbling about the word corruption is a distraction, because the underlying fault of the system is still present. The system claims to support snapshot isolation, but does not, and the result of that fact is that data will not be correct for users expecting snapshot isolation. [0] - http://lesswrong.com/lw/nv/replace_the_symbol_with_the_substance/ http://lesswrong.com/lw/nv/replace_the_symbol_with_the_subst...

fweespeech 11y ago

Galera documentation: > It is recommended that you avoid using SERIALIZABLE in Galera Cluster. Aphyr: > SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE Eh.