4 ms·

Disclaimer: I work for Percona. The docs are on the galeracluster.com page, which is owned and maintained by another company, so there's no way we could fix th

by fipar 11y ago

Disclaimer: I work for Percona. The docs are on the galeracluster.com page, which is owned and maintained by another company, so there's no way we could fix those. A staff member from this company (And one of the Galera authors) replied on the original 'Call me maybe' post indicating they would fix the docs, though. I think the 'corruption vs inconsistency' debate could seem as nitpicking, but anybody who has been working long enough on databases has a very specific concept for each word, and, given transaction processing (distributed or otherwise) is such a complex topic, it does not help to use the wrong terminology.

MichaelGG 11y ago
Perhaps update the article to make a note that you don't control that site's docs? It wasn't immediately apparent that there's yet another party involved. I'd suggest that the tone of the article be very clear that the problem described is accurate and isn't trying to be downplayed. Suggesting that Aphyr write another "FUD" article comes across as dismissive of the issue. And that'll make people read more into the corruption/inconsistent comment. (Unfair as it may be.)
sergiosgc 11y ago
Corruption is read by the database people as: "Your data is unreadable"; Inconsistency as: "Your data is wrong". The article implies that inconsistent is better than corrupt, and that mixing up the terms paints a worse picture. The assumption is flawed. Inconsistent is as bad or worse than corrupted. It is a silent failure, and silent failures are worse than visible ones.
- bsg75 11y ago
  What database people? Maybe those dealing with low value, fluff data in their attempt to be the next social media breakout, but for those of us who work with data containing anything with real business value, inconsistent == corrupt. No other definition is sufficient.
  - sergiosgc 11y ago
    Did you actually read my comment all the way through?
    - bsg75 11y ago
      It appears I misread it.
- bakhy 11y ago
  i'm kinda surprised by the prevalence of that attitude. in theory, yes, you are right. but that's really a bit of programmer purism. in practice, a DB that stops working can mean a sudden stop in doing business, which is a huge catastrophe. money lost every second. inconsistency is horrible, and theoretically the same thing, you are right! but it's something you might be able to fix, without the whole system failing, with maybe just a limited number of people knowing what happened and not the whole company and/or all customers. this is kinda fucked up, but i think it counts. (and PS, accountants find these errors. it's their job, and they have experience enough not to trust any proof of correctness, and check everything twice.)
  - tptacek 11y ago
    An account database that makes new money out of thin air is a huge catastrophe! I honestly don't understand how this is even a live issue for debate. The "inconsistency" we're talking about is terrifying.
    - bakhy 11y ago
      they are both a catastrophe, i never implied that inconsistency is not one. i'm merely involved in semantic bickering here :) both are bad in their own way, but which is worse will depend on the case. bear in mind that programmers make mistakes all the time. they can also cause such effects. which is why we often do things in a way where incorrect data can be corrected later. particularly in anything related to accounting, where the practice has existed for centuries. i'm really not interested in claiming that one catastrophe is better than the other, but a data corruption in a database can sometimes be more problematic. edit - sorry, i just noticed i actually did say, in another thread here, that corruption is worse. i would like to take that back now :D
  - MichaelGG 11y ago
    It's really case-specific. But it's not hard to see where an error page "SQL Connection Failed" is better than not recording user's orders, mis-crediting their account, etc. At least with a hard error it's obvious there's a problem and it must be fixed immediately. With inconsistencies, you might e.g. end up with customers expecting travel reservations only to show up and find this to not be the case. By the time the accountant finds the error, the business could be irreparably harmed.
pgaddict 11y ago
So where are the definitions of those terms, actually? The article merely complains that Tkachenko's personal definitions of those two terms do not match Aphyr's, but that's merely a personal opinion. I'd argue that "corrupted" is usually used in cases when the data get messed up due to external circumstances (say, disk error corrupting a page of data) or because of a clear bug (say, writing to incorrect place), while "consistency" is usually used for data violating some invariants (foreign keys, check constraints, unique constraints, or global constraints like "total sum of money is X"). But in the end this is utterly irrelevant because the database failed to provide correct data (to the extend implied by the advertised guarantees like isolation level, for example). For the user the end result is pretty much the same. And actually, corrupted data tend to be detected much sooner, because when you can't read the disk pace you know there's an error and can address it. The inconsistency is often creepy and takes very long until someone spots that.
- fipar 11y ago
  I don't know if there is some official definition, but my experience talking to others working with databases daily tells me they have the same concepts as I do, and a 5 second google search on "data corruption" followed by "data consistency" yield wikipedia articles in the top 3 results for both queries, showing concepts that, also, match those that I have.
  - dsp1234 11y ago
    If I sat down at one of our database consoles and fired off a few queries and saw that hundreds of thousands of dollars had appeared out of nowhere, I'd tell my boss that the database was corrupt. However, for the purposes of Aphyr's original article, it's a moot point specifically because Aphyr replaced the symbol with the substance[0]. He specifically states the conclusion, and in detail explains why it happened and what the ramifications are. Any quibbling about the word corruption is a distraction, because the underlying fault of the system is still present. The system claims to support snapshot isolation, but does not, and the result of that fact is that data will not be correct for users expecting snapshot isolation. [0] - http://lesswrong.com/lw/nv/replace_the_symbol_with_the_substance/ http://lesswrong.com/lw/nv/replace_the_symbol_with_the_subst...

by fipar 11y ago

MichaelGG 11y ago

Perhaps update the article to make a note that you don't control that site's docs? It wasn't immediately apparent that there's yet another party involved. I'd suggest that the tone of the article be very clear that the problem described is accurate and isn't trying to be downplayed. Suggesting that Aphyr write another "FUD" article comes across as dismissive of the issue. And that'll make people read more into the corruption/inconsistent comment. (Unfair as it may be.)

sergiosgc 11y ago

Corruption is read by the database people as: "Your data is unreadable"; Inconsistency as: "Your data is wrong". The article implies that inconsistent is better than corrupt, and that mixing up the terms paints a worse picture. The assumption is flawed. Inconsistent is as bad or worse than corrupted. It is a silent failure, and silent failures are worse than visible ones.

bsg75 11y ago
What database people? Maybe those dealing with low value, fluff data in their attempt to be the next social media breakout, but for those of us who work with data containing anything with real business value, inconsistent == corrupt. No other definition is sufficient.
- sergiosgc 11y ago
  Did you actually read my comment all the way through?
  - bsg75 11y ago
    It appears I misread it.
bakhy 11y ago
i'm kinda surprised by the prevalence of that attitude. in theory, yes, you are right. but that's really a bit of programmer purism. in practice, a DB that stops working can mean a sudden stop in doing business, which is a huge catastrophe. money lost every second. inconsistency is horrible, and theoretically the same thing, you are right! but it's something you might be able to fix, without the whole system failing, with maybe just a limited number of people knowing what happened and not the whole company and/or all customers. this is kinda fucked up, but i think it counts. (and PS, accountants find these errors. it's their job, and they have experience enough not to trust any proof of correctness, and check everything twice.)
- tptacek 11y ago
  An account database that makes new money out of thin air is a huge catastrophe! I honestly don't understand how this is even a live issue for debate. The "inconsistency" we're talking about is terrifying.
  - bakhy 11y ago
    they are both a catastrophe, i never implied that inconsistency is not one. i'm merely involved in semantic bickering here :) both are bad in their own way, but which is worse will depend on the case. bear in mind that programmers make mistakes all the time. they can also cause such effects. which is why we often do things in a way where incorrect data can be corrected later. particularly in anything related to accounting, where the practice has existed for centuries. i'm really not interested in claiming that one catastrophe is better than the other, but a data corruption in a database can sometimes be more problematic. edit - sorry, i just noticed i actually did say, in another thread here, that corruption is worse. i would like to take that back now :D
- MichaelGG 11y ago
  It's really case-specific. But it's not hard to see where an error page "SQL Connection Failed" is better than not recording user's orders, mis-crediting their account, etc. At least with a hard error it's obvious there's a problem and it must be fixed immediately. With inconsistencies, you might e.g. end up with customers expecting travel reservations only to show up and find this to not be the case. By the time the accountant finds the error, the business could be irreparably harmed.

pgaddict 11y ago

So where are the definitions of those terms, actually? The article merely complains that Tkachenko's personal definitions of those two terms do not match Aphyr's, but that's merely a personal opinion. I'd argue that "corrupted" is usually used in cases when the data get messed up due to external circumstances (say, disk error corrupting a page of data) or because of a clear bug (say, writing to incorrect place), while "consistency" is usually used for data violating some invariants (foreign keys, check constraints, unique constraints, or global constraints like "total sum of money is X"). But in the end this is utterly irrelevant because the database failed to provide correct data (to the extend implied by the advertised guarantees like isolation level, for example). For the user the end result is pretty much the same. And actually, corrupted data tend to be detected much sooner, because when you can't read the disk pace you know there's an error and can address it. The inconsistency is often creepy and takes very long until someone spots that.

fipar 11y ago
I don't know if there is some official definition, but my experience talking to others working with databases daily tells me they have the same concepts as I do, and a 5 second google search on "data corruption" followed by "data consistency" yield wikipedia articles in the top 3 results for both queries, showing concepts that, also, match those that I have.
- dsp1234 11y ago
  If I sat down at one of our database consoles and fired off a few queries and saw that hundreds of thousands of dollars had appeared out of nowhere, I'd tell my boss that the database was corrupt. However, for the purposes of Aphyr's original article, it's a moot point specifically because Aphyr replaced the symbol with the substance[0]. He specifically states the conclusion, and in detail explains why it happened and what the ramifications are. Any quibbling about the word corruption is a distraction, because the underlying fault of the system is still present. The system claims to support snapshot isolation, but does not, and the result of that fact is that data will not be correct for users expecting snapshot isolation. [0] - http://lesswrong.com/lw/nv/replace_the_symbol_with_the_substance/ http://lesswrong.com/lw/nv/replace_the_symbol_with_the_subst...