3 ms·
Maybe if antirez doesn't want to read others simplistic responses on Twitter, he shouldn't make simplistic statements on Twitter. It's clearly a medium that do
by notacoward 12y ago
Maybe if antirez doesn't want to read others simplistic responses on Twitter, he shouldn't make simplistic statements on Twitter. It's clearly a medium that doesn't convey nuance very well. There was a great deal of misinterpretation and talking past one another on all sides including his.
This was actually a continuation of a conversation that has been going on for several days, starting with an issue even more serious than 99th percentile latency - data loss due to a broken replica-repair strategy. Some very clueful people have pointed antirez at useful literature on the topic, and made specific suggestions to avoid the problem, only to be met with silence or excuses for why throwing away data was actually OK. Is it any wonder that they're frustrated with him, and ready to interpret an ambiguous statement like "the 99% percentile is bad" in a more negative way than he intended? For him to cherry-pick one exchange and present only one side only continues his own pattern of making constructive conversation almost impossible.
It's not a Twitter problem. It's a people problem.
- dangerlibrary 12y agoThis whole little teapot tempest may be a people problem, but many of the issues brought up in the article - removal of context, selective re-tweets, general nastiness - are Twitter problems.
- annamarie 12y agoI think they are people problems. Everyone likes being right + feeling important. Everyone is lazy. No one wants to be called out on it. If you are forced to keep it <140 char then no one can fault you for being lazy with your replies and critiques. Of course people are going to jump in and offer their brilliant opinions. Think about the low-cost high they can get.
- dangerlibrary 12y agoThis is a bit of a false dichotomy, I think. Of course there are good (and valid!) psychological explanations for the behavior patterns exhibited on Twitter. When I say something is a "Twitter problem," I mean that these patterns are less prevalent on other platforms. I mean that Twitter has created a scaffold for communication that encourages certain bad behaviors. One example: If I were to pluck the middle sentence from your response and criticize you for saying that "Everyone is lazy" here on HN, I'd be down voted into oblivion because I am obviously being a jackass. Twitter's structure can make it very hard to see when someone is being misquoted or their views misrepresented. It's the Fox News Soundbite version of online discussions.
- annamarie 12y agoYeah, I can see your point, to say that it's a "twitter problem" for leaving that door open.
- notacoward 12y agoI don't deny that Twitter constrains conversation in a way that can be negative, but is that what's going on in this case? Consider: antirez's blog post represents exactly the kind of one-sided un-nuanced polemic that he blames on Twitter, even though he has infinite space in his chosen medium to do better. What more could he have done to prove that the medium isn't the problem? The problem is desire for control of the message. People who want that sort of control should just issue press releases. People who try to use the Twitter megaphone to promote their ideas, their projects, or themselves have to understand that others are doing exactly the same thing and sometimes the messages will conflict. The community into which antirez dropped this particular comment is one full of people running their own data-storage projects, academics promoting their own ideas, and others with more abstract (but no less passionate) beliefs about things like data protection or 99th percentile latency. I'm part of that community, and I've certainly had to endure pot shots against me or my project because of my presence on Twitter. It's part of the territory - just as it is on sites like this, or has been since forever on Usenet and BBSes and all the way back to the first town square. Among a thousand competing voices, yours might not be heard or understood perfectly.
- seiji 12y agodata loss due to a broken replica-repair strategy. That's the thing: Redis has no read repair strategy. Redis has a "be an exact copy of your master" strategy. It's not a secret. It's the exact design. The complaints are like yelling at Linus when you rm -rf / your entire machine. Sure, it sucks, but it's a repercussion if your own actions, not a fault in the system. If you don't want to rm -rf / your machine, go use an OS designed for babies (cough ubuntu cough). (Plus, there are already Redis improvements (designed within a day or two of the original problem being reported) to provide workarounds to users who _do_ want to run that exact use case. The answer to problems is solutions—not complaining and blaming endlessly.) "the 99% percentile is bad" in a more negative way than he intended? Text. It's only text. You can't read the intonation and people want to read absolutes. People want to read anger. Always anger. Always confrontation. It's possible the author of the text didn't mean to insult your mother. Breathe. It'll be okay. The entire goal of the Internet is to get ALL THE ATTENTION YOURSELF. If anybody hates you online, it's because you got attention and they didn't. Nobody hates insignificant people. So, often times people with lower profiles/attention in conversations will try to increase their attention profile by arguing/hating the high-attention people. If people hate you, you've already won.
- notacoward 12y agoWhen you say "be an exact copy of your master" is the strategy, you're kind of missing the problem here - who should be the master? In this case, Redis made a really dumb choice of which node should be master, and brought everything in sync by replicating emptiness instead of replicating data. There are very few use cases where that would be preferable. People have pointed out well known and fairly simple solutions to the selection problem, which are leading to the fixes and workarounds you've mentioned (though without the courtesy of acknowledging where the ideas came from). Your analogy to "rm -rf /" is invalid, because that's well defined, well documented, and well known behavior. That's not true of Redis's autonomous and non-deterministic response to a failure (not to a user action). No spec or doc precluded choosing a different master and preserving data instead of discarding it. In the absence of such explicit guidance, preserving data should always be the default. How can it be user error when the user did nothing? Redis did the wrong thing because something was missed in its implementation, not because of any rational or deliberate choice.
- antirez 12y agoDid you read the post? "the 99% percentile is bad" is something I never wrote, it was part of a larger sentence where it was obvious it meant if the 99th percentile figure is too big. About pointing to relevant papers: things like pointing to timestamped replication paper, which is a CP system for replicated state machines, in response to the fact that Redis does not support slaves when the master disk persistence is turned off, is actually just another instance of why it is not possible to have decent tech conversations on Twitter. The problem experienced at Stripe is a result of Redis replication documented behavior, regardless of using Sentinel or not. Redis replication is a very simple system where replicas will try to exactly mimic the master, and if the link breaks, will try to connect with it again and again forever. There is no builtin HA, nor failover or alike. So before considering failover (and yet IMHO pointing to timestamped replication is not very informative even in this context, since in distributed systems the details matter, and you can't just give a random reference to a completely different system which happens to have just superficially similar issues to fix), there were different useful observations to do. Like: Hey @antirez, what about supporting master-slaves setups where the master can be configured without persistence at all, and yet when it reboots, it will not be considered viable for reconnections? Which is the fundamental problem: stopping the basic Redis replication behavior from working as it works, with slaves that always want to replicate the current master, which is not ok if we want a system supporing the master restarted without persistence (which wipes the dataset on restart). However the whole problem with that is that there are a lot of people like you that will regard linking to a paper as a great way to help, and as a very smart thing, while there are other that are instead trying to work to really make stuff better. Before commenting you should make the effort to understand exactly the problem domain and its subtleness: exact details or what you say is not relevant in a discussion which is all about details.
- notacoward 12y agoSee my response to seiji. Exact mimicry might be a defined behavior, but where is it mandated who should mimic whom? Where had you said, before this, that the node with no data must be the one whose (empty) view should prevail? As for your false dichotomy between reading the literature and getting stuff done, or your "make the effort to understand" ad hominem - go to hell. I've been doing this longer than you, I've been doing it better than you, and I've been writing about it as well. You make the effort to understand the problem before you shoot yourself in the foot yet again.