uvdn7

10 ms·

91.
by uvdn7 4y ago
I don't see that. Yes I did say I had to do A. Then I said that it doesn't imply B. Because even if the underlying system is strongly ordered, we can't order external events. Did I miss something?
92.
by uvdn7 4y ago
Good points. I will try to address them. > But "knowing" this is a big part of what people mean when they say cache invalidation is hard! I can see that. Memcache is a look-aside cache we have at scale. There are abstractions o
93.
by uvdn7 4y ago
I am glad that you liked the content. On the definition of cache invalidation, and specifically why it's hard. Can you send me a link to any definition of it? This is what's in wikipedia and I think it's reasonable. > Cach
94.
by uvdn7 4y ago
> De facto, it is a cache server, with the associated performance decrease. Isn't the main purpose of caching to improve performance? Polaris only receives cache invalidation event, and doesn't serve any client queries. > so
95.
by uvdn7 4y ago
I can see that. One can argue that the “difficulty” is front loaded. If or not you can identify the list of “context” is local. I guess you can probably come up with complicated dependencies and argue it’s hard to capture the dependencies a
96.
by uvdn7 4y ago
That's not the case though. Polaris acts as a client and only monitors client observable effect, and assumes no knowledge of the server internals. I am trying to help.
97.
by uvdn7 4y ago
> Suppose you have source-of-truth A (doesn't really matter if it's a key value store or whatever, it could be a function for all intents) and a few clients B1, B2, B3, ... that rely on the data from A. You have to keep them in
98.
by uvdn7 4y ago
It's a good observation that out-of-orderness and asynchrony contribute a lot to the challenge. > the inconsistency between your cache nodes is a consequence of the underlying design of your system I disagree with this. First of all
99.
by uvdn7 4y ago
OK. I am here to learn. Say we go with your definition of cache invalidation problem. Do you think that's THE hard part of cache invalidation? Say, you are just caching simple k/v data (no joins, nothing). Are you claiming cache i
100.
by uvdn7 4y ago
The tool that monitors cache consistency is easy to build. That by itself doesn't solve anything major. The most important contribution is a novel approach on consistency tracing that helps find out "why" caches are inconsist
101.
by uvdn7 4y ago
I think https://news.ycombinator.com/item?id=31672541 might address your question.
102.
by uvdn7 4y ago
I agree with everything you said. > Having a cache whose design allows it to be stale but only very rarely seems like a bad idea. And that's exactly why we first brought this number from 6 9's to more than 10 9's; and we a
103.
by uvdn7 4y ago
For the fun issues you described, read-modify-write (either using optimistic concurrency control or pessimistic) can work, if I understand your question correctly. Spanner is awesome and one of my favorite systems/papers. I think it wo
104.
by uvdn7 4y ago
Tracing is not in polaris. It's a separate library that runs in every cache host. Its main job is logging every cache state mutation for traced writes. So when polaris detects cache inconsistencies, we know what happened and why cache
105.
by uvdn7 4y ago
> What I thought the article would tackle is the dependencies between stored objects. Like tracking dependencies, so it knows what to invalidate on mutations?
106.
by uvdn7 4y ago
> It helps that TAO is a write through cache so clients can't really forget to invalidate, correct? That's not the case. > If someone were to directly write to MySQL shards there would be stale data For the sake of this disc
107.
by uvdn7 4y ago
> Is the database updated notification going out while the transaction is still in-progress? No, that's not what happened. > If a cache tries to update itself while there's an open transaction against the data it's tryi
108.
by uvdn7 4y ago
Good point! We have memcache which is a look-aside cache that serves this type of workload. What you described can be solved by adding one level of abstraction and letting reads and writes all go through it. Now on your read path, you can c
109.
by uvdn7 4y ago
This is a fair point! The exact same argument applies to normal exceptions and errors. It might be the case that out of quadrillions of queries a day, just none of them hit the error path, and hence we won't observe any issues. I can s
110.
by uvdn7 4y ago
Can you elaborate? I went into details in the post about what I believe is the root cause of why cache invalidation is hard, which seems different than what you are saying.
111.
by uvdn7 4y ago
This is a fair point. Lesson learned. Thank you :D
112.
by uvdn7 4y ago
We invalidate cache upon mutations. When else would you do it?
113.
by uvdn7 4y ago
> Instead, it feels like grandiose claims about solving cache invalidation itself. That is definitely something I worried about. But at the same time, I do think we solved the harder part of the cache invalidation problem. TAO and Memcac
114.
by uvdn7 4y ago
Polaris is actually fairly simple to build. The harder question to answer is "why cache is inconsistent" and how you debug. The second half of the post talks about consistency tracing, which tracks all cache data state mutations.
115.
by uvdn7 4y ago
> EDIT: Reading through "A real bug we found and fixed this year" and I'm still a bit confused. yeah that specific bug is convoluted. > It seems like a very contrived bug directed directly to how you deal with versionin
116.
by uvdn7 4y ago
> isn't the typical bug related to cache invalidation that a service forgets to contact the cache server for invalidation? That's not the case based on our experience at FB. At-least-once delivery is a solved problem basically.
117.
by uvdn7 4y ago
Great question! I will try to answer this one without too much implementation details. First of all, cache items can be evicted and are not durable, which is just a fact. But that doesn't mean we can't track progress (in terms of
118.
by uvdn7 4y ago
> until such time as the claim isn't pure speculation Which I don't think it is. > casually optimistic claims about complex and hard problems are the essence of click-bait A group of people at Facebook worked on this for man
119.
by uvdn7 4y ago
Accepted. Just to be clear, it’s not a link bait as far as I understand it for two reasons 1. I am dead serious about making cache invalidation a simpler problem 2. “Cache invalidation might no longer be a hard problem in Computer Science
120.
by uvdn7 4y ago
But that's not what I am saying though ... > you have tools to analyze the correctness of your solution That's half of it. Cache invalidation is hard not only because of the complexity of cache coherence protocols, some of whic

More ›