4 ms·

Hiya, by an index. The index of course isn’t cached. This model applies equally well to RDBMS as well as non-RDMS systems. You can index which is the most recen

by hashbo 15y ago

Hiya, by an index. The index of course isn’t cached. This model applies equally well to RDBMS as well as non-RDMS systems. You can index which is the most recent version without losing the benefit of caching the data (and potentially it’s de-serialized form within the application). Not suggesting one size fits all, just highlighting a useful strategy.

birken 15y ago
And how do you keep this consistent across different objects? IE object A and object B are updated at the same time, going from generation 1 to generation 2. I am on a remote machine reading from the cache. How can I ensure I don't get a view where I see object A at generation 1 and object B at generation 2? IE the basic point I am making here is that MVCC does a lot of relatively complicated stuff to handle this for you. I don't disagree that you are gaining a lot of power at your remote nodes to do things that would increase performance, but for 99% of applications I think it a huge amount of work for a small amount of benefit.
- hashbo 15y ago
  Hi so I believe you’re asking how does CoW solve consistency issues. This seems to be a recurring theme. I suspect and correct me if I’m wrong this comes from the fact we’re using a non-RDMS database or is it because our actual caching is by nature distributed. In either case the questions being asked are applicable to all distributed caching, if you need to have an absolute guarantee of consistency (nuclear power plant, large financial transactions) then I would personally avoid using caches anyway and reconcile these relationships in the database anyway. I might add this is not our use case :-) My experience has been in both high volume financial systems and consumer startups and I personally have found that sometimes ACID behaviour is near essential. Or at least near essential to keep the implementor sane :-) And other times it’s just a hinderance. Nothing in my article implied that I’m out to solve all these, interesting cases I hope - rather that a particular pattern which can be implemented in different ways has a set of benefits. For the record Neo4J supports ACID transactions, in case I have inferred otherwise. What’s interesting to me is that in various of the comments I have received seem to extrapolate my article into, I believe an attempt to remove or somehow change the working of RDMS systems. Which I can honestly say has me at a loss, goodness knows what I said to trigger that view. No offence to the commenters I just am uncertain where this came from. What I was discussing originally was a design pattern which I have had recent experience of within the context in my case of persistence. Again forgive me if I misunderstood but I believe you are talking the internals of your database when you’re talking MVCC. This wasn't the problem we’re trying to solve. In fact Neo4J is a very high speed transactional database that we’re more than happy with. In fact we’re not trying to solve a problem, we needed versioned data so we implemented that, now we’re getting lots of benefits including zero cache invalidations which means we have a safe write through cache, which of course can be caching application style representations of the data (i.e. objects). And the various other benefits I mentioned. I can only state my practical experience here, which is that we do get a massive benefit from being able to arbitrarily cache data without fear of stale data. But hey horses for courses, enough from me for tonite - a huge pile of code awaits me and a beta needs to get underway - if you’d like to get back to me, please drop a comment on the original article and I will be pinged. Always happy to chat. Thanks for the discussion! Neil
- gnaritas 15y ago
  > And how do you keep this consistent across different objects? IE object A and object B are updated at the same time, going from generation 1 to generation 2. I am on a remote machine reading from the cache. How can I ensure I don't get a view where I see object A at generation 1 and object B at generation 2? You use caches when it doesn't matter, which is much of the time. If it really really matters, then you use ACID semantics of the DB to ensure that.

by hashbo 15y ago

birken 15y ago

And how do you keep this consistent across different objects? IE object A and object B are updated at the same time, going from generation 1 to generation 2. I am on a remote machine reading from the cache. How can I ensure I don't get a view where I see object A at generation 1 and object B at generation 2? IE the basic point I am making here is that MVCC does a lot of relatively complicated stuff to handle this for you. I don't disagree that you are gaining a lot of power at your remote nodes to do things that would increase performance, but for 99% of applications I think it a huge amount of work for a small amount of benefit.

hashbo 15y ago
Hi so I believe you’re asking how does CoW solve consistency issues. This seems to be a recurring theme. I suspect and correct me if I’m wrong this comes from the fact we’re using a non-RDMS database or is it because our actual caching is by nature distributed. In either case the questions being asked are applicable to all distributed caching, if you need to have an absolute guarantee of consistency (nuclear power plant, large financial transactions) then I would personally avoid using caches anyway and reconcile these relationships in the database anyway. I might add this is not our use case :-) My experience has been in both high volume financial systems and consumer startups and I personally have found that sometimes ACID behaviour is near essential. Or at least near essential to keep the implementor sane :-) And other times it’s just a hinderance. Nothing in my article implied that I’m out to solve all these, interesting cases I hope - rather that a particular pattern which can be implemented in different ways has a set of benefits. For the record Neo4J supports ACID transactions, in case I have inferred otherwise. What’s interesting to me is that in various of the comments I have received seem to extrapolate my article into, I believe an attempt to remove or somehow change the working of RDMS systems. Which I can honestly say has me at a loss, goodness knows what I said to trigger that view. No offence to the commenters I just am uncertain where this came from. What I was discussing originally was a design pattern which I have had recent experience of within the context in my case of persistence. Again forgive me if I misunderstood but I believe you are talking the internals of your database when you’re talking MVCC. This wasn't the problem we’re trying to solve. In fact Neo4J is a very high speed transactional database that we’re more than happy with. In fact we’re not trying to solve a problem, we needed versioned data so we implemented that, now we’re getting lots of benefits including zero cache invalidations which means we have a safe write through cache, which of course can be caching application style representations of the data (i.e. objects). And the various other benefits I mentioned. I can only state my practical experience here, which is that we do get a massive benefit from being able to arbitrarily cache data without fear of stale data. But hey horses for courses, enough from me for tonite - a huge pile of code awaits me and a beta needs to get underway - if you’d like to get back to me, please drop a comment on the original article and I will be pinged. Always happy to chat. Thanks for the discussion! Neil
gnaritas 15y ago
> And how do you keep this consistent across different objects? IE object A and object B are updated at the same time, going from generation 1 to generation 2. I am on a remote machine reading from the cache. How can I ensure I don't get a view where I see object A at generation 1 and object B at generation 2? You use caches when it doesn't matter, which is much of the time. If it really really matters, then you use ACID semantics of the DB to ensure that.