4 ms·

OP adds confusion by calling data immutable. Data has a well accepted common usage of being updatable. Event is the term what OP wants. Event is a snapshot o

by ww520 15y ago

OP adds confusion by calling data immutable. Data has a well accepted common usage of being updatable. Event is the term what OP wants. Event is a snapshot of fact happened in time and cannot be changed and it's immutable. What OP described in general is the append-log based approach to storing and querying data. There are a whole bunch databases and file systems built on the append-log idea. It's well worth the time to check out the research in this area to see the benefits and pitfalls of the approach. Also check out the Complex Event Processing topic which bears similarity to what OP described.

nathanmarz 15y ago
Like I said, I was going to break your assumptions on data systems. Treating data as mutable (whether you use MVCC or append logs or any other approach) leads to major complexity problems, as I detailed in the post.
- ww520 15y ago
  Redefining the meaning of general term just creating more confusion about the point you want to make. Why not use precise definition. The term "data" is really really general and it's bad to convey what you really mean. Are you talking about data value? Data object? Data record? Data version? Data event? Quoting from your blog, "A piece of data is a fact that you know to be true at some moment of time." That's called an Event. That's a snapshot of fact in time. MVCC is one way to do transaction. The blog didn't deal with transaction at all so I don't see the comparison. Append log-based databases do the closest to what you described in avoiding changing data in place and recompute/precompute data at query time. It's a very simple mechanism. What complexity problem do you see with it?
RickHull 15y ago
> OP adds confusion by calling data immutable. Data has a well accepted common usage of being updatable. I find this point to be generally contentious. Certainly in the realm of programming languages, data is often held immutable. I think mutable vs. immutable data is an important distinction with wide-ranging implications, so I find being explicit here adds clarity rather than confusion.
- dasil003 15y ago
  I agree. Saying that data must be updatable is a truism. Of course the vast majority of systems must have updatable data, as that is a fundamental building block of common-sense, real-world reasoning—to the end user data is always mutable. But to the systems designer, thinking about data being immutable and reasoning from there allows deeper thinking that takes us to more interesting architectures.
  - nathanmarz 15y ago
    The key insight is that an "update" is really a query getting the most recent data record for some entity/type.
richhickey 15y ago
Sorry, no. It is calling data mutable that is the confusion. Try this exercise: Identify something you consider a datum. Write it down. Note that it has more than one part. Now imagine it 'changing' and write that datum down. Now, identify the 'it' that (you would say) changed. Note that the 'it' part is the same in the 2 datum. Let's call this 'it' part the identity. Now look at the part that is different. For discussion's sake let's say that part was 42 in the first case and 99 in the second. Did 42 change into 99, or was it replaced by it? Let's call 42 and 99 values. You'll not be able to describe something that 'changed', that incorporates both the identity and value, that doesn't involve some memory/disk/system that had a (just one!) 'place' for storing information about 'it', and it is that place that has had it's contents replaced/changed, not the datum/fact. Did Obama becoming president change the fact that Bush had been president? No. Could you make an information system that forgot Bush was president? Sure, but that's your fault, not a characteristic of data. Finally, check out this 30 year old paper that contains this interesting section: UPDATE IN PLACE: A poison apple? http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.108.7810 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.108.... As long as we continue to use lax language like mutable/updateable data we will be without the tools to describe, think about, and build better systems that independently manipulate identity, value and place, and correctly handle time.
- ww520 15y ago
  I'm sorry. I don't understand the point of the exercise. First, what kind of data are you talking about? Are you talking about data value? Data record? Data object with identity? Second, are you saying database records being mutable cannot model Bush was a president once Obama has become president? I don't know the reason for the aversion to mutable data record/object. Both mutable and immutable data record/objects have their places in modeling real world objects.
  - jcromartie 15y ago
    > I don't know the reason for the aversion to mutable data record/object. I've found that mutable data is a root of all kinds of evil. In fact, I'd guess that most bugs stem from a mis-management of mutable state. I know that pretty much all of the bugs I see every day are the result of failure to coordinate state changes, or poorly-designed state changes.

by ww520 15y ago

nathanmarz 15y ago

Like I said, I was going to break your assumptions on data systems. Treating data as mutable (whether you use MVCC or append logs or any other approach) leads to major complexity problems, as I detailed in the post.

ww520 15y ago
Redefining the meaning of general term just creating more confusion about the point you want to make. Why not use precise definition. The term "data" is really really general and it's bad to convey what you really mean. Are you talking about data value? Data object? Data record? Data version? Data event? Quoting from your blog, "A piece of data is a fact that you know to be true at some moment of time." That's called an Event. That's a snapshot of fact in time. MVCC is one way to do transaction. The blog didn't deal with transaction at all so I don't see the comparison. Append log-based databases do the closest to what you described in avoiding changing data in place and recompute/precompute data at query time. It's a very simple mechanism. What complexity problem do you see with it?

RickHull 15y ago

> OP adds confusion by calling data immutable. Data has a well accepted common usage of being updatable. I find this point to be generally contentious. Certainly in the realm of programming languages, data is often held immutable. I think mutable vs. immutable data is an important distinction with wide-ranging implications, so I find being explicit here adds clarity rather than confusion.

dasil003 15y ago
I agree. Saying that data must be updatable is a truism. Of course the vast majority of systems must have updatable data, as that is a fundamental building block of common-sense, real-world reasoning—to the end user data is always mutable. But to the systems designer, thinking about data being immutable and reasoning from there allows deeper thinking that takes us to more interesting architectures.
- nathanmarz 15y ago
  The key insight is that an "update" is really a query getting the most recent data record for some entity/type.

richhickey 15y ago

Sorry, no. It is calling data mutable that is the confusion. Try this exercise: Identify something you consider a datum. Write it down. Note that it has more than one part. Now imagine it 'changing' and write that datum down. Now, identify the 'it' that (you would say) changed. Note that the 'it' part is the same in the 2 datum. Let's call this 'it' part the identity. Now look at the part that is different. For discussion's sake let's say that part was 42 in the first case and 99 in the second. Did 42 change into 99, or was it replaced by it? Let's call 42 and 99 values. You'll not be able to describe something that 'changed', that incorporates both the identity and value, that doesn't involve some memory/disk/system that had a (just one!) 'place' for storing information about 'it', and it is that place that has had it's contents replaced/changed, not the datum/fact. Did Obama becoming president change the fact that Bush had been president? No. Could you make an information system that forgot Bush was president? Sure, but that's your fault, not a characteristic of data. Finally, check out this 30 year old paper that contains this interesting section: UPDATE IN PLACE: A poison apple? http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.108.7810 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.108.... As long as we continue to use lax language like mutable/updateable data we will be without the tools to describe, think about, and build better systems that independently manipulate identity, value and place, and correctly handle time.

ww520 15y ago
I'm sorry. I don't understand the point of the exercise. First, what kind of data are you talking about? Are you talking about data value? Data record? Data object with identity? Second, are you saying database records being mutable cannot model Bush was a president once Obama has become president? I don't know the reason for the aversion to mutable data record/object. Both mutable and immutable data record/objects have their places in modeling real world objects.
- jcromartie 15y ago
  > I don't know the reason for the aversion to mutable data record/object. I've found that mutable data is a root of all kinds of evil. In fact, I'd guess that most bugs stem from a mis-management of mutable state. I know that pretty much all of the bugs I see every day are the result of failure to coordinate state changes, or poorly-designed state changes.