4 ms·

"The truth is, it's all state. All of it." The problem is not state. The problem is being able to recreate the state: both for testing and for business purpos

"The truth is, it's all state. All of it." The problem is not state. The problem is being able to recreate the state: both for testing and for business purposes. The bigger problem is that most programmers, like you, are knee-jerking before the issue of "recreating the state" and declaring that "It cannot be done, because it's all state". And hence we have both languages who are build by considering mutability to be a virtue and back-end databases (your typical CRUD SQL DB) build upon the same false premises. I'll give you one example: monday morning the service desk calls because one user of your app, at 3:07pm on friday, experienced a bug. Can you "recreate the state" your application was in at that time as to be able to figure out what triggered the bug? You probably can't. Because your DB has changed meanwhile: you're SOL because the 'U' and the 'D' in CRUD are destructive. It's mutability. It's the ennemy of determinism. So now you're stuck calling your DB admin asking for a dump of the prod DB on last thursday evening and a dump of the log of the transactions that happened on friday... And you're spending hours and hours trying to recreate the state the environment was in when the sht hit the fan. And you may or may not be able to do it. It's just one hypotethical scenario but things like that is the daily lot of many* programmers. But software development, in many cases, shouldn't be that painful. If you were to use a CRA DB (Create Read Append) and languages favoring immutability and a more functional approach overall, you'd have a much much easier time recreating the state. If you think of it, it's all a gigantic determistic machine. So why can't we accept that the notion of time is an important one? Why can't you realize that the battle the likes of Rich Hickey are fighting are worth it? It is possible to use programming languages and DBs (or wrappers like Datomic in front of SQL DBs) that do definitely make it easier to reason about programs and that make it just so much easier to recreate the state and to query the past (which has a lot of business value). Why do you react like this: "The truth is, it's all state". Saying that as if nothing could be done and as if every single programmer's life should be Java/C# + ORM + XML + SQL hell? There are people trying to make our life as devs easier. Why not try to listen to what their saying? Is Rich Hickey "seeing things" with Clojure + Datomic? To me the combination of a functional language (or at least a language that can be used in a mostly functional way) and a CRA DB (Create Read Append) which incorporates the notion of time from the start is a godsend to our industry. Why do you close your eyes?

Lapsa 14y ago
Try googling around for "cqrs". There are many ideas on how to do exactly this effectively. All it takes is to swap "write" operation with "event" entity (all is state). Combine that with intention revealing commands (customer_died instead of delete from customers where id=1) and you hit jackpot. End result is nice stream of events which is ultimate source of truth and enables crazy queries you never thought you would ever need http://abdullin.com/journal/2010/6/3/time-machines-should-support-linq.html http://abdullin.com/journal/2010/6/3/time-machines-should-su...
- neeleshs 14y ago
  This is quite interesting. In practice though, the lags between command/query models when the datastores are different(almost always, for large data) becomes a real concern
  - mcgwiz 14y ago
    Pardon my pedantry, but CQRS doesn't necessitate eventual consistency. Denormalization of domain events into can happen synchronously. It all depends on the needs of the project. If the project information architecture and processes can support a degree of latency (which is implicit in almost any technology providing static screens anyway), then the performance gains of eventual consistency can be realized. A good, nuanced explanation of CQRS is here: http://codeofrob.com/entries/cqrs-is-too-complicated.html http://codeofrob.com/entries/cqrs-is-too-complicated.html
jeffdavis 14y ago
A CRA DB is essentially a SQL database with temporal capability. Even if you have perfect time-warping capabilities, there are still big challenges to solving bugs with that kind of analysis. The time the bug is noticed may be long after the root cause, and working backward could be a very slow process. That's why SQL has declarative constraints, to try to make the error noticed at a time closer to the root cause. Would I like full time-warping capabilities sometimes? Of course. But it's just one more option, and a very expensive one at that. To be any use at all, we'd need to be able to warp backwards through the entire OS and its scheduling decisions (not just a single process), because many difficult bugs involve race conditions. Really, we need better temporal capabilities, and more declarative constraints (that are immune from race conditions, like a UNIQUE constraint), better ways of avoiding race conditions, etc.; and they all need to work together. That's why I have worked on temporal capabilities[1] as well as declarative constraints [2] (immune from races) in Postgres. Also, you might be interested in truly serializable transactions in postgres, which eliminate race conditions between transactions without blocking[3]. I feel like the programming language community should work more with the database community. From the article, they mention how restarts can help recover from errors encountered after part of the state has already been changed. But that kind of recovery is taken for granted with the atomic nature of transactions in a database. Database theory is largely about detecting, containing, mitigating, recovering from, and preventing errors[4]. [1] http://www.postgresql.org/docs/9.2/static/rangetypes.html http://www.postgresql.org/docs/9.2/static/rangetypes.html [2] http://www.postgresql.org/docs/current/static/sql-createtable.html#SQL-CREATETABLE-EXCLUDE http://www.postgresql.org/docs/current/static/sql-createtabl... [3] http://drkp.net/drkp/papers/ssi-vldb12.pdf http://drkp.net/drkp/papers/ssi-vldb12.pdf [4] http://thoughts.davisjeff.com/2009/12/23/good-error-recovery-is-hard-so-use-postgresql/ http://thoughts.davisjeff.com/2009/12/23/good-error-recovery...

Lapsa 14y ago

Try googling around for "cqrs". There are many ideas on how to do exactly this effectively. All it takes is to swap "write" operation with "event" entity (all is state). Combine that with intention revealing commands (customer_died instead of delete from customers where id=1) and you hit jackpot. End result is nice stream of events which is ultimate source of truth and enables crazy queries you never thought you would ever need http://abdullin.com/journal/2010/6/3/time-machines-should-support-linq.html http://abdullin.com/journal/2010/6/3/time-machines-should-su...

neeleshs 14y ago
This is quite interesting. In practice though, the lags between command/query models when the datastores are different(almost always, for large data) becomes a real concern
- mcgwiz 14y ago
  Pardon my pedantry, but CQRS doesn't necessitate eventual consistency. Denormalization of domain events into can happen synchronously. It all depends on the needs of the project. If the project information architecture and processes can support a degree of latency (which is implicit in almost any technology providing static screens anyway), then the performance gains of eventual consistency can be realized. A good, nuanced explanation of CQRS is here: http://codeofrob.com/entries/cqrs-is-too-complicated.html http://codeofrob.com/entries/cqrs-is-too-complicated.html

jeffdavis 14y ago

A CRA DB is essentially a SQL database with temporal capability. Even if you have perfect time-warping capabilities, there are still big challenges to solving bugs with that kind of analysis. The time the bug is noticed may be long after the root cause, and working backward could be a very slow process. That's why SQL has declarative constraints, to try to make the error noticed at a time closer to the root cause. Would I like full time-warping capabilities sometimes? Of course. But it's just one more option, and a very expensive one at that. To be any use at all, we'd need to be able to warp backwards through the entire OS and its scheduling decisions (not just a single process), because many difficult bugs involve race conditions. Really, we need better temporal capabilities, and more declarative constraints (that are immune from race conditions, like a UNIQUE constraint), better ways of avoiding race conditions, etc.; and they all need to work together. That's why I have worked on temporal capabilities[1] as well as declarative constraints [2] (immune from races) in Postgres. Also, you might be interested in truly serializable transactions in postgres, which eliminate race conditions between transactions without blocking[3]. I feel like the programming language community should work more with the database community. From the article, they mention how restarts can help recover from errors encountered after part of the state has already been changed. But that kind of recovery is taken for granted with the atomic nature of transactions in a database. Database theory is largely about detecting, containing, mitigating, recovering from, and preventing errors[4]. [1] http://www.postgresql.org/docs/9.2/static/rangetypes.html http://www.postgresql.org/docs/9.2/static/rangetypes.html [2] http://www.postgresql.org/docs/current/static/sql-createtable.html#SQL-CREATETABLE-EXCLUDE http://www.postgresql.org/docs/current/static/sql-createtabl... [3] http://drkp.net/drkp/papers/ssi-vldb12.pdf http://drkp.net/drkp/papers/ssi-vldb12.pdf [4] http://thoughts.davisjeff.com/2009/12/23/good-error-recovery-is-hard-so-use-postgresql/ http://thoughts.davisjeff.com/2009/12/23/good-error-recovery...