11 ms·
Immutability Changes Everything (2016)
- skohan 5y agoCheeky headline
- baby 5y agoCan you change the title?
- samuell 5y agoOn my reading list for the summer: "The Art of Immutable Architecture: Theory and Practice of Data Management in Distributed Systems" [1] [1] https://www.goodreads.com/book/show/52653567-the-art-of-immutable-architecture https://www.goodreads.com/book/show/52653567-the-art-of-immu...
- throwaway81523 5y agoI'll try to look at that. At a more local level you might like "Purely Functional Data Structures" by Chris Okasaki. Functional programming revolves around those.
- BazookaMusic 5y ago+1 on "Purely Functional Data Structures", an excellent book to learn a different programming paradigm
- refset 5y agoPat also wrote a pretty great follow-up article that expands on the "Data on the Outside vs. Data on the Inside" section, covering the intersection of immutability, temporality, schema and relational querying: https://queue.acm.org/detail.cfm?id=341501 https://queue.acm.org/detail.cfm?id=341501
- news_hacker 5y agoYour link wasn't working for me, here's a different one: https://queue.acm.org/detail.cfm?id=3415014 https://queue.acm.org/detail.cfm?id=3415014
- refset 5y agoI somehow messed up the copy+paste and lost the last character, sorry :( Thank you for your persistence though and for chiming in with the right answer!
- jbjbjbjb 5y agoI feel like the programming tools haven’t kept up and so to do this we also need immutable requirements and impeccable programmers. Currently if I change the code in my application it gives different results. I’m not sure if there is a word for it, but I’m thinking of pure applications like we have pure functions. I don’t think the right tools exist for this. Also the “accountants don't use erasers” feels really glib to me, it’s hard work.
- dgb23 5y agoWe have Git but a commit hash has no notion of whether the consumers have to care about a change. The artifact we produce, typically immutable in a package repository, then has "semantic versioning" (possibly) to convey whether they should care.
- jbjbjbjb 5y agoYeah I my argument is that those tools are ok but a bit inadequate. Certainly not first class solutions. It’s just a thought, I’m not sure it would open up a world of possibilities but there could be more rigorous ways on showing how the logic has changed over time and keeping that alongside the current state of the logic.
- dgb23 5y agoYou might find Unison[0] interesting. Or the more recent developments (or intents) of Clojure spec[1] and the talk Speculation[2]. [0] https://www.unisonweb.org/ https://www.unisonweb.org/ [1] https://github.com/clojure/spec-alpha2 https://github.com/clojure/spec-alpha2 [2] https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk
- kaba0 5y agoAs others in the thread mentioned, the Unison language is exactly what you are getting at.
- belter 5y agoImmutability changes everything but does not Solves Everything. You are just changing to a different scenario that might help. "Tesler's Law" [1] :-) makes sure you are looking from a different perspective. Complexity is still present. "Immutability is not enough" https://codewords.recurse.com/issues/six/immutability-is-not-enough https://codewords.recurse.com/issues/six/immutability-is-not... [1] - "Tesler's Law" https://en.wikipedia.org/wiki/Law_of_conservation_of_complexity https://en.wikipedia.org/wiki/Law_of_conservation_of_complex...
- cuddlecake 5y agoThe article "Immutability is not enough" is sub-par in every aspect. The author never defines, what sort of problems immutable data can help avoid, or what the benefits of immutable data are. And still, after every paragraph, he wonders why functional programming didn't help him avoid bugs in his business logic. Like, what the hell, the author produced a bug because the order of your operations was wrong? Great, write a test and restructure your code to make it work, at least you know where to look. Functional Programming does not fix your Game Loop, and it never promised to.
- moldavi 5y agoThe article rang true for me. I heard the same FP promises that the author did, and eventually ran into the same issues in my FP programs, and left disillusioned. The article captured the experience perfectly.
- crdrost 5y agoFWIW I came to the comments because I also found the “Immutability changes everything” paper nearly unreadable. I just wanted to comment that I love your last paragraph from a conceptual level. You want immutability? Great, let's talk about what performant game design looks like in that context. Games are special, in that they kind of do everything that computers are good at, to varying degrees. The author's ideas about snapshots of relational databases inviting further schema changes in the pipeline—I guess I see the idea you're trying to introduce, but what does it actually look like in a chess program? Or a choose your own adventure game?
- bob1029 5y agoImmutability allows for incredible performance when you are working in the guts of database engines. Having assurances that a specific data log file offset cannot possibly be modified allows for substantial optimization throughout. For users who do not need a serialized view of events, access to "near-real-time" data can be made to scale as far as needed, even if a single machine is driving all the writes. You could theoretically have a billion readers working off a single synchronous core system. By utilizing careful abstractions, that single core system can write something on the order of 10-100 million transactions per second. We do it every day on the various financial exchanges around the world. In many cases, a single thread is what is holding up the entire exchange. Arrays of structs, hot cache lines, correct branch prediction and full pipelines are a hell of a combo. In some practical businesses apps, it is also feasible to coalesce logical writes before they need to enter a fully-synchronous context, so you can get some step-down ratio like 100:1 on transactions that actually need to go across the network and be handled by the core system.
- m12k 5y agoI'm increasingly coming to realize that in programming, flexibility is something you pay for in performance, whether you use it or not. Data being mutable even if you never mutate it. Automatic bounds checks for data that you've already made sure can't go out of bounds. vtable lookups even when you're not using inheritance, because who knows, someone else might. Compilers needing to do extra work in case two pointers are aliases, even though that never makes sense for them to be from the programmer's perspective. Objects allocated one at a time, scattered around memory, even if you don't need them to be, and wouldn't mind their innards being kept together in cache aligned arrays. They say you can solve any problem in computer science with another level of indirection - except performance, it seems, because we're paying the cost of all this indirection all the time, even if we don't need it most of the time. There's a crazy amount of unnecessary "housekeeping" done by compilers and runtimes to make sure everything always works all the time, which requires them to always do extra work to protect against pathological cases that almost never arise. Modern compilers and runtimes are getting better and better at recognizing the common "happy case" and optimizing for that, and that's good. But it still feels backwards - both that we can't explicitly opt out of unneeded flexibility, so instead the compiler needs to deduce that we aren't using it. But also that a lot of this isn't opt-in to begin with, rather than opt out - i.e. make things immutable by default. Or let the programmer tell the compiler a little bit more about what they need and don't need the code to do, so the compiler can do a much better job if it.
- davidgerard 5y ago(2016)
- JackMorgan 5y agoAnyone interested in what an immutable programming language looks like should check out Unison [0] written by Paul Chiasano. He is known for writing the book FP in Scala. The premise is that any edit produces a new function, but the old one still exists, so you need to migrate calls from the old version to the new version. The benefits really start to manifest in distributed computing and package management: much less need to worry about transitive dependencies and conflicting versions. The editable code is a projection of a data structure, so refactoring and static analysis tools will be very easy to write. Also the syntax is just a display issue and can be changed easily, everyone could read and write code in the syntax they prefer. It does increase some complexity, but I think it is essential complexity we currently have only poor ways to manage, and explicit knobs to turn will allow us overall less effort. [0]https://www.unisonweb.org/ https://www.unisonweb.org/
- fsloth 5y agoF# is immutable by default, so are all ML variants (e.g. OCaml, Scala) for that matter.
- DennisP 5y agoF# has immutable data by default. Your parent comment describes immutable code.
- CyberDildonics 5y agoWhat is that supposed to mean?
- DennisP 5y agoI mean, it's right there in the above comment with presumably more at the link. This seems like the key point: > any edit produces a new function, but the old one still exists, so you need to migrate calls from the old version to the new version I'm not the guy to expand on it further, but maybe the above commenter could answer questions.
- dgb23 5y ago> Normalization is not necessary in an immutable data set, however. The only reason to normalize immutable data sets may be to reduce the storage necessary for them. On the other hand, denormalized data sets may be easier and faster to process as inputs to a computation. I disagree here. To derive new data or from a immutable data source you still want normalized data. Denormalization can happen at some point for processing. Sometimes normalization is not feasible, because it often requires modelling/design. But it is the best general case scenario for data to be well-structured and normalized.
- mtVessel 5y agoYou're being kind. The author's statement is utter nonsense. If you store the same fact in two places and only change one of the them, it doesn't matter whether you update in-place, or create a new version of record -- your data is still corrupted.
- deleted 5y ago[deleted]
- chaz6 5y agoI do not think immutability is compatible with data protection legislation such as GDPR.
- pjc50 5y agoInterestingly France requires immutability (well, durable records, not quite the same thing) for point of sale systems, under "NF525": https://www.ikosoft.com/en-gb/all-knowledge-about-the-nf525-standard-for-cash-register-software/ https://www.ikosoft.com/en-gb/all-knowledge-about-the-nf525-...
- aszen 5y agoImmutability doesn't mean you can't delete old records.
- tyingq 5y agoIn practice, it could. Kafka, for example, doesn't provide a non-hacky way to delete a specific record from a topic.
- mightybyte 5y agoThis idea can be extended by observing the following analogy: purity is to functions as immutability is to data Immutability is definitely a valuable technique that can help make software faster (sometimes), more reliable, and easier to maintain. I would take it one step further and argue that we can get a similar boost by extending the idea to functions, not just data. And the way you do that is by writing pure functions (i.e. no side effects...or controlling/limiting them in some way) and using languages that can enforce that your functions are pure.
- sitkack 5y agoYes, and computation and data both become interchangeable facts. And by becoming facts, they lessen the tech debt, and start to fade into the background. I think the only way to have tractable large systems is to have pervasive immutability and purity.
- jhgb 5y ago> purity is to functions as immutability is to data I thought all data was just a bunch of nullary functions. ;)
- tsimionescu 5y agoUnfortunately, this just doesn't pan out in practice. There are various techniques that are much more inefficient to express with immutability and pure functions (e.g. computing a histogram). There is also a pretty big body of evidence already that doesn't show any hugely significant (say, 2x) changes in productivity, correctness or speed for using pure, strict functional languages and styles vs more traditional imperative, impure or even dynamic languages.
- benrbray 5y agoFor completeness, would you mind posting a link to some of this evidence so we don't have to take your word for it?
- username90 5y ago
- deleted 5y ago[deleted]
- wruza 5y agoAccountants don't use erasers; otherwise they may go to jail. All entries in a ledger remain in the ledger. Corrections can be made but only by making new entries in the ledger. This is not true. Accounting is usually a most mutable data source you ever have. Facts arrive at random times and then are entered on a schedule. Corrections await on someone to resolve analytical issues. Different accounting units have different timelines, and in a unit there is many threads of an unordered fact flow. It is all synced together only at specific points, usually when it’s time to report officially or internally. If you ask your accountant and the only thing they drop is “busy”, it’s one of these weeks of the year. When a company's quarterly results are published, they include small corrections to the previous quarter. Small fixes are OK. They are append-only, too. This is true, because public reports are legally binding. On topic: in my opinion, immutability at small scales is good, because modifications do not leak accidentally into the fields of your data structures (e.g. a mutable string is a bad idea in general). But at the bigger scale immutability gets in the way, if not hidden properly. It requires a less common sort of developers who have to think in immutable/functional way, and even when they can, this increases their cognitive distance to the business logic. Properly hidden immutability helps a lot though, and we mutable guys use it a lot, despite a widespread belief. It’s transactions and execution context isolation. There is nothing wrong with mutating the operative area if these mutations cannot escape it. But if you’re short on primitives (a context object, an object model vs global.store and raw fields of a one-for-all object) you end up with setting globally visible state that can be read from the outside and trouble creeps in. Moreover, a real world code rarely looks like you’ve seen in a tutorial and instead of a clear `assemble(this(), that())` you have to resort to non-ordinary functional tricks and concepts in the name of it. As a result, you have to hire (or be one of) these functional guys who may be better in general, but also spend half their cognitive abilities on things that you couldn’t explain to an accountant. So yes and no, immutability changes everything, but it’s not the only way to change it. It is just the easiest manual way to feel safe if you start naked in the woods.
- bruce343434 5y ago> Facts arrive at random times and then are entered on a schedule. Corrections await on someone to resolve analytical issues. Different accounting units have different timelines, and in a unit there is many threads of an unordered fact flow. It is all synced together only at specific points, usually when it’s time to report officially or internally. If you ask your accountant and the only thing they drop is “busy”, it’s one of these weeks of the year. I fail to see how any of this implies the data is mutable.
- whiddershins 5y agoSo I want to store historical sensor data, like room temperature, for many sensors across many rooms. And then events that happen less frequently. Hourly. Daily. There is no need to ever change that data, only append to it. Does anyone here know what I should be looking at for append-only data storage technologies?
- rzzzt 5y ago"Time series database" and "log-structured storage" might be two keywords for starting points.
- jasonwatkinspdx 5y agoFor industrial scale Kafka et all. For more modest scale you could use something like lmdb, where append only hits a happy case optimization.
- mbrodersen 5y agoJust use a single file on disk. Or (if you worry about things getting corrupted) SQLite.
- rurban 5y agoTaught us SICP already 45 years ago (1985)
- lewisjoe 5y agoCurious: Under what kind does redux's use of immutability come under?