3 ms·
I never meant to imply FP is already the right choice for big data (which is what I think you refer to by 'systems people') or high performance computing. The
by mtrimpe 12y ago
I never meant to imply FP is already the right choice for big data (which is what I think you refer to by 'systems people') or high performance computing.
The reality though is that well over 99% of the market works with data sets for which, with current hardware, the performance costs of these persistent data structures are very much irrelevant.
When you add FRP on top of that so that you can keep your managed state minimal you can actually often achieve quite dramatic performance improvements over the traditional hand-written data-to-UI synchronisation.
For persistent data structures to come into their own for big data quite a few more things are needed of course; but if you look at Nathan Marz's Lambda Architecture for example it's little more than a handwritten system for making large scale persistent data structures manageable with today's systems already.
I'm fairly sure that tooling for that will only improve over time until the very thought of manually managing derivation pipelines becomes as quaint as manual memory management.
As for HPC; yes... I would not be surprised at all if at that level mutable state no longer is a premature optimization. Even there I could imagine many of the more high-level control structures benefitting from persistent data structures though.
About the FRP part; which competing paradigm are you working on? I wouldn't be able to name a 'competitor' so I'm quite curious.
- seanmcdirmid 12y agoIt actually involves versioned data structures (similar, but a but more complicated than persistent data structures, and outwardly they appear mutable): http://research.microsoft.com/pubs/211297/managedtime.pdf http://research.microsoft.com/pubs/211297/managedtime.pdf
- mtrimpe 12y agoI already had a feeling we were essentially arguing the same thing... Yes; the managed time approach in Glitch is very interesting and indeed a competitor towards addressing the same problem I originally raised. The core of my original statement/problem definition was that real mutable state, i.e. throwing away history by default, was no longer the smart thing to do given current hardware. From what I can tell you would also agree with that...
- seanmcdirmid 12y agoI would, but it doesn't necessarily have to happen with persistent data structures, and I would also argue that they aren't really sufficient (the history has to be useful, after all). They are convenient, and I'm using trees in my current implementation.