6 ms·

Isn't this article just stating the obvious fact that programs at some point do I/O, so no program can be purely functional? Every Haskell tutorial seems to in

Isn't this article just stating the obvious fact that programs at some point do I/O, so no program can be purely functional? Every Haskell tutorial seems to introduce the IO Monad to illustrate this distinction.

mtrn 10y ago
While not new, I find the discussion around immutable storage important and interesting. Datomic[1] comes to mind. There are many scenarios and applications, where it would be nice to just write things to a structured log and build an application on that log. Unfortunately there are some roadblocks, like little information and conversation about the topic and no real mainstream implementation of these ideas. I've been working on a system, that has is based on an immutable data layer and it is just a wonderful thing to work with, as long as you have enough disk space. [1] http://www.datomic.com/ http://www.datomic.com/
- smallnamespace 10y ago
  I'm curious, how is this different from using an event oriented database, or using your normal DB but changing the schema to be oriented around immutable events?
  - mamcx 10y ago
    Now I trying to use Event Sourcing (mainly for sync) for a "normal" invoice app, I find several problems. The thing is that the event log is good for write but awful for read. Re-compute everything is problematic and the true is 90% of the time you want the last version, not all the history. Also, the split in logic from the app / engine is stupid. I think is good if we move past the idea that the DB engine is more dumb than a rock, and embrace it. But current tools are not ideal (sadly, I know what is live like this: I have used Visual FoxPro!) Where I think this could lead to something great is if the DB is alike this: - Relational - You have normal tables. Consider them your up-to-date cache. Most scenarios will be covered here (ie: The log recomputing is NOT at the app level, but at the DB level). - You have a event log where everything is stored, plus your "tables" for the info you need up-to-date. But you don't need all to be a table if not make sense, and do full re-computation if desired. - Your backup, sync and load history is around the event log. - You don't even need a "table" if you wanna only a "index", this lead to me: - You submit a data request (like a POST), and the DB convert it to the tables and log, but: - You can instead (or also!) hook here and build secondary indexes and do other stuff. You can do it async, put the computation in a queue, do validations, etc as fit. So, what is not here and need tooling is marry the "normal tables" + "event log" + "logic that route this", so putting: Data -> WAL -> Router -> Event Log | Tables | Index | ping external tools And Request Data -> Router -> Get it from:Event Log | Tables | Index | ping external tools
    - icebraining 10y ago
      Have you seen PipelineDB? It's Postgres, but with something called "continuous views", which are like regular views except they're cached and updated on writes, not re-calculated each time they're read. Sounds like one could use plain tables as the "event log", and these views as the up-to-date cache. And aggregation queries work too, so your regular table can be more than just a cache. I'm not affiliated, not even a user, it just sounded interesting when it was submitted here.
      - mamcx 10y ago
        I have not use it, I my first glance make me think is only about analytics, but maybe that could work. I will try and see how well...
      - Fergi 10y ago
        Jeff from PipelineDB here. Yes, PipelineDB is an analytics product designed to continuously query high volumes of streaming data. The top use cases we see are building realtime reporting dashboards and realtime monitoring and alerting systems. PipelineDB excels at workloads where you know the queries you want to run ahead of time, where your workload fits within the confines of SQL, and where there is a high degree of distillation (aggregations, sliding windows, etc.). We have large customers including Charter Cable / Time Warner Cable, MediaMath, SmartNews, MOAT Analytics, Cradlepoint Networks, Paddy Power Betfair, and others using the system at scale and we offer a clustered edition of PipelineDB under a commercial license, but PipelineDB's single server edition is open-source. We have a live chat room here for technical questions: https://gitter.im/pipelinedb/pipelinedb https://gitter.im/pipelinedb/pipelinedb
    - yrashk 10y ago
      You are quite right, event sourcing on its own is not great for read, and the "re-compute everything" (aka foldl) is problematic. CQRS solves this by producing a read-side that aggregates events into your domain. Classic event sourcing also attempts to solve the re-computation problem by snapshotting. It is great that you mention that 90% of the time you want "the last version". That was one of my discoveries with event sourcing as well. So what I came to is that re-computation (foldl) is a nice generalization, but isn't very practical. Instead, index the events and build the state on demand by querying the indices. This way, [plurality enabling] domain modelling moves to the code (as opposed to the database) and allows for quick changes in how you see the information. You can read more about it at https://blog.eventsourcing.com/why-use-eventsourcing-database-6b5e2ac61848#.o1q5yb3p9 https://blog.eventsourcing.com/why-use-eventsourcing-databas... and https://blog.eventsourcing.com/lazy-event-sourcing-ed7e59007e17#.v5fa8aqbt https://blog.eventsourcing.com/lazy-event-sourcing-ed7e59007... as well as other articles on that blog.
- clusmore 10y ago
  Offtopic to the OP but related to Datomic. I saw Rich Hickey's talk about Datomic recently and immediately thought it would be ideal as part of a distributed runtime. Without having actually programmed in Erlang, but having seen plenty of Joe Armstrong's talks, I understand that the point of replicating data in messages rather than passing around pointers is so that if a machine dies, the data isn't lost. But if you built a distributed runtime that used Datomic as a distributed heap on which you allocate large amounts of data which you expect to pass around a lot, you could instead pass around only pointers to the data and not worry about data loss, and save big time on message throughput. Does anybody know if something like this already exists?
wz1000 10y ago
I feel that one of the biggest misconceptions about monads and IO in Haskell is that they are an "escape hatch" in the language that allows you to perform imperative actions. This is plainly false. Monads are just another functional programming idiom, defined by two regular functions, `return` and `(>>=)`. What monads allow you to do is have a purely functional model to represent IO, among a great many other things. Haskell is, in a sense, the best imperative programming language ever created, because it lets you manipulate imperative statements in a purely functional manner. You can stick statements in data structures, compose them, replicate them, generate them and only evaluate them when you want to.
- js8 10y ago
  I agree. But this is a misconception about functional programming in general, that its purpose is to somehow replace imperative actions that are done by the computer system with something else which is not imperative. But that's not the goal. Functional programming is there to provide a better formalism for writing programs, it doesn't want to change the way the computers actually execute the programs (or how the systems eventually internally work). That means more work for the compilers to figure out how to make the same old imperative program out of a functional one, which is a good thing (it's generally the direction the compilers evolve). Benefits of functional programming are not in how programs are executed (quite to the contrary - it's actually more effort to make them as efficient as good imperative programs), but in how the programs are written by people. I think the article is a little confused about this. For another example to see this, even notion of non-pure function goes already in this direction. Today, having lots of (small) functions seems to have negligible downsides, but historically this wasn't the case at all, look at any better program that had to fit into memory of an 8-bit computer. These were rarely written in this style, because compilers were simply not capable enough to produce a reasonably compact spaghetti machine code out of those (by means of inlining, for instance).
  - wott 10y ago
    To sum up: harder to write, harder to compile, harder to make efficient. Hum... difficult to qualify it as 'better' after this.
seagreen 10y ago
no program can be purely functional Pure functional programming means building up a machine. This machine will eventually be executed by the runtime system (at which point IO will happen) but from the programmer's perspective a program can still be purely functional. By this I mean that a function that takes an Int and outputs an Int will have type `Int -> Int`, and a function that takes an Int, does logging, and outputs an Int will have type `Int -> IO Int`. Both are purely functional -- they're pure transformations from an Int to some other thing. For the first function that other thing is another Int, for the second that other thing is an action that can be performed by the machine which will return an Int. So from the perspective that matters to us a developers programs can be purely functional. (This becomes even more useful with something like PureScript's granular effect tracking https://github.com/purescript/purescript/wiki/Differences-from-Haskell#io-vs-eff https://github.com/purescript/purescript/wiki/Differences-fr...)
- smallnamespace 10y ago
  Yes, in Haskell monadic functions are strictly speaking still pure functions, but it's the set of mutations performed by the package of runtime + your code that matters. Do-notation shows that there's a trivial mapping between monadic code and imperative code. For all practical intents and purposes, things happening in the IO Monad are not 'functional', because your mental model needs to consider the mutations they represent. What functional programming encourages you to do (as opposed to OOP) is segregate the pure aspects of your program, under the theory that pure code is easier to write, maintain, and test.
  - catnaroek 10y ago
    Effect segregation (a more appropriate term than “purity”) is a matter of using types, and has nothing to do with functional vs. OOP.
    - smallnamespace 10y ago
      The type system has a huge impact on how easily you can do effect segregation. Haskell has a type system that enforces effect segregation for you. Java doesn't. Classic OOP encourages modeling your problem as a collection of inherently stateful objects, making effect segregation more difficult. Of course, you can write in a functional style in Java, or an imperative OOP style in Haskell, but the language isn't going to help you as much.
charlieflowers 10y ago
No, that's not exactly what he is saying. He's saying we write logic in pure functions, but then save the data in a mutable database. But he's saying we don't have to -- with event sourcing, even the data we store is immutable, so more of the system can be purely functional.
agumonkey 10y ago
All but benchmarks.
clusmore 10y ago
>Isn't this article just stating the obvious fact that programs at some point do I/O, so no program can be purely functional? I agree with the point that a program can't be purely functional, but lately I've been wondering if you could still have a programming language that is purely functional (preferably also total). It is designed for writing data transformation libraries and needs to be called into by a stateful language that handles IO and orchestration. If you've seen Gary Bernhardt's talk Boundariers [1], this language would be the purely functional core. [1] https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries

mtrn 10y ago

While not new, I find the discussion around immutable storage important and interesting. Datomic[1] comes to mind. There are many scenarios and applications, where it would be nice to just write things to a structured log and build an application on that log. Unfortunately there are some roadblocks, like little information and conversation about the topic and no real mainstream implementation of these ideas. I've been working on a system, that has is based on an immutable data layer and it is just a wonderful thing to work with, as long as you have enough disk space. [1] http://www.datomic.com/ http://www.datomic.com/

smallnamespace 10y ago
I'm curious, how is this different from using an event oriented database, or using your normal DB but changing the schema to be oriented around immutable events?
- mamcx 10y ago
  Now I trying to use Event Sourcing (mainly for sync) for a "normal" invoice app, I find several problems. The thing is that the event log is good for write but awful for read. Re-compute everything is problematic and the true is 90% of the time you want the last version, not all the history. Also, the split in logic from the app / engine is stupid. I think is good if we move past the idea that the DB engine is more dumb than a rock, and embrace it. But current tools are not ideal (sadly, I know what is live like this: I have used Visual FoxPro!) Where I think this could lead to something great is if the DB is alike this: - Relational - You have normal tables. Consider them your up-to-date cache. Most scenarios will be covered here (ie: The log recomputing is NOT at the app level, but at the DB level). - You have a event log where everything is stored, plus your "tables" for the info you need up-to-date. But you don't need all to be a table if not make sense, and do full re-computation if desired. - Your backup, sync and load history is around the event log. - You don't even need a "table" if you wanna only a "index", this lead to me: - You submit a data request (like a POST), and the DB convert it to the tables and log, but: - You can instead (or also!) hook here and build secondary indexes and do other stuff. You can do it async, put the computation in a queue, do validations, etc as fit. So, what is not here and need tooling is marry the "normal tables" + "event log" + "logic that route this", so putting: Data -> WAL -> Router -> Event Log | Tables | Index | ping external tools And Request Data -> Router -> Get it from:Event Log | Tables | Index | ping external tools
  - icebraining 10y ago
    Have you seen PipelineDB? It's Postgres, but with something called "continuous views", which are like regular views except they're cached and updated on writes, not re-calculated each time they're read. Sounds like one could use plain tables as the "event log", and these views as the up-to-date cache. And aggregation queries work too, so your regular table can be more than just a cache. I'm not affiliated, not even a user, it just sounded interesting when it was submitted here.
    - mamcx 10y ago
      I have not use it, I my first glance make me think is only about analytics, but maybe that could work. I will try and see how well...
    - Fergi 10y ago
      Jeff from PipelineDB here. Yes, PipelineDB is an analytics product designed to continuously query high volumes of streaming data. The top use cases we see are building realtime reporting dashboards and realtime monitoring and alerting systems. PipelineDB excels at workloads where you know the queries you want to run ahead of time, where your workload fits within the confines of SQL, and where there is a high degree of distillation (aggregations, sliding windows, etc.). We have large customers including Charter Cable / Time Warner Cable, MediaMath, SmartNews, MOAT Analytics, Cradlepoint Networks, Paddy Power Betfair, and others using the system at scale and we offer a clustered edition of PipelineDB under a commercial license, but PipelineDB's single server edition is open-source. We have a live chat room here for technical questions: https://gitter.im/pipelinedb/pipelinedb https://gitter.im/pipelinedb/pipelinedb
  - yrashk 10y ago
    You are quite right, event sourcing on its own is not great for read, and the "re-compute everything" (aka foldl) is problematic. CQRS solves this by producing a read-side that aggregates events into your domain. Classic event sourcing also attempts to solve the re-computation problem by snapshotting. It is great that you mention that 90% of the time you want "the last version". That was one of my discoveries with event sourcing as well. So what I came to is that re-computation (foldl) is a nice generalization, but isn't very practical. Instead, index the events and build the state on demand by querying the indices. This way, [plurality enabling] domain modelling moves to the code (as opposed to the database) and allows for quick changes in how you see the information. You can read more about it at https://blog.eventsourcing.com/why-use-eventsourcing-database-6b5e2ac61848#.o1q5yb3p9 https://blog.eventsourcing.com/why-use-eventsourcing-databas... and https://blog.eventsourcing.com/lazy-event-sourcing-ed7e59007e17#.v5fa8aqbt https://blog.eventsourcing.com/lazy-event-sourcing-ed7e59007... as well as other articles on that blog.
clusmore 10y ago
Offtopic to the OP but related to Datomic. I saw Rich Hickey's talk about Datomic recently and immediately thought it would be ideal as part of a distributed runtime. Without having actually programmed in Erlang, but having seen plenty of Joe Armstrong's talks, I understand that the point of replicating data in messages rather than passing around pointers is so that if a machine dies, the data isn't lost. But if you built a distributed runtime that used Datomic as a distributed heap on which you allocate large amounts of data which you expect to pass around a lot, you could instead pass around only pointers to the data and not worry about data loss, and save big time on message throughput. Does anybody know if something like this already exists?

wz1000 10y ago

I feel that one of the biggest misconceptions about monads and IO in Haskell is that they are an "escape hatch" in the language that allows you to perform imperative actions. This is plainly false. Monads are just another functional programming idiom, defined by two regular functions, `return` and `(>>=)`. What monads allow you to do is have a purely functional model to represent IO, among a great many other things. Haskell is, in a sense, the best imperative programming language ever created, because it lets you manipulate imperative statements in a purely functional manner. You can stick statements in data structures, compose them, replicate them, generate them and only evaluate them when you want to.

js8 10y ago
I agree. But this is a misconception about functional programming in general, that its purpose is to somehow replace imperative actions that are done by the computer system with something else which is not imperative. But that's not the goal. Functional programming is there to provide a better formalism for writing programs, it doesn't want to change the way the computers actually execute the programs (or how the systems eventually internally work). That means more work for the compilers to figure out how to make the same old imperative program out of a functional one, which is a good thing (it's generally the direction the compilers evolve). Benefits of functional programming are not in how programs are executed (quite to the contrary - it's actually more effort to make them as efficient as good imperative programs), but in how the programs are written by people. I think the article is a little confused about this. For another example to see this, even notion of non-pure function goes already in this direction. Today, having lots of (small) functions seems to have negligible downsides, but historically this wasn't the case at all, look at any better program that had to fit into memory of an 8-bit computer. These were rarely written in this style, because compilers were simply not capable enough to produce a reasonably compact spaghetti machine code out of those (by means of inlining, for instance).
- wott 10y ago
  To sum up: harder to write, harder to compile, harder to make efficient. Hum... difficult to qualify it as 'better' after this.

seagreen 10y ago

no program can be purely functional Pure functional programming means building up a machine. This machine will eventually be executed by the runtime system (at which point IO will happen) but from the programmer's perspective a program can still be purely functional. By this I mean that a function that takes an Int and outputs an Int will have type `Int -> Int`, and a function that takes an Int, does logging, and outputs an Int will have type `Int -> IO Int`. Both are purely functional -- they're pure transformations from an Int to some other thing. For the first function that other thing is another Int, for the second that other thing is an action that can be performed by the machine which will return an Int. So from the perspective that matters to us a developers programs can be purely functional. (This becomes even more useful with something like PureScript's granular effect tracking https://github.com/purescript/purescript/wiki/Differences-from-Haskell#io-vs-eff https://github.com/purescript/purescript/wiki/Differences-fr...)

smallnamespace 10y ago
Yes, in Haskell monadic functions are strictly speaking still pure functions, but it's the set of mutations performed by the package of runtime + your code that matters. Do-notation shows that there's a trivial mapping between monadic code and imperative code. For all practical intents and purposes, things happening in the IO Monad are not 'functional', because your mental model needs to consider the mutations they represent. What functional programming encourages you to do (as opposed to OOP) is segregate the pure aspects of your program, under the theory that pure code is easier to write, maintain, and test.
- catnaroek 10y ago
  Effect segregation (a more appropriate term than “purity”) is a matter of using types, and has nothing to do with functional vs. OOP.
  - smallnamespace 10y ago
    The type system has a huge impact on how easily you can do effect segregation. Haskell has a type system that enforces effect segregation for you. Java doesn't. Classic OOP encourages modeling your problem as a collection of inherently stateful objects, making effect segregation more difficult. Of course, you can write in a functional style in Java, or an imperative OOP style in Haskell, but the language isn't going to help you as much.

charlieflowers 10y ago

No, that's not exactly what he is saying. He's saying we write logic in pure functions, but then save the data in a mutable database. But he's saying we don't have to -- with event sourcing, even the data we store is immutable, so more of the system can be purely functional.

agumonkey 10y ago

All but benchmarks.

clusmore 10y ago

>Isn't this article just stating the obvious fact that programs at some point do I/O, so no program can be purely functional? I agree with the point that a program can't be purely functional, but lately I've been wondering if you could still have a programming language that is purely functional (preferably also total). It is designed for writing data transformation libraries and needs to be called into by a stateful language that handles IO and orchestration. If you've seen Gary Bernhardt's talk Boundariers [1], this language would be the purely functional core. [1] https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries