9 ms·

An Experiment in Purely Functional IO for Clojure

72 points by mikemarsh 11y ago

explorak 11y ago
That first little example is the best explanation of a functionally pure language I've read.
escherize 11y ago
I think I'm missing something: How is the function println not pure? It always returns nil (for inputs that it doesn't fail on). Doesn't that mean it's pure? Edit: I'd define pure in this sense as the same thing as referentially transparent, meaning f(x) will always return the same thing for a given x.
- ane 11y ago
  Well, if somebody breaks the kernel so that printing doesn't work, it will most likely return something strange, in the case of Clojure, an exception. A pure function always returns the same result because it only depends on its input parameters. An impure IO function does not.
  - escherize 11y ago
    I see that if there's a kernel panic, or power outage that f(x) might return something insane or nothing at all, but those aren't things I'm usually worried about. What's the practical upside to this? Is breaking the kernel something I should actually consider likely?
    - astrange 11y ago
      Compiler optimization are allowed to change how often a pure function is called, and you can freely change it in your code. If it had a printf inside it, you'd need a way to say you don't care about the side effect of printing a specific line one time, and not zero or two times. Of course there is lots of code that has side effects in implementation but not in their interface. Like malloc, or a read only data structure that has an internal cache it updates.
- gizmo686 11y ago
  The definition of purity in this post isn't quite correct. The common definition (which this post seems to be using) is that a pure function is one that has no side effects. This doesn't have much of an implication for reasoning about almost-pure functions like println, which have to effects on the program execution, but do cause some distinction. For example, with a pure function, the following are equivelent: var a=f(0); a=f(1) a=f(1) however, these are not the same when f=println.
  - zak_mc_kracken 11y ago
    How so? println returns `Unit` so certainly, what you say above doesn't hold. Even if `println` returned a value, it would still be considered pure as long as that value is strictly calculated from its input parameter and that parameter alone (say, the size of the string). Another often used criterion to determine purity of a function is whether inlining it everywhere in your program produces a similar program. There again, `println` passes this test. The only way `println` could not be pure is if you call it with the same string twice and it returns a different value.
    - gizmo686 11y ago
      Consider the following program. var a = println("Hello ") a = println(" World") This should output "Hello \n World\n" However, assuming println is pure, we can optimize this to just var a = println(" World") Which produces a different output. We could also convert: var a=println("Hello") var b=println("Hello") into var a=println("Hello") var b=a
      - lgas 11y ago
        My (quite possibly wrong) understanding is that purity has nothing to say about side effects. It's simply concerned with the inputs and outputs of functions, specifically whether the function will always return the same value given the same input. What zak_mc_kracken is saying is that under this definition println is obviously and trivially pure because it returns the same value for every input. You're correct that it won't have the proper side effects when optimized but under this definition that doesn't have anything to do with purity. Edit: If you want the "side effects" of the println statement to be considered part of the "output" then you want something like Haskell's semantics where the println statements are IO actions. Edit: zak_mc_kracken beat me to it.
- T-R 11y ago
  Surprised no one's brought up Haskell's Debug.Trace, which effectively does this, but returning whichever value you tell it to.
- tel 11y ago
  Pure usually has a stricter definition. First we need the ability to state whether two functions are equal. Then we need a function which is constantly unit. Finally we need composition such that (f >> g) is "f then g". Now, a function f is pure if and only if constantly_unit = f >> constantly_unit If you unpack that a bit it might translate as "if we throw away the return value of a function, it is exactly the same as if nothing is happening at all". If your notion of equality differentiates "println" and "constantly_unit" then we cannot call "println" pure. Note that this is a very powerful notion of purity. It's so powerful as to render Haskell impure as if we have the function loop x = loop x then `loop >> constantly_unit` never returns and is therefore easy to distinguish from `constantly_unit` itself. This just drives home that non-termination is an effect itself!
  - wz1000 11y ago
    Ah, but Haskell is lazy so loop will never be evaluated and a loop >> constantly_unit will return unit no matter what.
    - tel 11y ago
      Ah, yeah, I was being fast and loose with laziness. You have to `seq` the argument to `constantly_unit`, too.
  - im3w1l 11y ago
    Hmm I wonder if that loop example can be rehabilitated. If we interpret it, not as a description of how to evaluate loop, but rather as a constraint on loop. Then we see that the statement is simply a non-condition on loop. This would mean that non-termination is not a property of the function, but of the compiler/runtime, in that they failed to notice that loop was a partial function called with an input value for which it was not defined!
    - tel 11y ago
      That's probably possibly in this particular case but it sounds a lot like you're heading toward Halting Problem territory here :)
- noelwelsh 11y ago
  I find it most useful to define referential transparency in terms of substitution (which is the definition Wikipedia starts with). Namely: if I substitute a function call with its result is there an observable different in program behaviour? If there is, the program is not referentially transparent. This is clearly the case with `println`. (println "Hi there") is clearly not the same program as nil as the former causes output to the console, while the latter does not -- an observable difference. Using Scheme (because I don't know Clojure that well) (define (foo x) (let ([y 0]) (set! y 10) (+ x 1))) is a referentially transparent program. Although it contains a side-effect (the assignment to y) this effect is not observable outside `foo`, and thus any call to `foo` can be substituted with its result.
  - badsock 11y ago
    Ah, but println "Hi there" doesn't return nil, it returns a value of the type IO nil which can be transparently substituted for the call.
    - noelwelsh 11y ago
      Yes, but you're writing in Haskell whereas my example is in Clojure. Hence the difference :)
      - badsock 11y ago
        Dammit! Sorry, carry on!
- bhrgunatha 11y ago
  Since println is just an example, perhaps it isn't really a good example to use when trying to give an example of how impure functions can have side effects. Sending some ink to a printer or altering pixels on a screen isn't exactly a compelloing argument for side effects. Maybe a better example function is one that deletes data from your hard drive, or sens control signals to a robot that locks the doors or changes the A/C or starts some industrial process, releasing toxic chemicals. The point is println does have a side effect, it's just not a very interesting one. Or maybe the definition of side effect is what is causing the problem, after all - executing any function does have physical side effects - electrons move, electromagnetic styate chabnes, ambient temperature changes. Maybe the problem is where we draw the line for observable side effects.
- sgk284 11y ago
  > It always returns nil (for inputs that it doesn't fail on). > Doesn't that mean it's pure? Unfortunately not because it also modifies global state (the state of the console). A pure function implies that it is referentially transparent, meaning that: a = foo() b = a + a Is the exact same as: b = foo() + foo() But if foo() modifies global state, then this statement is not true. In this case the difference is printing something twice vs. once. If a function is pure, a compiler can optimize the second example into the first example. For a function to be pure, it must always return the same value given the same input and not modify any state that is observable outside of its definition.
yason 11y ago
And why would you want to redo the idiomatically complex part in Haskell (there are probably thousands of monad tutorials just to explain why I/O in Haskell needs to be wrapped in a monad) also in similarly a complex way in Clojure which, however, could actually handle I/O and side-effects in a controlled way just fine without making an (academic) mess out of it? You need to use monads to get around Haskell's limitations imposed by the decision to target complete purity just to do simple things like I/O but no matter what I/O is still not pure (something like readline can never be pure even if it wanted to) and thus you're, in one way or in another, forced to separate the static, pure and functional parts of your program and its dynamic part with side-effects. Haskell does it with monads which, as a concept in itself, is a generic way to reason about state, but IMHO the exactly best part of Clojure is that it offers several ways to manage dynamic state in a controlled way without forcing you to go 100% pure or 100% impure. Why break that, except as a mental exercise?
- tel 11y ago
  This is so misinformed it isn't even wrong. * IO doesn't have to be wrapped in a monad, there are other models * Clojure cannot handle IO/side effects very well (compared to any language with effect typing) * Input and output are not pure, but the `IO` type itself is pure. Rather, constructions of the IO type are pure and then the runtime can interpret IO values impurely. * The entire point is to separate the impure from the pure. It's not that you're forced to, it's that you desire to. * Monads are not, in themselves, generic ways to reason about state. They are far more general. * Anything that is not 100% pure is 100% impure. Without purity guarantees you cannot trust code you call upon to not do side effects. This breaks local reasoning.
  - taeric 11y ago
    I can't help but disagree with your final bullet. You won't have "contractual and checked by the compiler" trust in code you call upon, but it is quite common to trust the code that you call in any language to do just what it claims it will do. Consider, do you really think that code was lacking local reasoning before the likes of Haskel? It is arguable that things were more difficult then, but the argument is still out that things are easier now.
nopuremore 11y ago
The purpose of defining a function as pure is to allow the computer to make transformations. If you define println as a pure function and use pure to debug or trace your program you could discover that the compiler has eliminated some dead code and that some code has been executed out of order, so your debugging doesn't has any useful meaning. That's why is better to tell you that print isn't pure, the compiler could eliminate it if you define it as pure. Disclaimer: I only has read the first comments.
- kyllo 11y ago
  In Haskell this goes even deeper because the lazy evaluation strategy allows the compiler to optimize the order of evaluation of the expressions within a function and create thunks without you explicitly telling it to. Haskell needs to be pure in order to be lazy, because if it were impure it would be very difficult to reason about if, when, and in what order, the side effecting operations will take place. So like you said, if a side-effectful expression evaluates to nothing, its return value is not "needed" in a further computation, therefore it may not be evaluated at all. This doesn't really apply to Clojure because Clojure is strictly evaluated--except when you're explicitly working with a lazy stream data structure. If you were to put side-effectful expressions inside of a lazy stream, you would have a hard time controlling when they are evaluated, especially since Clojure "chunks" lazy streams by default in groups of 32 as an optimization. Here's a SO question demonstrating what happens when you mix laziness and side-effects and don't understand the implications--you find yourself trying to restrict the optimizations you allow the compiler to do (which will hurt performance): http://stackoverflow.com/questions/3407876/how-do-i-avoid-clojures-chunking-behavior-for-lazy-seqs-that-i-want-to-short-ci http://stackoverflow.com/questions/3407876/how-do-i-avoid-cl...

72 points by mikemarsh 11y ago

That first little example is the best explanation of a functionally pure language I've read.

I think I'm missing something: How is the function println not pure? It always returns nil (for inputs that it doesn't fail on). Doesn't that mean it's pure? Edit: I'd define pure in this sense as the same thing as referentially transparent, meaning f(x) will always return the same thing for a given x.

ane 11y ago
Well, if somebody breaks the kernel so that printing doesn't work, it will most likely return something strange, in the case of Clojure, an exception. A pure function always returns the same result because it only depends on its input parameters. An impure IO function does not.
- escherize 11y ago
  I see that if there's a kernel panic, or power outage that f(x) might return something insane or nothing at all, but those aren't things I'm usually worried about. What's the practical upside to this? Is breaking the kernel something I should actually consider likely?
  - astrange 11y ago
    Compiler optimization are allowed to change how often a pure function is called, and you can freely change it in your code. If it had a printf inside it, you'd need a way to say you don't care about the side effect of printing a specific line one time, and not zero or two times. Of course there is lots of code that has side effects in implementation but not in their interface. Like malloc, or a read only data structure that has an internal cache it updates.
gizmo686 11y ago
The definition of purity in this post isn't quite correct. The common definition (which this post seems to be using) is that a pure function is one that has no side effects. This doesn't have much of an implication for reasoning about almost-pure functions like println, which have to effects on the program execution, but do cause some distinction. For example, with a pure function, the following are equivelent: var a=f(0); a=f(1) a=f(1) however, these are not the same when f=println.
- zak_mc_kracken 11y ago
  How so? println returns `Unit` so certainly, what you say above doesn't hold. Even if `println` returned a value, it would still be considered pure as long as that value is strictly calculated from its input parameter and that parameter alone (say, the size of the string). Another often used criterion to determine purity of a function is whether inlining it everywhere in your program produces a similar program. There again, `println` passes this test. The only way `println` could not be pure is if you call it with the same string twice and it returns a different value.
  - gizmo686 11y ago
    Consider the following program. var a = println("Hello ") a = println(" World") This should output "Hello \n World\n" However, assuming println is pure, we can optimize this to just var a = println(" World") Which produces a different output. We could also convert: var a=println("Hello") var b=println("Hello") into var a=println("Hello") var b=a
    - lgas 11y ago
      My (quite possibly wrong) understanding is that purity has nothing to say about side effects. It's simply concerned with the inputs and outputs of functions, specifically whether the function will always return the same value given the same input. What zak_mc_kracken is saying is that under this definition println is obviously and trivially pure because it returns the same value for every input. You're correct that it won't have the proper side effects when optimized but under this definition that doesn't have anything to do with purity. Edit: If you want the "side effects" of the println statement to be considered part of the "output" then you want something like Haskell's semantics where the println statements are IO actions. Edit: zak_mc_kracken beat me to it.
T-R 11y ago
Surprised no one's brought up Haskell's Debug.Trace, which effectively does this, but returning whichever value you tell it to.
tel 11y ago
Pure usually has a stricter definition. First we need the ability to state whether two functions are equal. Then we need a function which is constantly unit. Finally we need composition such that (f >> g) is "f then g". Now, a function f is pure if and only if constantly_unit = f >> constantly_unit If you unpack that a bit it might translate as "if we throw away the return value of a function, it is exactly the same as if nothing is happening at all". If your notion of equality differentiates "println" and "constantly_unit" then we cannot call "println" pure. Note that this is a very powerful notion of purity. It's so powerful as to render Haskell impure as if we have the function loop x = loop x then `loop >> constantly_unit` never returns and is therefore easy to distinguish from `constantly_unit` itself. This just drives home that non-termination is an effect itself!
- wz1000 11y ago
  Ah, but Haskell is lazy so loop will never be evaluated and a loop >> constantly_unit will return unit no matter what.
  - tel 11y ago
    Ah, yeah, I was being fast and loose with laziness. You have to `seq` the argument to `constantly_unit`, too.
- im3w1l 11y ago
  Hmm I wonder if that loop example can be rehabilitated. If we interpret it, not as a description of how to evaluate loop, but rather as a constraint on loop. Then we see that the statement is simply a non-condition on loop. This would mean that non-termination is not a property of the function, but of the compiler/runtime, in that they failed to notice that loop was a partial function called with an input value for which it was not defined!
  - tel 11y ago
    That's probably possibly in this particular case but it sounds a lot like you're heading toward Halting Problem territory here :)
noelwelsh 11y ago
I find it most useful to define referential transparency in terms of substitution (which is the definition Wikipedia starts with). Namely: if I substitute a function call with its result is there an observable different in program behaviour? If there is, the program is not referentially transparent. This is clearly the case with `println`. (println "Hi there") is clearly not the same program as nil as the former causes output to the console, while the latter does not -- an observable difference. Using Scheme (because I don't know Clojure that well) (define (foo x) (let ([y 0]) (set! y 10) (+ x 1))) is a referentially transparent program. Although it contains a side-effect (the assignment to y) this effect is not observable outside `foo`, and thus any call to `foo` can be substituted with its result.
- badsock 11y ago
  Ah, but println "Hi there" doesn't return nil, it returns a value of the type IO nil which can be transparently substituted for the call.
  - noelwelsh 11y ago
    Yes, but you're writing in Haskell whereas my example is in Clojure. Hence the difference :)
    - badsock 11y ago
      Dammit! Sorry, carry on!
bhrgunatha 11y ago
Since println is just an example, perhaps it isn't really a good example to use when trying to give an example of how impure functions can have side effects. Sending some ink to a printer or altering pixels on a screen isn't exactly a compelloing argument for side effects. Maybe a better example function is one that deletes data from your hard drive, or sens control signals to a robot that locks the doors or changes the A/C or starts some industrial process, releasing toxic chemicals. The point is println does have a side effect, it's just not a very interesting one. Or maybe the definition of side effect is what is causing the problem, after all - executing any function does have physical side effects - electrons move, electromagnetic styate chabnes, ambient temperature changes. Maybe the problem is where we draw the line for observable side effects.
sgk284 11y ago
> It always returns nil (for inputs that it doesn't fail on). > Doesn't that mean it's pure? Unfortunately not because it also modifies global state (the state of the console). A pure function implies that it is referentially transparent, meaning that: a = foo() b = a + a Is the exact same as: b = foo() + foo() But if foo() modifies global state, then this statement is not true. In this case the difference is printing something twice vs. once. If a function is pure, a compiler can optimize the second example into the first example. For a function to be pure, it must always return the same value given the same input and not modify any state that is observable outside of its definition.

yason 11y ago

And why would you want to redo the idiomatically complex part in Haskell (there are probably thousands of monad tutorials just to explain why I/O in Haskell needs to be wrapped in a monad) also in similarly a complex way in Clojure which, however, could actually handle I/O and side-effects in a controlled way just fine without making an (academic) mess out of it? You need to use monads to get around Haskell's limitations imposed by the decision to target complete purity just to do simple things like I/O but no matter what I/O is still not pure (something like readline can never be pure even if it wanted to) and thus you're, in one way or in another, forced to separate the static, pure and functional parts of your program and its dynamic part with side-effects. Haskell does it with monads which, as a concept in itself, is a generic way to reason about state, but IMHO the exactly best part of Clojure is that it offers several ways to manage dynamic state in a controlled way without forcing you to go 100% pure or 100% impure. Why break that, except as a mental exercise?

tel 11y ago
This is so misinformed it isn't even wrong. * IO doesn't have to be wrapped in a monad, there are other models * Clojure cannot handle IO/side effects very well (compared to any language with effect typing) * Input and output are not pure, but the `IO` type itself is pure. Rather, constructions of the IO type are pure and then the runtime can interpret IO values impurely. * The entire point is to separate the impure from the pure. It's not that you're forced to, it's that you desire to. * Monads are not, in themselves, generic ways to reason about state. They are far more general. * Anything that is not 100% pure is 100% impure. Without purity guarantees you cannot trust code you call upon to not do side effects. This breaks local reasoning.
- taeric 11y ago
  I can't help but disagree with your final bullet. You won't have "contractual and checked by the compiler" trust in code you call upon, but it is quite common to trust the code that you call in any language to do just what it claims it will do. Consider, do you really think that code was lacking local reasoning before the likes of Haskel? It is arguable that things were more difficult then, but the argument is still out that things are easier now.

nopuremore 11y ago

The purpose of defining a function as pure is to allow the computer to make transformations. If you define println as a pure function and use pure to debug or trace your program you could discover that the compiler has eliminated some dead code and that some code has been executed out of order, so your debugging doesn't has any useful meaning. That's why is better to tell you that print isn't pure, the compiler could eliminate it if you define it as pure. Disclaimer: I only has read the first comments.

kyllo 11y ago
In Haskell this goes even deeper because the lazy evaluation strategy allows the compiler to optimize the order of evaluation of the expressions within a function and create thunks without you explicitly telling it to. Haskell needs to be pure in order to be lazy, because if it were impure it would be very difficult to reason about if, when, and in what order, the side effecting operations will take place. So like you said, if a side-effectful expression evaluates to nothing, its return value is not "needed" in a further computation, therefore it may not be evaluated at all. This doesn't really apply to Clojure because Clojure is strictly evaluated--except when you're explicitly working with a lazy stream data structure. If you were to put side-effectful expressions inside of a lazy stream, you would have a hard time controlling when they are evaluated, especially since Clojure "chunks" lazy streams by default in groups of 32 as an optimization. Here's a SO question demonstrating what happens when you mix laziness and side-effects and don't understand the implications--you find yourself trying to restrict the optimizations you allow the compiler to do (which will hurt performance): http://stackoverflow.com/questions/3407876/how-do-i-avoid-clojures-chunking-behavior-for-lazy-seqs-that-i-want-to-short-ci http://stackoverflow.com/questions/3407876/how-do-i-avoid-cl...