16 ms·

Clojure for the Brave and True – Functional Programming

162 points by nonrecursive 13y ago

pat_shaughnessy 13y ago
Another great post, Daniel... keep 'em coming!
- nonrecursive 13y ago
  Thanks!
jafaku 13y ago
As someone who still doesn't get functional programming, I would like to see a real-world example of how to deal with side effects. Eg: write something into a file or a DB. Because of course methods/functions with no side effects are easier to deal with, it sounds good, but I don't think it would be easy nor convenient to separate every side effect in a real-world program (as opposed to academic programs, where you just write a Fibonacci or whatever). Even in good OOP, you can have lots of side effects in a method. Halp!
- ufo 13y ago
  Using side effects in Haskell is easy. The only thing is that they need to be explicit and you aren't allowed to hide them inside pure functions. For a database example, here is the chapter from Real World Haskell about databases (a bit old but still relevant): http://book.realworldhaskell.org/read/using-databases.html http://book.realworldhaskell.org/read/using-databases.html And here is some example code: do conn <- connectSqlite3 "test1.db" stmt <- prepare conn "INSERT INTO test VALUES (?, ?)" execute stmt [toSql 1, toSql "one"] execute stmt [toSql 2, toSql "two"] execute stmt [toSql 3, toSql "three"] execute stmt [toSql 4, SqlNull] commit conn disconnect conn As you can see, it looks very similar to how you would write it in a regular imperative language. The main difference is that since they database operations are IO operations then they need to be in the special do-notation block and cannot be mixed with regular code. As for the question of wanting to mix pure and impure code that much I don't think it actually comes up that much. First of all, I find that I don't often find myself with an impure function that I want to add a side effect in the middle seamlessly - firstly, side-effecting stuff tend to be side-effecting from the beggining and secondly if I do want to turn a pure function into an impure one then forcing me to change the interface helps make sure that I am not breaking any code that used to work because it assumed it was working with pure code. Additionally, Haskell is really good at abstracting code so its not that hard to take a big chunk of logic and split it into a pure and an impure part. Finally, converting code from pure to impure is not that bad - sure, you need to change the syntax a bit but the type checker helps you find all the places that need to change and also helps you check all the code that calles the method you changed so you can be sure that you aren't breaking those. side effecting operations in the IO monad so you can't run them in the middle of any function and instead must run them inside a special "do-notation block" like I did. You can have impure code call both other impure code and pure functions and all you can't do is call impure functions from inside pure functions. Sure, its a bit annoying if you ever have to convert a pure function to an impure one (you need to change some of the syntax and all the places that call tha tfunction need to start treating it as impure) but its not that bad: first of all the type checker tells you all the spots where you need to update your code and secondly, Haskell is super good at abstracting code so its not hard to take a big chunk of code with some side effects in the middle and split it into a pure and an impure part. > to academic program Its a little pet peeve of mine but it seems that nowadays "academic" doesn't have a real meaning and just tends to refer to whatever people don't like. :)
  - dllthomas 13y ago
    "do" is just syntax. You could write the above as: connectSqlite3 "test1.db" >>= \ conn -> prepare conn "INSERT INTO test VALUES (?, ?)" >>= \ stmt -> >> execute stmt [toSql 1, toSql "one"] >> execute stmt [toSql 2, toSql "two"] >> execute stmt [toSql 3, toSql "three"] >> execute stmt [toSql 4, SqlNull] >> commit conn >> disconnect conn and it's "just" a bunch of nested lambdas with no "do" in sight. What's different from "regular" code is that those lambdas 1) return IO actions, and 2) are strung together with monadic bind (>>=) to build one big IO action. To be sure, in this case the do version is much easier to read (that's why "do notation" exists). It can be used with any monad, though - nothing ties it particularly to IO.
    - ufo 13y ago
      Maybe I should have been clearer but the basic point is that monadic code has a different interface bar (foo x) --pure Haskell vs foo x >>= bar --monadic Haskell vs bar(foo(x)) -- in C you write both cases like this.
      - dllthomas 13y ago
        You're close, but it's not really so clear cut. foo x ++ bar -- "pure" or "monadic"? foo x >>= baz -- "pure" or "monadic"? What if (foo x) is a list in both cases? (>>=) is just an operator. The values that it operates on are anything with a Monad instance, just like the values that (+) operates on are anything with a Num instance. There is nothing voodoo about monads or about bind (>>=); such voodoo as there is lies solely in IO, which is interacted with the same way as other values (though certainly what you can do with (IO a) is more limited than what you can do with (Maybe a)).
        Dewie 13y ago
        I used to think that Monad was some alternative realm in Haskell, a realm where purity didn't exist and everything was written in an alternative, built-in language (do-notation). That's because people tend to refer to using Monads as "working in the X-Monad", as if it is some... place. But then it turned out... Oh, so it's basically just a signature/interface/type class.
- daGrevis 13y ago
  I suggest to dig in some web framework for functional programming. One example would be Snap for Haskell. http://snapframework.com/ http://snapframework.com/
  - jafaku 13y ago
    Does it make sense to have to learn Haskell and a Haskell framework just to understand Clojure? Because I'm already sold on the idea that Clojure is the best language™, being a lisp, portable, with a great community, already with a mature library for everything (because it has access to the JVM), and with some dirty shortcuts for convenience. But the frameworks are lacking. The only web framework available got dissolved, and I don't have the energy to wire everything myself in a new language and paradigm that I don't even fully understand).
    - daGrevis 13y ago
      > Does it make sense to have to learn Haskell and a Haskell framework just to understand Clojure? I have a strong feeling that once you understand either Haskell or Clojure, it will be a breeze to learn other. My point was to get framework for functional programming that deals with a lot of I/O and side-effects and see docs for real-life examples of usage.
      - hisham_hm 13y ago
        > I have a strong feeling that once you understand either Haskell or Clojure, it will be a breeze to learn other. Does Clojure use lazy evaluation?
        dllthomas 13y ago
        It also isn't statically typed.
        cgag 13y ago
        Not everywhere, but most of the sequence functions like map/filter return lazy sequences.
- nonrecursive 13y ago
  One thing I find myself doing over and over is taking as much logic as possible out of side effecting functions and placing that in a pure function, then passing the return value to the side effecting function.
- TheZenPsycho 13y ago
  In the end, there is always some kind of side effect. The value that functional programming adds is you keep the part of the program that does the side effect in one spot, doing one specific well defined thoroughly debugged thing and you limit its scope so it can't screw up the rest of your program's operation no matter how bad it goes wrong. It only goes "out". it doesn't cause any side effect inside your program. One example of a way to structure a functional program is categorising functions according to whether they are a "source" a "filter" or a "sink", (terms stolen from circuit design). The source being your inputs (not pure), filters doing some transformation on the input, over time (pure) and handing off to a sink that causes some side effect (not pure). In the "real world" You've probably made a lot of simple programs that mostly follow this structure if you've used unix pipes. If you go along with this, you find that once you start making more sophisticated programs, you need some kind of "memory" so you can progress. Strictly speaking, memory, as we typically use it in imperative languages, is a kind of side effect. In the model above, though, you can make a special kind of "source" which is something like a feedback loop- a source that contains the output of the previous iteration of the program- or some sub portion of the program. Then maintaining a memory becomes a process of continually taking some input, transforming it, creating an output, and then using that output as an input again. (what you might call recursion) Why would you do this? Because each part of a program like this is self contained, and its ability to do damage to some distant part of a program is extremely limited. It becomes much easier to reason about such a program. When you find a bug (and these are still possible) you can trace the origin from output to source, or source to output. There is no possibility of some phantom distant subroutine screwing up the dataflow. And this kind of structure is very useful for games, interactive multimedia programs, and other such graphical interactive program. A commonly cited "real world" example is xmonad. What I find more insightful is this article on replicating pacman in a functional language. http://prog21.dadgum.com/23.html http://prog21.dadgum.com/23.html
- brudgers 13y ago
  The idea is to generate side effects that don't effect program flow. This keyboard generates side effects as I type because it changes the state of the display. But pressing the "B" to start this sentence did not change the way the keyboard operates - each key press produced the same results as it would had I pressed "H". And had I pressed "H" and begun the previous sentence with "However," the screen layout would have been somewhat different, but not in a way I typically care about when writing (as opposed to doing typography). Now this is not to say that my keyboard is purely functional. The shift key creates state. But that state is isolated and short lived to the point where we can treat it as functional across two key combinations. It is the caps lock key which creates an unpredictable state - I do not know what output I will get without checking its status - the problem with state might be illustrated by the results of hitting capslock when I meant to hit tab while typing a program.
  - jafaku 13y ago
    So how should you program the capslock? You (the functional guys) keep telling me what's wrong, but you don't tell me how to do it right. Not having a capslock key is not an option, people need it.
    - TheZenPsycho 13y ago
      Actually a lot of people have decided they don't need it. They remap caps lock to something like spotlight search. The alternative to the caps lock key is the shift key. And this is a really bad metaphor for functional programming. Sorry GP. I suppose the important point here is it's not possible for something going wrong with your monitor or speakers to suddenly remap your keyboard without warning.
      - brudgers 13y ago
        I'm not saying it's a great analogy, but among it's strengths is handiness. Among it's other strengths is that functional programs can be built bottom up from lots of little pieces and when we get unexpected results, we can track them down. If suddenly my keyboard's "b" key stops working, the problem is almost certainly with the switch beneath it and not because of something the "h" key is doing. Another strength of the analogy is that the most disorienting behaviors of a keyboard are tied to state: the worst one for me is numlock off where the behavior shifts to dependence on the state of the display.
- tel 13y ago
  "Side-effect free" doesn't mean there are no side-effects, it simply means that their existence is controlled, separated, and minimized. For instance, in Haskell, you have the pure function "unwords . map upcase . words" which, reading right to left as is the convention for function composition, breaks a string on whitespace, upcases each chunk, then glues them back together with spaces again. It's net signature is "String -> String". By itself, this function is useless, but "interact" is a function with signature "(String -> String) -> IO ()" which means that it turns our "unwords . map upcase . words" into an interaction with the environment. There are whole debate here as to how interact is actually still pure, but for the purposes of this argument, interact is an impure effect. The net program however is very simple: "interact (unwords . map upcase . words)". The pure part is decomposed into many other pure pieces, each with simple, small contracts and independent testability. The impure part also has a simple contract which is described in terms of pure computation. Another great example is some of the streaming libraries available in Haskell today. Pipes for instance will let you compose a pipeline of pure and impure computations and have them work together nicely. "fibsPipe >-> Pipes.take 20 >-> Pipes.print" The first two segments are pure, but have empty "placeholder" slots for impure effects (identity monadic layers). These get merged seamlessly with the "Pipes.print" segment which has impure effects. All together, you get an infinite producer "fibsPipe" producing an impure effect "Pipes.print" while having its resource utilization and termination controlled by another pure computation "Pipes.take 20" which is actually maintaining its own implicit state (the current count of numbers it's let through). But then you can decompose each piece and understand it on its own in a pure or simple impure fashion.
- ajuc 13y ago
  Compiler is a good example of big program that's suitable for functional programming. You have specified input and output, no need for interactivity, and you can localize all mutations of state in one place (writing generated files to disk). Other batch processing programs works as well.
  - Athas 13y ago
    This is true. I'm currently working on a research compiler targeting GPUs. It's 7500 lines of Haskell with all comments and blank lines removed, and the only impure part is the command line interface. Avoiding IO is not really painful for these kinds of programs.
- saosebastiao 13y ago
  I'm not a Haskeller, nor do I write large amounts of software, but the one thing I took away from my very short foray into Haskell is that you want all of your state management and IO in one place. Clojure doesn't enforce this on you at all, but ever since learning that concept, my Clojure code has become an order of magnitude cleaner and less buggy.
- dragonwriter 13y ago
  > As someone who still doesn't get functional programming, I would like to see a real-world example of how to deal with side effects. Eg: write something into a file or a DB. Plenty of that here: http://book.realworldhaskell.org/read/ http://book.realworldhaskell.org/read/
- breckinloggins 13y ago
  You probably don't need Yet Another Reply to your question, but I can't resist. As others have noted, there's really no such thing as a "purely functional program" unless all you're looking for is turning your CPU into a space heater. That said, there's a very real sense in which the parts of your program that ARE pure are much safer and easier to reason about. The best way to visualize this is that you can theoretically turn an impure function into a pure one, watch: void printAGreeting(string greeting) { IO.printLn(greeting); } That's impure. The string goes into a black hole and does nothing from the program's perspective, but changes the state of the world from everyone else's perspective (it probably makes pixels change on a display somewhere). So how do you make that function pure? Like this: TheEntireUniverse printGreeting(string greeting, TheEntireUniverse u) { return u.makeSomeAtomsMoveAroundBobsConsole(greeting); } This function takes not only the string greeting, but everything else in the universe. You, me, everyone who's ever lived, bob's console, everything. It performs the same function by returning a new universe with everything the same except for the atoms moved to caused the greeting to appear on Bob's console. Obviously we can't do this, but (in Haskell, say) the IO monad is a context in which we understand that anything done inside of it has the affect of changing the "real world". If you pretend that you really had to do the whole "universe dragging" thing around for anything in the universe that WASN'T part of your program (we're assuming it itself is in some black hole somewhere to avoid paradoxes and such), then you would darn well make sure you kept this pain-in-the-ass operation as "thin" as possible. Why? Well, you're lazy! When you look at things that way, there's an analogy to writing stuff on bare metal hardware. You know you're going to have to write some assembly language (or maybe even direct machine code if you don't have an assembler). What you want is to get to the nice cozy world of C as soon as possible, so what you're going to do is use assembly to code only the smallest surface area you need so that writing in C makes sense, and everything else that doesn't need to be in assembly for performance sake you do in C. So it is in a purely functional language. The idea is to "get out of IO land" as fast as possible. To use a DB as an example, you'd like to live in a world where your little view of the DB could be expressed in purely functional terms (writing functions that result in a modified version of that view) and only at the VERY LAST MINUTE would you send that DB view to an IO function that actually sent the view down the wire to the DB. The same goes for files, sockets, gamepad IO, you name it. And you'd be surprised how little you need to actually interact with the world between getting your data and pushing out state changes. A game, for example, can do a WHOLE LOT OF STUFF between reading the input state and writing graphics commands.
gbog 13y ago
I have been tempted by functional programming, and it is interesting, but I'm not sure how would be the result for highly complex code.
- eru 13y ago
  It is especially suited to highly complex tasks. (Your code should stay as simple as possible.)
brudgers 13y ago
Rich Hickey has a great description of how to think about working with immutable values - what we usually want when we change a variable is simply the next value and so long as we are getting the value we expected, there's no need to name it. In other words if our current position in an array is 2, what we need to access the next position is the value 3. Creating a variable int i=2; and then mutating it i++; introduces the possibility of side effects. This not to say that at an abstraction layer below our programming language a register won't get incremented, only that our brains don't need to worry about the mechanisms most of the time if we use a functional language.
- dllthomas 13y ago
  Or a language with a foreach construct...
  - brudgers 13y ago
    Exactly. There's nothing magical about functional programming - it is no more than syntactic sugar to help programmers control the state of computing devices. Foreach is a higher level abstraction that makes reasoning about our code easier. The bits still change.
    - dllthomas 13y ago
      Functional programming has deep semantic implications. Calling it "syntactic sugar" is incorrect. Foreach in particular is arguably syntatic sugar, though, to be sure - functional or not.
    - Dewie 13y ago
      > There's nothing magical about functional programming - it is no more than syntactic sugar to help programmers control the state of computing devices. In the same way that objects in Java are just syntactic sugar for encapsulating state. In the same way that a compiler is just the machine that takes a language of the syntactic sugar of language A to the actual language A. > Foreach is a higher level abstraction that makes reasoning about our code easier. The bits still change. So how do you write a Foreach loop from a regular for-loop in your typical imperative language? How do you compose that Foreach loop to make even higher level abstractions like map, filter and fold/reduce?
taeric 13y ago
Has there been any exploration into an idea where the problem with side effects is not that they make functions impure, but they are often against the metaphor of the instructions being given? That is, if the metaphor that one is trying to model is traditional mathematics, then of course side effects are terrible. However, consider a program where the main metaphor is controlling something. Logo, for example. Few people argue, I would think, that the traditional imperative styling there hurts and confuses things. More extreme, consider stack based languages. These are strictly based on the current state of the program, yet my understanding is if you can fit your mind to that metaphor, it works very very well. Or, my favorite category, cookbooks. Look at traditional baking directions: "Begin heating oven to XXX, mix dry ingredients in bowl, add butter, whip, ..." Doesn't get any more imperative than that, and yet people around the world often have great success replicating the desired results. (Granted, I often think that the difference in programming and teaching is that humans make an effort to understand what you were communicating, computers typically don't.) Does this make sense?
- billsix 13y ago
  Guy Lewis Steele, Jr. and Gerald Jay Sussman. "The Art of the Interpreter of, the Modularity Complex (Parts Zero, One, and Two)". MIT AI Lab. AI Lab Memo AIM-453. May 1978 http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-453.pdf http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-...
  - taeric 13y ago
    Holy crap, just from the abstract I can't help but think many today would balk at the claim "More general side effects are also found to be necessary to promote modular style." Thank you very much for this link! I should have known to search for work by these authors. I can only dream of a world where I could have found their works more prominent in something like infoq.
  - agentultra 13y ago
    Thanks for the link to that paper! I recommend reading the section on "Top Level vs Referential Transparency" after reading this article. It argues that the trade-offs for absolute transparency require the programmer to define the entire world ahead of time. This trade-off is not acceptable by the authors of the paper for reasons elucidated in earlier sections and in SICP and so they demonstrate a version of EVAL whose environment allows for forward-references to top-level forms which we know is useful to writing dynamically compiled programs and decoupled modules. Every pure-FP language implementation has to make trade-offs with impure, mutable state and data. Haskell had to add the IO monad. Clojure has Var and Ref. Engineering isn't a discipline of absolutes. http://this-plt-life.tumblr.com/post/44462204757/simon-peyton-jones-adding-the-io-monad-to-haskell http://this-plt-life.tumblr.com/post/44462204757/simon-peyto...
SeoxyS 13y ago
The third example has an error: ((rand) > 0.5) should say (> (rand) 0.5)
- nonrecursive 13y ago
  Thanks, fixed!
  - JPKab 13y ago
    I like your post, and think we need more people like you doing this. One critique: I think you need to do a much better job explaining your first part on recursion. You need to walk through exactly what you are doing in your sum function, because it is not at all clear to the uninitiated where in the hell the "acc" variable comes from. Instead of explaining any of this part, which is perhaps the most difficult part for a non-FP person to grasp, you jumped straight to "by the way, use loop when there's a lot of numbers." My coworker's comments when I sent him the link: "Ok, I get it...... what's going on with the sum?..... whaaat?" Thanks for doing what you're doing. I look forward to reading more of your work.
    - nonrecursive 13y ago
      Thanks, this is exactly the kind of feedback I need. I'll work on clearing that up!
    - nonrecursive 13y ago
      OK - I've updated that section. I hope it helps!
      - JPKab 13y ago
        Yes, that is a fantastic explanation. My coworker immediately got it, and was simultaneously blown away by the concept of "arity." Please keep it up. One area you may be interested in: I've recently been playing around with Incanter (a library for data analysis, similar to R or Python Pandas). There is a distinct lack of tutorials for using it, and the documentation isn't exactly friendly.... The reason I mention it is because I think data analysis is a place where Clojure absolutely shines, and someone with your expertise could probably work it into useful examples.
kineticfocus 13y ago
Just watched this decent intro released yesterday... (OSCON 2013: "Functional Thinking" - Neal Ford) http://www.youtube.com/watch?v=7aYS9PcAITQ http://www.youtube.com/watch?v=7aYS9PcAITQ
alexvay 13y ago
I found LightTable* as a great way to grok functional programming - you can actually see how data moves around. Really brilliant. *Lighttable IDE: http://www.chris-granger.com/lighttable/ http://www.chris-granger.com/lighttable/
geuis 13y ago
I want to take a leap and talk about 2 core concepts that I got after spending last week learning Clojure. I'd love some comments about them. I've been writing javascript for years but functional programming didn't make sense until I realized: 1) It's essentially inverted callbacks. You can read and reason about your code by working from the inside out. The results of a function are passed up to the outside. 2) When you stop modifying variables and just pass them around, it becomes a lot easier to think about code in abstract ways. There are of course some situations where modifying a variable is needed, but it really feels icky and you think twice about doing it. The problem I found trying to learn this stuff is that once you learn to think functional its hard to think like someone who doesn't get it. That makes it hard to translate concepts in ways that non-functional people can get easier. I'm keeping mental tabs on ideas as I understand them to hopefully help with this problem. Also, macros. Clojure people love talking about them. They aren't explained well most of the time though. Simplest explanation is that you can write a bunch of your own functions, even ones that will override native functions, i.e macros. Since clojure is a compiled language, at compile time all the source code gets read in and pre-processed. When the compiler reaches some code that was defined in a macro, it keeps rewriting that code until all the macros are processed and the code has been re-written. Then everything is compiled to bytecode for the jvm.
- cgag 13y ago
  I think inverted callbacks is a strange way to describe just standard function calls. Unless you also call 'a.b' a callback, I don't think 'f a' could be called an inverted callback. It's just a standard function call without any special treatment of it's arguments (like self in most OO languages). I agree with you that it's hard to remember not thinking functionally. I've only been doing it for maybe 3 years and it's hard to remember what it felt like when I was first trying to wrap my head around it. With macro's, I think you mean to say your own special forms, not functions. They allow you to take in code as arguments, and determine when that code will get evaluated, letting you write things like "or" as macros, when it most languages they have to be implemented as built-in primitives (special forms).
  - geuis 13y ago
username223 13y ago
> It always returns the same result given the same arguments. This is call "referential transparency" and you can add it to your list of five-dollar programming terms. > It doesn't cause any side effects, e.g. it doesn't "change the external world" by changing external mutable objects or outputting to i/o. These have little to do with one another. "sort :: [a] -> [a]" sorts a list, perhaps using quicksort. If you want to randomize the pivots, it's "sort :: [a] -> IO [a]", and everything in your program that sorts things has a different type, even though it produces the same value.

162 points by nonrecursive 13y ago

pat_shaughnessy 13y ago

Another great post, Daniel... keep 'em coming!

nonrecursive 13y ago
Thanks!

jafaku 13y ago

As someone who still doesn't get functional programming, I would like to see a real-world example of how to deal with side effects. Eg: write something into a file or a DB. Because of course methods/functions with no side effects are easier to deal with, it sounds good, but I don't think it would be easy nor convenient to separate every side effect in a real-world program (as opposed to academic programs, where you just write a Fibonacci or whatever). Even in good OOP, you can have lots of side effects in a method. Halp!

ufo 13y ago
Using side effects in Haskell is easy. The only thing is that they need to be explicit and you aren't allowed to hide them inside pure functions. For a database example, here is the chapter from Real World Haskell about databases (a bit old but still relevant): http://book.realworldhaskell.org/read/using-databases.html http://book.realworldhaskell.org/read/using-databases.html And here is some example code: do conn <- connectSqlite3 "test1.db" stmt <- prepare conn "INSERT INTO test VALUES (?, ?)" execute stmt [toSql 1, toSql "one"] execute stmt [toSql 2, toSql "two"] execute stmt [toSql 3, toSql "three"] execute stmt [toSql 4, SqlNull] commit conn disconnect conn As you can see, it looks very similar to how you would write it in a regular imperative language. The main difference is that since they database operations are IO operations then they need to be in the special do-notation block and cannot be mixed with regular code. As for the question of wanting to mix pure and impure code that much I don't think it actually comes up that much. First of all, I find that I don't often find myself with an impure function that I want to add a side effect in the middle seamlessly - firstly, side-effecting stuff tend to be side-effecting from the beggining and secondly if I do want to turn a pure function into an impure one then forcing me to change the interface helps make sure that I am not breaking any code that used to work because it assumed it was working with pure code. Additionally, Haskell is really good at abstracting code so its not that hard to take a big chunk of logic and split it into a pure and an impure part. Finally, converting code from pure to impure is not that bad - sure, you need to change the syntax a bit but the type checker helps you find all the places that need to change and also helps you check all the code that calles the method you changed so you can be sure that you aren't breaking those. side effecting operations in the IO monad so you can't run them in the middle of any function and instead must run them inside a special "do-notation block" like I did. You can have impure code call both other impure code and pure functions and all you can't do is call impure functions from inside pure functions. Sure, its a bit annoying if you ever have to convert a pure function to an impure one (you need to change some of the syntax and all the places that call tha tfunction need to start treating it as impure) but its not that bad: first of all the type checker tells you all the spots where you need to update your code and secondly, Haskell is super good at abstracting code so its not hard to take a big chunk of code with some side effects in the middle and split it into a pure and an impure part. > to academic program Its a little pet peeve of mine but it seems that nowadays "academic" doesn't have a real meaning and just tends to refer to whatever people don't like. :)
- dllthomas 13y ago
  "do" is just syntax. You could write the above as: connectSqlite3 "test1.db" >>= \ conn -> prepare conn "INSERT INTO test VALUES (?, ?)" >>= \ stmt -> >> execute stmt [toSql 1, toSql "one"] >> execute stmt [toSql 2, toSql "two"] >> execute stmt [toSql 3, toSql "three"] >> execute stmt [toSql 4, SqlNull] >> commit conn >> disconnect conn and it's "just" a bunch of nested lambdas with no "do" in sight. What's different from "regular" code is that those lambdas 1) return IO actions, and 2) are strung together with monadic bind (>>=) to build one big IO action. To be sure, in this case the do version is much easier to read (that's why "do notation" exists). It can be used with any monad, though - nothing ties it particularly to IO.
  - ufo 13y ago
    Maybe I should have been clearer but the basic point is that monadic code has a different interface bar (foo x) --pure Haskell vs foo x >>= bar --monadic Haskell vs bar(foo(x)) -- in C you write both cases like this.
    - dllthomas 13y ago
      You're close, but it's not really so clear cut. foo x ++ bar -- "pure" or "monadic"? foo x >>= baz -- "pure" or "monadic"? What if (foo x) is a list in both cases? (>>=) is just an operator. The values that it operates on are anything with a Monad instance, just like the values that (+) operates on are anything with a Num instance. There is nothing voodoo about monads or about bind (>>=); such voodoo as there is lies solely in IO, which is interacted with the same way as other values (though certainly what you can do with (IO a) is more limited than what you can do with (Maybe a)).
      - Dewie 13y ago
        I used to think that Monad was some alternative realm in Haskell, a realm where purity didn't exist and everything was written in an alternative, built-in language (do-notation). That's because people tend to refer to using Monads as "working in the X-Monad", as if it is some... place. But then it turned out... Oh, so it's basically just a signature/interface/type class.
daGrevis 13y ago
I suggest to dig in some web framework for functional programming. One example would be Snap for Haskell. http://snapframework.com/ http://snapframework.com/
- jafaku 13y ago
  Does it make sense to have to learn Haskell and a Haskell framework just to understand Clojure? Because I'm already sold on the idea that Clojure is the best language™, being a lisp, portable, with a great community, already with a mature library for everything (because it has access to the JVM), and with some dirty shortcuts for convenience. But the frameworks are lacking. The only web framework available got dissolved, and I don't have the energy to wire everything myself in a new language and paradigm that I don't even fully understand).
  - daGrevis 13y ago
    > Does it make sense to have to learn Haskell and a Haskell framework just to understand Clojure? I have a strong feeling that once you understand either Haskell or Clojure, it will be a breeze to learn other. My point was to get framework for functional programming that deals with a lot of I/O and side-effects and see docs for real-life examples of usage.
    - hisham_hm 13y ago
      > I have a strong feeling that once you understand either Haskell or Clojure, it will be a breeze to learn other. Does Clojure use lazy evaluation?
      - dllthomas 13y ago
        It also isn't statically typed.
      - cgag 13y ago
        Not everywhere, but most of the sequence functions like map/filter return lazy sequences.
nonrecursive 13y ago
One thing I find myself doing over and over is taking as much logic as possible out of side effecting functions and placing that in a pure function, then passing the return value to the side effecting function.
TheZenPsycho 13y ago
In the end, there is always some kind of side effect. The value that functional programming adds is you keep the part of the program that does the side effect in one spot, doing one specific well defined thoroughly debugged thing and you limit its scope so it can't screw up the rest of your program's operation no matter how bad it goes wrong. It only goes "out". it doesn't cause any side effect inside your program. One example of a way to structure a functional program is categorising functions according to whether they are a "source" a "filter" or a "sink", (terms stolen from circuit design). The source being your inputs (not pure), filters doing some transformation on the input, over time (pure) and handing off to a sink that causes some side effect (not pure). In the "real world" You've probably made a lot of simple programs that mostly follow this structure if you've used unix pipes. If you go along with this, you find that once you start making more sophisticated programs, you need some kind of "memory" so you can progress. Strictly speaking, memory, as we typically use it in imperative languages, is a kind of side effect. In the model above, though, you can make a special kind of "source" which is something like a feedback loop- a source that contains the output of the previous iteration of the program- or some sub portion of the program. Then maintaining a memory becomes a process of continually taking some input, transforming it, creating an output, and then using that output as an input again. (what you might call recursion) Why would you do this? Because each part of a program like this is self contained, and its ability to do damage to some distant part of a program is extremely limited. It becomes much easier to reason about such a program. When you find a bug (and these are still possible) you can trace the origin from output to source, or source to output. There is no possibility of some phantom distant subroutine screwing up the dataflow. And this kind of structure is very useful for games, interactive multimedia programs, and other such graphical interactive program. A commonly cited "real world" example is xmonad. What I find more insightful is this article on replicating pacman in a functional language. http://prog21.dadgum.com/23.html http://prog21.dadgum.com/23.html
brudgers 13y ago
The idea is to generate side effects that don't effect program flow. This keyboard generates side effects as I type because it changes the state of the display. But pressing the "B" to start this sentence did not change the way the keyboard operates - each key press produced the same results as it would had I pressed "H". And had I pressed "H" and begun the previous sentence with "However," the screen layout would have been somewhat different, but not in a way I typically care about when writing (as opposed to doing typography). Now this is not to say that my keyboard is purely functional. The shift key creates state. But that state is isolated and short lived to the point where we can treat it as functional across two key combinations. It is the caps lock key which creates an unpredictable state - I do not know what output I will get without checking its status - the problem with state might be illustrated by the results of hitting capslock when I meant to hit tab while typing a program.
- jafaku 13y ago
  So how should you program the capslock? You (the functional guys) keep telling me what's wrong, but you don't tell me how to do it right. Not having a capslock key is not an option, people need it.
  - TheZenPsycho 13y ago
    Actually a lot of people have decided they don't need it. They remap caps lock to something like spotlight search. The alternative to the caps lock key is the shift key. And this is a really bad metaphor for functional programming. Sorry GP. I suppose the important point here is it's not possible for something going wrong with your monitor or speakers to suddenly remap your keyboard without warning.
    - brudgers 13y ago
      I'm not saying it's a great analogy, but among it's strengths is handiness. Among it's other strengths is that functional programs can be built bottom up from lots of little pieces and when we get unexpected results, we can track them down. If suddenly my keyboard's "b" key stops working, the problem is almost certainly with the switch beneath it and not because of something the "h" key is doing. Another strength of the analogy is that the most disorienting behaviors of a keyboard are tied to state: the worst one for me is numlock off where the behavior shifts to dependence on the state of the display.
tel 13y ago
"Side-effect free" doesn't mean there are no side-effects, it simply means that their existence is controlled, separated, and minimized. For instance, in Haskell, you have the pure function "unwords . map upcase . words" which, reading right to left as is the convention for function composition, breaks a string on whitespace, upcases each chunk, then glues them back together with spaces again. It's net signature is "String -> String". By itself, this function is useless, but "interact" is a function with signature "(String -> String) -> IO ()" which means that it turns our "unwords . map upcase . words" into an interaction with the environment. There are whole debate here as to how interact is actually still pure, but for the purposes of this argument, interact is an impure effect. The net program however is very simple: "interact (unwords . map upcase . words)". The pure part is decomposed into many other pure pieces, each with simple, small contracts and independent testability. The impure part also has a simple contract which is described in terms of pure computation. Another great example is some of the streaming libraries available in Haskell today. Pipes for instance will let you compose a pipeline of pure and impure computations and have them work together nicely. "fibsPipe >-> Pipes.take 20 >-> Pipes.print" The first two segments are pure, but have empty "placeholder" slots for impure effects (identity monadic layers). These get merged seamlessly with the "Pipes.print" segment which has impure effects. All together, you get an infinite producer "fibsPipe" producing an impure effect "Pipes.print" while having its resource utilization and termination controlled by another pure computation "Pipes.take 20" which is actually maintaining its own implicit state (the current count of numbers it's let through). But then you can decompose each piece and understand it on its own in a pure or simple impure fashion.
ajuc 13y ago
Compiler is a good example of big program that's suitable for functional programming. You have specified input and output, no need for interactivity, and you can localize all mutations of state in one place (writing generated files to disk). Other batch processing programs works as well.
- Athas 13y ago
  This is true. I'm currently working on a research compiler targeting GPUs. It's 7500 lines of Haskell with all comments and blank lines removed, and the only impure part is the command line interface. Avoiding IO is not really painful for these kinds of programs.
saosebastiao 13y ago
I'm not a Haskeller, nor do I write large amounts of software, but the one thing I took away from my very short foray into Haskell is that you want all of your state management and IO in one place. Clojure doesn't enforce this on you at all, but ever since learning that concept, my Clojure code has become an order of magnitude cleaner and less buggy.
dragonwriter 13y ago
> As someone who still doesn't get functional programming, I would like to see a real-world example of how to deal with side effects. Eg: write something into a file or a DB. Plenty of that here: http://book.realworldhaskell.org/read/ http://book.realworldhaskell.org/read/
breckinloggins 13y ago
You probably don't need Yet Another Reply to your question, but I can't resist. As others have noted, there's really no such thing as a "purely functional program" unless all you're looking for is turning your CPU into a space heater. That said, there's a very real sense in which the parts of your program that ARE pure are much safer and easier to reason about. The best way to visualize this is that you can theoretically turn an impure function into a pure one, watch: void printAGreeting(string greeting) { IO.printLn(greeting); } That's impure. The string goes into a black hole and does nothing from the program's perspective, but changes the state of the world from everyone else's perspective (it probably makes pixels change on a display somewhere). So how do you make that function pure? Like this: TheEntireUniverse printGreeting(string greeting, TheEntireUniverse u) { return u.makeSomeAtomsMoveAroundBobsConsole(greeting); } This function takes not only the string greeting, but everything else in the universe. You, me, everyone who's ever lived, bob's console, everything. It performs the same function by returning a new universe with everything the same except for the atoms moved to caused the greeting to appear on Bob's console. Obviously we can't do this, but (in Haskell, say) the IO monad is a context in which we understand that anything done inside of it has the affect of changing the "real world". If you pretend that you really had to do the whole "universe dragging" thing around for anything in the universe that WASN'T part of your program (we're assuming it itself is in some black hole somewhere to avoid paradoxes and such), then you would darn well make sure you kept this pain-in-the-ass operation as "thin" as possible. Why? Well, you're lazy! When you look at things that way, there's an analogy to writing stuff on bare metal hardware. You know you're going to have to write some assembly language (or maybe even direct machine code if you don't have an assembler). What you want is to get to the nice cozy world of C as soon as possible, so what you're going to do is use assembly to code only the smallest surface area you need so that writing in C makes sense, and everything else that doesn't need to be in assembly for performance sake you do in C. So it is in a purely functional language. The idea is to "get out of IO land" as fast as possible. To use a DB as an example, you'd like to live in a world where your little view of the DB could be expressed in purely functional terms (writing functions that result in a modified version of that view) and only at the VERY LAST MINUTE would you send that DB view to an IO function that actually sent the view down the wire to the DB. The same goes for files, sockets, gamepad IO, you name it. And you'd be surprised how little you need to actually interact with the world between getting your data and pushing out state changes. A game, for example, can do a WHOLE LOT OF STUFF between reading the input state and writing graphics commands.

gbog 13y ago

I have been tempted by functional programming, and it is interesting, but I'm not sure how would be the result for highly complex code.

eru 13y ago
It is especially suited to highly complex tasks. (Your code should stay as simple as possible.)

brudgers 13y ago

Rich Hickey has a great description of how to think about working with immutable values - what we usually want when we change a variable is simply the next value and so long as we are getting the value we expected, there's no need to name it. In other words if our current position in an array is 2, what we need to access the next position is the value 3. Creating a variable int i=2; and then mutating it i++; introduces the possibility of side effects. This not to say that at an abstraction layer below our programming language a register won't get incremented, only that our brains don't need to worry about the mechanisms most of the time if we use a functional language.

dllthomas 13y ago
Or a language with a foreach construct...
- brudgers 13y ago
  Exactly. There's nothing magical about functional programming - it is no more than syntactic sugar to help programmers control the state of computing devices. Foreach is a higher level abstraction that makes reasoning about our code easier. The bits still change.
  - dllthomas 13y ago
    Functional programming has deep semantic implications. Calling it "syntactic sugar" is incorrect. Foreach in particular is arguably syntatic sugar, though, to be sure - functional or not.
  - Dewie 13y ago
    > There's nothing magical about functional programming - it is no more than syntactic sugar to help programmers control the state of computing devices. In the same way that objects in Java are just syntactic sugar for encapsulating state. In the same way that a compiler is just the machine that takes a language of the syntactic sugar of language A to the actual language A. > Foreach is a higher level abstraction that makes reasoning about our code easier. The bits still change. So how do you write a Foreach loop from a regular for-loop in your typical imperative language? How do you compose that Foreach loop to make even higher level abstractions like map, filter and fold/reduce?

taeric 13y ago

Has there been any exploration into an idea where the problem with side effects is not that they make functions impure, but they are often against the metaphor of the instructions being given? That is, if the metaphor that one is trying to model is traditional mathematics, then of course side effects are terrible. However, consider a program where the main metaphor is controlling something. Logo, for example. Few people argue, I would think, that the traditional imperative styling there hurts and confuses things. More extreme, consider stack based languages. These are strictly based on the current state of the program, yet my understanding is if you can fit your mind to that metaphor, it works very very well. Or, my favorite category, cookbooks. Look at traditional baking directions: "Begin heating oven to XXX, mix dry ingredients in bowl, add butter, whip, ..." Doesn't get any more imperative than that, and yet people around the world often have great success replicating the desired results. (Granted, I often think that the difference in programming and teaching is that humans make an effort to understand what you were communicating, computers typically don't.) Does this make sense?

billsix 13y ago
Guy Lewis Steele, Jr. and Gerald Jay Sussman. "The Art of the Interpreter of, the Modularity Complex (Parts Zero, One, and Two)". MIT AI Lab. AI Lab Memo AIM-453. May 1978 http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-453.pdf http://repository.readscheme.org/ftp/papers/ai-lab-pubs/AIM-...
- taeric 13y ago
  Holy crap, just from the abstract I can't help but think many today would balk at the claim "More general side effects are also found to be necessary to promote modular style." Thank you very much for this link! I should have known to search for work by these authors. I can only dream of a world where I could have found their works more prominent in something like infoq.
- agentultra 13y ago
  Thanks for the link to that paper! I recommend reading the section on "Top Level vs Referential Transparency" after reading this article. It argues that the trade-offs for absolute transparency require the programmer to define the entire world ahead of time. This trade-off is not acceptable by the authors of the paper for reasons elucidated in earlier sections and in SICP and so they demonstrate a version of EVAL whose environment allows for forward-references to top-level forms which we know is useful to writing dynamically compiled programs and decoupled modules. Every pure-FP language implementation has to make trade-offs with impure, mutable state and data. Haskell had to add the IO monad. Clojure has Var and Ref. Engineering isn't a discipline of absolutes. http://this-plt-life.tumblr.com/post/44462204757/simon-peyton-jones-adding-the-io-monad-to-haskell http://this-plt-life.tumblr.com/post/44462204757/simon-peyto...

SeoxyS 13y ago

The third example has an error: ((rand) > 0.5) should say (> (rand) 0.5)

nonrecursive 13y ago
Thanks, fixed!
- JPKab 13y ago
  I like your post, and think we need more people like you doing this. One critique: I think you need to do a much better job explaining your first part on recursion. You need to walk through exactly what you are doing in your sum function, because it is not at all clear to the uninitiated where in the hell the "acc" variable comes from. Instead of explaining any of this part, which is perhaps the most difficult part for a non-FP person to grasp, you jumped straight to "by the way, use loop when there's a lot of numbers." My coworker's comments when I sent him the link: "Ok, I get it...... what's going on with the sum?..... whaaat?" Thanks for doing what you're doing. I look forward to reading more of your work.
  - nonrecursive 13y ago
    Thanks, this is exactly the kind of feedback I need. I'll work on clearing that up!
  - nonrecursive 13y ago
    OK - I've updated that section. I hope it helps!
    - JPKab 13y ago
      Yes, that is a fantastic explanation. My coworker immediately got it, and was simultaneously blown away by the concept of "arity." Please keep it up. One area you may be interested in: I've recently been playing around with Incanter (a library for data analysis, similar to R or Python Pandas). There is a distinct lack of tutorials for using it, and the documentation isn't exactly friendly.... The reason I mention it is because I think data analysis is a place where Clojure absolutely shines, and someone with your expertise could probably work it into useful examples.

kineticfocus 13y ago

Just watched this decent intro released yesterday... (OSCON 2013: "Functional Thinking" - Neal Ford) http://www.youtube.com/watch?v=7aYS9PcAITQ http://www.youtube.com/watch?v=7aYS9PcAITQ

alexvay 13y ago

I found LightTable* as a great way to grok functional programming - you can actually see how data moves around. Really brilliant. *Lighttable IDE: http://www.chris-granger.com/lighttable/ http://www.chris-granger.com/lighttable/

geuis 13y ago

I want to take a leap and talk about 2 core concepts that I got after spending last week learning Clojure. I'd love some comments about them. I've been writing javascript for years but functional programming didn't make sense until I realized: 1) It's essentially inverted callbacks. You can read and reason about your code by working from the inside out. The results of a function are passed up to the outside. 2) When you stop modifying variables and just pass them around, it becomes a lot easier to think about code in abstract ways. There are of course some situations where modifying a variable is needed, but it really feels icky and you think twice about doing it. The problem I found trying to learn this stuff is that once you learn to think functional its hard to think like someone who doesn't get it. That makes it hard to translate concepts in ways that non-functional people can get easier. I'm keeping mental tabs on ideas as I understand them to hopefully help with this problem. Also, macros. Clojure people love talking about them. They aren't explained well most of the time though. Simplest explanation is that you can write a bunch of your own functions, even ones that will override native functions, i.e macros. Since clojure is a compiled language, at compile time all the source code gets read in and pre-processed. When the compiler reaches some code that was defined in a macro, it keeps rewriting that code until all the macros are processed and the code has been re-written. Then everything is compiled to bytecode for the jvm.

cgag 13y ago
I think inverted callbacks is a strange way to describe just standard function calls. Unless you also call 'a.b' a callback, I don't think 'f a' could be called an inverted callback. It's just a standard function call without any special treatment of it's arguments (like self in most OO languages). I agree with you that it's hard to remember not thinking functionally. I've only been doing it for maybe 3 years and it's hard to remember what it felt like when I was first trying to wrap my head around it. With macro's, I think you mean to say your own special forms, not functions. They allow you to take in code as arguments, and determine when that code will get evaluated, letting you write things like "or" as macros, when it most languages they have to be implemented as built-in primitives (special forms).
- geuis 13y ago

username223 13y ago

> It always returns the same result given the same arguments. This is call "referential transparency" and you can add it to your list of five-dollar programming terms. > It doesn't cause any side effects, e.g. it doesn't "change the external world" by changing external mutable objects or outputting to i/o. These have little to do with one another. "sort :: [a] -> [a]" sorts a list, perhaps using quicksort. If you want to randomize the pivots, it's "sort :: [a] -> IO [a]", and everything in your program that sorts things has a different type, even though it produces the same value.