6 ms·

Really nice writeup of patterns. What turned me off Clojure is that the dynamic typing makes it very hard to refactor. e.g. even renaming a function is really q

Really nice writeup of patterns. What turned me off Clojure is that the dynamic typing makes it very hard to refactor. e.g. even renaming a function is really quite difficult, let alone adding function parameters. Forget extracting a function. The dynamic typing also means a typo can yield a nil that travels far from the mistake site making debugging difficult. There are hints towards better debugging in clojure, but last time I checked it was still very much a work in progress. Also lazy lists are easy to screw up, e.g. if you put a concat inside a reduce then stack go boom. Still this is a great post because it teaches you both good Clojure and the design patterns. Since java has closures now you can take a lot of what you learn from Clojure and put it in your well typed and refactorable Java programs.

yenda 11y ago
Refactoring is a breeze when your editor is connected to the repl, you should try clj-refactor for instance. I'm not sure what you mean with the typo story since most of the functions in your program should be free of side effects you should be able to test them as you write them.
- daxfohl 11y ago
  I'm always curious about the latter sentiment. Maybe I just haven't been on enough interesting projects, but I've never had a program that didn't have/need side-effecting code nearly every other line. Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc. Pure data manipulation has always been a fairly minor component limited to dashboard statistics for me. Is purity that really your experience? What industry / types of projects?
  - brandonbloom 11y ago
    Thinking in data & functions just takes some practice. Traditionally imperative code can often be reimagined in terms of reducing over a command sequence, or interpreting some more sophisticated representation. The benefits for implementation, testing, debugging, refactoring, etc can be massive. Consider your chain of examples for "Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc." You could represent this as a sequence of steps which could be interpreted by a central side-effecting loop. Let's call it an "action plan", in this case for your "upload" operation: (defn upload-plan [req] [{:action :upload :file (get-in req [:params :doc]) :max-size 2000 :name :doc} {:action :notify :message (fn [state] (get-in state [:data :doc])) :target [:somebody]}]) Now you can process this plan centrally: (defmulti take-action (fn [state {:keys [action]}] action)) (defmethod take-action :upload ...) (defmethod take-action :notify ...) (defn execute [plan] (let [{:keys [status]} (reduce take-action {:status :ok, :data {}} plan)] (when (not= status :ok) (log "omg!")))) While this is initially a tad more verbose, there's great reasons to do this sort of thing. For example, now have a central place to mock all side effects! You could substitute a different execute method during testing, or automatically log the "action" object that failed. More importantly, you can develop each of your "plan" functions independently. You can reuse them, re-order them, etc. You can also do hypotheticals, for example: "Which plan would cause less network round trips?" can be answered before committing to a plan. You can _record_ plans. Instrument your execute method to record and save plans to a file, then you can just check them right in as test cases. You can optimize plans. If five different functions all notify the same target of the same thing, your central execution method can filter subsequent calls. Or you could separate your "optimization" from your "interpretation" and you can ask "After accounting for idempotence, which operations will be performed?" The list goes on and on...
    - phamilton 11y ago
      As an aside, you just (sort of) implented the IO monad. The simplest explanation of the IO monad is that your Haskell application outputs a recipe or "execution plan". This output is side effect free and pure. The runtime then executes your plan, producing those side effects.
      - brandonbloom 11y ago
        One key difference between this and the IO monad or a free monad: There's no continuation. That is, you can't communicate between the code that produces the command sequence and the code that interprets it. This is either a shortcoming, or a dramatic simplification, depending on your needs. I find that it's often the latter, since it enforces a one-way information flow.
        phamilton 11y ago
        I suppose that would be the "monad" part.
        daxfohl 11y ago
        I looked at this, and isn't it inside-out though? Given that code/data equivalence being the fundamental thing that makes a lisp a lisp, wouldn't the lisp way to go about it be, to write a whole-program-transformation macro that can replace certain function calls with specified equivalents? That way you can write code just like normal, but have it do completely different things based on the context. The idea of being able to do this is what made me consider a lisp in the first place. Yet I don't see much evidence of people working in that direction.
        brandonbloom 11y ago
        There's a lot of stuff packed in to your comment here. The code/data equivalence isn't necessary to make this data-first approach work. Consider JavaScript, where it's quite popular to power complex processes with simple JSON (maybe + extensions, like functions). The approach also works well in typed functional languages, albeit with greater declaration burden. Moreover, it would even work well in languages like Java, if you're willing to buck the XML trend. That said, data being a subset of code (which is also implied by equivalence) is what makes this technique much more popular in Lisps (and JavaScript). Being able to easily move forms between evaluation, quoting, interpretation, etc is critical to evolving the right approach. Regarding whole program transforms: They don't compose, at least not without a substantially more sophisticated macro system (like Racket's) or evaluator (see work on "extensible interpreters"). Check out Racket's rich literature on macros if you want to see the state of the art here. Lastly, changing behavior based on context can be accomplished by many techniques including interpretation, dynamically scoped binding, etc. Read about Common Lisp's condition system, or Eff's "Algebraic Effects and Handlers" to see the state of the art of the dynamic substitution of effects. However, each new context needs to carefully reason about the interleaving of effects, since time is linear. If you want more flexibility with time, you have to turn it in to space. That's what happens when you take a step-by-step procedure and turn it in to data you can interpret. Consider code that prints a lot of stuff. Instead, you can return a lazy sequence of strings and then at the very end do something like `(apply str parts)` to concatenate it all together. Now that each sub-printer doesn't have side effects, you can re-order and refactor the calls without worrying about messing up the printed output. You've decoupled the evaluation timeline from the linear order of the output.
elsen 11y ago
A typo would yield a "Unable to resolve symbol" Error in Clojure and a Warning in Clojurescript. Both at compile time. Cursive IDE (IntelliJ based) has no problem renaming functions. These are all tooling issues, not specific to Clojure nor Dynamic Typing. Give it time & give it a try!
- richcole2 11y ago
  Try :mssage instead of :message and see if you get a compile time error
catnaroek 11y ago
Not that you should necessarily use it as a production language, but I'd suggest checking Racket if you want to see dynamic typing done right. `#lang racket` doesn't have static types, but it has dynamically checked contracts with a blame feature that pinpoints at the part of the program that is responsible for a contract violation. `#lang typed/racket` has a more traditional type system (programs are checked before they even run), but the type checker is implemented entirely as a series of Racket macros, and the resulting program integrates seamlessly with untyped Racket code. Regarding refactoring, DrRacket does a pretty good job of identifying dependencies between defined symbols, even across modules. It's true that more popular dynamic languages (JavaScript, Python, etc.) are bad at tracking useful static information, but you shouldn't generalize from this to all dynamic languages.

yenda 11y ago

Refactoring is a breeze when your editor is connected to the repl, you should try clj-refactor for instance. I'm not sure what you mean with the typo story since most of the functions in your program should be free of side effects you should be able to test them as you write them.

daxfohl 11y ago
I'm always curious about the latter sentiment. Maybe I just haven't been on enough interesting projects, but I've never had a program that didn't have/need side-effecting code nearly every other line. Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc. Pure data manipulation has always been a fairly minor component limited to dashboard statistics for me. Is purity that really your experience? What industry / types of projects?
- brandonbloom 11y ago
  Thinking in data & functions just takes some practice. Traditionally imperative code can often be reimagined in terms of reducing over a command sequence, or interpreting some more sophisticated representation. The benefits for implementation, testing, debugging, refactoring, etc can be massive. Consider your chain of examples for "Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc." You could represent this as a sequence of steps which could be interpreted by a central side-effecting loop. Let's call it an "action plan", in this case for your "upload" operation: (defn upload-plan [req] [{:action :upload :file (get-in req [:params :doc]) :max-size 2000 :name :doc} {:action :notify :message (fn [state] (get-in state [:data :doc])) :target [:somebody]}]) Now you can process this plan centrally: (defmulti take-action (fn [state {:keys [action]}] action)) (defmethod take-action :upload ...) (defmethod take-action :notify ...) (defn execute [plan] (let [{:keys [status]} (reduce take-action {:status :ok, :data {}} plan)] (when (not= status :ok) (log "omg!")))) While this is initially a tad more verbose, there's great reasons to do this sort of thing. For example, now have a central place to mock all side effects! You could substitute a different execute method during testing, or automatically log the "action" object that failed. More importantly, you can develop each of your "plan" functions independently. You can reuse them, re-order them, etc. You can also do hypotheticals, for example: "Which plan would cause less network round trips?" can be answered before committing to a plan. You can _record_ plans. Instrument your execute method to record and save plans to a file, then you can just check them right in as test cases. You can optimize plans. If five different functions all notify the same target of the same thing, your central execution method can filter subsequent calls. Or you could separate your "optimization" from your "interpretation" and you can ask "After accounting for idempotence, which operations will be performed?" The list goes on and on...
  - phamilton 11y ago
    As an aside, you just (sort of) implented the IO monad. The simplest explanation of the IO monad is that your Haskell application outputs a recipe or "execution plan". This output is side effect free and pure. The runtime then executes your plan, producing those side effects.
    - brandonbloom 11y ago
      One key difference between this and the IO monad or a free monad: There's no continuation. That is, you can't communicate between the code that produces the command sequence and the code that interprets it. This is either a shortcoming, or a dramatic simplification, depending on your needs. I find that it's often the latter, since it enforces a one-way information flow.
      - phamilton 11y ago
        I suppose that would be the "monad" part.
      - daxfohl 11y ago
        I looked at this, and isn't it inside-out though? Given that code/data equivalence being the fundamental thing that makes a lisp a lisp, wouldn't the lisp way to go about it be, to write a whole-program-transformation macro that can replace certain function calls with specified equivalents? That way you can write code just like normal, but have it do completely different things based on the context. The idea of being able to do this is what made me consider a lisp in the first place. Yet I don't see much evidence of people working in that direction.
        brandonbloom 11y ago
        There's a lot of stuff packed in to your comment here. The code/data equivalence isn't necessary to make this data-first approach work. Consider JavaScript, where it's quite popular to power complex processes with simple JSON (maybe + extensions, like functions). The approach also works well in typed functional languages, albeit with greater declaration burden. Moreover, it would even work well in languages like Java, if you're willing to buck the XML trend. That said, data being a subset of code (which is also implied by equivalence) is what makes this technique much more popular in Lisps (and JavaScript). Being able to easily move forms between evaluation, quoting, interpretation, etc is critical to evolving the right approach. Regarding whole program transforms: They don't compose, at least not without a substantially more sophisticated macro system (like Racket's) or evaluator (see work on "extensible interpreters"). Check out Racket's rich literature on macros if you want to see the state of the art here. Lastly, changing behavior based on context can be accomplished by many techniques including interpretation, dynamically scoped binding, etc. Read about Common Lisp's condition system, or Eff's "Algebraic Effects and Handlers" to see the state of the art of the dynamic substitution of effects. However, each new context needs to carefully reason about the interleaving of effects, since time is linear. If you want more flexibility with time, you have to turn it in to space. That's what happens when you take a step-by-step procedure and turn it in to data you can interpret. Consider code that prints a lot of stuff. Instead, you can return a lazy sequence of strings and then at the very end do something like `(apply str parts)` to concatenate it all together. Now that each sub-printer doesn't have side effects, you can re-order and refactor the calls without worrying about messing up the printed output. You've decoupled the evaluation timeline from the linear order of the output.

elsen 11y ago

A typo would yield a "Unable to resolve symbol" Error in Clojure and a Warning in Clojurescript. Both at compile time. Cursive IDE (IntelliJ based) has no problem renaming functions. These are all tooling issues, not specific to Clojure nor Dynamic Typing. Give it time & give it a try!

richcole2 11y ago
Try :mssage instead of :message and see if you get a compile time error

catnaroek 11y ago

Not that you should necessarily use it as a production language, but I'd suggest checking Racket if you want to see dynamic typing done right. `#lang racket` doesn't have static types, but it has dynamically checked contracts with a blame feature that pinpoints at the part of the program that is responsible for a contract violation. `#lang typed/racket` has a more traditional type system (programs are checked before they even run), but the type checker is implemented entirely as a series of Racket macros, and the resulting program integrates seamlessly with untyped Racket code. Regarding refactoring, DrRacket does a pretty good job of identifying dependencies between defined symbols, even across modules. It's true that more popular dynamic languages (JavaScript, Python, etc.) are bad at tracking useful static information, but you shouldn't generalize from this to all dynamic languages.