3 ms·
Thinking in data & functions just takes some practice. Traditionally imperative code can often be reimagined in terms of reducing over a command sequence, or in
by brandonbloom 11y ago
Thinking in data & functions just takes some practice. Traditionally imperative code can often be reimagined in terms of reducing over a command sequence, or interpreting some more sophisticated representation. The benefits for implementation, testing, debugging, refactoring, etc can be massive.
Consider your chain of examples for "Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc." You could represent this as a sequence of steps which could be interpreted by a central side-effecting loop. Let's call it an "action plan", in this case for your "upload" operation:
(defn upload-plan [req]
[{:action :upload
:file (get-in req [:params :doc])
:max-size 2000
:name :doc}
{:action :notify
:message (fn [state] (get-in state [:data :doc]))
:target [:somebody]}])
Now you can process this plan centrally:
(defmulti take-action (fn [state {:keys [action]}] action))
(defmethod take-action :upload ...)
(defmethod take-action :notify ...)
(defn execute [plan]
(let [{:keys [status]} (reduce take-action {:status :ok, :data {}} plan)]
(when (not= status :ok)
(log "omg!"))))
While this is initially a tad more verbose, there's great reasons to do this sort of thing.
For example, now have a central place to mock all side effects! You could substitute a different execute method during testing, or automatically log the "action" object that failed.
More importantly, you can develop each of your "plan" functions independently. You can reuse them, re-order them, etc.
You can also do hypotheticals, for example: "Which plan would cause less network round trips?" can be answered before committing to a plan.
You can _record_ plans. Instrument your execute method to record and save plans to a file, then you can just check them right in as test cases.
You can optimize plans. If five different functions all notify the same target of the same thing, your central execution method can filter subsequent calls. Or you could separate your "optimization" from your "interpretation" and you can ask "After accounting for idempotence, which operations will be performed?"
The list goes on and on...
- phamilton 11y agoAs an aside, you just (sort of) implented the IO monad. The simplest explanation of the IO monad is that your Haskell application outputs a recipe or "execution plan". This output is side effect free and pure. The runtime then executes your plan, producing those side effects.
- brandonbloom 11y agoOne key difference between this and the IO monad or a free monad: There's no continuation. That is, you can't communicate between the code that produces the command sequence and the code that interprets it. This is either a shortcoming, or a dramatic simplification, depending on your needs. I find that it's often the latter, since it enforces a one-way information flow.
- phamilton 11y agoI suppose that would be the "monad" part.
- daxfohl 11y agoI looked at this, and isn't it inside-out though? Given that code/data equivalence being the fundamental thing that makes a lisp a lisp, wouldn't the lisp way to go about it be, to write a whole-program-transformation macro that can replace certain function calls with specified equivalents? That way you can write code just like normal, but have it do completely different things based on the context. The idea of being able to do this is what made me consider a lisp in the first place. Yet I don't see much evidence of people working in that direction.
- brandonbloom 11y agoThere's a lot of stuff packed in to your comment here. The code/data equivalence isn't necessary to make this data-first approach work. Consider JavaScript, where it's quite popular to power complex processes with simple JSON (maybe + extensions, like functions). The approach also works well in typed functional languages, albeit with greater declaration burden. Moreover, it would even work well in languages like Java, if you're willing to buck the XML trend. That said, data being a subset of code (which is also implied by equivalence) is what makes this technique much more popular in Lisps (and JavaScript). Being able to easily move forms between evaluation, quoting, interpretation, etc is critical to evolving the right approach. Regarding whole program transforms: They don't compose, at least not without a substantially more sophisticated macro system (like Racket's) or evaluator (see work on "extensible interpreters"). Check out Racket's rich literature on macros if you want to see the state of the art here. Lastly, changing behavior based on context can be accomplished by many techniques including interpretation, dynamically scoped binding, etc. Read about Common Lisp's condition system, or Eff's "Algebraic Effects and Handlers" to see the state of the art of the dynamic substitution of effects. However, each new context needs to carefully reason about the interleaving of effects, since time is linear. If you want more flexibility with time, you have to turn it in to space. That's what happens when you take a step-by-step procedure and turn it in to data you can interpret. Consider code that prints a lot of stuff. Instead, you can return a lazy sequence of strings and then at the very end do something like `(apply str parts)` to concatenate it all together. Now that each sub-printer doesn't have side effects, you can re-order and refactor the calls without worrying about messing up the printed output. You've decoupled the evaluation timeline from the linear order of the output.