10 ms·
Robust Clojure: The best way to handle nil
- djtango 8y agoThis is a good article. One gotcha I ran into using fnil is that you don't get variable-arity functions for free from it in Clojure (YMMV in cljs). This caused an issue when I was using (fnil + 0 0) in a reduce function - as it would barf on an empty collection - I got around this by providing an init value to the reduce call, but it caught me off guard when (reduce + []) works
- delish 8y agoI enjoyed this article, and I like the author surveys several ways to handle `nil` before giving his preferred. I'm curious: Do any of you use the Cats library? Do you like it?
- hellofunk 8y agoI prefer to expect nothing to be nil and assert for this rather than write defensively in case it is accidentally nil. This requires a more structured approach but is a better solution for me. I don’t like nil and prefer not to think of it as a valid value at all. Too many null pointer exceptions has ruined my appreciation even for the bandaids described in this article. It’s the only pain point I have in Clojure. I just think nil is a terrible idea in any language, and the way it is overloaded in Clojure is among the most troubling among languages, in my experience.
- sz4kerto 8y agoYou have to deal with it in some way. If a function should return a boolean and something bad happens then it can return Union<boolean, failure> or Optional<boolean> or nullable boolean or throw an exception. The difference is basically in compiler assistance. Null pointers are bad because the lack of good static analysis support.
- hellofunk 8y agoBut you are talking about a statically typed way of handling nil. Options and Unions are not nil, which is the point. But you don’t get this in Clojure and nil can mean a lot of different things in different contexts, which is more complicated than a simple optional.
- degrews 8y agoSo how _do_ you handle nil without those things? You've said you try to just avoid it, but many functions return nil and there's no easy way around it (e.g. trying to get the max value of an empty list)
- qop 8y agoRepresent the problem programatically. If there's some reason data is missing, use a symbol :idk, :not-provided, etc. Programs DO things, use what you're doing to describe your program, whenever possible. In a better world, max([]) # -> None(T) But sometimes we have to get creative...
- hellofunk 8y agoFunctions only return nil when nil is a valid value in your logic.
- 3pt14159 8y agoWhat do you do when you don't know a value? Say you're making an application that tracks corporate fraud and one of the changes that you need to make is to include the date of incorporation onto the corporation model. For the existing corporations in your database what do you do? What I see people do in the wild is to use a value like 1900-01-01. Or they make a ton of tables so that the nil is now just a non-existent key instead of a non-existent value. I dislike nil as well, and I try to avoid it if possible, but I can't understand how to handle certain situations without it or without re-creating it.
- hellofunk 8y ago> What do you do when you don't know a value? I use an explicit value to represent the absence of something specific like :id-unknown which is much more useful and safer than just throwing nil in there when a project scales. If something is nil in Clojure, you have to ask why is it nil? The answer is not always simple. It could be a bug, it could be legit missing data, a number of things. A simple typo in Clojure can give you nil. You don’t want to interpret that the wrong way.
- 8ytecoder 8y agoWell, now this id-unknown would be your new null and on top of it it changes by the data type as well.
- hellofunk 8y agoI disagree. A keyword is not a null. And nil is a different “type” than something not nil anyway.
- maxilulu 8y agoIt will not crash the program and is domain specific. Definitely an improvement over nil.
- MaxBarraclough 8y ago
- dmichulke 8y agoI always use `or` because it's IMO most readable: (or x 5) ;;=> returns x if x is not nil, else 5 E.g., (-> {:a 1 :b 2} :c (or 5.5) inc)
- delish 8y ago> ;;=> returns x if x is nil, else 5 Should this be: > ;;=> returns x if x is not nil, else 5
- kimi 8y agoFranky, this looks great and should be in the article :-)
- serpix 8y agoyes! you have eliminated repeating yourself and returning nil. Other ways to do this very simple example: `(some-> m :c (inc 5))` there is no need to defend against nil values. Hickey has your back on built in functions.
- lkitching 8y agoOne problem with using or is if the key is associated with false, then it will be ignored in favour of the default. Keywords support a second argument which is the default to use on not-found so you can just write: (-> {:a 1 :b 2} (:c 5.5) inc)
- masklinn 8y ago> Clojure has a function that does basically the same thing. The get function will return nil if a key in a dictionary isn’t present: Isn't that already what happens when you use the map as a function directly? Even the sub-feature of being able to provide a default value if the key is not found is supported.
- hellofunk 8y agoBut what if the map itself is nil? That’s why using maps as functions is generally not done in the wild.
- masklinn 8y agoThat's nil punning, which the article specifically does not care for if not outright decries in its conclusion: > Over at Lispcast, Eric Normand argues for the “nil-punning” approach, which is fine. But I think this approach requires a confused notion of what nil/Nothing actually means. […] It is much simpler to understand nil as Nothing, i.e. the absence of a value (which is a type). Assuming that, the behaviour of `get` is specifically one you do not want. Furthermore, calling the map actually behaves as "Haskell and other ML-ish languages". `Data.Map.lookup` takes a Map, not a Maybe Map.
- hellofunk 8y ago‘get’ is a function that will always exist, but using a map in function position is very fragile and everything breaks if the map itself is nil. If you advocating for this, it is contrary to general consensus in the community; you will almost never see this in real code. Responding to your edit: it’s safe to do it in Haskell because you know it isn’t nil at compile time.
- masklinn 8y ago> ‘get’ is a function that will always exist, but using a map in function position is very fragile and everything breaks if the map itself is nil. It's no more fragile than any other function which doesn't do nil punning, and again the essay we're supposedly discussing advocates not using nil punning.
- pgt 8y agoNil-checking is a form of input validation. Validation features often leak into helper functions lower down which don't have the context or control to solve the source of a problem, where they don't belong. Separation of concerns means assigning management of risk to the bearer of profit (the calling code), where it can "fail/retry/failover" fast, before leaking validation code into functions that could be pure. Therefore, I prefer to hoist missing value handling up to the caller, akin to "smart vs dumb components", where the caller has sufficient context and control to recover from errors. For example, Instead of: (defonce bus (ZeroMQ. "some-url")) ;; this might be null, but it'll leak into helpers that should only care about values (defn transmit! [bus payload] (if bus ;; why??? (if payload) ;; gross (.send bus (zmq/encode-string payload)) (log/error "nil payload whoops") (log/error "something bad happens that the caller doesn't understand and can't do anything about") (defn tick! [ms] ;; athe wrong place to handle initialization problems (if-let [input (.read bus buffer-length)] ;; multiple failure modes (transmit! bus [:some (inc input)]) (comment "do nothing"))) Instead, handle initialization and input validation nearer the source and pass in system components explicitly to simplify mocking and testing: (defn transmit! [^ZeroMQ bus payload] ;; type hint {:pre [bus payload]} ;; basic guards (->> payload ;; notice lack of nil-checking. you have to enforce contracts somewhere (zmq/encode-string) (.send bus))) (defn init! [{:as config :keys [zmq-url]}] {:bus (ZeroMq. zmq-url) :redis ... :logger ...}) (defn tick! [{:as system :keys [bus]} ms] (if-let [parsed (parse-input (.read bus buffer-length))] (transmit! bus [:some (inc parsed)]) ;; parse-input always returns valid data (log/error "it's fine to handle runtime parsing errors here")) (defonce !running (atom true)) (defn -main [] (let [system (init! {:some "config-url"})] ;; attempt bus failover here (if-let [bus (:bus system)] (go-loop [] (if @!running?) (do (tick! (time/now)) (Thread/sleep 30) (recur)) (log/info "Shutting down...")) (log/error "Retrying serial bus connection... ."))))
- dustingetz 8y agoAre you suggesting that there is a meaningful difference between (:firstname record) and (.-firstName record), or (func x) and (.method x), the first of which is nil safe and the second isn't? (It doesn't seem you are, because you're focused on the ifs, but nil punning is about eliding ifs)
- j_m_b 8y ago>The best way >The Right way You presented a solution, not the solution.
- moomin 8y agoSo, I’ve programmed in Haskell and Clojure and I really wish people would give over on the Haskell requires category theory meme. What Haskell offers is _generalisation of abstraction_. So, the function that satisfies the behaviour of fnil is called fmap, but it’s also the function that maps a function over all the values of a list, or the return values of a function. Equally, the equivalent of some->> (do notation) works for all of the above and more. None of this requires you to know what a Kliesi category is.
- dmead 8y agoThis is an important point to stress. However the moment people want to get more complicated than bind and return while doing IO the "C" word inevitably creeps into scope. So while you don't need to know anything about abstract algebras to do Haskell, that knowledge is lurking so close to the surface that it is really more of an issue than is generally acknowledged.
- yogthos 8y agoWhether you need to know category theory or not, the reality is that Haskell is a much more complex language than Clojure. My team works with Clojure, and we regularly hire co-op students who typically have no exposure to FP. They're able to to start writing useful code within a couple of weeks or so on average.
- Peaker 8y agoI've mentored programmers in Haskell and they were also productive within a few weeks. I'm pretty sure I could get a student to write useful Haskell code in that time span with just mentoring and existing documentation material.
- moomin 8y agoI think the thing is, if you want to write at a Clojure-level-abstraction, Haskell won’t stop you doing that and I get the impression that it’s actually the way a lot of Haskell programmers operate. (Chas Emerick has recently been advocating this approach and I get the impression it’s actually the style of GHC itself.) We don’t all have to be at EK, BM or GGs level.
- dustingetz 8y agoNil is monoid-zero – what do you think of this idea? assoc {} (assoc nil :a 10) => {:a 10} concat '() (concat nil '(a b c)) => (a b c) str "" (str "abc" nil) => "abc" Except where host language interop is prioritized: comp identity ((comp nil inc) 42) => NPE + 0 (+ 1 nil) => NPE * 1 (* 1 nil) => NPE AFAICT, everywhere where nil could be sensibly defined as zero without complicating interop, it is. And we can of course pave over the interop examples with `fnil`.
- dmead 8y agoHi dustin.
- dmead 8y agoPaving over the difference could seem convenient but it smells like a possible source of non obvious coding errors. AFAIK it's still worthwhile to know the difference between function identity and the value that produces identities. Though, lots of this is inherited from common lisp where the list operations have more to do with lambda calc and the physical construction of the old lisp machines than they do anything else.
- hexo 8y agooh boi, best way to handle nil is to have no nil at all
- davidgrenier 8y agoInvalid Haskell code in the article: case getSomeData of Nothing -> Nothing Just a -> a Unless there's something about a that I don't know.
- dmead 8y agohow is this invalid? it's covering both constructors for maybe.
- davidgrenier 8y agoThe resulting type of the expression is inconsistent. First case returns a Maybe, second case returns whatever the type of a is. edit: replaced option with maybe.
- dmead 8y agoAh right you are. It's missing something to unwrap from the maybe type at least.
- moomin 8y agoI think it’d work if the initial type was “Maybe Maybe x”, (in which the function is join) but I seriously doubt that was what was intended. The roach motel aspect of monads is not underestimating.
- moomin 8y agoGrief, I need a proof-reader.
- maxilulu 8y ago> If you remember to write all of your Clojure code like this The whole problem with Clojure, you have to remember how to write anything instead being restricted to a subset of a safe idiom.
- platz 8y agoBingo
- Barrin92 8y agoyep, being on alert for nils should not be a human task, it shouldn't just be a convention because it is a trivial task that the type system can handle and enforce. The article is good and the author is right, but it should not be the author's problem in the first place.
- etatoby 8y agoNull / nil is the gaping hole in the safety net of static typing. I dislike and distrust any language where null / nil is a thing. Yes, that includes SQL, which otherwise would be a great language. At the same time, I've found Haskell to be too mind-bending to use for any but the most trivial of tasks. (Eg. monad pyramids and the most cryptic error messages in the history of CS!) Fortunately I've found a language that covers all bases: OCaml. It has static typing with full inference; algebraic datatypes; eager evaluation by default (yes, I believe Haskell's default of lazy to be counter-productive); no null / nil value; fast compilation to machine code and to Javascript; a compact syntax; and good editor support, including a modern gofmt-inspired auto-formatter.
- xfer 8y agomonad pyramid is also present in ocaml :) Both concurrency libraries are monadic.
- superlopuh 8y agoNot with monadic let bindings [0]. They're not the prettiest but I think have a net positive effect on the code. [0] https://github.com/janestreet/ppx_let https://github.com/janestreet/ppx_let
- daxfohl 8y agoDoes core.spec add anything here? I stopped Clojure a while back due to this problem but was considering trying again if spec helps.
- robto 8y agoYes, in my experience it does. In my projects I fdef all my functions and then turn on instrumentation (with the Orchestra[0] library, so that it also checks :ret specs). It helped me find several instances where I was incorrectly assuming that I would never get nil as an input. Couple it with Expound[1] to make reading the spec errors a bit easier, and you've got a much better story for handling nil in Clojure. [0]https://github.com/jeaye/orchestra https://github.com/jeaye/orchestra [1]https://github.com/bhb/expound https://github.com/bhb/expound
- bgongfu 8y agoThere are as many kinds of nils as programming languages. Cixl [0] doesn't derive nil from every other type, instead Nil and most other types derive Opt; which means that user code may specify Opt to accept nil or any other type to have them automatically trapped by function dispatch. I find this to be a nice compromise between wrapped optionals and shooting from the hip. [0] https://github.com/basic-gongfu/cixl#optionals https://github.com/basic-gongfu/cixl#optionals
- vemv 8y agoMy approach: - validate agressively at the system boundaries (IO- or user- facing stuff) - inside the system, clojure implementation code isn't particularly nil-aware - inside each namespace, the most important functions (typically their public part) are covered with pre/post conditions. This checks/documents the arguments/return type + (implicitly) non-nilness. - pre/post conditions are implicitly exercised with integration tests. There's clojure.spec/fdef if you prefer that to :pre/:post.
- specialist 8y agoAll this kabuki around nil, null, nullable. Did the world just completely forget about Null Object (design pattern)? With a JIT, the calls often get converted to NOPs. Makes walking a graph clean, fast, trivial.