3 ms·
Julia has language-level support for missing values (https://docs.julialang.org/en/v1/manual/missing/ https://docs.julialang.org/en/v1/manual/missing/) which ac
by ChrisRackauckas 3y ago
Julia has language-level support for missing values (https://docs.julialang.org/en/v1/manual/missing/ https://docs.julialang.org/en/v1/manual/missing/) which acts like the solution here, but generally for any value that can be put in an array by doing `Union{Missing,T}`. The array type will automatically optimize by having a bitmap of represented values (for any singleton type this works, not just Missing).
There were lots of pre-v1.0 design discussions on this vs sentinels, such as https://discourse.julialang.org/t/representing-nullable-values/4729 https://discourse.julialang.org/t/representing-nullable-valu... which can be dug up. The conclusion was more about the safety than performance, since sentinel values are fundamentally unsafe in generally since they can take a valid value and make it "special" (like in the blog post, minimum integer). Thus Julia has the following 3 choices:
* Use `nothing`, which throws an error at each operation like `nothing + 1`. This requires `if x === nothing` handling everywhere, is the safest option, but is generally considered clunky for statistics use. As `nothing` is a singleton type, it has the same optimizations as what's mentioned in the blog post.
* Use `missing`, which propagates, i.e. `missing + 1 === missing`. This allows for any `Union{Missing,T}` to use a scheme similar to the blog post: as `nothing` is a singleton type, it has the same optimizations as what's mentioned in the blog post.
* Use a sentinel of your choice, risks known and dependent on the type.
And you can build your stats libraries however you so choose, though JuliaStats has strongly gravitated towards the middle option as something close in ergonomics to R's NA but while having a bit more safety guarantees and generality.
- derbOac 3y agoInteresting. I was familiar with Missing and nothing in Julia but had never really thought about other options from a safety angle. It makes sense though. I have run into issues where it would be nice to have some sort of missing type/class hierarchy though, in that the reason for missingness is coded somewhere and then how to represent that in those three categories gets complicated. At some level the different missing types get treated the same for many functions (such as your missing + 1 propagation example) but for other things not. Then you end up being tempted to use the third option which has its own issues. I suppose you could create another variable with type of missing information but then that also becomes complex quickly.