7 ms·

Algebraic Data Types in Elixir

87 points by unripe_syntax 4y ago

throwaway932432 4y ago
> We Elixir programmers usually don’t think in terms of sum types. yikes Elixir is one of many languages that are red flagged in my job filter. I conditionally allow python where python >= 3.10, and ruby where sorbet is strictly enforced, )perhaps I'll make an exception for dialyzer) I'm bullish on golang being in my filter for years to come.. Also I feel that people that care about ADTs wouldn't be using Elixir in the first place, so the evangelism feels wasted here.
- throwawaymaths 4y ago
  People who don't care about ADTs are often caring about other things, like UX, DX, OX, readability, maintainability, etc.
- ashton314 4y ago
  That’s just like… your opinion man. Some of us who do care are also working in domains where we need really high concurrency, where Elixir frequently is the best tool, ecosystem, etc. for the job. Elixir has a great ecosystem and tooling; I’ve found it to be a joy to work in. Dialyzer is a must for me; I wrote above that it’s a love/hate relationship, but it’s something. I’d enjoy a more strongly typed language to work with for sure.
- aspett 4y ago
  > We Elixir programmers usually don’t think in terms of sum types. This person speaks.. for themselves only. > perhaps I'll make an exception for dialyzer If your opinion is to disregard Elixir, then dialyzer shouldn't change your mind. It's pretty ineffective and painful to use. That said, I'm a big proponent of Elixir; it's a great language and makes me happy to use it.
ashton314 4y ago
Nice article. I must say though, I hate dialyzer. Did I get it wired up at my company? Oh yeah. I made it part of our CI pipeline in my first few weeks. My team and I have liked the help it has given us. But where it falls down is when it’s errors are stupidly obtuse or when it doesn’t catch errors. By construction it is an unsound check: some errant programs will not be caught because Elixir (and Erlang) make it really hard to infer types. The creators wanted something that could check existing Erlang code, [1] so they had to sacrifice soundness for completeness. [2] So often I really want soundness checks, and I have spent many an hour chasing down stupid little type errors that dialyzer doesn’t tell me how to find. So that’s the hate. Overall, though, it does catch a lot of stupid mistakes, so I still appreciate its efforts. :) [1]: Lindahl and Sagonas, “Practical Type Inference Based on Success Typings.” [2]: I have a write up on the difference between some of these terms here: https://lambdaland.org/posts/2022-03-02_sound_complete_and_duals/ https://lambdaland.org/posts/2022-03-02_sound_complete_and_d...
- bmitc 4y ago
  I was recently having trouble with it on a project I'm working on along with a book. I am still a bit stumped because I couldn't make it fail. I started to get suspicious when I knew a spec was incorrect (I just hadn't updated a type yet), but yet Dialyzer was happy. I even started to worry it wasn't configured properly or there was a state problem, but deleting the `_build` directory didn't help. Still not sure. I've used it successfully in another personal project. Do you know of a good reference that discusses what types of problems Dialyzer is good for and what problems it isn't good for?
  - ashton314 4y ago
    Woof. I feel your pain. That paper by Lindahl and Sagonas is pretty decent at exploring some of the theoretical limitations of the system. (DM me on Matrix if you can’t find it: I’m @ashton314:lambdaland.org Email is also in my resume which is on my blog; see bio) Beyond that… er no, unfortunately. I’ve done many a web search trying to figure out some funky behaviour. :-P
pmontra 4y ago
I'm copy pasting my usual comment about types in Ruby: if I'd want static types I'd be using one of the many static typed languages out there. One among the many reasons I like Ruby and Elixir is that they are dynamically typed. I'm happy to see that the official type annotation system for Ruby is basically hiding them under the carpet by writing them into a separate optional file where they hopefully go to die in the dark. The @spec line in Elixir is just not DRY and inconvenient. Sometimes you fail to match it to the actual function definition because typos happen. I add an anecdote from the largest Elixir project I've been working on. One of the original developers added @specs to all the modules he developed. The other two of us did that too for many reasons. We're all senior developers but he's more senior and we'd like to see how it turned out. That developer left the project pretty soon, with some appearances now and then. New developers joined the project, mostly junior. After five years no new code has @specs AFAIK. I remember a couple of cases where static typing would have spared us a half hour of debugging. No type declaration saved us hours of typing and thinking though. I can't think about a single test that we wouldn't have to write if we had static typing because we're testing for functionality, not for input and output types. Finally, pattern matching in function definition is a great substitute for static typing because it limits the shape of the data that code can work on. There is no way not to have two Person arguments in function with a declaration of def marry(%Person{name: name1}, %Person{name: name2}) do IO.puts("#{name1} and #{name2} are married") end You can't marry a dog and a cat there, or an int and a string.
- nagasadhu 4y ago
  > if I'd want static types I'd be using one of the many static typed languages out there. Some of us want to have the concurrency and resilience of the BEAM with ADTs for data modelling and other static type advantages. Caramel, gleam and purerl are worthy efforts in this direction. A more production ready half-ass alternative is to use elixir with whatever type advantages you can get.
  - pmontra 4y ago
    In an ideal world you would be using a static typed language on the BEAM. Maybe there are some. I'm not saying that you should look for them or create one, because there are obvious advantages in using a language with a large community and creating a new community is a years long process, apart from the technical complexity of designing and implementing a language. You and I are probably paid for something else.
losvedir 4y ago
This is a good overview on ADTs and how to write them in your typespecs. Unfortunately, it doesn't dive into how exactly they interact with dialyzer, which in my experience is not perfect. Since dialyzer does "success typing", it will allow code that could work in the happy path, even if it has type issues, which is very limiting. You can make it stricter with `overspecs` and `underspecs`, which will flag if the inferred type is either stricter or looser than the given spec, and these are so close to what I want, but they don't quite work with sum types as in the article here. The issue is dialyzer considers "too loose" or "too strict" on both function inputs and outputs, but really what you want is to warn only if the function returns more than the spec allows or accepts less than the spec allows. (This is, I believe, covariance and contravariance in types[0].) In other words, you want half of each of underspecs and overspecs. Consider the example in the article of `@type direction :: east | north | west | south`. Then, it's perfectly valid for a function to return only a subset of those: @spec common_wind() :: direction def common_winds(), do: :east But with `underspecs` this gets flagged. (This would be fine in Rust, for example. If you say a function returns an enum, you don't need it to return every variant of that enum.) On the other hand, this is what you want on the input. If you specify `@spec foo(direction)`, then it better handle every variant of that enum. The opposite problem is a function that can, in theory, handle more than specified. For example: @spec func_that_can_handle_null(direction | null) :: boolean def func_that_can_handle_null(arg), do: ... @spec my_func(direction) :: boolean def my_func(arg) do func_that_can_handle_null(arg) end In this case, you want to express to callers of your function that `null` is not acceptable. However, incidentally from the implementation, it is (for now). In this case, dialyzer will infer that the function can handle `direction | null`, and see that your spec only has `direction`, and so the `underspecs` flag will now warn against it! But there's no problem here. The problem function inputs can have is the reverse: if you specify `direction | null` but then don't handle the `null`. That's the `overspecs` flag, but then that will mess you up on the first example! OTP 25 introduced `missing_returns` and `extra_returns`, which is "half" of it, only checking the return side, so I'm thrilled we're finally going in the right direction. But until we have `missing_inputs` and `extra_inputs`, I will continue to think that dialyzer doesn't really support ADTs. [0] https://docs.microsoft.com/en-us/dotnet/standard/generics/covariance-and-contravariance https://docs.microsoft.com/en-us/dotnet/standard/generics/co...
cercatrova 4y ago
See also, Witchcraft [0] (adds ADTs to Elixir in a subjectively better way than Dialyzer) and Gleam [1] (not Elixir but its own full blown language that uses BEAM but with much more of a Haskell-like flavor) which both offer algebraic data types in their own ways. I used to do a lot of Python and heard of Elixir as a marrying of BEAM and dynamic typing, so I started learning it. However, as I used it more, I actually moved to TypeScript and more recently Rust after I found that I actually liked thinking in types, and I truly did miss them from Elixir. [0] https://github.com/witchcrafters/witchcraft https://github.com/witchcrafters/witchcraft [1] https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam

87 points by unripe_syntax 4y ago

> We Elixir programmers usually don’t think in terms of sum types. yikes Elixir is one of many languages that are red flagged in my job filter. I conditionally allow python where python >= 3.10, and ruby where sorbet is strictly enforced, )perhaps I'll make an exception for dialyzer) I'm bullish on golang being in my filter for years to come.. Also I feel that people that care about ADTs wouldn't be using Elixir in the first place, so the evangelism feels wasted here.

throwawaymaths 4y ago
People who don't care about ADTs are often caring about other things, like UX, DX, OX, readability, maintainability, etc.
ashton314 4y ago
That’s just like… your opinion man. Some of us who do care are also working in domains where we need really high concurrency, where Elixir frequently is the best tool, ecosystem, etc. for the job. Elixir has a great ecosystem and tooling; I’ve found it to be a joy to work in. Dialyzer is a must for me; I wrote above that it’s a love/hate relationship, but it’s something. I’d enjoy a more strongly typed language to work with for sure.
aspett 4y ago
> We Elixir programmers usually don’t think in terms of sum types. This person speaks.. for themselves only. > perhaps I'll make an exception for dialyzer If your opinion is to disregard Elixir, then dialyzer shouldn't change your mind. It's pretty ineffective and painful to use. That said, I'm a big proponent of Elixir; it's a great language and makes me happy to use it.

ashton314 4y ago

Nice article. I must say though, I hate dialyzer. Did I get it wired up at my company? Oh yeah. I made it part of our CI pipeline in my first few weeks. My team and I have liked the help it has given us. But where it falls down is when it’s errors are stupidly obtuse or when it doesn’t catch errors. By construction it is an unsound check: some errant programs will not be caught because Elixir (and Erlang) make it really hard to infer types. The creators wanted something that could check existing Erlang code, [1] so they had to sacrifice soundness for completeness. [2] So often I really want soundness checks, and I have spent many an hour chasing down stupid little type errors that dialyzer doesn’t tell me how to find. So that’s the hate. Overall, though, it does catch a lot of stupid mistakes, so I still appreciate its efforts. :) [1]: Lindahl and Sagonas, “Practical Type Inference Based on Success Typings.” [2]: I have a write up on the difference between some of these terms here: https://lambdaland.org/posts/2022-03-02_sound_complete_and_duals/ https://lambdaland.org/posts/2022-03-02_sound_complete_and_d...

bmitc 4y ago
I was recently having trouble with it on a project I'm working on along with a book. I am still a bit stumped because I couldn't make it fail. I started to get suspicious when I knew a spec was incorrect (I just hadn't updated a type yet), but yet Dialyzer was happy. I even started to worry it wasn't configured properly or there was a state problem, but deleting the `_build` directory didn't help. Still not sure. I've used it successfully in another personal project. Do you know of a good reference that discusses what types of problems Dialyzer is good for and what problems it isn't good for?
- ashton314 4y ago
  Woof. I feel your pain. That paper by Lindahl and Sagonas is pretty decent at exploring some of the theoretical limitations of the system. (DM me on Matrix if you can’t find it: I’m @ashton314:lambdaland.org Email is also in my resume which is on my blog; see bio) Beyond that… er no, unfortunately. I’ve done many a web search trying to figure out some funky behaviour. :-P

pmontra 4y ago

I'm copy pasting my usual comment about types in Ruby: if I'd want static types I'd be using one of the many static typed languages out there. One among the many reasons I like Ruby and Elixir is that they are dynamically typed. I'm happy to see that the official type annotation system for Ruby is basically hiding them under the carpet by writing them into a separate optional file where they hopefully go to die in the dark. The @spec line in Elixir is just not DRY and inconvenient. Sometimes you fail to match it to the actual function definition because typos happen. I add an anecdote from the largest Elixir project I've been working on. One of the original developers added @specs to all the modules he developed. The other two of us did that too for many reasons. We're all senior developers but he's more senior and we'd like to see how it turned out. That developer left the project pretty soon, with some appearances now and then. New developers joined the project, mostly junior. After five years no new code has @specs AFAIK. I remember a couple of cases where static typing would have spared us a half hour of debugging. No type declaration saved us hours of typing and thinking though. I can't think about a single test that we wouldn't have to write if we had static typing because we're testing for functionality, not for input and output types. Finally, pattern matching in function definition is a great substitute for static typing because it limits the shape of the data that code can work on. There is no way not to have two Person arguments in function with a declaration of def marry(%Person{name: name1}, %Person{name: name2}) do IO.puts("#{name1} and #{name2} are married") end You can't marry a dog and a cat there, or an int and a string.

nagasadhu 4y ago
> if I'd want static types I'd be using one of the many static typed languages out there. Some of us want to have the concurrency and resilience of the BEAM with ADTs for data modelling and other static type advantages. Caramel, gleam and purerl are worthy efforts in this direction. A more production ready half-ass alternative is to use elixir with whatever type advantages you can get.
- pmontra 4y ago
  In an ideal world you would be using a static typed language on the BEAM. Maybe there are some. I'm not saying that you should look for them or create one, because there are obvious advantages in using a language with a large community and creating a new community is a years long process, apart from the technical complexity of designing and implementing a language. You and I are probably paid for something else.

losvedir 4y ago

This is a good overview on ADTs and how to write them in your typespecs. Unfortunately, it doesn't dive into how exactly they interact with dialyzer, which in my experience is not perfect. Since dialyzer does "success typing", it will allow code that could work in the happy path, even if it has type issues, which is very limiting. You can make it stricter with `overspecs` and `underspecs`, which will flag if the inferred type is either stricter or looser than the given spec, and these are so close to what I want, but they don't quite work with sum types as in the article here. The issue is dialyzer considers "too loose" or "too strict" on both function inputs and outputs, but really what you want is to warn only if the function returns more than the spec allows or accepts less than the spec allows. (This is, I believe, covariance and contravariance in types[0].) In other words, you want half of each of underspecs and overspecs. Consider the example in the article of `@type direction :: east | north | west | south`. Then, it's perfectly valid for a function to return only a subset of those: @spec common_wind() :: direction def common_winds(), do: :east But with `underspecs` this gets flagged. (This would be fine in Rust, for example. If you say a function returns an enum, you don't need it to return every variant of that enum.) On the other hand, this is what you want on the input. If you specify `@spec foo(direction)`, then it better handle every variant of that enum. The opposite problem is a function that can, in theory, handle more than specified. For example: @spec func_that_can_handle_null(direction | null) :: boolean def func_that_can_handle_null(arg), do: ... @spec my_func(direction) :: boolean def my_func(arg) do func_that_can_handle_null(arg) end In this case, you want to express to callers of your function that `null` is not acceptable. However, incidentally from the implementation, it is (for now). In this case, dialyzer will infer that the function can handle `direction | null`, and see that your spec only has `direction`, and so the `underspecs` flag will now warn against it! But there's no problem here. The problem function inputs can have is the reverse: if you specify `direction | null` but then don't handle the `null`. That's the `overspecs` flag, but then that will mess you up on the first example! OTP 25 introduced `missing_returns` and `extra_returns`, which is "half" of it, only checking the return side, so I'm thrilled we're finally going in the right direction. But until we have `missing_inputs` and `extra_inputs`, I will continue to think that dialyzer doesn't really support ADTs. [0] https://docs.microsoft.com/en-us/dotnet/standard/generics/covariance-and-contravariance https://docs.microsoft.com/en-us/dotnet/standard/generics/co...

cercatrova 4y ago

See also, Witchcraft [0] (adds ADTs to Elixir in a subjectively better way than Dialyzer) and Gleam [1] (not Elixir but its own full blown language that uses BEAM but with much more of a Haskell-like flavor) which both offer algebraic data types in their own ways. I used to do a lot of Python and heard of Elixir as a marrying of BEAM and dynamic typing, so I started learning it. However, as I used it more, I actually moved to TypeScript and more recently Rust after I found that I actually liked thinking in types, and I truly did miss them from Elixir. [0] https://github.com/witchcrafters/witchcraft https://github.com/witchcrafters/witchcraft [1] https://github.com/gleam-lang/gleam https://github.com/gleam-lang/gleam