5 ms·

Is there anyone with experience on a large Elixir/Phoenix code base? Is the lack of static type system a problem?

by koevet 4y ago

Liberonostrud 4y ago
This is another problem of Phoenix. There are some big projects running on it, like cars.com but the code is not public and also they have a team of people who are good programmers and can bug-fix or pay people who can help bug-fix things for them. Watch the presentation of their switch to Phoenix and the gotchas they have encountered - for example how at first they didn't take into account the impact on Phoenix when users of sites like cars.com have opened 10 or 20 tabs (that have to be updated via websockets) instead of just 1 or 2 when they are comparing their dream car and how they thought that switching to Phoenix was a bad idea. Of course, they are smart ()or have money to pay smart programmers) and they solved this problem and now are happy with Phoenix ;). But as I said in previous posts, unlike with PHP that is very forgiving for sloppy code, Elixir does need higher skill level and the barrier to master it much, much higher than PHP.
- krstf13 4y ago
  You’re misrepresenting what’s being said in this talk (the relevant part of the talk is around the 15 minute mark at : https://youtu.be/XzAupUHiryg https://youtu.be/XzAupUHiryg ). The tldr was that they had patched some of the library code without considering the ramifications of the change. At no point in the talk is it said that “switching to Phoenix was a bad idea”. Also, none of the issues they talk about are related to the lack of static typing.
  - Liberonostrud 4y ago
    https://github.com/laravel/laravel/pull/6010 https://github.com/laravel/laravel/pull/6010
    - krstf13 4y ago
      So? Laravel uses type hinting, ok. This has absolutely nothing to do with your previous post grossly misrepresenting a talk. You hadn’t even mentioned Laravel in it. Type hinting is not equivalent to a static type system, which is what the parent was asking about. Finally, in either cases, it changes nothing to the fact that the pain points mentioned in the talk were not caused by the lack of a static type system (which is not to say it cannot cause pain points)
- ethangunderson 4y ago
  Hi, software developer at Cars.com here. While I appreciate the ego boost of thinking that we're somehow better than average, I don't believe this to be true. You can get away with sloppy Elixir code just as easily as you can in PHP, Ruby, Python, etc. > they thought that switching to Phoenix was a bad idea If you have the time I recommend re-watching Zack's talk. This is not a take away, or implied.
- ch4s3 4y ago
  I work on an application that's probably about as large as Cars.com and we've never run into an issue that was caused by phoenix or elixir specifically. We've had to write a few things in house that may be available off the shelf in other languages, but it's never been especially difficult or time consuming. Most of us are average devs, and it the tools just work. We love how easily we can run things in parallel and we have a lot of soft real time problems that the language makes trivial. We also onboard a lot of people with no elixir experience, and most of them are committing code within a day or two. The docs are great and the surface area is pretty small. On boarding just isn't an issue.
sanswork 4y ago
Check on the elixirconf videos on YouTube.
conradfr 4y ago
In my experience not really because you can type hint structs, which combined with pattern matching, guards and typespecs will cover most cases. ex: def foo(%Bar{} = bar, how_many) when is_number(how_many) Also avoiding/limiting variable rebinding should mitigate problems as the language is immutable otherwise. So IMHO it affects the developer productivity more than create type bugs, because you have to write all that and intellisense kind of sucks (at least in Jetbrains' IDE with the unofficial Elixir plugin).
cybrox 4y ago
There are optional type annotations and a tool to statically check them (dialyzer) The learning curve is a bit steep as dialyzer works on erlang terms internally but we've been using it for a rather large codebase for 4+ years now and it does a consistently good job. The Elixir core team actually mentioned that they want to make improved type support a priority in the last keynotes.
dns_snek 4y ago
As someone who swore by Typescript over Javascript, it's an annoyance. Dialyzer type specs create more pain than they're worth imho and you rarely get useful autocomplete in return since Elixir IDEs (VS Code/Jetbrains + FOSS plugins) aren't as advanced as they are with Typescript. As another comment has mentioned, what works for me is defining structs for more complex parameters and using them like so: def some_function(param1, param2, bag_of_options = %BagOfOptions{}) do With this you'll get autocomplete when you type `bag_of_options.`, a compilation warning* [1] if you try to access properties of `bag_of_options` that don't exist, and it's going to raise an error if you run the code and try to call the function with anything other than a %BagOfOptions{} struct. TypedStruct library is particularly useful when defining structs, allowing you to easily define and document fields, assign them Dyalizer types and set their default values. This is good enough for me not to significantly hinder my productivity, but YMMV. [1] Compilation warnings can be treated as errors using --warnings-as-errors, I use it for all production builds
lpil 4y ago
I find that a large Elixir codebase is easier to work with than a larger Ruby or JavaScript codebase, but it still has the same difficulties to some degree. If you're a typing fan as I am, and you are interested in the BEAM then you may want to check out [Gleam, a sibling language of Elixir](https://github.com/gleam-lang/gleam/ https://github.com/gleam-lang/gleam/) which has an Elm/OCaml/F# style static type system. It provides many of the strengths of Elixir but with that statically typed programming style which you may prefer. (disclosure: I'm the lead dev!)
benmmurphy 4y ago
I think the biggest problems are with untested code. If you have tests for all your code paths then its less of an issue. If you have code paths that are not tested then it's quite easy to have typing issues in those code paths. In my experience this is a big problem with error handling and also with `with` statements since when you are using a `with` statement there is an implicit branch for every statement in the `with` block. People are probably not going to be writing a test for every single statement short-circuiting a `with` block and its likely to be the `error` path so it's quite easy for something with the wrong type to escape the `with` block. I also think there is this trap of using unnamed tuples and unnamed maps in the code instead of Structs. I suspect this is fine for a small team and a small code base but if you have a large codebase then it makes it harder to familiarise yourself with the code quickly. Having functions where you can't easily see the types of the inputs and the outputs is not ideal, but once you work out what they are its worse when they don't have names. Of course there are `specs` for this but if you are writing `specs` then what is the point of a dynamic language. I don't really see the advantage of writing specs and not having the compiler do any checking vs writing type definitions and having the compiler check everything is sound.
ak_111 4y ago
What is a popular static type system/framework/language for spa backend development that is competitive with the likes of Elixir/Phoenix, Larval/PHP, Python/Django?
brightball 4y ago
I think a lot of people look to lean on static types coming from other languages, but you're learning a little bit of a different way of writing code when you come to Elixir. It's strongly typed, but not statically typed so you get a lot of the benefits of static typing with a lot of the flexibility that comes from dynamic typing. IMO it's a ideal balance with pattern matching, it's a very smooth experience all around. It's best to come into the language with the understanding that this language is different. Copy on write, tail call optimized recursion and pattern matching are wonderful, but they take some getting used to.

Is there anyone with experience on a large Elixir/Phoenix code base? Is the lack of static type system a problem?

by koevet 4y ago

Is there anyone with experience on a large Elixir/Phoenix code base? Is the lack of static type system a problem?

This is another problem of Phoenix. There are some big projects running on it, like cars.com but the code is not public and also they have a team of people who are good programmers and can bug-fix or pay people who can help bug-fix things for them. Watch the presentation of their switch to Phoenix and the gotchas they have encountered - for example how at first they didn't take into account the impact on Phoenix when users of sites like cars.com have opened 10 or 20 tabs (that have to be updated via websockets) instead of just 1 or 2 when they are comparing their dream car and how they thought that switching to Phoenix was a bad idea. Of course, they are smart ()or have money to pay smart programmers) and they solved this problem and now are happy with Phoenix ;). But as I said in previous posts, unlike with PHP that is very forgiving for sloppy code, Elixir does need higher skill level and the barrier to master it much, much higher than PHP.

krstf13 4y ago
You’re misrepresenting what’s being said in this talk (the relevant part of the talk is around the 15 minute mark at : https://youtu.be/XzAupUHiryg https://youtu.be/XzAupUHiryg ). The tldr was that they had patched some of the library code without considering the ramifications of the change. At no point in the talk is it said that “switching to Phoenix was a bad idea”. Also, none of the issues they talk about are related to the lack of static typing.
- Liberonostrud 4y ago
  https://github.com/laravel/laravel/pull/6010 https://github.com/laravel/laravel/pull/6010
  - krstf13 4y ago
    So? Laravel uses type hinting, ok. This has absolutely nothing to do with your previous post grossly misrepresenting a talk. You hadn’t even mentioned Laravel in it. Type hinting is not equivalent to a static type system, which is what the parent was asking about. Finally, in either cases, it changes nothing to the fact that the pain points mentioned in the talk were not caused by the lack of a static type system (which is not to say it cannot cause pain points)
ethangunderson 4y ago
Hi, software developer at Cars.com here. While I appreciate the ego boost of thinking that we're somehow better than average, I don't believe this to be true. You can get away with sloppy Elixir code just as easily as you can in PHP, Ruby, Python, etc. > they thought that switching to Phoenix was a bad idea If you have the time I recommend re-watching Zack's talk. This is not a take away, or implied.
ch4s3 4y ago
I work on an application that's probably about as large as Cars.com and we've never run into an issue that was caused by phoenix or elixir specifically. We've had to write a few things in house that may be available off the shelf in other languages, but it's never been especially difficult or time consuming. Most of us are average devs, and it the tools just work. We love how easily we can run things in parallel and we have a lot of soft real time problems that the language makes trivial. We also onboard a lot of people with no elixir experience, and most of them are committing code within a day or two. The docs are great and the surface area is pretty small. On boarding just isn't an issue.

sanswork 4y ago

Check on the elixirconf videos on YouTube.

conradfr 4y ago

In my experience not really because you can type hint structs, which combined with pattern matching, guards and typespecs will cover most cases. ex: def foo(%Bar{} = bar, how_many) when is_number(how_many) Also avoiding/limiting variable rebinding should mitigate problems as the language is immutable otherwise. So IMHO it affects the developer productivity more than create type bugs, because you have to write all that and intellisense kind of sucks (at least in Jetbrains' IDE with the unofficial Elixir plugin).

cybrox 4y ago

There are optional type annotations and a tool to statically check them (dialyzer) The learning curve is a bit steep as dialyzer works on erlang terms internally but we've been using it for a rather large codebase for 4+ years now and it does a consistently good job. The Elixir core team actually mentioned that they want to make improved type support a priority in the last keynotes.

dns_snek 4y ago

As someone who swore by Typescript over Javascript, it's an annoyance. Dialyzer type specs create more pain than they're worth imho and you rarely get useful autocomplete in return since Elixir IDEs (VS Code/Jetbrains + FOSS plugins) aren't as advanced as they are with Typescript. As another comment has mentioned, what works for me is defining structs for more complex parameters and using them like so: def some_function(param1, param2, bag_of_options = %BagOfOptions{}) do With this you'll get autocomplete when you type `bag_of_options.`, a compilation warning* [1] if you try to access properties of `bag_of_options` that don't exist, and it's going to raise an error if you run the code and try to call the function with anything other than a %BagOfOptions{} struct. TypedStruct library is particularly useful when defining structs, allowing you to easily define and document fields, assign them Dyalizer types and set their default values. This is good enough for me not to significantly hinder my productivity, but YMMV. [1] Compilation warnings can be treated as errors using --warnings-as-errors, I use it for all production builds

lpil 4y ago

I find that a large Elixir codebase is easier to work with than a larger Ruby or JavaScript codebase, but it still has the same difficulties to some degree. If you're a typing fan as I am, and you are interested in the BEAM then you may want to check out [Gleam, a sibling language of Elixir](https://github.com/gleam-lang/gleam/ https://github.com/gleam-lang/gleam/) which has an Elm/OCaml/F# style static type system. It provides many of the strengths of Elixir but with that statically typed programming style which you may prefer. (disclosure: I'm the lead dev!)

benmmurphy 4y ago

I think the biggest problems are with untested code. If you have tests for all your code paths then its less of an issue. If you have code paths that are not tested then it's quite easy to have typing issues in those code paths. In my experience this is a big problem with error handling and also with `with` statements since when you are using a `with` statement there is an implicit branch for every statement in the `with` block. People are probably not going to be writing a test for every single statement short-circuiting a `with` block and its likely to be the `error` path so it's quite easy for something with the wrong type to escape the `with` block. I also think there is this trap of using unnamed tuples and unnamed maps in the code instead of Structs. I suspect this is fine for a small team and a small code base but if you have a large codebase then it makes it harder to familiarise yourself with the code quickly. Having functions where you can't easily see the types of the inputs and the outputs is not ideal, but once you work out what they are its worse when they don't have names. Of course there are `specs` for this but if you are writing `specs` then what is the point of a dynamic language. I don't really see the advantage of writing specs and not having the compiler do any checking vs writing type definitions and having the compiler check everything is sound.

ak_111 4y ago

What is a popular static type system/framework/language for spa backend development that is competitive with the likes of Elixir/Phoenix, Larval/PHP, Python/Django?

brightball 4y ago

I think a lot of people look to lean on static types coming from other languages, but you're learning a little bit of a different way of writing code when you come to Elixir. It's strongly typed, but not statically typed so you get a lot of the benefits of static typing with a lot of the flexibility that comes from dynamic typing. IMO it's a ideal balance with pattern matching, it's a very smooth experience all around. It's best to come into the language with the understanding that this language is different. Copy on write, tail call optimized recursion and pattern matching are wonderful, but they take some getting used to.