9 ms·

Beyond 10,000 Lines: Lessons Learned from a Large Phoenix Project

239 points by mikecarlton 10y ago

jblow 10y ago
10,000 lines is supposed to be 'big'? wtf? I write something like 25kLoC/year (of shipping code, generally very complex stuff) and I don't even program full-time. The two projects I am working on now are 35kloc (the smaller one) and 250kloc (the medium-sized one). If someone thinks 10kloc is big, I have a hard time thinking of that person as a professional programmer. (Numbers listed here exclude blank lines and comments.)
- partycoder 10y ago
  Run a code duplication detector in your code. I am curious to see if you actually need 250,000 SLOC, or if it's actually just copying and pasting around.
- tonyarkles 10y ago
  One interesting thing I've found with my limited experience with Elixir is that the code is very compact. What would often take 10+ LoC in other languages can be succinctly expressed as 1-2 lines. But I do in general agree with your take on it :)
  - jblow 10y ago
    > would often take 10+ LoC in other languages can be succinctly expressed as 1-2 lines. Every time I have heard this kind of claim (with modern languages), it turned out not to be true except for trivial code or straw-man bad code in the 'bigger' language. So if you have real-world examples that have real-world effort put in, I'd like to see them! (I would be happy to be wrong.) 5x-10x productivity increase would be huge if it actually existed; it would be so unstoppable that everyone would switch to the new really-great language immediately. That hasn't happened, which should be a clue that maybe the increase is not there. Even a 20% decrease in cost of engineering would be so large as to be unignorable.
    - kornish 10y ago
      > 5x-10x productivity increase would be huge if it actually existed When writing software, how fast you can type the code is rarely the limiting factor for speed of development – the architecting and consideration of interplay between components takes the bulk of the time. The grandparent claimed code reduction (which has intrinsic maintainability benefits) but made no statements about general cost of engineering.
      - jblow 10y ago
        > the architecting and consideration of interplay between components takes the bulk of the time. Which is supposed to be what is simplified as LOC goes down. So if a supposed 5x-10x code reduction (which I've never seen real evidence of) doesn't lead to 5x-10x productivity increase, how much increase is there supposed to be? Surely more than zero?
        kornish 10y ago
        > Which is supposed to be what is simplified as LOC goes down. I don't think so. If you can express the same concepts with the same interfaces and functionality in 1kloc vs 10kloc, most of your time has probably still gone into figuring out the interfaces and connections. > So if a supposed 5x-10x code reduction (which I've never seen real evidence of) doesn't lead to 5x-10x productivity increase, how much increase is there supposed to be? Surely more than zero? Oh, certainly more than zero! Sometimes much more. But there's simply not a one-size-fits-all formula for the relationship between lines of code written and productivity. Anyways, not really sure what you're getting at. Your original comment was that 10kloc isn't "big"; the rebuttal is that lines of code is a naive way of looking at system complexity, which is presumably what you mean by "big".
        Scarbutt 10y ago
        You find having to read 1kloc vs 10kloc the same?
        10y ago
  - fleetfox 10y ago
    Even if you multiply it by 3 it's a small codebase. And Phoenix does bunch of code generation from what I've seen.
- sotojuan 10y ago
  This is why I'm wary of posts about Elixir. I love the language a lot but the blog posts and articles on it are mostly hype by people without much experience.
- rozap 10y ago
  two things: 1) elixir is relatively young so there aren't many projects that have had a chance to grow to be huge. so yea, take everything you read about it with that grain of salt. people haven't grown to hate it yet...maybe everyone will always love it, but I wouldn't bet on it. 2) when used in certain contexts, elixir applications may end up being quite a bit smaller than comparable java/scala/python/whatever apps. A lot of what we end up writing with microservices is just basic RPC and fault tolerance boilerplate stuff, which OTP takes care of out of the box. Example, pinterest re-writing a 10kloc java app in 1kloc of elixir, running it on fewer servers: https://engineering.pinterest.com/blog/introducing-new-open-source-tools-elixir-community https://engineering.pinterest.com/blog/introducing-new-open-...
- dudul 10y ago
  10,000 lines is actually a lot for Elixir. Its syntax and built-in standard library makes the code very compact.
- enraged_camel 10y ago
  Pro-tip: never look at number of LOCs to measure the size or complexity of a project, unless you are comparing two systems written in the same language. Even then, comparisons might be tricky due to factors such as libraries used.
- aikah 10y ago
  I don't really think one can judge the work of someone else by the line of codes he writes a year. I thought IBM management failures demonstrated that kind of metric is meaningless. Furthermore, 10,000 lines of elixir/Ruby is not like 10,000 of say Java that don't do much except wiring dependencies.
  - Roboprog 10y ago
    INDEED. 10K lines RoR = 100K lines JEE. More or less. (OK, I made those numbers up, but reading Java code is bloody tiring due to the silly walks and typical paste-o-graphic work styles)
- strmpnk 10y ago
  Some of the gain in concision is from the language it seems but I think this case is very much from the view of leveraging frameworks and libraries which themselves are quite a bit bigger than the app. I cloned a few of the repositories and excluded tooling that was used for documentation, testing, and so on. It's non-exhaustive since I only wanted to get a ballpark figure but we have: 1.4 million lines of Erlang 309 thousand lines of C 120 thousand lines of Elixir ... And lots of other code and things I've excluded. These are lines of code, not counting blank or comment-lines. Now we could argue that we could unwrap this count at any layer? Why stop at Erlang/OTP? Why not include the C runtime and the Kernel? The real effect here is that a lot of this code implement things that are directly used by Phoenix apps, so I think it's fair to say, it's concision comes from being very good at leveraging other code. (I don't write Elixir code but I've seen some good results from teams who have. None of it is magic though.)
- xutopia 10y ago
  What does it matter how many you write? How many do you delete? I have a project that is 5k lines of code and roughly 11k of tests and specs(cucumber). It is a rewrite of a project that was 50k lines of code with 1.2k lines of tests and had less functionality and features than what it does now. Lines of code are meaningless when it comes to how much value they provide. I personally prefer when a codebase is smaller because it means some thought was put into it and most likely has less bugs as a result.
  - jblow 10y ago
    I said shipping code, by which I specifically mean that which is left after all deletions (of which there are many).
  - cgag 10y ago
    I believe jblows games (braid, the witness) have less lines of code than one would typically expect. I think you should assume hes not writing code like a government contractor.
    - mars4rp 10y ago
      upvote for government contractor, I once saw 5k lines of code for simple change of address in the front end alone !!!
- jamon51 10y ago
  It's relatively large for a Phoenix web app, given that there aren't that many out there yet.
- velox_io 10y ago
  I'd say 10k LoC is where design and architecture start to become vital, you can't just hack away without breaking stuff. I personally think copy & paste is an anti-pattern (instead, use features such as generics), but many think it is a godsend. Using code reuse rather than copying code can make a massive difference in size and maintainability, less is definitely more!
  - Roboprog 10y ago
    Dealing with that at work now. Dev added to project won't use the existing structure, just recreates and pastes stuff in over and over :-( Part of that is a political problem, of course, but was not an issue earlier.
- pmontra 10y ago
  I spent the best part of yesterday's afternoon writing a complex SQL query. I could have written a few straightforward queries instead and still be here writing, debugging and testing the code to merge their results. I have this single LOC now instead of 100+. That line is worth 100.
- StreamBright 10y ago
  I regularly delete or replace several thousands of lines of code without changing the functionality of application, put in by programmers like you, who are obsessed with the amount of LOC they produce. I am not sure why anybody thinks that a lengthy solution is great or we should judge programmers by the LOC they produce. I strongly believe that the better the programmer is the shorter the solution will be produced by her/him.
  - Scea91 10y ago
    It's like judging the quality of a book by number of pages.
pmontra 10y ago
Typo in the title : it's Phoenix, not Phoeniz.
- i336_ 10y ago
  Not sure if the words "admin", "admins" or "mods" trigger anything? Mentioning just in case.
  - rattray 10y ago
    try also dang
    - sctb 10y ago
      No, those don't have any magic, but if something like this persists you can always email us at hn@ycombinator.com.
      - i336_ 10y ago
        Ah, okay then. I was curious, thanks for the info. Idea: make users with high karma capable of using benign, unassuming and non-common parts of speech to ping the admins? ...I think I've been thinking about security too much
        reitanqild 10y ago
        Just click the post timestamp and click flag first.
        i336_ 10y ago
        ...I am so dense. That makes perfect sense, thanks.
        reitanqild 10y ago
        > I am so dense. Maybe just busy with real work?
rdtsc 10y ago
Also I found Dialyzer (success typing checker) to be helpful in large projects: http://learnyousomeerlang.com/dialyzer http://learnyousomeerlang.com/dialyzer The way success typing works it is a bit like static typing but when it cannot deduce the types it assumes success. However when it finds a discrepancy it is always right. So the more type annotations you add, and the more precise they are the more benefits you get from it. It has also been there for many years. I think Python only very recently has started getting the same kind of things via MyPy. Of course, they had to call it something differently (Optional Static Typing). Elixir has a wrapper around it, it seems: https://github.com/jeremyjh/dialyxir https://github.com/jeremyjh/dialyxir but never used it (I user Erlang mostly).
- crbelaus 10y ago
  Totaly agree. Dialyzer/Dialyxir can provide a high degree of confidence about types. The wonderful thing about it is that you don't have to get all type annotations since the begining, you can add them over the time. This allows to use "unspecified" types for quick prototyping, and make them more specific when the project evolves, which is a major complain of traditional type systems such as Java's. Another tool that I find useful is Credo[1] which "is a static code analysis tool for the Elixir language with a focus on teaching and code consistency". The Ecto[2] project uses a tool called Ebert[3] that automatically runs Credo for each pull-request and comments with the issues found. Here you can see an example of Ebert's bot commenting on a PR[4] [1] https://github.com/rrrene/credo https://github.com/rrrene/credo [2] https://github.com/elixir-ecto/ecto https://github.com/elixir-ecto/ecto [3] https://ebertapp.io/ https://ebertapp.io/ [4] https://github.com/elixir-ecto/ecto/pull/1785 https://github.com/elixir-ecto/ecto/pull/1785
saalaa 10y ago
> Lessons Learned from a Large Phoenix Project More like "large" projects.
yellowapple 10y ago
I feel like the bit about OTP misses the point of OTP. Erlang is not just about distributed computing (in fact, it never was; any affinity for distributed computing was more of a side-effect of Erlang's design). Rather, it's about fault-tolerance. Supervision trees and "let it crash" are the cornerstone of Erlang programming, and therefore by extension the cornerstone of Elixir programming. Meanwhile, OTP applications build on this in a way that permits composability. It's kind of like microservices behind the scenes, bit they feel like a monolith; you build up your system from lots of different OTP applications that work together to provide a unified whole. Elixir and Erlang web frameworks (Phoenix, Sugar, Chicago Boss (IIRC), etc.) already do a lot of this for you by kicking off various OTP dependencies; for example, your average Phoenix or Sugar application will in turn start Plug, Ecto, and various other OTP apps, and these will in turn spin up their own dependencies (like Cowboy and Postgrex, respectively). Basically, it's not quite right to equate OTP to just distributed computing. OTP is at the heart and soul of the vast majority of software written for BEAM.
misterbowfinger 10y ago
> 5. Avoid Non-RESTful Routes Every "large" project gets to a point where some routes aren't totally RESTful. It happens. But it's not great advice to say that non-RESTful routes are always a code smell.
- arms 10y ago
  Agreed. I'm working on an API where most of the endpoints are RESTful, but there's a handful where it makes total sense to break the rules in order to make everyone's lives easier.
- aikah 10y ago
  What a total RESTful route to begin with ? I thought REST had nothing to do with "routes", url or resources but how the state of the application is communicated in an API ? It seems to me the Roy wrote something with a certain perspective, someone came up, read the paper and decided to apply his writings to something totally unrelated because he didn't like SOAP ... I personally stopped using that word so I don't have to confront and debate with "REST purists" and just went back to talking about web service, because that's what it is.
  - Arcaire 10y ago
    The idea is to limit exposed routes to the standard CRUD routes: create (/new), read (/show), update (/edit), delete (/destroy).
    - aikah 10y ago
      I thought the idea was to make an API behave like a webpage by using hyperlinks that represents the state of the application, so in theory a "smart" client could navigate an API autonomously like a human interacts with a web page ? Because otherwise it's no different from RPC.
      - anthonybullard 10y ago
        I think you are thinking of HATEOAS. Read about it here: https://en.wikipedia.org/wiki/HATEOAS?wprov=sfla1 https://en.wikipedia.org/wiki/HATEOAS?wprov=sfla1
thijsvandien 10y ago
Could anyone more knowledgable explain whether or not my feeling is correct that Elixir becoming (relatively) popular is making Erlang a less viable choice? Whereas the total number of libraries may be increasing, code that might have been written in Erlang is now written in Elixir and established Erlang codebases (RabbitMQ for one) have started to migrate (parts) to Elixir. Calling Erlang from Elixir is easy, but what about the reverse? It reminds of me of how, at least in the early days, Play Framework (largely Scala) could be used from Java but not without much friction. Should I prefer Erlang as a language for its maturity and fewer constructs, it would be a pity to then miss out on new libraries or frameworks, such as Phoenix, that revolve around Elixir.
- dsri 10y ago
  Yes, you can call Elixir from Erlang: http://elixir-lang.org/crash-course.html#adding-elixir-to-existing-erlang-programs http://elixir-lang.org/crash-course.html#adding-elixir-to-ex...
  - thijsvandien 10y ago
    I did expect it to be theoretically possible, but how realistic/sensible is it really to do an Erlang project with Elixir libraries or frameworks? Elixir brings a lot of its own tools, conventions and so on. Will it soon take over my project, making it a better option to just use Elixir for everything, or would it be very manageable and not interfere much with the rest?
    - qaq 10y ago
      If anyone from Erlang Solutions is around they prob would be most qualified to answer.
      - claudioesl 10y ago
        Claudio from Erlang Solutions here. I'll ask around in the company for more information. I've worked briefly on wrapping an existing erlang app in a Elixir (Nerves) container and I did have a few issues around dependency management which required small changes here and there, but nothing major. One area where I usually spend more time than I'd like is to get type specifications in good shape so that dialyzer doesn't report too many warnings, but that gets better with every Erlang/Elixir release.
crasm 10y ago
I was expecting the IT and devops book, The Phoenix Project: http://itrevolution.com/books/phoenix-project-devops-book/ http://itrevolution.com/books/phoenix-project-devops-book/
akavel 10y ago
In section titled "3. Write Fewer, Valuable Tests", I don't understand the terminology ("controllers and plugs"), which I assume is some stuff in the Phoenix framework that I have no idea about. Can somebody possibly explain to me what are the "controllers and plugs" here? I'm very interested in learning a more general idea (how to effectively reduce lines of tests) from this nugget, that I could hopefully translate to different languages & frameworks I use... Quoting the relevant fragment in full: "We focused our automated tests on our controller actions and plugs rather than going for 100% test coverage. Since these are the main ways that the Phoenix application interfaces with the outside world, they’re the critical points of failure. Controller tests also exercise a lot of the code paths in your application, making it less necessary to unit test every single module. As a result, you end up with fewer tests, which makes it easier to do refactoring, provided your changes preserve the behavior of the controllers and plugs."
- gcb0 10y ago
  they did black box unit test instead of proper unit test. e.g. if a method is private, it's not tested. that's the usual way when full reliability is not worth the dev time, or when devs think functional tests only are enough but still want a tap on the back for having unit tests and coverage numbers. in their case it's the later as you can see for: "Controller tests also exercise a lot of the code paths in your application" classical functional tests being called unit tests excuse. not that functional is better or worse, but correct names are better no matter what.
  - jurre 10y ago
    They don't call them unit tests though do they?
  - Jtsummers 10y ago
    The thing about unit testing is that you have to decide what your "unit" is. It could be down to the function/method level, but that's not always appropriate. Such a strict testing framework can leave you immobilized when you want or need to refactor. Their HTTP API is their interface for the application. It makes a great deal of sense to focus on that level for their testing (note: they didn't say they didn't test internals, they said they focused on the external interfaces).
chx 10y ago
Anyone has a small-ish gig where I could learn Phoenix/Elixir? I am a senior level programmer but all my experience is with PHP/Drupal. I would very much like to work on a smaller side (15-20 hrs a week) gig learning Phoenix/Elixir. A lot of experience is going to transfer, I hope (problem solving is not really language dependent) and so I think I offer a pretty good deal: you could get a very experienced coder for a language learner's rates :)
- messel 10y ago
  I've been hoping to pick it up on weekends & time off but startup hours don't leave much free time. I worked on a few simple heroku / json phoenix apps last year. Hoping I can put together a useful but small web app next.
  - dvcrn 10y ago
    I can recommend to both of you to get the book "Elixir in Action". It's fantastic! Starts very simple and step by step explores the must-need concepts of Elixir. I started phoenix before fully understanding Elixir and had a hard time. Then I got that book and when I was halfway through I loved Elixir already. Now I use phoenix for almost everything web related.
    - jon-wood 10y ago
      Programming Phoenix is also excellent. I think it helps that it was written by the core contributors to Phoenix itself (including Jose Valim, who created Elixir). I really appreciated not only looking at how things work, but also why they work in the way they do. The authors clearly have a real sense of excitement about Phoenix and its contagious.
- brightball 10y ago
  The RedFour instructional is a solid intro to Elixir as well.
- tortilla 10y ago
  Not mine, but here's the CMS behind thechangelog.com: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com Also take a look at the code powering hex.pm: https://github.com/hexpm/hex_web https://github.com/hexpm/hex_web
Roboprog 10y ago
Trivial, but it warms my heart to see short-ish identifiers with gaps in the names (underlines, but other languages use dashes, dots, etc), rather than reallyLongStudlyCaps names. I love that Elixir uses the Ruby convention of short names with underscores in them. C++ is the tragedy that keeps smiting the industry with its children. (e.g. - Java, and its silly names) "But we need long names, and need to compress them since they contain so many words". No, you need to better partition things so there is enough context around what the name is attached to, so that the name can be simple.
kriro 10y ago
Started meddling with Phoenix recently so this is an interesting data point. I have to say that the very basic architecture diagram from the PragProg book basically sold me on using it for my next side project(s). connection |> endpoint |> router |> pipeline |> controller Not sure what it is but it must be a combination of "|> seems pretty awesome" and "well yeah it obviously makes sense to get a request as a struct and chain functions in this way". It just clicked and aligned so perfectly with my mental model :)

239 points by mikecarlton 10y ago

jblow 10y ago

10,000 lines is supposed to be 'big'? wtf? I write something like 25kLoC/year (of shipping code, generally very complex stuff) and I don't even program full-time. The two projects I am working on now are 35kloc (the smaller one) and 250kloc (the medium-sized one). If someone thinks 10kloc is big, I have a hard time thinking of that person as a professional programmer. (Numbers listed here exclude blank lines and comments.)

partycoder 10y ago
Run a code duplication detector in your code. I am curious to see if you actually need 250,000 SLOC, or if it's actually just copying and pasting around.
tonyarkles 10y ago
One interesting thing I've found with my limited experience with Elixir is that the code is very compact. What would often take 10+ LoC in other languages can be succinctly expressed as 1-2 lines. But I do in general agree with your take on it :)
- jblow 10y ago
  > would often take 10+ LoC in other languages can be succinctly expressed as 1-2 lines. Every time I have heard this kind of claim (with modern languages), it turned out not to be true except for trivial code or straw-man bad code in the 'bigger' language. So if you have real-world examples that have real-world effort put in, I'd like to see them! (I would be happy to be wrong.) 5x-10x productivity increase would be huge if it actually existed; it would be so unstoppable that everyone would switch to the new really-great language immediately. That hasn't happened, which should be a clue that maybe the increase is not there. Even a 20% decrease in cost of engineering would be so large as to be unignorable.
  - kornish 10y ago
    > 5x-10x productivity increase would be huge if it actually existed When writing software, how fast you can type the code is rarely the limiting factor for speed of development – the architecting and consideration of interplay between components takes the bulk of the time. The grandparent claimed code reduction (which has intrinsic maintainability benefits) but made no statements about general cost of engineering.
    - jblow 10y ago
      > the architecting and consideration of interplay between components takes the bulk of the time. Which is supposed to be what is simplified as LOC goes down. So if a supposed 5x-10x code reduction (which I've never seen real evidence of) doesn't lead to 5x-10x productivity increase, how much increase is there supposed to be? Surely more than zero?
      - kornish 10y ago
        > Which is supposed to be what is simplified as LOC goes down. I don't think so. If you can express the same concepts with the same interfaces and functionality in 1kloc vs 10kloc, most of your time has probably still gone into figuring out the interfaces and connections. > So if a supposed 5x-10x code reduction (which I've never seen real evidence of) doesn't lead to 5x-10x productivity increase, how much increase is there supposed to be? Surely more than zero? Oh, certainly more than zero! Sometimes much more. But there's simply not a one-size-fits-all formula for the relationship between lines of code written and productivity. Anyways, not really sure what you're getting at. Your original comment was that 10kloc isn't "big"; the rebuttal is that lines of code is a naive way of looking at system complexity, which is presumably what you mean by "big".
        Scarbutt 10y ago
        You find having to read 1kloc vs 10kloc the same?
        10y ago
- fleetfox 10y ago
  Even if you multiply it by 3 it's a small codebase. And Phoenix does bunch of code generation from what I've seen.
sotojuan 10y ago
This is why I'm wary of posts about Elixir. I love the language a lot but the blog posts and articles on it are mostly hype by people without much experience.
rozap 10y ago
two things: 1) elixir is relatively young so there aren't many projects that have had a chance to grow to be huge. so yea, take everything you read about it with that grain of salt. people haven't grown to hate it yet...maybe everyone will always love it, but I wouldn't bet on it. 2) when used in certain contexts, elixir applications may end up being quite a bit smaller than comparable java/scala/python/whatever apps. A lot of what we end up writing with microservices is just basic RPC and fault tolerance boilerplate stuff, which OTP takes care of out of the box. Example, pinterest re-writing a 10kloc java app in 1kloc of elixir, running it on fewer servers: https://engineering.pinterest.com/blog/introducing-new-open-source-tools-elixir-community https://engineering.pinterest.com/blog/introducing-new-open-...
dudul 10y ago
10,000 lines is actually a lot for Elixir. Its syntax and built-in standard library makes the code very compact.
enraged_camel 10y ago
Pro-tip: never look at number of LOCs to measure the size or complexity of a project, unless you are comparing two systems written in the same language. Even then, comparisons might be tricky due to factors such as libraries used.
aikah 10y ago
I don't really think one can judge the work of someone else by the line of codes he writes a year. I thought IBM management failures demonstrated that kind of metric is meaningless. Furthermore, 10,000 lines of elixir/Ruby is not like 10,000 of say Java that don't do much except wiring dependencies.
- Roboprog 10y ago
  INDEED. 10K lines RoR = 100K lines JEE. More or less. (OK, I made those numbers up, but reading Java code is bloody tiring due to the silly walks and typical paste-o-graphic work styles)
strmpnk 10y ago
Some of the gain in concision is from the language it seems but I think this case is very much from the view of leveraging frameworks and libraries which themselves are quite a bit bigger than the app. I cloned a few of the repositories and excluded tooling that was used for documentation, testing, and so on. It's non-exhaustive since I only wanted to get a ballpark figure but we have: 1.4 million lines of Erlang 309 thousand lines of C 120 thousand lines of Elixir ... And lots of other code and things I've excluded. These are lines of code, not counting blank or comment-lines. Now we could argue that we could unwrap this count at any layer? Why stop at Erlang/OTP? Why not include the C runtime and the Kernel? The real effect here is that a lot of this code implement things that are directly used by Phoenix apps, so I think it's fair to say, it's concision comes from being very good at leveraging other code. (I don't write Elixir code but I've seen some good results from teams who have. None of it is magic though.)
xutopia 10y ago
What does it matter how many you write? How many do you delete? I have a project that is 5k lines of code and roughly 11k of tests and specs(cucumber). It is a rewrite of a project that was 50k lines of code with 1.2k lines of tests and had less functionality and features than what it does now. Lines of code are meaningless when it comes to how much value they provide. I personally prefer when a codebase is smaller because it means some thought was put into it and most likely has less bugs as a result.
- jblow 10y ago
  I said shipping code, by which I specifically mean that which is left after all deletions (of which there are many).
- cgag 10y ago
  I believe jblows games (braid, the witness) have less lines of code than one would typically expect. I think you should assume hes not writing code like a government contractor.
  - mars4rp 10y ago
    upvote for government contractor, I once saw 5k lines of code for simple change of address in the front end alone !!!
jamon51 10y ago
It's relatively large for a Phoenix web app, given that there aren't that many out there yet.
velox_io 10y ago
I'd say 10k LoC is where design and architecture start to become vital, you can't just hack away without breaking stuff. I personally think copy & paste is an anti-pattern (instead, use features such as generics), but many think it is a godsend. Using code reuse rather than copying code can make a massive difference in size and maintainability, less is definitely more!
- Roboprog 10y ago
  Dealing with that at work now. Dev added to project won't use the existing structure, just recreates and pastes stuff in over and over :-( Part of that is a political problem, of course, but was not an issue earlier.
pmontra 10y ago
I spent the best part of yesterday's afternoon writing a complex SQL query. I could have written a few straightforward queries instead and still be here writing, debugging and testing the code to merge their results. I have this single LOC now instead of 100+. That line is worth 100.
StreamBright 10y ago
I regularly delete or replace several thousands of lines of code without changing the functionality of application, put in by programmers like you, who are obsessed with the amount of LOC they produce. I am not sure why anybody thinks that a lengthy solution is great or we should judge programmers by the LOC they produce. I strongly believe that the better the programmer is the shorter the solution will be produced by her/him.
- Scea91 10y ago
  It's like judging the quality of a book by number of pages.

pmontra 10y ago

Typo in the title : it's Phoenix, not Phoeniz.

i336_ 10y ago
Not sure if the words "admin", "admins" or "mods" trigger anything? Mentioning just in case.
- rattray 10y ago
  try also dang
  - sctb 10y ago
    No, those don't have any magic, but if something like this persists you can always email us at hn@ycombinator.com.
    - i336_ 10y ago
      Ah, okay then. I was curious, thanks for the info. Idea: make users with high karma capable of using benign, unassuming and non-common parts of speech to ping the admins? ...I think I've been thinking about security too much
      - reitanqild 10y ago
        Just click the post timestamp and click flag first.
        i336_ 10y ago
        ...I am so dense. That makes perfect sense, thanks.
        reitanqild 10y ago
        > I am so dense. Maybe just busy with real work?

rdtsc 10y ago

Also I found Dialyzer (success typing checker) to be helpful in large projects: http://learnyousomeerlang.com/dialyzer http://learnyousomeerlang.com/dialyzer The way success typing works it is a bit like static typing but when it cannot deduce the types it assumes success. However when it finds a discrepancy it is always right. So the more type annotations you add, and the more precise they are the more benefits you get from it. It has also been there for many years. I think Python only very recently has started getting the same kind of things via MyPy. Of course, they had to call it something differently (Optional Static Typing). Elixir has a wrapper around it, it seems: https://github.com/jeremyjh/dialyxir https://github.com/jeremyjh/dialyxir but never used it (I user Erlang mostly).

crbelaus 10y ago
Totaly agree. Dialyzer/Dialyxir can provide a high degree of confidence about types. The wonderful thing about it is that you don't have to get all type annotations since the begining, you can add them over the time. This allows to use "unspecified" types for quick prototyping, and make them more specific when the project evolves, which is a major complain of traditional type systems such as Java's. Another tool that I find useful is Credo[1] which "is a static code analysis tool for the Elixir language with a focus on teaching and code consistency". The Ecto[2] project uses a tool called Ebert[3] that automatically runs Credo for each pull-request and comments with the issues found. Here you can see an example of Ebert's bot commenting on a PR[4] [1] https://github.com/rrrene/credo https://github.com/rrrene/credo [2] https://github.com/elixir-ecto/ecto https://github.com/elixir-ecto/ecto [3] https://ebertapp.io/ https://ebertapp.io/ [4] https://github.com/elixir-ecto/ecto/pull/1785 https://github.com/elixir-ecto/ecto/pull/1785

saalaa 10y ago

> Lessons Learned from a Large Phoenix Project More like "large" projects.

yellowapple 10y ago

I feel like the bit about OTP misses the point of OTP. Erlang is not just about distributed computing (in fact, it never was; any affinity for distributed computing was more of a side-effect of Erlang's design). Rather, it's about fault-tolerance. Supervision trees and "let it crash" are the cornerstone of Erlang programming, and therefore by extension the cornerstone of Elixir programming. Meanwhile, OTP applications build on this in a way that permits composability. It's kind of like microservices behind the scenes, bit they feel like a monolith; you build up your system from lots of different OTP applications that work together to provide a unified whole. Elixir and Erlang web frameworks (Phoenix, Sugar, Chicago Boss (IIRC), etc.) already do a lot of this for you by kicking off various OTP dependencies; for example, your average Phoenix or Sugar application will in turn start Plug, Ecto, and various other OTP apps, and these will in turn spin up their own dependencies (like Cowboy and Postgrex, respectively). Basically, it's not quite right to equate OTP to just distributed computing. OTP is at the heart and soul of the vast majority of software written for BEAM.

misterbowfinger 10y ago

> 5. Avoid Non-RESTful Routes Every "large" project gets to a point where some routes aren't totally RESTful. It happens. But it's not great advice to say that non-RESTful routes are always a code smell.

arms 10y ago
Agreed. I'm working on an API where most of the endpoints are RESTful, but there's a handful where it makes total sense to break the rules in order to make everyone's lives easier.
aikah 10y ago
What a total RESTful route to begin with ? I thought REST had nothing to do with "routes", url or resources but how the state of the application is communicated in an API ? It seems to me the Roy wrote something with a certain perspective, someone came up, read the paper and decided to apply his writings to something totally unrelated because he didn't like SOAP ... I personally stopped using that word so I don't have to confront and debate with "REST purists" and just went back to talking about web service, because that's what it is.
- Arcaire 10y ago
  The idea is to limit exposed routes to the standard CRUD routes: create (/new), read (/show), update (/edit), delete (/destroy).
  - aikah 10y ago
    I thought the idea was to make an API behave like a webpage by using hyperlinks that represents the state of the application, so in theory a "smart" client could navigate an API autonomously like a human interacts with a web page ? Because otherwise it's no different from RPC.
    - anthonybullard 10y ago
      I think you are thinking of HATEOAS. Read about it here: https://en.wikipedia.org/wiki/HATEOAS?wprov=sfla1 https://en.wikipedia.org/wiki/HATEOAS?wprov=sfla1

thijsvandien 10y ago

Could anyone more knowledgable explain whether or not my feeling is correct that Elixir becoming (relatively) popular is making Erlang a less viable choice? Whereas the total number of libraries may be increasing, code that might have been written in Erlang is now written in Elixir and established Erlang codebases (RabbitMQ for one) have started to migrate (parts) to Elixir. Calling Erlang from Elixir is easy, but what about the reverse? It reminds of me of how, at least in the early days, Play Framework (largely Scala) could be used from Java but not without much friction. Should I prefer Erlang as a language for its maturity and fewer constructs, it would be a pity to then miss out on new libraries or frameworks, such as Phoenix, that revolve around Elixir.

dsri 10y ago
Yes, you can call Elixir from Erlang: http://elixir-lang.org/crash-course.html#adding-elixir-to-existing-erlang-programs http://elixir-lang.org/crash-course.html#adding-elixir-to-ex...
- thijsvandien 10y ago
  I did expect it to be theoretically possible, but how realistic/sensible is it really to do an Erlang project with Elixir libraries or frameworks? Elixir brings a lot of its own tools, conventions and so on. Will it soon take over my project, making it a better option to just use Elixir for everything, or would it be very manageable and not interfere much with the rest?
  - qaq 10y ago
    If anyone from Erlang Solutions is around they prob would be most qualified to answer.
    - claudioesl 10y ago
      Claudio from Erlang Solutions here. I'll ask around in the company for more information. I've worked briefly on wrapping an existing erlang app in a Elixir (Nerves) container and I did have a few issues around dependency management which required small changes here and there, but nothing major. One area where I usually spend more time than I'd like is to get type specifications in good shape so that dialyzer doesn't report too many warnings, but that gets better with every Erlang/Elixir release.

crasm 10y ago

I was expecting the IT and devops book, The Phoenix Project: http://itrevolution.com/books/phoenix-project-devops-book/ http://itrevolution.com/books/phoenix-project-devops-book/

akavel 10y ago

In section titled "3. Write Fewer, Valuable Tests", I don't understand the terminology ("controllers and plugs"), which I assume is some stuff in the Phoenix framework that I have no idea about. Can somebody possibly explain to me what are the "controllers and plugs" here? I'm very interested in learning a more general idea (how to effectively reduce lines of tests) from this nugget, that I could hopefully translate to different languages & frameworks I use... Quoting the relevant fragment in full: "We focused our automated tests on our controller actions and plugs rather than going for 100% test coverage. Since these are the main ways that the Phoenix application interfaces with the outside world, they’re the critical points of failure. Controller tests also exercise a lot of the code paths in your application, making it less necessary to unit test every single module. As a result, you end up with fewer tests, which makes it easier to do refactoring, provided your changes preserve the behavior of the controllers and plugs."

gcb0 10y ago
they did black box unit test instead of proper unit test. e.g. if a method is private, it's not tested. that's the usual way when full reliability is not worth the dev time, or when devs think functional tests only are enough but still want a tap on the back for having unit tests and coverage numbers. in their case it's the later as you can see for: "Controller tests also exercise a lot of the code paths in your application" classical functional tests being called unit tests excuse. not that functional is better or worse, but correct names are better no matter what.
- jurre 10y ago
  They don't call them unit tests though do they?
- Jtsummers 10y ago
  The thing about unit testing is that you have to decide what your "unit" is. It could be down to the function/method level, but that's not always appropriate. Such a strict testing framework can leave you immobilized when you want or need to refactor. Their HTTP API is their interface for the application. It makes a great deal of sense to focus on that level for their testing (note: they didn't say they didn't test internals, they said they focused on the external interfaces).

chx 10y ago

Anyone has a small-ish gig where I could learn Phoenix/Elixir? I am a senior level programmer but all my experience is with PHP/Drupal. I would very much like to work on a smaller side (15-20 hrs a week) gig learning Phoenix/Elixir. A lot of experience is going to transfer, I hope (problem solving is not really language dependent) and so I think I offer a pretty good deal: you could get a very experienced coder for a language learner's rates :)

messel 10y ago
I've been hoping to pick it up on weekends & time off but startup hours don't leave much free time. I worked on a few simple heroku / json phoenix apps last year. Hoping I can put together a useful but small web app next.
- dvcrn 10y ago
  I can recommend to both of you to get the book "Elixir in Action". It's fantastic! Starts very simple and step by step explores the must-need concepts of Elixir. I started phoenix before fully understanding Elixir and had a hard time. Then I got that book and when I was halfway through I loved Elixir already. Now I use phoenix for almost everything web related.
  - jon-wood 10y ago
    Programming Phoenix is also excellent. I think it helps that it was written by the core contributors to Phoenix itself (including Jose Valim, who created Elixir). I really appreciated not only looking at how things work, but also why they work in the way they do. The authors clearly have a real sense of excitement about Phoenix and its contagious.
brightball 10y ago
The RedFour instructional is a solid intro to Elixir as well.
tortilla 10y ago
Not mine, but here's the CMS behind thechangelog.com: https://github.com/thechangelog/changelog.com https://github.com/thechangelog/changelog.com Also take a look at the code powering hex.pm: https://github.com/hexpm/hex_web https://github.com/hexpm/hex_web

Roboprog 10y ago

Trivial, but it warms my heart to see short-ish identifiers with gaps in the names (underlines, but other languages use dashes, dots, etc), rather than reallyLongStudlyCaps names. I love that Elixir uses the Ruby convention of short names with underscores in them. C++ is the tragedy that keeps smiting the industry with its children. (e.g. - Java, and its silly names) "But we need long names, and need to compress them since they contain so many words". No, you need to better partition things so there is enough context around what the name is attached to, so that the name can be simple.

kriro 10y ago

Started meddling with Phoenix recently so this is an interesting data point. I have to say that the very basic architecture diagram from the PragProg book basically sold me on using it for my next side project(s). connection |> endpoint |> router |> pipeline |> controller Not sure what it is but it must be a combination of "|> seems pretty awesome" and "well yeah it obviously makes sense to get a request as a struct and chain functions in this way". It just clicked and aligned so perfectly with my mental model :)