5 ms·

Learnings from One Year of Building an Open Source Elixir Application

197 points by achariam 9y ago

darkmarmot 9y ago
Having a sign up button or perhaps context or text on the site might've been helpful...
- achariam 9y ago
  You're completely right, it seems a little counterintuitive but one thing I learned early on while building a different community site (Designer News) was the way to build a wholesome community was to be invite only in the beginning. That and real first and last names. Which means it's a poor user experience. Sorry for the trouble, I wonder if any folks have better ideas here?
  - bpicolo 9y ago
    Not a single page even mentions that it's invite only =/. Context for that would be a good start heh
  - wolco 9y ago
    An invite only for designers because the barrier to entry is so low it makes sense. Not sure that would work or is as helpful in this case. The community seems more friendly than most and the general goals of many are to learn something new, powerful and fun. If you build a community around those themes I think it will attract the right crowd.
brosky117 9y ago
I really enjoyed this article and the one about the ambient notification cube! Well-written. It's nice to hear someone else talk about how hard it it to stay consistent with worthwhile projects while maintaining a day job. Congrats on sticking it out!
- achariam 9y ago
  Thank you for the kind words, really appreciate it :)
deedubaya 9y ago
One of the interesting things about the Elixir/Phoenix community is that building your own authentication system seems to be encouraged. Even if you're using one of the plugs available (guardian?) you still have to do a ton of manual lifting.
- rpazyaquian 9y ago
  This seems to be the case in many places outside of Rails. I know that Clojure, at least, seems to encourage implementing things yourself by way of leveraging smaller libraries and modules. Auth is genuinely hard, and turnkey solutions often aren't enough.
  - s0l1dsnak3123 9y ago
    Let's not also forget that lots of projects people are using Phoenix for are APIs (either for mobile apps or single page web applications). Because of that, it makes sense to glue together smaller libs to match your house style for how you want that auth contract to work.
    - throwaway47861 9y ago
      Can confirm. Currently working on several Elixir API apps, they are absolute joy to work on and you get tons of freedom on how exactly to structure them. The "cruft" from the Phoenix framework is absolutely minimal -- mandatory 4-6 files per project at the most. And they're quite small, too.
  - WillPostForFood 9y ago
    I agree Auth is hard, but that makes me want a good turnkey solution even more.
  - fny 9y ago
    Auth is genuinely hard, and a hand-rolled solution's definitely not enough... Given the massive attack surface for a web application, it's absurd to think someone could (or should*) develop an entire auth framework from scratch for all their projects. Turnkey solutions, like Ruby's Devise, are a godsend. Even in situations were a custom flow was needed, it's saved both me and my clients hundreds of hours. In addition, I benefit from the community around a turnkey solution. Think of all the years the software has been tested by in umpteen production environments. Think of all the people that have an eye on the code and report security flaws while you sleep. Your custom session management implementation will never have that benefit. Also, why the hell are you building auth when you could be building app? Is that really such a critical experiential part of your app that you can't possibly rely on something turnkey and then move on to the features that actually matter?
codegeek 9y ago
Great writing. Elixir/Phoenix has been on my radar for learning something exciting considering it is faster out of the box and can handle tons of concurrent connections. For the lazy, here is the github link from this article: https://github.com/achariam/elyxel https://github.com/achariam/elyxel
ploggingdev 9y ago
Interesting post, more so because I am working on a project that has many characteristics of a link aggregator. While implementing the ranking algorithm, which is very similar to the one mentioned in the article, I decided to run a periodic job every 60 seconds that updates the rank for each submission and stores it in the database so querying the ranked data is more efficient than recalculating the rank on every page request. Are you doing something similar or did you take a different approach? Ranking all stories works well if the total number of submissions is a small number, but I imagine the approach is a little different for large sites like HN. Ranking all submissions periodically seems like a waste since people rarely view submissions beyond 10 pages. One approach is just to rank submissions from the past n days, where n depends on the average daily submission volume. For the part that displays time since a submission was made, I implemented the HN model, where it displays only minutes, hours and days. Python code here : https://dpaste.de/5d1w https://dpaste.de/5d1w > Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something. Once you start to scale, the bottleneck is rarely the application layer. For the typical crud web app it's likely to be the database.
- tim333 9y ago
  Not sure if it's relevant but you can avoid having elapsed time in the formula and instead inflate the scores of the newer stories (https://news.ycombinator.com/item?id=9889750 https://news.ycombinator.com/item?id=9889750) Then you only have to recalculate when there is a submission or vote.
- pdimitar 9y ago
  > Once you start to scale, the bottleneck is rarely the application layer. For the typical crud web app it's likely to be the database. You seem to be downplaying the importance of an efficient app layer and I respectfully disagree. 1. Using well-optimized tech for the app sends you right at the DB scaling problems phase, so you never go through the "our app layer is too damn slow" phase -- which can kill a startup pretty damn quick (and history knows examples). So, it's a huge win in my eyes. 2. Most apps never even take off to the point where the DB is the bottleneck so a strong app layer is a godsend; I'd estimate 80% of the apps struggle with optimizing backends and caches and not DBs -- at least judging by my 15 years of career which, I admit, is anecdotal evidence and doesn't mean much. 3. Those apps that do scale that far are very grateful to their backend engineers who spared them all the "tune our Java / Ruby / PHP / Python VM's memory usage and GC behaviour parameters" dance (and that dance can last for weeks and weeks almost without sleep!).
agentgt 9y ago
> Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something. Thats much appreciated but I was kind of hoping you (the author) would go into more details about request time. Most people can (and should) do the above. However I would be more interested in what Elixer + CRUD is like particularly for TTFB. Like does the author do streaming (I don't necessarily mean websockets or comet)? After all if the TTFB is really slow the CSS optimizations and what not matter little. In traditional request per thread (or whatever is analagous) web framework paradigms the request is a single thread and often waits for the database to finish before moving on to display the page. I would imagine Elixir has a better answer at least for read only pages.
- abrookewood 9y ago
  Time To First Byte?
- pdimitar 9y ago
  On my virtual Debian, the development version of a relatively complex website -- which is probably anywhere between 5 to 30 times slower than production -- has 450ms for the first ever request after the server starts. Anything from then on goes between 0.2 to 12ms. Not really telling you much, I know, but these are pretty standard numbers for Elixir + Phoenix apps.
overcast 9y ago
It would be awesome if elyxel.com wasn't so esoteric.

197 points by achariam 9y ago

darkmarmot 9y ago

Having a sign up button or perhaps context or text on the site might've been helpful...

achariam 9y ago
You're completely right, it seems a little counterintuitive but one thing I learned early on while building a different community site (Designer News) was the way to build a wholesome community was to be invite only in the beginning. That and real first and last names. Which means it's a poor user experience. Sorry for the trouble, I wonder if any folks have better ideas here?
- bpicolo 9y ago
  Not a single page even mentions that it's invite only =/. Context for that would be a good start heh
- wolco 9y ago
  An invite only for designers because the barrier to entry is so low it makes sense. Not sure that would work or is as helpful in this case. The community seems more friendly than most and the general goals of many are to learn something new, powerful and fun. If you build a community around those themes I think it will attract the right crowd.

brosky117 9y ago

I really enjoyed this article and the one about the ambient notification cube! Well-written. It's nice to hear someone else talk about how hard it it to stay consistent with worthwhile projects while maintaining a day job. Congrats on sticking it out!

achariam 9y ago
Thank you for the kind words, really appreciate it :)

deedubaya 9y ago

One of the interesting things about the Elixir/Phoenix community is that building your own authentication system seems to be encouraged. Even if you're using one of the plugs available (guardian?) you still have to do a ton of manual lifting.

rpazyaquian 9y ago
This seems to be the case in many places outside of Rails. I know that Clojure, at least, seems to encourage implementing things yourself by way of leveraging smaller libraries and modules. Auth is genuinely hard, and turnkey solutions often aren't enough.
- s0l1dsnak3123 9y ago
  Let's not also forget that lots of projects people are using Phoenix for are APIs (either for mobile apps or single page web applications). Because of that, it makes sense to glue together smaller libs to match your house style for how you want that auth contract to work.
  - throwaway47861 9y ago
    Can confirm. Currently working on several Elixir API apps, they are absolute joy to work on and you get tons of freedom on how exactly to structure them. The "cruft" from the Phoenix framework is absolutely minimal -- mandatory 4-6 files per project at the most. And they're quite small, too.
- WillPostForFood 9y ago
  I agree Auth is hard, but that makes me want a good turnkey solution even more.
- fny 9y ago
  Auth is genuinely hard, and a hand-rolled solution's definitely not enough... Given the massive attack surface for a web application, it's absurd to think someone could (or should*) develop an entire auth framework from scratch for all their projects. Turnkey solutions, like Ruby's Devise, are a godsend. Even in situations were a custom flow was needed, it's saved both me and my clients hundreds of hours. In addition, I benefit from the community around a turnkey solution. Think of all the years the software has been tested by in umpteen production environments. Think of all the people that have an eye on the code and report security flaws while you sleep. Your custom session management implementation will never have that benefit. Also, why the hell are you building auth when you could be building app? Is that really such a critical experiential part of your app that you can't possibly rely on something turnkey and then move on to the features that actually matter?

codegeek 9y ago

Great writing. Elixir/Phoenix has been on my radar for learning something exciting considering it is faster out of the box and can handle tons of concurrent connections. For the lazy, here is the github link from this article: https://github.com/achariam/elyxel https://github.com/achariam/elyxel

ploggingdev 9y ago

Interesting post, more so because I am working on a project that has many characteristics of a link aggregator. While implementing the ranking algorithm, which is very similar to the one mentioned in the article, I decided to run a periodic job every 60 seconds that updates the rank for each submission and stores it in the database so querying the ranked data is more efficient than recalculating the rank on every page request. Are you doing something similar or did you take a different approach? Ranking all stories works well if the total number of submissions is a small number, but I imagine the approach is a little different for large sites like HN. Ranking all submissions periodically seems like a waste since people rarely view submissions beyond 10 pages. One approach is just to rank submissions from the past n days, where n depends on the average daily submission volume. For the part that displays time since a submission was made, I implemented the HN model, where it displays only minutes, hours and days. Python code here : https://dpaste.de/5d1w https://dpaste.de/5d1w > Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something. Once you start to scale, the bottleneck is rarely the application layer. For the typical crud web app it's likely to be the database.

tim333 9y ago
Not sure if it's relevant but you can avoid having elapsed time in the formula and instead inflate the scores of the newer stories (https://news.ycombinator.com/item?id=9889750 https://news.ycombinator.com/item?id=9889750) Then you only have to recalculate when there is a submission or vote.
pdimitar 9y ago
> Once you start to scale, the bottleneck is rarely the application layer. For the typical crud web app it's likely to be the database. You seem to be downplaying the importance of an efficient app layer and I respectfully disagree. 1. Using well-optimized tech for the app sends you right at the DB scaling problems phase, so you never go through the "our app layer is too damn slow" phase -- which can kill a startup pretty damn quick (and history knows examples). So, it's a huge win in my eyes. 2. Most apps never even take off to the point where the DB is the bottleneck so a strong app layer is a godsend; I'd estimate 80% of the apps struggle with optimizing backends and caches and not DBs -- at least judging by my 15 years of career which, I admit, is anecdotal evidence and doesn't mean much. 3. Those apps that do scale that far are very grateful to their backend engineers who spared them all the "tune our Java / Ruby / PHP / Python VM's memory usage and GC behaviour parameters" dance (and that dance can last for weeks and weeks almost without sleep!).

agentgt 9y ago

> Elyxel was designed and built with performance in mind. Styles and any additional flourishes were kept to a minimum. My choice of Elixir & Phoenix was driven by this consideration as well. Most of the pages are well under 100 kilobytes and load in less than 100 milliseconds5. I find it's always helpful to keep performance in the back of your mind when building something. Thats much appreciated but I was kind of hoping you (the author) would go into more details about request time. Most people can (and should) do the above. However I would be more interested in what Elixer + CRUD is like particularly for TTFB. Like does the author do streaming (I don't necessarily mean websockets or comet)? After all if the TTFB is really slow the CSS optimizations and what not matter little. In traditional request per thread (or whatever is analagous) web framework paradigms the request is a single thread and often waits for the database to finish before moving on to display the page. I would imagine Elixir has a better answer at least for read only pages.

abrookewood 9y ago
Time To First Byte?
pdimitar 9y ago
On my virtual Debian, the development version of a relatively complex website -- which is probably anywhere between 5 to 30 times slower than production -- has 450ms for the first ever request after the server starts. Anything from then on goes between 0.2 to 12ms. Not really telling you much, I know, but these are pretty standard numbers for Elixir + Phoenix apps.

overcast 9y ago

It would be awesome if elyxel.com wasn't so esoteric.