5 ms·

From zero to Production Elixir in 1 month

78 points by myth_drannon 9y ago

ramchip 9y ago
This feels like a bit... too much. If you’re really going for microservices, that means every service will be small and specialized. Do you really need to split it further into so many sub-apps? I feel like the S3 app is going to end up as one module with a handful of functions.
- rothbardrand 9y ago
  These are medium sized. Remember in elixir it's perfectly convenient to map across a list by spawning a process for each element in the list and sending the function you want to map to that process, and then they collapse down and get reduced. A process (or application) in elixir is no big deal, just think of it as a module or object that lives in its own process, probably has a set of processes and all that... If you were writing this in go you might call it a monolith because all of those "Apps" would be in one single binary executable.
  - ramchip 9y ago
    My comment is about code organization, not process usage. You can have 6 separate apps and still do everything in a single process. By splitting into lots of sub-apps, you end up with a deep folder structure like `my_service/apps/s3/lib/s3/query.ex`, each app having its own versioning, configuration, tests, etc. That makes sense for a sizable codebase, but for a microservice, I don't think the added complexity is worth the benefit. It can be organized very cleanly with just modules and folders.
- novembermike 9y ago
  These aren't sub applications, they're more similar functionally to classes, although they tend to end up a little larger in a lot of cases.
  - ramchip 9y ago
    They are applications. That's what an umbrella does, it groups multiple Erlang applications into one project.
- anon4728 9y ago
  Erlang processes are cheap, unlike OS processes. More, the merrier.
hartator 9y ago
The author seems to be unaware of being able to upload directly to S3 via JS eliminating the need of even the Phoenix app.
- sb8244 9y ago
  What type of security implications are there from this? Are things like rate-limiting (by time period, all-time) or access control handled properly?
  - kmf 9y ago
    I clicked on this article because I needed to do something similar this week, and ended up using the JS-based approach. I’ve built something like this with S3 before, but ended up doing it with Google Cloud Storage this time around. The process looked like this: 1) Client-side app receives an image (via drag-and-drop or file picker) 2) Client hits an authenticated endpoint on backend with the name of the file, which returns a “signed URL” (authenticated, with customizable expiration) from Google Cloud Storage 3) (This part is particular to GCS, and I think unneeded for AWS/S3) Hit this signed URL in the client, and receive a final, PUT-able URL 4) Upload the file using the final authenticated URL from Google Cloud Storage Like I said, I think with AWS/S3 you can just request a signed URL and send data to that URL directly. Delegating as much communication with GCS to the backend was important for my project [1] but you might be able to just manage it with a JS lib. [1] Streaming music, so trying to abstract the actual files from the client-side as much as possible to discourage downloading the raw audio files
    - blatyo 9y ago
      s3 requires a signature as well. Without requiring a signature, anyone could upload anything to your s3 account. With a signature, you choose when and where people can put their files.
    - pramodliv1 9y ago
      If we want to store the object's URL in a relational database, should the browser send another HTTP request to our service? Wouldn't we lose the object if there's a network error during the interval between the two requests?
pmontra 9y ago
Code?
colbyh 9y ago
as much as I love Elixir (and Erlang/BEAM) I get uncomfortable when people say "I heard it was a magic bullet for scalability and concurrency[...]. the language provides primitives that can, in certain circumstances, make it easier to scale horizontally, but there's no magic. and there are certainly tradeoffs (esp with regards to managing state) that can actually make your code harder to scale. dig the post, just an observation. for anyone new to the language reading this please understand that Elixir isn't a super fast/concurrent Ruby replacement. but it is a hell of a lot of fun to program in.
- weego 9y ago
  I feel like scala went through the same thing with akka, just with less blog hype. It's a panacea until someone asks how you manage state.

78 points by myth_drannon 9y ago

This feels like a bit... too much. If you’re really going for microservices, that means every service will be small and specialized. Do you really need to split it further into so many sub-apps? I feel like the S3 app is going to end up as one module with a handful of functions.

rothbardrand 9y ago
These are medium sized. Remember in elixir it's perfectly convenient to map across a list by spawning a process for each element in the list and sending the function you want to map to that process, and then they collapse down and get reduced. A process (or application) in elixir is no big deal, just think of it as a module or object that lives in its own process, probably has a set of processes and all that... If you were writing this in go you might call it a monolith because all of those "Apps" would be in one single binary executable.
- ramchip 9y ago
  My comment is about code organization, not process usage. You can have 6 separate apps and still do everything in a single process. By splitting into lots of sub-apps, you end up with a deep folder structure like `my_service/apps/s3/lib/s3/query.ex`, each app having its own versioning, configuration, tests, etc. That makes sense for a sizable codebase, but for a microservice, I don't think the added complexity is worth the benefit. It can be organized very cleanly with just modules and folders.
novembermike 9y ago
These aren't sub applications, they're more similar functionally to classes, although they tend to end up a little larger in a lot of cases.
- ramchip 9y ago
  They are applications. That's what an umbrella does, it groups multiple Erlang applications into one project.
anon4728 9y ago
Erlang processes are cheap, unlike OS processes. More, the merrier.

hartator 9y ago

The author seems to be unaware of being able to upload directly to S3 via JS eliminating the need of even the Phoenix app.

sb8244 9y ago
What type of security implications are there from this? Are things like rate-limiting (by time period, all-time) or access control handled properly?
- kmf 9y ago
  I clicked on this article because I needed to do something similar this week, and ended up using the JS-based approach. I’ve built something like this with S3 before, but ended up doing it with Google Cloud Storage this time around. The process looked like this: 1) Client-side app receives an image (via drag-and-drop or file picker) 2) Client hits an authenticated endpoint on backend with the name of the file, which returns a “signed URL” (authenticated, with customizable expiration) from Google Cloud Storage 3) (This part is particular to GCS, and I think unneeded for AWS/S3) Hit this signed URL in the client, and receive a final, PUT-able URL 4) Upload the file using the final authenticated URL from Google Cloud Storage Like I said, I think with AWS/S3 you can just request a signed URL and send data to that URL directly. Delegating as much communication with GCS to the backend was important for my project [1] but you might be able to just manage it with a JS lib. [1] Streaming music, so trying to abstract the actual files from the client-side as much as possible to discourage downloading the raw audio files
  - blatyo 9y ago
    s3 requires a signature as well. Without requiring a signature, anyone could upload anything to your s3 account. With a signature, you choose when and where people can put their files.
  - pramodliv1 9y ago
    If we want to store the object's URL in a relational database, should the browser send another HTTP request to our service? Wouldn't we lose the object if there's a network error during the interval between the two requests?

pmontra 9y ago

Code?

colbyh 9y ago

as much as I love Elixir (and Erlang/BEAM) I get uncomfortable when people say "I heard it was a magic bullet for scalability and concurrency[...]. the language provides primitives that can, in certain circumstances, make it easier to scale horizontally, but there's no magic. and there are certainly tradeoffs (esp with regards to managing state) that can actually make your code harder to scale. dig the post, just an observation. for anyone new to the language reading this please understand that Elixir isn't a super fast/concurrent Ruby replacement. but it is a hell of a lot of fun to program in.

weego 9y ago
I feel like scala went through the same thing with akka, just with less blog hype. It's a panacea until someone asks how you manage state.