5 ms·

Shell scripting in Elixir is like leaf-blowing your driveway with a helicopter.

diggan 3y ago
Awesome, but too expensive to be within reach of the ordinary person and more harmful to the environment compared to alternative methods?
- bryanrasmussen 3y ago
  elevator leaf blowing might be harmful to the environment but whats the argument about elixir in this analogy?
  - diggan 3y ago
    I have no idea what parent is trying to say with their analogy, I'm trying to tease out some meaning from what they said. If anything, Elixir would be more energy efficient than bash as far as I can tell.
    - fredrikholm 3y ago
      > Elixir would be more energy efficient than bash as far as I can tell For long running scripts perhaps. BEAM idles at ~50Mb, and startup is often more than 500ms. Eg: $ time ./hello hello world real 0m0,009s user 0m0,003s sys 0m0,007s $ time elixir hello.exs hello world real 0m0,755s user 0m0,992s sys 0m0,304s Where the first example is a BASH script.
      - throwawaymaths 3y ago
        Yes because when you load elixir you're loading the entire gOddamn Erlang VM. Which you might want for things like reliable parallel downloading, but you don't for hello world.
      - toast0 3y ago
        I don't know that you'll get it down to 10 ms, but you can get beam startup a lot faster if you adjust parameters. The default startup does a lot of things you simply don't need in most shell scripts. Starting a bunch of threads, setting up dist, I can't remember what else off the top of my head.
        fredrikholm 3y ago
        Even without the shell and with epmd turned off we're still in the ball bark of 200-300ms: $ time erl -eval 'io:format("~s~n", [os:cmd("echo hello world")]), halt(0, [{flush, false}]).' -noshell -start_epmd false hello world real 0m0,332s user 0m0,461s sys 0m0,092s I spent some time last year trying to get those numbers down but this is as far as I made it. Do you know of any more flags that might help?
        toast0 3y ago
        Try -boot no_dot_erlang A couple years ago that got me to 100 ms https://news.ycombinator.com/item?id=25243246 https://news.ycombinator.com/item?id=25243246 Today's measurements with FreeBSD 13.2 on the same Pentium G3470 time /bin/sh -c "echo hi" hi real 0m0.004s user 0m0.001s sys 0m0.002s time /usr/local/bin/bash -c "echo hi" hi hi real 0m0.006s user 0m0.007s sys 0m0.000s time /usr/local/lib/erlang26/bin/erl -eval 'io:format("hi~n"), halt(0, [{flush, false}])' Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [jit:ns] [dtrace] [sharing-preserving] hi real 0m0.197s user 0m0.178s sys 0m0.018s time /usr/local/lib/erlang26/bin/erl -eval 'io:format("hi~n"), halt(0, [{flush, false}])' -noshell -start_epmd false -boot no_dot_erlang hi real 0m0.183s user 0m0.181s sys 0m0.011s This doesn't look very good, actually! FreeBSD has OTP 21-26 available through pkg, I ran the same thing on each of them several times to warm up, and get (real time): 21: 81ms 22: 81ms 23: 74ms 24: 129ms 25: 135ms 26: 177ms So maybe there's some new stuff that needs to be avoided now.
        fredrikholm 3y ago
        It was your original post that set me on course for experimenting with startup times, cheers! Unfortunately, that final flag doesn't change much on 26 for me. IIRC I dropped a bunch of flags after concluding that no shell and no epmd were the only flags that I noticed improvements with.
- thibaut_barrere 3y ago
  If your script needs to do things like reliable HTTP querying and JSON parsing, I would say using Elixir (once you get used to it) won’t be harmful in any way, quite the contrary. Interestingly, what drove me to Ruby initially is exactly writing scripting tasks in a C#/C++ culture where Ruby would have been considered harmful. Now Elixir can be used for the same type of tasks, but only better (eg: trivial parallelized downloads!)
  - neonsunset 3y ago
    F# and C# are very different to C++ in this regard. Parallel downloads in C# boil down to using var http = new HttpClient(); var urls = new[] { "http://example.com/file1.jpg", "http://example.com/file2.jpg", /* ... */ }; var files = await Task.WhenAll( urls.Select(http.GetByteArrayAsync)); (you can replace '.GetByteArrayAsync' with .GetFromJsonAsync, .GetStreamAsync and more depending on your needs and e.g. forward the incoming data directly to files on the disk or another source)
lionkor 3y ago
You should, if you can?
- adityaathalye 3y ago
  Absolutely. What is life without any fun?
thibaut_barrere 3y ago
Ahaha, in a good way. Recently I have used Elixir scripts to mass-parallel-download numerous HTTP resources and do some stats on them, and it is exactly the feeling I had.
dtx1 3y ago
Suprisingly relevant https://www.youtube.com/watch?v=Ss6VBYLjMJQ https://www.youtube.com/watch?v=Ss6VBYLjMJQ
wturner 3y ago
I envision the creation of a GenServer for each and every file on your system, using this to duplicate them by the thousands, then deleting them in a timed contraction over and over and over again. I call this, "the blowcopter".
Too 3y ago
Still better than the inverse, that lots of scripters are doing; Building a helicopter out of a leaf-blower.

Awesome, but too expensive to be within reach of the ordinary person and more harmful to the environment compared to alternative methods?

bryanrasmussen 3y ago
elevator leaf blowing might be harmful to the environment but whats the argument about elixir in this analogy?
- diggan 3y ago
  I have no idea what parent is trying to say with their analogy, I'm trying to tease out some meaning from what they said. If anything, Elixir would be more energy efficient than bash as far as I can tell.
  - fredrikholm 3y ago
    > Elixir would be more energy efficient than bash as far as I can tell For long running scripts perhaps. BEAM idles at ~50Mb, and startup is often more than 500ms. Eg: $ time ./hello hello world real 0m0,009s user 0m0,003s sys 0m0,007s $ time elixir hello.exs hello world real 0m0,755s user 0m0,992s sys 0m0,304s Where the first example is a BASH script.
    - throwawaymaths 3y ago
      Yes because when you load elixir you're loading the entire gOddamn Erlang VM. Which you might want for things like reliable parallel downloading, but you don't for hello world.
    - toast0 3y ago
      I don't know that you'll get it down to 10 ms, but you can get beam startup a lot faster if you adjust parameters. The default startup does a lot of things you simply don't need in most shell scripts. Starting a bunch of threads, setting up dist, I can't remember what else off the top of my head.
      - fredrikholm 3y ago
        Even without the shell and with epmd turned off we're still in the ball bark of 200-300ms: $ time erl -eval 'io:format("~s~n", [os:cmd("echo hello world")]), halt(0, [{flush, false}]).' -noshell -start_epmd false hello world real 0m0,332s user 0m0,461s sys 0m0,092s I spent some time last year trying to get those numbers down but this is as far as I made it. Do you know of any more flags that might help?
        toast0 3y ago
        Try -boot no_dot_erlang A couple years ago that got me to 100 ms https://news.ycombinator.com/item?id=25243246 https://news.ycombinator.com/item?id=25243246 Today's measurements with FreeBSD 13.2 on the same Pentium G3470 time /bin/sh -c "echo hi" hi real 0m0.004s user 0m0.001s sys 0m0.002s time /usr/local/bin/bash -c "echo hi" hi hi real 0m0.006s user 0m0.007s sys 0m0.000s time /usr/local/lib/erlang26/bin/erl -eval 'io:format("hi~n"), halt(0, [{flush, false}])' Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:1] [jit:ns] [dtrace] [sharing-preserving] hi real 0m0.197s user 0m0.178s sys 0m0.018s time /usr/local/lib/erlang26/bin/erl -eval 'io:format("hi~n"), halt(0, [{flush, false}])' -noshell -start_epmd false -boot no_dot_erlang hi real 0m0.183s user 0m0.181s sys 0m0.011s This doesn't look very good, actually! FreeBSD has OTP 21-26 available through pkg, I ran the same thing on each of them several times to warm up, and get (real time): 21: 81ms 22: 81ms 23: 74ms 24: 129ms 25: 135ms 26: 177ms So maybe there's some new stuff that needs to be avoided now.
        fredrikholm 3y ago
        It was your original post that set me on course for experimenting with startup times, cheers! Unfortunately, that final flag doesn't change much on 26 for me. IIRC I dropped a bunch of flags after concluding that no shell and no epmd were the only flags that I noticed improvements with.
thibaut_barrere 3y ago
If your script needs to do things like reliable HTTP querying and JSON parsing, I would say using Elixir (once you get used to it) won’t be harmful in any way, quite the contrary. Interestingly, what drove me to Ruby initially is exactly writing scripting tasks in a C#/C++ culture where Ruby would have been considered harmful. Now Elixir can be used for the same type of tasks, but only better (eg: trivial parallelized downloads!)
- neonsunset 3y ago
  F# and C# are very different to C++ in this regard. Parallel downloads in C# boil down to using var http = new HttpClient(); var urls = new[] { "http://example.com/file1.jpg", "http://example.com/file2.jpg", /* ... */ }; var files = await Task.WhenAll( urls.Select(http.GetByteArrayAsync)); (you can replace '.GetByteArrayAsync' with .GetFromJsonAsync, .GetStreamAsync and more depending on your needs and e.g. forward the incoming data directly to files on the disk or another source)

lionkor 3y ago

You should, if you can?

adityaathalye 3y ago
Absolutely. What is life without any fun?

thibaut_barrere 3y ago

Ahaha, in a good way. Recently I have used Elixir scripts to mass-parallel-download numerous HTTP resources and do some stats on them, and it is exactly the feeling I had.

dtx1 3y ago

Suprisingly relevant https://www.youtube.com/watch?v=Ss6VBYLjMJQ https://www.youtube.com/watch?v=Ss6VBYLjMJQ

wturner 3y ago

I envision the creation of a GenServer for each and every file on your system, using this to duplicate them by the thousands, then deleting them in a timed contraction over and over and over again. I call this, "the blowcopter".

Too 3y ago

Still better than the inverse, that lots of scripters are doing; Building a helicopter out of a leaf-blower.