5 ms·

Sshell: Serverless Shell

64 points by otrack 5y ago

tobilg 5y ago
I built something similar 2.5 years ago in case you're interested: https://github.com/tobilg/lsh https://github.com/tobilg/lsh
- ignoramous 5y ago
  + dgsh [0], I wonder if one could build a serverless sh map-reduce? [0] https://news.ycombinator.com/item?id=21700014 https://news.ycombinator.com/item?id=21700014
  - tobilg 5y ago
    Yeah, that should be possible I guess. I slightly had another idea in the back of my head with LSH. E.g. see https://davidwells.io/blog/rise-of-embarrassingly-parallel-serverless-compute https://davidwells.io/blog/rise-of-embarrassingly-parallel-s...
  - alexpotato 5y ago
    If you count someone else's server as "serverless", then yes: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...
    - JasonCannon 5y ago
      >If you count someone else's server as "serverless" That's literally what serverless is. Serverless doesn't mean there are no servers. It is simply the ability to run your code without having to think about the server it is running on. More server agnostic than serverless.
- otrack 5y ago
  Very interesting work, thanks for the link! We actually missed it among the references in our paper [1] and will correct this in the journal version. Regarding the differences, lsh has an interactive mode but not sshell. On the other hand, sshell compile to native (to support massively parallel calls), and IPCs between such calls (thanks to the DSO layer). [1] https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpToTtfG/view https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpT...
Garlef 5y ago
I think the documentation should give the precise AWS permissions so that a role can be issued. Currently it only reads "# the role must have access to AWS Lambda".
- otrack 5y ago
  This is now detailed.
chespinoza 5y ago
Oops... I read this as "The Serverless hell", not joking.
- ducharmdev 5y ago
  “In the middle of the journey of our life I found myself within a dark woods where the straight way was lost.” – Dante Alighieri, Infern.io
  - chessmango 5y ago
    More upvotes here ^ Edit: And TIL infern.io is a thing too
    - chespinoza 5y ago
      what's that? I thought it'd be related with inferno OS.
      - deleted 5y ago
        [deleted]
- emptybottle 5y ago
  I mean, you're not wrong
deleted 5y ago
[deleted]
tony_cannistra 5y ago
Cool idea, but wouldn't applications of this be pretty significantly limited by lambda execution timeouts? The "process big data in the cloud" use case in the README seems like it's not really all that tenable, depending, I guess, on the structure of the data and the necessary processing steps.
- otrack 5y ago
  sshell inherits the limitations of the FaaS infrastructure, including the time limit. Regarding the big data use case, more work is necessary to assess this, but the applications we have written so far were competing both in terms of efficiency and pricing (Section 5 in [1]). [1] https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpToTtfG/view https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpT...

64 points by otrack 5y ago

I built something similar 2.5 years ago in case you're interested: https://github.com/tobilg/lsh https://github.com/tobilg/lsh

ignoramous 5y ago
+ dgsh [0], I wonder if one could build a serverless sh map-reduce? [0] https://news.ycombinator.com/item?id=21700014 https://news.ycombinator.com/item?id=21700014
- tobilg 5y ago
  Yeah, that should be possible I guess. I slightly had another idea in the back of my head with LSH. E.g. see https://davidwells.io/blog/rise-of-embarrassingly-parallel-serverless-compute https://davidwells.io/blog/rise-of-embarrassingly-parallel-s...
- alexpotato 5y ago
  If you count someone else's server as "serverless", then yes: https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...
  - JasonCannon 5y ago
    >If you count someone else's server as "serverless" That's literally what serverless is. Serverless doesn't mean there are no servers. It is simply the ability to run your code without having to think about the server it is running on. More server agnostic than serverless.
otrack 5y ago
Very interesting work, thanks for the link! We actually missed it among the references in our paper [1] and will correct this in the journal version. Regarding the differences, lsh has an interactive mode but not sshell. On the other hand, sshell compile to native (to support massively parallel calls), and IPCs between such calls (thanks to the DSO layer). [1] https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpToTtfG/view https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpT...

Garlef 5y ago

I think the documentation should give the precise AWS permissions so that a role can be issued. Currently it only reads "# the role must have access to AWS Lambda".

otrack 5y ago
This is now detailed.

chespinoza 5y ago

Oops... I read this as "The Serverless hell", not joking.

ducharmdev 5y ago
“In the middle of the journey of our life I found myself within a dark woods where the straight way was lost.” – Dante Alighieri, Infern.io
- chessmango 5y ago
  More upvotes here ^ Edit: And TIL infern.io is a thing too
  - chespinoza 5y ago
    what's that? I thought it'd be related with inferno OS.
    - deleted 5y ago
      [deleted]
emptybottle 5y ago
I mean, you're not wrong

deleted 5y ago

[deleted]

tony_cannistra 5y ago

Cool idea, but wouldn't applications of this be pretty significantly limited by lambda execution timeouts? The "process big data in the cloud" use case in the README seems like it's not really all that tenable, depending, I guess, on the structure of the data and the necessary processing steps.

otrack 5y ago
sshell inherits the limitations of the FaaS infrastructure, including the time limit. Regarding the big data use case, more work is necessary to assess this, but the applications we have written so far were competing both in terms of efficiency and pricing (Section 5 in [1]). [1] https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpToTtfG/view https://drive.google.com/file/d/1D7h0hoMep0W73XV_EdXPSEWUxpT...