15 ms·

Ruby: A great language for shell scripts

442 points by lucasoshiro 2y ago

djbusby 2y ago
These are great points, if you already have Ruby in your stack. For me, when Bash ain't enough I upgrade to the Project language (PHP, Python, JS, etc). For compiled project I reach for LSB language (Perl, Python) before introducing a new dependency.
- sestep 2y ago
  What does LSB stand for?
  - wisemang 2y ago
    Linux Standard Base. Already on the system.
  - nequo 2y ago
    I assume it means Linux Standard Base: https://refspecs.linuxbase.org/LSB_3.2.0/LSB-Languages/LSB-Languages/book1.html https://refspecs.linuxbase.org/LSB_3.2.0/LSB-Languages/LSB-L...
    - sestep 2y ago
      Thanks, I didn't know about that! I'm curious though; even LSB 5.0 only demands Python 2.4: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages/LSB-Languages/pyversion.html https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages... Do people really write scripts against Python 2 just so that they're guaranteed to be supported by LSB?
      - o11c 2y ago
        Nobody actually follows the LSB as written. But it remains useful as an idea of "stuff that was installed in old distros so usually has some newer version available". Assuming you can survive all the incompatible interpreter changes for Python etc., the main annoyance with LSB proper is shared library versions.
        nequo 2y ago
        That is one reason to use Perl I guess where you can request the feature set of a specific Perl version.
  - deleted 2y ago
    [deleted]
- cortesoft 2y ago
  Ruby is fantastic for the main project language, too! And is also standard on many Linux distros.
  - simoncion 2y ago
    Ruby's fine for small to medium-sized projects. It's pretty great for small DSLs. In my professional experience on medium to large-sized projects, its lack of explicit typing, the habit of Rubyists to use its fairly-substantial metaprogramming capabilities at the drop of a hat, and (for projects that include pieces or all of Rails) the system's implicit library loading make such projects a nightmare to reason about and maintain. I'm sure these aspects are manageable with a small group, or a very, very disciplined group, but when you have large-sized projects that have been continually worked on for ten, fifteen, more years, you're just not going to be able to guarantee that everyone involved will have the combination of knowledge and discipline require to ensure the thing remains easy to understand, extend, and maintain.
    - cortesoft 2y ago
      I work on a 12 year old Ruby project at my job, and while some of what you say is true, I still love working on it more than anything else.
      - simoncion 2y ago
        > ...while some of what you say is true... What parts of what I said is not true, and how?
    - amarshall 2y ago
      Implicit library loading isn’t even the real problem, it’s the global namespace. I think it, more than anything else, makes large projects difficult to manage due to the inability to grok the dependency graph.
      - simoncion 2y ago
- nightpool 2y ago
  Many distros come with Ruby standard, it's not very big and I think it has a lot to recommend it over Python or Perl when it comes to lightweight scripting for sysadmins and day to day automation. I wouldn't necessarily pull it into an open source project where they weren't already present, but I would definitely choose it for personal use well before Python or Perl
  - dymk 2y ago
    Ubuntu, Debian, Arch, CentOS - none of these, as far as I know, ship with a Ruby interpreter by default. I’d like to be wrong, but I don’t think many do.
    - drusepth 2y ago
      It's been a while since I reinstalled Ubuntu, but the latest version (24.04 LTS) definitely includes Ruby (3.2) according to their docs [0]. [0] https://canonical.com/blog/canonical-releases-ubuntu-24-04-noble-numbat#:~:text=Ubuntu%2024.04%20LTS%20includes%20Python,and%20Rust https://canonical.com/blog/canonical-releases-ubuntu-24-04-n....
      - mixmastamyk 2y ago
        It's in the repos, but not installed by default. Not on my Fedora box either.
        gavindean90 2y ago
        And Python 3 is everywhere
      - o11c 2y ago
        It might be installed by default from the graphical CD, but you can't assume everybody uses that, even outside of server environments.
    - bigstrat2003 2y ago
      Does Arch even ship with Python by default? Its whole thing is being minimalist by default and letting you customize what is and is not installed.
drusepth 2y ago
Ruby is an amazing language. I've seen some systems it's not already installed on and hop over to something like perl/python in those cases, but Ruby is by far my preferred hammer for small scripts. The code is beautiful. Small nit: your note in Feature 4 is actually supposed to be in Feature 5, I assume.
lambdaba 2y ago
The greatest feature for this is inline deps, something very rare to have built-in (I've only found Deno to have a similar feature): https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
- meiraleal 2y ago
  All browsers implemented it before deno with ESM and importmap.
- epage 2y ago
  Many languages have a third-party way of doing it and a few even have first-party support, see https://dbohdan.com/scripts-with-dependencies https://dbohdan.com/scripts-with-dependencies Since that document, Python PEP 723 was approved, see https://peps.python.org/pep-0723/ https://peps.python.org/pep-0723/ and https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata https://packaging.python.org/en/latest/specifications/inline... Similarly, Rust RFC 3502 was approved and an unstable implementation is available, see https://rust-lang.github.io/rfcs/3502-cargo-script.html https://rust-lang.github.io/rfcs/3502-cargo-script.html
- quechimba 2y ago
  Didn't know about this, very useful!
- pas 2y ago
  for Scala there's Ammonite which can do this https://ammonite.io/#MagicImports https://ammonite.io/#MagicImports
- corytheboyd 2y ago
  Yay I’m glad someone else knows about this and thinks it’s awesome too :) It’s crazy I didn’t find it until I was digging through bundler docs looking for something completely unrelated
- asa400 2y ago
  Elixir also has inline dependencies in scripts, via `Mix.install`: https://hexdocs.pm/mix/1.17.1/Mix.html#install/2 https://hexdocs.pm/mix/1.17.1/Mix.html#install/2. I've used it quite a bit, it works great.
codesnik 2y ago
I sometimes wonder why we don't see ruby used for shell stuff more often. It inherited most of the good stuff for shell scripting from Perl, and Perl took a lot of it's syntax from sh and sed and awk, so almost anything you can do in shell script you can do in ruby, but with an option of making it gradually less terse and more readable, while having sane variables and data handling from the start. Also ruby is great in allowing complexity to grow smoothly, no sudden hiccups. You start with just one line (everything goes into module main implicitly), extend it to a single-file script, require some built-in libraries, then add a module or helper class in the same file, and only then maybe extract those files to required files, add gems, whatever. No boilerplate whatsoever, no jumps, no big rewrites. meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why, it's not friendly for os manipulation at all, and number crunching power is not needed in many, many tasks of that sort.
- seabrookmx 2y ago
  I think golang is used because you can easily create a single static binary, which is incredibly easy to distribute. I often find non-trivial CLI tools written in Python cumbersome because of the dependency wrangling necessary.
  - np_tedious 2y ago
    Most shell-ish scripts probably use no dependencies and will not be picky about exact version
    - mmh0000 2y ago
      Mmm. I’d argue that all shell scripts use a ton of dependencies that are different across Unix/Linux. I. E. ‘sed -i’ is only in GNU sed. Same with ‘grep -P’.
      - passthejoe 2y ago
        This has bitten me many times
blahgeek 2y ago
Perl also satisfies all listed features
- Alifatisk 2y ago
  Not surprising since Ruby was inspired by Perl.
easylion 2y ago
Java for the win. Have used java for any kind of programming or running shell commands or literally anything since last 5 years professionally and personally. Language is just a tool, anything and everything works if you love it enough
- simoncion 2y ago
  In fairness, Java is still pretty slow to start. If you're looking for non-traditional "scripting" languages, you might look into Erlang. 'erl_call' is a particularly interesting little helper script that's packed into the standard system.
- cozzyd 2y ago
  java as a bash replacement is certainly a take I've never heard before...
- Alifatisk 2y ago
  Sounds like you’d be interested JRuby :)
nightpool 2y ago
Ruby is my favorite shell scripting language, I used it last year for a complex ffmpeg automation script (use blackdetect to detect long stretches of power-off and split a video file into much smaller components), and Ruby made it a breeze when I know it would have been a real struggle to get working in bash or powershell
cyclotron3k 2y ago
Totally agree! Other tricks I rely on: a) put a `binding.irb` (or `binding.pry`) in any rescue block you may have in your script - it'll allow you to jump in and see what went wrong in an interactive way. (You'll need a `require 'irb'` in your script too, ofc) b) I always use `Pathname` instead of `File` - it's part of the standard library, is a drop in replacement for `File` (and `Dir`) and generally has a much more natural API. c) Often backticks are all you need, but when you need something a little stronger (e.g. when handling filenames with spaces in them, or potentially hostile user input, etc), Ruby has a plethora of tools in its stdlib to handle any scenario. First step would be `system` which escapes inputs for you (but doesn't return stdout). d) Threads in Ruby are super easy, but using `Parallel` (not part of the stdlib) can make it even easier! A contrived example: `Parallel.map(url_list) { |url| Benchmark.measure { system('wget', url) }.real }.sum` to download a bunch of files in parallel and get the total time. MacOS has Ruby 2.6 installed by default which is perfectly serviceable, but it's EOL and there are plenty of features in 3+ that make the jump more than worthwhile.
- MatthiasPortzel 2y ago
  > You'll need a `require 'irb'` in your script too, ofc irb is a part of Ruby core, so this isn’t true. (It may have been at one point? I’m not sure.) I love binding.irb. I use it all the time.
  - delichon 2y ago
    Me too (binding.pry). I think of it as REPL based development.
  - cyclotron3k 2y ago
    Huh, TIL! Requiring `irb` has been unnecessary since 2.5 and I never noticed.
didip 2y ago
No it’s not a great language. Too many ways to do the same thing. Not packaged by default on most Linux. Monkey patching makes things even harder to debug.
- ekianjo 2y ago
  yeah. The not packaged by default is the killer issue.
  - Alifatisk 2y ago
    You are one command away from having ruby in your system, is the threshold really that high?
- dwheeler 2y ago
  Ruby is packaged for all significant Linux distros. It isn't installed by default in many, but installing it is a trivial one-time command, and it takes relatively little space.
- x3n0ph3n3 2y ago
  Monkey patching isn't the norm anymore and not really a valid complaint in this decade.
  - manume 2y ago
    Hearing people still mention "monkey patching" always makes me chuckle... I haven't "monkey patched" anything in Ruby in > 5 years, and I don't see it in any of the popular libraries/gems anymore either.
    - p_l 2y ago
      Especially when the big difference is that Ruby had proper OO system that allowed patching, when necessary, to be done in much saner way... whereas the origin of monkey patching seems to by Python with its totally broken magical method names and kitbashed object model.
dcchambers 2y ago
I work for a company that has a large Rails monolith. Although we use many more languages than just ruby these days, we still have a ton of scripting, config, and tooling that is all in Ruby. It's a joy to work with IMO. Another common pattern I see is people using embedded ruby in a shell script. It does make it a little harder to read/understand at a glance, but it's nice for being able to do most simple things in sh but drop into ruby for things that sh/bash suck at. That said, I get a feeling that the people that joined once we'd added stuff outside of the Rails monolith and don't know/use Ruby are...not big fans.
- mberning 2y ago
  I had the same experience. Somebody in our company inherited a Ruby script and was trying to modify it and was stuck. They came to me exasperated. The error message was something really trivial like addition is not defined for some object type. If you don’t understand the base level concepts of the language it’s going to be a very bad time. Sadly people are not that interested in learning about Ruby nowadays and look at it as a huge imposition to deal with. I love it still.
  - dcchambers 2y ago
    Yup that tracks with my experience. Ruby is a wonderful language worth learning (and it's really not difficult to pick up) but I see way more people push against learning it than they do other things (eg python).
Demiurge 2y ago
If Rails could be written in Python, I would probably prefer that. Ruby has too much of Perl. If I didn't choose to write a critical piece of software in Perl, which caused me numerous sleepless nights hunting for a missing quote or other weird character, I might have thought it was cool. By my experience with PHP and Perl is why I prefer Python. Clever is not what I ever want a language to be. Python > Lua > PHP > Ruby > JavaScript > Perl > Bash
- wkirby 2y ago
  Honestly same, but only because Python has types relatively sorted. While I actually enjoy the language, Ruby is… struggling… and after supporting multiple Rails apps in the wild for close to a decade, what me and my team miss the most is the ability to hint types easily.
- lucasoshiro 2y ago
  > Python > Lua > PHP > Ruby > JavaScript > Perl > Bash Those are different languages, with different characteristics. Perhaps the only two that can be directly compared as a 100% replacement to another are Ruby and Python.
derefr 2y ago
I love using Ruby for shell scripting, but there are also a ton of little nits I have to fix whenever I'm doing it. For example: Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception", ala bash `set -e`. So in many of my Ruby scripts there lives this little helper: def system!(*args, **kwargs) r = system(*args, **kwargs) fail "subprocess failed" unless $?.success? r end And I can't ask "is this command installed" in an efficient built-in way, so I end up throwing this one in frequently too (in this instance, whimsically attached to the metaclass of the ENV object): class << ENV def path @path ||= self['PATH'].split(':').map{ |d| Pathname.new(d) } end def which(cmd) cmd = cmd.to_s self.path.lazy.map{ |d| d + cmd }.find{ |e| e.file? && e.executable? } end end I have other little snippets like this for: • implicitly logging process-spawns (ala bash `set -x`) • High-level wrapper methods like `Pathname#readable_when_elevated?` that run elevated through IO.popen(['sudo', ...]) — the same way you'd use `sudo` in a bash script for least-privilege purposes • Recursive path helpers, e.g. a `Pathname#collapse_tree` method that recursively deletes empty subdirectories, with the option to consider directories that only contain OS errata files like `.DS_Store` "empty" (in other words, what you'd get back from of a git checkout, if you checked in the directory with a sensible .gitignore in play) ...and so forth. It really does end up adding up, to the point that I feel like what I really want is a Ruby-like language or a Ruby-based standalone DSL processor that's been optimized for sysadmin tasks.
- copirate 2y ago
  > Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception" Since Ruby 2.6 you can pass `exception: true` to `system` to make it behave like your `system!`. https://rubyreferences.github.io/rubychanges/2.6.html#system-has-exception-argument https://rubyreferences.github.io/rubychanges/2.6.html#system...
  - derefr 2y ago
    Didn't realize that! That's one snippet I can maybe eliminate now. (As to why I didn't know: the first thing in the RDoc for Kernel#system is still "see the docs for Kernel#spawn for options" — and then Kernel#spawn doesn't actually have that one, because it doesn't block until the process quits, and so returns you a pid, not a Process::Status. I stopped looking at the docs for Kernel#system itself a long time ago, just jumping directly to Kernel#spawn...) But come to think of it, if Kernel#system is just doing a blocking version of Kernel#spawn → Process#wait, then shouldn't Process#wait also take an exception: kwarg now? And also-also, sadly IO.popen doesn't take this kwarg. (And IO.popen is what I'm actually using most of the time. The system! function above is greatly simplified from the version of the snippet I actually use these days — which involves a DSL for hierarchical serial task execution that logs steps with nesting, and reflects command output from an isolated PTY.)
kazinator 2y ago
It seems like a waste of precious syntax to dedicate backticks to running shell commands. What's between the backticks is not even portable; the commands rely on an operating-system-specific command interpreter. > puts `ls`.lines.map { |name| name.strip.length } # prints the lengths of the filenames Fantastic example, except for the commandment violation: "thou shalt not parse the output of 'ls'"! You really want to read the directory and map over the stat function (or equivalent) to get the length. 2> (flow "." open-directory get-lines (mapcar [chain stat .size])) (727 2466 21 4096 643 16612 5724 163 707 319 352135 140 51 0 4096 114898 1172428 1328258 4096 4096 4096 29953 4096 4096 0 27 4096 4096 35585 8450 968 40960 14610 4096 14 755128 1325258 4096 17283 218 471 104 4096 99707 1255 4096 129 4096 721 9625 401 15658 4096 235 98 1861 664 23944 4286 4096 1024 0)
- Too 2y ago
  They have inherited the second biggest mistake of shellscripts; Requiring the user to manually check $? after each command. No thanks. Anything that doesn't have error handling enabled by default goes straight in the trash bin.
  - nomilk 2y ago
    I couldn't spot any way to make something like `ls -j` (j is an illegal option) throw an exception in ruby (as opposed to simply outputting the system error message). The closest I could find is what you suggest (checking $?), or using something like this [1], which would require changing syntax: system('ls -j', exception: true) Would be great to know if there's some easy callback or, ideally, a global setting one can make so a ruby exception is thrown if there's an error running system commands using the backtick syntax. [1] https://stackoverflow.com/a/54132729/5783745 https://stackoverflow.com/a/54132729/5783745
  - tasuki 2y ago
    It's not required to check $? after each command. It's only when calling out to the shell. How would you propose to handle that? Throw an exception on a shell command which fails? Some would say calling out to shell is an anti pattern by itself. Others would say exceptions are an anti pattern. (Just use appropriate return type and there's no need for exceptions ever!)
rednafi 2y ago
Ruby is slow and encourages an esoteric convention over configuration style of OO code. If you enjoy it, more power to you. However, Python is everyone's second favorite or least favorite language, and it runs laps around Ruby any day. Then there's Go if you need some extra oomph!
- jakogut 2y ago
  Go is a good language for when I want binaries an order of magnitude larger than necessary.
  - simoncion 2y ago
    Oh but golang's "deployment story" is SO SIMPLE! /s
    - notnmeyer 2y ago
      isn’t it though?
      - simoncion 2y ago
        I was arguing with myself about adding additional context. It looks like I should have put it in. In my professional experience, Go cannot consistently avoid producing binaries that fail when run on an incompatibly-different version of glibc than they were build and linked on. Also. for most deployment situations, Go's "single, enormous binary" doesn't matter. * In The Cloud(TM), you have full control over what you deploy. So, so, so often you have a custom VM or Docker image that you just squirt out there and automation to keep it updated and rebuilt, which makes deployment just trivial. * On Windows, you have Windows Installer, which you can instruct to check for and install any prereqs you require. (It's incredible how all of the pieces to build a proper package manager have been in Windows since the early 2000s, and yet no package manager came out of MSFT.) * On OSX, you either use the App Store or one of the handful of package managers... but I expect nearly zero non-Power-Users use the package managers. I guess there are also the nutballs who do 'curl $INSTALL_SCRIPT | sudo bash'. * On Linux... well nearly zero non-Power-Users run desktop Linux, and those that do likely already have Ruby, Python, Perl, and GCC installed.
adamnemecek 2y ago
I’ve been enjoying nushell
- __MatrixMan__ 2y ago
  Nushell is fantastic. Being able to just paste json into a terminal and immediately start working with it in a structured way feels like a superpower. The interactive help and well-thought-out error messages feel like a hug.
lr4444lr 2y ago
Not denying Ruby is good, but other than process forking, why should I prefer it to Python?
- katbyte 2y ago
  Not using white space as a delimiter
  - ianschmitz 2y ago
    Ruby is surprisingly picky with white space
    - manume 2y ago
      Can you give an example? I can't think of a single situation where whitespace matters in Ruby (unless of course you forget to put a space between two commands or something silly).
      - codesnik 2y ago
        if foo is a method then `foo + bar` and `foo+bar` are `foo()+bar`, but `foo +bar` is `foo(+bar)` ternary ? : also has some interesting whitespace dependent mixups with symbols, but I cannot remember what. I think that parser has many gotchas like that, but they are really really rare to bite you, because ruby's magic follows human intuition as much as possible.
        p_l 2y ago
        still less annoying than Python's semantic whitespace
      - ezrast 2y ago
        It's not really a problem in practice (and I love Ruby), but it's still wild to me that they made the parser do this: $ irb irb(main):001:0> def foo(x=70) = x => :foo irb(main):002:0> i = 2 => 2 irb(main):003:0> foo / 5/i => 7 irb(main):004:0> foo /5/i => /5/i
phendrenad2 2y ago
Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box. Are you going to add 5 minutes to your docker build to compile Ruby? Probably not. Luckily, I've found that Perl has most of the best features of Ruby, and it's installed everywhere. It's time to Make Perl Great Again.
- bigstrat2003 2y ago
  Why on earth would you add the compilation of a Ruby interpreter to your docker build? Just install it through the package manager of whatever distro your image is built on.
- manume 2y ago
  > Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box. Please name those 10 environments you are talking about. In my experience, a reasonably recent Ruby version is present almost everywhere. > add 5 minutes to your docker build Why on earth would it take 5 minutes to install anything? If you install Ruby through a package manager (it's present in pretty much all of them: https://www.ruby-lang.org/en/documentation/installation/#package-management-systems https://www.ruby-lang.org/en/documentation/installation/#pac...) it takes only seconds.
greenthrow 2y ago
By definition a shell script is one that is run by the shell. The term should not be overloaded to include scripts that require another interpreter.
- lucasoshiro 2y ago
  What do you suggest for the title?
samatman 2y ago
The things which make Ruby good for shell scripts are, to a large degree, things it inherited from Perl. Which was, and is, a great language for scripting. People use it a lot less these days, for a lot of reasons, some better than others. I myself do simple stuff in bash, and pull out some Python for more complex scripting. My Perl chops have withered, last time I was paid to use it was 21 years ago, but it really is a great scripting language, and you'll find it installed on a great deal more systems than Ruby. One of these days I'll give Raku a spin, just for old time's sake.
corytheboyd 2y ago
Hell yeah! I’ll never forget being fooled by HN that you have to use Perl if you want portable scripts that aren’t bash, writing a whole script with it, and having a coworker politely, yet firmly, tell me that I am dumb and it should just be Ruby… and it was a script I was checking into a Rails app! It’s even trivial to include dependencies with bundler inline https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
- notnmeyer 2y ago
  this is… somewhat horrific, right? would anyone do this without explicit version pinning? loosely pinned deps installed on execution sounds fucking awful.
  - deleted 2y ago
    [deleted]
  - corytheboyd 2y ago
    Well, don’t use this then. It’s a neat feature for quickly iterating, and you can quite easily go up from here to a proper Gemfile/Gemfile.lock.
  - manume 2y ago
    You can pin a gem to a specific version, of course. `gem "mygem"` installs the latest version. `gem "mygem", "~> 4.0.0"` installs >= 4.0.0 but < 4.1.0, which is what you probably want when using Semantic Versioning, which most gems adhere to, to get the latest patch version. `gem "mygem", "4.0.10"` installs exactly that version.
JohnMakin 2y ago
As a self proclaimed shell advocate I am… intrigued. As a self proclaimed shell advocate, I also have a revulsion to back ticks.
- justin_oaks 2y ago
  Yes, all hail $( ) I haven't used backticks in my shell scripts in years.
- pikelet 2y ago
  You can also use %x(ls some/dir), which is a syntax used in other places too, like %w[word word word] (array of word strings, notice you can choose the delimiter).
- lucasoshiro 2y ago
  > I also have a revulsion to back ticks. Yeah, I understand... But this is other language with a similar syntax for a similar functionality.
JohnMakin 2y ago
To simulate “types” in complex shell scripting I typically involve a lot of json objects acting as my data structures and the file system for a rudimentary database. They’re horrifying ugly, but they tend to work pretty reliably. This is probably easier.
- kchr 2y ago
  The `ip` network utility now supports JSON ouput via the `ip --json` invocation since a while back. I would love to see more tools implementing this as an option!
xgdgsc 2y ago
Or use Julia for scripts https://github.com/ninjaaron/administrative-scripting-with-julia https://github.com/ninjaaron/administrative-scripting-with-j...
- Alifatisk 2y ago
  There’s a section in the readme explaining why it’s a bad idea haha
bdcravens 2y ago
I spend most of my time writing Rails or other backend Ruby, and I prefer my system-level scripts in bash. Philosophically I don't want to have to manage dependencies or broken gems (though inline deps obviate that, and it's not like I've never had to wrestle with apt)
- manume 2y ago
  What do you mean by "broken gems"?
  - bdcravens 2y ago
    Either dependency issues with other gems, or gems that break due to some sort of library in the OS. If you pin all of your versions, and use it in one place, that's less of an issue, but many scripts are designed to have some level of portability (even if it's to a new instance of the server)
    - manume 2y ago
      In my experience, Bundler has improved a lot with regard to resolving dependency issues over the years. And OS libs are only really depended on by a few gems, no? 99% of them don't use FFI or call OS libs. Moreover, how often do you really move a script to a completely different OS, where you don't know which OS libs are installed? And wouldn't those missing OS libs also be a problem when writing the script in Bash or any other language?
- lucasoshiro 2y ago
  Well, but what you do with Bash can be also done with vanilla Ruby...
pipeline_peak 2y ago
Being able to call external commands with backticks alone makes it better suited than Python for shell scripting.
- __MatrixMan__ 2y ago
  If you've reached for something with more structure than the string-soup that is bash, why would you then embrace the string-soup that is invoking subprocesses with backticks? Involving an entirely separate parser just to split your args up feels like a footgun to me. Python's `sh` handles this rather nicely: http://sh.readthedocs.io/en/latest/ http://sh.readthedocs.io/en/latest/
  - deleted 2y ago
    [deleted]
- frou_dh 2y ago
  IMO that feature snatches defeat from the jaws of victory because if the command fails (exit code >0) it doesn't raise an exception. Same issue with the system(...) method by default.
corytheboyd 2y ago
Overall a nice lite write up! Bash is great, but it occasionally becomes untenable, usually around the time where HTTP and whatnot becomes involved. Same goes for shell exec exit codes, you can use an API like popen3 for this: https://ruby-doc.org/stdlib-2.4.1/libdoc/open3/rdoc/Open3.html#method-c-popen3 https://ruby-doc.org/stdlib-2.4.1/libdoc/open3/rdoc/Open3.ht... You mention using threads and regex match global variables in the same write up. Please use the regex match method response instead of the $1 variables to save yourself the potential awful debugging session. It even lets you access named capture groups in the match response using the already familiar Hash access API. Example: https://stackoverflow.com/a/18825787 https://stackoverflow.com/a/18825787 In general, just don’t use global variables in Ruby. It’s already SO easy to move “but it has to be global” functionality to static class methods or constants that I’ve encountered exactly zero cases where I have NEEDED global variables. Even if you need a “stateful constant” Ruby had a very convenient Singleton mixin that provides for a quick and easy solution. Besides, if you actually WERE to take advantage of them being global VARIABLES (reassigning the value) I would confidently bet that your downstream code would break, because I’m guessing said downstream code assumes the global value is constant. Just avoid them, there’s no point, use constants. This applies to any language TBH, but here we’re talking about Ruby :)
- lucasoshiro 2y ago
  Thanks for the comment! Even though I wrote this about Ruby, I must admit that I am not a specialist.
  - corytheboyd 2y ago
    No problem at all, and you don’t need to be a specialist to be excited and write something up :) I hope this didn’t come across as judgy, I’ve just lived through many a Ruby bug heh
xavdid 2y ago
Ruby's a great language- I've always enjoyed its ergonomics and clarity. But its editor tooling hasn't kept up with its one-time competitor, Python. I've mostly been in the Python ecosystem for the past few years and the LSP investment from Microsoft has really shown. Rich Python support in VSCode is seamless and simple. Coming back to Ruby after that caught me off guard - it feels like I'm writing syntax-highlighted plain text. There's an LSP extension from Shopify, but it's temperamental and I have trouble getting it working. Editor support isn't everything (the actual language design is still the most important), but it definitely affects how eager I am to use it. I basically never choose Ruby over Python given the option, which is too bad. Ruby's a cool language!
- anothername12 2y ago
  The “editor tooling” in Jetbrains’ Rubymine is fabulous.
  - xavdid 2y ago
    That's true, I've heard good things! I don't use Jetbrains products, but I'm glad they work well here.
    - anothername12 2y ago
      Yeah I think they have some special sauce that puts them a bit ahead of what LSP-based. Definitely check it out
- rafamvc 2y ago
  It got a lot better recently. You should take a second look.
  - xavdid 2y ago
    I mean, I was trying to set up editor support for a Ruby script last week. So unless it was improved really recently, it's not where I'd like it to be.
nomilk 2y ago
If you never coded in ruby before, but use macOS, you already have ruby installed. Just open terminal and type irb to bring up the ruby interpreter and try out the code in the article.
- anothername12 2y ago
  I think the built in one is deprecated though right?
  - Neywiny 2y ago
    Are there distributions where this doesn't happen? I feel like I'm often installing or compiling new versions of packages and languages because they're outdated on Ubuntu
    - medstrom 2y ago
      A rolling distro like Arch or Void.
      - Neywiny 2y ago
        Gotcha, thanks
- shagie 2y ago
  ~ % irb WARNING: This version of ruby is included in macOS for compatibility with legacy software. In future versions of macOS the ruby runtime will not be available by default, and may require you to install an additional package. irb(main):001:0>
kajika91 2y ago
No pipe (and I mean in parallel too) no love for me. Nice calling syntax though.
- copirate 2y ago
  You can pipe with the `pipeline*` method of open3 which is part of the stdlib: For example: require "open3" last_stdout, wait_threads = Open3.pipeline_r("cat /etc/passwd", ["grep", "root"]) last_stdout.read # => "root:x:0:0::/root:/bin/bash\n" wait_threads.map(&:value).map(&:success?) # => [true, true] https://ruby-doc.org/3.2.2/stdlibs/open3/Open3.html https://ruby-doc.org/3.2.2/stdlibs/open3/Open3.html
  - e12e 2y ago
    Can you easily chain these, though? (gzcat some.txt|grep foo|sort -u|head -10 etc?). Especially lazily, if the uncompressed stream is of modest size, like a couple of gigabytes?
    - RulerOf 2y ago
      I'd suspect you could do that with Open3, but if you are, why not just read the file and process with Ruby instead?
      - e12e 2y ago
        I'm currently working with 150MB worth of gzipped JSON - marshalling the full file from JSON to ruby hash eats up a lot of memory. One tweak that allows for easier lazy iteration over the file (while keeping temporary disk Io reasonable) is to pipe it through zcat, jq in stream mode to convert to ndjson, gzip again - for a temp file that ruby zlib can wrap for a stream convenient for lazy iteration per read_line...). Generally marshalling a gig or more of JSON (non-lazily) takes a lot of resources in ruby.
        RulerOf 2y ago
oliviergg 2y ago
For me, the problem with shell scripting is that I do it only occasionally. I’ve always been tempted by higher-level languages. The arrival of tools like ChatGPT has made me much more comfortable writing scripts of intermediate complexity. I find shell scripting more interesting now.
- manume 2y ago
  Similar case for me, but I think the author says it well: > That is, most of the cases Bash for me is enough, but if the script starts to become complex, I switch to Ruby. Even if ChatGPT lets you bang out more complex shell scripts easily, if you have to come back to it later on to fix an error or add a new feature, it's really hard to understand it (if you don't deal with such scripts on a daily basis). If you start with Ruby (or Python or similar) from the beginning, it's much easier to understand and extend later on.
fire_lake 2y ago
Nothing about dependencies! A scripting language needs a way to declare dependencies in a locked-down way, inside of the script that requires them. They must be portable across platforms.
- manume 2y ago
  Bundler inline works great for that: https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
aorth 2y ago
Side note: Firefox is offering to translate this blog post from Portuguese for me though the content is clearly in English. I noticed the `<html>` element has a `lang="pt"` attribute. The site is generated by Jekyll, which I have not used in years, so I'm wondering if this is a site-level setting or could be overridden in frontmatter...
- lucasoshiro 2y ago
  Thanks for the feedback! I need to check it because my mother language is Portuguese and my personal page was initially only in Portuguese (everything there has also a version in Portuguese if you click the Brazilian flag). Some years ago I started to write in English but I didn't want to delete my older posts.
hiAndrewQuinn 2y ago
I have what probably sounds like a niche use case, where most of the boxes I work on don't have access to the Internet. So for me, "is it installed in the base distribution" is the difference between being able to start immediately no matter which box I'm concerned with, and spending months trying to upstream a new program to be installed with our OS image team. I took a look around a vanilla Debian 12 box, and didn't see Ruby in there [1]. So, sadly, although I really like the way Ruby looks, I'm going to have to stick with Bash and Python 3 for the hard stuff. [1]: https://hiandrewquinn.github.io/til-site/posts/what-programming-languages-come-out-of-the-box-on-debian-12/ https://hiandrewquinn.github.io/til-site/posts/what-programm...
- lucasoshiro 2y ago
  Yeah, it makes sense.
floppy-disk 2y ago
While I write most of my scripts in Ruby and enjoy doing so, there is one gripe I have with it: its slow start-up time. On my machine, running an empty Ruby script takes about 100ms, compared to <10ms for Python, Perl, Lua, Bash. One can mitigate the problem somewhat using the `--disable-gems` flag, but that's not a good general solution.
- Borg3 2y ago
  100ms !!!! wtf? :) [borg@cube] time ruby -e 'nil' real 0m0.007s
  - floppy-disk 2y ago
    Whoa, interesting... What version are you running? Here's my system: $ ruby -v ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux] $ time ruby -e '' real 0m0.122s user 0m0.102s sys 0m0.020s I found an old Reddit thread also hinting at bad start-up times: https://old.reddit.com/r/ruby/comments/aqxepw/rubys_startup_time_seems_to_get_worse https://old.reddit.com/r/ruby/comments/aqxepw/rubys_startup_....
    - Borg3 2y ago
      Very old one :) Because its reliable. Version 1.8.7 here. I want controllable delays. I want controllable threads :) I want portability. Of course, if you need more peformance, you need to go for newer stuff.
      - Alifatisk 2y ago
        I’ll try this out sometime, but probably through docker https://hub.docker.com/r/hublogix/minimal-ruby https://hub.docker.com/r/hublogix/minimal-ruby
        Borg3 2y ago
        Yeah.. old ruby have interesting build system. It first build miniruby with is single selfcontained binary having all core ruby functionality. Then, it is used to run some more .rb build scripts to finaly make fully functional ruby. Actyally that miniruby is pretty usefull. For basic stuff. Also, its very easy to build static ruby containing all extra stuff you care about, like Win32 API for example. I myself have custom ruby binary on Win32 (Cygwin) with GRX library added in, So I can do basic graph stuff directly from ruby :) That stuff is written in it: http://borg.uu3.net/~borg/?gperf http://borg.uu3.net/~borg/?gperf
deleted 2y ago
[deleted]
bingemaker 2y ago
My first application of Ruby was to use it for shell auto-completions. I'm so grateful that I learnt Ruby first, and then Rails. Ruby is a great language to get some utility working out real fast. Rails is great for MVP. I fail to understand why people bitch about Ruby/Rails by comparing them to other languages/frameworks.
- deleted 2y ago
  [deleted]
- lucasoshiro 2y ago
  Yeah, as I mentioned, every time someone says something bad about Ruby for me, it is about Rails...
helpfulContrib 2y ago
[dead]
globular-toast 2y ago
Cool, I'm sold, added Ruby to my to learn list! I use Python a lot but I don't think it's as good for writing scripts as bash or Perl. Ruby looks like it fits that "better but not much harder" category much better.
- lucasoshiro 2y ago
  Well, I'm happy that I you learnt something new here :-). Ruby is quite similar to Python, perhaps you don't even need to study too much!
stefantalpalaru 2y ago
[dead]
softwaredoug 2y ago
Shoutout to Hannes Moser at Shopify and our regular pairing on the Disco CLI tool that orchestrated a lot of Elasticsearch config stuff. :) As a Python guy I found the setup for this sort of CLI too really refreshing!
nunez 2y ago
I agree. Ruby is a _fantastic_ language for getting things done quickly whose credibility was unfairly maligned by Rails. Unbelievably easy to read, and, with rspec, it is stupid easy to write tests for. No need to fuss with interfaces like you do with Golang; yes, that is the right thing to do, but when you need to ship _now_, it becomes a pain and generates serious boilerplate quickly. I've switched to Golang for most things these days, as it is a much safer language overall, but when shell scripts get too hard, Ruby's a great language to turn to.
- Alifatisk 2y ago
  Rspec changed my view on writing tests, it’s way to easy now!
0mwh 2y ago
I've been working (intermittently) on a project to do exactly this: https://github.com/0mWh/ruby-linux-initrd https://github.com/0mWh/ruby-linux-initrd
deleted 2y ago
[deleted]
Alifatisk 2y ago
I wonder why this post blew up so much

442 points by lucasoshiro 2y ago

These are great points, if you already have Ruby in your stack. For me, when Bash ain't enough I upgrade to the Project language (PHP, Python, JS, etc). For compiled project I reach for LSB language (Perl, Python) before introducing a new dependency.

sestep 2y ago
What does LSB stand for?
- wisemang 2y ago
  Linux Standard Base. Already on the system.
- nequo 2y ago
  I assume it means Linux Standard Base: https://refspecs.linuxbase.org/LSB_3.2.0/LSB-Languages/LSB-Languages/book1.html https://refspecs.linuxbase.org/LSB_3.2.0/LSB-Languages/LSB-L...
  - sestep 2y ago
    Thanks, I didn't know about that! I'm curious though; even LSB 5.0 only demands Python 2.4: https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages/LSB-Languages/pyversion.html https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages... Do people really write scripts against Python 2 just so that they're guaranteed to be supported by LSB?
    - o11c 2y ago
      Nobody actually follows the LSB as written. But it remains useful as an idea of "stuff that was installed in old distros so usually has some newer version available". Assuming you can survive all the incompatible interpreter changes for Python etc., the main annoyance with LSB proper is shared library versions.
      - nequo 2y ago
        That is one reason to use Perl I guess where you can request the feature set of a specific Perl version.
- deleted 2y ago
  [deleted]
cortesoft 2y ago
Ruby is fantastic for the main project language, too! And is also standard on many Linux distros.
- simoncion 2y ago
  Ruby's fine for small to medium-sized projects. It's pretty great for small DSLs. In my professional experience on medium to large-sized projects, its lack of explicit typing, the habit of Rubyists to use its fairly-substantial metaprogramming capabilities at the drop of a hat, and (for projects that include pieces or all of Rails) the system's implicit library loading make such projects a nightmare to reason about and maintain. I'm sure these aspects are manageable with a small group, or a very, very disciplined group, but when you have large-sized projects that have been continually worked on for ten, fifteen, more years, you're just not going to be able to guarantee that everyone involved will have the combination of knowledge and discipline require to ensure the thing remains easy to understand, extend, and maintain.
  - cortesoft 2y ago
    I work on a 12 year old Ruby project at my job, and while some of what you say is true, I still love working on it more than anything else.
    - simoncion 2y ago
      > ...while some of what you say is true... What parts of what I said is not true, and how?
  - amarshall 2y ago
    Implicit library loading isn’t even the real problem, it’s the global namespace. I think it, more than anything else, makes large projects difficult to manage due to the inability to grok the dependency graph.
    - simoncion 2y ago
nightpool 2y ago
Many distros come with Ruby standard, it's not very big and I think it has a lot to recommend it over Python or Perl when it comes to lightweight scripting for sysadmins and day to day automation. I wouldn't necessarily pull it into an open source project where they weren't already present, but I would definitely choose it for personal use well before Python or Perl
- dymk 2y ago
  Ubuntu, Debian, Arch, CentOS - none of these, as far as I know, ship with a Ruby interpreter by default. I’d like to be wrong, but I don’t think many do.
  - drusepth 2y ago
    It's been a while since I reinstalled Ubuntu, but the latest version (24.04 LTS) definitely includes Ruby (3.2) according to their docs [0]. [0] https://canonical.com/blog/canonical-releases-ubuntu-24-04-noble-numbat#:~:text=Ubuntu%2024.04%20LTS%20includes%20Python,and%20Rust https://canonical.com/blog/canonical-releases-ubuntu-24-04-n....
    - mixmastamyk 2y ago
      It's in the repos, but not installed by default. Not on my Fedora box either.
      - gavindean90 2y ago
        And Python 3 is everywhere
    - o11c 2y ago
      It might be installed by default from the graphical CD, but you can't assume everybody uses that, even outside of server environments.
  - bigstrat2003 2y ago
    Does Arch even ship with Python by default? Its whole thing is being minimalist by default and letting you customize what is and is not installed.

drusepth 2y ago

Ruby is an amazing language. I've seen some systems it's not already installed on and hop over to something like perl/python in those cases, but Ruby is by far my preferred hammer for small scripts. The code is beautiful. Small nit: your note in Feature 4 is actually supposed to be in Feature 5, I assume.

lambdaba 2y ago

The greatest feature for this is inline deps, something very rare to have built-in (I've only found Deno to have a similar feature): https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...

meiraleal 2y ago
All browsers implemented it before deno with ESM and importmap.
epage 2y ago
Many languages have a third-party way of doing it and a few even have first-party support, see https://dbohdan.com/scripts-with-dependencies https://dbohdan.com/scripts-with-dependencies Since that document, Python PEP 723 was approved, see https://peps.python.org/pep-0723/ https://peps.python.org/pep-0723/ and https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata https://packaging.python.org/en/latest/specifications/inline... Similarly, Rust RFC 3502 was approved and an unstable implementation is available, see https://rust-lang.github.io/rfcs/3502-cargo-script.html https://rust-lang.github.io/rfcs/3502-cargo-script.html
quechimba 2y ago
Didn't know about this, very useful!
pas 2y ago
for Scala there's Ammonite which can do this https://ammonite.io/#MagicImports https://ammonite.io/#MagicImports
corytheboyd 2y ago
Yay I’m glad someone else knows about this and thinks it’s awesome too :) It’s crazy I didn’t find it until I was digging through bundler docs looking for something completely unrelated
asa400 2y ago
Elixir also has inline dependencies in scripts, via `Mix.install`: https://hexdocs.pm/mix/1.17.1/Mix.html#install/2 https://hexdocs.pm/mix/1.17.1/Mix.html#install/2. I've used it quite a bit, it works great.

codesnik 2y ago

I sometimes wonder why we don't see ruby used for shell stuff more often. It inherited most of the good stuff for shell scripting from Perl, and Perl took a lot of it's syntax from sh and sed and awk, so almost anything you can do in shell script you can do in ruby, but with an option of making it gradually less terse and more readable, while having sane variables and data handling from the start. Also ruby is great in allowing complexity to grow smoothly, no sudden hiccups. You start with just one line (everything goes into module main implicitly), extend it to a single-file script, require some built-in libraries, then add a module or helper class in the same file, and only then maybe extract those files to required files, add gems, whatever. No boilerplate whatsoever, no jumps, no big rewrites. meanwhile, a lot of tooling nowadays is written in Go, and I have no idea why, it's not friendly for os manipulation at all, and number crunching power is not needed in many, many tasks of that sort.

seabrookmx 2y ago
I think golang is used because you can easily create a single static binary, which is incredibly easy to distribute. I often find non-trivial CLI tools written in Python cumbersome because of the dependency wrangling necessary.
- np_tedious 2y ago
  Most shell-ish scripts probably use no dependencies and will not be picky about exact version
  - mmh0000 2y ago
    Mmm. I’d argue that all shell scripts use a ton of dependencies that are different across Unix/Linux. I. E. ‘sed -i’ is only in GNU sed. Same with ‘grep -P’.
    - passthejoe 2y ago
      This has bitten me many times

blahgeek 2y ago

Perl also satisfies all listed features

Alifatisk 2y ago
Not surprising since Ruby was inspired by Perl.

easylion 2y ago

Java for the win. Have used java for any kind of programming or running shell commands or literally anything since last 5 years professionally and personally. Language is just a tool, anything and everything works if you love it enough

simoncion 2y ago
In fairness, Java is still pretty slow to start. If you're looking for non-traditional "scripting" languages, you might look into Erlang. 'erl_call' is a particularly interesting little helper script that's packed into the standard system.
cozzyd 2y ago
java as a bash replacement is certainly a take I've never heard before...
Alifatisk 2y ago
Sounds like you’d be interested JRuby :)

nightpool 2y ago

Ruby is my favorite shell scripting language, I used it last year for a complex ffmpeg automation script (use blackdetect to detect long stretches of power-off and split a video file into much smaller components), and Ruby made it a breeze when I know it would have been a real struggle to get working in bash or powershell

cyclotron3k 2y ago

Totally agree! Other tricks I rely on: a) put a `binding.irb` (or `binding.pry`) in any rescue block you may have in your script - it'll allow you to jump in and see what went wrong in an interactive way. (You'll need a `require 'irb'` in your script too, ofc) b) I always use `Pathname` instead of `File` - it's part of the standard library, is a drop in replacement for `File` (and `Dir`) and generally has a much more natural API. c) Often backticks are all you need, but when you need something a little stronger (e.g. when handling filenames with spaces in them, or potentially hostile user input, etc), Ruby has a plethora of tools in its stdlib to handle any scenario. First step would be `system` which escapes inputs for you (but doesn't return stdout). d) Threads in Ruby are super easy, but using `Parallel` (not part of the stdlib) can make it even easier! A contrived example: `Parallel.map(url_list) { |url| Benchmark.measure { system('wget', url) }.real }.sum` to download a bunch of files in parallel and get the total time. MacOS has Ruby 2.6 installed by default which is perfectly serviceable, but it's EOL and there are plenty of features in 3+ that make the jump more than worthwhile.

MatthiasPortzel 2y ago
> You'll need a `require 'irb'` in your script too, ofc irb is a part of Ruby core, so this isn’t true. (It may have been at one point? I’m not sure.) I love binding.irb. I use it all the time.
- delichon 2y ago
  Me too (binding.pry). I think of it as REPL based development.
- cyclotron3k 2y ago
  Huh, TIL! Requiring `irb` has been unnecessary since 2.5 and I never noticed.

didip 2y ago

No it’s not a great language. Too many ways to do the same thing. Not packaged by default on most Linux. Monkey patching makes things even harder to debug.

ekianjo 2y ago
yeah. The not packaged by default is the killer issue.
- Alifatisk 2y ago
  You are one command away from having ruby in your system, is the threshold really that high?
dwheeler 2y ago
Ruby is packaged for all significant Linux distros. It isn't installed by default in many, but installing it is a trivial one-time command, and it takes relatively little space.
x3n0ph3n3 2y ago
Monkey patching isn't the norm anymore and not really a valid complaint in this decade.
- manume 2y ago
  Hearing people still mention "monkey patching" always makes me chuckle... I haven't "monkey patched" anything in Ruby in > 5 years, and I don't see it in any of the popular libraries/gems anymore either.
  - p_l 2y ago
    Especially when the big difference is that Ruby had proper OO system that allowed patching, when necessary, to be done in much saner way... whereas the origin of monkey patching seems to by Python with its totally broken magical method names and kitbashed object model.

dcchambers 2y ago

I work for a company that has a large Rails monolith. Although we use many more languages than just ruby these days, we still have a ton of scripting, config, and tooling that is all in Ruby. It's a joy to work with IMO. Another common pattern I see is people using embedded ruby in a shell script. It does make it a little harder to read/understand at a glance, but it's nice for being able to do most simple things in sh but drop into ruby for things that sh/bash suck at. That said, I get a feeling that the people that joined once we'd added stuff outside of the Rails monolith and don't know/use Ruby are...not big fans.

mberning 2y ago
I had the same experience. Somebody in our company inherited a Ruby script and was trying to modify it and was stuck. They came to me exasperated. The error message was something really trivial like addition is not defined for some object type. If you don’t understand the base level concepts of the language it’s going to be a very bad time. Sadly people are not that interested in learning about Ruby nowadays and look at it as a huge imposition to deal with. I love it still.
- dcchambers 2y ago
  Yup that tracks with my experience. Ruby is a wonderful language worth learning (and it's really not difficult to pick up) but I see way more people push against learning it than they do other things (eg python).

Demiurge 2y ago

If Rails could be written in Python, I would probably prefer that. Ruby has too much of Perl. If I didn't choose to write a critical piece of software in Perl, which caused me numerous sleepless nights hunting for a missing quote or other weird character, I might have thought it was cool. By my experience with PHP and Perl is why I prefer Python. Clever is not what I ever want a language to be. Python > Lua > PHP > Ruby > JavaScript > Perl > Bash

wkirby 2y ago
Honestly same, but only because Python has types relatively sorted. While I actually enjoy the language, Ruby is… struggling… and after supporting multiple Rails apps in the wild for close to a decade, what me and my team miss the most is the ability to hint types easily.
lucasoshiro 2y ago
> Python > Lua > PHP > Ruby > JavaScript > Perl > Bash Those are different languages, with different characteristics. Perhaps the only two that can be directly compared as a 100% replacement to another are Ruby and Python.

derefr 2y ago

I love using Ruby for shell scripting, but there are also a ton of little nits I have to fix whenever I'm doing it. For example: Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception", ala bash `set -e`. So in many of my Ruby scripts there lives this little helper: def system!(*args, **kwargs) r = system(*args, **kwargs) fail "subprocess failed" unless $?.success? r end And I can't ask "is this command installed" in an efficient built-in way, so I end up throwing this one in frequently too (in this instance, whimsically attached to the metaclass of the ENV object): class << ENV def path @path ||= self['PATH'].split(':').map{ |d| Pathname.new(d) } end def which(cmd) cmd = cmd.to_s self.path.lazy.map{ |d| d + cmd }.find{ |e| e.file? && e.executable? } end end I have other little snippets like this for: • implicitly logging process-spawns (ala bash `set -x`) • High-level wrapper methods like `Pathname#readable_when_elevated?` that run elevated through IO.popen(['sudo', ...]) — the same way you'd use `sudo` in a bash script for least-privilege purposes • Recursive path helpers, e.g. a `Pathname#collapse_tree` method that recursively deletes empty subdirectories, with the option to consider directories that only contain OS errata files like `.DS_Store` "empty" (in other words, what you'd get back from of a git checkout, if you checked in the directory with a sensible .gitignore in play) ...and so forth. It really does end up adding up, to the point that I feel like what I really want is a Ruby-like language or a Ruby-based standalone DSL processor that's been optimized for sysadmin tasks.

copirate 2y ago
> Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception" Since Ruby 2.6 you can pass `exception: true` to `system` to make it behave like your `system!`. https://rubyreferences.github.io/rubychanges/2.6.html#system-has-exception-argument https://rubyreferences.github.io/rubychanges/2.6.html#system...
- derefr 2y ago
  Didn't realize that! That's one snippet I can maybe eliminate now. (As to why I didn't know: the first thing in the RDoc for Kernel#system is still "see the docs for Kernel#spawn for options" — and then Kernel#spawn doesn't actually have that one, because it doesn't block until the process quits, and so returns you a pid, not a Process::Status. I stopped looking at the docs for Kernel#system itself a long time ago, just jumping directly to Kernel#spawn...) But come to think of it, if Kernel#system is just doing a blocking version of Kernel#spawn → Process#wait, then shouldn't Process#wait also take an exception: kwarg now? And also-also, sadly IO.popen doesn't take this kwarg. (And IO.popen is what I'm actually using most of the time. The system! function above is greatly simplified from the version of the snippet I actually use these days — which involves a DSL for hierarchical serial task execution that logs steps with nesting, and reflects command output from an isolated PTY.)

kazinator 2y ago

It seems like a waste of precious syntax to dedicate backticks to running shell commands. What's between the backticks is not even portable; the commands rely on an operating-system-specific command interpreter. > puts `ls`.lines.map { |name| name.strip.length } # prints the lengths of the filenames Fantastic example, except for the commandment violation: "thou shalt not parse the output of 'ls'"! You really want to read the directory and map over the stat function (or equivalent) to get the length. 2> (flow "." open-directory get-lines (mapcar [chain stat .size])) (727 2466 21 4096 643 16612 5724 163 707 319 352135 140 51 0 4096 114898 1172428 1328258 4096 4096 4096 29953 4096 4096 0 27 4096 4096 35585 8450 968 40960 14610 4096 14 755128 1325258 4096 17283 218 471 104 4096 99707 1255 4096 129 4096 721 9625 401 15658 4096 235 98 1861 664 23944 4286 4096 1024 0)

Too 2y ago
They have inherited the second biggest mistake of shellscripts; Requiring the user to manually check $? after each command. No thanks. Anything that doesn't have error handling enabled by default goes straight in the trash bin.
- nomilk 2y ago
  I couldn't spot any way to make something like `ls -j` (j is an illegal option) throw an exception in ruby (as opposed to simply outputting the system error message). The closest I could find is what you suggest (checking $?), or using something like this [1], which would require changing syntax: system('ls -j', exception: true) Would be great to know if there's some easy callback or, ideally, a global setting one can make so a ruby exception is thrown if there's an error running system commands using the backtick syntax. [1] https://stackoverflow.com/a/54132729/5783745 https://stackoverflow.com/a/54132729/5783745
- tasuki 2y ago
  It's not required to check $? after each command. It's only when calling out to the shell. How would you propose to handle that? Throw an exception on a shell command which fails? Some would say calling out to shell is an anti pattern by itself. Others would say exceptions are an anti pattern. (Just use appropriate return type and there's no need for exceptions ever!)

rednafi 2y ago

Ruby is slow and encourages an esoteric convention over configuration style of OO code. If you enjoy it, more power to you. However, Python is everyone's second favorite or least favorite language, and it runs laps around Ruby any day. Then there's Go if you need some extra oomph!

jakogut 2y ago
Go is a good language for when I want binaries an order of magnitude larger than necessary.
- simoncion 2y ago
  Oh but golang's "deployment story" is SO SIMPLE! /s
  - notnmeyer 2y ago
    isn’t it though?
    - simoncion 2y ago
      I was arguing with myself about adding additional context. It looks like I should have put it in. In my professional experience, Go cannot consistently avoid producing binaries that fail when run on an incompatibly-different version of glibc than they were build and linked on. Also. for most deployment situations, Go's "single, enormous binary" doesn't matter. * In The Cloud(TM), you have full control over what you deploy. So, so, so often you have a custom VM or Docker image that you just squirt out there and automation to keep it updated and rebuilt, which makes deployment just trivial. * On Windows, you have Windows Installer, which you can instruct to check for and install any prereqs you require. (It's incredible how all of the pieces to build a proper package manager have been in Windows since the early 2000s, and yet no package manager came out of MSFT.) * On OSX, you either use the App Store or one of the handful of package managers... but I expect nearly zero non-Power-Users use the package managers. I guess there are also the nutballs who do 'curl $INSTALL_SCRIPT | sudo bash'. * On Linux... well nearly zero non-Power-Users run desktop Linux, and those that do likely already have Ruby, Python, Perl, and GCC installed.

adamnemecek 2y ago

I’ve been enjoying nushell

__MatrixMan__ 2y ago
Nushell is fantastic. Being able to just paste json into a terminal and immediately start working with it in a structured way feels like a superpower. The interactive help and well-thought-out error messages feel like a hug.

lr4444lr 2y ago

Not denying Ruby is good, but other than process forking, why should I prefer it to Python?

katbyte 2y ago
Not using white space as a delimiter
- ianschmitz 2y ago
  Ruby is surprisingly picky with white space
  - manume 2y ago
    Can you give an example? I can't think of a single situation where whitespace matters in Ruby (unless of course you forget to put a space between two commands or something silly).
    - codesnik 2y ago
      if foo is a method then `foo + bar` and `foo+bar` are `foo()+bar`, but `foo +bar` is `foo(+bar)` ternary ? : also has some interesting whitespace dependent mixups with symbols, but I cannot remember what. I think that parser has many gotchas like that, but they are really really rare to bite you, because ruby's magic follows human intuition as much as possible.
      - p_l 2y ago
        still less annoying than Python's semantic whitespace
    - ezrast 2y ago
      It's not really a problem in practice (and I love Ruby), but it's still wild to me that they made the parser do this: $ irb irb(main):001:0> def foo(x=70) = x => :foo irb(main):002:0> i = 2 => 2 irb(main):003:0> foo / 5/i => 7 irb(main):004:0> foo /5/i => /5/i

phendrenad2 2y ago

Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box. Are you going to add 5 minutes to your docker build to compile Ruby? Probably not. Luckily, I've found that Perl has most of the best features of Ruby, and it's installed everywhere. It's time to Make Perl Great Again.

bigstrat2003 2y ago
Why on earth would you add the compilation of a Ruby interpreter to your docker build? Just install it through the package manager of whatever distro your image is built on.
manume 2y ago
> Sadly, 9 out of 10 environments lack a Ruby interpreter out of the box. Please name those 10 environments you are talking about. In my experience, a reasonably recent Ruby version is present almost everywhere. > add 5 minutes to your docker build Why on earth would it take 5 minutes to install anything? If you install Ruby through a package manager (it's present in pretty much all of them: https://www.ruby-lang.org/en/documentation/installation/#package-management-systems https://www.ruby-lang.org/en/documentation/installation/#pac...) it takes only seconds.

greenthrow 2y ago

By definition a shell script is one that is run by the shell. The term should not be overloaded to include scripts that require another interpreter.

lucasoshiro 2y ago
What do you suggest for the title?

samatman 2y ago

The things which make Ruby good for shell scripts are, to a large degree, things it inherited from Perl. Which was, and is, a great language for scripting. People use it a lot less these days, for a lot of reasons, some better than others. I myself do simple stuff in bash, and pull out some Python for more complex scripting. My Perl chops have withered, last time I was paid to use it was 21 years ago, but it really is a great scripting language, and you'll find it installed on a great deal more systems than Ruby. One of these days I'll give Raku a spin, just for old time's sake.

corytheboyd 2y ago

Hell yeah! I’ll never forget being fooled by HN that you have to use Perl if you want portable scripts that aren’t bash, writing a whole script with it, and having a coworker politely, yet firmly, tell me that I am dumb and it should just be Ruby… and it was a script I was checking into a Rails app! It’s even trivial to include dependencies with bundler inline https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...

notnmeyer 2y ago
this is… somewhat horrific, right? would anyone do this without explicit version pinning? loosely pinned deps installed on execution sounds fucking awful.
- deleted 2y ago
  [deleted]
- corytheboyd 2y ago
  Well, don’t use this then. It’s a neat feature for quickly iterating, and you can quite easily go up from here to a proper Gemfile/Gemfile.lock.
- manume 2y ago
  You can pin a gem to a specific version, of course. `gem "mygem"` installs the latest version. `gem "mygem", "~> 4.0.0"` installs >= 4.0.0 but < 4.1.0, which is what you probably want when using Semantic Versioning, which most gems adhere to, to get the latest patch version. `gem "mygem", "4.0.10"` installs exactly that version.

JohnMakin 2y ago

As a self proclaimed shell advocate I am… intrigued. As a self proclaimed shell advocate, I also have a revulsion to back ticks.

justin_oaks 2y ago
Yes, all hail $( ) I haven't used backticks in my shell scripts in years.
pikelet 2y ago
You can also use %x(ls some/dir), which is a syntax used in other places too, like %w[word word word] (array of word strings, notice you can choose the delimiter).
lucasoshiro 2y ago
> I also have a revulsion to back ticks. Yeah, I understand... But this is other language with a similar syntax for a similar functionality.

JohnMakin 2y ago

To simulate “types” in complex shell scripting I typically involve a lot of json objects acting as my data structures and the file system for a rudimentary database. They’re horrifying ugly, but they tend to work pretty reliably. This is probably easier.

kchr 2y ago
The `ip` network utility now supports JSON ouput via the `ip --json` invocation since a while back. I would love to see more tools implementing this as an option!

xgdgsc 2y ago

Or use Julia for scripts https://github.com/ninjaaron/administrative-scripting-with-julia https://github.com/ninjaaron/administrative-scripting-with-j...

Alifatisk 2y ago
There’s a section in the readme explaining why it’s a bad idea haha

bdcravens 2y ago

I spend most of my time writing Rails or other backend Ruby, and I prefer my system-level scripts in bash. Philosophically I don't want to have to manage dependencies or broken gems (though inline deps obviate that, and it's not like I've never had to wrestle with apt)

manume 2y ago
What do you mean by "broken gems"?
- bdcravens 2y ago
  Either dependency issues with other gems, or gems that break due to some sort of library in the OS. If you pin all of your versions, and use it in one place, that's less of an issue, but many scripts are designed to have some level of portability (even if it's to a new instance of the server)
  - manume 2y ago
    In my experience, Bundler has improved a lot with regard to resolving dependency issues over the years. And OS libs are only really depended on by a few gems, no? 99% of them don't use FFI or call OS libs. Moreover, how often do you really move a script to a completely different OS, where you don't know which OS libs are installed? And wouldn't those missing OS libs also be a problem when writing the script in Bash or any other language?
lucasoshiro 2y ago
Well, but what you do with Bash can be also done with vanilla Ruby...

pipeline_peak 2y ago

Being able to call external commands with backticks alone makes it better suited than Python for shell scripting.

__MatrixMan__ 2y ago
If you've reached for something with more structure than the string-soup that is bash, why would you then embrace the string-soup that is invoking subprocesses with backticks? Involving an entirely separate parser just to split your args up feels like a footgun to me. Python's `sh` handles this rather nicely: http://sh.readthedocs.io/en/latest/ http://sh.readthedocs.io/en/latest/
- deleted 2y ago
  [deleted]
frou_dh 2y ago
IMO that feature snatches defeat from the jaws of victory because if the command fails (exit code >0) it doesn't raise an exception. Same issue with the system(...) method by default.

corytheboyd 2y ago

Overall a nice lite write up! Bash is great, but it occasionally becomes untenable, usually around the time where HTTP and whatnot becomes involved. Same goes for shell exec exit codes, you can use an API like popen3 for this: https://ruby-doc.org/stdlib-2.4.1/libdoc/open3/rdoc/Open3.html#method-c-popen3 https://ruby-doc.org/stdlib-2.4.1/libdoc/open3/rdoc/Open3.ht... You mention using threads and regex match global variables in the same write up. Please use the regex match method response instead of the $1 variables to save yourself the potential awful debugging session. It even lets you access named capture groups in the match response using the already familiar Hash access API. Example: https://stackoverflow.com/a/18825787 https://stackoverflow.com/a/18825787 In general, just don’t use global variables in Ruby. It’s already SO easy to move “but it has to be global” functionality to static class methods or constants that I’ve encountered exactly zero cases where I have NEEDED global variables. Even if you need a “stateful constant” Ruby had a very convenient Singleton mixin that provides for a quick and easy solution. Besides, if you actually WERE to take advantage of them being global VARIABLES (reassigning the value) I would confidently bet that your downstream code would break, because I’m guessing said downstream code assumes the global value is constant. Just avoid them, there’s no point, use constants. This applies to any language TBH, but here we’re talking about Ruby :)

lucasoshiro 2y ago
Thanks for the comment! Even though I wrote this about Ruby, I must admit that I am not a specialist.
- corytheboyd 2y ago
  No problem at all, and you don’t need to be a specialist to be excited and write something up :) I hope this didn’t come across as judgy, I’ve just lived through many a Ruby bug heh

xavdid 2y ago

Ruby's a great language- I've always enjoyed its ergonomics and clarity. But its editor tooling hasn't kept up with its one-time competitor, Python. I've mostly been in the Python ecosystem for the past few years and the LSP investment from Microsoft has really shown. Rich Python support in VSCode is seamless and simple. Coming back to Ruby after that caught me off guard - it feels like I'm writing syntax-highlighted plain text. There's an LSP extension from Shopify, but it's temperamental and I have trouble getting it working. Editor support isn't everything (the actual language design is still the most important), but it definitely affects how eager I am to use it. I basically never choose Ruby over Python given the option, which is too bad. Ruby's a cool language!

anothername12 2y ago
The “editor tooling” in Jetbrains’ Rubymine is fabulous.
- xavdid 2y ago
  That's true, I've heard good things! I don't use Jetbrains products, but I'm glad they work well here.
  - anothername12 2y ago
    Yeah I think they have some special sauce that puts them a bit ahead of what LSP-based. Definitely check it out
rafamvc 2y ago
It got a lot better recently. You should take a second look.
- xavdid 2y ago
  I mean, I was trying to set up editor support for a Ruby script last week. So unless it was improved really recently, it's not where I'd like it to be.

nomilk 2y ago

If you never coded in ruby before, but use macOS, you already have ruby installed. Just open terminal and type irb to bring up the ruby interpreter and try out the code in the article.

anothername12 2y ago
I think the built in one is deprecated though right?
- Neywiny 2y ago
  Are there distributions where this doesn't happen? I feel like I'm often installing or compiling new versions of packages and languages because they're outdated on Ubuntu
  - medstrom 2y ago
    A rolling distro like Arch or Void.
    - Neywiny 2y ago
      Gotcha, thanks
shagie 2y ago
~ % irb WARNING: This version of ruby is included in macOS for compatibility with legacy software. In future versions of macOS the ruby runtime will not be available by default, and may require you to install an additional package. irb(main):001:0>

kajika91 2y ago

No pipe (and I mean in parallel too) no love for me. Nice calling syntax though.

copirate 2y ago
You can pipe with the `pipeline*` method of open3 which is part of the stdlib: For example: require "open3" last_stdout, wait_threads = Open3.pipeline_r("cat /etc/passwd", ["grep", "root"]) last_stdout.read # => "root:x:0:0::/root:/bin/bash\n" wait_threads.map(&:value).map(&:success?) # => [true, true] https://ruby-doc.org/3.2.2/stdlibs/open3/Open3.html https://ruby-doc.org/3.2.2/stdlibs/open3/Open3.html
- e12e 2y ago
  Can you easily chain these, though? (gzcat some.txt|grep foo|sort -u|head -10 etc?). Especially lazily, if the uncompressed stream is of modest size, like a couple of gigabytes?
  - RulerOf 2y ago
    I'd suspect you could do that with Open3, but if you are, why not just read the file and process with Ruby instead?
    - e12e 2y ago
      I'm currently working with 150MB worth of gzipped JSON - marshalling the full file from JSON to ruby hash eats up a lot of memory. One tweak that allows for easier lazy iteration over the file (while keeping temporary disk Io reasonable) is to pipe it through zcat, jq in stream mode to convert to ndjson, gzip again - for a temp file that ruby zlib can wrap for a stream convenient for lazy iteration per read_line...). Generally marshalling a gig or more of JSON (non-lazily) takes a lot of resources in ruby.
      - RulerOf 2y ago

oliviergg 2y ago

For me, the problem with shell scripting is that I do it only occasionally. I’ve always been tempted by higher-level languages. The arrival of tools like ChatGPT has made me much more comfortable writing scripts of intermediate complexity. I find shell scripting more interesting now.

manume 2y ago
Similar case for me, but I think the author says it well: > That is, most of the cases Bash for me is enough, but if the script starts to become complex, I switch to Ruby. Even if ChatGPT lets you bang out more complex shell scripts easily, if you have to come back to it later on to fix an error or add a new feature, it's really hard to understand it (if you don't deal with such scripts on a daily basis). If you start with Ruby (or Python or similar) from the beginning, it's much easier to understand and extend later on.

fire_lake 2y ago

Nothing about dependencies! A scripting language needs a way to declare dependencies in a locked-down way, inside of the script that requires them. They must be portable across platforms.

manume 2y ago
Bundler inline works great for that: https://bundler.io/guides/bundler_in_a_single_file_ruby_script.html https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...

aorth 2y ago

Side note: Firefox is offering to translate this blog post from Portuguese for me though the content is clearly in English. I noticed the `<html>` element has a `lang="pt"` attribute. The site is generated by Jekyll, which I have not used in years, so I'm wondering if this is a site-level setting or could be overridden in frontmatter...

lucasoshiro 2y ago
Thanks for the feedback! I need to check it because my mother language is Portuguese and my personal page was initially only in Portuguese (everything there has also a version in Portuguese if you click the Brazilian flag). Some years ago I started to write in English but I didn't want to delete my older posts.

hiAndrewQuinn 2y ago

I have what probably sounds like a niche use case, where most of the boxes I work on don't have access to the Internet. So for me, "is it installed in the base distribution" is the difference between being able to start immediately no matter which box I'm concerned with, and spending months trying to upstream a new program to be installed with our OS image team. I took a look around a vanilla Debian 12 box, and didn't see Ruby in there [1]. So, sadly, although I really like the way Ruby looks, I'm going to have to stick with Bash and Python 3 for the hard stuff. [1]: https://hiandrewquinn.github.io/til-site/posts/what-programming-languages-come-out-of-the-box-on-debian-12/ https://hiandrewquinn.github.io/til-site/posts/what-programm...

lucasoshiro 2y ago
Yeah, it makes sense.

floppy-disk 2y ago

While I write most of my scripts in Ruby and enjoy doing so, there is one gripe I have with it: its slow start-up time. On my machine, running an empty Ruby script takes about 100ms, compared to <10ms for Python, Perl, Lua, Bash. One can mitigate the problem somewhat using the `--disable-gems` flag, but that's not a good general solution.

Borg3 2y ago
100ms !!!! wtf? :) [borg@cube] time ruby -e 'nil' real 0m0.007s
- floppy-disk 2y ago
  Whoa, interesting... What version are you running? Here's my system: $ ruby -v ruby 3.3.1 (2024-04-23 revision c56cd86388) [x86_64-linux] $ time ruby -e '' real 0m0.122s user 0m0.102s sys 0m0.020s I found an old Reddit thread also hinting at bad start-up times: https://old.reddit.com/r/ruby/comments/aqxepw/rubys_startup_time_seems_to_get_worse https://old.reddit.com/r/ruby/comments/aqxepw/rubys_startup_....
  - Borg3 2y ago
    Very old one :) Because its reliable. Version 1.8.7 here. I want controllable delays. I want controllable threads :) I want portability. Of course, if you need more peformance, you need to go for newer stuff.
    - Alifatisk 2y ago
      I’ll try this out sometime, but probably through docker https://hub.docker.com/r/hublogix/minimal-ruby https://hub.docker.com/r/hublogix/minimal-ruby
      - Borg3 2y ago
        Yeah.. old ruby have interesting build system. It first build miniruby with is single selfcontained binary having all core ruby functionality. Then, it is used to run some more .rb build scripts to finaly make fully functional ruby. Actyally that miniruby is pretty usefull. For basic stuff. Also, its very easy to build static ruby containing all extra stuff you care about, like Win32 API for example. I myself have custom ruby binary on Win32 (Cygwin) with GRX library added in, So I can do basic graph stuff directly from ruby :) That stuff is written in it: http://borg.uu3.net/~borg/?gperf http://borg.uu3.net/~borg/?gperf

deleted 2y ago

[deleted]

bingemaker 2y ago

My first application of Ruby was to use it for shell auto-completions. I'm so grateful that I learnt Ruby first, and then Rails. Ruby is a great language to get some utility working out real fast. Rails is great for MVP. I fail to understand why people bitch about Ruby/Rails by comparing them to other languages/frameworks.

deleted 2y ago
[deleted]
lucasoshiro 2y ago
Yeah, as I mentioned, every time someone says something bad about Ruby for me, it is about Rails...

helpfulContrib 2y ago

[dead]

globular-toast 2y ago

Cool, I'm sold, added Ruby to my to learn list! I use Python a lot but I don't think it's as good for writing scripts as bash or Perl. Ruby looks like it fits that "better but not much harder" category much better.

lucasoshiro 2y ago
Well, I'm happy that I you learnt something new here :-). Ruby is quite similar to Python, perhaps you don't even need to study too much!

stefantalpalaru 2y ago

[dead]

softwaredoug 2y ago

Shoutout to Hannes Moser at Shopify and our regular pairing on the Disco CLI tool that orchestrated a lot of Elasticsearch config stuff. :) As a Python guy I found the setup for this sort of CLI too really refreshing!

nunez 2y ago

I agree. Ruby is a _fantastic_ language for getting things done quickly whose credibility was unfairly maligned by Rails. Unbelievably easy to read, and, with rspec, it is stupid easy to write tests for. No need to fuss with interfaces like you do with Golang; yes, that is the right thing to do, but when you need to ship _now_, it becomes a pain and generates serious boilerplate quickly. I've switched to Golang for most things these days, as it is a much safer language overall, but when shell scripts get too hard, Ruby's a great language to turn to.

Alifatisk 2y ago
Rspec changed my view on writing tests, it’s way to easy now!

0mwh 2y ago

I've been working (intermittently) on a project to do exactly this: https://github.com/0mWh/ruby-linux-initrd https://github.com/0mWh/ruby-linux-initrd

deleted 2y ago

[deleted]

Alifatisk 2y ago

I wonder why this post blew up so much