8 ms·
Bunster: Compile bash scripts to self contained executables
- Alifatisk 2y agoVery cool, but since this transpiles Shell to Go, what makes this difficult to port to Windows?
- forgotpwd16 2y agoSeems one project's goal is to convert frequently used commands to builtins. So maybe because currently converted scripts still use external programs that are usually only available in Unix.
- stackskipton 2y agoAlso, Windows is not as fragmented so you don't tend to have different runtime environments where Ubuntu might include Y Utility but Rocky doesn't. Powershell written for Windows 2016 is likely to work fine on 2019/2022/2025.
- shrx 2y agoIt should be possible to run bash scripts on any system supported by jart's cosmopolitan library [1], which provides a platform-agnostic bash executable [2]. [1] https://justine.lol/cosmo3/ https://justine.lol/cosmo3/ [2] https://cosmo.zip/pub/cosmos/bin/ https://cosmo.zip/pub/cosmos/bin/
- shakna 2y agoYes and no. It does work - but there are some OS-specific stuff that can still pop up and explode on you. There are different guarantees around open/write on Windows and *nix, and cosmopolitan doesn't 100% paper over those gaping differences. It doesn't change the underlying file locking behaviour of the file system, for example. You can run into thread-time guarantees and other streaming problems, when piping from one thing to another.
- shrx 2y agoInteresting, thanks for pointing this out. Can I read more about potential pitfalls somewhere, or is it mostly trial and error?
- shakna 2y agoJustine's pretty easy to talk to and fairly available. I tend to try something, and then hit them up if things start falling apart.
- jart 2y agoWhat are these gaping differences? File locks are a horror show. No doubt. But I can't think of any truly terrible differences with the other things you mentioned.
- shakna 2y agoFile locks are the thing that has most been biting me, to be honest. The horrow show of NTFS, and Windows Defender, where a 'close' can become a costly operation.
- jart 2y agoWe could potentially create a background worker thread for closing handles.
- shakna 2y agoFrom memory, rustup ended up creating a worker for closing handles, to avoid dealing with Windows Defender. So it can be a valid approach. Though, cosmo might then run into Windows not being happy with two things attempting to open the same file, if the programmer then attempts to reopen the file. You could do some logic that cancels closing in the thread, and re-hands the handle to the user if the handle still exists, but that might be a little fragile. To be honest... There may not actually be a good answer that works 100% of the time. Working around the platform might introduce race conditions you absolutely don't want.
- jonathaneunice 2y agoAmbitious. Given the great diversity of shell scripting needed (even if just bash) across different variants of Linux and Unix and different platform versions, debugging the resulting transpiled executables is not something I'd be keen to take on. You'd want to be an expert in the Go ecosystem at minimum, and probably already committed to moving your utility programming into Go.
- koolba 2y agoDoes it support eval? Because then you could compile something like #!/usr/bin/env bash eval “$@“ And get a statically compiled bash!
- Imustaskforhelp 2y agoWhat does this do mate? (I tried to run it and it failed)
- mbreese 2y agoYou’d need to pass in arguments… All it does is evaluate the expression you pass in as arguments. ./evalme.sh echo hello world The joke being that if you could transpile this evalme.sh script to a static binary, you’d effectively have a static version of bash itself (transpiled to Go).
- koolba 2y agoIt evaluates the arguments to the command as bash commands. So if you save the file as foo.sh and add it to your PATH. You could run: $ foo.sh 'date ; ls ; foo=bar ; echo "Hello $foo"' Or really anything you'd like as the argument is treated as a bash script. NOTE: The original comment had the #! of the shebang backwards (as !#) due to a typo.
- vander_elst 2y agoAre there performance drawbacks in particular with long pipelines (e.g. something like `cat | grep | sed | bc | paste | ...`)?
- ComputerGuru 2y agoTo the contrary. They’re all run in parallel and the (standard) output goes directly from one to the next without being buffered by the shell. Unix overhead for process creation is very low compared to others, doing the same under, for example, Windows, would be more expensive. But if you have to run n processes, much better to run them in a single pipeline like that. (Source: I’m a shell developer. Fish-shell ftw!)
- mixedmath 2y agoI'm confronted with a similar problem frequently. I have a growing bash script and it's slowly growing in complexity. Once bash scripts become sufficiently long, I find editing them later to be very annoying. So instead, at some point I change the language entirely and write a utility in python/lua/c/whatever other language I want. As time goes on, my limit for "sufficient complexity" to justify leaving bash and using something like python has dropped radically. Now I follow the rule that as soon as I do something "nontrivial", it should be in a scripting language. As a side-effect, my bash scripting skills are worse than they once were. And now the scope of what I consider "trivial" is shrinking!
- AtlasBarfed 2y agoIsn't this perfect for LLM? You know, assuming they transpile well, I haven't tried a solid one yet. I wonder if kernel code rewrites in rust with Llama (obviously reviewed are up to snuff.
- maccard 2y agoI agree. My limit is pretty much one you start branching or looping, it should be in another tool. If that seems low to you, that’s the point
- fieu 2y agoI have exactly the same issue. I maintain a project called discord.sh which sends Discord webhooks via pure Bash (and a little bit of jq and curl). At some point I might switch over to Go or C. https://github.com/fieu/discord.sh https://github.com/fieu/discord.sh
- wiether 2y agoFirst of all, thank you for your work! I'm using it daily for many years now and it does exactly what I expect it to do. Now I'm a little concerned by the end of your message because it could make its usage a bit trickier... My main usecase is to curl the raw discord.sh file from GitHub in a Dockerfile and put in in /user/local/bin, so then I can _discord.sh_ anytime I need it. Mostly used for CI images. The only constraint is to install jq if it's not already installed on the base image. Switching to Go or C would make the setup much harder I'm afraid
- skulk 2y agoIf you want portable shell-scripts that come with their dependencies bundled, Nix also has a solution: writeShellApplication[0] (and more simpler ones like writeShellScript). writeShellApplication { name = "show-nixos-org"; runtimeInputs = [ curl w3m ]; text = '' curl -s 'https://nixos.org' | w3m -dump -T text/html ''; } writeShellApplication will call shellcheck[1] on your script and fail to build if there are any issues reported, which I think is the only sane default. [0]: https://nixos.org/manual/nixpkgs/stable/#trivial-builder-writeShellApplication https://nixos.org/manual/nixpkgs/stable/#trivial-builder-wri... [1]: https://www.shellcheck.net/ https://www.shellcheck.net/
- randall 2y agoomg i love nix so much.
- azeirah 2y agoNix is the best. If you're reading this and wondering how you can use this for yourself? You don't need nixos at all. You can install nix on any linux-like system, including on MacOS
- gchamonlive 2y agoI still haven't come around to using nix in my daily workflow. My concern is high entry bar, obscure errors and breaking changes, but also excessive use of storage either because that's how it works or because I won't know how to manage the store well. How's nix these days? How long would you expect someone with years of Linux management experience (bash, ansible, terraform, you name it, either onprem or on cloud) to get comfortable with nix? And what's would be the best roadmap to start introducing nix slowly in my workflow?
- epic9x 2y agoStart by using home-manager in your current environment. Once you can modularize your own config, start building other systems with it. It's a very deep rabbit hole, and starting off as a replacement for managing your own dotfile scripts and the like is a great way to try it out without having to replace whole systems.
- 752963e64 2y ago[dead]
- Phlogistique 2y agoThe README fails to address the elephant in the room, which is that usually shell scripts mainly call external commands; as far as I can tell there is no documentation of which built-ins are supported? That said, in a similar vein, you could probably create a bundler that takes a shell script and bundles it with busybox to create a static program.
- nodesocket 2y agoI also wondered this as well. How is something like "cat file.json | jq '.filename' | grep out.txt" implement into Go?
- beepbooptheory 2y agoI haven't looked at the code, but I assume this is just taking care of things like pipes, loops, variables, conditionals, etc, and leaving the actual binaries like jq as stubs assumed to be there. Its abstracting the shell, not the programs you run in the shell.
- mananaysiempre 2y agoSure, but why is that an interesting goal? Historically, bash has had very good backwards compatibility, and it’s unlikely that you need new features anyway.
- notnmeyer 2y agoyou can write bash but run the scripts on systems that may not have bash, is my first thought. packaging “shell” scripts into a scratch container or similar sounds pretty nice for certain use cases.
- adamc 2y agoIf that's all you want, is compiling it all into go really better than just having a portable bash?
- berbec 2y agoSeeing as how they just implemented the IF statement[0] two weeks ago, I'm going to hold of for a few more releases before testing. [0]: https://github.com/yassinebenaid/bunster/pull/88 https://github.com/yassinebenaid/bunster/pull/88
- withinboredom 2y agoI think you’d have to say more. It looks quite sane to me.
- ur-whale 2y agoI'm not able to fathom the security implications of this but my gut tells me ... ugh.
- elzbardico 2y ago[flagged]
- deleted 2y ago[deleted]
- nightowl_games 2y agoOne of the most critical elements of a shell script is that the source can be easily examined. Bringing this into your system seems like a huge liability. The syntax of shell scripts is terrible, but we write it to do simple things easily without needing more external tools. git-bash on windows is generally good enough to do the kind of things most shell scripts do. This tool feels like the worst of both worlds: bash syntax + external dependency.
- BeetleB 2y agoOh. I was just about to comment that it may be easier to understand what it does by decompiling the binary than by looking at the actual unreadable Bash language ;-)
- stabbles 2y agoA big advantage of shell scripts is that they're scripts and you can peek in the sources or run with `-x` to see what it does.
- kmclean 2y agoScience has gone too far.
- gtsop 2y agoIt is a very interesting technical feat to be able to do that... but should you do it? My gut feeling says no. Unless I am missing something.
- epic9x 2y agoPortability and other constraints I've discovered with the shell have always been a sign I need to reach for different tool. Bash is so often a "glue" language where accessibility and readability are it's primary feature right after the immediate utility of whatever it's automating. Writing POSIX compatible scripts is probably safer and can be validated with projects like shellcheck. That said - this is a neat project and I've seen plenty of "enterprise" use-cases where this kind of thing could be useful.
- sammnaser 2y agoI don't see what problem this solves, especially in its current form only supporting Unix. Bash scripts are already portable enough across Unix environments, the headaches come from dependency versioning (e.g. Mac ships non-GNU awk, etc). Except with this, when something breaks, I don't even get to debug bash (which is bad enough), but a binary compiled from Go transpiled from bash.
- rednafi 2y agoNeat project. Can’t say I’ve ever been in a situation where I thought, “If only this shell script were a standalone binary.” By the time I get to that point, I’ve usually outgrown shell syntax and just jump straight to Go. Still, I can see this being really handy for people who don’t speak Go or Rust but want to throw together a quick-and-dirty shell script and still need a standalone binary.
- extraduder_ire 2y agoI have. At one point I wanted to set a bash script to setuid/setgid. By the time I read up on why that didn't work and how to "fix" it, I decided it was a bad idea and tried something else.
- IshKebab 2y agoThis is fucking dumb. Sorry but this is just a paragon of everything wrong with Unix. The only reason to use shell in the first place is because I can't use a binary compiled from a sane language. This... Wow. This is like not having your cake and not eating it. The shitness of Bash combined with the non-portability of binaries! Sign me up! It's the opposite of https://amber-lang.com/ https://amber-lang.com/ which tries to (not sure it succeeds) provide a sane language with the portability of shell (ignoring Windows). That's a sensible project. This is just... Why does this exist?
- deleted 2y ago[deleted]
- josephcsible 2y ago> Password and Expiration Lock: Surprisingly, some people have asked for this feature. Basically, It allows you to choose an expiry date at build time. the generated program will not work after that date. Also you can choose to lock the script using a password. whenever you try to run it, it prompts for the password. Support for that makes me sad. It's antithetical to everything FOSS is.
- xyzzy_plugh 2y agoThis is has nothing to do with FOSS. Self-detonating code is a great idea, something my peers and I often joke about but rarely actually implement (though I have done depreciations that are similar). Here's some FOSS just for you: /* Copyright (c) 2025 xyzzy_plugh all rights reserved. Usage of the works is permitted provided that this instrument is retained with the works, so that any entity that uses the works is notified of this instrument. DISCLAIMER: THE WORKS ARE WITHOUT WARRANTY. */ if(time(NULL) > 1767225600) exit(1);
- josephcsible 2y agoYes, it's possible for FOSS programs to do those things. But the point I'm making is that since they're trivially removable from FOSS programs, I expect people who use those features for real to distribute closed-source binary-only programs.