25 ms·

env docker run

496 points by adtac 3y ago

tzury 3y ago
As a rule, if you can write your code in a file, and run it as a script, it is better than writing scrolls between <<EOF … EOF The why is obvious.
- mattigames 3y ago
  Some people like their personal full-programs inside a single-file, I think the appeal is that after opening you only have to keep scrolling to continue reading the other "files", or that if you need to attach it to an email or something similar you are sure it has no dependency on other files, but yeah the trade-off is not worth it.
  - devoutsalsa 3y ago
    And keeping it one file means you're reducing risk of a breaking change in the external script.
- jonahx 3y ago
  It's not obvious to me that the benefits outweigh the benefits of a single, easily readable file.
  - d-z-m 3y ago
    Some would disagree that heredoc'ing your scripts makes them easily readable.
- deleted 3y ago
  [deleted]
a_t48 3y ago
That’s cute - though typically the docker images I build need supporting infra around them anyhow - I’d have to forward to the build script.
- adtac 3y ago
  I'm probably getting banned for committing this war crime but have you considered a #!/usr/bin/env -S bash -c "docker run --privileged ..." FROM docker RUN <<EOF cat >/other.Dockerfile #!/usr/bin/env -S bash -c "docker run ..." FROM debian:buster EOF RUN chmod +x /other.Dockerfile CMD bash -c "/other.Dockerfile &; ./main"
  - ta988 3y ago
    could you make it curl a script as well
  - asn1parse 3y ago
    heck, why not toss in --net=host
    - bravetraveler 3y ago
      ... and privileged, then make the entrypoint 'nsenter' for PID 1
    - yonatan8070 3y ago
      -v /:/sysroot
  - a_t48 3y ago
    No, but that’s a fun idea. Most of my docker crimes involve working around the lack of REBASE or similar to transplant a layer from another stage. Instead I’m forced to abuse rsync.
    - 8organicbits 3y ago
      I use `COPY --from=image` to move data between images. Do you need some more advanced features of rsync?
    - efrecon 3y ago
      I've got that for rebasing: https://github.com/efrecon/docker-rebase https://github.com/efrecon/docker-rebase
pushedx 3y ago
The readme compares it to the cross-architecture Cosmopolitan libc, but Docker is anything but cross-platform. On any other platform besides Linux it requires a Linux VM. Linux containers are great (and I run Linux as my desktop OS), just pointing out the not-so-efficient nature of considering this cross-platform.
- erik_seaberg 3y ago
  Doesn’t Cosmopolitan rely on QEMU to emulate an x86_64 CPU when running on any other platform?
  - HumanOstrich 3y ago
    No
  - leonheld 3y ago
    No, it doesn't. You're probably thinking of binfmt https://docs.kernel.org/admin-guide/binfmt-misc.html https://docs.kernel.org/admin-guide/binfmt-misc.html.
    - erik_seaberg 3y ago
      Was QEMU replaced with another emulator or some kind of translation layer to run on a non-x86_64 CPU? I’m going by https://justine.lol/ape.html https://justine.lol/ape.html: > It'll be nice to know that any normal PC program we write will "just work" on Raspberry Pi and Apple ARM. All we have to do embed an ARM build of the emulator above within our x86 executables, and have them morph and re-exec appropriately, similar to how Cosmopolitan is already doing doing with qemu-x86_64, except that this wouldn't need to be installed beforehand.
- 8organicbits 3y ago
  I explored the idea of using the scratch image with a cosmopolitan binary to get something more cross-architecture, but you need a shell to run those binaries. I'd love to see cross architecture Docker images, if someone else can figure out a trick to make it work.
benatkin 3y ago
I feel comfortable with fenced code blocks. Using heredocs all the time, not so much. ```js title="/root/server.js" console.log('test') ``` or `/root/server.js` ```js console.log('test') ``` vs RUN <<EOF cat >/root/server.js console.log('test') EOF However the Markdown one is better if the syntax highlighting theme makes the code fence a color that doesn't stick out - either monochrome or closer to the background color.
- pushedx 3y ago
  The file is a Dockerfile with a shebang line that ignores the comments with a regex. Code fences would not be valid. The point of this isn't to share this code, it's a demo of the clever shebang line.
- bionhoward 3y ago
  Brilliant idea. Single markdown file for a whole app stack?
  - benatkin 3y ago
    Multiple markdown files, presenting code more like gists where you can read them top to bottom, plus with docs in between...
keepamovin 3y ago
Can we figure out a way to throw an exec into the shebang so the Docker process replaces the bash One?
IggleSniggle 3y ago
I like how the last code line reads ctx.stroke()
Kab1r 3y ago
This isn't POSIX compliant is it? I feel like I tried to do something different but trying to put arguments in a shebang and ran into trouble there a year or two ago.
- cpuguy83 3y ago
  It depends on the version of /usr/bin/env.
- error9348 3y ago
  Should be fine, you can even compile and run a C file using a shebang
- cerved 3y ago
  I believe it's compliant but only in the sense that the end result is unspecified by POSIX. I.e. you can't rely on this working on a POSIX compliant system
throwaway892238 3y ago
Cute trick, but it's not actually what the title claims. Since this is actually env calling bash first, not docker, this should just be a Bash script. You can still feed the Dockerfile to docker build via STDIN. But you'd gain the ability to shellcheck the Bash, the code would be easier to read, write, maintain, add comments to, etc. You could keep the filename the same, run it the same way, etc. The way they've done it here is just unnecessarily difficult.
- chii 3y ago
  > You can still feed the Dockerfile to docker build via STDIN. but you'd then have to work out how to "filter out" the bash commands inside this bash script to make it a valid docker file. Unless of course, you entirely store the docker file contents inside heredocs. That works fine, but it's not as "cool" as "executing" dockerfiles as a script.
- notso411 3y ago
  You can say it is wrong without being insufferably condescending
phone8675309 3y ago
Is this webscale?
- mirekrusin 3y ago
  It has web scales on it for sure.
- scrps 3y ago
  Is hyperscale.
  - jbverschoor 3y ago
    hyperscale^3-8
chubot 3y ago
Something like this should definitely exist, just not with Docker! Podman is better but it's also a bit coupled to a distro - https://news.ycombinator.com/item?id=38981844 https://news.ycombinator.com/item?id=38981844 The problem is the Linux kernel container primitives are a bit of a mess bubblewrap is a lot closer, although last I heard it's not in some distros for security reasons - https://news.ycombinator.com/item?id=30823164 https://news.ycombinator.com/item?id=30823164
- nickstinemates 3y ago
  another docker post filled with podman propaganda. despite it all, still no one uses it.
  - rtpg 3y ago
    I mean I know several people who run their infra with podman. But it's for personal things, I don't know if there is any level of usage at the enterprise level.
    - viraptor 3y ago
      Not enterprise level, but I made a choice of deploying podman at $job rather than docker for a few reasons.
      - KAMSPioneer 3y ago
        I'll chime in to say that I have started deploying podman over Docker where it's frictionless at $job as well. I'd say half (or more) of my new container deploys are podman. At home I use only podman because my tinkering doesn't affect anyone but me.
    - oso2k 3y ago
      DISCLAIMER: I work for Red Hat. I'm formerly an OpenShift Consultant and SA. podman has underpinned our Kubernetes distribution, OpenShift, since 4.0 was released in 2019. OpenShift is a $1B+ USD business for us (https://www.newsobserver.com/news/business/article271678707.html https://www.newsobserver.com/news/business/article271678707....). You can search and see a sample of who uses it for Enterprise level business.
renewiltord 3y ago
Haha very clever. I like it. L
forrestthewoods 3y ago
Can someone explain what this is and what it does? I have no idea. I use Windows and have never needed to use Docker for anything.
- BossingAround 3y ago
  It turns a Dockerfile into an executable script, so that by executing the Dockerfile, the shebang invokes docker to build and run the file. Pretty neat if you're using Dockerfiles, but also highly non-standard so you wouldn't use it in your company repo (unless you want to increase the "what-the-fuck" level of your repo). It's more of a "look, this is cool" kind of a thing if you're a Linux and container user.
  - amne 3y ago
    I can see this used to install dependencies (php composer?) to inspect code with references that resolve instead of having to spinup a whole toolchain just for that
- maronato 3y ago
  It’s a Docker shebang. Normally shebangs are used to define what shell to use when running a script, or in the case of a python script, to run it with “./myscript” instead of “python myscript.py”. Here OP created a little hack for building and running a docker container by adding a shebang to a Dockerfile. Usually it’s a two step process. You first use “docker build” to build the image and then “docker run” to create a container from it. With this little hack you just run “./Dockerfile” and it does both. It’s cool, but not really useful for most people.
WhyNotHugo 3y ago
Using spaces in a shebang is not a standard thing and doesn’t work in most shells.
- jstanley 3y ago
  The spaces are being handled by `env`: $ env "-S echo hello world" hello world https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#g_t_002dS_002f_002d_002dsplit_002dstring-usage-in-scripts https://www.gnu.org/software/coreutils/manual/html_node/env-...
- flakes 3y ago
  Curious for what systems this does not work? I start a lot of my shebangs with `#!/usr/bin/env <app>` such that I can rely on PATH for resolving application locations.
  - chuckadams 3y ago
    Just plain #!app also works. Probably less portable, but it does work on linux and macOS. Not sure if POSIX has anything to say about shebangs.
    - flakes 3y ago
      Huh, I guess I've never given it too much thought. I was under the impression for some reason that the first argument was supposed to be an absolute path.
      - chuckadams 3y ago
        It might have had to be absolute on ancient Unixen ... Unices? Seems POSIX has all of this to say about shebangs: If the first line of a file of shell commands starts with the characters "#!", the results are unspecified. So it's basically all down to convention, but one that's been followed long enough that you can rely on it. I still don't count on shebang taking more than one argument to the command though.
kitd 3y ago
For added excitement, you could go the whole hog, and generate, build and run via docker compose. Apart from anything else, you wouldn't need the 2-step build&run. I mean, you could. Whether you should, well ...
jcul 3y ago
Reminds me of the "self consuming script pattern". Seen in this super user answer. https://superuser.com/a/440059 https://superuser.com/a/440059 It embeds an awk (or any interpreter) script, and uses sed to cut out the script between tags in $0. I agree with other comments that this kind of thing can get messy, but sometimes it makes a lot of sense and let's you share a single file.
- jordemort 3y ago
  The upgrade files for a product I used to work on was (and perhaps still is) a .tar.gz file with a shell script prepended to them, to make a self-extracting/self-executing archive. The archive wasn't even base64 encoded or anything; just binary data with some text in front that can find the beginning of the binary.
  - chriswarbo 3y ago
    For those wanting to go down the self-extracting executable route, I recommend arx (it generates that sort of tarball-prepended-with-shell-script you describe) https://github.com/solidsnack/arx https://github.com/solidsnack/arx The `nix bundle` command can generate an arx file, which includes all of an application's dependencies. As an example, we started getting issues with an EC2 server whose image was an accumulation of changes over several years; whilst we worked on migrating to a saner setup (containers defined using Nix), as a stop-gap we got the server working again by using `nix bundle` to create an arx executable containing working versions of all the application's dependencies, which we could copy to the existing server as a drop-in replacement of the existing (broken) command.
  - jcul 3y ago
    Oh yeah, true, I've seen this pattern very often. Can be annoying sometimes when you just want to extract the files rather than run an installer script and they don't give an option.
mgaunard 3y ago
why not use docker build -q instead of that silly sha parsing?
- adtac 3y ago
  As you can imagine, it wasn't a fun developer experience building this incrementally without build logs. This was the only way I could find to have the cake (logs) and it eat it too (sha).
bionhoward 3y ago
This is genius and I love how this is a whole app meta-seed in a single file! I think I have docker trauma, why did we reach a point where we need computers inside our computers just for normal stuff to work? Container packing is cool, but is it just a security thing preventing us from using our normal hardware? Or versioning (NixOS)? Is wasm capable of doing this and is wasm still alive? I just feel like needing to run tests inception style inside and outside docker gets complicated and annoying and always try to just use Linux directly these days.
- petercooper 3y ago
  There are many reasons, but the simple idea of "containing" is a big part of it. You could run several versions of Python, database systems, etc. on a single machine, but it rapidly becomes confusing in most cases with dependency clashes, losing track of where everything is, etc. Anyone who worked on multiple projects ~20 years ago and didn't use VMs might remember how it felt. It's like if you have a workshop and you diligently organize all of the different parts into different trays in different units so it's easier to do all the types of work you need to do. You could just have a giant box in the corner where you chuck absolutely everything.. far less complex, but it'd make your day to day work a nightmare.
  - zarzavat 3y ago
    I'd argue that running all of those things inside docker containers also rapidly becomes confusing. The confusion is inherent to the complexity of the things you are running. I don't hate docker, but I find that it's just not that useful until you reach a certain scale. I stopped using it for personal projects and am much happier for it.
    - drsh0 3y ago
      What do you use for your personal projects now out of curiosity?
      - vrighter 3y ago
noname120 3y ago
The -S / --split-string option[1] of /usr/bin/env is a relatively recent addition to GNU Coreutils. It's available starting from GNU Coreutils 8.30[2], released on 2018-07-01. Beware of portability: it relies on a non-standard behavior from some operating systems. It only works on OSs that treat all the text after the first space as argument(s) to the shebanged executable; rather than just treating the whole string as an executable path (that can happen to contain spaces). Fortunately this non-standard behavior is more the norm than the exception: it works at least on modern GNU/Linux, BSDs, and macOS. [1] https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#g_t_002dS_002f_002d_002dsplit_002dstring-usage-in-scripts https://www.gnu.org/software/coreutils/manual/html_node/env-... [2] https://github.com/coreutils/coreutils/blob/b09dc6306e7affaf002f67350b9787c550ddb5c2/NEWS#L953-L955 https://github.com/coreutils/coreutils/blob/b09dc6306e7affaf...
- riedel 3y ago
  There is some some ways of doing this in a more portable way on Unix like systems [0] [0] https://unix.stackexchange.com/questions/399690/multiple-arguments-in-shebang/551025#551025 https://unix.stackexchange.com/questions/399690/multiple-arg...
- habitue 3y ago
  Not to be negative, but is this warning of non-standardness for like, AT&T unix or something? Beyond Linux, macos, and BSDs, I'm assuming you're running an ancient mainframe or something and are not worried about trying a cool docker shebang hack (probably because docker doesn't exist on your platform anyway)
kevincox 3y ago
This is cool hacking but I really don't get this obsession with "single file". Directories exist and can contain self-contained applications without the need to pack everything into some ugly script. They are into the slightest bit more difficult to ship around to different machines.
- da39a3ee 3y ago
  I think maybe it helps to think from the point of view of a developer for whom these single-file things are tools in their workshop. - Easier to grep a collection of single files - Easier to see what you've got in your collection in a directory listing (whether via a shell or in a web UI such as GitHib) - Easier to view the contents quickly (`cat`) - General philosophy that flat is better than nested
pedrovhb 3y ago
For an actually intentional, non-cursed version of this, see the nix-shell shebang [0]: > #! /usr/bin/env nix-shell > #! nix-shell -i python3 -p python3Packages.pillow python3Packages.ansicolor > > # scale image by 50% > import sys, PIL.Image, ansicolor > path = sys.argv[1] > image = PIL.Image.open(path) > factor = 0.5 > image = image.resize((round(image.width * factor), round(image.height * factor))) > path = path + ".s50.jpg" > image.save(path) > print(ansicolor.green(f"done {path}")) Just `chmod +x` and you have an executable with all dependencies you specify! [0] https://nixos.wiki/wiki/Nix-shell_shebang https://nixos.wiki/wiki/Nix-shell_shebang
- miduil 3y ago
  Totally, some practical use of that here as well: https://dpc.pw/posts/nix-users-you-can-start-using-rust-scripts-already/ https://dpc.pw/posts/nix-users-you-can-start-using-rust-scri...
- d0mine 3y ago
  There are pip-run, pipx run, etc for Python-specific use-cases.
- nmz 3y ago
  There's a 256byte limit for #! so this shouldn't work at all. EDIT: Now I see its badly formatted, Either way, be careful with #! size limits.
  - pronoiac 3y ago
    Ah. I think two leading spaces fix this? I'll try: #! /usr/bin/env nix-shell #! nix-shell -i python3 -p python3Packages.pillow python3Packages.ansicolor # scale image by 50% import sys, PIL.Image, ansicolor path = sys.argv[1] image = PIL.Image.open(path) factor = 0.5 image = image.resize((round(image.width \* factor), round(image.height \* factor))) path = path + ".s50.jpg" image.save(path) print(ansicolor.green(f"done {path}"))
- zopa 3y ago
  I use nix-shell, and mostly I love it. But it’s important to be aware that the above means “Go get the latest(*) versions of python, pillow and ansicolor and run this code in an environment where they’re available.” It doesn’t do any version-pinning of your dependencies. That might be what you want, but maybe not: it’s frustrating when a script that worked yesterday won’t work today, or will only work after some big download. My own rule of thumb is that nix-shell is great for quick one-offs and for sharing environments. For local tools and anything else I’m sharing with my future self, it’s usually better to write a nix expression and install it, which gives me access to Nix’s (excellent) rollback system, and lets me upgrade on my schedule, not upstream’s. * - ‘Latest’ according to whatever Nix channel checkout currently applies. Which you can change, of course, but the point is it’s external to the script.
teknopaul 3y ago
Simple file based solution. That's not abuse that's unix. Well; thats Unix before Pottering and Microsoft.
WhackyIdeas 3y ago
I had no idea a shebang could be used like this! After all of these years… Nice hack. Love it.
rekado 3y ago
There's also `guix shell` which can be used in shebang position. Example from the Guix manual: #!/usr/bin/env -S guix shell python python-numpy -- python3 import numpy print("This is numpy", numpy.version.version) It also works with manifest files specifying more complex environments.
Igor_Wiwi 3y ago
what is the practical use of it?
- dailykoder 3y ago
  Why is that important?
- cflewis 3y ago
  I mean, apart from the hacker mindset of the thing, if you’re talking about the _outcome_: there is real value in being able to distribute something to a customer without having to worry about whether they have the dependencies or not, what version of an OS they have, whether they are running on ephemeral VMs or long running machines they don’t want to pollute etc etc. At Google Cloud we did this on a team I was on. It was really the only way we could be sure of the environment we were handing off to the customer.
  - adtac 3y ago
    It's funny you mentioned Google's internal infra because my motivation for this was to hack together something to emulate the kind of static fat binaries deployed on Borg.
mike-cardwell 3y ago
I did this in Nov 2021 - https://www.grepular.com/Self_Building_and_Executing_Dockerfiles https://www.grepular.com/Self_Building_and_Executing_Dockerf... #!/usr/bin/env -S bash -c "podman run --rm -w /x -v "\$PWD:/x" \$(podman build -q - < \$0) \${@:1}"
lwneal 3y ago
You can create this type of thing (a self-contained single-file project) for any language or infrastructure, with or without a clever shebang. All you need are heredocs. For example, here's the same app but packaged as a regular bash script: https://gist.github.com/lwneal/a24ba363d9cc9f7a02282c3621afa43d https://gist.github.com/lwneal/a24ba363d9cc9f7a02282c3621afa...
- adtac 3y ago
  Of course! Bash script is Turing complete so it should be possible to implement everything in it :) The only upside to having an executable Dockerfile is that it's still a valid Dockerfile that you can use with docker build, docker-compose, etc. in addition to being able to execute it.
- bcjordan 3y ago
  Yes I love this approach, I use this exact format as a way to get ChatGPT to work with an entire multi file programming project in a single idempotent bootstrapping script. Then ask for changes to be given as the entire file again
- richdougherty 3y ago
  Agree, nesting files with cat >Dockerfile <<'EOF' and having a basic bash script seems way nicer than putting all the shell logic on the #! line.
fruktmix 3y ago
Can someone explain what this is about?
- fruktmix 3y ago
  Looks like docker everything is done in just one file?

496 points by adtac 3y ago

As a rule, if you can write your code in a file, and run it as a script, it is better than writing scrolls between <<EOF … EOF The why is obvious.

mattigames 3y ago
Some people like their personal full-programs inside a single-file, I think the appeal is that after opening you only have to keep scrolling to continue reading the other "files", or that if you need to attach it to an email or something similar you are sure it has no dependency on other files, but yeah the trade-off is not worth it.
- devoutsalsa 3y ago
  And keeping it one file means you're reducing risk of a breaking change in the external script.
jonahx 3y ago
It's not obvious to me that the benefits outweigh the benefits of a single, easily readable file.
- d-z-m 3y ago
  Some would disagree that heredoc'ing your scripts makes them easily readable.
deleted 3y ago
[deleted]

a_t48 3y ago

That’s cute - though typically the docker images I build need supporting infra around them anyhow - I’d have to forward to the build script.

adtac 3y ago
I'm probably getting banned for committing this war crime but have you considered a #!/usr/bin/env -S bash -c "docker run --privileged ..." FROM docker RUN <<EOF cat >/other.Dockerfile #!/usr/bin/env -S bash -c "docker run ..." FROM debian:buster EOF RUN chmod +x /other.Dockerfile CMD bash -c "/other.Dockerfile &; ./main"
- ta988 3y ago
  could you make it curl a script as well
- asn1parse 3y ago
  heck, why not toss in --net=host
  - bravetraveler 3y ago
    ... and privileged, then make the entrypoint 'nsenter' for PID 1
  - yonatan8070 3y ago
    -v /:/sysroot
- a_t48 3y ago
  No, but that’s a fun idea. Most of my docker crimes involve working around the lack of REBASE or similar to transplant a layer from another stage. Instead I’m forced to abuse rsync.
  - 8organicbits 3y ago
    I use `COPY --from=image` to move data between images. Do you need some more advanced features of rsync?
  - efrecon 3y ago
    I've got that for rebasing: https://github.com/efrecon/docker-rebase https://github.com/efrecon/docker-rebase

pushedx 3y ago

The readme compares it to the cross-architecture Cosmopolitan libc, but Docker is anything but cross-platform. On any other platform besides Linux it requires a Linux VM. Linux containers are great (and I run Linux as my desktop OS), just pointing out the not-so-efficient nature of considering this cross-platform.

erik_seaberg 3y ago
Doesn’t Cosmopolitan rely on QEMU to emulate an x86_64 CPU when running on any other platform?
- HumanOstrich 3y ago
  No
- leonheld 3y ago
  No, it doesn't. You're probably thinking of binfmt https://docs.kernel.org/admin-guide/binfmt-misc.html https://docs.kernel.org/admin-guide/binfmt-misc.html.
  - erik_seaberg 3y ago
    Was QEMU replaced with another emulator or some kind of translation layer to run on a non-x86_64 CPU? I’m going by https://justine.lol/ape.html https://justine.lol/ape.html: > It'll be nice to know that any normal PC program we write will "just work" on Raspberry Pi and Apple ARM. All we have to do embed an ARM build of the emulator above within our x86 executables, and have them morph and re-exec appropriately, similar to how Cosmopolitan is already doing doing with qemu-x86_64, except that this wouldn't need to be installed beforehand.
8organicbits 3y ago
I explored the idea of using the scratch image with a cosmopolitan binary to get something more cross-architecture, but you need a shell to run those binaries. I'd love to see cross architecture Docker images, if someone else can figure out a trick to make it work.

benatkin 3y ago

I feel comfortable with fenced code blocks. Using heredocs all the time, not so much. ```js title="/root/server.js" console.log('test') ``` or `/root/server.js` ```js console.log('test') ``` vs RUN <<EOF cat >/root/server.js console.log('test') EOF However the Markdown one is better if the syntax highlighting theme makes the code fence a color that doesn't stick out - either monochrome or closer to the background color.

pushedx 3y ago
The file is a Dockerfile with a shebang line that ignores the comments with a regex. Code fences would not be valid. The point of this isn't to share this code, it's a demo of the clever shebang line.
bionhoward 3y ago
Brilliant idea. Single markdown file for a whole app stack?
- benatkin 3y ago
  Multiple markdown files, presenting code more like gists where you can read them top to bottom, plus with docs in between...

keepamovin 3y ago

Can we figure out a way to throw an exec into the shebang so the Docker process replaces the bash One?

IggleSniggle 3y ago

I like how the last code line reads ctx.stroke()

Kab1r 3y ago

This isn't POSIX compliant is it? I feel like I tried to do something different but trying to put arguments in a shebang and ran into trouble there a year or two ago.

cpuguy83 3y ago
It depends on the version of /usr/bin/env.
error9348 3y ago
Should be fine, you can even compile and run a C file using a shebang
cerved 3y ago
I believe it's compliant but only in the sense that the end result is unspecified by POSIX. I.e. you can't rely on this working on a POSIX compliant system

throwaway892238 3y ago

Cute trick, but it's not actually what the title claims. Since this is actually env calling bash first, not docker, this should just be a Bash script. You can still feed the Dockerfile to docker build via STDIN. But you'd gain the ability to shellcheck the Bash, the code would be easier to read, write, maintain, add comments to, etc. You could keep the filename the same, run it the same way, etc. The way they've done it here is just unnecessarily difficult.

chii 3y ago
> You can still feed the Dockerfile to docker build via STDIN. but you'd then have to work out how to "filter out" the bash commands inside this bash script to make it a valid docker file. Unless of course, you entirely store the docker file contents inside heredocs. That works fine, but it's not as "cool" as "executing" dockerfiles as a script.
notso411 3y ago
You can say it is wrong without being insufferably condescending

phone8675309 3y ago

Is this webscale?

mirekrusin 3y ago
It has web scales on it for sure.
scrps 3y ago
Is hyperscale.
- jbverschoor 3y ago
  hyperscale^3-8

chubot 3y ago

Something like this should definitely exist, just not with Docker! Podman is better but it's also a bit coupled to a distro - https://news.ycombinator.com/item?id=38981844 https://news.ycombinator.com/item?id=38981844 The problem is the Linux kernel container primitives are a bit of a mess bubblewrap is a lot closer, although last I heard it's not in some distros for security reasons - https://news.ycombinator.com/item?id=30823164 https://news.ycombinator.com/item?id=30823164

nickstinemates 3y ago
another docker post filled with podman propaganda. despite it all, still no one uses it.
- rtpg 3y ago
  I mean I know several people who run their infra with podman. But it's for personal things, I don't know if there is any level of usage at the enterprise level.
  - viraptor 3y ago
    Not enterprise level, but I made a choice of deploying podman at $job rather than docker for a few reasons.
    - KAMSPioneer 3y ago
      I'll chime in to say that I have started deploying podman over Docker where it's frictionless at $job as well. I'd say half (or more) of my new container deploys are podman. At home I use only podman because my tinkering doesn't affect anyone but me.
  - oso2k 3y ago
    DISCLAIMER: I work for Red Hat. I'm formerly an OpenShift Consultant and SA. podman has underpinned our Kubernetes distribution, OpenShift, since 4.0 was released in 2019. OpenShift is a $1B+ USD business for us (https://www.newsobserver.com/news/business/article271678707.html https://www.newsobserver.com/news/business/article271678707....). You can search and see a sample of who uses it for Enterprise level business.

renewiltord 3y ago

Haha very clever. I like it. L

forrestthewoods 3y ago

Can someone explain what this is and what it does? I have no idea. I use Windows and have never needed to use Docker for anything.

BossingAround 3y ago
It turns a Dockerfile into an executable script, so that by executing the Dockerfile, the shebang invokes docker to build and run the file. Pretty neat if you're using Dockerfiles, but also highly non-standard so you wouldn't use it in your company repo (unless you want to increase the "what-the-fuck" level of your repo). It's more of a "look, this is cool" kind of a thing if you're a Linux and container user.
- amne 3y ago
  I can see this used to install dependencies (php composer?) to inspect code with references that resolve instead of having to spinup a whole toolchain just for that
maronato 3y ago
It’s a Docker shebang. Normally shebangs are used to define what shell to use when running a script, or in the case of a python script, to run it with “./myscript” instead of “python myscript.py”. Here OP created a little hack for building and running a docker container by adding a shebang to a Dockerfile. Usually it’s a two step process. You first use “docker build” to build the image and then “docker run” to create a container from it. With this little hack you just run “./Dockerfile” and it does both. It’s cool, but not really useful for most people.

WhyNotHugo 3y ago

Using spaces in a shebang is not a standard thing and doesn’t work in most shells.

jstanley 3y ago
The spaces are being handled by `env`: $ env "-S echo hello world" hello world https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#g_t_002dS_002f_002d_002dsplit_002dstring-usage-in-scripts https://www.gnu.org/software/coreutils/manual/html_node/env-...
flakes 3y ago
Curious for what systems this does not work? I start a lot of my shebangs with `#!/usr/bin/env <app>` such that I can rely on PATH for resolving application locations.
- chuckadams 3y ago
  Just plain #!app also works. Probably less portable, but it does work on linux and macOS. Not sure if POSIX has anything to say about shebangs.
  - flakes 3y ago
    Huh, I guess I've never given it too much thought. I was under the impression for some reason that the first argument was supposed to be an absolute path.
    - chuckadams 3y ago
      It might have had to be absolute on ancient Unixen ... Unices? Seems POSIX has all of this to say about shebangs: If the first line of a file of shell commands starts with the characters "#!", the results are unspecified. So it's basically all down to convention, but one that's been followed long enough that you can rely on it. I still don't count on shebang taking more than one argument to the command though.

kitd 3y ago

For added excitement, you could go the whole hog, and generate, build and run via docker compose. Apart from anything else, you wouldn't need the 2-step build&run. I mean, you could. Whether you should, well ...

jcul 3y ago

Reminds me of the "self consuming script pattern". Seen in this super user answer. https://superuser.com/a/440059 https://superuser.com/a/440059 It embeds an awk (or any interpreter) script, and uses sed to cut out the script between tags in $0. I agree with other comments that this kind of thing can get messy, but sometimes it makes a lot of sense and let's you share a single file.

jordemort 3y ago
The upgrade files for a product I used to work on was (and perhaps still is) a .tar.gz file with a shell script prepended to them, to make a self-extracting/self-executing archive. The archive wasn't even base64 encoded or anything; just binary data with some text in front that can find the beginning of the binary.
- chriswarbo 3y ago
  For those wanting to go down the self-extracting executable route, I recommend arx (it generates that sort of tarball-prepended-with-shell-script you describe) https://github.com/solidsnack/arx https://github.com/solidsnack/arx The `nix bundle` command can generate an arx file, which includes all of an application's dependencies. As an example, we started getting issues with an EC2 server whose image was an accumulation of changes over several years; whilst we worked on migrating to a saner setup (containers defined using Nix), as a stop-gap we got the server working again by using `nix bundle` to create an arx executable containing working versions of all the application's dependencies, which we could copy to the existing server as a drop-in replacement of the existing (broken) command.
- jcul 3y ago
  Oh yeah, true, I've seen this pattern very often. Can be annoying sometimes when you just want to extract the files rather than run an installer script and they don't give an option.

mgaunard 3y ago

why not use docker build -q instead of that silly sha parsing?

adtac 3y ago
As you can imagine, it wasn't a fun developer experience building this incrementally without build logs. This was the only way I could find to have the cake (logs) and it eat it too (sha).

bionhoward 3y ago

This is genius and I love how this is a whole app meta-seed in a single file! I think I have docker trauma, why did we reach a point where we need computers inside our computers just for normal stuff to work? Container packing is cool, but is it just a security thing preventing us from using our normal hardware? Or versioning (NixOS)? Is wasm capable of doing this and is wasm still alive? I just feel like needing to run tests inception style inside and outside docker gets complicated and annoying and always try to just use Linux directly these days.

petercooper 3y ago
There are many reasons, but the simple idea of "containing" is a big part of it. You could run several versions of Python, database systems, etc. on a single machine, but it rapidly becomes confusing in most cases with dependency clashes, losing track of where everything is, etc. Anyone who worked on multiple projects ~20 years ago and didn't use VMs might remember how it felt. It's like if you have a workshop and you diligently organize all of the different parts into different trays in different units so it's easier to do all the types of work you need to do. You could just have a giant box in the corner where you chuck absolutely everything.. far less complex, but it'd make your day to day work a nightmare.
- zarzavat 3y ago
  I'd argue that running all of those things inside docker containers also rapidly becomes confusing. The confusion is inherent to the complexity of the things you are running. I don't hate docker, but I find that it's just not that useful until you reach a certain scale. I stopped using it for personal projects and am much happier for it.
  - drsh0 3y ago
    What do you use for your personal projects now out of curiosity?
    - vrighter 3y ago

noname120 3y ago

The -S / --split-string option[1] of /usr/bin/env is a relatively recent addition to GNU Coreutils. It's available starting from GNU Coreutils 8.30[2], released on 2018-07-01. Beware of portability: it relies on a non-standard behavior from some operating systems. It only works on OSs that treat all the text after the first space as argument(s) to the shebanged executable; rather than just treating the whole string as an executable path (that can happen to contain spaces). Fortunately this non-standard behavior is more the norm than the exception: it works at least on modern GNU/Linux, BSDs, and macOS. [1] https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html#g_t_002dS_002f_002d_002dsplit_002dstring-usage-in-scripts https://www.gnu.org/software/coreutils/manual/html_node/env-... [2] https://github.com/coreutils/coreutils/blob/b09dc6306e7affaf002f67350b9787c550ddb5c2/NEWS#L953-L955 https://github.com/coreutils/coreutils/blob/b09dc6306e7affaf...

riedel 3y ago
There is some some ways of doing this in a more portable way on Unix like systems [0] [0] https://unix.stackexchange.com/questions/399690/multiple-arguments-in-shebang/551025#551025 https://unix.stackexchange.com/questions/399690/multiple-arg...
habitue 3y ago
Not to be negative, but is this warning of non-standardness for like, AT&T unix or something? Beyond Linux, macos, and BSDs, I'm assuming you're running an ancient mainframe or something and are not worried about trying a cool docker shebang hack (probably because docker doesn't exist on your platform anyway)

kevincox 3y ago

This is cool hacking but I really don't get this obsession with "single file". Directories exist and can contain self-contained applications without the need to pack everything into some ugly script. They are into the slightest bit more difficult to ship around to different machines.

da39a3ee 3y ago
I think maybe it helps to think from the point of view of a developer for whom these single-file things are tools in their workshop. - Easier to grep a collection of single files - Easier to see what you've got in your collection in a directory listing (whether via a shell or in a web UI such as GitHib) - Easier to view the contents quickly (`cat`) - General philosophy that flat is better than nested

pedrovhb 3y ago

For an actually intentional, non-cursed version of this, see the nix-shell shebang [0]: > #! /usr/bin/env nix-shell > #! nix-shell -i python3 -p python3Packages.pillow python3Packages.ansicolor > > # scale image by 50% > import sys, PIL.Image, ansicolor > path = sys.argv[1] > image = PIL.Image.open(path) > factor = 0.5 > image = image.resize((round(image.width * factor), round(image.height * factor))) > path = path + ".s50.jpg" > image.save(path) > print(ansicolor.green(f"done {path}")) Just `chmod +x` and you have an executable with all dependencies you specify! [0] https://nixos.wiki/wiki/Nix-shell_shebang https://nixos.wiki/wiki/Nix-shell_shebang

miduil 3y ago
Totally, some practical use of that here as well: https://dpc.pw/posts/nix-users-you-can-start-using-rust-scripts-already/ https://dpc.pw/posts/nix-users-you-can-start-using-rust-scri...
d0mine 3y ago
There are pip-run, pipx run, etc for Python-specific use-cases.
nmz 3y ago
There's a 256byte limit for #! so this shouldn't work at all. EDIT: Now I see its badly formatted, Either way, be careful with #! size limits.
- pronoiac 3y ago
  Ah. I think two leading spaces fix this? I'll try: #! /usr/bin/env nix-shell #! nix-shell -i python3 -p python3Packages.pillow python3Packages.ansicolor # scale image by 50% import sys, PIL.Image, ansicolor path = sys.argv[1] image = PIL.Image.open(path) factor = 0.5 image = image.resize((round(image.width \* factor), round(image.height \* factor))) path = path + ".s50.jpg" image.save(path) print(ansicolor.green(f"done {path}"))
zopa 3y ago
I use nix-shell, and mostly I love it. But it’s important to be aware that the above means “Go get the latest(*) versions of python, pillow and ansicolor and run this code in an environment where they’re available.” It doesn’t do any version-pinning of your dependencies. That might be what you want, but maybe not: it’s frustrating when a script that worked yesterday won’t work today, or will only work after some big download. My own rule of thumb is that nix-shell is great for quick one-offs and for sharing environments. For local tools and anything else I’m sharing with my future self, it’s usually better to write a nix expression and install it, which gives me access to Nix’s (excellent) rollback system, and lets me upgrade on my schedule, not upstream’s. * - ‘Latest’ according to whatever Nix channel checkout currently applies. Which you can change, of course, but the point is it’s external to the script.

teknopaul 3y ago

Simple file based solution. That's not abuse that's unix. Well; thats Unix before Pottering and Microsoft.

WhackyIdeas 3y ago

I had no idea a shebang could be used like this! After all of these years… Nice hack. Love it.

rekado 3y ago

There's also `guix shell` which can be used in shebang position. Example from the Guix manual: #!/usr/bin/env -S guix shell python python-numpy -- python3 import numpy print("This is numpy", numpy.version.version) It also works with manifest files specifying more complex environments.

Igor_Wiwi 3y ago

what is the practical use of it?

dailykoder 3y ago
Why is that important?
cflewis 3y ago
I mean, apart from the hacker mindset of the thing, if you’re talking about the _outcome_: there is real value in being able to distribute something to a customer without having to worry about whether they have the dependencies or not, what version of an OS they have, whether they are running on ephemeral VMs or long running machines they don’t want to pollute etc etc. At Google Cloud we did this on a team I was on. It was really the only way we could be sure of the environment we were handing off to the customer.
- adtac 3y ago
  It's funny you mentioned Google's internal infra because my motivation for this was to hack together something to emulate the kind of static fat binaries deployed on Borg.

mike-cardwell 3y ago

I did this in Nov 2021 - https://www.grepular.com/Self_Building_and_Executing_Dockerfiles https://www.grepular.com/Self_Building_and_Executing_Dockerf... #!/usr/bin/env -S bash -c "podman run --rm -w /x -v "\$PWD:/x" \$(podman build -q - < \$0) \${@:1}"

lwneal 3y ago

You can create this type of thing (a self-contained single-file project) for any language or infrastructure, with or without a clever shebang. All you need are heredocs. For example, here's the same app but packaged as a regular bash script: https://gist.github.com/lwneal/a24ba363d9cc9f7a02282c3621afa43d https://gist.github.com/lwneal/a24ba363d9cc9f7a02282c3621afa...

adtac 3y ago
Of course! Bash script is Turing complete so it should be possible to implement everything in it :) The only upside to having an executable Dockerfile is that it's still a valid Dockerfile that you can use with docker build, docker-compose, etc. in addition to being able to execute it.
bcjordan 3y ago
Yes I love this approach, I use this exact format as a way to get ChatGPT to work with an entire multi file programming project in a single idempotent bootstrapping script. Then ask for changes to be given as the entire file again
richdougherty 3y ago
Agree, nesting files with cat >Dockerfile <<'EOF' and having a basic bash script seems way nicer than putting all the shell logic on the #! line.

fruktmix 3y ago

Can someone explain what this is about?

fruktmix 3y ago
Looks like docker everything is done in just one file?