4 ms·

Just learnt about whiteout files from this, thanks! Trying to understand if you purposely included a filename into a layer with the same whiteout prefix “.wh.”,

by mortar 2y ago

Just learnt about whiteout files from this, thanks! Trying to understand if you purposely included a filename into a layer with the same whiteout prefix “.wh.”, if it would mess with the process that is meant to obfuscate that prefix from subsequent layers.

m463 2y ago
I learned about $_ echo abc && echo $_ abc abc except it's used with wget... wget URL && tar -xvf $_ does this work? Shouldn't tar take a filename? hmm... also, it says there is an alpine layer with "FROM scratch"??
- godelski 2y ago
  $_ is the last argument. Here's a better example to illustrate > echo 'Hello' 'world' 'my' 'name' 'is' 'godelski' Hello world my name is godelski > echo $_ godelski > !:0 !:1 !:2 "I'm" "$_" Hello world I'm godelski The reference manual is here[0] and here's a more helpful list[1] One of my favorites is > git diff some/file/ugh/hierarchy.cpp > git add $_ ## Alternatively, but this is more cumbersome (but more flexible) !!:s^diff^add So what is happening with wget is > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz && tar -xvf $_ ## Becomes > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz > tar -xvf https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz Which you are correct, doesn't work. It should actually be something like this > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz -O alpine.tar.gz && tar xzf $_ This would work as the last parameter is correct. I also added `z` to the tar and removed `-` because it isn't needed. Note that `v` often makes untaring files MUCH slower [0] https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html https://www.gnu.org/software/bash/manual/html_node/Bash-Vari... [1] https://www.gnu.org/software/bash/manual/html_node/Variable-Index.html https://www.gnu.org/software/bash/manual/html_node/Variable-...
  - ryencoke 2y ago
    If you want to add in another bash trick called Parameter Expansion[0] you can parse out the filename automatically with the special variable $_. Something like: > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz && tar xzf ${_##*/} [0] https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html https://www.gnu.org/software/bash/manual/html_node/Shell-Par...
    - MyOutfitIsVague 2y ago
      I want to make a small note, in that `$_` is a special Bashism (though it is supported widely), but Parameter Expansion is POSIX-standard and will work on all POSIX-compliant shells, not just Bash. https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#tag_02_06_02 https://pubs.opengroup.org/onlinepubs/009604499/utilities/xc...
      - vendiddy 2y ago
        You know I just realized after all these years I still don't quite know what a shell is. You have iTerm, Terminal, etc. But what do those do? Those are not the shells themselves right?
        kritr 2y ago
        iTerm and Terminal are pieces of software emulate a physical terminal environment. They take the output of programs/shells output characters and control codes to render text, clear the screen, etc. The terminal emulator receives keyboard input via your operating system, and passes it to the shell program via stdin. The shell is responsible for prompting you and handling whatever you type. For example the “$ “ waits for next character from the terminal emulator until you hit newline. The shell is responsible for parsing your input, executing any child programs “ls” for example, outputting their content to stdout, and prompting you again.
        mdaniel 2y ago
        I wanted to offer that (if you are "C handy") writing your own shell is a super informative exercise. We had to write our own shell in my operating system class at GT and I actually got it working well enough that I could "exec ./myshell" and use it for some day to day stuff. I felt empowered I tried to dig up the course but naturally things are wwaaaaaaay different now than back in my day. But OCW has something similar https://ocw.mit.edu/courses/6-828-operating-system-engineering-fall-2012/resources/mit6_828f12_lab5/ https://ocw.mit.edu/courses/6-828-operating-system-engineeri... and does ship the source files https://ocw.mit.edu/courses/6-828-operating-system-engineering-fall-2012/resources/lab5-tar/ https://ocw.mit.edu/courses/6-828-operating-system-engineeri... although I have no idea why that's only present in a graduate level class
- mdaniel 2y ago
  It's not an alpine layer, it's a Dockerfile construct representing basically an empty tar file layer: <https://docs.docker.com/build/building/base-images/#create-a-minimal-base-image-using-scratch https://docs.docker.com/build/building/base-images/#create-a...> and <https://github.com/moby/moby/pull/8827 https://github.com/moby/moby/pull/8827>
  - m463 2y ago
    He says: FROM scratch COPY ./hello /root/ ENTRYPOINT ["./hello"] > Here, our image contains 2 layers. The first layer comes from the base image, the alpine official docker image i.e. the root filesystem with all the standard shell tools that come along with an alpine distribution. But I thought "FROM scratch" was an empty container, while "FROM alpine" is a container with alpine libs/executables. otherwise using "FROM scratch" to populate for example an ubuntu image would pollute the container.
- prakashdanish 2y ago
  You're right, that doesn't work the way it is shown. Thankfully, a reader(I'm not sure if it's you) pointed this out with a solution[1] that I plan to add to the post shortly. Again, thanks for pointing this out! [1] - https://github.com/danishprakash/danishpraka.sh/issues/30 https://github.com/danishprakash/danishpraka.sh/issues/30

by mortar 2y ago

m463 2y ago

I learned about $_ echo abc && echo $_ abc abc except it's used with wget... wget URL && tar -xvf $_ does this work? Shouldn't tar take a filename? hmm... also, it says there is an alpine layer with "FROM scratch"??

godelski 2y ago
$_ is the last argument. Here's a better example to illustrate > echo 'Hello' 'world' 'my' 'name' 'is' 'godelski' Hello world my name is godelski > echo $_ godelski > !:0 !:1 !:2 "I'm" "$_" Hello world I'm godelski The reference manual is here[0] and here's a more helpful list[1] One of my favorites is > git diff some/file/ugh/hierarchy.cpp > git add $_ ## Alternatively, but this is more cumbersome (but more flexible) !!:s^diff^add So what is happening with wget is > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz && tar -xvf $_ ## Becomes > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz > tar -xvf https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz Which you are correct, doesn't work. It should actually be something like this > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz -O alpine.tar.gz && tar xzf $_ This would work as the last parameter is correct. I also added `z` to the tar and removed `-` because it isn't needed. Note that `v` often makes untaring files MUCH slower [0] https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html https://www.gnu.org/software/bash/manual/html_node/Bash-Vari... [1] https://www.gnu.org/software/bash/manual/html_node/Variable-Index.html https://www.gnu.org/software/bash/manual/html_node/Variable-...
- ryencoke 2y ago
  If you want to add in another bash trick called Parameter Expansion[0] you can parse out the filename automatically with the special variable $_. Something like: > wget https://dl-cdn.alpinelinux.org/alpine/v3.18/releases/x86_64/alpine-minirootfs-3.18.4-x86_64.tar.gz && tar xzf ${_##*/} [0] https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html https://www.gnu.org/software/bash/manual/html_node/Shell-Par...
  - MyOutfitIsVague 2y ago
    I want to make a small note, in that `$_` is a special Bashism (though it is supported widely), but Parameter Expansion is POSIX-standard and will work on all POSIX-compliant shells, not just Bash. https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#tag_02_06_02 https://pubs.opengroup.org/onlinepubs/009604499/utilities/xc...
    - vendiddy 2y ago
      You know I just realized after all these years I still don't quite know what a shell is. You have iTerm, Terminal, etc. But what do those do? Those are not the shells themselves right?
      - kritr 2y ago
        iTerm and Terminal are pieces of software emulate a physical terminal environment. They take the output of programs/shells output characters and control codes to render text, clear the screen, etc. The terminal emulator receives keyboard input via your operating system, and passes it to the shell program via stdin. The shell is responsible for prompting you and handling whatever you type. For example the “$ “ waits for next character from the terminal emulator until you hit newline. The shell is responsible for parsing your input, executing any child programs “ls” for example, outputting their content to stdout, and prompting you again.
      - mdaniel 2y ago
        I wanted to offer that (if you are "C handy") writing your own shell is a super informative exercise. We had to write our own shell in my operating system class at GT and I actually got it working well enough that I could "exec ./myshell" and use it for some day to day stuff. I felt empowered I tried to dig up the course but naturally things are wwaaaaaaay different now than back in my day. But OCW has something similar https://ocw.mit.edu/courses/6-828-operating-system-engineering-fall-2012/resources/mit6_828f12_lab5/ https://ocw.mit.edu/courses/6-828-operating-system-engineeri... and does ship the source files https://ocw.mit.edu/courses/6-828-operating-system-engineering-fall-2012/resources/lab5-tar/ https://ocw.mit.edu/courses/6-828-operating-system-engineeri... although I have no idea why that's only present in a graduate level class
mdaniel 2y ago
It's not an alpine layer, it's a Dockerfile construct representing basically an empty tar file layer: <https://docs.docker.com/build/building/base-images/#create-a-minimal-base-image-using-scratch https://docs.docker.com/build/building/base-images/#create-a...> and <https://github.com/moby/moby/pull/8827 https://github.com/moby/moby/pull/8827>
- m463 2y ago
  He says: FROM scratch COPY ./hello /root/ ENTRYPOINT ["./hello"] > Here, our image contains 2 layers. The first layer comes from the base image, the alpine official docker image i.e. the root filesystem with all the standard shell tools that come along with an alpine distribution. But I thought "FROM scratch" was an empty container, while "FROM alpine" is a container with alpine libs/executables. otherwise using "FROM scratch" to populate for example an ubuntu image would pollute the container.
prakashdanish 2y ago
You're right, that doesn't work the way it is shown. Thankfully, a reader(I'm not sure if it's you) pointed this out with a solution[1] that I plan to add to the post shortly. Again, thanks for pointing this out! [1] - https://github.com/danishprakash/danishpraka.sh/issues/30 https://github.com/danishprakash/danishpraka.sh/issues/30