3 ms·

Mine's getting command output out of docker. For long builds (I had one that took 4 hours), it was gutting to have it fail a long way in and not be able to see

by nomilk 3y ago

Mine's getting command output out of docker. For long builds (I had one that took 4 hours), it was gutting to have it fail a long way in and not be able to see the output of the RUN commands for thorough debugging. So I devised a stupidly simple way: add && echo "asdfasdfsadf" after each RUN command. I mashed the keyboard each time to come up with some nonsense token. That way, docker would see RUN lines as different each time it built, which would prevent it using the cached layer, and thus would provide the commands' output. I wrote the same thing (more completely) here: https://stackoverflow.com/a/73893889/5783745 https://stackoverflow.com/a/73893889/5783745 (a comment on that answer provides an even better solution - use the timestamp to generate the nonsense token for you) As stupid as this solution is, I've yet to find a better way.

sopooneo 3y ago
I'm a Docker amateur, so this will be a dumb question, but if you were using that technique after every run line in a DockerFile, wouldn't they be the same every time they're run? Like, it's random, but it's the same random values stored in the file, so wouldn't the lines get cached? Or did you adjust the DockerFile each time? Or am I completely missunderstanding?
- nomilk 3y ago
  > Or am I completely missunderstanding? No, I just didn't explain it very well. You have to mash the keyboard each time (i.e. each build) to come up with some new token. The reason this (dumb) idea was so useful was it was a choice between that (dumb idea) and either run with --no-cache (i.e. wait 2-3 hours) or build normally and not have a clear idea why it failed (since no console output for cached layers), so taking a moment to mash the keyboard in a few places (as absolutely stupid as that is) was way better than the alternative of not having complete console output (docker provides no way to --no-cache on a per layer basis, hence my stupid way of achieving it).
hyperhopper 3y ago
Why cant you echo a newly generated guid?
- nomilk 3y ago
  I had tried && openssl rand -base64 12 but docker assumes if the command hasn’t changed (even though it generates a new random value) then it can rely on the cache for that layer. A comment below the answer on stack overflow points out you could do the same thing using an environment variable (with guid or datetime to ensure no duplicates), that’s a nice approach
jddj 3y ago
This reminded me of an ongoing hack that I have running in a production webapp. When I was writing it there was no easy opt-in cache busting (renaming scripts at build time, or applying a static random querystring to the script requests, etc) in the minimal framework I was using and I was getting away with not using a task runner or anything like that. It was one of those "would be nice, but effort is better spent elsewhere" things, so to this day before I do releases I still bump a hardcoded version number that I pass to the templating engine for each route. I should fix that.
sim7c00 3y ago
a lot of my debugging involves just echo or print 'hello-N' tokens. some systems just need this :D love it. "hello-9". yay it almost worked xD.
arccy 3y ago
consider --progress plain ?
- nomilk 3y ago
  That's definitely part of the solution, but alone it will get output only for uncached layers. The && echo "lakjdflkjsdf" hack above tricks docker into thinking that layer isn't cached (since, technically, it has changed and hence should be re-run). Otherwise, even with --progress plain, it will not re-run a previously run (cached) layer and hence not provide any console output.

by nomilk 3y ago

sopooneo 3y ago

I'm a Docker amateur, so this will be a dumb question, but if you were using that technique after every run line in a DockerFile, wouldn't they be the same every time they're run? Like, it's random, but it's the same random values stored in the file, so wouldn't the lines get cached? Or did you adjust the DockerFile each time? Or am I completely missunderstanding?

nomilk 3y ago
> Or am I completely missunderstanding? No, I just didn't explain it very well. You have to mash the keyboard each time (i.e. each build) to come up with some new token. The reason this (dumb) idea was so useful was it was a choice between that (dumb idea) and either run with --no-cache (i.e. wait 2-3 hours) or build normally and not have a clear idea why it failed (since no console output for cached layers), so taking a moment to mash the keyboard in a few places (as absolutely stupid as that is) was way better than the alternative of not having complete console output (docker provides no way to --no-cache on a per layer basis, hence my stupid way of achieving it).

hyperhopper 3y ago

Why cant you echo a newly generated guid?

nomilk 3y ago
I had tried && openssl rand -base64 12 but docker assumes if the command hasn’t changed (even though it generates a new random value) then it can rely on the cache for that layer. A comment below the answer on stack overflow points out you could do the same thing using an environment variable (with guid or datetime to ensure no duplicates), that’s a nice approach

jddj 3y ago

This reminded me of an ongoing hack that I have running in a production webapp. When I was writing it there was no easy opt-in cache busting (renaming scripts at build time, or applying a static random querystring to the script requests, etc) in the minimal framework I was using and I was getting away with not using a task runner or anything like that. It was one of those "would be nice, but effort is better spent elsewhere" things, so to this day before I do releases I still bump a hardcoded version number that I pass to the templating engine for each route. I should fix that.

sim7c00 3y ago

a lot of my debugging involves just echo or print 'hello-N' tokens. some systems just need this :D love it. "hello-9". yay it almost worked xD.

arccy 3y ago

consider --progress plain ?

nomilk 3y ago
That's definitely part of the solution, but alone it will get output only for uncached layers. The && echo "lakjdflkjsdf" hack above tricks docker into thinking that layer isn't cached (since, technically, it has changed and hence should be re-run). Otherwise, even with --progress plain, it will not re-run a previously run (cached) layer and hence not provide any console output.