15 ms·
Shrinking my static site
- mattacular 6y agoSometimes you can get space savings on docker images from seemingly odd sources. For example, I found that running a chown command on files after they've been COPY'd in bloats the image size significantly (100s of MB). However, at some point Docker added a "--chown" flag to the COPY command which brings it back in line.
- rumanator 6y agoThis post says a lot more about the javascript ecosystem than Docker. Multi-stage image builds are nothing new or extraordinary, and in fact it's Docker 101. However, being forced to install 500MB worth of tooling and dependencies just to deploy a measly 30MB static website on nginx is something unbelievable.
- throwaway8941 6y agoHow is that any different from build tooling for any other language? On my system, gcc with a bunch of commonly used dependencies requires just about the same space (and it's a full Linux system, not a trimmed down container).
- parhamn 6y agoObvious: Because GCC is a compiler not a runtime. Node is a runtime. You (usually) don't ship the compiler. Less obvious: You probably wont need Nginx, especially for small static files, in languages where the native servers are fast enough (e.g. http.FileServer)
- abraham 6y agoFor a static site Node is a compiler not a runtime dependency.
- JustFinishedBSG 6y agoNode is a compiler too..
- masklinn 6y ago> How is that any different from build tooling for any other language? On the one hand yes, on the other hand it's a large amount of crap just to build a static site. `npm install @11ty/eleventy` pulls 600 packages and yields a 100MB node_modules folder. Even Sphinx (whose scope goes way beyond static site generation) "only" pulls in about 75MB worth of stuff (a third of that being Babel) over two dozen dependencies (half a dozen being sphinx's own subcomponents).
- parhamn 6y agoTo be fair, you still have a tree shaking and minification step that would reduce this quite a bit.
- ashishb 6y agoThe problem with JS ecosystem is less about size and more about the number of files.
- onion2k 6y agoThere are real problems with NPM, like the proliferation of packages that include things they shouldn't (babel included a picture of Guy Feiri for a while...) but the simplistic "there's a lot of files" argument is nonsense. If you need a lot of files then you need a lot of files.
- filleduchaos 6y ago> babel included a picture of Guy Feiri for a while... I refuse to believe that there are real people so eager to jump on the JS hate bandwagon that they unironically believed every word of an obviously satirical article. (babel has never included a picture of Guy Fieri.)
- oplav 6y agoIt might not have been a picture, but there was an ASCI art of his face that got checked in. https://github.com/babel/babel/blob/f36d07d30334f86412a9d2771880cb566a82a9b6/packages/babel-core/src/api/node.js https://github.com/babel/babel/blob/f36d07d30334f86412a9d277...
- indymike 6y agoUm... deploy the binary, don't build it in the image.
- andybak 6y agoA closer comparison - an average node folder weighs a lot more than an average Python virtualenv.
- globular-toast 6y agoWhy even have a docker image for a static site? If the site has to be built like this one then just put the output of the build process behind some webserver. We were doing this back in the 90s and didn't have to write blog posts about how not to make your site 400MB.
- ig0r0 6y agoI use Hugo as my static site generator, single binary, no dependencies, generating hundreds of pages in milliseconds ... so reading this feels so wrong, I want to call it JavaScript masochism. Taking a simple concept like a static site and adding a ton of complex tooling around it because it is the trend now? Why would you even need a docker image to run a static website? The best thing about a static website us you can host it everyone without requiring any extra resources, like putting it directly to some CDN as files, etc.
- fock 6y agobest thing: you could easily pack your static Hugo binary in a container with nothing else. But somehow nobody does this without talking about microkernels (and probably internal google-apps)
- saagarjha 6y agoWhy do you need a container in the first place?
- aledalgrande 6y agoThat's exactly how I was doing it years ago: Hugo and script to upload to S3. Engineers like to overengineer stuff... get away from that habit folks.
- oplav 6y agoAt work, we build a couple SPA React application for internal tools. All of these SPAs talk to other internal APIs via REST, so you can think of these as static websites. In production, we do exactly what you suggest: serve through a CDN. However, for our dev environments and PR builds, programmatically setting up CDN to handle these use cases is not easily integrated with our CI/CD workflow. Our CI/CD environment is, however, good at quickly deploying containerized services. For this, using an nginx container to serve the produced static bundle enables us to easily have PR builds for the dev team and product owners to see changes without having to check out and build the code.
- 6y ago
- symgryph 6y agoUse docker-slim too!
- symgryph 6y agoHe could have also used deno a very nice js intrepreter and upx (makes HUGE go progrms small)
- microcolonel 6y agoI'm glad they shrunk the image to make space for the HUGE documentation link.
- francislavoie 6y agoYou can probably shrink it even more. The Caddy alpine image is 14MB compressed. https://hub.docker.com/_/caddy/ https://hub.docker.com/_/caddy/ You also get automatic TLS certificate management and tons of other goodies that nginx doesn't offer out of the box.
- herohamp 6y agoall of my docker containers are placed behind traefik which handles TLS certifactes, HTTPS redirect, compression, and routing to the correct container
- francislavoie 6y agoThis post talks about a single-container setup though, with the static site bundled with Nginx. That's what I'm replying to, not the usecase where you have many containers you need to proxy to.
- 1337shadow 6y agoSo, if you want to host multiple sites on the same port of your server, from the same process, then you can also have Caddy behind Traefik, but then why would you need Caddy ? Traefik got docker load balancing right: watch docker.sock and self configure on the fly. I missed the whole Caddy thing anyway because the source code was not completely open back in the days I was looking for nginx alternatives that provided with LetsEncrypt automatically. However, it seems Caddy can be used as an alternative to Traefik if you use a plugin: https://caddyserver.com/v1/docs/docker https://caddyserver.com/v1/docs/docker Apparently it requires swarm though, which I don't need anyway, so, still no reason for me to try Caddy unfortunately because it does look pretty cool now.
- mholt 6y ago> the source code was not open back in the days The Caddy source code has always been open source (Apache 2.0 licensed) from day one all the way to today, and will continue to be in the future. Beware when using other Let's Encrypt integrations. Caddy will keep your site online when other servers don't. (We saw this recently when Let's Encrypt had a revocation incident and when OCSP responders / other network infrastructure went down. Caddy kept the sites up, where other sites went down or left sysadmins scrambling to renew their certs.)
- quezzle 6y agoStatic site and docker shouldn’t be seen together.
- antsar 6y agoSo if I have a kubernetes env hosting all my stuff, with standardized CI /deployment flows, and one site is static, hosting it from a container like everything else would be a cardinal sin? Thats a strangely black-and-white way to put it...
- tailsdog 6y agoCould you elaborate on that statement, why should static site and docker not be seen together?
- alexhornbake 6y agoIt depends what you’re optimizing for. One of the beautiful things about a static site is it’s ability to be served by object stores like S3 as your origin, and cached by a CDN. From an operations standpoint, you are not responsible for maintaining much of anything, the performance is super fast, highly available, and relatively cheap (no dedicated servers, just paying for bandwidth and storage costs) Contrast that with a docker container as your origin... it must be running (and that is your problem to ensure it is). If you’re optimizing for developer convenience, your traffic is low or not mission critical, or maybe you have a globally distributed highly available k8s cluster and that is “the way” your company does all the things... sure why not
- jrockway 6y agoDocker is nice because you end up with the same configuration in development and production. There are many hidden details that "just let someone else host your site" or "just rsync your files to a server" gloss over. Who is renewing your TLS certificate? Where do you configure headers, redirects, mime type mappings, etc.? Where do access logs go? How do you update the version of the web server? What effects does that update have? When you manually do these things, you rely on a bunch of implicit defaults. Maybe your production server and your workstation happen to have the same version of nginx, and happen to set the same defaults. So you can test a change on your workstation and the same change works in production. But more likely, that is not the case. So you get weird differences between development and production, and you only notice when you push to production. That is not ideal. Building an image with your webserver and static files ensures that you see the same things in both places. There is no need to test anything in production, as you have a copy of the exact code and data that is going to be running in production, locally. You can tweak and poke to your heart's content, confident that you'll have the same effect when you push to production. There is no need to maintain documentation about how to build your project and what versions of things you use; you specify them in a machine-readable format and the machine dutifully builds the project correctly every single time. (One disadvantage of clean builds, though, is that sometimes you want old artifacts to exist. Consider a case where you use webpack to generate javascript. Typically, you'll output a bundle like "main.abc123.js" which is loaded from "index.html" via a script tag. What happens when the browser loads index.html from your last build, then you the next request goes to an updated server, which says to get "main.def456.js" instead? The page silently breaks, because the server doesn't have a file called "main.def456.js" anymore. "rsync --delete" has the same problem. And if you never delete anything, you eventually use an infinite amount of disk space. So there is definitely room for improvement here, but "it will probably work if I don't think about it and cross my fingers" is not the improvement we're looking for.)
- philshem 6y agoThe initial build time is “about 3 minutes” but I’d like to know the build time of the final image.
- heroic 6y agoYou could even use nginx on scratch to remove alpine as well. https://github.com/gunjank/docker-nginx-scratch/blob/master/Dockerfile https://github.com/gunjank/docker-nginx-scratch/blob/master/...
- pvtmert 6y agotl;dr author discovers multi-stage build to throw away useless nodejs dependencies. Same also applies to even Java. Maven downloads tons of stuff these days. You may only be using single static string from a dependency.
- superkuh 6y agoBack in 2015 when cloud offerings were still marginally new, a lot of big providers were gettting into the game with Docker offerings (ie, IBM Bluemix) where the charge was based entirely on RAM*Hours. Naturally this lead to me gaming the system and making my docker images as in RAM usage small as possible. In the end I even abandoned SSH as too heavy and switched to shadowsocks (2MB resident) for networking the docker instances together.
- azangru 6y ago> This docker image resulted in a 419MB final image and took about 3 minutes to build. There are some obvious issues with this. For instance every-time I change any file it must go through and reinstall all of my node_modules. He doesn't tell whether there have been any build time improvements after the changes to the Dockerfile. Will the builder docker images get cached and thus reduce the build and deployment time?
- licebmi__at__ 6y agoWell, the article mentions an obvious improvement. The node_moduoes won't be rebuilt after any changes to the code, only changes specific to the packages.json. Basically the main step is the COPY . . step (previously COPY . /app) which will invalidate the cache every change to the code. Also that on the builder, the steps beyond npm run are not needed, but well they won't improve much on the performance or space of the overall process.
- 1337shadow 6y agoTBH I have such a setup on yourlabs.io/oss/blog because I like SASS too (SASS, the nice CSS language that integrates nicely with webpack, as long as there's node-gyp which needs python2 and g++ to build CSS, guess I'm not doing it right ><), and to serve as a template project for others that would require more elaborated frontends... ... But in my experience it's pretty boring to wait for the whole JS rebuild when you just add a post, I think next time I'm going to remove the JS build from CI and just commit the built things when I change the frontend code, which is not often.
- jrururufuf666 6y agoin my eyes this is all madness. deploying sites via github to some docker shit. how about good old ftp and a cheap shared webhost? like its been done for 30 years.
- api 6y agoThat doesn't have enough buzzwords.
- jrururufuf666 6y agoseems to me like some hipster wannabe hackers in need to inflate trivial tasks into rocket science
- epmaybe 6y agoTo be honest, GitHub sites have been amazing for me. I just have a simple static website with personal information. HTML and CSS with minimal JS. Purchasing a shared webhost still means extra cost on top of domain registration. With GitHub, I just followed their instructions with my domain, and everything just worked. I'm also not paying anything per month or year besides domain registration which is really nice when you have to start budgeting tightly. I don't know about docker and all of that, though.
- gen3 6y agoIt’s been awesome for me too. I’ve had a static site hosted on github since high school. It’s been a cheap way to host a website for free (I use github + cloudflare) since before I had a credit card. I’m willing to bet it helped me get at least one of my internships. It’s been a great way for me to learn, as I’ve re-written the site every 2 years or so.
- 1337shadow 6y agoWith GitLab Pages it's even easier, just fork one of the examples in gitlab.com/pages, push a content update in a commit and it's online a minute later thanks to GitLab-CI and GitLab Pages.
- Drdrdrq 6y agoAside: one should not use package.json to install dependencies. Use either package-lock.json (and command "npm ci") or yarn.lock (and... I forget). Keep the lock file as part of repo too, or each build could be different.
- slezyr 6y ago> or each build could be different. Or not working.
- nikeee 6y agoI'd suggest changing this: COPY package.json . RUN npm install to this: COPY package.json package-lock.json . RUN npm ci `npm ci` installs the exact dependencies specified in the lockfile. This way, transitive dependencies that were upgraded via `npm audit fix` are guaranteed to be installed. It therefore forces the image to be rebuilt when a transitive dependency changes. Copying only the package.json wouldn't do that. It also errors if the lockfile and the package.json are inconsistent. https://docs.npmjs.com/cli/ci.html https://docs.npmjs.com/cli/ci.html
- noahtallen 6y ago+1. npm ci is typically quite a bit faster too in my experience.
- oefrha 6y ago3 minutes to build a static blog (with a grand total of five posts) that doesn’t look any different from decade-old blogs. Pulling hundreds of MB from the Internet in the process. Wow. https://github.com/herohamp/eleventy-blog/tree/master/posts https://github.com/herohamp/eleventy-blog/tree/master/posts
- herohamp 6y agoyes it is sub-optimal, but that is not because building it takes so much time, it is because of the node_modules. I am looking into migrating to Hugo as has been suggested by MANY people
- discordance 6y agoHow about binary patching your container? - pretty sure I've seen this done somewhere but can't find the link to it.
- alpb 6y agoIf you're using multiple steps anyway, there's no need to use nginx base on every step. FROM nginx:1.17.10-alpine as npmpackages RUN apk add --update nodejs npm Just do: FROM node:10 RUN npm [...]
- herohamp 6y agothat is true, ill switch to that when i get around to it
- jt2190 6y agoThis approach is akin to installing all of the build tooling inside of Docker, then generating the build artifact. I'd think it'd be even slimmer to generate the build artifact first, then just copy that into the container. Is there an advantage to building inside of Docker?
- oplav 6y agoWe've found one advantage to be more reproducible builds since you don't have to worry about different versions of build tooling affecting the artifact.
- steve_adams_86 6y agoWouldn't the advantage be consistent build fragments? If you use the build tools outside of docker, you won't get the same repeatable build artifact across different machines (or as your own host system changes). Maybe I'm not understanding you though.
- IanCal 6y ago> I'd think it'd be even slimmer to generate the build artifact first, then just copy that into the container. That's what this is doing. It creates builder containers, uses them to make the artifact and copies it in: FROM nginx:1.17.10-alpine RUN rm -r /usr/share/nginx/html/ COPY --from=builder /app/_site/ /usr/share/nginx/html/ EXPOSE 80 The final image is nginx with the static files copied over.
- 1337shadow 6y agoI have no idea why they need an nginx image in their second FROM, they are just doing some npm.
- herohamp 6y agoyeah, that is a mistake. I just did not fully rethink my code when I moved the build layers. I will be removing that tonight along with a few changed suggested here
- 1337shadow 6y agoOn a second thought, I also don't understand why do npm in two different images, why not just copy the webpack bundles from the builder image into the nginx image ? For me the cause of the big image size was in COPY --from=npmpackages /app /app From your third Dockerfile, it seems replacing the above with the following would have done the trick without adding an extra stage COPY --from=npmpackages /app/_site/ /usr/share/nginx/html/
- herohamp 6y agoI do npm in two different images so that the node_modules can be cached between builds. this massively speeds up my build. The npmpackages layer only installs the npm modules
- 1337shadow 6y agoI see, this speeds up when you change a dependency, because at least then your whole node_modules is not thrown away is that right ?
- kissgyorgy 6y agoYOU DON'T NEED DOCKER FOR A STATIC SITE. That's why it's called "static". There should be no moving parts in it.
- CJefferson 6y agoI have had several problems with static sites being broken by library updates. I now have a vagrant disc image which will let me always build my site.
- saagarjha 6y agoStatic in this context means "served statically", not "generated statically".
- miganga 6y agoWhy are we scared of disk space in 2020?
- watersb 6y agoI'm scared of everything in 2020.
- saagarjha 6y agoCan we put "Docker image" in the title somewhere? Otherwise, it seems like the article is talking about the site itself (i.e. having less JavaScript, optimizing the images, …)