4 ms·

For those using firecracker, what's your method for building a rootfs?

progamer2018 5y ago
see document here https://github.com/firecracker-microvm/firecracker/blob/main/docs/rootfs-and-kernel-setup.md https://github.com/firecracker-microvm/firecracker/blob/main...
deivid 5y ago
I build them from docker images: https://blog.davidv.dev/docker-based-images-on-baremetal.html https://blog.davidv.dev/docker-based-images-on-baremetal.htm...
- deleted 5y ago
  [deleted]
lgierth 5y ago
I build them using Buildah, then mkfs.ext4, mount, cp, umount. It's a little bit annoying that I'm still using root at least for the mount part. OpenWrt's build system has a method of building rootfs ext4 and squashfs images without any root, it's somewhere in that large Makefile mess. (sorry, reposting this as I first replied to the wrong parent)
- mikepurvis 5y ago
  I similarly have built bootable disk images with various tools including buildah and have never been able to fully get away from needing root for various chrooty/loopbacky parts of the process. In principle, it should be very possible to point grub at a filesystem-in-a-file and be like "install to that", but I could never make it happen; it always wanted to be trying to infer things about how to configure itself from examining the host system. And yes, I've studied the OpenWRT build to no avail. I would be delighted for someone to dissect whatever it is that goes on in there and write it up.
- bminor13 5y ago
  No experience with Firecracker specifically, but if squashfs images are sufficient, one should be able to build a tar archive of the filesystem without root (where all the files have the correct owners, mode, etc.) and then convert it to squashfs using `tar2sqfs` in https://github.com/AgentD/squashfs-tools-ng https://github.com/AgentD/squashfs-tools-ng, also without root - I've done something like this to create squashfs images in constrained build environments, which worked well.
rad_gruchalski 5y ago
Directly from Docker image: https://github.com/combust-labs/firebuild#high-level-example https://github.com/combust-labs/firebuild#high-level-example. I’m the author.
- gavinray 5y ago
  This is incredibly interesting. Is it alright if I email you with some questions?
  - rad_gruchalski 5y ago
    Sure. My email address: radek@gruchalski.com.
- ignoramous 5y ago
  nb, building from docker is also what fly.io does: https://news.ycombinator.com/item?id=22514017 https://news.ycombinator.com/item?id=22514017
  - rad_gruchalski 5y ago
    Yes, I can also build from Dockerfile: sudo $GOPATH/bin/firebuild rootfs \ --profile=standard \ --dockerfile=git+https://github.com/hashicorp/docker-consul.git:/0.X/Dockerfile \ --cni-network-name=machine-builds \ --ssh-user=alpine \ --vmlinux-id=vmlinux-v5.8 \ --tag=combust-labs/consul:1.9.4 That SSH bit is no longer required as I'm using MMDS instead.
  - tptacek 5y ago
    We're fancier now, and use containerd with LVM. https://fly.io/blog/docker-without-docker/ https://fly.io/blog/docker-without-docker/
    - ignoramous 5y ago
      That's super fancy, indeed. Thanks for these write-ups. They make for a riveting read, even when more than half of it usually is beyond my technical know-how.

progamer2018 5y ago

see document here https://github.com/firecracker-microvm/firecracker/blob/main/docs/rootfs-and-kernel-setup.md https://github.com/firecracker-microvm/firecracker/blob/main...

deivid 5y ago

I build them from docker images: https://blog.davidv.dev/docker-based-images-on-baremetal.html https://blog.davidv.dev/docker-based-images-on-baremetal.htm...

deleted 5y ago
[deleted]

lgierth 5y ago

I build them using Buildah, then mkfs.ext4, mount, cp, umount. It's a little bit annoying that I'm still using root at least for the mount part. OpenWrt's build system has a method of building rootfs ext4 and squashfs images without any root, it's somewhere in that large Makefile mess. (sorry, reposting this as I first replied to the wrong parent)

mikepurvis 5y ago
I similarly have built bootable disk images with various tools including buildah and have never been able to fully get away from needing root for various chrooty/loopbacky parts of the process. In principle, it should be very possible to point grub at a filesystem-in-a-file and be like "install to that", but I could never make it happen; it always wanted to be trying to infer things about how to configure itself from examining the host system. And yes, I've studied the OpenWRT build to no avail. I would be delighted for someone to dissect whatever it is that goes on in there and write it up.
bminor13 5y ago
No experience with Firecracker specifically, but if squashfs images are sufficient, one should be able to build a tar archive of the filesystem without root (where all the files have the correct owners, mode, etc.) and then convert it to squashfs using `tar2sqfs` in https://github.com/AgentD/squashfs-tools-ng https://github.com/AgentD/squashfs-tools-ng, also without root - I've done something like this to create squashfs images in constrained build environments, which worked well.

rad_gruchalski 5y ago

Directly from Docker image: https://github.com/combust-labs/firebuild#high-level-example https://github.com/combust-labs/firebuild#high-level-example. I’m the author.

gavinray 5y ago
This is incredibly interesting. Is it alright if I email you with some questions?
- rad_gruchalski 5y ago
  Sure. My email address: radek@gruchalski.com.
ignoramous 5y ago
nb, building from docker is also what fly.io does: https://news.ycombinator.com/item?id=22514017 https://news.ycombinator.com/item?id=22514017
- rad_gruchalski 5y ago
  Yes, I can also build from Dockerfile: sudo $GOPATH/bin/firebuild rootfs \ --profile=standard \ --dockerfile=git+https://github.com/hashicorp/docker-consul.git:/0.X/Dockerfile \ --cni-network-name=machine-builds \ --ssh-user=alpine \ --vmlinux-id=vmlinux-v5.8 \ --tag=combust-labs/consul:1.9.4 That SSH bit is no longer required as I'm using MMDS instead.
- tptacek 5y ago
  We're fancier now, and use containerd with LVM. https://fly.io/blog/docker-without-docker/ https://fly.io/blog/docker-without-docker/
  - ignoramous 5y ago
    That's super fancy, indeed. Thanks for these write-ups. They make for a riveting read, even when more than half of it usually is beyond my technical know-how.