7 ms·

The unreasonable effectiveness of VMs in hacker pedagogy

43 points by hiAndrewQuinn 3y ago

ekimehtor 3y ago
I like this process much better than using pre-compiled virtual machine images that let's just say certain State controlled businesses are offering up as an easier way to install dependencies and or tool – chains needed for their "Open source" hardware. vagrant plugin install vagrant-share Seems like a useful tool also!
hobs 3y ago
The only problem with this article is "Everyone knows what a virtual machine is" - no, full stop, they do not.
- ergonaught 3y ago
  The title alone conveys this difficulty without entering into the content.
kreetx 3y ago
I'd say nix is the more deterministic way to get all, and only, the things you absolutely need. Docker has the same issue as vagrant that you'll get a different debian depending on the day you init the VM, run apt, and so on.
- oneplane 3y ago
  It depends. Depends on a whole lot of things, and Nix isn't special in this: Do you re-use an existing artefact or do you build a new one? You can put Nix in Docker and in Vagrant and get the same result every time. You an also not use Nix, create a Docker image or a Vagrant box, and whenever you instantiate one of those you get the same result every time. You an also involve Nix and not pin to a specific release and now you get a different result every time. That would be the same as creating a Dockerfile with a mutable tag or Vagrant file with a dynamic setup. Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need.". That is something that is probably only really useful in CI when you need reproducible builds. For everything else you probably want to be tracking a stable release for the version you target. Especially since most people aren't working to get a deterministic result, but a 'close enough' result to get the job done.
  - kreetx 3y ago
    In nix you can pin your environment down to a commit in nixpkgs, from which whenever you build your system (i.e, artefact), the result will be the same. How do you achieve this in docker or vagrant? > Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no?
    - oneplane 3y ago
      In docker and vagrant you do it the same way, the hash of the specific artefact. That can be a commit ID, but you can also do this without Git and the likes and use a content hash. Those are guaranteed to always refer to the same thing. >> Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." >IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no? That is why I wrote about the specific exception for CI, but since we're talking about humans experimenting in a box, we're not talking about CI.
      - kreetx 3y ago
        > In docker and vagrant you do it the same way, the hash of the specific artefact. No, in nix you pin the commit hash of nixpkgs, which gives you the exact same package set every time. But with a Docerfile, even if you add it to your project's repo then any `apt-get install something` will still give you a different something depending on when you build the image. In nix, you always get the same something. > That is why I wrote about the specific exception for CI I meant that you keep the dev environment the same as CI because if you develop in a different environment than the CI then whenever CI runs then you'll get new issues (because you weren't using the same environment).
        oneplane 3y ago
        I wasn't referring to Nix, how Nix uses hashing, what specific hash, or source implementation it is using, or that it might be using an artefact hash. I was referring to the referencing method: a hash. This is the same method in the three mentioned products (nix, docker, vagrant): an immutable pointer to a specific resource. As for CI, dev environments, not relevant here. I only mentioned in passing because CI is a common use case for hardcoded immutable reference. We're talking about someone wanting a box to experiment in, not CI. Not even software engineering either.
- nunez 3y ago
  You can avoid that unpredictability if you don't use the latest tag, or, better, create images or start containers from SHA256 digests. Those are guaranteed to be immutable. I.e instead of: Docker run --rm debian:trixie You'd do Docker run --rm debian:trixie@sha256:0ee9224bb31d3622e84d15260cf759bf37f08d652cc0411ca7b1f785f87ac19c The only disadvantage of the digest approach is that you would need to manually resolve the digest that's correct for your processor arch. Using bare tags like "debian:trixie" can resolve to manifest lists (if so configured) that has Docker automatically find the right digest for your arch.
  - kreetx 3y ago
    What if the environment needs more packages installed via apt, do you save those hashes somewhere, too?
    - codethief 3y ago
      Debian & Ubuntu provide daily snapshots of their repositories, see snapshot.debian.org and https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-predictability-and-consistency-in-cloud-deployments https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-p... See e.g. https://github.com/reproducible-containers/repro-sources-list.sh https://github.com/reproducible-containers/repro-sources-lis... for how to use them. Also, at least for Debian there are official images available that use snapshot.debian.org as package repository from the get-go – unfortunately those images are not published on a daily basis yet. It seems the overall topic if reproducibility is on the Docker team's to do list, though, see https://github.com/docker-library/official-images/issues/16044 https://github.com/docker-library/official-images/issues/160...
      - kreetx 3y ago
        But if you read those links, you'll see that it's not quite there yet. Docker in general can't solve reproducibility - it's the package manager within any container that does that.
        3y ago
- hiAndrewQuinn 3y ago
  I deem that you are correct! I have investigated NixOS in a Vagrant + Virtualbox combo very similar to what I did in this post, and yes, this is a much more deterministic solution to this idea. If the people you are writing a tutorial for can be expected to be familiar with Nix, this is probably a much more future-proofed solution. That's still a small camp of people at the moment, but it does include the creator of Vagrant himself, Mitchell Hashimoto. I maintain that a specific Debian version is a better "lingua franca" for a general audience - that may change in coming years, of course. I have begun writing a new tutorial called "NixOS: Three Big Ideas" to expose people like myself who haven't given Nix the time of day to explore its concepts in about 20 minutes or so.
_xerces_ 3y ago
He doesn't want me to have to go figure out what Docker is, but now instead I have to figure out what Vagrant is or does, he doesn't say. I go to the Vagrant site and see: "Single workflow to build and manage virtual machine environments" If I was new enough to not know what Docker is, I am pretty sure I would be a little confused about that. I don't want a "workflow" do I, I thought I was trying to follow a tutorial. Do I need to "manage virtual machine environments?". Do I need to install both Virtual Box and this Vagrant tool? I think this is a nice idea and useful, but I wouldn't couch it in terms of being some great technique for beginner tutorials.
- tengbretson 3y ago
  There's a wide chasm between not knowing what docker does and knowing docker well enough that it isn't a major drain on your productivity.
deleted 3y ago
[deleted]
hoffs 3y ago
> “But Andrew, Docker is already a thing. It’s been a thing for some time, even.” Correct. But everyone knows what a virtual machine is. Not everyone understands Docker. Do you want to risk sending someone off on a wild goose chase learning what containerization is, when they could be learning your thing instead? Meet your student where they are, not where you wish they were. Ironic
oneplane 3y ago
"effectiveness of VMs" is probably not what really mattered, but rather "disposable environments". A VM can be a type of re-creatable and disposable environment, if you also include an OS and configuration. But if all you need is a CLI context, the same could be said for chroot, LXC, Docker etc. A VM can be an easier mental model or box for people, but no all people are the same.
andrewflnr 3y ago
Re Docker, chroot, etc: the boundary around a VM is both easier to understand and more hermetic than around a container. The learner already has a pretty good idea of where a computer stops. They can pretty much transfer that knowledge to a VM and not have any problems for a long time. Whereas with Docker, what's a filesystem? What's a process namespace? And, well, I wouldn't do anything in a container that I couldn't afford to leak out into my host system. In a chroot especially, the files are just sitting there, waiting for a newbie to touch them from the wrong context.
- hiAndrewQuinn 3y ago
  Exactly! The key thing to realize here is that tutorials have to be aimed at newbies. If they know containers, great. If they don't, and later want to learn, writing and running a Dockerfile based on instructions for the cool thing they successfully ran once saying "Here's our VM, here are the commands we run, and here's the program running in action" is a great little practice problem.
ggiesen 3y ago
Vagrant is also great for filing bug reports in software. It allows you to give the developers all the commands to reproduce the bug in a clean environment.
lupusreal 3y ago
The unreasonable obnoxiousness of the overuse of the "unreasonable effectiveness" meme: I know I should let this bother me, but the "unreasonable effectiveness" of math in the natural sciences was not merely an assertion that math is useful in science, but an observation that math is so useful that it must hint at some underlying metaphysical or philosophical truth about the very nature of reality. I don't think the usefulness of VMs in pedagogy is hinting at anything so profound. They're useful in a reasonable way.
skeeter2020 3y ago
I used to do all my development in VMs but found they were too heavy-weight, too slow and too difficult to keep "clean". Docker doesn't give you the same isolation at the resource level, which is a major benefit of VMs, but I also found this was more of a hinderance than a help. For me sharing the artifacts of development or the dependencies (i.e. building Docker images or using containers for parts of the system not under development) is where the value is, vs a completely isolated, reproducable software/hardware environment. A baseline VM is still great for reproducing production issues, and avoids the "Works on my machine" badge.
tromp 3y ago
I followed the given instructions on my MacOSX 10.15.7 following brew installs of vagrant and virtualbox, but running vagrant up resulted in ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 (guest) => 2222 (host) (adapter 1) ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Remote connection disconnect. Retrying... ... Timed out while waiting for the machine to boot. This means that Vagrant was unable to communicate with the guest machine within the configured ("config.vm.boot_timeout" value) time period. I cannot find any config.vm.boot_timeout in Vagrantfile
- raincom 3y ago
  Default value is 300 seconds: https://developer.hashicorp.com/vagrant/docs/vagrantfile/machine_settings https://developer.hashicorp.com/vagrant/docs/vagrantfile/mac... So, just set config.vm.boot_timeout to 900 in the vagrant file, and see what happens.
  - tromp 3y ago
    Re-running vagrant up gives Bringing machine 'default' up with 'virtualbox' provider... ==> default: Checking if box 'debian/bookworm64' version '12.20240212.1' is up to date... ==> default: Machine 'default' has a post `vagrant up` message. This is a message ==> default: from the creator of the Vagrantfile, and not from Vagrant itself: ==> default: ==> default: Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports almost immediately. Looks like timing out was not the real issue. I can run vagrant ssh now. Realizing there's no cc and no git, I try $ apt-get update Reading package lists... Done E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) Hmm, still some challenges ahead...
    - bpfrh 3y ago
crote 3y ago
In my experience Vagrant over-promises and under-delivers. Granted, this was a few years ago. First, there's the whole "VM" part. You are constantly reminded of the border between your machine and the VM: having to explicitly start/stop the VM, SSH into it, set up port forwarding, figure out a core count & RAM size which is large enough to be useful yet small enough to work on everyone's machine, having to set up rsync wrappers because the NFS host mount is unusably slow... Second, it doesn't really deal well with change. When you're working in a team, you want everyone to have an identical environment. Vagrant is great for setting it up, but it doesn't really have a solution to changing it. Want to add a package? Nuke the VMs, start from scratch, and hope nobody runs into install issues. Docker, especially with docker-compose, provides pretty much the same but better. It was a no-no when Vagrant first started due to poor Windows support, but I understand that WSL has significantly changed that. You'd have to pay me to go back to Vagrant.
- znpy 3y ago
  > First, there's the whole "VM" part. You are constantly reminded of the border between your machine and the VM: having to explicitly start/stop the VM, SSH into it, set up port forwarding, figure out a core count & RAM size which is large enough to be useful yet small enough to work on everyone's machine, having to set up rsync wrappers because the NFS host mount is unusably slow... What are you talking about? I used to do all that you're mentioning in the Vagrantfile, via code. Granted, ruby code (which doesn't help), but still in code. > Second, it doesn't really deal well with change. When you're working in a team, you want everyone to have an identical environment. Vagrant is great for setting it up, but it doesn't really have a solution to changing it. Want to add a package? Nuke the VMs, start from scratch, and hope nobody runs into install issues. Uh, adding the vagrant file to the code repository, maybe? So that maybe you can decide if a package goes into the vm via a code review? Maybe using the provisioning methods, so that changes can both be reviewed and git-tracked. Like, have you actually read the Vagrant's documentation ? Or are you one of those people that criticize stuff just because the getting started one-page tutorial doesn't solve 100% of their use-case ? I've used Vagrant in a team in the past, and it feels like either you have no idea what you're talking about or you have used it only briefly without actually reading any documentation. > Docker, especially with docker-compose, provides pretty much the same but better. There are some things docker cannot do, and some things vagrant cannot do. They're completely different pieces of technology that happen to be able to do similar things (and happen to be used for similar purposes).
nocsi 3y ago
You probably don’t need vagrant anymore.. now that you can throw VMs into any OCI compliant registry. Prefer the container way and want multi-arch support? You can then convert the container disks into vms.
- derefr 3y ago
  > Prefer the container way and want multi-arch support? You can then convert the container disks into vms. The prior step is a lossy conversion, though; there are things arbitrary VMs can know about/control, that OCI container images cannot. Like UEFI, or host core affinity, or the layout of the guest’s physical memory, or host disk storage formatting, or nonstandard behavior for paravirtualized devices like NICs. Ideally the conversion from VM to OCI image would have some sort of standard + portable schema for encoding this type of VM “pragma” info as latent metadata — which container runtimes would ignore, but which a back-conversion into a VM could pick up and translate into hypervisor-specific VM config. I think, if that was done, then OCI images would indeed become the only portable format you need for any type of container or VM.
tambourine_man 3y ago
I was a heavy Vagrant user many years ago, but I thought the project was mostly dead due to changes in business model or focus from HasiCorp, IIRC, and of course, the popularity of Docker. Also, isn’t Vagrant dependent on VirtualBox? Because the latter doesn’t run on ARM Macs.
- hiAndrewQuinn 3y ago
  >isn’t Vagrant dependent on VirtualBox? Added a footnote suggesting a workaround with Parallels for any confused ARM Mac users out there, searching desparately for this fabled Virtualbox program. Appreciate the catch.
  - tambourine_man 3y ago
    Parallels is closed source. I suggest going with a QEMU based solution, such as multipass
    - Jtsummers 3y ago
      I like Multipass because it does actually work pretty well, but its purpose is to run Ubuntu VMs: > Ubuntu VMs on demand for any workstation Which may be suitable for many Linux-based tutorials, but not necessarily all. For instance, we deploy software to RHEL (historical reasons more than anything else) at work and might want a RHEL-based tutorial for things unique to administrating it versus administrating Ubuntu or other distributions.
- _the_inflator 3y ago
  I used Vagrant and VMs heavily for customer projects between 2013-2016. Added benefit was mainly some sort of workspace isolation since you could simply “vagrant in” you different projects via WebStorm/PHPstorm in parallel. I was given a MacBook Pro 13” with then massive 16gb exactly for this reason: to handle VMs via Vagrant. Before that, everything was kind of tricky and fragile, mostly relying on a LAMP stack without isolation. Vagrant was fantastic at the time for the job.
- 3y ago
tambourine_man 3y ago
> But Andrew, Docker is already a thing… Docker is a lot better on Linux, everywhere else you must also spin a VM of sorts. It makes a difference if you’re running multiple instances since you only pay for the virtualized kernel once.
- btbuildem 3y ago
  My first question reading the article was "wth is vagrant?". I use Docker to containerize projects all the time.
- j45 3y ago
  Docker is infinitely better on linux. There is a lot of merit to the idea that if you're building for linux servers, build in a linux environment. So much life-force is wasted in trying to make Docker on another OS behave like linux. It is ironically, easier to run Docker, an linux inside something like a Virtual box an enjoy near native experience.
  - j45 3y ago
    Also, the author of the article said there was no virtualbox for Apple Silicon. It does appear to be in some kind of beta https://www.virtualbox.org/wiki/Testbuilds https://www.virtualbox.org/wiki/Testbuilds
znpy 3y ago
I miss having a decently working Vagrant setup. VirtualBox nowadays is un-trustable. If you end up installing the wrong package some Oracle lawyer will come and ask you for money. I'd love for Vagrant to make libvirt/qemu a first-class citizen, to be honest.
Timber-6539 3y ago
Well, docker is basically the same thing without the excess resource usage (wastage in many cases) of a type 2 hypervisor. Also, had a long-forgotten setup with vagrant that I later came back to and couldn't start the VM after a couple of upgrades to vagrant & its dependencies. So take that as you will.
starkparker 3y ago
Vagrant/VirtualBox was a heavy heavy part of tech support workflows 5-6 years ago, since it gave you a way to effectively script consistent, quasi-reproducible scenarios _including reproductions of wacky customer network and storage topologies_. That's not as important anymore, I think, since wacky network/storage topoligies are now mostly wacky k8s topologies in stuff like helm charts. But I imagine that old use case isn't completely gone.
mikewarot 3y ago
I got a 404 error, so here's the Internet Archive capture of the page https://web.archive.org/web/20240331100215/https://hiandrewquinn.github.io/til-site/posts/the-unreasonable-effectiveness-of-vms-in-hacker-pedagogy/ https://web.archive.org/web/20240331100215/https://hiandrewq...
kjkjadksj 3y ago
VM centric development seems so wasteful to me especially doing it on your hacker home lab. Say you have six cores, overprovision 3x, that leaves you with 18 vms each running One Thing in the vm ethos. Meanwhile, open a pc or a mac and see how many services are running on the bare metal, its probably over 100 and not even impacting you. The author wants you to run parallels to run vms on macs… its madness
- hiAndrewQuinn 3y ago
  To be clear, I am advocating for using disposable, blank slate VMs in tutorials, so a student can walk through the exact steps they would need to get the thing built, installed, and running, with a high chance of success. Nothing more, nothing less.
pmarreck 3y ago
I swear that there should be a tag called #PeopleWhoDontKnowAboutNix Every single developer on my team has exactly the same environment thanks to a single flake.nix file that I wrote. (And it wasn't that hard. Certainly easier than wasting time troubleshooting individual dev's configuration differences. And certainly less frustrating than Docker.)
- marcolinux 3y ago
  While you are at it, also: #PeopleWhoDontKnow_QuasarVue, when the topic is frontend/webUI.

43 points by hiAndrewQuinn 3y ago

I like this process much better than using pre-compiled virtual machine images that let's just say certain State controlled businesses are offering up as an easier way to install dependencies and or tool – chains needed for their "Open source" hardware. vagrant plugin install vagrant-share Seems like a useful tool also!

hobs 3y ago

The only problem with this article is "Everyone knows what a virtual machine is" - no, full stop, they do not.

ergonaught 3y ago
The title alone conveys this difficulty without entering into the content.

kreetx 3y ago

I'd say nix is the more deterministic way to get all, and only, the things you absolutely need. Docker has the same issue as vagrant that you'll get a different debian depending on the day you init the VM, run apt, and so on.

oneplane 3y ago
It depends. Depends on a whole lot of things, and Nix isn't special in this: Do you re-use an existing artefact or do you build a new one? You can put Nix in Docker and in Vagrant and get the same result every time. You an also not use Nix, create a Docker image or a Vagrant box, and whenever you instantiate one of those you get the same result every time. You an also involve Nix and not pin to a specific release and now you get a different result every time. That would be the same as creating a Dockerfile with a mutable tag or Vagrant file with a dynamic setup. Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need.". That is something that is probably only really useful in CI when you need reproducible builds. For everything else you probably want to be tracking a stable release for the version you target. Especially since most people aren't working to get a deterministic result, but a 'close enough' result to get the job done.
- kreetx 3y ago
  In nix you can pin your environment down to a commit in nixpkgs, from which whenever you build your system (i.e, artefact), the result will be the same. How do you achieve this in docker or vagrant? > Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no?
  - oneplane 3y ago
    In docker and vagrant you do it the same way, the hash of the specific artefact. That can be a commit ID, but you can also do this without Git and the likes and use a content hash. Those are guaranteed to always refer to the same thing. >> Usually you don't really want "the more deterministic way to get all, and only, the things you absolutely need." >IMO, you do, as otherwise you'll need to fix additional issues every time your CI runs, no? That is why I wrote about the specific exception for CI, but since we're talking about humans experimenting in a box, we're not talking about CI.
    - kreetx 3y ago
      > In docker and vagrant you do it the same way, the hash of the specific artefact. No, in nix you pin the commit hash of nixpkgs, which gives you the exact same package set every time. But with a Docerfile, even if you add it to your project's repo then any `apt-get install something` will still give you a different something depending on when you build the image. In nix, you always get the same something. > That is why I wrote about the specific exception for CI I meant that you keep the dev environment the same as CI because if you develop in a different environment than the CI then whenever CI runs then you'll get new issues (because you weren't using the same environment).
      - oneplane 3y ago
        I wasn't referring to Nix, how Nix uses hashing, what specific hash, or source implementation it is using, or that it might be using an artefact hash. I was referring to the referencing method: a hash. This is the same method in the three mentioned products (nix, docker, vagrant): an immutable pointer to a specific resource. As for CI, dev environments, not relevant here. I only mentioned in passing because CI is a common use case for hardcoded immutable reference. We're talking about someone wanting a box to experiment in, not CI. Not even software engineering either.
nunez 3y ago
You can avoid that unpredictability if you don't use the latest tag, or, better, create images or start containers from SHA256 digests. Those are guaranteed to be immutable. I.e instead of: Docker run --rm debian:trixie You'd do Docker run --rm debian:trixie@sha256:0ee9224bb31d3622e84d15260cf759bf37f08d652cc0411ca7b1f785f87ac19c The only disadvantage of the digest approach is that you would need to manually resolve the digest that's correct for your processor arch. Using bare tags like "debian:trixie" can resolve to manifest lists (if so configured) that has Docker automatically find the right digest for your arch.
- kreetx 3y ago
  What if the environment needs more packages installed via apt, do you save those hashes somewhere, too?
  - codethief 3y ago
    Debian & Ubuntu provide daily snapshots of their repositories, see snapshot.debian.org and https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-predictability-and-consistency-in-cloud-deployments https://ubuntu.com/blog/ubuntu-snapshots-on-azure-ensuring-p... See e.g. https://github.com/reproducible-containers/repro-sources-list.sh https://github.com/reproducible-containers/repro-sources-lis... for how to use them. Also, at least for Debian there are official images available that use snapshot.debian.org as package repository from the get-go – unfortunately those images are not published on a daily basis yet. It seems the overall topic if reproducibility is on the Docker team's to do list, though, see https://github.com/docker-library/official-images/issues/16044 https://github.com/docker-library/official-images/issues/160...
    - kreetx 3y ago
      But if you read those links, you'll see that it's not quite there yet. Docker in general can't solve reproducibility - it's the package manager within any container that does that.
      - 3y ago
hiAndrewQuinn 3y ago
I deem that you are correct! I have investigated NixOS in a Vagrant + Virtualbox combo very similar to what I did in this post, and yes, this is a much more deterministic solution to this idea. If the people you are writing a tutorial for can be expected to be familiar with Nix, this is probably a much more future-proofed solution. That's still a small camp of people at the moment, but it does include the creator of Vagrant himself, Mitchell Hashimoto. I maintain that a specific Debian version is a better "lingua franca" for a general audience - that may change in coming years, of course. I have begun writing a new tutorial called "NixOS: Three Big Ideas" to expose people like myself who haven't given Nix the time of day to explore its concepts in about 20 minutes or so.

_xerces_ 3y ago

He doesn't want me to have to go figure out what Docker is, but now instead I have to figure out what Vagrant is or does, he doesn't say. I go to the Vagrant site and see: "Single workflow to build and manage virtual machine environments" If I was new enough to not know what Docker is, I am pretty sure I would be a little confused about that. I don't want a "workflow" do I, I thought I was trying to follow a tutorial. Do I need to "manage virtual machine environments?". Do I need to install both Virtual Box and this Vagrant tool? I think this is a nice idea and useful, but I wouldn't couch it in terms of being some great technique for beginner tutorials.

tengbretson 3y ago
There's a wide chasm between not knowing what docker does and knowing docker well enough that it isn't a major drain on your productivity.

deleted 3y ago

[deleted]

hoffs 3y ago

> “But Andrew, Docker is already a thing. It’s been a thing for some time, even.” Correct. But everyone knows what a virtual machine is. Not everyone understands Docker. Do you want to risk sending someone off on a wild goose chase learning what containerization is, when they could be learning your thing instead? Meet your student where they are, not where you wish they were. Ironic

oneplane 3y ago

"effectiveness of VMs" is probably not what really mattered, but rather "disposable environments". A VM can be a type of re-creatable and disposable environment, if you also include an OS and configuration. But if all you need is a CLI context, the same could be said for chroot, LXC, Docker etc. A VM can be an easier mental model or box for people, but no all people are the same.

andrewflnr 3y ago

Re Docker, chroot, etc: the boundary around a VM is both easier to understand and more hermetic than around a container. The learner already has a pretty good idea of where a computer stops. They can pretty much transfer that knowledge to a VM and not have any problems for a long time. Whereas with Docker, what's a filesystem? What's a process namespace? And, well, I wouldn't do anything in a container that I couldn't afford to leak out into my host system. In a chroot especially, the files are just sitting there, waiting for a newbie to touch them from the wrong context.

hiAndrewQuinn 3y ago
Exactly! The key thing to realize here is that tutorials have to be aimed at newbies. If they know containers, great. If they don't, and later want to learn, writing and running a Dockerfile based on instructions for the cool thing they successfully ran once saying "Here's our VM, here are the commands we run, and here's the program running in action" is a great little practice problem.

ggiesen 3y ago

Vagrant is also great for filing bug reports in software. It allows you to give the developers all the commands to reproduce the bug in a clean environment.

lupusreal 3y ago

The unreasonable obnoxiousness of the overuse of the "unreasonable effectiveness" meme: I know I should let this bother me, but the "unreasonable effectiveness" of math in the natural sciences was not merely an assertion that math is useful in science, but an observation that math is so useful that it must hint at some underlying metaphysical or philosophical truth about the very nature of reality. I don't think the usefulness of VMs in pedagogy is hinting at anything so profound. They're useful in a reasonable way.

skeeter2020 3y ago

I used to do all my development in VMs but found they were too heavy-weight, too slow and too difficult to keep "clean". Docker doesn't give you the same isolation at the resource level, which is a major benefit of VMs, but I also found this was more of a hinderance than a help. For me sharing the artifacts of development or the dependencies (i.e. building Docker images or using containers for parts of the system not under development) is where the value is, vs a completely isolated, reproducable software/hardware environment. A baseline VM is still great for reproducing production issues, and avoids the "Works on my machine" badge.

tromp 3y ago

I followed the given instructions on my MacOSX 10.15.7 following brew installs of vagrant and virtualbox, but running vagrant up resulted in ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration... default: Adapter 1: nat ==> default: Forwarding ports... default: 22 (guest) => 2222 (host) (adapter 1) ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: SSH address: 127.0.0.1:2222 default: SSH username: vagrant default: SSH auth method: private key default: Warning: Remote connection disconnect. Retrying... ... Timed out while waiting for the machine to boot. This means that Vagrant was unable to communicate with the guest machine within the configured ("config.vm.boot_timeout" value) time period. I cannot find any config.vm.boot_timeout in Vagrantfile

raincom 3y ago
Default value is 300 seconds: https://developer.hashicorp.com/vagrant/docs/vagrantfile/machine_settings https://developer.hashicorp.com/vagrant/docs/vagrantfile/mac... So, just set config.vm.boot_timeout to 900 in the vagrant file, and see what happens.
- tromp 3y ago
  Re-running vagrant up gives Bringing machine 'default' up with 'virtualbox' provider... ==> default: Checking if box 'debian/bookworm64' version '12.20240212.1' is up to date... ==> default: Machine 'default' has a post `vagrant up` message. This is a message ==> default: from the creator of the Vagrantfile, and not from Vagrant itself: ==> default: ==> default: Vanilla Debian box. See https://app.vagrantup.com/debian for help and bug reports almost immediately. Looks like timing out was not the real issue. I can run vagrant ssh now. Realizing there's no cc and no git, I try $ apt-get update Reading package lists... Done E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) Hmm, still some challenges ahead...
  - bpfrh 3y ago

crote 3y ago

In my experience Vagrant over-promises and under-delivers. Granted, this was a few years ago. First, there's the whole "VM" part. You are constantly reminded of the border between your machine and the VM: having to explicitly start/stop the VM, SSH into it, set up port forwarding, figure out a core count & RAM size which is large enough to be useful yet small enough to work on everyone's machine, having to set up rsync wrappers because the NFS host mount is unusably slow... Second, it doesn't really deal well with change. When you're working in a team, you want everyone to have an identical environment. Vagrant is great for setting it up, but it doesn't really have a solution to changing it. Want to add a package? Nuke the VMs, start from scratch, and hope nobody runs into install issues. Docker, especially with docker-compose, provides pretty much the same but better. It was a no-no when Vagrant first started due to poor Windows support, but I understand that WSL has significantly changed that. You'd have to pay me to go back to Vagrant.

znpy 3y ago
> First, there's the whole "VM" part. You are constantly reminded of the border between your machine and the VM: having to explicitly start/stop the VM, SSH into it, set up port forwarding, figure out a core count & RAM size which is large enough to be useful yet small enough to work on everyone's machine, having to set up rsync wrappers because the NFS host mount is unusably slow... What are you talking about? I used to do all that you're mentioning in the Vagrantfile, via code. Granted, ruby code (which doesn't help), but still in code. > Second, it doesn't really deal well with change. When you're working in a team, you want everyone to have an identical environment. Vagrant is great for setting it up, but it doesn't really have a solution to changing it. Want to add a package? Nuke the VMs, start from scratch, and hope nobody runs into install issues. Uh, adding the vagrant file to the code repository, maybe? So that maybe you can decide if a package goes into the vm via a code review? Maybe using the provisioning methods, so that changes can both be reviewed and git-tracked. Like, have you actually read the Vagrant's documentation ? Or are you one of those people that criticize stuff just because the getting started one-page tutorial doesn't solve 100% of their use-case ? I've used Vagrant in a team in the past, and it feels like either you have no idea what you're talking about or you have used it only briefly without actually reading any documentation. > Docker, especially with docker-compose, provides pretty much the same but better. There are some things docker cannot do, and some things vagrant cannot do. They're completely different pieces of technology that happen to be able to do similar things (and happen to be used for similar purposes).

nocsi 3y ago

You probably don’t need vagrant anymore.. now that you can throw VMs into any OCI compliant registry. Prefer the container way and want multi-arch support? You can then convert the container disks into vms.

derefr 3y ago
> Prefer the container way and want multi-arch support? You can then convert the container disks into vms. The prior step is a lossy conversion, though; there are things arbitrary VMs can know about/control, that OCI container images cannot. Like UEFI, or host core affinity, or the layout of the guest’s physical memory, or host disk storage formatting, or nonstandard behavior for paravirtualized devices like NICs. Ideally the conversion from VM to OCI image would have some sort of standard + portable schema for encoding this type of VM “pragma” info as latent metadata — which container runtimes would ignore, but which a back-conversion into a VM could pick up and translate into hypervisor-specific VM config. I think, if that was done, then OCI images would indeed become the only portable format you need for any type of container or VM.

tambourine_man 3y ago

I was a heavy Vagrant user many years ago, but I thought the project was mostly dead due to changes in business model or focus from HasiCorp, IIRC, and of course, the popularity of Docker. Also, isn’t Vagrant dependent on VirtualBox? Because the latter doesn’t run on ARM Macs.

hiAndrewQuinn 3y ago
>isn’t Vagrant dependent on VirtualBox? Added a footnote suggesting a workaround with Parallels for any confused ARM Mac users out there, searching desparately for this fabled Virtualbox program. Appreciate the catch.
- tambourine_man 3y ago
  Parallels is closed source. I suggest going with a QEMU based solution, such as multipass
  - Jtsummers 3y ago
    I like Multipass because it does actually work pretty well, but its purpose is to run Ubuntu VMs: > Ubuntu VMs on demand for any workstation Which may be suitable for many Linux-based tutorials, but not necessarily all. For instance, we deploy software to RHEL (historical reasons more than anything else) at work and might want a RHEL-based tutorial for things unique to administrating it versus administrating Ubuntu or other distributions.
_the_inflator 3y ago
I used Vagrant and VMs heavily for customer projects between 2013-2016. Added benefit was mainly some sort of workspace isolation since you could simply “vagrant in” you different projects via WebStorm/PHPstorm in parallel. I was given a MacBook Pro 13” with then massive 16gb exactly for this reason: to handle VMs via Vagrant. Before that, everything was kind of tricky and fragile, mostly relying on a LAMP stack without isolation. Vagrant was fantastic at the time for the job.
3y ago

tambourine_man 3y ago

> But Andrew, Docker is already a thing… Docker is a lot better on Linux, everywhere else you must also spin a VM of sorts. It makes a difference if you’re running multiple instances since you only pay for the virtualized kernel once.

btbuildem 3y ago
My first question reading the article was "wth is vagrant?". I use Docker to containerize projects all the time.
j45 3y ago
Docker is infinitely better on linux. There is a lot of merit to the idea that if you're building for linux servers, build in a linux environment. So much life-force is wasted in trying to make Docker on another OS behave like linux. It is ironically, easier to run Docker, an linux inside something like a Virtual box an enjoy near native experience.
- j45 3y ago
  Also, the author of the article said there was no virtualbox for Apple Silicon. It does appear to be in some kind of beta https://www.virtualbox.org/wiki/Testbuilds https://www.virtualbox.org/wiki/Testbuilds

znpy 3y ago

I miss having a decently working Vagrant setup. VirtualBox nowadays is un-trustable. If you end up installing the wrong package some Oracle lawyer will come and ask you for money. I'd love for Vagrant to make libvirt/qemu a first-class citizen, to be honest.

Timber-6539 3y ago

Well, docker is basically the same thing without the excess resource usage (wastage in many cases) of a type 2 hypervisor. Also, had a long-forgotten setup with vagrant that I later came back to and couldn't start the VM after a couple of upgrades to vagrant & its dependencies. So take that as you will.

starkparker 3y ago

Vagrant/VirtualBox was a heavy heavy part of tech support workflows 5-6 years ago, since it gave you a way to effectively script consistent, quasi-reproducible scenarios _including reproductions of wacky customer network and storage topologies_. That's not as important anymore, I think, since wacky network/storage topoligies are now mostly wacky k8s topologies in stuff like helm charts. But I imagine that old use case isn't completely gone.

mikewarot 3y ago

I got a 404 error, so here's the Internet Archive capture of the page https://web.archive.org/web/20240331100215/https://hiandrewquinn.github.io/til-site/posts/the-unreasonable-effectiveness-of-vms-in-hacker-pedagogy/ https://web.archive.org/web/20240331100215/https://hiandrewq...

kjkjadksj 3y ago

VM centric development seems so wasteful to me especially doing it on your hacker home lab. Say you have six cores, overprovision 3x, that leaves you with 18 vms each running One Thing in the vm ethos. Meanwhile, open a pc or a mac and see how many services are running on the bare metal, its probably over 100 and not even impacting you. The author wants you to run parallels to run vms on macs… its madness

hiAndrewQuinn 3y ago
To be clear, I am advocating for using disposable, blank slate VMs in tutorials, so a student can walk through the exact steps they would need to get the thing built, installed, and running, with a high chance of success. Nothing more, nothing less.

pmarreck 3y ago

I swear that there should be a tag called #PeopleWhoDontKnowAboutNix Every single developer on my team has exactly the same environment thanks to a single flake.nix file that I wrote. (And it wasn't that hard. Certainly easier than wasting time troubleshooting individual dev's configuration differences. And certainly less frustrating than Docker.)

marcolinux 3y ago
While you are at it, also: #PeopleWhoDontKnow_QuasarVue, when the topic is frontend/webUI.