13 ms·

Virtualizing development environments in 2023

113 points by hugodutka 3y ago

easton 3y ago
> Many cloud providers, like AWS inside VM-based EC2 instances, won't let you run a VM since they don't support nested virtualization Does anyone know if there's a technical limitation to nested virt on EC2 or if it's just so people use .metal? I know Azure recently (1-2 years?) started offering it.
- colinchartier 3y ago
  Nested virt doesn't work with some things like OS snapshot/restore, so they might not support it to allow those features.
- segfaltnh 3y ago
  I can't imagine amazon prefers the use of metal vs. non-metal. As I understand the pricing they're equivalent.
SkyMarshal 3y ago
Author doesn’t mention it but I wonder if tried or considered Nix/NixOS’s reproducible developer environments and ruled them out for any reason. I couldn’t tell from the article if there’s something unique to his requirements that disqualifies them. https://nixos.org/explore https://nixos.org/explore https://nix.dev/ https://nix.dev/
- hugodutka 3y ago
  Nix solves a different problem than Hocus. Nix lets you define a development environment, Hocus gives you a way to run it on a remote server. Right now we use Dockerfiles to let users define the packages they need in their dev env, but we would like to support Nix in the future too. Interestingly, you can use custom BuildKit syntax https://docs.docker.com/build/dockerfile/frontend/ https://docs.docker.com/build/dockerfile/frontend/ to build Nix environments with Docker https://github.com/reproducible-containers/buildkit-nix https://github.com/reproducible-containers/buildkit-nix, and that's probably what we will end up supporting.
  - Nezteb 3y ago
    From what I can tell, devenv [1] and devbox [2] (both built on Nix) can also deploy to remote servers. [1] https://devenv.sh https://devenv.sh [2] https://www.jetpack.io/devbox https://www.jetpack.io/devbox
    - pxc 3y ago
      There are a lot of other ways to deploy to remote servers with Nix. Many of them are NixOS-based, but some don't require NixOS at all. Whatever way you're using Nix for developer environments, you can reuse most of that work to define a package suitable for remote deployment.
  - midchildan 3y ago
    I think Nix is relevant here, because being able to run software across different machines reproducibly is one of its major selling point. I particularly like that it doesn't rely on virtualization or containerization to do that. It's up to the user to decide how to isolate the runtime environment from the host or whether they even should. Alternatively, tools building upon Nix can make that decision for them. Either way, it allows for a more flexible approach when you have to weigh the pros and cons of different isolation strategies. Development environments defined by Nix tend to compose well too, as a result of this design.
  - debosmit 3y ago
    see this is the problem w/ all these devtools - i need to pair together 5 different things when i just want a reproducible, ephemeral environment someone needs to bring a heroku-like experience but for cloud-native development
    - suryao 3y ago
      That's the mission we're on at Argonaut. I'd love to know more about how you think about it if you're up for a chat.
      - fragmede 3y ago
nickstinemates 3y ago
I am curious how important 100GB dev environment support and efficiently binpacking dev environments and rat-holing on memory efficiency actually matters for the first X enterprise customers for Hocus. Memory and CPU cores are cheap. Getting the UX, resource allocation/sizing/instance type, and general pattern for multi-tenant dev boxes seems more fruitful. Would love to see any data supporting this prioritization, and what the answer is for Workstation / HEDT-type environments given the industry's current focus on AI.
pphysch 3y ago
This blog doesn't convince me why I should care about Hocus over solutions like VSCode devcontainer + Vagrant. That is more what it is competing with, not raw Docker/VM.
- deleted 3y ago
  [deleted]
- hugodutka 3y ago
  Oh yes, in this post I was not trying to. Hocus gives you a web interface that lets you spin up a dev env with a single click of a button. We also implemented a git-integrated CI system that prebuilds your dev env on new commits. It’s basically a self-hosted Gitpod or GitHub Codespaces.
stephenr 3y ago
This whole article is weird to me. I don't really understand the point of a "dev box" that's hosted in a DC and shared, at all - at least not the way they're painting it here. Hardware capabilities for even consumer level laptops and desktops have progressed much faster than average network connections. Having testing/preview/branch named environments in a DC? Sure. But this line: > They should be able to run any software they want inside their own workspaces without impacting each other. What does that even mean? Is this about someone working on a feature branch that uses some new dependency that needs to be installed? That's 100% the sort of thing your local development environment is for, until you're ready to push it to your hosted test/whatever environment. > But when it's running in a VM, the VM gobbles up as much memory as it can and, by itself, shows no inclination to give it back. Someone, anyone, tell me which hypervisor doesn't enforce a memory limit on VMs?
- Terretta 3y ago
  See VSCode ssh remote: https://code.visualstudio.com/docs/remote/ssh-tutorial https://code.visualstudio.com/docs/remote/ssh-tutorial See also: https://code.visualstudio.com/docs/devcontainers/containers https://code.visualstudio.com/docs/devcontainers/containers People pay for this. https://github.com/features/codespaces https://github.com/features/codespaces
  - ajdude 3y ago
    This is exactly what I've been doing. I pay for a vps and just use vscode to do all of my work, I tend to switch between various computers and even various laptops, and it doesn't matter which one I'm on and just ssh into my dev station.
- scarmig 3y ago
  Enterprise likes it, because it gives them more visibility and control over what's going on in the dev box. That gives more angles to control against e.g. exfiltration threats. It also makes dev environments much more homogeneous: you worry less about client machines. In principle you could have a fleet of stateless Chromebooks whose main function is ssh and http, all connecting to identical virtualized desktops that are easily provisioned and deprovisioned.
foxbyte 3y ago
Consider exploring LXC for a more mature alternative to Docker. It's not confined to the OCI ecosystem and offers a higher degree of isolation for development environments.
- seabrookmx 3y ago
  I don't know if I'd use the word "mature" (Docker is quite mature after all), but as a long-time Docker user I did jump into lxc/lxd and can confidently say it's "system container" approach is better for dev environments. I used to use WSL 2 or a VM, but on a Linux host LXD is a really nice workflow. I can create a fully isolated Linux instance with all my dev tooling on it (optionally script/automate the install of all my tools), and with nested containers enabled I can even run _Docker in that LXD container_ so that when I type `docker ps` on that instance, and `docker ps` on my host, they each have their own set of containers running. For example, I have an ElasticSearch and Redis instance running in the dev box but Syncthing on the host. Then I can use snapshots to back up the dev environment, or blow it away completely once it gathers too much cruft all without affecting my host. It's really great. Pair it with VS Code remote SSH and you have a very feature rich setup with little effort.
- m463 3y ago
  LXC doesn't have a Dockerfile concept though. I think a Dockerfile as a recipe for an environment is pretty elegant. I've used LXC with proxmox and managing what's in a container is kind of like being a sysadmin.
  - stephenr 3y ago
    There is an LXC provider for vagrant, which gets you the one-file concept, with the benefits that not everyone on the project has to use LXC, they just need a provider that works on their host for the specified base box. This is vagrants' real superpower.
  - tiew9Vii 3y ago
    Docker is cattle, LXC is pets. Docker packages applications, and should only really have one app inside it. LXC is similar to a vm in it presents itself as a stand-alone machine but uses the host kernel, it’s not like a vm in it’s not as isolated as sharing the host kernel. So with LXC I agree, managing it is like being a sysadmin as that’s what it’s designed to be. I use Docker for things that are stateless, maybe throw away, or just test an app quickly. I use LXC for things I want to run multiple services inside, more statefull, typically where people plumb a bunch of Docker images together in I’ll use LXC. The advantage in Proxmox is I tell Proxmos to backup my LXC nightly as it’s treated similar to a VM. For making LXC feel less like needing to be a sysadmin, you can use Nix to build your LXC images and import in to Proxmox. Your LXC container becomes declarative and not to dissimilar to using a Dockerfile, it’s a far more powerful Dockerfile. What I’ve done is create a bare minimal NixOS LXC with some basic config and use that as a template then edit ‘/etc/nixos/configuration.nix’ inside the LXC on first boot. However as it’s just nixos you can build push the config remotely, use NixOps etc. It’s a really good workflow using NixOS with LXC however it took me a while to get it as the docs are a bit thin and an old+new version of docs with the new version skipping things mentioned in the old you need to do, I.e change the tty to /dev/console to get a shell inside proxmox console.
ravenstine 3y ago
Good god, no. Or, at least, let us never make this a sort of standard way of developing that you're going to push on to your engineers. Containers have their place, they can be great, but I've tried the whole "containerize everything" approach, and it's no less time consuming than not containers, and now you have another Thing You Have To Learn (TM). This isn't to say that virtualization is a bad thing for a development environment. It can be a very good thing, actually, but you don't need containers. A development environment, in my opinion, should be well organized, easy for a new developer to pick up, but also scrappy and adaptable to new and interesting situations. Containers, at least in their current conception, are antithetical to this. They're best kept to deployments or running things in a scenario where you know for sure you're not going to tinker with anything. I don't think I've worked on a software project where, at some point or another, I didn't need to reach under the hood in some unconventional way to investigate an issue or simulate some circumstance. Yes, you can do those things with containers, but now you have to deal with that complexity, which can get in the way by merely being confusing and indirect. Having given up on using containers for development, when I want to use virtualization, I just spin up minimalist instances of Debian under Qemu (using the UTM GUI). This way, I can run a Thing (TM) in isolation, easily open up a shell, or SSH into it from the host shell, or start a GUI from within it if I need to, or install however many services I want within it and open up ports to the host, or have shared folders, etc. etc. etc. But what if I need multiple of a Thing (TM) running at the same time? In that case, I just clone the VM. No Dockerfiles, no docker-compose.yml, no "volumes", no orchestration, no images failing to build because reasons, no qcow2 files blowing up with Docker For Mac (though that situation has improved), no pruning of orphaned containers/images, no worrying about whether your container should only be running a single process/service, and so on. The only drawback to this is that, if the host is Linux, then it's indeed kinda dumb to be running other Linux kernels in VMs. But hey, it's 2023 and the difference in performance may not be meaningful depending on what you're doing. I know that, on an M1 Mac, I can run graphical applications at monitor resolution with incredible speed (perhaps moreso with Parallels instead of Qemu), so I'd struggle to care about any performance drawback if my host was Linux on the same hardware. I'm sure that, for some, containers in development will be totally worthwhile. But if they don't seem worthwhile to you, dear reader, then that's probably because they wouldn't be.
therealmarv 3y ago
Gitpod, just use Gitpod. You can even run docker inside there.
- debosmit 3y ago
  yea, great for small projects but no good when you're trying to expand into enterprise capabilities -- I have to get one tool for dev, have that config diverge from CI, and then from staging then i have to hire a large devops team to manage it all -- super inefficient
- emptysongglass 3y ago
  It's frustrating neither Hocus or Gitpod support the devcontainers spec, which at this point should be a given if you're getting into this space.
siliconc0w 3y ago
Eh, might as well use the crazy power of that MacBook you're provisioning for each engineer.
swozey 3y ago
I know this is an article specific to hocus so I don't mean to take away from that but the title is much more open ended and plural so I'd like to ask this - What has been your favorite local dev environment? I personally docker-compose everything w/ .env overrides and I don't really feel like I've ever needed anything more than this.
- black3r 3y ago
  Docker Compose with .env overrides is very close to an ideal local dev environment, the only downside is if you don't use Linux you can hit some I/O bottlenecks (it got better with VirtioFS in Docker Desktop but still not native-speed).. I still wait for properly usable remote work environment that trumps local in performance and convenience.... I feel like the VS Code approach with server-client IDE architecture and the client running as a web application is going in the right direction but not quite there yet.., Especially if you can leverage lots of RAM and GPU power in your workflow and often work from different locations, don't want to carry your laptop around but still want to use the same power, environment & configuration on any machine you work from....
- CGamesPlay 3y ago
  How do you handle tooling? Install it in the parent or in the container? I quite like the idea of VS Code dev containers since it does a lot of the bootstrapping work to make the container be more like a full system, and I wish this “remote environment” idea was more widely supported amongst editors.
  - swozey 3y ago
    Hmm. By tooling are you referring to things like jq and other "uncommon" cli/tui apps I use to work against a code base locally? I think that's what you mean. One of the best practices when using docker-compose to develop is actually to mount your code base as a volume. This lets you modify the code locally without requiring a complete rebuild of the container each time code is modified. You might have to no-hup a service or run it not as pid1 (init container or supervisord) so it can restart without killing the container but that depends on the work being done. I'm trying to find a good example to show you but I'm not finding much. If you have a medium account this might be decent, I don't have one but it starts off going through the process https://medium.com/swlh/tutorial-how-to-use-docker-volumes-to-code-faster-245f15a60534 https://medium.com/swlh/tutorial-how-to-use-docker-volumes-t... With that said I use literally the exact same containers that are pushed to production.
ohthehugemanate 3y ago
TFA goes right into the core complexity and misses it. "Dev environment" depends on what kind of work you are doing. What a web developer needs is worlds apart from what a driver developer needs in terms of capability, resourcing, isolation, and rebuilds. Trying to offer a single environment for anyone who builds anything using a computer, is analogous to a single environment for anyone who uses a screwdriver. An electrician is using the same tool, sure, but they're doing fundamentally different work from a kitchen installer or a plumber and have vastly different needs. The single environment with all the capabilities that any computer developer needs is called a "computer." IMO this is the genius of VSCode remotes. They built a core capability of running the bulk of the IDE "somewhere else" with just the display layer on your local machine. Then they can allow the developer to choose if container(s), VM(s), and local or remote are right for their case, without having to learn new tooling. They can even let you choose if "owning a computer" is required, or if a web browser is sufficient.
rvdginste 3y ago
I prefer having a powerful development machine and install and use development environments natively. The only times I am prepared to use a (local or remote) VM is when the development environment cannot be installed on the OS of my development machine, or when I absolutely do not want to install those tools natively (because they might impact my main OS). I am often involved with multiple projects at the same time, but if those projects are set up well (and the toolset is decent), that does not cause any issues. In practice that means that developers always have to take into account that the project itself includes references to the specific versions of the tools that the project needs. And that those tools are -as-much-as-possible- installed in the context of those projects instead of in the context of the whole machine. A concrete example in node world is to use "nvm" and to always install tools locally in the context of a project instead of globally. In dotnet world, use global.json to specify the version of the tools you want to work with, install dotnet tools locally in the context of a project instead of globally. I have worked on projects where this is a problem. For example, projects that depend on hardcoded paths. Or projects that depend on hardcoded connection strings for a database. Or projects that do not use something like nvm to specify the version of node/npm they are compatible with. Each of those things is an issue with the setup of the project and is something that you will bump against when you try to setup a CI/CD pipeline for the project. Although, I agree that it can be useful to use docker containers with a correct version of the tools to create builds in a build pipeline. The reason that I prefer to develop natively is that it is faster, snappier and more enjoyable. I also believe that it is easier to diagnose issues and investigate performance. Aside from that, renting a virtual machine with the same performance as my development machine, would be a lot more expensive, and I would still have to deal with the lag. I am aware of recent changes that allow you to for example run a JetBrains IDE locally (similar to what VSCode already had for a long time?) that is connected with a remote headless version of the IDE. I do not have any experience with that kind of setup yet, but I assume that would remove most of the lag.
overnight5349 3y ago
I keep trying to use Jetbrains Gateway. It spins up the IDE on a remote machine and renders it locally. I honestly can't find a use for it. My personal machines are ancient quad core setups, but I have a beefy server at my disposal. But the benefit of faster build times isn't worth the hassle. Between the suboptimal performance of the remote interface, syncing files back to my local machine, and all of the little idiosyncrasies of setting up the remote environment, it's still faster to develop locally. I just go grab a glass of water when I start my compile. This might make more sense if you're working on some C++ monstrosity with millions of lines of code that takes hours to build on a mainframe, but if you're in the region of minutes per build, there's just no benefit. Or I guess if all you have is a thin client, this might be better than nothing. But then again, you can get an IDE and compiler for most languages on an android tablet. If you have a machine with a processor, it can compile your code.

113 points by hugodutka 3y ago

> Many cloud providers, like AWS inside VM-based EC2 instances, won't let you run a VM since they don't support nested virtualization Does anyone know if there's a technical limitation to nested virt on EC2 or if it's just so people use .metal? I know Azure recently (1-2 years?) started offering it.

colinchartier 3y ago
Nested virt doesn't work with some things like OS snapshot/restore, so they might not support it to allow those features.
segfaltnh 3y ago
I can't imagine amazon prefers the use of metal vs. non-metal. As I understand the pricing they're equivalent.

SkyMarshal 3y ago

Author doesn’t mention it but I wonder if tried or considered Nix/NixOS’s reproducible developer environments and ruled them out for any reason. I couldn’t tell from the article if there’s something unique to his requirements that disqualifies them. https://nixos.org/explore https://nixos.org/explore https://nix.dev/ https://nix.dev/

hugodutka 3y ago
Nix solves a different problem than Hocus. Nix lets you define a development environment, Hocus gives you a way to run it on a remote server. Right now we use Dockerfiles to let users define the packages they need in their dev env, but we would like to support Nix in the future too. Interestingly, you can use custom BuildKit syntax https://docs.docker.com/build/dockerfile/frontend/ https://docs.docker.com/build/dockerfile/frontend/ to build Nix environments with Docker https://github.com/reproducible-containers/buildkit-nix https://github.com/reproducible-containers/buildkit-nix, and that's probably what we will end up supporting.
- Nezteb 3y ago
  From what I can tell, devenv [1] and devbox [2] (both built on Nix) can also deploy to remote servers. [1] https://devenv.sh https://devenv.sh [2] https://www.jetpack.io/devbox https://www.jetpack.io/devbox
  - pxc 3y ago
    There are a lot of other ways to deploy to remote servers with Nix. Many of them are NixOS-based, but some don't require NixOS at all. Whatever way you're using Nix for developer environments, you can reuse most of that work to define a package suitable for remote deployment.
- midchildan 3y ago
  I think Nix is relevant here, because being able to run software across different machines reproducibly is one of its major selling point. I particularly like that it doesn't rely on virtualization or containerization to do that. It's up to the user to decide how to isolate the runtime environment from the host or whether they even should. Alternatively, tools building upon Nix can make that decision for them. Either way, it allows for a more flexible approach when you have to weigh the pros and cons of different isolation strategies. Development environments defined by Nix tend to compose well too, as a result of this design.
- debosmit 3y ago
  see this is the problem w/ all these devtools - i need to pair together 5 different things when i just want a reproducible, ephemeral environment someone needs to bring a heroku-like experience but for cloud-native development
  - suryao 3y ago
    That's the mission we're on at Argonaut. I'd love to know more about how you think about it if you're up for a chat.
    - fragmede 3y ago

nickstinemates 3y ago

I am curious how important 100GB dev environment support and efficiently binpacking dev environments and rat-holing on memory efficiency actually matters for the first X enterprise customers for Hocus. Memory and CPU cores are cheap. Getting the UX, resource allocation/sizing/instance type, and general pattern for multi-tenant dev boxes seems more fruitful. Would love to see any data supporting this prioritization, and what the answer is for Workstation / HEDT-type environments given the industry's current focus on AI.

pphysch 3y ago

This blog doesn't convince me why I should care about Hocus over solutions like VSCode devcontainer + Vagrant. That is more what it is competing with, not raw Docker/VM.

deleted 3y ago
[deleted]
hugodutka 3y ago
Oh yes, in this post I was not trying to. Hocus gives you a web interface that lets you spin up a dev env with a single click of a button. We also implemented a git-integrated CI system that prebuilds your dev env on new commits. It’s basically a self-hosted Gitpod or GitHub Codespaces.

stephenr 3y ago

This whole article is weird to me. I don't really understand the point of a "dev box" that's hosted in a DC and shared, at all - at least not the way they're painting it here. Hardware capabilities for even consumer level laptops and desktops have progressed much faster than average network connections. Having testing/preview/branch named environments in a DC? Sure. But this line: > They should be able to run any software they want inside their own workspaces without impacting each other. What does that even mean? Is this about someone working on a feature branch that uses some new dependency that needs to be installed? That's 100% the sort of thing your local development environment is for, until you're ready to push it to your hosted test/whatever environment. > But when it's running in a VM, the VM gobbles up as much memory as it can and, by itself, shows no inclination to give it back. Someone, anyone, tell me which hypervisor doesn't enforce a memory limit on VMs?

Terretta 3y ago
See VSCode ssh remote: https://code.visualstudio.com/docs/remote/ssh-tutorial https://code.visualstudio.com/docs/remote/ssh-tutorial See also: https://code.visualstudio.com/docs/devcontainers/containers https://code.visualstudio.com/docs/devcontainers/containers People pay for this. https://github.com/features/codespaces https://github.com/features/codespaces
- ajdude 3y ago
  This is exactly what I've been doing. I pay for a vps and just use vscode to do all of my work, I tend to switch between various computers and even various laptops, and it doesn't matter which one I'm on and just ssh into my dev station.
scarmig 3y ago
Enterprise likes it, because it gives them more visibility and control over what's going on in the dev box. That gives more angles to control against e.g. exfiltration threats. It also makes dev environments much more homogeneous: you worry less about client machines. In principle you could have a fleet of stateless Chromebooks whose main function is ssh and http, all connecting to identical virtualized desktops that are easily provisioned and deprovisioned.

foxbyte 3y ago

Consider exploring LXC for a more mature alternative to Docker. It's not confined to the OCI ecosystem and offers a higher degree of isolation for development environments.

seabrookmx 3y ago
I don't know if I'd use the word "mature" (Docker is quite mature after all), but as a long-time Docker user I did jump into lxc/lxd and can confidently say it's "system container" approach is better for dev environments. I used to use WSL 2 or a VM, but on a Linux host LXD is a really nice workflow. I can create a fully isolated Linux instance with all my dev tooling on it (optionally script/automate the install of all my tools), and with nested containers enabled I can even run _Docker in that LXD container_ so that when I type `docker ps` on that instance, and `docker ps` on my host, they each have their own set of containers running. For example, I have an ElasticSearch and Redis instance running in the dev box but Syncthing on the host. Then I can use snapshots to back up the dev environment, or blow it away completely once it gathers too much cruft all without affecting my host. It's really great. Pair it with VS Code remote SSH and you have a very feature rich setup with little effort.
m463 3y ago
LXC doesn't have a Dockerfile concept though. I think a Dockerfile as a recipe for an environment is pretty elegant. I've used LXC with proxmox and managing what's in a container is kind of like being a sysadmin.
- stephenr 3y ago
  There is an LXC provider for vagrant, which gets you the one-file concept, with the benefits that not everyone on the project has to use LXC, they just need a provider that works on their host for the specified base box. This is vagrants' real superpower.
- tiew9Vii 3y ago
  Docker is cattle, LXC is pets. Docker packages applications, and should only really have one app inside it. LXC is similar to a vm in it presents itself as a stand-alone machine but uses the host kernel, it’s not like a vm in it’s not as isolated as sharing the host kernel. So with LXC I agree, managing it is like being a sysadmin as that’s what it’s designed to be. I use Docker for things that are stateless, maybe throw away, or just test an app quickly. I use LXC for things I want to run multiple services inside, more statefull, typically where people plumb a bunch of Docker images together in I’ll use LXC. The advantage in Proxmox is I tell Proxmos to backup my LXC nightly as it’s treated similar to a VM. For making LXC feel less like needing to be a sysadmin, you can use Nix to build your LXC images and import in to Proxmox. Your LXC container becomes declarative and not to dissimilar to using a Dockerfile, it’s a far more powerful Dockerfile. What I’ve done is create a bare minimal NixOS LXC with some basic config and use that as a template then edit ‘/etc/nixos/configuration.nix’ inside the LXC on first boot. However as it’s just nixos you can build push the config remotely, use NixOps etc. It’s a really good workflow using NixOS with LXC however it took me a while to get it as the docs are a bit thin and an old+new version of docs with the new version skipping things mentioned in the old you need to do, I.e change the tty to /dev/console to get a shell inside proxmox console.

ravenstine 3y ago

Good god, no. Or, at least, let us never make this a sort of standard way of developing that you're going to push on to your engineers. Containers have their place, they can be great, but I've tried the whole "containerize everything" approach, and it's no less time consuming than not containers, and now you have another Thing You Have To Learn (TM). This isn't to say that virtualization is a bad thing for a development environment. It can be a very good thing, actually, but you don't need containers. A development environment, in my opinion, should be well organized, easy for a new developer to pick up, but also scrappy and adaptable to new and interesting situations. Containers, at least in their current conception, are antithetical to this. They're best kept to deployments or running things in a scenario where you know for sure you're not going to tinker with anything. I don't think I've worked on a software project where, at some point or another, I didn't need to reach under the hood in some unconventional way to investigate an issue or simulate some circumstance. Yes, you can do those things with containers, but now you have to deal with that complexity, which can get in the way by merely being confusing and indirect. Having given up on using containers for development, when I want to use virtualization, I just spin up minimalist instances of Debian under Qemu (using the UTM GUI). This way, I can run a Thing (TM) in isolation, easily open up a shell, or SSH into it from the host shell, or start a GUI from within it if I need to, or install however many services I want within it and open up ports to the host, or have shared folders, etc. etc. etc. But what if I need multiple of a Thing (TM) running at the same time? In that case, I just clone the VM. No Dockerfiles, no docker-compose.yml, no "volumes", no orchestration, no images failing to build because reasons, no qcow2 files blowing up with Docker For Mac (though that situation has improved), no pruning of orphaned containers/images, no worrying about whether your container should only be running a single process/service, and so on. The only drawback to this is that, if the host is Linux, then it's indeed kinda dumb to be running other Linux kernels in VMs. But hey, it's 2023 and the difference in performance may not be meaningful depending on what you're doing. I know that, on an M1 Mac, I can run graphical applications at monitor resolution with incredible speed (perhaps moreso with Parallels instead of Qemu), so I'd struggle to care about any performance drawback if my host was Linux on the same hardware. I'm sure that, for some, containers in development will be totally worthwhile. But if they don't seem worthwhile to you, dear reader, then that's probably because they wouldn't be.

therealmarv 3y ago

Gitpod, just use Gitpod. You can even run docker inside there.

debosmit 3y ago
yea, great for small projects but no good when you're trying to expand into enterprise capabilities -- I have to get one tool for dev, have that config diverge from CI, and then from staging then i have to hire a large devops team to manage it all -- super inefficient
emptysongglass 3y ago
It's frustrating neither Hocus or Gitpod support the devcontainers spec, which at this point should be a given if you're getting into this space.

siliconc0w 3y ago

Eh, might as well use the crazy power of that MacBook you're provisioning for each engineer.

swozey 3y ago

I know this is an article specific to hocus so I don't mean to take away from that but the title is much more open ended and plural so I'd like to ask this - What has been your favorite local dev environment? I personally docker-compose everything w/ .env overrides and I don't really feel like I've ever needed anything more than this.

black3r 3y ago
Docker Compose with .env overrides is very close to an ideal local dev environment, the only downside is if you don't use Linux you can hit some I/O bottlenecks (it got better with VirtioFS in Docker Desktop but still not native-speed).. I still wait for properly usable remote work environment that trumps local in performance and convenience.... I feel like the VS Code approach with server-client IDE architecture and the client running as a web application is going in the right direction but not quite there yet.., Especially if you can leverage lots of RAM and GPU power in your workflow and often work from different locations, don't want to carry your laptop around but still want to use the same power, environment & configuration on any machine you work from....
CGamesPlay 3y ago
How do you handle tooling? Install it in the parent or in the container? I quite like the idea of VS Code dev containers since it does a lot of the bootstrapping work to make the container be more like a full system, and I wish this “remote environment” idea was more widely supported amongst editors.
- swozey 3y ago
  Hmm. By tooling are you referring to things like jq and other "uncommon" cli/tui apps I use to work against a code base locally? I think that's what you mean. One of the best practices when using docker-compose to develop is actually to mount your code base as a volume. This lets you modify the code locally without requiring a complete rebuild of the container each time code is modified. You might have to no-hup a service or run it not as pid1 (init container or supervisord) so it can restart without killing the container but that depends on the work being done. I'm trying to find a good example to show you but I'm not finding much. If you have a medium account this might be decent, I don't have one but it starts off going through the process https://medium.com/swlh/tutorial-how-to-use-docker-volumes-to-code-faster-245f15a60534 https://medium.com/swlh/tutorial-how-to-use-docker-volumes-t... With that said I use literally the exact same containers that are pushed to production.

ohthehugemanate 3y ago

TFA goes right into the core complexity and misses it. "Dev environment" depends on what kind of work you are doing. What a web developer needs is worlds apart from what a driver developer needs in terms of capability, resourcing, isolation, and rebuilds. Trying to offer a single environment for anyone who builds anything using a computer, is analogous to a single environment for anyone who uses a screwdriver. An electrician is using the same tool, sure, but they're doing fundamentally different work from a kitchen installer or a plumber and have vastly different needs. The single environment with all the capabilities that any computer developer needs is called a "computer." IMO this is the genius of VSCode remotes. They built a core capability of running the bulk of the IDE "somewhere else" with just the display layer on your local machine. Then they can allow the developer to choose if container(s), VM(s), and local or remote are right for their case, without having to learn new tooling. They can even let you choose if "owning a computer" is required, or if a web browser is sufficient.

rvdginste 3y ago

I prefer having a powerful development machine and install and use development environments natively. The only times I am prepared to use a (local or remote) VM is when the development environment cannot be installed on the OS of my development machine, or when I absolutely do not want to install those tools natively (because they might impact my main OS). I am often involved with multiple projects at the same time, but if those projects are set up well (and the toolset is decent), that does not cause any issues. In practice that means that developers always have to take into account that the project itself includes references to the specific versions of the tools that the project needs. And that those tools are -as-much-as-possible- installed in the context of those projects instead of in the context of the whole machine. A concrete example in node world is to use "nvm" and to always install tools locally in the context of a project instead of globally. In dotnet world, use global.json to specify the version of the tools you want to work with, install dotnet tools locally in the context of a project instead of globally. I have worked on projects where this is a problem. For example, projects that depend on hardcoded paths. Or projects that depend on hardcoded connection strings for a database. Or projects that do not use something like nvm to specify the version of node/npm they are compatible with. Each of those things is an issue with the setup of the project and is something that you will bump against when you try to setup a CI/CD pipeline for the project. Although, I agree that it can be useful to use docker containers with a correct version of the tools to create builds in a build pipeline. The reason that I prefer to develop natively is that it is faster, snappier and more enjoyable. I also believe that it is easier to diagnose issues and investigate performance. Aside from that, renting a virtual machine with the same performance as my development machine, would be a lot more expensive, and I would still have to deal with the lag. I am aware of recent changes that allow you to for example run a JetBrains IDE locally (similar to what VSCode already had for a long time?) that is connected with a remote headless version of the IDE. I do not have any experience with that kind of setup yet, but I assume that would remove most of the lag.

overnight5349 3y ago

I keep trying to use Jetbrains Gateway. It spins up the IDE on a remote machine and renders it locally. I honestly can't find a use for it. My personal machines are ancient quad core setups, but I have a beefy server at my disposal. But the benefit of faster build times isn't worth the hassle. Between the suboptimal performance of the remote interface, syncing files back to my local machine, and all of the little idiosyncrasies of setting up the remote environment, it's still faster to develop locally. I just go grab a glass of water when I start my compile. This might make more sense if you're working on some C++ monstrosity with millions of lines of code that takes hours to build on a mainframe, but if you're in the region of minutes per build, there's just no benefit. Or I guess if all you have is a thin client, this might be better than nothing. But then again, you can get an IDE and compiler for most languages on an android tablet. If you have a machine with a processor, it can compile your code.