11 ms·

Developing inside a virtual machine

190 points by disintegrator 2y ago

moritonal 2y ago
Thr Dev Container ecosystem for VsCode really is quite impressive at the moment. All your dev dependencies, wrapped up in a docker image per repo.
- invalidname 2y ago
  100%. I've been using that and also the DevContainer support in IntelliJ/IDEA which is good but has some limitations (e.g. I can connect IntelliJ but not CLion at the same time).
- psyclobe 2y ago
  Heh if you can work around ALL its weird quirks, especially on windows.
  - elcritch 2y ago
    Having supported team members where we were all running this setup there was constant fiddling with docker. Containers would freeze and often required restarting docker directly. Especially with WSL on windows. It’s slick when it works.
    - moritonal 2y ago
      I noticed similar things, and found most these problems went away with more memory, and less usage of the "{}.features[]" in the `devcontainer.json` file.
  - urronglol 2y ago
    What quirks have you come across. The only annoyance for me was ssh agent having to uninstall the windows store version
jareklupinski 2y ago
is there a way to forward usb/serial ports from my local machine to the dev container? maintaining consistent firmware development environments using containers is a great idea, and current solutions involving proxying the compiled binary work well for flashing quickly, but switching back and forth between UART and Serial Debug is always more convenient when the IDE can handle it all
- nottorp 2y ago
  I've done linux/arm yocto development from VMs. The best solution to pass usb/serial was still VMWare the last time I had to.
- deivid 2y ago
  Not sure what your host OS is, if Linux, QEMU can pass usb devices to the guest. Otherwise you could pipe serial over TCP
  - jareklupinski 2y ago
    interesting, specifically I'm on a host Mac and trying to use the ESP IDF dev containers in VSCode (https://docs.espressif.com/projects/vscode-esp-idf-extension/en/latest/additionalfeatures/qemu.html https://docs.espressif.com/projects/vscode-esp-idf-extension...) will try again with sockets/TCP, thought i hit a roadblock after reading https://github.com/docker/for-mac/issues/5263#issuecomment-762955441 https://github.com/docker/for-mac/issues/5263#issuecomment-7...
- nneonneo 2y ago
  VMWare has an excellent implementation of this which can selectively connect devices to the guest, and it properly remembers the action for each device you connect to your computer.
- watermelon0 2y ago
  You mean forwarding to VSCode dev container? If you are using Docker Engine directly on Linux, you can forward a device to a container via docker-compose `devices` setting. If you are using Docker Desktop (or similar), there is no native way. However, there are ways to share USB devices via network (USB/IP is an open source implementation of this), in which case you setup server on host device (can be macOS/Windows/Linux), and then run a client software inside a container.
amelius 2y ago
Beats developing inside a docker container.
- cachvico 2y ago
  Does it though? I've developed in remote VMs before and the advantages are clear, but having a fully containerized development environment is really nice too because you can tear the whole thing down and rebuild at the drop of a hat. You can achieve that with a VM and scripts, but a Dockerfile is very lightweight and standard. Edit: Unless you literally mean "editing code in a container with vi". In which case yes I'd go for the VM too!
  - Multicomp 2y ago
    I am currently doing development on a VM with remote SSH, but I use the terminal on said VM to run a docker container when I need to actually run and build the thing, so it is possible to get both remote SSH tooling and containerization benefits, without needing to build a docker container and SSH into it from vs code, which might be what GP was saying.
  - amelius 2y ago
    Well, if what you're developing is an editor, you'll be editing inside the docker container either way ...
mongol 2y ago
I do something similar but using WSL on Windows. But something I really, really hate is dealing with special certificate handling required to pass the corporate Zscaler proxy. I think it works somewhat transparent on the Windows host, but repeating the setup in every VM is such a pain.
- chrisweekly 2y ago
  Given how much you hate it, any chance you documented how you did it?
  - mongol 2y ago
    No not really. The employer has some documentation, it is not complete but is starting point when issues pop up. For example, a JDK neeed to have special certificates installed if Java tries to talk SSL. And when you juggle with different JDKs for different Java versions it becomes a nightmare. The best you can do is to try not to touch anything when it eventually works but eventually something unforeseen breaks it anyways.
  - Atotalnoob 2y ago
    All you really need to do is install the zscaler cert in the appropriate trust store. It’s really a 1 step process in your dockerfile or other location.
- deergomoo 2y ago
  Heh, my employer is rolling out Zscaler this year. The limited trial a few months ago was hell for folks using WSL primarily, with Docker images adding an additional layer of pain. The people in the trial got very little done until it was decided to pause it, and I do not have high hopes for when it’s tried again. It strikes me as basically running malware in the name of security.
  - sieabahlpark 2y ago
    [dead]
  - 2y ago
dsfsaff 2y ago
I have used a Virtual Box VM with a Ubuntu guest for years and it has worked great. It's as close to the VM's in prod you can get.
- malux85 2y ago
  I used to do that, but now that all of our microservices are dockerized every microservice has its own docker container Vscode supports remote containers, so everyone in the org just develops INSIDE a replica of the prod container All containers run remotely on enormous machines with 800+ GB of RAM and 8+ GPUs It’s trivial to share environments now because you just open the project and the dev container starts up and installs all the deps, devcontainer.json is just a few kb so just check it into git Engineering, DevOps, Data science all use this setup, push around your devcontainer.json and everyone gets the same GPU accelerated dev environment with near unlimited RAM and hundreds of CPU cores, none of this hardware is local so you can code on the balcony/beach on your MacBook Air, light and easy to travel with. We put VMs in the same country as our staff so latency has never been an issue This is the dev setup I’ve wanted for ages, and it’s a joy to use
  - eikenberry 2y ago
    What do devs who don't use VSCode do to work in this environment?
    - MawKKe 2y ago
      VScode devcontainer can build from existing Dockerfile. You can develop the project image as usual, and then reference the Dockerfile from devcontainer.json. This means you can build and run from the command line via `docker` command if needed. The VScode extension just makes this slightly easier. Not sure how GP's company does it, but that is how I would configure it. Caveat: the default devcontainer initialization workflow does _not_ create the Dockerfile, only the .json. At $work we don't use devcontainer.json, but we can launch the development environment image such that you can SSH into it as if it was a regular VM.
    - malux85 2y ago
mrbluecoat 2y ago
If you like NixOs and virtual development environments, perhaps try https://www.jetify.com/devbox https://www.jetify.com/devbox or https://flox.dev/ https://flox.dev/
tkiolp4 2y ago
Why does the author need a “remote ssh” plugin in their VSCode? I usually develop inside a VM as well, with my IDE running in the host… but what I do is to mount a shared directory for the code between the host and the VM. Works pretty fast. Don’t understand the need for Tailscale either. When I’m running services or dbs inside the VM, I can easily access them if needed from the host (either by IP or by the hostname I gave to the VM on start up)
- kevingadd 2y ago
  SSH remote in VS code has way better latency and performance characteristics than mounting a shared directory. Stuff like disk change monitoring also works a lot better. The one mixed/negative thing is that language servers will run inside the VM instead of the host where the editor is "running", which can defy your expectations. I find that a plus since language servers love to tie up multiple cores and eat up memory and having that happen inside the constrained VM environment stops my host system from getting bogged down. I used to edit in a shared mount before and moving to the vscode ssh remote model was a noticeable improvement. It's just faster.
  - kijin 2y ago
    SSH remote absolutely rocks. It's the #1 reason I chose VS code instead of some other editor with an SFTP plugin. I'm working on a Windows host with a bunch of Linux VMs. Although I can share directories between the host and guests, I prefer to rely on SSH remote because I want to work in the VM's filesystem and its environment. For example, I don't want to care which version of python and what kind of libraries are installed on the host. The VM is supposed to be a container for all that stuff, and different projects have different requirements.
  - 1718627440 2y ago
    What would prevent you from mounting your directory with ssh?
- askonomm 2y ago
  So how does your IDE pick up on the tooling inside the VM? E.g if you build Python projects, how does it pick up the Python executable, .venv, etc? Or if PHP, then the PHP runtime, or if C then its stuff ... etc? If you install these on your host machine to make your IDE work well then I'm afraid that defeats the point of having a VM.
nneonneo 2y ago
You can use pbcopy/pbpaste in a Linux VM on Mac by making a shell script wrapper in the VM that calls “ssh mac-host pb{copy|paste}” - that is, basically ssh back from the guest to the host to use its clipboard. It’s seamless and fast since it’s basically a local network connection. My specific setup is that I use an authorized_keys entry on the host that restricts the guest to running a specific command, which limits what a compromised guest can do to the host. The command is set to a script that has a list of specific permitted actions. This is a good option if you’re looking for a bit of additional isolation between host and guest.
- disintegrator 2y ago
  Brilliant tip! I’m going to give it a shot tomorrow and update the post (with attribution).
- hamandcheese 2y ago
  In the past, I set up something similar, except I would reverse forward my local ssh port to my remote servers (so that I could easily ssh back regardless of network topology). Ultimately I didn't keep it out of security concerns -- I had done nothing to limit the commands. On the topic of limiting the possible commands - for my use case I only needed pbcopy. Maybe think twice before letting an insecure VM or remote host read your clipboard contents with pbpaste.
  - nneonneo 2y ago
    Yep - good point. Another option would be to set a confirmation in front of every paste attempt - for example, putting Touch ID in front of any pbpaste call from the guest (which you can enforce with the authorized_keys command). That should be low-friction enough that it isn’t a major delay to your development process, while still being reasonably secure and providing the convenience of pasteboard access.
- fulafel 2y ago
  Having that kind of ssh access from guest to host negates the security barrier benefits that using a dev VM might have.
firesteelrain 2y ago
We have been developing code inside VMs for over 15 years at my company with various flavors of VMware on the backend and lately more moving to Azure. I assumed this was normal. We reach our VMs via VDI.
- regularfry 2y ago
  I have heard VDI described variously as "an abomination", "unusable except when you absolutely have no other option", and "don't". That might be down to the implementations in play at the time, though. I most often see this sort of thing where corporate IT can't stomach devs getting root on their own machines. It's a very specific sort of corporate dysfunction.
  - firesteelrain 2y ago
    lol yep no root here But, I am in an airgapped environment that is tightly regulated if you get it.
disintegrator 2y ago
Author here. Thank you for all the tips. I especially like the idea of using ssh from guest to host to enable pbcopy/pbpaste and open. Now I know what all the WSL users experience seamlessly with their setups. Glad I have something that comes close.
- StreakyCobra 2y ago
  Thanks for the post, an interesting read! Side note: I checked out your other blog post, and it resonates with my own first post, which I wrote just two days ago: https://fabiendubosson.com/blog/overcoming-perfectionism/ https://fabiendubosson.com/blog/overcoming-perfectionism/. You’re definitely not alone in battling anxiety, perfectionism, and procrastination when it comes to blogging. Keep writing! :)
  - disintegrator 2y ago
    Thanks! Really appreciate your comment :)
tonymet 2y ago
Windows 11 and WSL manages this well . For those developing linux apps & containers using VS Code, you'll find the Windows 11 experience to be very good. You can code against WSL which offers the more popular distros, or use HyperV to run your own custom VMs.
- makeitdouble 2y ago
  In general, yes. One weird quirk: networking can be peculiar. Windows creates a magic bridging between the host and WSL, and as anything magic, it can break for specific use cases. VPN is one [0]: my WSL instances lose outgoing networking when connecting to our company VPN. There are workarounds but none are trivial. [0] https://superuser.com/questions/1715764/wsl2-has-no-connectivity-when-windows-is-connected-to-vpn https://superuser.com/questions/1715764/wsl2-has-no-connecti...
  - tonymet 2y ago
    Good to know. My vpn works but I believe it’s wireguard-based. I wonder if yours is TUN/TAP or another driver
    - makeitdouble 2y ago
      It seems to be TAP (layer 2 tunneling) Thinking about it, as the whole machine is under MDM (I only have the VPN on my dedicated work machine), there might be additional quirks that mess with the networking as well. Even bridging the Wi-fi to it was kind of a PITA.
  - k8sToGo 2y ago
    This has been fixed for many months now as you can switch between different networking types for WSL. https://learn.microsoft.com/en-us/windows/wsl/networking#mirrored-mode-networking https://learn.microsoft.com/en-us/windows/wsl/networking#mir...
raihansaputra 2y ago
Are you installing the project dependencies on the VM directly or in a docker container? I'm curious how well docker on top of the Ubuntu vm works. Orbstack is great for personal use, but some companies don't want to pay for it, and this might be an alternative to have a better docker experience on macOS.
- UltraSane 2y ago
  docker in a VM works fine because docker isn't a VM and just uses Linux features like pivot-root, namespaces, and cgroups to isolate programs. At least on x86 CPUs you can even do nested virtualization if the CPU supports it.
- arkh 2y ago
  > I'm curious how well docker on top of the Ubuntu vm works. Works well enough. Better than Docker on barebone MacOS if you have lot of file access in a volume. That's one of the thing which surprised me when I started using VMs to develop. First time was with a postgres backed app: I expected to lose performance when moving everything in a VM. But got the exact opposite result at the time. Postgres really liked the linux filesystem more than the windows one, enough to do more than offset the VM tax.
- disintegrator 2y ago
  I work directly in my guest os and clone my projects and run them in there. There are some projects that are driven through docker-compose and that works nicely. The one caveat is that I had to install `apt install binfmt-support qemu-user-static` so that docker can run x86_64 images on my arm64 VM.
tonymet 2y ago
For the rest of you, don't be fooled by Darwin, it's a dusty BSD in Linux clothing
mmwelt 2y ago
This all looks fine for developing server apps that don't need a GUI, particularly as long as 3D accelerated graphics aren't needed. You don't even need to be using/developing a 3D game or application, just using a modern GUI without too much lag now seems to require 3D acceleration.
- weikju 2y ago
  3D acceleration is pretty well supported in VMware and sorta works well in UTM
  - 01HNNWZ0MV43FF 2y ago
    Complete pain in the butt for anything else like qemu
    - weikju 2y ago
      UTM's backend is qemu, isn't it? so one could check what they're doing
    - kwk1 2y ago
      It's pretty manageable with libvirt + virt-manager sitting on top of QEMU, though.
apt-apt-apt-apt 2y ago
I accidentally typed 'npm install axioss' (extra s typo) this morning. When it successfully installed, it was terrifying to think that all source code, private files were instantly shared with malicious actors. Not only that, there was the prospect of having to somehow wipe and ensure all files were clean, reinstall the OS, and the possibility of some bootloader remnant still lurking. In this case, it seems that a security package had replaced a previous malicious package, making this instance benign. But it feels like I am only one typo away from an absolute catastrophe every time I install a package. VM seems like a good way to add some protection.
- jeswin 2y ago
  > VM seems like a good way to add some protection. Yeah, but someone should try to fix this anyway. It's not a nodejs-specific problem, but it's badly needed in node. Any of the 100s of authors whose packages I depend on might have made a typo, or just been careless. Software development requires a scary level of trust. I am also increasingly moving to VMs. I want tools (such as VSCode) to run on the main machine, but actual execution to happen in the vm. It's a bit painful and a drag on productivity, especially debugging.
  - skydhash 2y ago
    The one trick I found that work well is to move everything in the VM. I usually opt for either emacs or vim, and if I need an IDE, I install i3. It just takes a moment to copy my dotfiles over.
    - quectophoton 2y ago
      I'm doing something similar. My development environment for work is defined in a Dockerfile, and I have a small shell script whose only purpose is to call `docker run` with that image, mount a few volumes for caching, mount the CWD in "/workspace", and start a shell in there. Development is done with nvim. If I need Docker Compose, I run it from the host. For projects that I find unpleasant to work with in this way, I use GitHub Codespaces. I hadn't thought about using a GUI IDE from within a VM, so thanks to your comment (EDIT: and also the submitted article) I now have something new to try!
hrtk 2y ago
Have you tried Lima?
zokier 2y ago
I use similar setup on Windows (with vmware/virtualbox/hyper-v at different times), which kinda highlights one additional upside: it doesn't matter that much what the host system is, you can do your work all the same regardless if it is macos/windows (or even linux). As long as it can run the vm and vscode, you are good to go. Although admittedly Apple going with aarch64 throws a small wrench in the equation. It is especially nice in corporate environment, where the host system is generally managed by IT and the devices are largely impersonal (standardized configuration, standardized software). You can carve out a corner to make your own and work there. <insert rant on ineffective corporate IT>
TowerTall 2y ago
I run my entire work computer inside of a VM. I work from home and have a powerfull desktop i use for my private stuff hooked up to my 3 monitors. My work pc is a vmware vm running inside of vmware pro. I can minimize the work vm and work it out my sight or I can selective choose that the vm should use 1, 2 or 3 monitors and it is easy to switch back and forth between work and private without have any work related data on my private desktop. The work vm is domain joined, o365 enterprise joined and loocked down in ridicilous ways by cooperate IT but now i can run that from my private powerfull PC without worring that corp IT messes up my private computer.
- vladvasiliu 2y ago
  I was thinking of doing something similar, especially since work mostly insists on running Windows. Do you use conferencing software or such from the VM? If so, how's the performance?
  - TheTxT 2y ago
    I did the same thing for many months running teams. It was about what you expect from teams. I didn’t have any other significant issues, but eventually stopped using this setup due to increasing security requirements of the VPN software.
  - TowerTall 2y ago
    I started working from home around 12 years ago and have been using this setup in various incarnations. I experience zero issues related to running work from inside of a VM. I work for an MS Shop, so we are using teams. The main issue with my setup is that VMware or Windows 11 (my host OS) can't use the GPU when rendering the UI of the VM (I'm not sure if VMware or MS is to be blamed or both) despite having a discrete GPU card installed. The rendering of the image must all be done by the CPU, which requires a lot of RAM. After switching to 3 x 4K monitors, the VM requires 62GB of RAM to be able to run on full screen on all monitors (63720 MB to be precise). I recall that I somehow managed to get it working while "only" using around 32GB, but it became unstable. 62GB is the sweet spot where everything runs smoothly. Haven't tried to adjust the settings in years. It was a pain to get working in the first place with 4k monitors, and since I have 256 GB installed, I just left the settings as they are. At next motherboard upgrade I might revisit, but I think that VmWare still can't take advantages of the host GPU in this regard so I expect the RAM requirement to stay. Some computer problems can be solved by throwing hardware at it. This is one of those. Give VMware and Windows an obscene amount of RAM, and you can have Teams running smoothly and flawlessly inside a VM on a Windows Client Host in 4K.
sushidev 2y ago
Started using code-server on a remote server. Pretty good. Going to switch working like that. Its a bit like remote vscode only that the vscode ui is also remote and served via a web browser. Coming from intellij I was surprised that the user experience in terms of responsiveness is actually better with the remote setup. IntelliJ these days just lagging on everything.
arkh 2y ago
> I’ve found developers frowning up Ubuntu and preaching for folks to use NixOS, Arch, Debian or other distros. My setup is mostly one VM per project group / online identity. Most of them using Ubuntu. The problem is when I want to work on an old project to check how it likes new technology I tend to stumble into the "you should have kept the OS up to date" problem. Ubuntu does not make it easy to upgrade if you miss more than a year of update. And even if you keep up to date, they tend to break things often (loved the X11 to weyland switch when working with screen capture libraries) so new VMs are using debian.
- Asooka 2y ago
  Don't you get the same problem with upgrades with Debian? As for Wayland, seeing the progress over the last 17 years, I estimate it will be ready for regular use sometime during the 41st millennium.
  - thequux 2y ago
    With debian, is you develop on stable, you only need to do the upgrade song and dance every year and a half or so and upgrades rarely break anything. If you develop on unstable, you can use the snapshot archive to either upgrade 6 months at a time or move back to the next stable and then walk through stable releases.
  - arkh 2y ago
    > Don't you get the same problem with upgrades with Debian? I don't think you have to do some manual configuration to upgrade your distribution because it is a couple years old and current scripts don't support that (like going from a 22.10 to 24.04 is a fun game).
Abishek_Muthian 2y ago
I do a switcharoo, I develop for iOS inside a macOS VM with Linux host. After a 5 years hiatus I started developing mobile apps again and I was frustrated to learn that Apple doesn't allow renewing the developer license on web anymore, I don't own a Mac and even Apple developer app on iPhone didn't allow me to renew my license. After I signed into a macOS VM, I was able to renew my license through Apple Developer App on iPhone as macOS version of the app requires T2 chip. Now I have PTSD flashbacks of why I left mobile development in first place.
- theoreticalmal 2y ago
  Can you share more info on your MacOS VM setup? I’ve managed to set one up precisely once with proxmox, and getting iCloud/imessage to work required me to contact Apple support
  - Abishek_Muthian 2y ago
    I used Kholia's scripts[1] on Qemu with Virsh. I didn't have much trouble other than some SELinux permission issues which I resolved quickly. iCloud/imessage have always been finicky with Hackintosh, but in my experience setting the correct serial number with appropriate Mac model is the key to resolve those issues. With just couple of years of Hackintosh scene left before support for x86 Macs are dropped completely, a VM Hackintosh makes more sense to me than building a physical Hackintosh. [1]https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM
    - LeFantome 2y ago
      Even a VM will stop working soon as Intel support is removed from the OS and apps start demanding newer OS versions. Or is it possible to emulate Apple Silicon on a VM now?
      - Abishek_Muthian 2y ago
        Yes macOS VM will also be useless too in some years but a useless VM is better than a useless purpose built computer for hackintosh. Qemu is capable of ARM emulation but I don't think its capable of emulating sophisticated SoC like Apple Silicon.
mootoday 2y ago
Pretty sure you'd have a more lightweight experience with https://www.jetify.com/devbox https://www.jetify.com/devbox. Happy to set it up and demo if you can share (or DM) a repo URL.
- rcarmo 2y ago
  That sets up a VM with Docker as well.
  - mootoday 2y ago
    No
    - rcarmo 2y ago
      You can't run Docker on a Mac without a VM.
rcarmo 2y ago
I use Proxmox on my LAN and RDP connections to Linux and Windows desktops of various kinds. It’s great.
pshirshov 2y ago
I use Nix for exactly the same purpose (dependency management for code generators), that's much more efficient and easier to maintain than VMs.
Too 2y ago
Any benefit of using a VM over docker container here? Since you seem to use the terminal only, without any graphical applications. Containers should be more lightweight and dockerfiles allow quick and reproducible changes to the guest OS.
- kasey_junk 2y ago
  Docker uses virtualization on Macs
TacticalCoder 2y ago
Curiosity: is anyone here developing inside a VM, with GPU passthrough, and with the monitor directly connected to the GPU used by the VM? (as in: showing the UI of the VM, without any need for RDP or the like) Such a setup works (I'd know for I have one at home doing just that but it's not my main PC) but how's it like to work like that? The GPU hooked to the hypervisor can either be on another monitor or on another input (in the latter case you'd "go" to the hypervisor by changing the monitor's input).
- thehamkercat 2y ago
  I have a 7800X3D with 64GB Ram, it's overkill for programming So I've installed proxmox on it to utilize it's resources for other stuff as well For my personal use, I created an Arch VM with GPU and USB PCIE passthrough, all 3 monitors directly connected to that GPU It's so seamless and fast that I don't even feel I'm working inside a virtual machine I have other headless VMs hosted to do other things (opensense etc)
- TheTxT 2y ago
  I used such a setup for gaming using a windows and a Linux (later proxmox) host. With everything passed through you basically don’t realize that you’re sitting in front of a VM, it’s great.
lizknope 2y ago
Are most companies this flexible in allowing developers to install whatever they want? > My physical machine is a 2023 MacBook Pro with M2 Pro CPU > I’m using VMWare Fusion Pro > Quite often I’ve found developers frowning up Ubuntu and preaching for folks to use NixOS, Arch, Debian or other distros. The reality for me was Ubuntu was the fastest way to get set up and now I'm in integrated circuit / semiconductor design. At every big company over the last 30 years we are given a computer and we can change the desktop environment but we aren't installing our own operating system. The people I know in software have a common OS, compiler, and build environment. They aren't dictating what text editor you use but you aren't working on projects individually but together. So if everyone at the author's company is doing their own thing do they have problems integrating all the code together? "Oh you used version 2.3.4 but I used version 2.4.7 which fixed this issue, what are we using to ship with?" Or is this not a problem?
- IsTom 2y ago
  There's typically a list of dependencies of a specific project as a part of it, (hopefully) handled automatically by some part of build pipeline.
  - whatevaa 2y ago
    Build pipelines don't have much to do with actual development. What, do you edit stuff but never build it locally?
    - IsTom 2y ago
      I should have written "build process" instead I suppose.
- bluehatbrit 2y ago
  I work mostly in web / server side development, it's not really a problem I've had for a number of years now. Some of my colleagues use various linux distros, others macos. No one is using Windows that I know of. Each project we have requires a specific tool chain version (python, elixir, ...) and specific versions of things like postgres. All dependencies are listed with some kind dependency definition file (pyproject.toml, package.json, mix.exs). If you bump a package it's done in the definition file as part of your changes and goes through CI for packaging and releasing. The rest of the team will get the new package version as soon as they pull your changes and run `just deps` or whatever. CI is the ultimate determining factor of whether your code actually "works". We also package and deploy with containers, but this isn't the real determining factor for any of the above.
urronglol 2y ago
Devcontainers running as non root. Trivial t set up. Don’t need to fanny around with a Vm
- disintegrator 2y ago
  I've tried devcontainers in the past and the performance compared to my current setup was pretty bad. This was a few years ago when it was known that filesystem-heavy workloads on Docker for Mac were sub-optimal. I remember having to define several bind mounts which improved the overall performance. I do intend to revisit this solution next time I need to set up a dev environment but rest assured there was nothing substantially more complex about a VM versus devcontainers.
  - urronglol 2y ago
    So why are you commenting on it now as if it hasn’t changed or evolved since then? Give it another try now. It is really good.
  - bluehatbrit 2y ago
    I use it at the moment and don't really find any noticable difference between running directly on my host, and in a dev container. If I were to measure the performance I'm sure there would be something, but it's not noticable in my development cycle. They also seem to be pushing it beyond vscode and into something which is editor agnostic. It's not quite there yet on that front, but I'm excited for it as I've been dabbling with other editors recently which don't support devcontainers directly and it always pulls me back to vscode. It's on a journey for sure, but I've had no performance issues when using it straight out of the box over the past year.
secondcoming 2y ago
If you're using a personal device for developing your employer's code always use a VM. You can just nuke the whole thing when your employment ends and not have to worry about any of their IP remaining on your machine.
cranium 2y ago
(Tangentially related) I had to run the desktop version of Excel to develop a quick VBA macro for a client. Problem: I've been developing on a Linux box for years and the idea of leaving my cozy dev environment for a plain Windows install gave me chills. After failing to install Windows in a VM (thanks TPM), I found a way to run Windows apps nearly natively (https://github.com/winapps-org/winapps https://github.com/winapps-org/winapps). It works by starting a Windows docker image and streaming the application frame with RDP. As the RDP client handles the copy/paste and other niceties such as shared directories, it's way easier to integrate in my env than the other options.
- rescbr 2y ago
  This is great!
gbraad 2y ago
Consider using tailscale drives to expose certain folders using webdav. Have been doing this for de containers and VMs since it got demoted and since has replaced using winscp or other ways to share files
- disintegrator 2y ago
  Just trying this out and it seems amazing so far. Thanks for the tip!
pjmlp 2y ago
At work, development with VMs has been a given since the early days of Amazon EC2 in 2010. Likewise when Windows 7 came out, I stop bothering with dual booting hassles and using VMare Workstation instead for whatever Linux. The exception being a netbook from the Asus Linux netbooks glory days, a price category nowadays replaced by tablets.
crabbone 2y ago
But why tho? The only kind of plausible explanation the author gives is that it's "more secure" because the imaginary attacker will have to take an extra step to get the password from the VM instead of the host OS? -- This seems like such an inconsequential / worthless benefit to jump through the hoops of running things in a VM... Like... I wasn't sold on this approach from the get go, and this pitch makes it sound like I was right all along? Other non-starter "bonuses" include not installing developer tools on your laptop that you have for... drum roll... development. Why? It's sole purpose is to be used for development, why not install development tools on it? Just doesn't make any sense...
nkko 2y ago
An alternative approach we're working on is Daytona (https://github.com/daytonaio/daytona https://github.com/daytonaio/daytona) - it orchestrates dev environments on your infrastructure or even on your local machine. Oh, and it is Apache 2.0. The interesting bit is how it handles any IDE integration and various providers for running your dev envs on. Disclosure: I work on this project

190 points by disintegrator 2y ago

Thr Dev Container ecosystem for VsCode really is quite impressive at the moment. All your dev dependencies, wrapped up in a docker image per repo.

invalidname 2y ago
100%. I've been using that and also the DevContainer support in IntelliJ/IDEA which is good but has some limitations (e.g. I can connect IntelliJ but not CLion at the same time).
psyclobe 2y ago
Heh if you can work around ALL its weird quirks, especially on windows.
- elcritch 2y ago
  Having supported team members where we were all running this setup there was constant fiddling with docker. Containers would freeze and often required restarting docker directly. Especially with WSL on windows. It’s slick when it works.
  - moritonal 2y ago
    I noticed similar things, and found most these problems went away with more memory, and less usage of the "{}.features[]" in the `devcontainer.json` file.
- urronglol 2y ago
  What quirks have you come across. The only annoyance for me was ssh agent having to uninstall the windows store version

jareklupinski 2y ago

is there a way to forward usb/serial ports from my local machine to the dev container? maintaining consistent firmware development environments using containers is a great idea, and current solutions involving proxying the compiled binary work well for flashing quickly, but switching back and forth between UART and Serial Debug is always more convenient when the IDE can handle it all

nottorp 2y ago
I've done linux/arm yocto development from VMs. The best solution to pass usb/serial was still VMWare the last time I had to.
deivid 2y ago
Not sure what your host OS is, if Linux, QEMU can pass usb devices to the guest. Otherwise you could pipe serial over TCP
- jareklupinski 2y ago
  interesting, specifically I'm on a host Mac and trying to use the ESP IDF dev containers in VSCode (https://docs.espressif.com/projects/vscode-esp-idf-extension/en/latest/additionalfeatures/qemu.html https://docs.espressif.com/projects/vscode-esp-idf-extension...) will try again with sockets/TCP, thought i hit a roadblock after reading https://github.com/docker/for-mac/issues/5263#issuecomment-762955441 https://github.com/docker/for-mac/issues/5263#issuecomment-7...
nneonneo 2y ago
VMWare has an excellent implementation of this which can selectively connect devices to the guest, and it properly remembers the action for each device you connect to your computer.
watermelon0 2y ago
You mean forwarding to VSCode dev container? If you are using Docker Engine directly on Linux, you can forward a device to a container via docker-compose `devices` setting. If you are using Docker Desktop (or similar), there is no native way. However, there are ways to share USB devices via network (USB/IP is an open source implementation of this), in which case you setup server on host device (can be macOS/Windows/Linux), and then run a client software inside a container.

amelius 2y ago

Beats developing inside a docker container.

cachvico 2y ago
Does it though? I've developed in remote VMs before and the advantages are clear, but having a fully containerized development environment is really nice too because you can tear the whole thing down and rebuild at the drop of a hat. You can achieve that with a VM and scripts, but a Dockerfile is very lightweight and standard. Edit: Unless you literally mean "editing code in a container with vi". In which case yes I'd go for the VM too!
- Multicomp 2y ago
  I am currently doing development on a VM with remote SSH, but I use the terminal on said VM to run a docker container when I need to actually run and build the thing, so it is possible to get both remote SSH tooling and containerization benefits, without needing to build a docker container and SSH into it from vs code, which might be what GP was saying.
- amelius 2y ago
  Well, if what you're developing is an editor, you'll be editing inside the docker container either way ...

mongol 2y ago

I do something similar but using WSL on Windows. But something I really, really hate is dealing with special certificate handling required to pass the corporate Zscaler proxy. I think it works somewhat transparent on the Windows host, but repeating the setup in every VM is such a pain.

chrisweekly 2y ago
Given how much you hate it, any chance you documented how you did it?
- mongol 2y ago
  No not really. The employer has some documentation, it is not complete but is starting point when issues pop up. For example, a JDK neeed to have special certificates installed if Java tries to talk SSL. And when you juggle with different JDKs for different Java versions it becomes a nightmare. The best you can do is to try not to touch anything when it eventually works but eventually something unforeseen breaks it anyways.
- Atotalnoob 2y ago
  All you really need to do is install the zscaler cert in the appropriate trust store. It’s really a 1 step process in your dockerfile or other location.
deergomoo 2y ago
Heh, my employer is rolling out Zscaler this year. The limited trial a few months ago was hell for folks using WSL primarily, with Docker images adding an additional layer of pain. The people in the trial got very little done until it was decided to pause it, and I do not have high hopes for when it’s tried again. It strikes me as basically running malware in the name of security.
- sieabahlpark 2y ago
  [dead]
- 2y ago

dsfsaff 2y ago

I have used a Virtual Box VM with a Ubuntu guest for years and it has worked great. It's as close to the VM's in prod you can get.

malux85 2y ago
I used to do that, but now that all of our microservices are dockerized every microservice has its own docker container Vscode supports remote containers, so everyone in the org just develops INSIDE a replica of the prod container All containers run remotely on enormous machines with 800+ GB of RAM and 8+ GPUs It’s trivial to share environments now because you just open the project and the dev container starts up and installs all the deps, devcontainer.json is just a few kb so just check it into git Engineering, DevOps, Data science all use this setup, push around your devcontainer.json and everyone gets the same GPU accelerated dev environment with near unlimited RAM and hundreds of CPU cores, none of this hardware is local so you can code on the balcony/beach on your MacBook Air, light and easy to travel with. We put VMs in the same country as our staff so latency has never been an issue This is the dev setup I’ve wanted for ages, and it’s a joy to use
- eikenberry 2y ago
  What do devs who don't use VSCode do to work in this environment?
  - MawKKe 2y ago
    VScode devcontainer can build from existing Dockerfile. You can develop the project image as usual, and then reference the Dockerfile from devcontainer.json. This means you can build and run from the command line via `docker` command if needed. The VScode extension just makes this slightly easier. Not sure how GP's company does it, but that is how I would configure it. Caveat: the default devcontainer initialization workflow does _not_ create the Dockerfile, only the .json. At $work we don't use devcontainer.json, but we can launch the development environment image such that you can SSH into it as if it was a regular VM.
  - malux85 2y ago

mrbluecoat 2y ago

If you like NixOs and virtual development environments, perhaps try https://www.jetify.com/devbox https://www.jetify.com/devbox or https://flox.dev/ https://flox.dev/

tkiolp4 2y ago

Why does the author need a “remote ssh” plugin in their VSCode? I usually develop inside a VM as well, with my IDE running in the host… but what I do is to mount a shared directory for the code between the host and the VM. Works pretty fast. Don’t understand the need for Tailscale either. When I’m running services or dbs inside the VM, I can easily access them if needed from the host (either by IP or by the hostname I gave to the VM on start up)

kevingadd 2y ago
SSH remote in VS code has way better latency and performance characteristics than mounting a shared directory. Stuff like disk change monitoring also works a lot better. The one mixed/negative thing is that language servers will run inside the VM instead of the host where the editor is "running", which can defy your expectations. I find that a plus since language servers love to tie up multiple cores and eat up memory and having that happen inside the constrained VM environment stops my host system from getting bogged down. I used to edit in a shared mount before and moving to the vscode ssh remote model was a noticeable improvement. It's just faster.
- kijin 2y ago
  SSH remote absolutely rocks. It's the #1 reason I chose VS code instead of some other editor with an SFTP plugin. I'm working on a Windows host with a bunch of Linux VMs. Although I can share directories between the host and guests, I prefer to rely on SSH remote because I want to work in the VM's filesystem and its environment. For example, I don't want to care which version of python and what kind of libraries are installed on the host. The VM is supposed to be a container for all that stuff, and different projects have different requirements.
- 1718627440 2y ago
  What would prevent you from mounting your directory with ssh?
askonomm 2y ago
So how does your IDE pick up on the tooling inside the VM? E.g if you build Python projects, how does it pick up the Python executable, .venv, etc? Or if PHP, then the PHP runtime, or if C then its stuff ... etc? If you install these on your host machine to make your IDE work well then I'm afraid that defeats the point of having a VM.

nneonneo 2y ago

You can use pbcopy/pbpaste in a Linux VM on Mac by making a shell script wrapper in the VM that calls “ssh mac-host pb{copy|paste}” - that is, basically ssh back from the guest to the host to use its clipboard. It’s seamless and fast since it’s basically a local network connection. My specific setup is that I use an authorized_keys entry on the host that restricts the guest to running a specific command, which limits what a compromised guest can do to the host. The command is set to a script that has a list of specific permitted actions. This is a good option if you’re looking for a bit of additional isolation between host and guest.

disintegrator 2y ago
Brilliant tip! I’m going to give it a shot tomorrow and update the post (with attribution).
hamandcheese 2y ago
In the past, I set up something similar, except I would reverse forward my local ssh port to my remote servers (so that I could easily ssh back regardless of network topology). Ultimately I didn't keep it out of security concerns -- I had done nothing to limit the commands. On the topic of limiting the possible commands - for my use case I only needed pbcopy. Maybe think twice before letting an insecure VM or remote host read your clipboard contents with pbpaste.
- nneonneo 2y ago
  Yep - good point. Another option would be to set a confirmation in front of every paste attempt - for example, putting Touch ID in front of any pbpaste call from the guest (which you can enforce with the authorized_keys command). That should be low-friction enough that it isn’t a major delay to your development process, while still being reasonably secure and providing the convenience of pasteboard access.
fulafel 2y ago
Having that kind of ssh access from guest to host negates the security barrier benefits that using a dev VM might have.

firesteelrain 2y ago

We have been developing code inside VMs for over 15 years at my company with various flavors of VMware on the backend and lately more moving to Azure. I assumed this was normal. We reach our VMs via VDI.

regularfry 2y ago
I have heard VDI described variously as "an abomination", "unusable except when you absolutely have no other option", and "don't". That might be down to the implementations in play at the time, though. I most often see this sort of thing where corporate IT can't stomach devs getting root on their own machines. It's a very specific sort of corporate dysfunction.
- firesteelrain 2y ago
  lol yep no root here But, I am in an airgapped environment that is tightly regulated if you get it.

disintegrator 2y ago

Author here. Thank you for all the tips. I especially like the idea of using ssh from guest to host to enable pbcopy/pbpaste and open. Now I know what all the WSL users experience seamlessly with their setups. Glad I have something that comes close.

StreakyCobra 2y ago
Thanks for the post, an interesting read! Side note: I checked out your other blog post, and it resonates with my own first post, which I wrote just two days ago: https://fabiendubosson.com/blog/overcoming-perfectionism/ https://fabiendubosson.com/blog/overcoming-perfectionism/. You’re definitely not alone in battling anxiety, perfectionism, and procrastination when it comes to blogging. Keep writing! :)
- disintegrator 2y ago
  Thanks! Really appreciate your comment :)

tonymet 2y ago

Windows 11 and WSL manages this well . For those developing linux apps & containers using VS Code, you'll find the Windows 11 experience to be very good. You can code against WSL which offers the more popular distros, or use HyperV to run your own custom VMs.

makeitdouble 2y ago
In general, yes. One weird quirk: networking can be peculiar. Windows creates a magic bridging between the host and WSL, and as anything magic, it can break for specific use cases. VPN is one [0]: my WSL instances lose outgoing networking when connecting to our company VPN. There are workarounds but none are trivial. [0] https://superuser.com/questions/1715764/wsl2-has-no-connectivity-when-windows-is-connected-to-vpn https://superuser.com/questions/1715764/wsl2-has-no-connecti...
- tonymet 2y ago
  Good to know. My vpn works but I believe it’s wireguard-based. I wonder if yours is TUN/TAP or another driver
  - makeitdouble 2y ago
    It seems to be TAP (layer 2 tunneling) Thinking about it, as the whole machine is under MDM (I only have the VPN on my dedicated work machine), there might be additional quirks that mess with the networking as well. Even bridging the Wi-fi to it was kind of a PITA.
- k8sToGo 2y ago
  This has been fixed for many months now as you can switch between different networking types for WSL. https://learn.microsoft.com/en-us/windows/wsl/networking#mirrored-mode-networking https://learn.microsoft.com/en-us/windows/wsl/networking#mir...

raihansaputra 2y ago

Are you installing the project dependencies on the VM directly or in a docker container? I'm curious how well docker on top of the Ubuntu vm works. Orbstack is great for personal use, but some companies don't want to pay for it, and this might be an alternative to have a better docker experience on macOS.

UltraSane 2y ago
docker in a VM works fine because docker isn't a VM and just uses Linux features like pivot-root, namespaces, and cgroups to isolate programs. At least on x86 CPUs you can even do nested virtualization if the CPU supports it.
arkh 2y ago
> I'm curious how well docker on top of the Ubuntu vm works. Works well enough. Better than Docker on barebone MacOS if you have lot of file access in a volume. That's one of the thing which surprised me when I started using VMs to develop. First time was with a postgres backed app: I expected to lose performance when moving everything in a VM. But got the exact opposite result at the time. Postgres really liked the linux filesystem more than the windows one, enough to do more than offset the VM tax.
disintegrator 2y ago
I work directly in my guest os and clone my projects and run them in there. There are some projects that are driven through docker-compose and that works nicely. The one caveat is that I had to install `apt install binfmt-support qemu-user-static` so that docker can run x86_64 images on my arm64 VM.

tonymet 2y ago

For the rest of you, don't be fooled by Darwin, it's a dusty BSD in Linux clothing

mmwelt 2y ago

This all looks fine for developing server apps that don't need a GUI, particularly as long as 3D accelerated graphics aren't needed. You don't even need to be using/developing a 3D game or application, just using a modern GUI without too much lag now seems to require 3D acceleration.

weikju 2y ago
3D acceleration is pretty well supported in VMware and sorta works well in UTM
- 01HNNWZ0MV43FF 2y ago
  Complete pain in the butt for anything else like qemu
  - weikju 2y ago
    UTM's backend is qemu, isn't it? so one could check what they're doing
  - kwk1 2y ago
    It's pretty manageable with libvirt + virt-manager sitting on top of QEMU, though.

apt-apt-apt-apt 2y ago

I accidentally typed 'npm install axioss' (extra s typo) this morning. When it successfully installed, it was terrifying to think that all source code, private files were instantly shared with malicious actors. Not only that, there was the prospect of having to somehow wipe and ensure all files were clean, reinstall the OS, and the possibility of some bootloader remnant still lurking. In this case, it seems that a security package had replaced a previous malicious package, making this instance benign. But it feels like I am only one typo away from an absolute catastrophe every time I install a package. VM seems like a good way to add some protection.

jeswin 2y ago
> VM seems like a good way to add some protection. Yeah, but someone should try to fix this anyway. It's not a nodejs-specific problem, but it's badly needed in node. Any of the 100s of authors whose packages I depend on might have made a typo, or just been careless. Software development requires a scary level of trust. I am also increasingly moving to VMs. I want tools (such as VSCode) to run on the main machine, but actual execution to happen in the vm. It's a bit painful and a drag on productivity, especially debugging.
- skydhash 2y ago
  The one trick I found that work well is to move everything in the VM. I usually opt for either emacs or vim, and if I need an IDE, I install i3. It just takes a moment to copy my dotfiles over.
  - quectophoton 2y ago
    I'm doing something similar. My development environment for work is defined in a Dockerfile, and I have a small shell script whose only purpose is to call `docker run` with that image, mount a few volumes for caching, mount the CWD in "/workspace", and start a shell in there. Development is done with nvim. If I need Docker Compose, I run it from the host. For projects that I find unpleasant to work with in this way, I use GitHub Codespaces. I hadn't thought about using a GUI IDE from within a VM, so thanks to your comment (EDIT: and also the submitted article) I now have something new to try!

hrtk 2y ago

Have you tried Lima?

zokier 2y ago

I use similar setup on Windows (with vmware/virtualbox/hyper-v at different times), which kinda highlights one additional upside: it doesn't matter that much what the host system is, you can do your work all the same regardless if it is macos/windows (or even linux). As long as it can run the vm and vscode, you are good to go. Although admittedly Apple going with aarch64 throws a small wrench in the equation. It is especially nice in corporate environment, where the host system is generally managed by IT and the devices are largely impersonal (standardized configuration, standardized software). You can carve out a corner to make your own and work there. <insert rant on ineffective corporate IT>

TowerTall 2y ago

I run my entire work computer inside of a VM. I work from home and have a powerfull desktop i use for my private stuff hooked up to my 3 monitors. My work pc is a vmware vm running inside of vmware pro. I can minimize the work vm and work it out my sight or I can selective choose that the vm should use 1, 2 or 3 monitors and it is easy to switch back and forth between work and private without have any work related data on my private desktop. The work vm is domain joined, o365 enterprise joined and loocked down in ridicilous ways by cooperate IT but now i can run that from my private powerfull PC without worring that corp IT messes up my private computer.

vladvasiliu 2y ago
I was thinking of doing something similar, especially since work mostly insists on running Windows. Do you use conferencing software or such from the VM? If so, how's the performance?
- TheTxT 2y ago
  I did the same thing for many months running teams. It was about what you expect from teams. I didn’t have any other significant issues, but eventually stopped using this setup due to increasing security requirements of the VPN software.
- TowerTall 2y ago
  I started working from home around 12 years ago and have been using this setup in various incarnations. I experience zero issues related to running work from inside of a VM. I work for an MS Shop, so we are using teams. The main issue with my setup is that VMware or Windows 11 (my host OS) can't use the GPU when rendering the UI of the VM (I'm not sure if VMware or MS is to be blamed or both) despite having a discrete GPU card installed. The rendering of the image must all be done by the CPU, which requires a lot of RAM. After switching to 3 x 4K monitors, the VM requires 62GB of RAM to be able to run on full screen on all monitors (63720 MB to be precise). I recall that I somehow managed to get it working while "only" using around 32GB, but it became unstable. 62GB is the sweet spot where everything runs smoothly. Haven't tried to adjust the settings in years. It was a pain to get working in the first place with 4k monitors, and since I have 256 GB installed, I just left the settings as they are. At next motherboard upgrade I might revisit, but I think that VmWare still can't take advantages of the host GPU in this regard so I expect the RAM requirement to stay. Some computer problems can be solved by throwing hardware at it. This is one of those. Give VMware and Windows an obscene amount of RAM, and you can have Teams running smoothly and flawlessly inside a VM on a Windows Client Host in 4K.

sushidev 2y ago

Started using code-server on a remote server. Pretty good. Going to switch working like that. Its a bit like remote vscode only that the vscode ui is also remote and served via a web browser. Coming from intellij I was surprised that the user experience in terms of responsiveness is actually better with the remote setup. IntelliJ these days just lagging on everything.

arkh 2y ago

> I’ve found developers frowning up Ubuntu and preaching for folks to use NixOS, Arch, Debian or other distros. My setup is mostly one VM per project group / online identity. Most of them using Ubuntu. The problem is when I want to work on an old project to check how it likes new technology I tend to stumble into the "you should have kept the OS up to date" problem. Ubuntu does not make it easy to upgrade if you miss more than a year of update. And even if you keep up to date, they tend to break things often (loved the X11 to weyland switch when working with screen capture libraries) so new VMs are using debian.

Asooka 2y ago
Don't you get the same problem with upgrades with Debian? As for Wayland, seeing the progress over the last 17 years, I estimate it will be ready for regular use sometime during the 41st millennium.
- thequux 2y ago
  With debian, is you develop on stable, you only need to do the upgrade song and dance every year and a half or so and upgrades rarely break anything. If you develop on unstable, you can use the snapshot archive to either upgrade 6 months at a time or move back to the next stable and then walk through stable releases.
- arkh 2y ago
  > Don't you get the same problem with upgrades with Debian? I don't think you have to do some manual configuration to upgrade your distribution because it is a couple years old and current scripts don't support that (like going from a 22.10 to 24.04 is a fun game).

Abishek_Muthian 2y ago

I do a switcharoo, I develop for iOS inside a macOS VM with Linux host. After a 5 years hiatus I started developing mobile apps again and I was frustrated to learn that Apple doesn't allow renewing the developer license on web anymore, I don't own a Mac and even Apple developer app on iPhone didn't allow me to renew my license. After I signed into a macOS VM, I was able to renew my license through Apple Developer App on iPhone as macOS version of the app requires T2 chip. Now I have PTSD flashbacks of why I left mobile development in first place.

theoreticalmal 2y ago
Can you share more info on your MacOS VM setup? I’ve managed to set one up precisely once with proxmox, and getting iCloud/imessage to work required me to contact Apple support
- Abishek_Muthian 2y ago
  I used Kholia's scripts[1] on Qemu with Virsh. I didn't have much trouble other than some SELinux permission issues which I resolved quickly. iCloud/imessage have always been finicky with Hackintosh, but in my experience setting the correct serial number with appropriate Mac model is the key to resolve those issues. With just couple of years of Hackintosh scene left before support for x86 Macs are dropped completely, a VM Hackintosh makes more sense to me than building a physical Hackintosh. [1]https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM
  - LeFantome 2y ago
    Even a VM will stop working soon as Intel support is removed from the OS and apps start demanding newer OS versions. Or is it possible to emulate Apple Silicon on a VM now?
    - Abishek_Muthian 2y ago
      Yes macOS VM will also be useless too in some years but a useless VM is better than a useless purpose built computer for hackintosh. Qemu is capable of ARM emulation but I don't think its capable of emulating sophisticated SoC like Apple Silicon.

mootoday 2y ago

Pretty sure you'd have a more lightweight experience with https://www.jetify.com/devbox https://www.jetify.com/devbox. Happy to set it up and demo if you can share (or DM) a repo URL.

rcarmo 2y ago
That sets up a VM with Docker as well.
- mootoday 2y ago
  No
  - rcarmo 2y ago
    You can't run Docker on a Mac without a VM.

rcarmo 2y ago

I use Proxmox on my LAN and RDP connections to Linux and Windows desktops of various kinds. It’s great.

pshirshov 2y ago

I use Nix for exactly the same purpose (dependency management for code generators), that's much more efficient and easier to maintain than VMs.

Too 2y ago

Any benefit of using a VM over docker container here? Since you seem to use the terminal only, without any graphical applications. Containers should be more lightweight and dockerfiles allow quick and reproducible changes to the guest OS.

kasey_junk 2y ago
Docker uses virtualization on Macs

TacticalCoder 2y ago

Curiosity: is anyone here developing inside a VM, with GPU passthrough, and with the monitor directly connected to the GPU used by the VM? (as in: showing the UI of the VM, without any need for RDP or the like) Such a setup works (I'd know for I have one at home doing just that but it's not my main PC) but how's it like to work like that? The GPU hooked to the hypervisor can either be on another monitor or on another input (in the latter case you'd "go" to the hypervisor by changing the monitor's input).

thehamkercat 2y ago
I have a 7800X3D with 64GB Ram, it's overkill for programming So I've installed proxmox on it to utilize it's resources for other stuff as well For my personal use, I created an Arch VM with GPU and USB PCIE passthrough, all 3 monitors directly connected to that GPU It's so seamless and fast that I don't even feel I'm working inside a virtual machine I have other headless VMs hosted to do other things (opensense etc)
TheTxT 2y ago
I used such a setup for gaming using a windows and a Linux (later proxmox) host. With everything passed through you basically don’t realize that you’re sitting in front of a VM, it’s great.

lizknope 2y ago

Are most companies this flexible in allowing developers to install whatever they want? > My physical machine is a 2023 MacBook Pro with M2 Pro CPU > I’m using VMWare Fusion Pro > Quite often I’ve found developers frowning up Ubuntu and preaching for folks to use NixOS, Arch, Debian or other distros. The reality for me was Ubuntu was the fastest way to get set up and now I'm in integrated circuit / semiconductor design. At every big company over the last 30 years we are given a computer and we can change the desktop environment but we aren't installing our own operating system. The people I know in software have a common OS, compiler, and build environment. They aren't dictating what text editor you use but you aren't working on projects individually but together. So if everyone at the author's company is doing their own thing do they have problems integrating all the code together? "Oh you used version 2.3.4 but I used version 2.4.7 which fixed this issue, what are we using to ship with?" Or is this not a problem?

IsTom 2y ago
There's typically a list of dependencies of a specific project as a part of it, (hopefully) handled automatically by some part of build pipeline.
- whatevaa 2y ago
  Build pipelines don't have much to do with actual development. What, do you edit stuff but never build it locally?
  - IsTom 2y ago
    I should have written "build process" instead I suppose.
bluehatbrit 2y ago
I work mostly in web / server side development, it's not really a problem I've had for a number of years now. Some of my colleagues use various linux distros, others macos. No one is using Windows that I know of. Each project we have requires a specific tool chain version (python, elixir, ...) and specific versions of things like postgres. All dependencies are listed with some kind dependency definition file (pyproject.toml, package.json, mix.exs). If you bump a package it's done in the definition file as part of your changes and goes through CI for packaging and releasing. The rest of the team will get the new package version as soon as they pull your changes and run `just deps` or whatever. CI is the ultimate determining factor of whether your code actually "works". We also package and deploy with containers, but this isn't the real determining factor for any of the above.

urronglol 2y ago

Devcontainers running as non root. Trivial t set up. Don’t need to fanny around with a Vm

disintegrator 2y ago
I've tried devcontainers in the past and the performance compared to my current setup was pretty bad. This was a few years ago when it was known that filesystem-heavy workloads on Docker for Mac were sub-optimal. I remember having to define several bind mounts which improved the overall performance. I do intend to revisit this solution next time I need to set up a dev environment but rest assured there was nothing substantially more complex about a VM versus devcontainers.
- urronglol 2y ago
  So why are you commenting on it now as if it hasn’t changed or evolved since then? Give it another try now. It is really good.
- bluehatbrit 2y ago
  I use it at the moment and don't really find any noticable difference between running directly on my host, and in a dev container. If I were to measure the performance I'm sure there would be something, but it's not noticable in my development cycle. They also seem to be pushing it beyond vscode and into something which is editor agnostic. It's not quite there yet on that front, but I'm excited for it as I've been dabbling with other editors recently which don't support devcontainers directly and it always pulls me back to vscode. It's on a journey for sure, but I've had no performance issues when using it straight out of the box over the past year.

secondcoming 2y ago

If you're using a personal device for developing your employer's code always use a VM. You can just nuke the whole thing when your employment ends and not have to worry about any of their IP remaining on your machine.

cranium 2y ago

(Tangentially related) I had to run the desktop version of Excel to develop a quick VBA macro for a client. Problem: I've been developing on a Linux box for years and the idea of leaving my cozy dev environment for a plain Windows install gave me chills. After failing to install Windows in a VM (thanks TPM), I found a way to run Windows apps nearly natively (https://github.com/winapps-org/winapps https://github.com/winapps-org/winapps). It works by starting a Windows docker image and streaming the application frame with RDP. As the RDP client handles the copy/paste and other niceties such as shared directories, it's way easier to integrate in my env than the other options.

rescbr 2y ago
This is great!

gbraad 2y ago

Consider using tailscale drives to expose certain folders using webdav. Have been doing this for de containers and VMs since it got demoted and since has replaced using winscp or other ways to share files

disintegrator 2y ago
Just trying this out and it seems amazing so far. Thanks for the tip!

pjmlp 2y ago

At work, development with VMs has been a given since the early days of Amazon EC2 in 2010. Likewise when Windows 7 came out, I stop bothering with dual booting hassles and using VMare Workstation instead for whatever Linux. The exception being a netbook from the Asus Linux netbooks glory days, a price category nowadays replaced by tablets.

crabbone 2y ago

But why tho? The only kind of plausible explanation the author gives is that it's "more secure" because the imaginary attacker will have to take an extra step to get the password from the VM instead of the host OS? -- This seems like such an inconsequential / worthless benefit to jump through the hoops of running things in a VM... Like... I wasn't sold on this approach from the get go, and this pitch makes it sound like I was right all along? Other non-starter "bonuses" include not installing developer tools on your laptop that you have for... drum roll... development. Why? It's sole purpose is to be used for development, why not install development tools on it? Just doesn't make any sense...

nkko 2y ago

An alternative approach we're working on is Daytona (https://github.com/daytonaio/daytona https://github.com/daytonaio/daytona) - it orchestrates dev environments on your infrastructure or even on your local machine. Oh, and it is Apache 2.0. The interesting bit is how it handles any IDE integration and various providers for running your dev envs on. Disclosure: I work on this project