10 ms·
Lima: A nice way to run Linux VMs on Mac
- MuffinFlavored 3y agohttps://github.com/search?q=repo%3Alima-vm%2Flima%20qemu&type=code https://github.com/search?q=repo%3Alima-vm%2Flima%20qemu&typ... It's a wrapper that interfaces with QEMU in the background to make things nicer/easier in case anybody was wondering
- hatf0 3y agoWould be cool if we see something come out that uses the Apple native Virtualization.framework so you can use the nested Rosetta extensions on M1. Dunno if that's been done yet.
- stetrain 3y agoIt is an option in Lima, I’m not sure if they handle setting up Rosetta yet though: https://github.com/lima-vm/lima/blob/master/docs/vmtype.md https://github.com/lima-vm/lima/blob/master/docs/vmtype.md
- hatf0 3y agoOh sick! According to some of their GitHub issues (https://github.com/lima-vm/lima/issues/1202 https://github.com/lima-vm/lima/issues/1202) they do setup Rosetta. I'll have to check this out then!
- filleduchaos 3y agoUTM uses either Virtualization.framework or QEMU depending on what you specify.
- Scarbutt 3y agoso it's doing software virtualization?
- kwk1 3y agoLooks like it defaults to using `-accel hvf`, roughly analogous to KVM, so no.
- MuffinFlavored 3y agohvf = hypervisor framework lima now supports "macOS Virtualization.Framework" https://zarinfam.medium.com/what-are-the-advantages-of-the-new-virtualization-framework-in-macos-big-sur-7685c3aca0f7 https://zarinfam.medium.com/what-are-the-advantages-of-the-n... https://news.ycombinator.com/item?id=36184400 https://news.ycombinator.com/item?id=36184400 I guess it's better/different and not just roughly the same thing wrapped in new packaging?
- stetrain 3y agoAlso worth checking out is Colima, which uses Lima to give you a linux container environment without needing to install Docker Desktop: https://github.com/abiosoft/colima https://github.com/abiosoft/colima
- fithisux 3y agolove it.
- dewey 3y agoColima is great, compared to Podman it's a lot more of a drop-in replacement for my use case. I've always had issues with Podman volumes but with Colima it was as simple as uninstalling Docker Desktop and running "colima start". There's one issue I'm running into where it becomes unresponsive after a while and "docker ps" hangs forever though.
- belthesar 3y agoBorrowing this thread to add additional context, Rancher Desktop on macOS also uses Lima to make VMs for running k8s (I think it's actually k3s?) on your workstation. I've been meaning to try out Colima, since, while nerdctl is pretty functional and things work, sometimes dealing with the nuances when I don't really need a real Kubernetes environment for most of my dev tasks is more overhead than I'd like. That said, if you do need a proper k8s environment on macOS, Rancher Desktop does work quite well, and makes a lot of sense especially if your shared k8s environments are managed by Rancher.
- vicaya 3y agocolima pretty much solves dev experience for docker and k8s on mac, esp. for apple silicons (m1/m2), where you can build multi-arch containers with ease. Some interesting caveats: * By default, system packages don't persist, as the default alpine distribution runs on tmpfs and doesn't have a overlay. This is a reasonable default, as it keeps the default VM storage small. * If you want to have additional system packages, you can turn on a ubuntu overlay that supports additional systemd services just fine. Of course, storage would balloon to a few GBs from a few hundred MBs. Edit: typos. BTW, the result of docker build is immediately available to the k8s (k3s) cluster without any insecure registry and/or side loading/caching steps, thanks to the seamless buildkit integration.
- golanggeek 3y agoVery nice. To have some sub directories as read-write would be super useful.
- adamgordonbell 3y agoIt totally works with read-write enabled and in fact rancher uses it this way. Make this change: mounts: - location: "~/sandbox" writable: true Lima cautions against this: # Setting `writable` to true is possible, but untested and dangerous. But I never hit any problems when I played around with it. Here are my notes: https://earthly.dev/blog/lima/ https://earthly.dev/blog/lima/
- davidcollantes 3y agoVery helpful notes on your blog, thank you for sharing!
- 727564797069706 3y agoHow does it compare to https://multipass.run/ https://multipass.run/?
- antod 3y agoMultipass for me suffered from a bunch of Macos networking bugs when on managed Macs. Kernel panics and vms that you couldn't connect to etc. UTM also suffered from these too. Apparently some have been fixed by now though.
- ChrisTownsend 3y agoHi! Yes, the dreaded Apple Firewall bug:( The cause is that somehow the firewall would block macOS's own bootpd process and that it the process responsible for handing out IP addresses to the virtual machine instances. We could never find out why macOS chose to block bootpd though. Anecdotally, the number of users mentioning this issue have been much less lately, but I'm not sure if that is because the new Multipass release which has a newer QEMU made it better, a macOS update fixed it, or users have just given up.
- antod 3y agoI gave up on using VMs on my work Mac (Linux user at home). When I get round to trying the Ventura upgrade, I'll give it another go. The annoying thing was that multipass and UTM would initially work before the firewall killed it a few days later. Makes me think it was learning from heuristics or something rather than being a defined policy.
- dakiol 3y agoSame question. I have been using multipass on my Mac (M1), and so far so good. The current limitation of multipass is that it only runs Ubuntu VMs. Also, setting up fixed IPs for multiple VMs is a bit tricky (if possible at all, I don't remember right now). I have a bash script that uses multipass to setup a few VMs... but still it feels "primitive" compared to what I was using when I had an intel Mac (I was using Vagrant, but the Vagrant experience on M1 is awful: I have tried it with VMWare and it's not very stable in my experience).
- gpsz 3y agoAlso worth looking at UTM [1] which uses the apple virtualization framework, while also being a simple frontend to QEMU. [1] https://github.com/utmapp/UTM https://github.com/utmapp/UTM
- mrweasel 3y agoI really like UTM for VM that I just need to shell into. It's not as complete as something like VirtualBox or VMware Fusion, but it does what I need. The only thing I'd like to have is some sort of background daemon, so my VM don't stop if I accidentally close the UTM window.
- ChrisLTD 3y agoThis is what I've been using. Very nice app.
- SpriglyElixir12 3y agoIs there a way to persist volumes in UTM? I want a persistent fedora vm but haven't been able to set it up.
- codetrotter 3y agoDid you run the Fedora installer in the VM, or just boot from live-CD?
- espe 3y agolima is nice. just beware that os updates can accidentially nuke the vm. got to try out utm sometime.
- antimatter15 3y agoI've recently been enjoying OrbStack (https://orbstack.dev/ https://orbstack.dev/), which I've found easier to get started with than Lima, starts up faster, and automatically mounts volumes so you can access things from Finder It's unfortunately not fully open source
- ElCapitanMarkla 3y agoIt is a really nice looking product but I wish they would set down a firmer plan for how they are going to charge for it.
- jzelinskie 3y agoI honestly think this is a feature and not a bug. The FAQ shows an attention to detail for the trade-offs of various pricing models[0]. It's clear that Danny cares about monetizing the project in a thoughtful way. I moved away from Docker Desktop to colima for a couple years and would not pay for Docker Desktop, but after a few weeks of swapping back to OrbStack now that it's public beta, I can definitely see myself paying. OrbStack just works and gets out of the way. [0]: https://docs.orbstack.dev/faq#free https://docs.orbstack.dev/faq#free
- tomcam 3y agoThat jumped right out at me too
- haberman 3y agoThis is a couple months old, but is a reasonably concrete proposal: https://twitter.com/OrbStack/status/1656326409995055104 https://twitter.com/OrbStack/status/1656326409995055104
- kdrag0n 3y ago(dev here) Yes, this is still the proposal that I'm planning to move forward with. It's always possible that it doesn't work out and will need to be changed, but I think it's a pretty solid start. Just updated the docs with this info: https://docs.orbstack.dev/faq#free https://docs.orbstack.dev/faq#free
- dmarinus 3y agoI've been searching for a live iso for Windows to use with lima (wima, Windows VM on Mac).
- itsrobforreal 3y agohttps://developer.microsoft.com/en-us/windows/downloads/virtual-machines/ https://developer.microsoft.com/en-us/windows/downloads/virt...
- dirteater_ 3y agoFunny to see this on the front page the same morning that I switched back to using a remote dev box. Lima constantly has i/o issues (usually network hangs, local connections within the VM). It's pretty rough to use when I need to do docker in Lima.
- espe 3y agoit's all sshfs.. afaik they're looking into changing that. i don't have the i/o issues but always backup any files in the vm as they tend to get lost at some point.
- mark_l_watson 3y ago+1 for a remote dev box. ssh/tmux/mosh and Emacs. I really like being able to read some tech on my iPad, see something I want to try, and 5 seconds later I am SSHed/Mosh'ed into my remote dev VPS. EDIT: that said, I do have Lima installed on my M1 MacBook Pro.
- nine_k 3y agoIt's interesting to compare how soon the VPS's total cost will reach the coat of an MBP. Even a relatively beefy VPS which is $30 / mo is only $360 / year, like a cheap used Linux laptop. On the other hand, it has much much less CPU than an MBP, and usually no GPU at all. On the third hand, the cloud allows one to quickly start more VMs, and to shut them down as quickly, something that's possible but a bit more limited on a laptop / desktop.
- Nanana909 3y agoWhy pay for a whole month though? Depending on your work, spin it up and down as needed. If you're just building a binary you only need it up for minutes or seconds. Been on my mind lately. Lot more sandboxing too. And if you want to build fast you can just scale up your instance since its so short lived. Want to switch from x86 to ARM? no problem. Spin up 40 of your current app? no problem.
- markstos 3y agoIs it pronounced like Lima the bean (LEYE MA) or Lima the city in Peru (LEE MA)?
- blacksmith_tb 3y agoI was assuming it was lim-UH, for LInux+MAc?
- throw0101a 3y agoThe post mentions Homebrew, but seems to also be available under MacPorts: * https://ports.macports.org/port/lima/ https://ports.macports.org/port/lima/
- jonhohle 3y agoNice to see MacPorts love!
- deleted 3y ago[deleted]
- nathants 3y agoat a certain point it may be less work to make linux macos like than to make macos linux like.
- pjmlp 3y agoThat is the thing when people buy Apple, but actually want GNU/Linux for the actual work.
- mst 3y agoFor the past twenty years I've bought thinkpads which during productive time have generally run (a) Firefox (b) some sort of system that lets me get full screen X11+xterm+ssh to do the actual work on remote development servers. I've gone through VMs, colinux, cygwin, and currently WSL2+VcXsrv for (b) and so far it's basically worked out fine. (I'm aware that desktop linux wrangling is far, far easier than it used to be but this setup very rarely annoys me and if I'm going to do some recreational geekery on a "because I can" basis it's far more likely to be something like playing with a new programming language than reinstalling my laptop)
- hank_z 3y agoCurious to know whether the Linux VMs on Apple Silicon Macs are also based on arm or not.
- _cenw 3y agoConfigurable. Also runs ARM on Intel Macs if you want.
- sgt 3y agoI see that lima has an option to choose between qemu and vz. What are the pros/cons to each? Is vz performance better? Update: I edited the YAML file for the Lima VM and changed from qemu to vz, also made sure the mount was using virtiofs. Observations - on the surface, no performance difference but I haven't really done much yet. I noticed that there is no longer a qemu process running (obviously), and I see that /System/Library/Frameworks/Virtualization.framework/Versions/A/XPCServices/com.apple.Virtualization.VirtualMachine.xpc/Contents/MacOS/com.apple.Virtualization.VirtualMachine is now running. Having a look at https://developer.apple.com/documentation/virtualization?language=swift https://developer.apple.com/documentation/virtualization?lan... for the documentation. It definitely looks like an interesting and well built framework.
- sgt 3y agoStrange - com.apple.Virtualization.VirtualMachine goes into 400% CPU and the Ubuntu VM freezes. I've now reproduced it twice. Not sure why this happens.
- maxyurk 3y agoVZ is much much faster for me on Mac, especially M1/M2.
- maxyurk 3y agoI saw the difference with heavy apps (multiple large processes inside the same container, lot's of I/O)
- deleted 3y ago[deleted]
- jamifsud 3y agoAnyone have good (preferably open source but not required) tools for running MacVMs on a Mac? Would love a way to programmatically control MacVMs (create new from image, start, stop, etc) as part of our Mac build server setup. GitHub actions Mac CI minutes are so expensive so we run our own setup and VM level isolation seems to be the best way to keep the build processes from stepping on each other.
- mannyv 3y agoVirtualBox supports it, as does VMWare Player. There's a VirtualBox build for Apple Silicon, which might do the right thing.
- stephenr 3y agoVbox for Arm macs doesn’t run Arm VMs: it emulates 32bit x86 hardware extremely slowly.
- halostatue 3y agoEclectic Light has Viable: https://eclecticlight.co/virtualisation-on-apple-silicon/ https://eclecticlight.co/virtualisation-on-apple-silicon/
- gurjeet 3y agoI’ve had a good experience using Tart. It checks all the boxes you asked. https://tart.run/ https://tart.run/
- ransackdev 3y agoYou can use qemu/libvirt/kvm on any Linux host to run macOS pretty easily these days[1]. I run Ventura on unraid with nvidea gpu passthrough (w/ ryzen cpu even!) and it’s been fairly painless. You can also run macOS in docker, but it’s ultimately running through qemu/kvm as well[2] 1. https://github.com/kholia/OSX-KVM https://github.com/kholia/OSX-KVM 2. https://github.com/sickcodes/Docker-OSX https://github.com/sickcodes/Docker-OSX
- e12e 3y agoI found UTM great for running a macOS vm on Mac: https://github.com/utmapp/UTM https://github.com/utmapp/UTM
- lelanthran 3y agoWhat I'd really like is to run MacOS on Linux. When I want to make sure my software works on MacOS, it'd be nice if I could do that without having to have a whole other computer sitting in front of me.
- n8henrie 3y agoAs a novice, I was surprised how easy it was to get the basics working with plain old libvirt (on my M1 at least).
- eoinboylan 3y agoOh this is very nice, I spent about 2 hours getting around some bullshit bug with Vagrant and VirtualBox to work on ARM OSX at the weekend. This took 5 minutes to setup.
- vuln 3y agoHas anyone figured out a way to run VMs on the M series iPad Pros?
- chazeon 3y agoHave you tried iSH[1]? [1]: https://ish.app/ https://ish.app/
- fredoliveira 3y agoI believe UTM will let you do just that.
- drozycki 3y agoYou can install UTM via AltStore but you’ll be limited to slow battery-hungry emulation. You might be able to use the hypervisor for hardware virtualization support via an exploit like TrollStore running on an older iPadOS. The OS will kill UTM soon after backgrounding to free up memory and preserve battery life so you’ll need to keep it onscreen using Split View or Stage Manager. Overall I found it to be too much of a hassle compared to remote dev using Blink shell or vscode.dev
- AceJohnny2 3y agoI want to amplify this: > why not use containers? [...] > on Mac you need to run containers inside a Linux VM anyway, so I’d rather use a VM directly and not introduce another unnecessary layer I was long confused at how Docker functioned on macOS, until I learned that it's "just" running a Linux VM within which it runs the container images. There is no other magic happening to run a (linux-assuming) container on macOS.
- favourable 3y agoHow do you pass files into/out of the VM? I know Virtualbox has the Guest Additions software which is very handy. I also know of things like the Spice Project[0]. Does Lima have its own solution? [0] https://www.spice-space.org https://www.spice-space.org
- rawbot 3y agoThe author said that it mounts MacOS home dir by default, so that would mean you can pass files that way.
- favourable 3y agoOops, glossed over that
- signa11 3y agoi normally use sshfs, which works just fine.
- dabedee 3y agoI recommend having a look at Macpine [1] which allows you to run lightweight alpine VMs on MacOS with easy port forwarding, file sharing; you can also easily run docker inside of it and use docker context to target it. [1] https://github.com/beringresearch/macpine https://github.com/beringresearch/macpine
- jbverschoor 3y agoI’d just want the following: In a cli, I want to start “vm” It should check the current directly and go up each time similar to.asdf or .rbenv, looking for a .virtconfig dir Depending on the config, I want: 1: it running a foreground instance, soo I don’t need ssh, and I know that it will shut down when I end the shell 2: I want to configure my shares/mounts, which by default don’t go up from the .virtconfig dir 3: I have to think about read only instances and multiple instances of the vm The idea is that when you later cd into a project directory, .direnv (I think) can automatically turn into a Linux shell, or other OS which is also sandboxed. I’d also want a single command that spans a Linux instance with the current director mounted (r or Re) to Linux. This way you get some sandboxing when trying someone else’s code
- amelius 3y agoAnyone knowing a good way to do the reverse? I.e., run MacOS on Linux?
- mil22 3y agoYou can use VMware Workstation Player and an unlocker like Auto-Unlocker (https://github.com/paolo-projects/auto-unlocker https://github.com/paolo-projects/auto-unlocker) to enable Mac OS as a guest. It works, but it's very slow because the Mac guest runs without GPU acceleration.
- willmeyers 3y agoI'll take a look at lima, but I've had nothing but problems using colima as a docker alternative on my macbook air m1. Could be user incompetence, but always got issues of images failing to pull and containers erroring out in mysterious ways.
- kirbyfan64sos 3y agoI want to point out that Lima also works on Linux, and it's super useful there for development scenarios where a container just doesn't cut it.
- deleted 3y ago[deleted]
- w10-1 3y agoYou can also just run the VM's using Apple's own API's. Their sample code includes a resizable display, shared drive, network access, etc. You can run x86 Linux on Apple Silicon (including running pre-built x86 binaries with Rosetta). On the latest release, you can save and restore the VM state to avoid boot initialization and application setup. Run headless linux https://developer.apple.com/documentation/virtualization/running_linux_in_a_virtual_machine https://developer.apple.com/documentation/virtualization/run... Run GUI linux https://developer.apple.com/documentation/virtualization/running_gui_linux_in_a_virtual_machine_on_a_mac https://developer.apple.com/documentation/virtualization/run... Run intel binaries in linux vms on appleSilicon with rosetta https://developer.apple.com/documentation/virtualization/running_intel_binaries_in_linux_vms_with_rosetta https://developer.apple.com/documentation/virtualization/run... Configuring Virtio shared directory https://developer.apple.com/documentation/virtualization/vzvirtiofilesystemdeviceconfiguration/ https://developer.apple.com/documentation/virtualization/vzv... And you can run MacOS VM's on Apple silicon: https://developer.apple.com/documentation/virtualization/installing_macos_on_a_virtual_machine https://developer.apple.com/documentation/virtualization/ins... https://developer.apple.com/videos/play/wwdc2022/10002/ https://developer.apple.com/videos/play/wwdc2022/10002/
- freetime2 3y agoDoes this offer any performance benefit over using Lima, particularly when running x86 VMs on Apple Silicon? Or is Lima already using the same APIs?
- treesknees 3y agoTo be clear, you can't run an x86 Linux VM on Apple Silicon using Apple's virtualization framework. It says right in the docs, >The kernel and RAM disk image must support the CPU architecture of your Mac. You _can_ run x86 via Rosetta inside the VM. But that's not the same thing as running an x86 VM. To answer your question, Lima supports two VM types - QEMU and vz. QEMU is the default and is what's used by other projects to run x86. If you select a vz VM type then it uses MacOS' native APIs. I wouldn't expect a performance improvement in using a vz-typed VM with Lima and Apple's own instructions.
- xvilka 3y agoIt's even possible to run FreeBSD too[1] [1] https://github.com/lima-vm/lima/pull/1509 https://github.com/lima-vm/lima/pull/1509
- ShadowBanThis01 3y agoHasn't Ubuntu fallen out of favor lately? I don't remember what people's beef is... Anybody?
- Aaron2222 3y agoForced Snaps I think.
- ShadowBanThis01 3y agoYes, that was it, thanks!
- yewenjie 3y agoIs it possible to run a Linux OS (NixOS) on M2 Macs and forget about MacOS toyally and expect reasonable performance?
- vaxman 3y agoIn a word, no. Apple has kicked errbody out of kernel space over security concerns, so even companies with 25 years of experience writing network virtualization kernel extensions must now instead use Apple's vastly inferior macOS facility that can reduce performance by as much as 80%.
- vaxman 3y ago..never mind if it cost the equivalent of several m2ultras discovering what I wrote, -1 doh. OTOH, I don’t believe much of what I read on here because of such direct negative experiences like this (being downvoted on stuff I have first hand knowledge of) —and that truly is priceless.
- midoBB 3y agoThe performance isn't the issue but the stability is. Anyways this is a hardware compat and tutorial on how to install nixOS on the M series of apple computers : https://github.com/tpwrules/nixos-apple-silicon/blob/main/docs/uefi-standalone.md https://github.com/tpwrules/nixos-apple-silicon/blob/main/do...
- therealmocker 3y agoCheck out what Mitchell Hashimoto (from HashiCorp) does w/ Nix on Mac for developer setup - https://www.youtube.com/watch?v=ubDMLoWz76U https://www.youtube.com/watch?v=ubDMLoWz76U
- timberfox 3y agoAs an alternative, here's a really minimalist command-line wrapper to run VMs in the macOS Virtualization.framework: https://github.com/evansm7/vftool https://github.com/evansm7/vftool
- RWejlgaard 3y agoHow do you know people use Nix? They will tell you