7 ms·

Replacing a Development VPS with Linux on OSX

91 points by remi 15y ago

typicalrunt 15y ago
There are some useful tips in that article. Using qemu in a no-gui mode is pretty cool. I've used VirtualBox, VMWare Fusion, and Parallels on OSX before, but I hate having to deal with the GUI window (and having the system reserve resources for it). Since OSX is based on Unix under the hood, it shouldn't be too difficult to for a developer to use either OSX or Linux as their development machine. It's not like the knowledge gap between Windows and OSX/*nix machines. So other than messing around with Linux, the article fails to mention why he doesn't use OSX (other than some unfounded 'flaky' and 'impossible' critiques).
- ch0wn 15y ago
  I used to work on Django project on a Mac. Dependencies like python-mysqldb can be quite tricky to install on OSX, whereas Ubuntu only requires you to type 'apt-get install python-mysqldb' and you're done. More importantly, though, I do not like the idea of having all those services constantly running on my desktop system. I'm using Ubuntu for both development and deployment now and I still use a separate VM.
- Bockit 15y ago
  I may be missing something but I think you can do what is proposed with VirtualBox. I use it for my virtual dev machines and run them in headless mode. For example the command: vboxmanage startvm "Name of VM" --type=headless to boot an image up, takes about 10 seconds to be ready to ssh into (You can ssh immediately it just waits till it has booted to connect). vboxmanage controlvm "Name of VM" poweroff If you want to turn it off from the outside.
- jmlacroix 15y ago
  Hello, I'm the author. I mostly use OSX for everything. The only times I switch to the VM is when I'm having a hard time installing something (for an example, see my earlier comment - http://news.ycombinator.com/item?id=2969509 http://news.ycombinator.com/item?id=2969509) or want to try a tool that's not available under OSX. I admit the post would have been better with examples and without then unfounded critiques, taking notes for the next ones.
  - martijn_himself 15y ago
    Just wondering if you could think of anything obvious I'm doing wrong (I'm pretty much a newbie when it comes to this)... I'm following the instructions in your post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' I am using the netinstall Arch Linux ISO. Many thanks!
- halostatue 15y ago
  VirtualBox also can run without GUI windows: VBoxManage start vmname -type=headless If you need to see the GUI, you need the VNC extensions installed (which are, IIRC, not FOSS).
- chrischen 15y ago
  It's often a good idea to simulate your production server as closely as possible.
- zokier 15y ago
  I think one of the main reasons why I like VMs so much is that they allow my base OS to stay relatively clean. Especially as I like to try out all kind of new stuff constantly, VMs make that easy without having to care how they affect the rest of my system. Even if you aren't trying out new stuff, keeping separate projects mostly isolated makes me feel happier. imho also configuring stuff becomes easier when your machines become basically single-purpose systems.
- mentat 15y ago
  If you want no-gui VirtualBox that "just works" try vagrant.
tombell 15y ago
Wonder if the author has considered checking out http://vagrantup.com/ http://vagrantup.com/ unless it does a bit too much for what they're after.
- lisperforlife 15y ago
  +1 for vagrant. It is awesome. It integrates well with Chef for provisioning. I would definitely recommend it.
1880 15y ago
I currently use a similar environment but with VirtualBox. Does qemu offer any advantages over it? I was considering recreating my VM so I may as well try qemu.
- jmlacroix 15y ago
  It's been a while since I last used VirtualBox, so I can't really compare. What I like in QEMU, is its simplicity. The learning curve is a bit steeper, but when you're used to it there's no turning back.
mise 15y ago
On a tangent to this, what is the "Linuxy" stuff you don't get to do on OS X? (My own frame of reference being an Ubuntu user.)
- zokier 15y ago
  Apt with 20000+ packages waiting to be installed.
- mhd 15y ago
  The author might have a different experience, but for me it's mainly anything that a) depends too heavily on a plethora of GNU tools and b) anything that's about deployment and installation. You're basically running on a different Unix system, so if you deploy to a Linux server, you'll probably encounter quite a lot of differences - mostly deployment, but also libraries, syscalls etc. Quite reminiscent of the days of different commercial Unices, where the compatibility of e.g. AIX and HP-UX wasn't exactly 100% (I'm sorry if that caused some violent flashbacks for veteran developers) Also, OS X is usually behind even compared to Linux distros with fixed releases (i.e. not rolling). You get your basic Unix software in one fell swoop, with the occasional update through a dot version. Apple doesn't provide a pkgsrc or even a rpm/deb packaging system. There are third party efforts, like MacPorts, Fink or – as the author mentioned – Homebrew, which is more similar to BSD's package tree or Arch's AUR (basically a git repo of Ruby build scripts). Nevertheless, these add on to your system, so aren't really integrated perfectly. Also, compared to Debian or Arch, you'll have a smaller package list (esp. true for Homebrew). So it might just be easier to install some stuff from within a headless Linux VM. Heck, I've been doing something similar even when I was running Linux as a dev environment. Your deployment environment might be quite different, and I like to do some local testing first – doubly so if your company doesn't even have a proper testing/integration environment. I had an Arch PC, but deployed to both somewhat recent Ubuntu machines and ancient SuSe installations. Now, I often work on OS X and deploy to a CentOS VPS. All good use cases for VMs – and the way described, where it's just a small headless instance in the background instead of a useless GNOME desktop-within-a-desktop is a pretty good approach.
  - jmlacroix 15y ago
zobzu 15y ago
OSX is an OS where you run VMs for Linux and Windows. I keep seeing people saying how great and fast it is to do stuff on their mac and run Windows for a few apps incl. mail client, office suites, some dev tools and many java apps (cause they're often not so nice on OSX) Then I see them with the Linux VM for all the dev tools, the hack tools, and so forth, because, well, hacking them up to work in OSX is always a pain, and even using stuff like brew is often a pain (+ delay) Well I dunno, I run Linux native and I get stuff done faster than the ones running Linux in a VM of course. I also have Linux VMs under Linux when necessary, with KSM and it's a lot faster and memory efficient than when running OSX native on the same hardware. I install stuff in about <5s where it takes a VM to boot/resume in OSX, and about 5-10min to install from scratch when the app _supports_ OSX (ages if it doesn't but has more or less compliant sources)
- MatthewPhillips 15y ago
  Yeah, me too. I have a linux computer for most development, a Macbook when I want to mess around with iOS development, and a Thinkpad for .NET development. All of these are tax write-offs and since I use the linux computer (which twice as old as the other two, by the way) 90% of the time because it's simply easier to install development related stuff. Whenever I want to install a gem that uses native headers requiring a -dev package in OSX I just quit.
- ubercore 15y ago
  I'm the opposite. I like all of my OS X dev tools (navicat, textmate/macvim (I know, gvim...)/sublime text/github for os x/things), and it's a much more usable OS overall for me. The parts I don't like, compiling libraries, running servers, etc, I automate away with vagrant. I'm more productive with this setup than if I ran everything directly in Linux (I've tried).
- barumrho 15y ago
  Your points are valid, but running VMs, and using unix tools are not all I do with my computer. For me, running OS X is a good compromise. I used Linux exclusively for more than four years, and I miss many things about it, but I don't think I can go back, at least yet.
pmarsh 15y ago
Unless you're on an ancient laptop I don't see why you wouldn't run a Linux VM if you are developing web apps. What is the advantage of trying to make OS X your development environment and then deploying to a Linux host? Unless you have an older laptop with less than 4 gigs of ram why not? Even with 2gigs it's not that system intensive to run a Linux guest os.
- riobard 15y ago
  Let's face it: SSH into an VM and write code there is not an enjoyable experience. Yes I can do a proper deployment into the VM right after I make a minor change in color of the 10px square box located at the bottom of the page, but it's really a pain in the ass to have the deployment delay. Oh, right, the deployment process just takes 10 seconds!
  - pmarsh 15y ago
    Why SSH into a VM? With Virtualbox I have a folder shared out that I can use like a regular folder and edit with whatever editor on my Host. Or you could turn on SMB shares and use those. Or I could run a lightweight X environment (or even gnome it really doesn't need much to run) and just edit there. Everyone seems to be on the "OMG Vim/Emacs are so awesome, TextMate is so yesterday" part of the every going cycle anyway. Might as well use them on the guest OS as well.
    - riobard 15y ago
      Virtual shared folder is rather flaky for me on OS X. It magically disappears and reappears in Linux guests.
- olefoo 15y ago
  And it's worth running a vm for development even if you're hosting it on a linux machine; presumably your workspace has a bunch of things that don't need to be installed in production.
maratd 15y ago
There is a problem with doing things this way. I want my development server to be always on, always available. It does things in the background and sometimes I use it when I'm away from the office. Yes, you can keep your desktop machine on all the time, but that's a huge power suck. Shut down your desktop when you're not using it. Most development servers just don't need to be that beefy anyway. A better approach, if you're absolutely committed to virtualization, is to just spin up an instance on Rackspace or Amazon. You'll pay about 10 bucks a month, which is less than you would for keeping your desktop running 24/7. On top of that, if your desktop dies, you won't lose your data.
- dchest 15y ago
  Not everybody needs to run development server 24/7. Plus, the title and the content of this post is "Replacing a Development VPS with Linux on OSX", and you're suggesting the opposite.
  - maratd 15y ago
    > Not everybody needs to run development server 24/7. Great. Shut down the instance. You pay nothing when it's off. > Plus, the title and the content of this post is ... I know what the article says. I'm telling you it's not a good idea, for the reasons listed above. I do run a few local images here and there, but that's only for situations where I don't have access to the internet.
    - dchest 15y ago
      > Great. Shut down the instance. You pay nothing when it's off. Not true for Rackspace.
      - maratd 15y ago
        > Not true for Rackspace. You're right. From what I hear, it may become a feature once they convert everything to openstack.
- 15y ago
emehrkay 15y ago
I recently got into an endless back and forth with a guy on reddit because I said that as a web developer, I require a mac (instead of a windows machine). When asked why I said that it is a hell of a lot easier to mimic testing/dev/production environments on a mac than windows and the only reason to choose a mac over a linux distro is because of the adobe dependance. His stance was that it isn't important to have your machine muddled up trying to replicate the dev/staging/production machines when you simply can develop on a remote box via some sort of ssh/file mounting. To which I replied with not always having access to remote dev machines, etc. What is your take on this issue? I was actually surprised that people want to develop on remote machines. It reminds me of "ftp to see a change"
- shadow_s 15y ago
  That's how I develop. Since I use vim anyways, I just ssh into a linux dev server from my windows box and code right on the server with the only delay being latency. 99% of what I do can be done in the command line so I end up with few problems.
- nkassis 15y ago
  bah, not really if you use editors like emacs or vim. I'd suggest also using screen to maintain a dev session going. screen lets you start shells and detach from a session without closing any of them (useful for long running jobs too). In the end my remote environment doesn't differ much from my local environment.
  - technomancy 15y ago
    I do most of my development on my server at home from my laptop (either at home or at coffee shops), but if I hadn't already invested in a beefy box I would definitely be using http://pair.io http://pair.io instead.
- dpritchett 15y ago
  I develop (Django/Rails) with Chrome running locally and my dev environment running in vim/tmux on a Ubuntu box that I ssh into. As long as the dev server automatically reloads when it detects a file change I see no issues. What are you thinking you'd miss in a setup like this?
anon_koder 15y ago
archlinux!
riobard 15y ago
Anyone knows the difference in performance between QEMU and Virtualbox both running on OS X?
jinushaun 15y ago
I also "deploy" to my local Linux VM for when I want to test a multi-server set up or try something new without messing up my main computer. Otherwise, I code and test almost exclusively on OSX. Haven't run into any situations yet where I wished I was coding in Linux as well. Linux doesn't have Photoshop or iTunes.
martijn_himself 15y ago
I'm following the instructions in the post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' Does anyone have any ideas why this is not working? I am using the netinstall Arch Linux ISO. Many thanks!
itodd 15y ago
Protip for those of you virtualizing Linux using VMware Fusion: You can force quit VMWare Fusion and your VMs will continue to run in the background.

91 points by remi 15y ago

There are some useful tips in that article. Using qemu in a no-gui mode is pretty cool. I've used VirtualBox, VMWare Fusion, and Parallels on OSX before, but I hate having to deal with the GUI window (and having the system reserve resources for it). Since OSX is based on Unix under the hood, it shouldn't be too difficult to for a developer to use either OSX or Linux as their development machine. It's not like the knowledge gap between Windows and OSX/*nix machines. So other than messing around with Linux, the article fails to mention why he doesn't use OSX (other than some unfounded 'flaky' and 'impossible' critiques).

ch0wn 15y ago
I used to work on Django project on a Mac. Dependencies like python-mysqldb can be quite tricky to install on OSX, whereas Ubuntu only requires you to type 'apt-get install python-mysqldb' and you're done. More importantly, though, I do not like the idea of having all those services constantly running on my desktop system. I'm using Ubuntu for both development and deployment now and I still use a separate VM.
Bockit 15y ago
I may be missing something but I think you can do what is proposed with VirtualBox. I use it for my virtual dev machines and run them in headless mode. For example the command: vboxmanage startvm "Name of VM" --type=headless to boot an image up, takes about 10 seconds to be ready to ssh into (You can ssh immediately it just waits till it has booted to connect). vboxmanage controlvm "Name of VM" poweroff If you want to turn it off from the outside.
jmlacroix 15y ago
Hello, I'm the author. I mostly use OSX for everything. The only times I switch to the VM is when I'm having a hard time installing something (for an example, see my earlier comment - http://news.ycombinator.com/item?id=2969509 http://news.ycombinator.com/item?id=2969509) or want to try a tool that's not available under OSX. I admit the post would have been better with examples and without then unfounded critiques, taking notes for the next ones.
- martijn_himself 15y ago
  Just wondering if you could think of anything obvious I'm doing wrong (I'm pretty much a newbie when it comes to this)... I'm following the instructions in your post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' I am using the netinstall Arch Linux ISO. Many thanks!
halostatue 15y ago
VirtualBox also can run without GUI windows: VBoxManage start vmname -type=headless If you need to see the GUI, you need the VNC extensions installed (which are, IIRC, not FOSS).
chrischen 15y ago
It's often a good idea to simulate your production server as closely as possible.
zokier 15y ago
I think one of the main reasons why I like VMs so much is that they allow my base OS to stay relatively clean. Especially as I like to try out all kind of new stuff constantly, VMs make that easy without having to care how they affect the rest of my system. Even if you aren't trying out new stuff, keeping separate projects mostly isolated makes me feel happier. imho also configuring stuff becomes easier when your machines become basically single-purpose systems.
mentat 15y ago
If you want no-gui VirtualBox that "just works" try vagrant.

tombell 15y ago

Wonder if the author has considered checking out http://vagrantup.com/ http://vagrantup.com/ unless it does a bit too much for what they're after.

lisperforlife 15y ago
+1 for vagrant. It is awesome. It integrates well with Chef for provisioning. I would definitely recommend it.

1880 15y ago

I currently use a similar environment but with VirtualBox. Does qemu offer any advantages over it? I was considering recreating my VM so I may as well try qemu.

jmlacroix 15y ago
It's been a while since I last used VirtualBox, so I can't really compare. What I like in QEMU, is its simplicity. The learning curve is a bit steeper, but when you're used to it there's no turning back.

mise 15y ago

On a tangent to this, what is the "Linuxy" stuff you don't get to do on OS X? (My own frame of reference being an Ubuntu user.)

zokier 15y ago
Apt with 20000+ packages waiting to be installed.
mhd 15y ago
The author might have a different experience, but for me it's mainly anything that a) depends too heavily on a plethora of GNU tools and b) anything that's about deployment and installation. You're basically running on a different Unix system, so if you deploy to a Linux server, you'll probably encounter quite a lot of differences - mostly deployment, but also libraries, syscalls etc. Quite reminiscent of the days of different commercial Unices, where the compatibility of e.g. AIX and HP-UX wasn't exactly 100% (I'm sorry if that caused some violent flashbacks for veteran developers) Also, OS X is usually behind even compared to Linux distros with fixed releases (i.e. not rolling). You get your basic Unix software in one fell swoop, with the occasional update through a dot version. Apple doesn't provide a pkgsrc or even a rpm/deb packaging system. There are third party efforts, like MacPorts, Fink or – as the author mentioned – Homebrew, which is more similar to BSD's package tree or Arch's AUR (basically a git repo of Ruby build scripts). Nevertheless, these add on to your system, so aren't really integrated perfectly. Also, compared to Debian or Arch, you'll have a smaller package list (esp. true for Homebrew). So it might just be easier to install some stuff from within a headless Linux VM. Heck, I've been doing something similar even when I was running Linux as a dev environment. Your deployment environment might be quite different, and I like to do some local testing first – doubly so if your company doesn't even have a proper testing/integration environment. I had an Arch PC, but deployed to both somewhat recent Ubuntu machines and ancient SuSe installations. Now, I often work on OS X and deploy to a CentOS VPS. All good use cases for VMs – and the way described, where it's just a small headless instance in the background instead of a useless GNOME desktop-within-a-desktop is a pretty good approach.
- jmlacroix 15y ago

zobzu 15y ago

OSX is an OS where you run VMs for Linux and Windows. I keep seeing people saying how great and fast it is to do stuff on their mac and run Windows for a few apps incl. mail client, office suites, some dev tools and many java apps (cause they're often not so nice on OSX) Then I see them with the Linux VM for all the dev tools, the hack tools, and so forth, because, well, hacking them up to work in OSX is always a pain, and even using stuff like brew is often a pain (+ delay) Well I dunno, I run Linux native and I get stuff done faster than the ones running Linux in a VM of course. I also have Linux VMs under Linux when necessary, with KSM and it's a lot faster and memory efficient than when running OSX native on the same hardware. I install stuff in about <5s where it takes a VM to boot/resume in OSX, and about 5-10min to install from scratch when the app _supports_ OSX (ages if it doesn't but has more or less compliant sources)

MatthewPhillips 15y ago
Yeah, me too. I have a linux computer for most development, a Macbook when I want to mess around with iOS development, and a Thinkpad for .NET development. All of these are tax write-offs and since I use the linux computer (which twice as old as the other two, by the way) 90% of the time because it's simply easier to install development related stuff. Whenever I want to install a gem that uses native headers requiring a -dev package in OSX I just quit.
ubercore 15y ago
I'm the opposite. I like all of my OS X dev tools (navicat, textmate/macvim (I know, gvim...)/sublime text/github for os x/things), and it's a much more usable OS overall for me. The parts I don't like, compiling libraries, running servers, etc, I automate away with vagrant. I'm more productive with this setup than if I ran everything directly in Linux (I've tried).
barumrho 15y ago
Your points are valid, but running VMs, and using unix tools are not all I do with my computer. For me, running OS X is a good compromise. I used Linux exclusively for more than four years, and I miss many things about it, but I don't think I can go back, at least yet.

pmarsh 15y ago

Unless you're on an ancient laptop I don't see why you wouldn't run a Linux VM if you are developing web apps. What is the advantage of trying to make OS X your development environment and then deploying to a Linux host? Unless you have an older laptop with less than 4 gigs of ram why not? Even with 2gigs it's not that system intensive to run a Linux guest os.

riobard 15y ago
Let's face it: SSH into an VM and write code there is not an enjoyable experience. Yes I can do a proper deployment into the VM right after I make a minor change in color of the 10px square box located at the bottom of the page, but it's really a pain in the ass to have the deployment delay. Oh, right, the deployment process just takes 10 seconds!
- pmarsh 15y ago
  Why SSH into a VM? With Virtualbox I have a folder shared out that I can use like a regular folder and edit with whatever editor on my Host. Or you could turn on SMB shares and use those. Or I could run a lightweight X environment (or even gnome it really doesn't need much to run) and just edit there. Everyone seems to be on the "OMG Vim/Emacs are so awesome, TextMate is so yesterday" part of the every going cycle anyway. Might as well use them on the guest OS as well.
  - riobard 15y ago
    Virtual shared folder is rather flaky for me on OS X. It magically disappears and reappears in Linux guests.
olefoo 15y ago
And it's worth running a vm for development even if you're hosting it on a linux machine; presumably your workspace has a bunch of things that don't need to be installed in production.

maratd 15y ago

There is a problem with doing things this way. I want my development server to be always on, always available. It does things in the background and sometimes I use it when I'm away from the office. Yes, you can keep your desktop machine on all the time, but that's a huge power suck. Shut down your desktop when you're not using it. Most development servers just don't need to be that beefy anyway. A better approach, if you're absolutely committed to virtualization, is to just spin up an instance on Rackspace or Amazon. You'll pay about 10 bucks a month, which is less than you would for keeping your desktop running 24/7. On top of that, if your desktop dies, you won't lose your data.

dchest 15y ago
Not everybody needs to run development server 24/7. Plus, the title and the content of this post is "Replacing a Development VPS with Linux on OSX", and you're suggesting the opposite.
- maratd 15y ago
  > Not everybody needs to run development server 24/7. Great. Shut down the instance. You pay nothing when it's off. > Plus, the title and the content of this post is ... I know what the article says. I'm telling you it's not a good idea, for the reasons listed above. I do run a few local images here and there, but that's only for situations where I don't have access to the internet.
  - dchest 15y ago
    > Great. Shut down the instance. You pay nothing when it's off. Not true for Rackspace.
    - maratd 15y ago
      > Not true for Rackspace. You're right. From what I hear, it may become a feature once they convert everything to openstack.
15y ago

emehrkay 15y ago

I recently got into an endless back and forth with a guy on reddit because I said that as a web developer, I require a mac (instead of a windows machine). When asked why I said that it is a hell of a lot easier to mimic testing/dev/production environments on a mac than windows and the only reason to choose a mac over a linux distro is because of the adobe dependance. His stance was that it isn't important to have your machine muddled up trying to replicate the dev/staging/production machines when you simply can develop on a remote box via some sort of ssh/file mounting. To which I replied with not always having access to remote dev machines, etc. What is your take on this issue? I was actually surprised that people want to develop on remote machines. It reminds me of "ftp to see a change"

shadow_s 15y ago
That's how I develop. Since I use vim anyways, I just ssh into a linux dev server from my windows box and code right on the server with the only delay being latency. 99% of what I do can be done in the command line so I end up with few problems.
nkassis 15y ago
bah, not really if you use editors like emacs or vim. I'd suggest also using screen to maintain a dev session going. screen lets you start shells and detach from a session without closing any of them (useful for long running jobs too). In the end my remote environment doesn't differ much from my local environment.
- technomancy 15y ago
  I do most of my development on my server at home from my laptop (either at home or at coffee shops), but if I hadn't already invested in a beefy box I would definitely be using http://pair.io http://pair.io instead.
dpritchett 15y ago
I develop (Django/Rails) with Chrome running locally and my dev environment running in vim/tmux on a Ubuntu box that I ssh into. As long as the dev server automatically reloads when it detects a file change I see no issues. What are you thinking you'd miss in a setup like this?

anon_koder 15y ago

archlinux!

riobard 15y ago

Anyone knows the difference in performance between QEMU and Virtualbox both running on OS X?

jinushaun 15y ago

I also "deploy" to my local Linux VM for when I want to test a multi-server set up or try something new without messing up my main computer. Otherwise, I code and test almost exclusively on OSX. Haven't run into any situations yet where I wished I was coding in Linux as well. Linux doesn't have Photoshop or iTunes.

martijn_himself 15y ago

I'm following the instructions in the post on my Macbook Pro (2010); however I get 'Booting from DVD/CD... 180MB medium detected. Boot failed: Could not read from CDROM (code 0005) No bootable device.' Does anyone have any ideas why this is not working? I am using the netinstall Arch Linux ISO. Many thanks!

itodd 15y ago

Protip for those of you virtualizing Linux using VMware Fusion: You can force quit VMWare Fusion and your VMs will continue to run in the background.