9 ms·

Why I stuck with Windows for 6 years while developing Discourse

100 points by sams99 8y ago

xvilka 8y ago
By the way, is there something like Discourse but in Go?
- sams99 8y ago
  Unless you are looking to contribute to a project and kick off a project I would recommend sticking with the bigger players our there like nodebb/vanilla and so on. Getting stuff right is not easy, very few forum software platforms out there for example have a bug bounty like (https://hackerone.com/discourse https://hackerone.com/discourse ), last thing you want to do is deploy an XSS hive out there to the public. As far as I know there is no small-medium well supported Discourse alternative written in Go. There is a Slack alternative though written in Go called mattermost which I can recommend.
  - edoceo 8y ago
    My company just switched from Slack to Mattermost, love it.
- favadi 8y ago
  Why the language matters here?
  - xvilka 8y ago
    Because of the performance, size, and ease of deployment.
    - mwcampbell 8y ago
      I wonder what annoys the Discourse team more: actual problems with Ruby, or discussions of how much better language X would have been compared to Ruby. I think Sam made a goode point on this sub-thread; the maturity and overall health of a project are more important than what language it uses. (Disclosure: I did a few small contributions to Discourse a few months ago, but never got more deeply involved than that.)
  - scoutt 8y ago
    I don't know on what depends, but I run a small Discourse forum... and I can tell you is a major PAIN! I'm not a web developer (I develop mostly firmware) so I'm not into all this kind of stuff, but I know it is written in Ruby and runs in a Docker container. It requires at least 10GB just to be able to install and 20GB just to update once in a while. It takes ages to compile. It's a PITA to update. Plus docker gets stuck with old images (not sure it's the right term) and you'll have to manually clear them. 20GB!! It's a freakin' forum software!!! Some triple A games are around 8GB. I'm glad someone is asking for alternatives. I think all this overhead is because maybe redistributing a Ruby website it's a PITA and requires a full environment installation. And hell yeah I believe a simple test suite can take 10 minutes to run in that poorly conceived environment.
    - ksec 8y ago
      It surely doesn't requires 10GB, although it does require a significant amount of space in Multiple GB due to the way it works in Docker. May be something could be done to the base image. Or just try running ./launcher cleanup
      - scoutt 8y ago
        I don't want to seem rude, but to my ignorant eyes and those of its users, it's a forum software that requires a 10GB hard disk to install, and 20GB to update. I know that there will be a lot of technical details that will justify (or not) why, but, hey, that's a lot for a forum and there is no sane alternative way to install it. I've found that I'm not alone: https://meta.discourse.org/search?q=disk%20space https://meta.discourse.org/search?q=disk%20space There are entire pages of users complaining about the same or related problems. At the end of the day, it's (almost) free... so what are we complaining for?
        ksec 8y ago
        I know. It surely sounds silly, but my guess is that instead of having to troubleshoot thousands of different possibl configuration with different dependency and OS library. The way it is currently distributed is not just "forum" software, it is the whole OS + Database and All other Software required along with Backup etc.
- lazyjones 8y ago
  https://github.com/godiscourse/godiscourse https://github.com/godiscourse/godiscourse https://github.com/disintegration/bebop https://github.com/disintegration/bebop
StillBored 8y ago
? What I see when I look at that benchmark is a comparison between linux running in a VM, via WSL and natively. AKA, VMs are slower than bare metal. News at 10. Sure there are things windows tends to be slower at, but similarly there are things linux tends to be slower at. For general desktop usage I think you will find two things, windows does quite well, and desktop level virtualization tends to be slower than server virtualization because most desktop users will be using a virtio/emulation IO access method vs servers which tend to be punching PCIe adapters into the VMs. Either way, the overhead of additional translation layers and VM exits for various things is always going to be worse than just running on bare metal. Wether that works out to barely noticeable for compute heavy benchmarks that are TLB friendly, 2x for small packet edge case IO's, or somewhere in between is completely application dependent.
- sams99 8y ago
  WSL is not technically a VM... The news is not the it is slower, it is how much slower it is. I found the number surprising given the prevailing wisdom out there was ... sure its slower, about 5-10% slower. The other news is that mac is slower and it does not do any virtualising.
  - AstralStorm 8y ago
    WSL is quite slow indeed, compared even to a low performance VM like Virtualbox w/ a bit of hardware help. Much less to something like KVM with paravirtualization drivers, or even Hyper-V likewise.
- mappu 8y ago
  > VMs are slower than bare metal. News at 10. The other interesting part of the benchmark is the i7-8559U (Coffee Lake, 4C8T, 2.7GHz) macOS vs i7-8750H (Coffee Lake, 6C12T, 2.2GHz) Ubuntu, on a single-threaded benchmark ("We are going to adopt parallel testing for our dev environments this year"). MacOS took 1.6x as long as Ubuntu, despite a 1.2x GHz advantage, for an overall 1.9x slowdown core-per-core on the same microarchitecture. Would you like 2x performance? Just replace macOS with Ubuntu. "There is no other operating system out there that competes against us at this time" - Greg Kroah-Hartman, 2018
mwcampbell 8y ago
I wonder how big a role Windows Defender, or other antivirus software, plays in the notorious inefficiency of reading and (especially) writing many small files on Windows. Sam, did you disable Windows Defender real-time protection, or add exceptions for it?
- sams99 8y ago
  Absolutely disabled defender. When I started running the spec suite in WSL, defender went way up (so I gave it a break), pretty sure stuff would have been way worse with defender on. The prevailing recommendation though from the WSL team is to leave defender on for now. So I kind of cheated by disabling it.
0x8BADF00D 8y ago
VM will always be shit at compilation tasks. All of the existing solutions out there (including Docker) will lead to slower builds, especially if you need to build an embedded Linux distribution. Native is better, for those tasks. The haters he’s replying to are literally splitting hairs. Wow, they shave off 3 mins to run an interpreted language test suite. This is the ABSOLUTE state of the industry right now.
- jimktrains2 8y ago
  > VM will always be shit at compilation tasks. All of the existing solutions out there (including Docker) Docker isn't a VM; it runs with the same kernel and directly on the host processor like any application.
  - 013a 8y ago
    Except the filesystem. Which is what everyone seems to forget. And that's the big thing we're talking about here in reference to Ruby and tests being slow.
    - edude03 8y ago
      The file system is native too depending on what filesystem your host is running. If the host is running a Copy on Write filesystems like ZFS or BTRFS docker can use it natively.
    - pmontra 8y ago
      Even if you mount the development directory on the host file system? This is probably necessary to be able to edit files. In the case of Ruby maybe you should mount also the language manager directory with all the gems and the interpreter.
  - throwaway2048 8y ago
    Unless your host system isn't linux (like in this case)
mastazi 8y ago
I've used Linux as my main desktop OS for about 3 years, from around 2010 to around 2013 (so maybe some things changed in the meantime). The article says that you pay a "VM tax" by running in a Linux VM in Windows or by using WSL, but by using Linux directly you end up "paying" in other areas, for example in terms of less-than-perfect drivers (which leads to problems such as worse battery life, worse graphical performance etc). In an ideal world, laptop and GPU makers would put as much effort developing Linux drivers as they do when they create Windows drivers, but unfortunately this is not the case. Another point is that, even if you are running Linux on your dev machine, it is often challenging to have a local environment that is as close as possible to your production one; for example where I work we have Ubuntu 16.04, Ubuntu 18.04 and CentOS (and with various combinations of installed packages), even if I was using a Linux laptop, I would still need containers to get all the different environments right. The only case where you could use no containers at all is if you had all of your servers running the exactly the same environment. Finally there is the issue of some pieces of software not being available for Linux, 95% of the times you can find a Linux equivalent that works for you but, in my experience, there were still rare cases where I had to resort to Wine or some Win VM just to run some specific tool that was needed to do my job. PS My job, at the time I was using Linux as my main desktop OS, and during the couple of years after I switched back to Windows, was about PHP, Python and Java web apps deployed on Linux servers, so my comments above should be taken in this context. Maybe in other fields of software development there are different factors to take into account. PPS I have also tried MacOS, on the one hand it's great that Mac is UNIX, but on the other hand it's not Linux so, if you want to closely replicate your production Linux environment, you will still need VMs or containers.
- baroffoos 8y ago
  I picked a Dell XPS for development and it runs linux amazingly well. Only bit that doesn't work out of the box is the fingerprint scanner but there are unofficial drivers for it. With web development its so trivial to do everything you need on linux and almost all of the time its easier to set up on linux or mac than on windows.
  - themodelplumber 8y ago
underwater 8y ago
If the time taken to run the test suite is so critical, then how is the "winning" result of 7m55s any better? In my experience any multi-minute operation invites a context switch or coffee break, and end up being roughly equivalent in practice.
- sams99 8y ago
  I completely understand the concern here. If every change required an 8 minute wait it would be real sad for us. We use a tool called `rake autospec` at Discourse which is a smart test runner. I personally like the vim integration we built. The way it works... 1. Run `bin/rake autospec` in a terminal 2. I head to the code I want to change, I change the code 3. The spec runner figures out what the right test is run and runs that. 4. While it is busy running specs at any time it can be interrupted by saving a "*.spec" file in which case the spec runner will run the spec at the cursor This has worked wonderfully well for us.
  - majewsky 8y ago
    FYI, list formatting works best when you leave an empty line between each item. 1. just 2. like 3. this
- ufmace 8y ago
  That's exactly what I thought - a 50% faster test suite run is nice I guess, but they're all still way too slow to be running all the time. I'll run the tests in 1 file or subset of files or rspec describe block or something that's relevant to what I'm doing and finishes in <10s. Save the multi-minute full runs for CI or big changes where you have no idea what will happen.
NightlyDev 8y ago
I've been using Linux as my main OS for over 10 years now, mainly Ubuntu. The best part is the performance and a realistic bare metal environment. You can profile code and actually trust the results. You can't say the same for virtualizarion, containers or other platforms.
ryan-allen 8y ago
> Here I am freaking out about a measly 38% perf hit when I could be running stuff concurrently and probably be able to run our entire test suite in 2 minutes on my current machine on Windows in a VM. I think this is the point, given you can get an inexpensive desktop CPU with 6 cores and 12 threads, it wouldn't matter so much if you had a 20% perf tax with WSL when you have approx 12x the throughput. You could always offload full test suites to an external CI box upon check-in and work with a subset of unit tests locally, once the suite becomes prohibitive for a single machine.
evancox100 8y ago
Meanwhile, in ASIC/hardware testing land, we're lucky to get a regression suite done in under 24 hours.
amanzi 8y ago
Curious to know why you were using VMware on Windows instead of Hyper-V? I would expect Hyper-V to be significantly faster than VMware.
- gorjusborg 8y ago
  I gave Hyper-V way too much of my time last tried setting up a development environment on windows (around September 2018). The Hyper-V tooling just isn't good enough for the developer use case compared to Virtualbox and VMWare, and once you enable Hyper-V, it owns the virtualization environment within Windows, meaning you can't run other hypervisors on the OS. That was a deal-breaker for me, as Docker for Windows wouldn't even run out of the box.
  - vbezhenar 8y ago
    Docker by default runs on Hypet-V.
    - bragh 8y ago
      Did it always used to do that, wasn't it VirtualBox only in the beginning? Also, https://github.com/MicrosoftDocs/Virtualization-Documentation/issues/945 https://github.com/MicrosoftDocs/Virtualization-Documentatio...
      - vbezhenar 8y ago
        I'm not sure about the beginning, but it uses Hyper-V for a while. I don't know about that bug, I used it with Windows 10 Pro, I guess bugs happens, probably they will fix it soon if it's widespread. I know that you need a special Docker package (Docker Toolbox) for VirtualBox and default Docker package from their website requires Hyper-V.
- wvenable 8y ago
  I don't see why Hyper-V would be faster but VMware is much nicer to work with (in my opinion).
lazyjones 8y ago
Why do people run their server software on their desktop systems instead of, say, a proper server? Back in the days our desktop systems were VT102s, NCD X-Terminals or underpowered laptops and developing on the server through rsh/ssh or even VNC was natural. Does Ruby somehow require locally running GUI processes?
- mwcampbell 8y ago
  There's certainly nothing stopping you from doing Ruby or JS dev work on a remote server. I run Windows on my laptop, so when I wanted to start contributing to Discourse on the side a few months ago, I ordered a VPS at DigitalOcean and did my work there over SSH. I used SSH tunneling to access the dev HTTP server. Edit: It helps that I use vim as my editor.
  - pletnes 8y ago
    I’ve done similar things with pycharm. I can run pycharm locally but run a remote python process. .py files are automatically uploaded and I can debug over the network just fine. I know eclipse, and presumsbly others, can do the same.
- hamandcheese 8y ago
  At my current company we do exactly this. It’s nice because it’s someone else’s problem to make sure my dev environment works, I can recreate my dev environment with a single command, and I have a shareable URL to my Rails instance that anyone on the eng VPN can use (makes it easy to show works in progress to teammates or to the product team). It does kind of suck in some ways though. I can’t develop without internet, and debugging is worse than debugging locally. The debugging issue could probably be alleviated with some more investment in developer tooling/editor integrations.
- kristianp 8y ago
  Because you can, Convenience, and compatibility. Since linux became the OS of the internet, people have been able to run the same software on their own computer. It's more convenient to just run the "server software" on your own machine and you have full control over it.
robocat 8y ago
It is unfortunate that there is no comparison against an Ubuntu host with an Ubuntu guest. "I was paying a pretty high tax for deciding to stick with Windows” was likely "I was paying a pretty high tax for deciding to use a VM”. I have little love for Windows, but I would prefer to see numbers that place the blame correctly.
- mwcampbell 8y ago
  I'm thinking of renting a Vultr bare metal server for an hour or two to do the test you're suggesting. What's the easiest way to spin up a VM with KVM from the command line these days? Is it still libvirt's virsh?
- ubercow13 8y ago
  Why? The tax is that using Windows means you have to use a VM, not that VMs are less performant on Windows
  - robocat 8y ago
    Well, I use an Ubuntu host on a Dell XPS (since it just works without extra configuration). But I also use an Ubuntu guest VM for development so that I can shove the guest onto a different machine easily (yeah, I know I should use continuous integration, but I haven't yet). Using a VM definitely has a big performance cost... Even though I value the benefits I accept the costs.
    - ubercow13 8y ago
      You might be able to use containers on Linux to achieve the same thing without the performance cost
      - robocat 8y ago
        Very likely. But I know that in X years time I can move a copy of my development VM to run as a guest on any OS I might happen to use. I haven't yet spent any time learning how to use containers, or what their compromises are, so for the moment I am still ignorant of that solution.
jacobsenscott 8y ago
Having used windows for several years, linux for several years, and currently macos for several years (because that's what work gave me) - linux is by far the most stable and easiest to manage OS, after you get past the initial learning curve. The main reason is you can actually fix problems. You pretty much know why every single bit of software is there for and how to configure it. You google the problem and someone has released a patch, or instructions on what configuration changes need to be made. Or you can roll back to old versions of software easily. Also you are a hero because you catch and fix production problems nobody else sees or understands because production is running linux. If you have a problem on OSX you just get 500 people telling you to reset your pram, and really you just need to wait for the next osx update. You certainly can't rollback an osx update. Nobody knows whats wrong because nobody has access to the code. Even if you did - each update is millions of lines of code. It is crazy town. Same for windows except there is not pram to reset. You just reboot until the next update I guess.
- deleted 8y ago
  [deleted]
- titanix2 8y ago
  > linux is by far the most stable and easiest to manage OS Nice April fools.
  - kuzimoto 8y ago
    Anecdotally, this is true for me. My Arch machine runs for months with no issues. My XPS running 10 blue screens once every several weeks, and generally has issues after running too long. To be fair it could be due to the general buggy-ness of the TB16 Thunderbolt dock.
    - titanix2 8y ago
      Yeah of course, different people will have different experiences. Mine was being never able to make wifi work, random destruction of the GUI when I plugged an external monitor, difficulty to install and configure eastern languages IME, sound issues, etc. That was a very instructive period as undergraduate in informatics, but I got tired after a while to constantly fix my system instead of getting things done. Still using it as a server OS, where it shines.
craz8 8y ago
I added parallel tests to our app last year. My 4 core/8 thread system runs the tests pretty well with a process count of 6. The suite runs about 4 times faster, which makes a real difference No need to wait for Rails 6 to get this boost
ncmncm 8y ago
It never occurred to him to run Linux on his machine, for real work, and keep Windows in a VM for when he really needs it? Surely someone, somewhere has explained that Windows runs faster in a VM than on bare metal, because Linux is better at file systems and buffer management? Or that relying on drivers from all over makes your system less reliable than the fixed set used in the VM? Me, I cannot imagine running MSWindows on bare metal. It just feels wrong.
pixelmonkey 8y ago
If you run Linux on vanilla Intel ultrabook or Intel desktop/server hardware, it is not only blazing fast, but all the hardware works out of the box, too. I wrote about this in the context of Lenovo X1C laptop here: http://amontalenti.com/2017/09/01/lenovo-linux http://amontalenti.com/2017/09/01/lenovo-linux I also recently built a Linux desktop from ~$950 of commodity parts (including a pair of free GPUs a crypto friend donated to me after his startup died). I use it as a devserver and a GPU rig for playing with CUDA, PyTorch, and TensorFlow. Aside from the requirement of proprietary Nvidia drivers, this whole box works perfectly in Linux too, is blazing fast, and operates silently and with low power consumption. (The case itself is bulky, but it’s stationary.) I think an equivalent Mac would cost a 2-3x multiple and run less well for developer workloads. https://pcpartpicker.com/user/amontalenti/saved/r4rLJx https://pcpartpicker.com/user/amontalenti/saved/r4rLJx Say what you will about Linux, but if you choose your hardware carefully, it truly does “just work” these days. And you can’t beat having access to scriptable everything and source code everywhere. That said, I keep a Mac Mini around because there is some proprietary stuff you can’t avoid in the Apple ecosystem (e.g. XCode for iOS, Safari Debugger, Keynote/Pages, ...)
bgdnyxbjx 8y ago
This whole Twitter thread and blog post gave me real pause for ever wanting to work at Discourse.
- mwcampbell 8y ago
  Why, exactly?
impostir 8y ago
I have been using Linux as my daily driver for a little over 2 years and day to day, I haven't had practically any problems. Every once in a while, I will mess with something and then things break, but I know I can fix it. Knowing I can find a fix to any day to day problem has become absolutely crucial to me; I can't go back to chasing windows error codes.
pmontra 8y ago
Ruby has always been a nightmare on Windows. Every time a new junior freelance developer is onboarded in a Rails project everybody hopes he's got a Mac or Linux laptop. A significant number of them uses Windows and we always suggest them to run a Linux VM. Some of them resist the suggestion. We make clear we're not wasting time to support their environment (we're deploying on Linux anyway) and that they are on their own. After not to much time they comply or leave. After months some of them could even buy a Mac or dual boot on Linux (Windows is good at videogames.) Still, AFAIK VirtualBox has a problem with supporting multiple cores. Single core VM is heavily handicapped even for Ruby. Parallel testing was not in Rails but it's definitely a thing.
- zubspace 8y ago
  Maybe your company should provide the optimal hardware which everyone must use? Each employee using a similar environment is a productivity boost. And I can tell you, that if everyone would use the same system, even windows, you would have less problems. Sounds alot simpler than letting everyone use their own environment..
  - pmontra 8y ago
    Not my company (I'm a freelancer) and every freelancer brings his own laptop. Furthermore all of us work from remote, often even employees do. I've seen this happen everywhere with the small/medium customers I'm working for.
  - bluedino 8y ago
    We were in a similar situation (20/80 Linux/Mac devs, respectively) You can use Windows, but none of the rest of us do so you're on your own. You can have a MBP, iMac, or ThinkPad with Linux when you start. Your choice. We had two developers actually say they were wary of learning Mac/Linux and didn't take the job for that reason.
rcarmo 8y ago
I’ve been doing an increasing amount of Node development in WSL, and the thing that helped me the most where it regards disk performance was adding an exclusion path to Windows Defender so that it wouldn’t constantly scan my WSL working tree. Also, I get it that some kinds of testing “feel better” when done locally, but... why not have those run on a cloud VM?
snarfy 8y ago
In the late 90s I installed Slackware on a 386DX 40mhz with kernel 1.2.8, XFree86, elvis vi clone, etc. I've been using Linux for a long time, on and off as my main desktop. I have various Linux machines around the house, but it's not my main desktop today. There's too much fiddling around to get it working right as a desktop. Audio has always been a ghetto. Wifi drivers are still binary blobs. Fonts, hidpi, multi monitor support, wayland, systemd are all still issues today. I really don't want my development machine to match production. Production is a stripped down image for a reason. I spend far more time writing code than running it, so the VM performance hit is a moot point. To be fair, Windows 10 has plenty of issues of its own that need tweaking, like having the only way to disable cortana is through group policy edits. And then they reenable it on the next update, and give me some candy crush ads in my start menu. It's pretty infuriating, but still less so than manually fixing wifi drivers through USB boot drives.
nickjj 8y ago
WSL isn't too bad nowadays. I would go as far as saying it's very good. I've been using it for full time development for over a year now. I've run 100,000+ line Rails apps in WSL (well technically through Docker, which I connect to through WSL) and I never noticed a slowdown that was bad enough to make me think "this sucks". It's always been pretty good. I run all sorts of Rails, Flask, Phoenix and Webpack driven apps and all of them run fast enough where I don't think twice about it. Personally, I find the WSL set up somewhat close to native Linux in terms of the user experience. I'm not talking about I/O performance, but I mean how it feels to use the OS in general. For example: I spend 99% of my time in a WSL terminal using tmux + terminal Vim + ranger. So that takes care of coding and managing files. Then I use a browser and other graphical apps (image / video editors) that run directly in Windows. Dexpot sets up virtual screens with key binds that are comparable to how i3wm lets you switch and move windows to another screen. Keypirinha lets you launch apps with a fuzzy finder (like dmenu but better IMO) AutoHotkey lets you easily manage global hotkeys (just like i3 does) and more When you put all of that together, you get a really really good development experience that also doubles for gaming and running programs that don't have a good alternative on Linux (such as Camtasia on Windows). Then for the icing on the cake, since you're running Ubuntu 18.04 in WSL, you can provision WSL itself with the same exact configuration management scripts you would use to provision a production box. For me specifically I run all of the same Ansible roles against WSL that I do in production. I can set the whole thing up with 1 Ansible command. Plus my dotfiles also happen to work exactly how they do on my native Linux laptop so it's easy to keep things in sync and feeling the same. This all runs from a i5 3.2ghz / 16GB of RAM / SSD / etc. $750 desktop built 5 years ago. Even if Apple tax didn't exist I would still use this Windows / WSL set up if I weren't in a position where I could run Linux natively.
bluedino 8y ago
I love OS X, but it's just 'slow'. I have no problem admitting that. It's much better than it was in the PowerPC days, when (IMO) OS X was almost too slow to be usable. We have it good these days with faster graphics and 12-core CPU's and super fast SSD's... But you give these same advantages to Windows/Linux and OS X is just handily beaten in just about every benchmark you can think of.

100 points by sams99 8y ago

xvilka 8y ago

By the way, is there something like Discourse but in Go?

sams99 8y ago
Unless you are looking to contribute to a project and kick off a project I would recommend sticking with the bigger players our there like nodebb/vanilla and so on. Getting stuff right is not easy, very few forum software platforms out there for example have a bug bounty like (https://hackerone.com/discourse https://hackerone.com/discourse ), last thing you want to do is deploy an XSS hive out there to the public. As far as I know there is no small-medium well supported Discourse alternative written in Go. There is a Slack alternative though written in Go called mattermost which I can recommend.
- edoceo 8y ago
  My company just switched from Slack to Mattermost, love it.
favadi 8y ago
Why the language matters here?
- xvilka 8y ago
  Because of the performance, size, and ease of deployment.
  - mwcampbell 8y ago
    I wonder what annoys the Discourse team more: actual problems with Ruby, or discussions of how much better language X would have been compared to Ruby. I think Sam made a goode point on this sub-thread; the maturity and overall health of a project are more important than what language it uses. (Disclosure: I did a few small contributions to Discourse a few months ago, but never got more deeply involved than that.)
- scoutt 8y ago
  I don't know on what depends, but I run a small Discourse forum... and I can tell you is a major PAIN! I'm not a web developer (I develop mostly firmware) so I'm not into all this kind of stuff, but I know it is written in Ruby and runs in a Docker container. It requires at least 10GB just to be able to install and 20GB just to update once in a while. It takes ages to compile. It's a PITA to update. Plus docker gets stuck with old images (not sure it's the right term) and you'll have to manually clear them. 20GB!! It's a freakin' forum software!!! Some triple A games are around 8GB. I'm glad someone is asking for alternatives. I think all this overhead is because maybe redistributing a Ruby website it's a PITA and requires a full environment installation. And hell yeah I believe a simple test suite can take 10 minutes to run in that poorly conceived environment.
  - ksec 8y ago
    It surely doesn't requires 10GB, although it does require a significant amount of space in Multiple GB due to the way it works in Docker. May be something could be done to the base image. Or just try running ./launcher cleanup
    - scoutt 8y ago
      I don't want to seem rude, but to my ignorant eyes and those of its users, it's a forum software that requires a 10GB hard disk to install, and 20GB to update. I know that there will be a lot of technical details that will justify (or not) why, but, hey, that's a lot for a forum and there is no sane alternative way to install it. I've found that I'm not alone: https://meta.discourse.org/search?q=disk%20space https://meta.discourse.org/search?q=disk%20space There are entire pages of users complaining about the same or related problems. At the end of the day, it's (almost) free... so what are we complaining for?
      - ksec 8y ago
        I know. It surely sounds silly, but my guess is that instead of having to troubleshoot thousands of different possibl configuration with different dependency and OS library. The way it is currently distributed is not just "forum" software, it is the whole OS + Database and All other Software required along with Backup etc.
lazyjones 8y ago
https://github.com/godiscourse/godiscourse https://github.com/godiscourse/godiscourse https://github.com/disintegration/bebop https://github.com/disintegration/bebop

StillBored 8y ago

? What I see when I look at that benchmark is a comparison between linux running in a VM, via WSL and natively. AKA, VMs are slower than bare metal. News at 10. Sure there are things windows tends to be slower at, but similarly there are things linux tends to be slower at. For general desktop usage I think you will find two things, windows does quite well, and desktop level virtualization tends to be slower than server virtualization because most desktop users will be using a virtio/emulation IO access method vs servers which tend to be punching PCIe adapters into the VMs. Either way, the overhead of additional translation layers and VM exits for various things is always going to be worse than just running on bare metal. Wether that works out to barely noticeable for compute heavy benchmarks that are TLB friendly, 2x for small packet edge case IO's, or somewhere in between is completely application dependent.

sams99 8y ago
WSL is not technically a VM... The news is not the it is slower, it is how much slower it is. I found the number surprising given the prevailing wisdom out there was ... sure its slower, about 5-10% slower. The other news is that mac is slower and it does not do any virtualising.
- AstralStorm 8y ago
  WSL is quite slow indeed, compared even to a low performance VM like Virtualbox w/ a bit of hardware help. Much less to something like KVM with paravirtualization drivers, or even Hyper-V likewise.
mappu 8y ago
> VMs are slower than bare metal. News at 10. The other interesting part of the benchmark is the i7-8559U (Coffee Lake, 4C8T, 2.7GHz) macOS vs i7-8750H (Coffee Lake, 6C12T, 2.2GHz) Ubuntu, on a single-threaded benchmark ("We are going to adopt parallel testing for our dev environments this year"). MacOS took 1.6x as long as Ubuntu, despite a 1.2x GHz advantage, for an overall 1.9x slowdown core-per-core on the same microarchitecture. Would you like 2x performance? Just replace macOS with Ubuntu. "There is no other operating system out there that competes against us at this time" - Greg Kroah-Hartman, 2018

mwcampbell 8y ago

I wonder how big a role Windows Defender, or other antivirus software, plays in the notorious inefficiency of reading and (especially) writing many small files on Windows. Sam, did you disable Windows Defender real-time protection, or add exceptions for it?

sams99 8y ago
Absolutely disabled defender. When I started running the spec suite in WSL, defender went way up (so I gave it a break), pretty sure stuff would have been way worse with defender on. The prevailing recommendation though from the WSL team is to leave defender on for now. So I kind of cheated by disabling it.

0x8BADF00D 8y ago

VM will always be shit at compilation tasks. All of the existing solutions out there (including Docker) will lead to slower builds, especially if you need to build an embedded Linux distribution. Native is better, for those tasks. The haters he’s replying to are literally splitting hairs. Wow, they shave off 3 mins to run an interpreted language test suite. This is the ABSOLUTE state of the industry right now.

jimktrains2 8y ago
> VM will always be shit at compilation tasks. All of the existing solutions out there (including Docker) Docker isn't a VM; it runs with the same kernel and directly on the host processor like any application.
- 013a 8y ago
  Except the filesystem. Which is what everyone seems to forget. And that's the big thing we're talking about here in reference to Ruby and tests being slow.
  - edude03 8y ago
    The file system is native too depending on what filesystem your host is running. If the host is running a Copy on Write filesystems like ZFS or BTRFS docker can use it natively.
  - pmontra 8y ago
    Even if you mount the development directory on the host file system? This is probably necessary to be able to edit files. In the case of Ruby maybe you should mount also the language manager directory with all the gems and the interpreter.
- throwaway2048 8y ago
  Unless your host system isn't linux (like in this case)

mastazi 8y ago

I've used Linux as my main desktop OS for about 3 years, from around 2010 to around 2013 (so maybe some things changed in the meantime). The article says that you pay a "VM tax" by running in a Linux VM in Windows or by using WSL, but by using Linux directly you end up "paying" in other areas, for example in terms of less-than-perfect drivers (which leads to problems such as worse battery life, worse graphical performance etc). In an ideal world, laptop and GPU makers would put as much effort developing Linux drivers as they do when they create Windows drivers, but unfortunately this is not the case. Another point is that, even if you are running Linux on your dev machine, it is often challenging to have a local environment that is as close as possible to your production one; for example where I work we have Ubuntu 16.04, Ubuntu 18.04 and CentOS (and with various combinations of installed packages), even if I was using a Linux laptop, I would still need containers to get all the different environments right. The only case where you could use no containers at all is if you had all of your servers running the exactly the same environment. Finally there is the issue of some pieces of software not being available for Linux, 95% of the times you can find a Linux equivalent that works for you but, in my experience, there were still rare cases where I had to resort to Wine or some Win VM just to run some specific tool that was needed to do my job. PS My job, at the time I was using Linux as my main desktop OS, and during the couple of years after I switched back to Windows, was about PHP, Python and Java web apps deployed on Linux servers, so my comments above should be taken in this context. Maybe in other fields of software development there are different factors to take into account. PPS I have also tried MacOS, on the one hand it's great that Mac is UNIX, but on the other hand it's not Linux so, if you want to closely replicate your production Linux environment, you will still need VMs or containers.

baroffoos 8y ago
I picked a Dell XPS for development and it runs linux amazingly well. Only bit that doesn't work out of the box is the fingerprint scanner but there are unofficial drivers for it. With web development its so trivial to do everything you need on linux and almost all of the time its easier to set up on linux or mac than on windows.
- themodelplumber 8y ago

underwater 8y ago

If the time taken to run the test suite is so critical, then how is the "winning" result of 7m55s any better? In my experience any multi-minute operation invites a context switch or coffee break, and end up being roughly equivalent in practice.

sams99 8y ago
I completely understand the concern here. If every change required an 8 minute wait it would be real sad for us. We use a tool called `rake autospec` at Discourse which is a smart test runner. I personally like the vim integration we built. The way it works... 1. Run `bin/rake autospec` in a terminal 2. I head to the code I want to change, I change the code 3. The spec runner figures out what the right test is run and runs that. 4. While it is busy running specs at any time it can be interrupted by saving a "*.spec" file in which case the spec runner will run the spec at the cursor This has worked wonderfully well for us.
- majewsky 8y ago
  FYI, list formatting works best when you leave an empty line between each item. 1. just 2. like 3. this
ufmace 8y ago
That's exactly what I thought - a 50% faster test suite run is nice I guess, but they're all still way too slow to be running all the time. I'll run the tests in 1 file or subset of files or rspec describe block or something that's relevant to what I'm doing and finishes in <10s. Save the multi-minute full runs for CI or big changes where you have no idea what will happen.

NightlyDev 8y ago

I've been using Linux as my main OS for over 10 years now, mainly Ubuntu. The best part is the performance and a realistic bare metal environment. You can profile code and actually trust the results. You can't say the same for virtualizarion, containers or other platforms.

ryan-allen 8y ago

> Here I am freaking out about a measly 38% perf hit when I could be running stuff concurrently and probably be able to run our entire test suite in 2 minutes on my current machine on Windows in a VM. I think this is the point, given you can get an inexpensive desktop CPU with 6 cores and 12 threads, it wouldn't matter so much if you had a 20% perf tax with WSL when you have approx 12x the throughput. You could always offload full test suites to an external CI box upon check-in and work with a subset of unit tests locally, once the suite becomes prohibitive for a single machine.

evancox100 8y ago

Meanwhile, in ASIC/hardware testing land, we're lucky to get a regression suite done in under 24 hours.

amanzi 8y ago

Curious to know why you were using VMware on Windows instead of Hyper-V? I would expect Hyper-V to be significantly faster than VMware.

gorjusborg 8y ago
I gave Hyper-V way too much of my time last tried setting up a development environment on windows (around September 2018). The Hyper-V tooling just isn't good enough for the developer use case compared to Virtualbox and VMWare, and once you enable Hyper-V, it owns the virtualization environment within Windows, meaning you can't run other hypervisors on the OS. That was a deal-breaker for me, as Docker for Windows wouldn't even run out of the box.
- vbezhenar 8y ago
  Docker by default runs on Hypet-V.
  - bragh 8y ago
    Did it always used to do that, wasn't it VirtualBox only in the beginning? Also, https://github.com/MicrosoftDocs/Virtualization-Documentation/issues/945 https://github.com/MicrosoftDocs/Virtualization-Documentatio...
    - vbezhenar 8y ago
      I'm not sure about the beginning, but it uses Hyper-V for a while. I don't know about that bug, I used it with Windows 10 Pro, I guess bugs happens, probably they will fix it soon if it's widespread. I know that you need a special Docker package (Docker Toolbox) for VirtualBox and default Docker package from their website requires Hyper-V.
wvenable 8y ago
I don't see why Hyper-V would be faster but VMware is much nicer to work with (in my opinion).

lazyjones 8y ago

Why do people run their server software on their desktop systems instead of, say, a proper server? Back in the days our desktop systems were VT102s, NCD X-Terminals or underpowered laptops and developing on the server through rsh/ssh or even VNC was natural. Does Ruby somehow require locally running GUI processes?

mwcampbell 8y ago
There's certainly nothing stopping you from doing Ruby or JS dev work on a remote server. I run Windows on my laptop, so when I wanted to start contributing to Discourse on the side a few months ago, I ordered a VPS at DigitalOcean and did my work there over SSH. I used SSH tunneling to access the dev HTTP server. Edit: It helps that I use vim as my editor.
- pletnes 8y ago
  I’ve done similar things with pycharm. I can run pycharm locally but run a remote python process. .py files are automatically uploaded and I can debug over the network just fine. I know eclipse, and presumsbly others, can do the same.
hamandcheese 8y ago
At my current company we do exactly this. It’s nice because it’s someone else’s problem to make sure my dev environment works, I can recreate my dev environment with a single command, and I have a shareable URL to my Rails instance that anyone on the eng VPN can use (makes it easy to show works in progress to teammates or to the product team). It does kind of suck in some ways though. I can’t develop without internet, and debugging is worse than debugging locally. The debugging issue could probably be alleviated with some more investment in developer tooling/editor integrations.
kristianp 8y ago
Because you can, Convenience, and compatibility. Since linux became the OS of the internet, people have been able to run the same software on their own computer. It's more convenient to just run the "server software" on your own machine and you have full control over it.

robocat 8y ago

It is unfortunate that there is no comparison against an Ubuntu host with an Ubuntu guest. "I was paying a pretty high tax for deciding to stick with Windows” was likely "I was paying a pretty high tax for deciding to use a VM”. I have little love for Windows, but I would prefer to see numbers that place the blame correctly.

mwcampbell 8y ago
I'm thinking of renting a Vultr bare metal server for an hour or two to do the test you're suggesting. What's the easiest way to spin up a VM with KVM from the command line these days? Is it still libvirt's virsh?
ubercow13 8y ago
Why? The tax is that using Windows means you have to use a VM, not that VMs are less performant on Windows
- robocat 8y ago
  Well, I use an Ubuntu host on a Dell XPS (since it just works without extra configuration). But I also use an Ubuntu guest VM for development so that I can shove the guest onto a different machine easily (yeah, I know I should use continuous integration, but I haven't yet). Using a VM definitely has a big performance cost... Even though I value the benefits I accept the costs.
  - ubercow13 8y ago
    You might be able to use containers on Linux to achieve the same thing without the performance cost
    - robocat 8y ago
      Very likely. But I know that in X years time I can move a copy of my development VM to run as a guest on any OS I might happen to use. I haven't yet spent any time learning how to use containers, or what their compromises are, so for the moment I am still ignorant of that solution.

jacobsenscott 8y ago

Having used windows for several years, linux for several years, and currently macos for several years (because that's what work gave me) - linux is by far the most stable and easiest to manage OS, after you get past the initial learning curve. The main reason is you can actually fix problems. You pretty much know why every single bit of software is there for and how to configure it. You google the problem and someone has released a patch, or instructions on what configuration changes need to be made. Or you can roll back to old versions of software easily. Also you are a hero because you catch and fix production problems nobody else sees or understands because production is running linux. If you have a problem on OSX you just get 500 people telling you to reset your pram, and really you just need to wait for the next osx update. You certainly can't rollback an osx update. Nobody knows whats wrong because nobody has access to the code. Even if you did - each update is millions of lines of code. It is crazy town. Same for windows except there is not pram to reset. You just reboot until the next update I guess.

deleted 8y ago
[deleted]
titanix2 8y ago
> linux is by far the most stable and easiest to manage OS Nice April fools.
- kuzimoto 8y ago
  Anecdotally, this is true for me. My Arch machine runs for months with no issues. My XPS running 10 blue screens once every several weeks, and generally has issues after running too long. To be fair it could be due to the general buggy-ness of the TB16 Thunderbolt dock.
  - titanix2 8y ago
    Yeah of course, different people will have different experiences. Mine was being never able to make wifi work, random destruction of the GUI when I plugged an external monitor, difficulty to install and configure eastern languages IME, sound issues, etc. That was a very instructive period as undergraduate in informatics, but I got tired after a while to constantly fix my system instead of getting things done. Still using it as a server OS, where it shines.

craz8 8y ago

I added parallel tests to our app last year. My 4 core/8 thread system runs the tests pretty well with a process count of 6. The suite runs about 4 times faster, which makes a real difference No need to wait for Rails 6 to get this boost

ncmncm 8y ago

It never occurred to him to run Linux on his machine, for real work, and keep Windows in a VM for when he really needs it? Surely someone, somewhere has explained that Windows runs faster in a VM than on bare metal, because Linux is better at file systems and buffer management? Or that relying on drivers from all over makes your system less reliable than the fixed set used in the VM? Me, I cannot imagine running MSWindows on bare metal. It just feels wrong.

pixelmonkey 8y ago

If you run Linux on vanilla Intel ultrabook or Intel desktop/server hardware, it is not only blazing fast, but all the hardware works out of the box, too. I wrote about this in the context of Lenovo X1C laptop here: http://amontalenti.com/2017/09/01/lenovo-linux http://amontalenti.com/2017/09/01/lenovo-linux I also recently built a Linux desktop from ~$950 of commodity parts (including a pair of free GPUs a crypto friend donated to me after his startup died). I use it as a devserver and a GPU rig for playing with CUDA, PyTorch, and TensorFlow. Aside from the requirement of proprietary Nvidia drivers, this whole box works perfectly in Linux too, is blazing fast, and operates silently and with low power consumption. (The case itself is bulky, but it’s stationary.) I think an equivalent Mac would cost a 2-3x multiple and run less well for developer workloads. https://pcpartpicker.com/user/amontalenti/saved/r4rLJx https://pcpartpicker.com/user/amontalenti/saved/r4rLJx Say what you will about Linux, but if you choose your hardware carefully, it truly does “just work” these days. And you can’t beat having access to scriptable everything and source code everywhere. That said, I keep a Mac Mini around because there is some proprietary stuff you can’t avoid in the Apple ecosystem (e.g. XCode for iOS, Safari Debugger, Keynote/Pages, ...)

bgdnyxbjx 8y ago

This whole Twitter thread and blog post gave me real pause for ever wanting to work at Discourse.

mwcampbell 8y ago
Why, exactly?

impostir 8y ago

I have been using Linux as my daily driver for a little over 2 years and day to day, I haven't had practically any problems. Every once in a while, I will mess with something and then things break, but I know I can fix it. Knowing I can find a fix to any day to day problem has become absolutely crucial to me; I can't go back to chasing windows error codes.

pmontra 8y ago

Ruby has always been a nightmare on Windows. Every time a new junior freelance developer is onboarded in a Rails project everybody hopes he's got a Mac or Linux laptop. A significant number of them uses Windows and we always suggest them to run a Linux VM. Some of them resist the suggestion. We make clear we're not wasting time to support their environment (we're deploying on Linux anyway) and that they are on their own. After not to much time they comply or leave. After months some of them could even buy a Mac or dual boot on Linux (Windows is good at videogames.) Still, AFAIK VirtualBox has a problem with supporting multiple cores. Single core VM is heavily handicapped even for Ruby. Parallel testing was not in Rails but it's definitely a thing.

zubspace 8y ago
Maybe your company should provide the optimal hardware which everyone must use? Each employee using a similar environment is a productivity boost. And I can tell you, that if everyone would use the same system, even windows, you would have less problems. Sounds alot simpler than letting everyone use their own environment..
- pmontra 8y ago
  Not my company (I'm a freelancer) and every freelancer brings his own laptop. Furthermore all of us work from remote, often even employees do. I've seen this happen everywhere with the small/medium customers I'm working for.
- bluedino 8y ago
  We were in a similar situation (20/80 Linux/Mac devs, respectively) You can use Windows, but none of the rest of us do so you're on your own. You can have a MBP, iMac, or ThinkPad with Linux when you start. Your choice. We had two developers actually say they were wary of learning Mac/Linux and didn't take the job for that reason.

rcarmo 8y ago

I’ve been doing an increasing amount of Node development in WSL, and the thing that helped me the most where it regards disk performance was adding an exclusion path to Windows Defender so that it wouldn’t constantly scan my WSL working tree. Also, I get it that some kinds of testing “feel better” when done locally, but... why not have those run on a cloud VM?

snarfy 8y ago

In the late 90s I installed Slackware on a 386DX 40mhz with kernel 1.2.8, XFree86, elvis vi clone, etc. I've been using Linux for a long time, on and off as my main desktop. I have various Linux machines around the house, but it's not my main desktop today. There's too much fiddling around to get it working right as a desktop. Audio has always been a ghetto. Wifi drivers are still binary blobs. Fonts, hidpi, multi monitor support, wayland, systemd are all still issues today. I really don't want my development machine to match production. Production is a stripped down image for a reason. I spend far more time writing code than running it, so the VM performance hit is a moot point. To be fair, Windows 10 has plenty of issues of its own that need tweaking, like having the only way to disable cortana is through group policy edits. And then they reenable it on the next update, and give me some candy crush ads in my start menu. It's pretty infuriating, but still less so than manually fixing wifi drivers through USB boot drives.

nickjj 8y ago

WSL isn't too bad nowadays. I would go as far as saying it's very good. I've been using it for full time development for over a year now. I've run 100,000+ line Rails apps in WSL (well technically through Docker, which I connect to through WSL) and I never noticed a slowdown that was bad enough to make me think "this sucks". It's always been pretty good. I run all sorts of Rails, Flask, Phoenix and Webpack driven apps and all of them run fast enough where I don't think twice about it. Personally, I find the WSL set up somewhat close to native Linux in terms of the user experience. I'm not talking about I/O performance, but I mean how it feels to use the OS in general. For example: I spend 99% of my time in a WSL terminal using tmux + terminal Vim + ranger. So that takes care of coding and managing files. Then I use a browser and other graphical apps (image / video editors) that run directly in Windows. Dexpot sets up virtual screens with key binds that are comparable to how i3wm lets you switch and move windows to another screen. Keypirinha lets you launch apps with a fuzzy finder (like dmenu but better IMO) AutoHotkey lets you easily manage global hotkeys (just like i3 does) and more When you put all of that together, you get a really really good development experience that also doubles for gaming and running programs that don't have a good alternative on Linux (such as Camtasia on Windows). Then for the icing on the cake, since you're running Ubuntu 18.04 in WSL, you can provision WSL itself with the same exact configuration management scripts you would use to provision a production box. For me specifically I run all of the same Ansible roles against WSL that I do in production. I can set the whole thing up with 1 Ansible command. Plus my dotfiles also happen to work exactly how they do on my native Linux laptop so it's easy to keep things in sync and feeling the same. This all runs from a i5 3.2ghz / 16GB of RAM / SSD / etc. $750 desktop built 5 years ago. Even if Apple tax didn't exist I would still use this Windows / WSL set up if I weren't in a position where I could run Linux natively.

bluedino 8y ago

I love OS X, but it's just 'slow'. I have no problem admitting that. It's much better than it was in the PowerPC days, when (IMO) OS X was almost too slow to be usable. We have it good these days with faster graphics and 12-core CPU's and super fast SSD's... But you give these same advantages to Windows/Linux and OS X is just handily beaten in just about every benchmark you can think of.