13 ms·

Switching to Linux: A Windows developer’s view

57 points by edw519 17y ago

phugoid 17y ago
Smacks of poor engineering to me; switching to another platform instead of finding the cause of the problem. "I was hitting some issues with either the scheduler or the memory subsystem, as my code didn’t have any I/O in it."
- RiderOfGiraffes 17y ago
  Whether there's an underlying fault or not, it's an interesting datum that it ran 5 to 10 times faster under Linux. If nothing else, compiling and running under both platforms gives clues to places problems are happening. We target three different platforms and seeing the performance change from one to the other is really useful.
  - blub 17y ago
    I was under the impression that he used OpenMP on Linux while using low-level threading on Windows.
    - flogic 17y ago
      That could be the cause but if he couldn't use OpenMP under windows then it's still a legitimate complaint.
      - blub 17y ago
        He could have, but he couldn't get it to compile. Either way, he should have just used QtConcurrent (since he's using Qt). The performance issue looks like more of an excuse to me; I can think of at least five things to try before changing the OS would even enter my mind.
        billswift 17y ago
        Some people have small minds without much room for new ideas to enter.
- ajross 17y ago
  Developers are responsible for debugging their platform then? He states that he had a significant speedup in Vista too, which is pretty good evidence that he'd isolated the problem to something in XP. So he needs to decide on what to do, and fixing XP retroactively doesn't seem to be an option. And in any case, the thread scalability of the allocator some of MS's older C runtimes is a known performance issue. I'm not surprised by this at all.
  - blub 17y ago
    What happened to: first check your code, then your compiler and as a last resort your OS? The problem was most likely in his hand-crafted threading code.
- rbanffy 17y ago
  All facts indicate the problem _was_ the platform. The program ran much faster on Vista, even with the original manual threading. When ported to Linux with OpenMP the problem went away, suggesting it never really existed inside the program.
- joe_the_user 17y ago
  Uh, When you start to hit issues that you have good reason to believe have nothing to do with you, finding the root cause of the issues is not always the best choice.
  - rbanffy 17y ago
    It's nice to find the root cause, but, sometimes, it can't be done unless you have access to the underlying source code. Too bad Windows doesn't come with it.
- Periodic 17y ago
  He stated that one of the biggest reasons he did the switch was because Linux allowed him more freedom in terms of platform, libraries, and tools, while his windows options were limited by his company to Visual Studio 2005 on Windows XP x64. He wanted to just start working on the newer operating systems, but was not allowed to due to corporate budgets or the like.
- jacquesm 17y ago
  Finding the cause of the problem is something that Microsoft might be able to do if they choose to do so. This is closed source, remember.
- agazso 17y ago
  free() under Windows XP can be incredibly slow. I once created a big hash table with more than 10000 allocated hash buckets, and freeing it took around 3 seconds (!) on a 3GHz P4. The same code on Linux and OSX took around 3 milliseconds.
joe_the_user 17y ago
I think QT is real gem as far as application frameworks go. It's the easiest C++ framework I've ever worked with.
justindz 17y ago
This was interesting inside view. One thing that I did notice is the statement that the level of changes on the MSFT side was causing him grief. I'd be curious, as a follow-up, to see if he continues to develop on Linux. It seems like he has generalized the ease of the port into other perceived benefits on Linux (e.g. rate of change, lower need to adapt code) that might not be true in practice, or at least as true as he feels they will be based on the porting experience. Sounded a little bit like some kind of initial success euphoria bias. Yes, I just made that up. I remember it with Rails too. This isn't a knock on Rails--just using it as an example--but a lot of people had such a blast porting code (me included) that they assumed everything was peaches and cream until they realized that the big, dirty things in web apps like scaling, localization and the impact of poor planning didn't magically disappear.
- Periodic 17y ago
  I think there's a honeymoon period with just about any new technology. It generally starts once you've switched to solve a problem and the new technology solves it well. At this point any faults or frustrations are generally just because you don't know the technology well enough to do it right. It isn't until you become more familiar and try to expand what you're doing that you start to find and explore the limits of the new technology. All technology has limitations and frustrations. The question is which works best for each person. I think this might work well for him, as it sounds like he is doing some things that do port very easily.
modeless 17y ago
Choosing Linux for stable APIs would be rather silly. X11 is evolving rapidly nowadays and audio APIs are in a state of constant flux, where the different APIs often interfere with each other and there's no obvious best solution. Linux changes ABIs all the time and is notorious for C library incompatibilities. This recent mess with filesystem write semantics (http://lwn.net/Articles/322823/ http://lwn.net/Articles/322823/ ) has proven you can't even rely on POSIX. Meanwhile, Win32 has been far more stable over the years and Microsoft has put a ton of effort into backwards compatibility. Don't get me wrong; I like Linux but API stability is not the reason to choose it over Windows.
- Periodic 17y ago
  There are benefits and drawbacks to maintaining an API vs updating it. Switching every few months will alienate your developers, but making sure everything is 20 years backwards compatible will lead to strange errors (particularly because programmers will expect their idiosyncratic workarounds to continue to work) I think the more important question is how the groups responsible will handle the transition. A well planned and executed transition will give many developers a chance to switch gradually and by the time the old API isn't supported the older versions of the program will likely only be run on the older versions of the API anyway.
- rbanffy 17y ago
  "This recent mess with filesystem write semantics (http://lwn.net/Articles/322823/ http://lwn.net/Articles/322823/ ) has proven you can't even rely on POSIX." I have to disagree. The "problem" resides in the way ext4 writes data to the disk. If you need crash-resistance (the data-loss occurs on computer crashes) ext4 is not for you. And keep in mind if you really need it, ext3 isn't for you either. JFS or XFS, perhaps, and ZFS, but I never intentionally crashed a machine with unwritten data since, perhaps, my Apple II days. This has nothing to do with POSIX.
  - jacquesm 17y ago
    I have several hundred million files stored on XFS, not sure if that's a lot or a little by HN standards, but maybe it counts for something. We will never ever have another XFS setup with a filesystem where deletions are to be expected. XFS works fine as long as you are just adding files to the filesystem, deletions are pathologically slow. We've tried just about every trick in the book and have finally made the decision to switch back to ext3, which given the number of files will take a while. So, that's nothing against XFS from a reliability point of view, but definitely from a performance point of view.
GrandMasterBirt 17y ago
I always been a windows developer. 3 mo ago my co-worker suggested I give linux a shot. Fuck it why not. I realized the following: 1) Linux is more stable. (NEVER have to reboot it, unless I chose to conserve energy at night). 2) Linux boots faster. By the time linux boots, auto launches firefox and eclipse, windows is only reaching the desktop at which point its still 1-2 minutes before it becomes usable. 3) I still use the same tools in linux and windows (I still need a VM to run Toad). (Eclipse, grep, firefox, a linux command prompt (god cygwin is not as good as the real thing). Which is great since toad makes windows 95 appear stable. Putting toad in a vm is the best way to make it work well :). Also I get to test IE6,7,8 without any problems because I already have vms set up. 4) Multiple desktops = easier at times. 5) Windows made me need tools for mounting like sftp -> drive. Linux comes with sftpfs (ok fine u need to apt-get it). Its all free too so cost savings. Switching to linux only raised one question: Why didn't I do it earlier. The only problem is KDE fonts suck! Gnome ones are good out-of-the-box. Also KDE apps (like plasma) crashes like a wasted, doped-up, narcoleptic. Sooooo ubuntu, not kubuntu wins for me.
- halo 17y ago
  If you have to reboot Windows for anything other than updates you should be looking into problems with your hardware and drivers. The "Windows is unstable" meme should really be dead and buried by now. I can believe Ubuntu is faster than Windows at booting, especially considering they focused on it in their last release, but for me the much bigger issue was flaky Sleep/Suspend-to-RAM and Hibernate/Suspend-to-Disk support. I suspend my laptop several times a day but only reboot on updates.
  - jacquesm 17y ago
    > If you have to reboot Windows for anything other than updates you should be looking into problems with your hardware and drivers. The "Windows is unstable" meme should really be dead and buried by now. It it should be 'dead and buried' by now then why does it persist ? It seems that there are enough people that share these experiences (not necessarily your own) to stop that from happening. My own take is that it is getting better, but it isn't there yet. One thing that I personally don't understand at all is why the industry did not long ago standardize on ECC RAM, the cost would have been marginal (due to the increase in volume) and the potential benefits in perceived stability would be huge. Servers have it pretty much as a standard, any bit flipped in critical RAM can cause an OS crash (both for Linux and for Windows, RAM failures are OS agnostic).
steverb 17y ago
Looking at what he was doing (Python + C++) it seems that switching was a no brainer. Hell, I'm a MS fanboy (seriously love .NET) and I use Linux for some projects (Java/Tomcat and investigating Mono).
demallien 17y ago
Maybe I'm missing something, but who develops on Windows for applications that aren't using the Windows APIs anyway? I mean, the whole point of Windows is it's APIs. If you aren't going to use them, then sure, go and move to a unix system - indeed you should have done so years ago. On the other hand, if your product is designed for Windows, then you can't really switch, can you? (that's a real question by the way, not rhetorical).
- blub 17y ago
  The whole point of Windows is that it's an operating system. Depending on your market/skills you either develop for it or not. I prefer to develop for it because when it comes to C++ programming, it's the best out there. This story made porting look easy, but rest assured it is hard, and he won't be so thrilled when the problems start cropping up.
- cakeface 17y ago
  I develop java applications on Windows. Within my company there are at least 3 other software engineers that are doing web development targeted towards deployments that are not on windows servers that still use windows as development boxes. We also do this by choice. If you are familiar with Windows XP then it is a fairly stable, simple, and reliable OS.
sandGorgon 17y ago
http://braid-game.com/news/?p=364 http://braid-game.com/news/?p=364 Jonathan Blow, the developer of Braid, has had a thread on his blog (made from last year, when he gave up trying to port Braid on linux, till July of this year) about the (sorry?) state of developer tools on Linux. An interesting read - given that his primary editor was emacs and he has been using *nixes for 20 years. Long story short - sorry state of sound on Linux, sorry state of debugging tools on linux, OpenAL+SDL dont cut it.
Keyframe 17y ago
I still use windows mostly because of my primary tools (Maya, Photoshop and Fusion - occasional game), however I have a VM with linux in it and just run a fullscreen putty on a second monitor. For django dev, for example, I have a vertically split screen where bottom is a :resize of 10 lines and running manage.py runserver_plus and top one I run bash or cycle through several (whatever I need). Similar stuff happens when I dev in D or C++. I edit files via tramp on that VM dev server. Neat combination is that I move VM from workstation to laptop and vice versa, basically having a portable dev server all the time with me. It would be nice though if I could find a binary diff copy for files, since copying VM over network can take some time (luckily I don't do that all the time).
scorpion032 17y ago
It's not for nothing Google took 8 months to port Chrome to Linux. Porting can't be this simple.
- pyre 17y ago
  Didn't it also take that long to port the OS X version as well? (I seem to remember the 'developer' builds of Chrome for Linux and Chrome for Mac coming out at the same time) Keep in mind that he seems to be mostly using Python and doing mostly number crunching (which would seem to be rather OS independent) from what I could glean.
- ZeroGravitas 17y ago
  Isn't the whole point of Chrome to take advantage of OS level protection mechanisms so that stuff coming from the web doesn't get the chance to mess with your machine. I believe they're doing completely different things in this regard on each OS, basically building sandboxes from scratch.

57 points by edw519 17y ago

Smacks of poor engineering to me; switching to another platform instead of finding the cause of the problem. "I was hitting some issues with either the scheduler or the memory subsystem, as my code didn’t have any I/O in it."

RiderOfGiraffes 17y ago
Whether there's an underlying fault or not, it's an interesting datum that it ran 5 to 10 times faster under Linux. If nothing else, compiling and running under both platforms gives clues to places problems are happening. We target three different platforms and seeing the performance change from one to the other is really useful.
- blub 17y ago
  I was under the impression that he used OpenMP on Linux while using low-level threading on Windows.
  - flogic 17y ago
    That could be the cause but if he couldn't use OpenMP under windows then it's still a legitimate complaint.
    - blub 17y ago
      He could have, but he couldn't get it to compile. Either way, he should have just used QtConcurrent (since he's using Qt). The performance issue looks like more of an excuse to me; I can think of at least five things to try before changing the OS would even enter my mind.
      - billswift 17y ago
        Some people have small minds without much room for new ideas to enter.
ajross 17y ago
Developers are responsible for debugging their platform then? He states that he had a significant speedup in Vista too, which is pretty good evidence that he'd isolated the problem to something in XP. So he needs to decide on what to do, and fixing XP retroactively doesn't seem to be an option. And in any case, the thread scalability of the allocator some of MS's older C runtimes is a known performance issue. I'm not surprised by this at all.
- blub 17y ago
  What happened to: first check your code, then your compiler and as a last resort your OS? The problem was most likely in his hand-crafted threading code.
rbanffy 17y ago
All facts indicate the problem _was_ the platform. The program ran much faster on Vista, even with the original manual threading. When ported to Linux with OpenMP the problem went away, suggesting it never really existed inside the program.
joe_the_user 17y ago
Uh, When you start to hit issues that you have good reason to believe have nothing to do with you, finding the root cause of the issues is not always the best choice.
- rbanffy 17y ago
  It's nice to find the root cause, but, sometimes, it can't be done unless you have access to the underlying source code. Too bad Windows doesn't come with it.
Periodic 17y ago
He stated that one of the biggest reasons he did the switch was because Linux allowed him more freedom in terms of platform, libraries, and tools, while his windows options were limited by his company to Visual Studio 2005 on Windows XP x64. He wanted to just start working on the newer operating systems, but was not allowed to due to corporate budgets or the like.
jacquesm 17y ago
Finding the cause of the problem is something that Microsoft might be able to do if they choose to do so. This is closed source, remember.
agazso 17y ago
free() under Windows XP can be incredibly slow. I once created a big hash table with more than 10000 allocated hash buckets, and freeing it took around 3 seconds (!) on a 3GHz P4. The same code on Linux and OSX took around 3 milliseconds.

joe_the_user 17y ago

I think QT is real gem as far as application frameworks go. It's the easiest C++ framework I've ever worked with.

justindz 17y ago

This was interesting inside view. One thing that I did notice is the statement that the level of changes on the MSFT side was causing him grief. I'd be curious, as a follow-up, to see if he continues to develop on Linux. It seems like he has generalized the ease of the port into other perceived benefits on Linux (e.g. rate of change, lower need to adapt code) that might not be true in practice, or at least as true as he feels they will be based on the porting experience. Sounded a little bit like some kind of initial success euphoria bias. Yes, I just made that up. I remember it with Rails too. This isn't a knock on Rails--just using it as an example--but a lot of people had such a blast porting code (me included) that they assumed everything was peaches and cream until they realized that the big, dirty things in web apps like scaling, localization and the impact of poor planning didn't magically disappear.

Periodic 17y ago
I think there's a honeymoon period with just about any new technology. It generally starts once you've switched to solve a problem and the new technology solves it well. At this point any faults or frustrations are generally just because you don't know the technology well enough to do it right. It isn't until you become more familiar and try to expand what you're doing that you start to find and explore the limits of the new technology. All technology has limitations and frustrations. The question is which works best for each person. I think this might work well for him, as it sounds like he is doing some things that do port very easily.

modeless 17y ago

Choosing Linux for stable APIs would be rather silly. X11 is evolving rapidly nowadays and audio APIs are in a state of constant flux, where the different APIs often interfere with each other and there's no obvious best solution. Linux changes ABIs all the time and is notorious for C library incompatibilities. This recent mess with filesystem write semantics (http://lwn.net/Articles/322823/ http://lwn.net/Articles/322823/ ) has proven you can't even rely on POSIX. Meanwhile, Win32 has been far more stable over the years and Microsoft has put a ton of effort into backwards compatibility. Don't get me wrong; I like Linux but API stability is not the reason to choose it over Windows.

Periodic 17y ago
There are benefits and drawbacks to maintaining an API vs updating it. Switching every few months will alienate your developers, but making sure everything is 20 years backwards compatible will lead to strange errors (particularly because programmers will expect their idiosyncratic workarounds to continue to work) I think the more important question is how the groups responsible will handle the transition. A well planned and executed transition will give many developers a chance to switch gradually and by the time the old API isn't supported the older versions of the program will likely only be run on the older versions of the API anyway.
rbanffy 17y ago
"This recent mess with filesystem write semantics (http://lwn.net/Articles/322823/ http://lwn.net/Articles/322823/ ) has proven you can't even rely on POSIX." I have to disagree. The "problem" resides in the way ext4 writes data to the disk. If you need crash-resistance (the data-loss occurs on computer crashes) ext4 is not for you. And keep in mind if you really need it, ext3 isn't for you either. JFS or XFS, perhaps, and ZFS, but I never intentionally crashed a machine with unwritten data since, perhaps, my Apple II days. This has nothing to do with POSIX.
- jacquesm 17y ago
  I have several hundred million files stored on XFS, not sure if that's a lot or a little by HN standards, but maybe it counts for something. We will never ever have another XFS setup with a filesystem where deletions are to be expected. XFS works fine as long as you are just adding files to the filesystem, deletions are pathologically slow. We've tried just about every trick in the book and have finally made the decision to switch back to ext3, which given the number of files will take a while. So, that's nothing against XFS from a reliability point of view, but definitely from a performance point of view.

GrandMasterBirt 17y ago

I always been a windows developer. 3 mo ago my co-worker suggested I give linux a shot. Fuck it why not. I realized the following: 1) Linux is more stable. (NEVER have to reboot it, unless I chose to conserve energy at night). 2) Linux boots faster. By the time linux boots, auto launches firefox and eclipse, windows is only reaching the desktop at which point its still 1-2 minutes before it becomes usable. 3) I still use the same tools in linux and windows (I still need a VM to run Toad). (Eclipse, grep, firefox, a linux command prompt (god cygwin is not as good as the real thing). Which is great since toad makes windows 95 appear stable. Putting toad in a vm is the best way to make it work well :). Also I get to test IE6,7,8 without any problems because I already have vms set up. 4) Multiple desktops = easier at times. 5) Windows made me need tools for mounting like sftp -> drive. Linux comes with sftpfs (ok fine u need to apt-get it). Its all free too so cost savings. Switching to linux only raised one question: Why didn't I do it earlier. The only problem is KDE fonts suck! Gnome ones are good out-of-the-box. Also KDE apps (like plasma) crashes like a wasted, doped-up, narcoleptic. Sooooo ubuntu, not kubuntu wins for me.

halo 17y ago
If you have to reboot Windows for anything other than updates you should be looking into problems with your hardware and drivers. The "Windows is unstable" meme should really be dead and buried by now. I can believe Ubuntu is faster than Windows at booting, especially considering they focused on it in their last release, but for me the much bigger issue was flaky Sleep/Suspend-to-RAM and Hibernate/Suspend-to-Disk support. I suspend my laptop several times a day but only reboot on updates.
- jacquesm 17y ago
  > If you have to reboot Windows for anything other than updates you should be looking into problems with your hardware and drivers. The "Windows is unstable" meme should really be dead and buried by now. It it should be 'dead and buried' by now then why does it persist ? It seems that there are enough people that share these experiences (not necessarily your own) to stop that from happening. My own take is that it is getting better, but it isn't there yet. One thing that I personally don't understand at all is why the industry did not long ago standardize on ECC RAM, the cost would have been marginal (due to the increase in volume) and the potential benefits in perceived stability would be huge. Servers have it pretty much as a standard, any bit flipped in critical RAM can cause an OS crash (both for Linux and for Windows, RAM failures are OS agnostic).

steverb 17y ago

Looking at what he was doing (Python + C++) it seems that switching was a no brainer. Hell, I'm a MS fanboy (seriously love .NET) and I use Linux for some projects (Java/Tomcat and investigating Mono).

demallien 17y ago

Maybe I'm missing something, but who develops on Windows for applications that aren't using the Windows APIs anyway? I mean, the whole point of Windows is it's APIs. If you aren't going to use them, then sure, go and move to a unix system - indeed you should have done so years ago. On the other hand, if your product is designed for Windows, then you can't really switch, can you? (that's a real question by the way, not rhetorical).

blub 17y ago
The whole point of Windows is that it's an operating system. Depending on your market/skills you either develop for it or not. I prefer to develop for it because when it comes to C++ programming, it's the best out there. This story made porting look easy, but rest assured it is hard, and he won't be so thrilled when the problems start cropping up.
cakeface 17y ago
I develop java applications on Windows. Within my company there are at least 3 other software engineers that are doing web development targeted towards deployments that are not on windows servers that still use windows as development boxes. We also do this by choice. If you are familiar with Windows XP then it is a fairly stable, simple, and reliable OS.

sandGorgon 17y ago

http://braid-game.com/news/?p=364 http://braid-game.com/news/?p=364 Jonathan Blow, the developer of Braid, has had a thread on his blog (made from last year, when he gave up trying to port Braid on linux, till July of this year) about the (sorry?) state of developer tools on Linux. An interesting read - given that his primary editor was emacs and he has been using *nixes for 20 years. Long story short - sorry state of sound on Linux, sorry state of debugging tools on linux, OpenAL+SDL dont cut it.

Keyframe 17y ago

I still use windows mostly because of my primary tools (Maya, Photoshop and Fusion - occasional game), however I have a VM with linux in it and just run a fullscreen putty on a second monitor. For django dev, for example, I have a vertically split screen where bottom is a :resize of 10 lines and running manage.py runserver_plus and top one I run bash or cycle through several (whatever I need). Similar stuff happens when I dev in D or C++. I edit files via tramp on that VM dev server. Neat combination is that I move VM from workstation to laptop and vice versa, basically having a portable dev server all the time with me. It would be nice though if I could find a binary diff copy for files, since copying VM over network can take some time (luckily I don't do that all the time).

scorpion032 17y ago

It's not for nothing Google took 8 months to port Chrome to Linux. Porting can't be this simple.

pyre 17y ago
Didn't it also take that long to port the OS X version as well? (I seem to remember the 'developer' builds of Chrome for Linux and Chrome for Mac coming out at the same time) Keep in mind that he seems to be mostly using Python and doing mostly number crunching (which would seem to be rather OS independent) from what I could glean.
ZeroGravitas 17y ago
Isn't the whole point of Chrome to take advantage of OS level protection mechanisms so that stuff coming from the web doesn't get the chance to mess with your machine. I believe they're doing completely different things in this regard on each OS, basically building sandboxes from scratch.