20 ms·

Windows NT Kernel Contributor Explains Why Performance is Behind Other OS

656 points by mrb 14y ago

chamanbuga 14y ago
What's wrong with 9-5? Can't you be passionate about what you work on, excel in you career, yet stick to dedicating ~50% of your time awake to your job?
- freditup 14y ago
  And 8hrs is probably too much time to really productively be working on a job at that.
- kika 14y ago
  Because you spend 9-noon and 1pm-3pm on meetings and you still need to code.
  - kperusko 14y ago
    That sounds like an organizational problem to me if you spend 75% of your time at meetings everyday instead of coding (if coding is actually your job).
- pjmlp 14y ago
  It is an American thing. In the rest of the world we can happily do 9-5 without feeling bad for it.
  - kperusko 14y ago
    Unfortunately that kind of thinking is no longer only an American thing. I see it more and more every day. Employers are pushing fresh college grads to work as many hours as possible for a low wage just so they can have a bigger profit. If they burn out, there are others to replace them.
  - chamanbuga 14y ago
    It's amusing watching Americans boast about working past the norm 9-5 hours and calling the rest of the world lazy. My response, "What are you proud about? After working twice as hard, you have the same quality of life as friends in Canada, and statistically speaking you'll drop dead sooner because of stress." That always wipes the smug grin of their faces.
    - illuminate 14y ago
      "That always wipes the smug grin of their faces." While I agree with you, nothing could wipe the smug grin off these persons' faces. They are beyond reality, and I'd be much happier with America if something so simple could give the "socialists are universally lazy" rabble whenever labor rights are mentioned in any sense.
      - pjmlp 14y ago
        Yes, here in the socialist Germany I can enjoy my free time after 4:30 pm and if my boss wants me to stay longer I get paid overtime.
jsolson 14y ago
> Another reason for the quality gap is that that we've been having trouble keeping talented people. Google and other large Seattle-area companies keep poaching our best, most experienced developers, and we hire youths straight from college to replace them. I will say all of the ex-Microsoft folks I've encountered at Google Seattle have been fantastic. On a related note, it's stupidly easy to get code accepted by another team at Google. Also we're hiring.
- dvt 14y ago
  >Also we're hiring. Insult to injury? ;)
- tracker1 14y ago
  Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries.
  - hythloday 14y ago
    I have zero formal education past A levels, but I've had two in-person interviews at Google and had an offer, and I have their charming recruiters chasing me to make sure I wouldn't be happier working for Google every 6 months or so. :) It does take a ferociously long time to get through the initial screening if you just send them a CV, and I'm sure a lot of people fall through the cracks - my suggestion would be to find a Googler or Xoogler and ask them to recommend you, I understand you get a lot more traction that way. Chin up.
  - ajross 14y ago
    To be fair, and in context: neither does Microsoft.
    - tracker1 14y ago
      I've been approached a number of times by MS... once fairly aggressively. Never a word or response from Google though.
dietrichepp 14y ago
I think there are two complementary parts that Linux gets right here. One is as cited in the article, that you get personal glory for improvements you make to the kernel even if they are fairly small. The other part is that there is for Linux someone who will say "no" to a patch. Linus will certainly do it, and other trusted devs will do it too. Linus is perhaps even famous for telling people when they are wrong. I've seen a number of open-source projects where you get all the personal glory for your additions but there is nobody who takes the responsibility to tell people "no". These projects, almost universally, turn into bloated messes over time. Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.
- philsnow 14y ago
  > Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games. I just listened to an episode of Roguelike Radio from the mid-30s (maybe the one with Red Rogue or Thomas Biskup) and Andrew / Keith (IIRC) go on and on about how all the "major" roguelikes are just accretions of features. This rings very true to me; I grew up playing Nethack [0], and Slash (and then Slash'Em) just got silly with all the things they were adding. Light sabers ? In my roguelike ? Come on, fhgwgads. Nethack at least has a mysterious Dev Team who ostensibly are the guardians of quality... but they _did_ bring Sokoban back from Slash'EM into vanilla. [0] since Nethack was my first roguelike, I didn't realize until lately just how crappy it is in many ways, most notably that it's very difficult to win before you've figured out all the sources of instadeath (whether you've figured them out by spoilers (written or source-diving) or through (really frickin hard-won) experience).
  - p0nce 14y ago
    Try brogue, it's a breath of fresh air.
    - philsnow 14y ago
      I <3 the crap out of brogue; I'm currently playing through some roguelikes that I've never looked at before [0], and I keep coming back to brogue. In particular, brogue's food clock is excellent in that if I forget about it too long and go back to my natural "explore everywhere, kill everything" tendency, the food clock _will_ kill me. [0] something that jumped out at me is just how much modern roguelike development is windows-first (or even windows-only), and how much of it is distributed primarily as runnable binaries vs source. I had to bend way over backwards to get PrincessRL running: I ended up installing a 32-bit ubuntu userspace virtualbox instance so that mono could run the prepackaged binary.
quotemstr 14y ago
I actually enjoy working at Microsoft --- I'm in Phone, working on telemetry and various other things --- and I've met a ton of very smart people people. I've also made the cross-team (and cross-org) contributions that the OP says are nearly impossible (just today, even). While the OP makes a few good points (some teams are kinda reluctant to take patches), I think he's grossly exaggerating the problems, and the level of vitriol really isn't called for either. He's also slightly off-base on some of the technical criticisms: there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent. Besides, change _does_ happen: Arun Kishan (who is smarter than I'll ever be) broke the dispatcher lock a while ago (see http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewell-to-the-Windows-Kernel-Dispatcher-Lock/ http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewe...) when nobody thought it could be done.
- quotemstr 14y ago
  By the way: there's actually quite a lot of information available on Windows Internals. In fact, there's a whole book, Windows Internals (http://technet.microsoft.com/en-us/sysinternals/bb963901.aspx http://technet.microsoft.com/en-us/sysinternals/bb963901.asp...) on exactly how the NT kernel and related system components work. The book is highly recommended reading and demystifies a lot of the odder parts of our API surface. We use it internally all the time, and reading it, you'll repeatedly say to yourself, "Ah, so that's why Foo does Bar and not Qux! Yeah, that's a good reason. I didn't think of it that way."
- angersock 14y ago
  "there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent." One of the issues the author seemed to be alluding to is that the loss of experienced developers makes it really hard to keep track of "line 123 of foo_dispatch.c does this because of vital business reason xyz or compatibility reason uvw" and "line 123 of foo_dispatch.c was written while I was hungover and trying to make a deadline--it looks clever, but feel free to junk it.". This issue is compounded when you are hiring safe, and you have a culture where making gross fuckups while trying to make progress is discouraged. It is neither good nor bad--after all, I too enjoy using stable software--but there is a price to be paid if devs don't feel comfortable making breaking changes.
pg 14y ago
"His post has been deleted! Why the censorship?" We didn't delete it. It was deleted by whoever posted it.
- mrb 14y ago
  Thanks. I added the explanation.
  - skeletonjelly 14y ago
    To be fair to pg, I feel like this was the obvious conclusion (that an anonymous user posting via tor would redact)
- jamesaguilar 14y ago
  No delete button on the internet v125786123. Sidebar: do deleted comments appear with showdead on, or just ones that are deleted on HN side.
  - deleted 14y ago
    [deleted]
  - hollerith 14y ago
    >do [deleted-by-author] comments appear with showdead on[?] No, they do not. Specifically, I have showdead on, and when trying to view the comment in question, I see only "[deleted]".
softbuilder 14y ago
>We fill headcount with nine-to-five-with-kids types, then >We occasionally get good people anyway Uh... nine-to-five-with-kids type here. Thanks for the stereotyping... from your safe corporate nest. Otherwise an insightful post.
- temphn 14y ago
  If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates. Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely. I understand the reason this fiction is maintained: people with kids need the job even more than people without, and have an interest in denouncing people who claim kids make you less productive. There's also the second order effect in which "with kids" is correlated with "older". The net of it though is that devs with kids tend to assign work a lower priority, to take fewer risks, to need more money, and to be older and hence less familiar with new technologies (and too busy to learn in their free time). Society doesn't have a good answer for this situation yet. In times past, technology didn't move so fast that experience was mostly obsolete (and hence useless) in a decade's time. A 40 year old farmer with kids in 1713 would probably have much to teach a young whippersnapper. The same isn't true for a 40 year old programmer with kids in 2013.
  - madewulf 14y ago
    If your experience of programming becomes worthless after 10 years, then you probably did not focus on the good skills. I don't see how learning how low level hardware works, some time tried languages (C for example), data structures, networking details, debugging protocols of all kinds, structuring programs, working in teams and so on and so forth... could become worthless. For the idea that parents and older programmers have different priorities, that may be true, but you downplay experience way too much. That said, trying to find 150% dedicated younglings, that don't spend a lot of time on side projects and just enjoying life is not that easy either
tzs 14y ago
It's possible he deleted his post because he realized that he included something in it that could identify him, which could be bad for his career. It might have been better to rewrite his post into your own words, and take out some of the unnecessary detail, rather than literally repost.
- logn 14y ago
  I'd agree. I know the OP is trying to spread knowledge and this is a great read, but I think this was slightly rude, not to mention questionable fair use.
  - dangrossman 14y ago
    There's no "fair use" argument at all. Copying an entire work, then distributing it with no transformative change or criticism, is inexcusable infringement should the author want to enforce his rights.
    - robryan 14y ago
      Although in this case I think it would be hard for this author to enforce his/her rights and still remain completely anonymous?
      - iso8859-1 14y ago
        which just makes it even more wrong...
outside1234 14y ago
Windows is dying just like DOS. Azure is the new Windows.
- illuminate 14y ago
  I don't know what this is even supposed to mean.
kevingadd 14y ago
It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do. From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory - sad to hear that's actually partly true. I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code. It really is too bad, though. There are parts of the NT kernel (and even the Win32 API) that I consider a joy to use - I love IOCP, despite its warts, and APIs like MsgWaitForMultipleObjects are great tools for building higher-level primitives. Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering, despite the fact that parts of it date back to before Windows 3.1. Some really smart people did some really good API design over time over at Microsoft...
- quotemstr 14y ago
  Actually, I think one NT's largest advantages over POSIX systems is process management: yes, the venerable CreateProcess API. See, in Windows, processes are first class kernel objects. You have handles (read: file descriptors) that refer to them. Processes have POSIX-style PIDs too, but you don't use a PID to manipulate a process the way you would with kill(2): you use a PID to open a handle to a process, then you manipulate the process using the handle. This approach, at a stroke, solves all the wait, wait3, wait4, SIGCHLD, etc. problems that plague Unixish systems to this day. (Oh, and while you have a handle to a process open, its process ID won't be re-used.) It's as if we live in a better, alternate universe where fork(2) returns a file descriptor. You can wait on process handles (the handle becomes signaled and the wait completes when the process exits). You can perform this waiting using the same functions you use to wait on anything else, and you can use WaitForMultipleObjects as a kind of super-select to wait on anything. If you want to wait on a socket, a process, and a global mutex and wake up when any of these things becomes available, you can do that. The Unix APIs for doing the same thing are a mess. Don't even get me started on SysV IPC. Another thing I really like about NT is job objects (http://msdn.microsoft.com/en-us/library/windows/desktop/ms684161%28v=vs.85%29.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/ms68...). They're a bit like cgroups, but a bit simpler (IMHO) to set up and use. You can apply memory use, scheduling, UI, and other restrictions to processes in job objects. Most conveniently of all, you can arrange for the OS to kill everything in a job object if the last handle to that job dies --- the closest Linux has is PR_SET_PDEATHSIG, which needs to be set up individually for each child and which doesn't work for setuid children. (Oh, and you can arrange for job objects to send notifications to IO completion ports.) Yes, Windows gets a lot wrong, but it gets a lot right.
termie 14y ago
UTF-16 everywhere doesn't help either. M\0o\0r\0e\0 \0d\0a\0t\0a.\0
- quotemstr 14y ago
  You do realize that NT is from 1989 and that UTF-8 is from 1992, right?
  - termie 14y ago
    Yeah. Which makes it even more irritating.. MS is still off in the double-wide stix 11 years later and still promoting UTF-16LE in userland. From http://msdn.microsoft.com/en-us/library/dd374081%28VS.85%29.aspx http://msdn.microsoft.com/en-us/library/dd374081%28VS.85%29.... "Unicode-enabled functions are described in Conventions for Function Prototypes. These functions use UTF-16 (wide character) encoding, which is the most common encoding of Unicode and the one used for native Unicode encoding on Windows operating systems. Each code value is 16 bits wide .. New Windows applications should use UTF-16 as their internal data representation."
    - philsnow 14y ago
      "Most common" encoding is best encoding! ugh
    - jeltz 14y ago
      I seriously doubt that UTF-16 still is the most common encoding. The web is mostly UTF-8 and so are most smartphones.
      - sratner 14y ago
        JavaScript strings are UCS-2 or UTF-16.
  - clhodapp 14y ago
    Wikipedia tells me that Windows NT was started in 1988 and shipped in 1993, while Linux's first release was in 1991, giving them approximately the same age. Furthermore, NT shipped with UCS-2 support, not UTF-16, as UTF-16 did not exist until 1996. UTF-16 was a migration that they completed in Windows 2000. UTF-8 was first presented in 1993 and is, therefore, older than UTF-16. With all of these facts being true (I hope), today things are such that the Windows world mostly (but not completely -- they sometimes assume two-byte characters) correctly implements UTF-16, while the Linux world correctly implements UTF-8.
kyllo 14y ago
Look at recent Microsoft releases: we don't fix old features, but accrete new ones. New features help much more at review time than improvements to old ones. (That's literally the explanation for PowerShell. Many of us wanted to improve cmd.exe, but couldn't.) Ahh, I was wondering about that. So, I guess I'll just keep using cygwin. On another note, I recently asked a friend who works at Microsoft how work is going. His reply: "Well, it's calibration time, so lots of sucking up to the boss." Must be hard to get much actual work done when you're worried about that all the time.
- aaronbrethorst 14y ago
  Yep, sounds about right. Building new stuff is rewarded far more than fixing broken shit. Do you want to make it right or do you want to get promoted?
- yuhong 14y ago
  michaelochurch has talked about problems with this, performance reviews and closed allocation before.
  - skrebbel 14y ago
    well, he doesn't talk about anything else :) I do wonder, however, whether 'boo! MS's rating system is suboptimal, to say the least!' is a very useful comment here (which might by why we haven't seen Michael write here yet)
  - kyllo 14y ago
    Yeah, and that was about Google, which has a much better reputation than Microsoft in this area. I wonder how he would like working at MS. I doubt he'd last a week there!
- invisible 14y ago
  For those unfamiliar with what "calibration" is in Microsoft: http://minimsft.blogspot.com/2011/04/microsofts-new-review-and-compensation.html http://minimsft.blogspot.com/2011/04/microsofts-new-review-a...
- 14y ago
chetanahuja 14y ago
tl;dr : corporatism and careerism. It's the death of creativity and productivity. No large organization is immune to it. Never will be. (and yes, that includes Google... it's just much smaller and newer than Microsoft is right now)
- yuhong 14y ago
  michaelochurch has talked about this before, including the fundamental flaws behind it.
  - randomfool 14y ago
    He's also a vindictive windbag who is extremely disgruntled about his shortcomings at Google.
    - deleted 14y ago
      [deleted]
    - yekko 14y ago
      And he's totally correct.
deleted 14y ago
[deleted]
deleted 14y ago
[deleted]
millerc 14y ago
Sad post IMO, which will unfortunately and unduly tarnish Microsoft's reputation. I would suppose it was written by a developer who has just become senior enough to see some of what's going on, but has not yet realized it's everywhere like that. One day, he'll understand that those challenges too need to be managed. Growing pains, all around.
gizmo686 14y ago
Slightly off topic, but is there anyway we can actually confirm that "the SHA-1 hash of revision #102 of [redacted] is [redacted]." EDIT: redacted information. Still, when that information we present, how would anyone be able to confirm it?
- JoachimSchipper 14y ago
  The article redacted that data on the request of the original poster; please remove it from your post.
- joe5150 14y ago
  I had the same question but was too embarrassed to ask. Thanks for doing it for me. :)
- RyJones 14y ago
  I assume some other dev with source access at Microsoft could verify that fact, broadcast that the fact is true, and thereby stamp the OP as actually who he says he is.
- marshray 14y ago
  From what I've heard (unofficially) the source is typically watermarked to identify the sources of any leaks. He may have given himself away with the SHA-1.
  - gizmo686 14y ago
    I doubt it. Not only would that involve generating a different watermark for each person with access to the code, but in doing so you also raise complications for source control. Additionally, it is almost impossible for developers not to notice if supposedly identical files are not identical, not to mention the fact that implementing the type of system without the conscious corporation of the developers seems impossible.
wfunction 14y ago
At some point Microsoft might want to consider expanding its kernel team into Silicon Valley.
dchest 14y ago
Ah, so the symlink plan is true: https://gist.github.com/800407/3588729d57abb7731985c99c492a2a08459d7b2f https://gist.github.com/800407/3588729d57abb7731985c99c492a2...
Mo1oKo 14y ago
"The rot has already set in" Great words for concluding a video game intro
pjmlp 14y ago
The typical scenario you will find in any big corporation developing software. I started to understand better how Microsoft works, after getting into the Fortune 500 enterprise world. Many of the bad things geeks associate with Microsoft are actually present in any development unit of big clunky Fortune 500 companies.
js4all 14y ago
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture. https://news.ycombinator.com/item?id=2841934 https://news.ycombinator.com/item?id=2841934
- Zigurd 14y ago
  The NT-based kernels were not as big a problem for Microsoft as CE-based OSs were. Nokia transitioned away from a Linux-based smartphone OS to Windows Phone 7, based on Windows Embedded CE 6. It is clear that Windows Phone 7 was uncompetitive with other smartphone OSs. That put Nokia in limbo for a year when they could least afford it. Now the problem is not so much the kernel, but all the cruft a Windows-based OS drags along with it. The most visible symptom is the amount of storage used up by stuff that adds little or no readily apparent value to Windows RT. Windows OSs are heavy compared to Android and other Linux-based OSs. That means inexpensive tablets and phones won't run Windows, and Windows is shut out of mobile device markets in India, Africa, and other developing areas. These markets are how Yulong (who??!!) became a top 10 mobile OEM.
- United857 14y ago
  People are praising BSD like it was all hot and new. OS X is built on a BSD which has it's roots in 60's and 70's OS design, just like the VMS roots of WinNT. OS X didn't change the world by bringing some great new underlying architecture to the table. In fact, their kernel and filesystem are arguably getting long in the tooth. The value that OS X brought to the table was the fantastic Carbon and Cocoa development platforms. And they have continued to execute and iterate on these platforms, providing the "Core" series of APIs (CoreGraphics, CoreAnimation, CoreAudio, etc.) to make certain HW services more accessible. There's very little cool stuff to be gained in the windows world by developing a new kernel from scratch. A quantum leap would not solve MS's problem. The problem is the platform. What's really dead and bloated is the Win32 subsystem. The kernel doesn't need major tweaking. In fact, the NT kernel was designed from the beginning such that it could easily run the old busted Win32 subsystem alongside a new subsystem without needing to resort to expensive virtualization. Unfortunately, the way Microsoft is built today it have a fatal organizational flaw that prevents creating the next great Windows platform. The platform/dev tools team and the OS team are in completely different business groups within the company. The platform team develops the wonderful .NET platform for small/medium applications and server apps while the OS team keeps crudging along with Win32. Managed languages have their place, but they have yet to gain traction for any top shelf large-scale windows client application vendors (Adobe, even Microsoft Office itself, etc.) Major client application development still relies on unmanaged APIs, and IMHO the Windows unmanaged APIs are arguably the worst (viable) development platform available today. What Windows needs is a new subsystem/development platform to break with Win32, providing simplified, extensible unmanaged application development, with modern easy-to-use abstractions for hardware services such as graphics, data, audio and networking. This is starting to come to fruition with WinRT, but the inertia in large scale apps is unbelievable.
js4all 14y ago
The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture. https://news.ycombinator.com/item?id=2841934 https://news.ycombinator.com/item?id=2841934
dottrap 14y ago
We just can't be fucked to implement C11 support I thought Microsoft refusing to update their decades behind, ancient C compiler was just to piss me off and make life difficult cross-platform developers that need to work in C. Interesting to see this applies to their own employees too.
- bitwize 14y ago
  I believe the official policy is that C++ is the upgrade path from C, and C is therefore deprecated.
  - pherz 14y ago
    I think a lot of people that care about C support are already aware of Herb Sutter's stance on the matter. But knowing why he made his decision doesn't really change anything, its not like C++ never crossed their minds before. And classing it up by calling it official policy doesn't make the pill go down any easier. Microsoft's position in the market is already big enough that they'll never really be proven wrong on C unless they relent. But they're also not large enough to kill it outright.
    - dottrap 14y ago
      Yeah, Herb Sutter can GFHS. We use C for good reasons. C++ is not an upgrade, it is a completely different language. I just remembered Microsoft wrote this scathing document: C++ for Kernel Mode Drivers: Pros and Cons http://msdn.microsoft.com/en-us/library/windows/hardware/gg487420.aspx http://msdn.microsoft.com/en-us/library/windows/hardware/gg4... I didn't find an actual pro. Makes sense that a Microsoft Kernel developer would also be frustrated by their lack of modern C support.
drorweiss 14y ago
As an outsider, it looks like MSFT does not grasp kernel performance as a super important feature of Windows. They add so much bells & whistles over the UI, but the inside sucks release after release
- glhaynes 14y ago
  As someone else posted in this same thread: is modern Windows substantially slower than the OSes it competes with (in anything other than a few uncommon niches)? If it's more than a few percentage points slower for anything but rare scenarios, I'd be surprised.
api 14y ago
I've even seen this mentality in startups. It does have some business rationale, provided you are thinking short term and focused only on near-term goals. One of the reason businesses have trouble really innovating is that it's hard in a business to work on long-term things when markets are very short sighted. Only mega-corps, monopolies, and governments can usually do that... or hobbyists / lifestyle businesses who are more casual about hard business demands. That being said, MS is surely cash-rich enough to think long term. So this doesn't apply as much here. I've also found that of all things optimization almost gets you looked down upon in most teams -- even young ones. "Premature optimization is the root of all evil," and all that, which is usually misinterpreted as "optimization is naive and a waste of time." It's seen as indicative of an amateur or someone who isn't goal-focused. If you comment "optimized X" in a commit, you're likely to get mocked or reprimanded. In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.") Good optimization is actually a high-skill thing. It requires deep knowledge of internals, ability to really comprehend profiling, and precisely the kind of higher-order algorithmic reasoning you want in good developers. Most good optimizations are algorithmic improvements, not micro-optimizations. Even good micro-optimization requires deep knowledge-- like understanding how pipelines and branch prediction and caches work. To micro-optimize well you've got to understand soup-to-nuts everything that happens when your code is compiled and run. Personally I think speed is really important. As a customer I know that slow sites, slow apps, and slow server code can be a reason for me to stop using a product. Even if the speed difference doesn't impact things much, a faster "smoother" piece of code will convey a sense of quality. Slow code that kerchunks around "feels" inferior, like I can see the awful mess it must be inside. It's sort of like how luxury car engines are expected to "purr." An example: before I learned it and realized what an innovative paradigm shift it was, speed is what sold me on git. The first time I did a git merge on a huge project I was like "whoa, it's done already?" SVN would have been kerchunking forever. It wasn't that the speed mattered that much. It was that the speed communicated to me "this thing is the product of a very good programmer who took their craft very seriously as they wrote it." It told me to expect quality. Another example: I tried Google Drive, but uninstalled it after a day. It used too much CPU. In this case it actually mattered -- on a laptop this shortens battery life and my battery life noticeably declined. This was a while ago, but I have not been motivated to try it again. The slowness told me "this was a quick hack, not a priority." I use DropBox because their client barely uses the CPU at all, even when I modify a lot of files. Google Drive gives me more storage, but I'm not content to sacrifice an hour of battery life for that. (Side note: on mobile devices, CPU efficiency has a much more rigid cost function. Each cycle costs battery.) Speed is a stealth attribute too. Customers will almost never bring it up in a survey or a focus group unless it impacts their business. So it never becomes a business priority. Edit: relevant: http://ubiquity.acm.org/article.cfm?id=1513451 http://ubiquity.acm.org/article.cfm?id=1513451
mariuolo 14y ago
I suspect another aspect is for linux developers to be less worried about keeping the API/ABI stable.
- gizmo686 14y ago
  The Linux kernel developers take backwards compatibility very seriously. Here is a tame excerpt from Linux on the subject: "Seriously. Binary compatibility is so important that I do not want to have anything to do with kernel developers who don't understand that importance. If you continue to pooh-pooh the issue, you only show yourself to be unreliable. Don't do it." [1] [1]https://lkml.org/lkml/2012/3/8/495 https://lkml.org/lkml/2012/3/8/495
  - slacka 14y ago
    Why is it that I can run my Radeon x1900 under Windows 7/8 with Windows Vista drivers? Runs StarCraft2 and a bunch of newer FPS games just fine. Linux on the other hand is a disaster. The kernel devs are so determined to break binary compatibility, I haven't been able to run with ATI's proprietary binary drivers for years. While AMD was a good open source citizen and released the specs, the open source drivers for my card are useless for anything other than 2D.
    - lmz 14y ago
      Linus is talking about userspace binary compatibility. Your graphics drivers are relying on kernel module binary compatibility, which is not guaranteed.
      - slacka 14y ago
        Yes, but without working drivers for my system, what good is user space binary compatibility? MS has maintained compatibly for their drivers, something kernel developers seem to careless about. Which was exactly the point of the original poster.
crookedbeer 14y ago
Isn't Windows NT used only in large corporations that are afraid to upgrade their servers?
- ksherlock 14y ago
  The Windows NT _Kernel_ is used in Windows NT, 2000, XP, Vista, 7, 8, etc. (Versus the DOS kernel used in DOS, 95, 98, and Me)
  - yuhong 14y ago
    Yea, that is another pet peeve of mine. Windows 2000 is technically NT 5.0 (and was called that during the beta). XP is NT 5.1 etc...
  - CurtHagenlocher 14y ago
    Not to mention Windows Phone 8.
bitwize 14y ago
We can't touch named pipes. Let's add %INTERNAL_NOTIFICATION_SYSTEM%! And let's make it inconsistent with virtually every other named NT primitive. Linux does this motherfucking bullshit too. "Oh, systemd is piss slow? We'll bung d-bus straight into the kernel, right along side the over 9000 other IPC mechanisms. Everybody uses d-bus these days, it's an Essential System Component. What? Systemd is a crap idea to start with? People like you should be committed."
- ajross 14y ago
  Considering linux actually hasn't done this yet, that seems like a weak argument. Are there better examples?
- xfs 14y ago
  Update in the original article: Besides: you guys have systemd, which if I'm going to treat it the same way I treated NTFS, is an all-devouring octopus monster about crawl out of the sea and eat Tokyo and spit it out as a giant binary logfile.
deleted 14y ago
[deleted]
gdonelli 14y ago
"our good people keep retiring or moving to other large technology companies, and there are few new people achieving the level of technical virtuosity needed to replace the people who leave." in the post PC era Kernels are not as hot as they were in 2000
pdknsk 14y ago
> These junior developers also have a tendency to make improvements to the system by implementing brand-new features instead of improving old ones. That's exactly the problem that's plaguing Google Chrome right now, although probably for a different reason, as many senior developers still seem to be on board. Google keeps adding new features at high pace and doesn't care what brakes in the process. The amount of unfixed (albeit mostly minor) bugs is huge.
corresation 14y ago
Is performance behind other OS'? Serious question, because empirically I'm not aware of any particular performance lag.
brudgers 14y ago
Backward compatibility is an OS performance metric. Maybe sales is too. Microsoft has to think long and hard about any kernel change. In some irony, Microsoft doesn't own the Windows code, and any individual can own the Linux kernel - i.e. Windows lacks forks. That Microsoft discourages individual junior developers from cowboying, is a point in their favor. Optimization for its own sake is not what benefits their users - real research does.
cpncrunch 14y ago
I think the main performance problem with NT is its scheduler. Compared to unix-type schedulers, it doesn't give very good interactive performance when there are cpu-bound processes. There have been a lot of hacks over the years (like throwing cpu-starved processes a bone when they haven't had any action in 4 seconds), but unix/linux just has a better designed scheduler that gives better performance in most situations.
giulivo 14y ago
Given the topic of the article I understand my question could be seen as OT but I'd like to dig more into the following: > The NT kernel is still much better than Linux in some ways --- you guys be trippin' with your overcommit-by-default MM nonsense What's wrong with the overcommit-by-default behaviour? Or how/why the NT kernel is supposed to do better?
- asveikau 14y ago
  I don't know what the author was thinking of specifically, but the OOM killer seems like a bit of a hack to me. I can think of situations where I'd rather have an allocation fail upfront predictably than be randomly killed by the system. A lot of people seem to think you're screwed when you're out of memory and so think killing the process is acceptable, but I've worked on code bases where it's actually handled somewhat gracefully. (Although, on NT, if you run out of nonpaged pool you start to get weird, random I/O failures.)
  - giulivo 14y ago
    but again, if it is just about the default behaviour which we debate, one can simply disable the overcommit vm.overcommit_memory = 2 see https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
    - asveikau 14y ago
      And that was exactly the criticism, wasn't it? "Overcommit-by-default nonsense". I didn't write the original, just trying to parse it. (And with that aspect I kind of agree, it's a goofy behavior.) I guess there's another, somewhat social aspect of making that the default: everyone else coding on the platform assumes the behavior is the other way, so I suspect when you set that flag to off, your other software starts crashing on null pointer dereferences because no one thinks malloc can fail.
Zigurd 14y ago
The description sounds accurate compared to what I heard when I was in contact with Microsoft insiders. That was about 15 years ago. What's surprising to me is that it is still this way. A lot of the justification for being exceedingly conservative about performance improvements was based on the difficulty of doing testing that is sufficient to leave no doubt that changes don't cause regressions nor any side effects bearing on compatibility. I'm surprised that hasn't changed, or that the value equation about possibly breaking customer software hasn't changed. By now this approach must be a very entrenched culture.
rjdagost 14y ago
There is a great quote in the post: "Incremental improvements just annoy people and are, at best, neutral for your career." Working for several different Fortune 500 companies this has been my experience many times. I have worked on products where I made a large number of incremental improvements that never got incorporated into the released product. Taken on their own the improvements don't change things much, but in the aggregate they make a significantly better product. Why didn't most of my incremental improvements get incorporated into the product? Every change, no matter how minor, is perceived by management to be a highly risky proposition. Product managers won't get promoted because the product is 4% better, but they could potentially get fired if that same incremental improvement had an unforeseen negative consequence. Once the short term sales mindset gets power, hit products are viewed as cows to be milked and not as things that need continued maintenance and improvement. As an engineer in such an enterprise you watch helplessly as your competitors slowly but steadily pass you, and then someday upper management wonders why the magic money machine broke down. Some large enterprises avoid this fate but they seem to be the exception.
concision 14y ago
I'm a recent college graduate starting on a Windows Core team in the next few months. Whee. Talk about a downer to start my day.
- NamTaf 14y ago
  Look at it this way: you're lined up to work in a company with far more capability and potential than almost every other western organisation. When you're compared to the likes of Google and Apple as examples of where you might be lacking, you're doing good. Besides, even if everything is as bad as they say and you absolutely cannot work in that environment, interning is the perfect time to discover that without as significant commitments. It will also prove to be a valuable lesson that you can then take to other companies. Chin up, I'd love to work even at Microsoft.
kabdib 14y ago
I keep coming here, writing a bunch of stuff, then removing it. The author isn't far off. He's over the top on some things, but on the whole it's a good take on some of the reasons Microsoft culture is toxic. My take: 1. The review system needs to be fixed. They have to stop losing good engineers. 2. They have to somehow get rid of a bunch of toxic people. Many of these are in management. It's hard. 3. They have to stop treating Windows as "the thing we put in all the other things". Windows is a fine OS; it could be better, and the OP points out a lot of reasons why it isn't. But it's not a great fit for embedded systems, nor game consoles, nor anything where you're paying for the hardware. But I keep coming back to the review system rewarding the wrong people, causing good people to leave. The underlying backstabbing and cutthroat culture needs to go; it's hurting the bottom line, and I'm surprised the board has been willing to let it happen.
pootch 14y ago
Pig say "what? I aint no chicken!"
marshray 14y ago
Disclosure: I'm a new guy at Microsoft, not related to Kernel or Windows stuff. So where does he back up the claim that NT kernel performance is, in fact, "behind other OS"?
PaulHoule 14y ago
I support conservative engineering for file systems. NTFS is highly reliable and Microsoft wants to build on that success with ReFS. In years of working with NTFS I've never had a serious filesystem wreck (although I did have one that was never quite the same after I made a few hundred million small files) The NTFS story isn't that different from the ext(2|3|4) story on Linux. ext-N filesystems are boring and not particularly fast, but they're reliable. I've tried other filesystems on Linux but have often had wrecks or noticed data corruption. I like boring filesystems.

656 points by mrb 14y ago

What's wrong with 9-5? Can't you be passionate about what you work on, excel in you career, yet stick to dedicating ~50% of your time awake to your job?

freditup 14y ago
And 8hrs is probably too much time to really productively be working on a job at that.
kika 14y ago
Because you spend 9-noon and 1pm-3pm on meetings and you still need to code.
- kperusko 14y ago
  That sounds like an organizational problem to me if you spend 75% of your time at meetings everyday instead of coding (if coding is actually your job).
pjmlp 14y ago
It is an American thing. In the rest of the world we can happily do 9-5 without feeling bad for it.
- kperusko 14y ago
  Unfortunately that kind of thinking is no longer only an American thing. I see it more and more every day. Employers are pushing fresh college grads to work as many hours as possible for a low wage just so they can have a bigger profit. If they burn out, there are others to replace them.
- chamanbuga 14y ago
  It's amusing watching Americans boast about working past the norm 9-5 hours and calling the rest of the world lazy. My response, "What are you proud about? After working twice as hard, you have the same quality of life as friends in Canada, and statistically speaking you'll drop dead sooner because of stress." That always wipes the smug grin of their faces.
  - illuminate 14y ago
    "That always wipes the smug grin of their faces." While I agree with you, nothing could wipe the smug grin off these persons' faces. They are beyond reality, and I'd be much happier with America if something so simple could give the "socialists are universally lazy" rabble whenever labor rights are mentioned in any sense.
    - pjmlp 14y ago
      Yes, here in the socialist Germany I can enjoy my free time after 4:30 pm and if my boss wants me to stay longer I get paid overtime.

jsolson 14y ago

> Another reason for the quality gap is that that we've been having trouble keeping talented people. Google and other large Seattle-area companies keep poaching our best, most experienced developers, and we hire youths straight from college to replace them. I will say all of the ex-Microsoft folks I've encountered at Google Seattle have been fantastic. On a related note, it's stupidly easy to get code accepted by another team at Google. Also we're hiring.

dvt 14y ago
>Also we're hiring. Insult to injury? ;)
tracker1 14y ago
Too bad you guys don't seem to give a second glance at someone without any formal education, despite working on systems for government, training, banking security and airline industries.
- hythloday 14y ago
  I have zero formal education past A levels, but I've had two in-person interviews at Google and had an offer, and I have their charming recruiters chasing me to make sure I wouldn't be happier working for Google every 6 months or so. :) It does take a ferociously long time to get through the initial screening if you just send them a CV, and I'm sure a lot of people fall through the cracks - my suggestion would be to find a Googler or Xoogler and ask them to recommend you, I understand you get a lot more traction that way. Chin up.
- ajross 14y ago
  To be fair, and in context: neither does Microsoft.
  - tracker1 14y ago
    I've been approached a number of times by MS... once fairly aggressively. Never a word or response from Google though.

dietrichepp 14y ago

I think there are two complementary parts that Linux gets right here. One is as cited in the article, that you get personal glory for improvements you make to the kernel even if they are fairly small. The other part is that there is for Linux someone who will say "no" to a patch. Linus will certainly do it, and other trusted devs will do it too. Linus is perhaps even famous for telling people when they are wrong. I've seen a number of open-source projects where you get all the personal glory for your additions but there is nobody who takes the responsibility to tell people "no". These projects, almost universally, turn into bloated messes over time. Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games.

philsnow 14y ago
> Open-sourced computer games seem to fall down this path more easily, since everyone and her little dog too has ideas about features to add to games. I just listened to an episode of Roguelike Radio from the mid-30s (maybe the one with Red Rogue or Thomas Biskup) and Andrew / Keith (IIRC) go on and on about how all the "major" roguelikes are just accretions of features. This rings very true to me; I grew up playing Nethack [0], and Slash (and then Slash'Em) just got silly with all the things they were adding. Light sabers ? In my roguelike ? Come on, fhgwgads. Nethack at least has a mysterious Dev Team who ostensibly are the guardians of quality... but they _did_ bring Sokoban back from Slash'EM into vanilla. [0] since Nethack was my first roguelike, I didn't realize until lately just how crappy it is in many ways, most notably that it's very difficult to win before you've figured out all the sources of instadeath (whether you've figured them out by spoilers (written or source-diving) or through (really frickin hard-won) experience).
- p0nce 14y ago
  Try brogue, it's a breath of fresh air.
  - philsnow 14y ago
    I <3 the crap out of brogue; I'm currently playing through some roguelikes that I've never looked at before [0], and I keep coming back to brogue. In particular, brogue's food clock is excellent in that if I forget about it too long and go back to my natural "explore everywhere, kill everything" tendency, the food clock _will_ kill me. [0] something that jumped out at me is just how much modern roguelike development is windows-first (or even windows-only), and how much of it is distributed primarily as runnable binaries vs source. I had to bend way over backwards to get PrincessRL running: I ended up installing a 32-bit ubuntu userspace virtualbox instance so that mono could run the prepackaged binary.

quotemstr 14y ago

I actually enjoy working at Microsoft --- I'm in Phone, working on telemetry and various other things --- and I've met a ton of very smart people people. I've also made the cross-team (and cross-org) contributions that the OP says are nearly impossible (just today, even). While the OP makes a few good points (some teams are kinda reluctant to take patches), I think he's grossly exaggerating the problems, and the level of vitriol really isn't called for either. He's also slightly off-base on some of the technical criticisms: there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent. Besides, change _does_ happen: Arun Kishan (who is smarter than I'll ever be) broke the dispatcher lock a while ago (see http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewell-to-the-Windows-Kernel-Dispatcher-Lock/ http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewe...) when nobody thought it could be done.

quotemstr 14y ago
By the way: there's actually quite a lot of information available on Windows Internals. In fact, there's a whole book, Windows Internals (http://technet.microsoft.com/en-us/sysinternals/bb963901.aspx http://technet.microsoft.com/en-us/sysinternals/bb963901.asp...) on exactly how the NT kernel and related system components work. The book is highly recommended reading and demystifies a lot of the odder parts of our API surface. We use it internally all the time, and reading it, you'll repeatedly say to yourself, "Ah, so that's why Foo does Bar and not Qux! Yeah, that's a good reason. I didn't think of it that way."
angersock 14y ago
"there are often good reasons for doing things a certain way, and these reasons aren't always immediately apparent." One of the issues the author seemed to be alluding to is that the loss of experienced developers makes it really hard to keep track of "line 123 of foo_dispatch.c does this because of vital business reason xyz or compatibility reason uvw" and "line 123 of foo_dispatch.c was written while I was hungover and trying to make a deadline--it looks clever, but feel free to junk it.". This issue is compounded when you are hiring safe, and you have a culture where making gross fuckups while trying to make progress is discouraged. It is neither good nor bad--after all, I too enjoy using stable software--but there is a price to be paid if devs don't feel comfortable making breaking changes.

pg 14y ago

"His post has been deleted! Why the censorship?" We didn't delete it. It was deleted by whoever posted it.

mrb 14y ago
Thanks. I added the explanation.
- skeletonjelly 14y ago
  To be fair to pg, I feel like this was the obvious conclusion (that an anonymous user posting via tor would redact)
jamesaguilar 14y ago
No delete button on the internet v125786123. Sidebar: do deleted comments appear with showdead on, or just ones that are deleted on HN side.
- deleted 14y ago
  [deleted]
- hollerith 14y ago
  >do [deleted-by-author] comments appear with showdead on[?] No, they do not. Specifically, I have showdead on, and when trying to view the comment in question, I see only "[deleted]".

softbuilder 14y ago

>We fill headcount with nine-to-five-with-kids types, then >We occasionally get good people anyway Uh... nine-to-five-with-kids type here. Thanks for the stereotyping... from your safe corporate nest. Otherwise an insightful post.

temphn 14y ago
If your first priority is your kids, your first priority is not your work. That's fine, it's a choice, but there is a frequent claim that the massive incremental time demand of kids makes one so much more time efficient at work that it more-than-compensates. Insofar as one is the same human being, with the same energy reservoirs and time management skills as before your child was born, this is unlikely. I understand the reason this fiction is maintained: people with kids need the job even more than people without, and have an interest in denouncing people who claim kids make you less productive. There's also the second order effect in which "with kids" is correlated with "older". The net of it though is that devs with kids tend to assign work a lower priority, to take fewer risks, to need more money, and to be older and hence less familiar with new technologies (and too busy to learn in their free time). Society doesn't have a good answer for this situation yet. In times past, technology didn't move so fast that experience was mostly obsolete (and hence useless) in a decade's time. A 40 year old farmer with kids in 1713 would probably have much to teach a young whippersnapper. The same isn't true for a 40 year old programmer with kids in 2013.
- madewulf 14y ago
  If your experience of programming becomes worthless after 10 years, then you probably did not focus on the good skills. I don't see how learning how low level hardware works, some time tried languages (C for example), data structures, networking details, debugging protocols of all kinds, structuring programs, working in teams and so on and so forth... could become worthless. For the idea that parents and older programmers have different priorities, that may be true, but you downplay experience way too much. That said, trying to find 150% dedicated younglings, that don't spend a lot of time on side projects and just enjoying life is not that easy either

tzs 14y ago

It's possible he deleted his post because he realized that he included something in it that could identify him, which could be bad for his career. It might have been better to rewrite his post into your own words, and take out some of the unnecessary detail, rather than literally repost.

logn 14y ago
I'd agree. I know the OP is trying to spread knowledge and this is a great read, but I think this was slightly rude, not to mention questionable fair use.
- dangrossman 14y ago
  There's no "fair use" argument at all. Copying an entire work, then distributing it with no transformative change or criticism, is inexcusable infringement should the author want to enforce his rights.
  - robryan 14y ago
    Although in this case I think it would be hard for this author to enforce his/her rights and still remain completely anonymous?
    - iso8859-1 14y ago
      which just makes it even more wrong...

outside1234 14y ago

Windows is dying just like DOS. Azure is the new Windows.

illuminate 14y ago
I don't know what this is even supposed to mean.

kevingadd 14y ago

It's nice to see that internal developers feel the same way about XNA that external developers (who used to build XNA games, or still build XNA games) do. From the outside I always assumed the constant flood of new, half-baked features instead of fixes and improvements to old ones was caused by interns and junior devs looking for glory - sad to hear that's actually partly true. I always considered frameworks like WPF (or Flex, for that matter) 'intern code' - not that interns necessarily wrote them, but they reek of not-experienced-enough engineers trying to solve problems by writing a bunch of new code, instead of fixing existing code. It really is too bad, though. There are parts of the NT kernel (and even the Win32 API) that I consider a joy to use - I love IOCP, despite its warts, and APIs like MsgWaitForMultipleObjects are great tools for building higher-level primitives. Plus, say what you want about GDI (there's a lot wrong with it at this point), but it's still a surprisingly efficient and flexible way to do 2D rendering, despite the fact that parts of it date back to before Windows 3.1. Some really smart people did some really good API design over time over at Microsoft...

quotemstr 14y ago
Actually, I think one NT's largest advantages over POSIX systems is process management: yes, the venerable CreateProcess API. See, in Windows, processes are first class kernel objects. You have handles (read: file descriptors) that refer to them. Processes have POSIX-style PIDs too, but you don't use a PID to manipulate a process the way you would with kill(2): you use a PID to open a handle to a process, then you manipulate the process using the handle. This approach, at a stroke, solves all the wait, wait3, wait4, SIGCHLD, etc. problems that plague Unixish systems to this day. (Oh, and while you have a handle to a process open, its process ID won't be re-used.) It's as if we live in a better, alternate universe where fork(2) returns a file descriptor. You can wait on process handles (the handle becomes signaled and the wait completes when the process exits). You can perform this waiting using the same functions you use to wait on anything else, and you can use WaitForMultipleObjects as a kind of super-select to wait on anything. If you want to wait on a socket, a process, and a global mutex and wake up when any of these things becomes available, you can do that. The Unix APIs for doing the same thing are a mess. Don't even get me started on SysV IPC. Another thing I really like about NT is job objects (http://msdn.microsoft.com/en-us/library/windows/desktop/ms684161%28v=vs.85%29.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/ms68...). They're a bit like cgroups, but a bit simpler (IMHO) to set up and use. You can apply memory use, scheduling, UI, and other restrictions to processes in job objects. Most conveniently of all, you can arrange for the OS to kill everything in a job object if the last handle to that job dies --- the closest Linux has is PR_SET_PDEATHSIG, which needs to be set up individually for each child and which doesn't work for setuid children. (Oh, and you can arrange for job objects to send notifications to IO completion ports.) Yes, Windows gets a lot wrong, but it gets a lot right.

termie 14y ago

UTF-16 everywhere doesn't help either. M\0o\0r\0e\0 \0d\0a\0t\0a.\0

quotemstr 14y ago
You do realize that NT is from 1989 and that UTF-8 is from 1992, right?
- termie 14y ago
  Yeah. Which makes it even more irritating.. MS is still off in the double-wide stix 11 years later and still promoting UTF-16LE in userland. From http://msdn.microsoft.com/en-us/library/dd374081%28VS.85%29.aspx http://msdn.microsoft.com/en-us/library/dd374081%28VS.85%29.... "Unicode-enabled functions are described in Conventions for Function Prototypes. These functions use UTF-16 (wide character) encoding, which is the most common encoding of Unicode and the one used for native Unicode encoding on Windows operating systems. Each code value is 16 bits wide .. New Windows applications should use UTF-16 as their internal data representation."
  - philsnow 14y ago
    "Most common" encoding is best encoding! ugh
  - jeltz 14y ago
    I seriously doubt that UTF-16 still is the most common encoding. The web is mostly UTF-8 and so are most smartphones.
    - sratner 14y ago
      JavaScript strings are UCS-2 or UTF-16.
- clhodapp 14y ago
  Wikipedia tells me that Windows NT was started in 1988 and shipped in 1993, while Linux's first release was in 1991, giving them approximately the same age. Furthermore, NT shipped with UCS-2 support, not UTF-16, as UTF-16 did not exist until 1996. UTF-16 was a migration that they completed in Windows 2000. UTF-8 was first presented in 1993 and is, therefore, older than UTF-16. With all of these facts being true (I hope), today things are such that the Windows world mostly (but not completely -- they sometimes assume two-byte characters) correctly implements UTF-16, while the Linux world correctly implements UTF-8.

kyllo 14y ago

Look at recent Microsoft releases: we don't fix old features, but accrete new ones. New features help much more at review time than improvements to old ones. (That's literally the explanation for PowerShell. Many of us wanted to improve cmd.exe, but couldn't.) Ahh, I was wondering about that. So, I guess I'll just keep using cygwin. On another note, I recently asked a friend who works at Microsoft how work is going. His reply: "Well, it's calibration time, so lots of sucking up to the boss." Must be hard to get much actual work done when you're worried about that all the time.

aaronbrethorst 14y ago
Yep, sounds about right. Building new stuff is rewarded far more than fixing broken shit. Do you want to make it right or do you want to get promoted?
yuhong 14y ago
michaelochurch has talked about problems with this, performance reviews and closed allocation before.
- skrebbel 14y ago
  well, he doesn't talk about anything else :) I do wonder, however, whether 'boo! MS's rating system is suboptimal, to say the least!' is a very useful comment here (which might by why we haven't seen Michael write here yet)
- kyllo 14y ago
  Yeah, and that was about Google, which has a much better reputation than Microsoft in this area. I wonder how he would like working at MS. I doubt he'd last a week there!
invisible 14y ago
For those unfamiliar with what "calibration" is in Microsoft: http://minimsft.blogspot.com/2011/04/microsofts-new-review-and-compensation.html http://minimsft.blogspot.com/2011/04/microsofts-new-review-a...
14y ago

chetanahuja 14y ago

tl;dr : corporatism and careerism. It's the death of creativity and productivity. No large organization is immune to it. Never will be. (and yes, that includes Google... it's just much smaller and newer than Microsoft is right now)

yuhong 14y ago
michaelochurch has talked about this before, including the fundamental flaws behind it.
- randomfool 14y ago
  He's also a vindictive windbag who is extremely disgruntled about his shortcomings at Google.
  - deleted 14y ago
    [deleted]
  - yekko 14y ago
    And he's totally correct.

deleted 14y ago

[deleted]

deleted 14y ago

[deleted]

millerc 14y ago

Sad post IMO, which will unfortunately and unduly tarnish Microsoft's reputation. I would suppose it was written by a developer who has just become senior enough to see some of what's going on, but has not yet realized it's everywhere like that. One day, he'll understand that those challenges too need to be managed. Growing pains, all around.

gizmo686 14y ago

Slightly off topic, but is there anyway we can actually confirm that "the SHA-1 hash of revision #102 of [redacted] is [redacted]." EDIT: redacted information. Still, when that information we present, how would anyone be able to confirm it?

JoachimSchipper 14y ago
The article redacted that data on the request of the original poster; please remove it from your post.
joe5150 14y ago
I had the same question but was too embarrassed to ask. Thanks for doing it for me. :)
RyJones 14y ago
I assume some other dev with source access at Microsoft could verify that fact, broadcast that the fact is true, and thereby stamp the OP as actually who he says he is.
marshray 14y ago
From what I've heard (unofficially) the source is typically watermarked to identify the sources of any leaks. He may have given himself away with the SHA-1.
- gizmo686 14y ago
  I doubt it. Not only would that involve generating a different watermark for each person with access to the code, but in doing so you also raise complications for source control. Additionally, it is almost impossible for developers not to notice if supposedly identical files are not identical, not to mention the fact that implementing the type of system without the conscious corporation of the developers seems impossible.

wfunction 14y ago

At some point Microsoft might want to consider expanding its kernel team into Silicon Valley.

dchest 14y ago

Ah, so the symlink plan is true: https://gist.github.com/800407/3588729d57abb7731985c99c492a2a08459d7b2f https://gist.github.com/800407/3588729d57abb7731985c99c492a2...

Mo1oKo 14y ago

"The rot has already set in" Great words for concluding a video game intro

pjmlp 14y ago

The typical scenario you will find in any big corporation developing software. I started to understand better how Microsoft works, after getting into the Fortune 500 enterprise world. Many of the bad things geeks associate with Microsoft are actually present in any development unit of big clunky Fortune 500 companies.

js4all 14y ago

The problems with the NT kernel are nothing new. When I suggested for Windows to use a BSD kernel to overcome its current limitations, I got lots of disagreement. Microsoft developers seem to be happy with the current state and have no vision of a totally new architecture. https://news.ycombinator.com/item?id=2841934 https://news.ycombinator.com/item?id=2841934

Zigurd 14y ago
The NT-based kernels were not as big a problem for Microsoft as CE-based OSs were. Nokia transitioned away from a Linux-based smartphone OS to Windows Phone 7, based on Windows Embedded CE 6. It is clear that Windows Phone 7 was uncompetitive with other smartphone OSs. That put Nokia in limbo for a year when they could least afford it. Now the problem is not so much the kernel, but all the cruft a Windows-based OS drags along with it. The most visible symptom is the amount of storage used up by stuff that adds little or no readily apparent value to Windows RT. Windows OSs are heavy compared to Android and other Linux-based OSs. That means inexpensive tablets and phones won't run Windows, and Windows is shut out of mobile device markets in India, Africa, and other developing areas. These markets are how Yulong (who??!!) became a top 10 mobile OEM.
United857 14y ago
People are praising BSD like it was all hot and new. OS X is built on a BSD which has it's roots in 60's and 70's OS design, just like the VMS roots of WinNT. OS X didn't change the world by bringing some great new underlying architecture to the table. In fact, their kernel and filesystem are arguably getting long in the tooth. The value that OS X brought to the table was the fantastic Carbon and Cocoa development platforms. And they have continued to execute and iterate on these platforms, providing the "Core" series of APIs (CoreGraphics, CoreAnimation, CoreAudio, etc.) to make certain HW services more accessible. There's very little cool stuff to be gained in the windows world by developing a new kernel from scratch. A quantum leap would not solve MS's problem. The problem is the platform. What's really dead and bloated is the Win32 subsystem. The kernel doesn't need major tweaking. In fact, the NT kernel was designed from the beginning such that it could easily run the old busted Win32 subsystem alongside a new subsystem without needing to resort to expensive virtualization. Unfortunately, the way Microsoft is built today it have a fatal organizational flaw that prevents creating the next great Windows platform. The platform/dev tools team and the OS team are in completely different business groups within the company. The platform team develops the wonderful .NET platform for small/medium applications and server apps while the OS team keeps crudging along with Win32. Managed languages have their place, but they have yet to gain traction for any top shelf large-scale windows client application vendors (Adobe, even Microsoft Office itself, etc.) Major client application development still relies on unmanaged APIs, and IMHO the Windows unmanaged APIs are arguably the worst (viable) development platform available today. What Windows needs is a new subsystem/development platform to break with Win32, providing simplified, extensible unmanaged application development, with modern easy-to-use abstractions for hardware services such as graphics, data, audio and networking. This is starting to come to fruition with WinRT, but the inertia in large scale apps is unbelievable.

js4all 14y ago

dottrap 14y ago

We just can't be fucked to implement C11 support I thought Microsoft refusing to update their decades behind, ancient C compiler was just to piss me off and make life difficult cross-platform developers that need to work in C. Interesting to see this applies to their own employees too.

bitwize 14y ago
I believe the official policy is that C++ is the upgrade path from C, and C is therefore deprecated.
- pherz 14y ago
  I think a lot of people that care about C support are already aware of Herb Sutter's stance on the matter. But knowing why he made his decision doesn't really change anything, its not like C++ never crossed their minds before. And classing it up by calling it official policy doesn't make the pill go down any easier. Microsoft's position in the market is already big enough that they'll never really be proven wrong on C unless they relent. But they're also not large enough to kill it outright.
  - dottrap 14y ago
    Yeah, Herb Sutter can GFHS. We use C for good reasons. C++ is not an upgrade, it is a completely different language. I just remembered Microsoft wrote this scathing document: C++ for Kernel Mode Drivers: Pros and Cons http://msdn.microsoft.com/en-us/library/windows/hardware/gg487420.aspx http://msdn.microsoft.com/en-us/library/windows/hardware/gg4... I didn't find an actual pro. Makes sense that a Microsoft Kernel developer would also be frustrated by their lack of modern C support.

drorweiss 14y ago

As an outsider, it looks like MSFT does not grasp kernel performance as a super important feature of Windows. They add so much bells & whistles over the UI, but the inside sucks release after release

glhaynes 14y ago
As someone else posted in this same thread: is modern Windows substantially slower than the OSes it competes with (in anything other than a few uncommon niches)? If it's more than a few percentage points slower for anything but rare scenarios, I'd be surprised.

api 14y ago

I've even seen this mentality in startups. It does have some business rationale, provided you are thinking short term and focused only on near-term goals. One of the reason businesses have trouble really innovating is that it's hard in a business to work on long-term things when markets are very short sighted. Only mega-corps, monopolies, and governments can usually do that... or hobbyists / lifestyle businesses who are more casual about hard business demands. That being said, MS is surely cash-rich enough to think long term. So this doesn't apply as much here. I've also found that of all things optimization almost gets you looked down upon in most teams -- even young ones. "Premature optimization is the root of all evil," and all that, which is usually misinterpreted as "optimization is naive and a waste of time." It's seen as indicative of an amateur or someone who isn't goal-focused. If you comment "optimized X" in a commit, you're likely to get mocked or reprimanded. In reality, "premature optimization is the root of all evil" is advice given to new programmers so they don't waste time dinking around with micro-optimizations instead of thinking about algorithms, data structures, and higher order reasoning. (Or worse, muddying their code up to make it "fast.") Good optimization is actually a high-skill thing. It requires deep knowledge of internals, ability to really comprehend profiling, and precisely the kind of higher-order algorithmic reasoning you want in good developers. Most good optimizations are algorithmic improvements, not micro-optimizations. Even good micro-optimization requires deep knowledge-- like understanding how pipelines and branch prediction and caches work. To micro-optimize well you've got to understand soup-to-nuts everything that happens when your code is compiled and run. Personally I think speed is really important. As a customer I know that slow sites, slow apps, and slow server code can be a reason for me to stop using a product. Even if the speed difference doesn't impact things much, a faster "smoother" piece of code will convey a sense of quality. Slow code that kerchunks around "feels" inferior, like I can see the awful mess it must be inside. It's sort of like how luxury car engines are expected to "purr." An example: before I learned it and realized what an innovative paradigm shift it was, speed is what sold me on git. The first time I did a git merge on a huge project I was like "whoa, it's done already?" SVN would have been kerchunking forever. It wasn't that the speed mattered that much. It was that the speed communicated to me "this thing is the product of a very good programmer who took their craft very seriously as they wrote it." It told me to expect quality. Another example: I tried Google Drive, but uninstalled it after a day. It used too much CPU. In this case it actually mattered -- on a laptop this shortens battery life and my battery life noticeably declined. This was a while ago, but I have not been motivated to try it again. The slowness told me "this was a quick hack, not a priority." I use DropBox because their client barely uses the CPU at all, even when I modify a lot of files. Google Drive gives me more storage, but I'm not content to sacrifice an hour of battery life for that. (Side note: on mobile devices, CPU efficiency has a much more rigid cost function. Each cycle costs battery.) Speed is a stealth attribute too. Customers will almost never bring it up in a survey or a focus group unless it impacts their business. So it never becomes a business priority. Edit: relevant: http://ubiquity.acm.org/article.cfm?id=1513451 http://ubiquity.acm.org/article.cfm?id=1513451

mariuolo 14y ago

I suspect another aspect is for linux developers to be less worried about keeping the API/ABI stable.

gizmo686 14y ago
The Linux kernel developers take backwards compatibility very seriously. Here is a tame excerpt from Linux on the subject: "Seriously. Binary compatibility is so important that I do not want to have anything to do with kernel developers who don't understand that importance. If you continue to pooh-pooh the issue, you only show yourself to be unreliable. Don't do it." [1] [1]https://lkml.org/lkml/2012/3/8/495 https://lkml.org/lkml/2012/3/8/495
- slacka 14y ago
  Why is it that I can run my Radeon x1900 under Windows 7/8 with Windows Vista drivers? Runs StarCraft2 and a bunch of newer FPS games just fine. Linux on the other hand is a disaster. The kernel devs are so determined to break binary compatibility, I haven't been able to run with ATI's proprietary binary drivers for years. While AMD was a good open source citizen and released the specs, the open source drivers for my card are useless for anything other than 2D.
  - lmz 14y ago
    Linus is talking about userspace binary compatibility. Your graphics drivers are relying on kernel module binary compatibility, which is not guaranteed.
    - slacka 14y ago
      Yes, but without working drivers for my system, what good is user space binary compatibility? MS has maintained compatibly for their drivers, something kernel developers seem to careless about. Which was exactly the point of the original poster.

crookedbeer 14y ago

Isn't Windows NT used only in large corporations that are afraid to upgrade their servers?

ksherlock 14y ago
The Windows NT _Kernel_ is used in Windows NT, 2000, XP, Vista, 7, 8, etc. (Versus the DOS kernel used in DOS, 95, 98, and Me)
- yuhong 14y ago
  Yea, that is another pet peeve of mine. Windows 2000 is technically NT 5.0 (and was called that during the beta). XP is NT 5.1 etc...
- CurtHagenlocher 14y ago
  Not to mention Windows Phone 8.

bitwize 14y ago

We can't touch named pipes. Let's add %INTERNAL_NOTIFICATION_SYSTEM%! And let's make it inconsistent with virtually every other named NT primitive. Linux does this motherfucking bullshit too. "Oh, systemd is piss slow? We'll bung d-bus straight into the kernel, right along side the over 9000 other IPC mechanisms. Everybody uses d-bus these days, it's an Essential System Component. What? Systemd is a crap idea to start with? People like you should be committed."

ajross 14y ago
Considering linux actually hasn't done this yet, that seems like a weak argument. Are there better examples?
xfs 14y ago
Update in the original article: Besides: you guys have systemd, which if I'm going to treat it the same way I treated NTFS, is an all-devouring octopus monster about crawl out of the sea and eat Tokyo and spit it out as a giant binary logfile.

deleted 14y ago

[deleted]

gdonelli 14y ago

"our good people keep retiring or moving to other large technology companies, and there are few new people achieving the level of technical virtuosity needed to replace the people who leave." in the post PC era Kernels are not as hot as they were in 2000

pdknsk 14y ago

> These junior developers also have a tendency to make improvements to the system by implementing brand-new features instead of improving old ones. That's exactly the problem that's plaguing Google Chrome right now, although probably for a different reason, as many senior developers still seem to be on board. Google keeps adding new features at high pace and doesn't care what brakes in the process. The amount of unfixed (albeit mostly minor) bugs is huge.

corresation 14y ago

Is performance behind other OS'? Serious question, because empirically I'm not aware of any particular performance lag.

brudgers 14y ago

Backward compatibility is an OS performance metric. Maybe sales is too. Microsoft has to think long and hard about any kernel change. In some irony, Microsoft doesn't own the Windows code, and any individual can own the Linux kernel - i.e. Windows lacks forks. That Microsoft discourages individual junior developers from cowboying, is a point in their favor. Optimization for its own sake is not what benefits their users - real research does.

cpncrunch 14y ago

I think the main performance problem with NT is its scheduler. Compared to unix-type schedulers, it doesn't give very good interactive performance when there are cpu-bound processes. There have been a lot of hacks over the years (like throwing cpu-starved processes a bone when they haven't had any action in 4 seconds), but unix/linux just has a better designed scheduler that gives better performance in most situations.

giulivo 14y ago

Given the topic of the article I understand my question could be seen as OT but I'd like to dig more into the following: > The NT kernel is still much better than Linux in some ways --- you guys be trippin' with your overcommit-by-default MM nonsense What's wrong with the overcommit-by-default behaviour? Or how/why the NT kernel is supposed to do better?

asveikau 14y ago
I don't know what the author was thinking of specifically, but the OOM killer seems like a bit of a hack to me. I can think of situations where I'd rather have an allocation fail upfront predictably than be randomly killed by the system. A lot of people seem to think you're screwed when you're out of memory and so think killing the process is acceptable, but I've worked on code bases where it's actually handled somewhat gracefully. (Although, on NT, if you run out of nonpaged pool you start to get weird, random I/O failures.)
- giulivo 14y ago
  but again, if it is just about the default behaviour which we debate, one can simply disable the overcommit vm.overcommit_memory = 2 see https://www.kernel.org/doc/Documentation/vm/overcommit-accounting https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
  - asveikau 14y ago
    And that was exactly the criticism, wasn't it? "Overcommit-by-default nonsense". I didn't write the original, just trying to parse it. (And with that aspect I kind of agree, it's a goofy behavior.) I guess there's another, somewhat social aspect of making that the default: everyone else coding on the platform assumes the behavior is the other way, so I suspect when you set that flag to off, your other software starts crashing on null pointer dereferences because no one thinks malloc can fail.

Zigurd 14y ago

The description sounds accurate compared to what I heard when I was in contact with Microsoft insiders. That was about 15 years ago. What's surprising to me is that it is still this way. A lot of the justification for being exceedingly conservative about performance improvements was based on the difficulty of doing testing that is sufficient to leave no doubt that changes don't cause regressions nor any side effects bearing on compatibility. I'm surprised that hasn't changed, or that the value equation about possibly breaking customer software hasn't changed. By now this approach must be a very entrenched culture.

rjdagost 14y ago

There is a great quote in the post: "Incremental improvements just annoy people and are, at best, neutral for your career." Working for several different Fortune 500 companies this has been my experience many times. I have worked on products where I made a large number of incremental improvements that never got incorporated into the released product. Taken on their own the improvements don't change things much, but in the aggregate they make a significantly better product. Why didn't most of my incremental improvements get incorporated into the product? Every change, no matter how minor, is perceived by management to be a highly risky proposition. Product managers won't get promoted because the product is 4% better, but they could potentially get fired if that same incremental improvement had an unforeseen negative consequence. Once the short term sales mindset gets power, hit products are viewed as cows to be milked and not as things that need continued maintenance and improvement. As an engineer in such an enterprise you watch helplessly as your competitors slowly but steadily pass you, and then someday upper management wonders why the magic money machine broke down. Some large enterprises avoid this fate but they seem to be the exception.

concision 14y ago

I'm a recent college graduate starting on a Windows Core team in the next few months. Whee. Talk about a downer to start my day.

NamTaf 14y ago
Look at it this way: you're lined up to work in a company with far more capability and potential than almost every other western organisation. When you're compared to the likes of Google and Apple as examples of where you might be lacking, you're doing good. Besides, even if everything is as bad as they say and you absolutely cannot work in that environment, interning is the perfect time to discover that without as significant commitments. It will also prove to be a valuable lesson that you can then take to other companies. Chin up, I'd love to work even at Microsoft.

kabdib 14y ago

I keep coming here, writing a bunch of stuff, then removing it. The author isn't far off. He's over the top on some things, but on the whole it's a good take on some of the reasons Microsoft culture is toxic. My take: 1. The review system needs to be fixed. They have to stop losing good engineers. 2. They have to somehow get rid of a bunch of toxic people. Many of these are in management. It's hard. 3. They have to stop treating Windows as "the thing we put in all the other things". Windows is a fine OS; it could be better, and the OP points out a lot of reasons why it isn't. But it's not a great fit for embedded systems, nor game consoles, nor anything where you're paying for the hardware. But I keep coming back to the review system rewarding the wrong people, causing good people to leave. The underlying backstabbing and cutthroat culture needs to go; it's hurting the bottom line, and I'm surprised the board has been willing to let it happen.

pootch 14y ago

Pig say "what? I aint no chicken!"

marshray 14y ago

Disclosure: I'm a new guy at Microsoft, not related to Kernel or Windows stuff. So where does he back up the claim that NT kernel performance is, in fact, "behind other OS"?

PaulHoule 14y ago

I support conservative engineering for file systems. NTFS is highly reliable and Microsoft wants to build on that success with ReFS. In years of working with NTFS I've never had a serious filesystem wreck (although I did have one that was never quite the same after I made a few hundred million small files) The NTFS story isn't that different from the ext(2|3|4) story on Linux. ext-N filesystems are boring and not particularly fast, but they're reliable. I've tried other filesystems on Linux but have often had wrecks or noticed data corruption. I like boring filesystems.