You are making a classic economic error, here, of seeing a company with good marketing that grew huge by selling people something, and assuming that it is that particular company's fault for causing the market to want that something. A much simpler explanation for the evidence is that people already did want that thing, but had nowhere to get it; and that the company grew huge by responding to untapped demand.
People, by-and-large, like magic. People want appliance-like technology that they don't have to understand. It is not Microsoft's fault (or Apple's, or Google's, or IBM's, or Nintendo's, for that matter.) People have wanted appliance-like computers since before there were computers.
The computer-like devices in speculative fiction novels from the early 20th century (written by, and for, the nerdiest of nerds) almost never resembled "general-purpose computers"; they mostly envisioned something roughly combining Skype, a wiki, and Siri. Englebart's Mother Of All Demos is a good example of what people were dreaming about: fluid communication and information sharing/retrieval. "Programmability" was never really a feature given much concern, even by the most idealistic futurist. People just don't want programmability. They don't lust after it like they do other whiz-bang features. They don't dream of a world where your tablet can be used to write programs that run on your toaster.
As for why...
As far as I've observed, programming a computer is, to most people, a mind-numbing consideration; a complex task, with a lot of rules to understand, facts to memorize, considerations to hold in one's head, and requiring heavy rigor. It's Engineering, in other words.
Most people aren't engineers; there is a specific mindset required, and most people don't seem to like being in that mindset, let alone staying in it for long enough to get good at using it. It seems to come down to "logical thinking" (or "System 2" thinking[1], or "Formal operational" thinking[2].) From what I've seen in trying to teach people to program, for most people, it hurts--possibly as far as causing an actual headache--to engage in rigorous, conscious logical analysis. People both try to avoid it for as long as possible, and to escape it as quickly as possible once they've started.
We--I don't know which "we", maybe "hackers" or "engineers" or "nerds" or some other term--we here don't seem to have this problem. Logical thinking comes simply to us; we get jobs requiring almost constant use of it, and then we engage in it in our off-hours as well. We might never stop using it. We are a minority.
This is why I say that "programming literacy" is a bad metaphor for becoming skilled at programming: "literacy" is something that (excepting learning disabilities) everybody is equally capable of attaining. Imagine if 90% of the population--forced to practice reading for years--could still only manage to read five or six words per minute, and doing so for 20 minutes or so caused them to need to take a walk outside to "clear their head." That's what happens to the "bottom hump" of the bimodal distribution in a CS101 class--and they're the ones prequalified by being in a CS101 class; the population at large does even worse.
So, there is no reason the average computer needs (serious) programming tools pre-installed on it, since there is no reason to expect the average person to want to be a programmer. Sure, programming tools should be made available to anyone who wants them, from a young age--as should supplies for any other craft a child might want to explore (paints, a keyboard, clay, a chemistry set, etc.) But every computer these days has the internet--and most of them even have app stores!--so it's easy enough to find, and then download, something like Shoes or Scratch if you want to play around and learn. I could even see including a learning environment like that (or maybe something more like Hypercard; if only~) with the OS, for the same reason Windows ships with MSPaint, and OSX ships with Garage Band.
But shipping Visual Studio, or XCode, or even gcc, with a computer, is more like giving everyone AutoCAD than giving them MSPaint. It's silly and overkill: only 0.1% of the population would even know what it was if they accidentally managed to get it open. Of those, most of them wouldn't be at a level to understand it anyway; they would need something easier to start with.
(All of this is separate from the "the system relies on the compiler toolchain to build or update parts of itself, so it needs to be there" argument. You might need to ship gcc with your OS, and might even use it to compile packages the user installs (i.e. in Gentoo), but that doesn't mean the user should be aware of--or care--that compilation is going on. Nobody notices that the .NET framework ships with a compiler (ngen.exe) which is then used to locally compile all the .NET stdlib modules in the background after installation; it just happens. You don't think "I have a CLR compiler on my system" when you install the .NET framework; you only think that once you've installed MSVC, because that gives it a user-accessible UI.
The same goes for "system runtimes"--the only thing anyone is recommended to do when their OS includes a copy of Ruby (usually for the sake of executing system scripts written in Ruby, not for the user) is to install a fresh copy through RVM and avoid the "system Ruby" like a plague. If it's so distasteful to the language's programmers, why are we as OS builders even exposing it to them? Stick the binaries in sbin instead of bin; that's what it's for.)
[1] http://en.wikipedia.org/wiki/Dual_process_theory http://en.wikipedia.org/wiki/Dual_process_theory
[2] http://en.wikipedia.org/wiki/Piaget's_theory_of_cognitive_development#Formal_operational_stage http://en.wikipedia.org/wiki/Piaget's_theory_of_cognitive_de...
I'm not saying the demand for personal computer as a click-able facebook or spreadsheet machine wasn't a demand MS created. But what MS did was heavily obfuscate its platform's inherent general-purpose compute functionality by having no standard script language besides batch files (I'm unaware of if 95+ had a visual basic interpreter). They didn't provide a compiler - hell, they changed you for it - until the MSDN happened around 2000, development material was sparse and expensive (compared to man pages existing for 30 years before that).
My point is the barrier to translate from interactive brick tool -> hackable general purpose compute platform was a huge divide for 15 years because Windows tools were hard to come by, hard to find, or otherwise inaccessible.
Also, I don't buy a concern about space consumption about runtimes. Microsoft was profiting off of selling MSVCC for two decades and plenty happy with the pocket change. You can pack mono, java, python, ruby, lua, node, perl, gcc, clang, and a beefy IDE on a linux install cd and... (checks)...
on Arch, all those packages combined consume (in order) 45.2 + 15.8 + 13.9 + 3.7 + .2 + 31 + 13.6 +20.3 + 21 + 9.8 (codeblocks) = 174.5 MB. By comparison, the libreoffice package contains a combined 345MB of archives. (pacman -Si <package names> | grep "Download Size")
So every dev tool under the sun takes up half the space of the office environment on Linux, and if I wanted to I could throw both those, and kde (full package is 450MB, wallpapers add 80MB) and still use up around a gigabyte. Windows 7 iso (I have a copy) is 3.3GB. So 3 times the space, spend on what, I have no idea. It doesn't even come with .net 4.
> the barrier to translate from interactive brick tool -> hackable general purpose compute platform was a huge divide for 15 years
I'll admit: back in the 90s, without the Internet, it was harder to bootstrap a programming environment on Windows (though in 1985 batch files weren't all that bad, relatively speaking.) A lot of the time, though, you'd be learning to program from someone--a friend, or a teacher--and they'd give you a compiler on a floppy/CD. Programming books also came with compiler toolchains on disks in the jacket sleeve quite a bit of the time, if I recall (usually DJGPP.) I'm not quite sure how you'd be learning to program but stuck for the compiler part.
But I started really trying to learn to program in 2001, not 1985, and I had Internet access. By 2001, it didn't matter what Microsoft was or wasn't doing. I could download MinGW and code some C within an hour or two. You know what the hardest part of that process was? Learning C. The "barrier" of finding a C compiler on the Internet, downloading it, installing it, etc. was immaterial, even then. In fact, the biggest "barrier" was figuring out that the thing I should be looking for is called a "C compiler"!
But I didn't necessarily need a C compiler at the time, really. If I had wanted to code VB, WSH was already on the machine. If I had wanted to code Javascript, IE would have let me. And if my copy of Windows came with Office, there was always VBA, too.[1]
Today, of course, it's even easier. Ignoring the actively-didactic software like Shoes or Scratch or Squeak, you can just download Light Table and twenty seconds later be coding Clojure. Or Love2d and be making a game in Lua. Or you can code "in the cloud" in your browser without downloading anything, if you like. Windows 8 ships with Powershell installable from the "Programs and Features" menu, I believe; and there are several compiler/IDE/whatevers in the Windows App Store.
In short, you can be programming as soon as you want to be. The hardest part of programming on any OS, for the last 15 years at least, has just been wanting to--and then putting up with all that rigorous, logical thinking.
> a concern about space consumption about runtimes
I didn't say anything about space consumption.
- Every package an OS includes in the base install is a package they have to security-audit, update, and support.
- Every base package is something a user could call in with a problem regarding, and so something customer service reps have to understand.
- Every base package is part of the OS's attack surface.
- Every base package is something developers will presume to rely on to be part of the OS in the future, and so the OS maker will have to commit to supporting indefinitely for as long as they support the OS.[2]
- And every base package is something that other internal developers might build other components of the OS to rely on.
In short, there are good reasons to ship as little with your OS as you can, reasons that have nothing to do with the size of the resulting installation.[3]
---
[1] I once programmed a JRPG-style game as a PowerPoint presentation using slides for views and VBA for controllers. Not as bad as you'd think; worked a lot like Flash + ActionScript, with the same "timeline" concept, but without an actor model.
[2] If Microsoft shipped Windows with a VB4 IDE, there would be programs, even today, wanting to go in and diddle with that IDE over COM to "programmatically" generate another program. Microsoft still supports DDE in the OS in part because there were so many programs written that used it to diddle with Excel workbooks--and Excel is an optional package.
[2] Despite that, I should note that installing the latest Visual Studio takes something more like 10GB than 174MB. You don't install Visual Studio on a computer you're not going to use for Visual Studio, even if the DVD is laying right there. It's not a "freebie" to give someone when setting up their computer, like Google Earth or VLC. It's more like VMWare: it takes over your computer and puts hooks everywhere, because it needs to do things like trap segfaults and handle them itself, or test-install services nobody's codesigned yet, etc.