10 ms·

Preventing the Collapse of Civilization [video]

231 points by dmit 7y ago

jstewartmobile 7y ago
I was with him until the middle. Lack of inter-generational knowledge transfer doesn't cut it. Most of the people who rolled this stuff are still alive. And as for the whipper-snappers: people don't get very far writing programming languages/video games/operating systems without knowing their stuff. The real boogeyman is feature combinatorics. When making a tightly-integrated product (which people tend to expect these days), adding "just" one new feature (when you already have 100 of them) means touching several (if not all 100) things. Take OpenBSD for example: When you have a volunteer project by nerds for nerds, prioritizing getting it right (over having the fastest benchmark or feature-parity with X) is still manageable. Bring that into a market scenario (where buyers have a vague to non-existent understanding of what they're even buying), and we get what we get. Software companies live and die by benchmark and feature parity, and as long as it crashes and frustrates less than the other guy's product, the cash will keep coming in.
- houseinthewoods 7y ago
  heh thx for editing out the weird cynical part ;) moved from phone to computer to argue but it was too late
  - jstewartmobile 7y ago
    My bad
- microcolonel 7y ago
  I tend to agree that OpenBSD is hitting the spot a lot better, but the problem I have is that there's not enough momentum that it keeps up with hardware releases. They had been maintaining kernel drivers for AMD GPUs for a while, but it seems they stopped updating regularly. I now own no hardware from the last decade that OpenBSD can get accelerated graphics on, and I need accelerated graphics to power the displays that allow me to be productive (by showing me enough information at once that I can understand what I'm doing). I was having a conversation with somebody the other day about a privacy concern they were addressing, where a company was offering to monitor cell signals for some retail analytics purpose; and it was genuinely surprising to them that mobile phones broadcast and otherwise leak information that can be used to fingerprint the device. I think it's rather shocking the amount of ignorance people allow themselves to have when it comes to things like this. Furthermore, the way she was talking about it, it seems she thought it was the responsibility of basically anyone but the owners of these devices to consider things like this, or even ask the questions that would tell you something like this exists.
- TeMPOraL 7y ago
  > When making a tightly-integrated product (which people tend to expect these days) Do they? It was my impression that the recent evolution of user-facing software (i.e. the web, mostly) was about less integation due to reduced scope and capabilities of any single piece of software. > adding "just" one new feature (when you already have 100 of them) means touching several (if not all 100) things. This sounds true on first impression, but I'm not sure how true it really is. Consider that I could start rewriting this as "adding 'just' one new program when you already have 100 of them installed on your computer"... and it doesn't make sense anymore. A feature to a program is like a program to OS, and yet most software doesn't involve extensive use, or changes, of the operating system. The most complex and feature-packed software I've seen (e.g. 3D modelling tools, Emacs, or hell, Windows or Linux) doesn't trigger combinatorial explosion; every new feature is developed almost in isolation from all others, and yet tight integration is achieved.
  - z3t4 7y ago
    Programs can often be extended with new features using the plugin pattern.
    - pixl97 7y ago
      And plugins can include logic that reacts in completely unexpected ways with existing plugins. Testing all possible scenarios can be near impossible.
      - majkinetor 7y ago
        And this is actually more rule then exception - once you have more then 2 plugins, the chance of plugins colliding or not allowing updates of the main software in the future are more or less norm. Plugin minimization is mandatory IMO.
        z3t4 7y ago
        Plugins should not depend on other plugins. Sometimes you have to move functionality to core.
- std_throwaway 7y ago
  > Lack of inter-generational knowledge transfer doesn't cut it. Most of the people who rolled this stuff are still alive. I think he means generations in terms of the workplace/politics where you can have a generation change every few years. Meaning that most old guys go and new guys come. Technically you could ask the old guys because most are still alive but it doesn't happen for a lot of different reasons.
austincheney 7y ago
As a JavaScript developer I strongly resonate with the quote at 14:50 into the video. In summary all of the silicon industrys' chips at the time were full of defects and often the same defects between various vendors. The industry was completely aware of this. The problem is that the original generation of chips were designed by old guys who figured it out. The current generation of chips (at that time) was designed by youngsters working in the shadow of the prior generation and not asking the right questions because they were not aware of what those questions were. A decade ago JavaScript developers had little or no trouble working cross browser, writing small applications, and churning out results that work reasonably well very quickly. It isn't that cross browser compatibility had been solved, far from it, but that you simply worked to the problem directly and this was part of regular testing. That older generation did not have the benefit of helpful abstractions like jQuery or React. They had to know what the APIs were and they worked to them directly. The biggest problem with this is that there weren't many people who could do this work well. Then shortly after the helpful abstractions appeared and suddenly there was an explosion of competent enough developers, but many of these good enough developers did not and cannot work without their favorite abstractions. These abstractions impose a performance penalty, increase product size, impose additional maintenance concerns, and complicate requirements. The ability to work below the abstractions is quickly becoming lost knowledge. Many commercial websites load slower now than they did 20 years ago despite radical increases in connection speeds. To the point of the video this loss of knowledge is not static and results in degrading quality over time that is acceptable to later generations of developers who don't know the proper questions to ask.
- shams93 7y ago
  Well I came from the pre jQuery era but the whole industry is obsessed with react. There are tons of tools available web components are becoming standard, ultimately using native apis is far more performent but it's also more standard and consistent and easier than the old days to write pure vanilla js.
  - austincheney 7y ago
    I have heard people say this in justification of their framework, library, abstraction, whatever. The reality is that, from my experience, I can easily replace maybe 8 other JavaScript developers. I am not saying that out of arrogance or some uniformed guess at my radical superiority. I am saying this out of experience. It isn't because I am smart or a strong programmer. This has proven true for me, because when writing vanilla JS I am the bottleneck and the delay. I am not waiting on tools, performance delays, code complexity, or anything else. If there is bad code its because I put it there and I am at fault, and so I have nothing to blame but myself. Knowing this, and I mean as a non-cognitive emotional certainty, means I can solve the problem immediately with the shortest effort necessary or it isn't getting solved ever and that changes my priority of effort. It also means not tolerating a slow shitty product since you are fully in control (at fault). When people go through project management training I tell them there are only two kinds of problems: natural disasters and bad human decisions. When you can remove blame from the equation the distance between you and the problem/solution becomes immensely shorter.
magicbuzz 7y ago
He obviously uses Windows for his anecdotal examples, but I don’t think you can point the finger at Windows specifically. I think it’s consistent throughout OSes. I see regressions in iOS since the earlier versions, as well as in Linux and the applications I use. My IDE stopped providing menus. It’s open source so I just shrug and track the issue in Github.
earenndil 7y ago
People don't care about five 9s anymore? It's not as important as it was (I assume—I wasn't really around at the time), but cloud providers definitely advertise their number of 9s.
- throwaway2048 7y ago
  Good thing they have regular service outages that mysteriously don't quality for SLA (or show up on dashboards)
earenndil 7y ago
I challenge his assertion around 32:50 that something is lost. I've done assembly programming. C programming. I might venture to say I'm pretty good at it. I even dabbled a bit in baremetal programming, was going to make my own OS, but lost interest. Wanna know why? Take a look at this[1] article. Yep. If, on x86, you want to know what memory you're allowed to access; how much of it and where it is, there is literally no good, or standard way to do that. "Well," you (or jon blow) might say, "just use grub (or another multiboot bootloader), it'll give you the memory map." But wait, wasn't that what we were trying to avoid? If you do this, you'll say "I'm smart, I'm sparing myself the effort," but really there is a loss of capability, you don't really know where these BIOS calls are going, what the inner workings of this bootloader are, and something is lost there. This is a bit of a contrived and exaggerated example, but it serves to prove my point which is that these things really do scale linearly: you give up the same amount you get back by going up a layer of abstraction (in understanding/productivity, not talking about performance yet). Low-level programming languages aren't more productive than high-level programming languages. Low-level programmers are more productive than high-level ones because it takes more discipline to get good at low-level programming so the ones that make it in low-level programming are likely to be more skilled or, at least, to have acquired more skill. Think about the story of mel[2]. Does anyone honestly think, with any kind of conviction, that mel would have been less productive had he programmed in python and not thought about how machine instruction would be loaded? As I've mentioned, I have done, and gotten reasonably good at, low-level programming, and yet my current favourite language is perl6. A language that is about as far from the cpu as it gets, on a par with javascript or haskell. Why? Because nothing is lost. Nothing is lost, and quite a lot is gained. There are things I can do with perl6 that I cannot do with c—but, of course, the reverse is also true. And I think that jon blow's perspective is rather coloured by his profession—game development—where performance is important and it really does pay, sometimes, to think about how your variables are going in memory. He has had, I'm sure, negative interactions with proponents of dynamic languages, because he sees their arguments as (maybe that's what their arguments are, I don't know) "c is useless, javascript is good enough for everything." Maybe the people who truly think that have lost something, but I do not think that mel, or jon blow, or I, would lose much by using perl6 instead of c where perl6 is sufficient. 1: https://wiki.osdev.org/Detecting_Memory_(x86) https://wiki.osdev.org/Detecting_Memory_(x86) 2: http://www.catb.org/~esr/jargon/html/story-of-mel.html http://www.catb.org/~esr/jargon/html/story-of-mel.html
cryptica 7y ago
I think it's because companies always try to commoditize software developers but it doesn't work. You can't replace a good software developer with 10 mediocre ones plus thousands of unit tests. The only way to become a good software developer is with experience.
mike00632 7y ago
It seems like Jonathan Blow completely forgot about the blue screen of death and how common it was.
- deleted 7y ago
  [deleted]
reactspa 7y ago
Fantastic talk. Portable Apps on Windows are a hedge against some of the angst he describes. (E.g. the part about updates changing a lot of things around or causing failure-to-launch problems). E.g., I still use WinAmp to play mp3's. It's a portable version, doesn't need installation (so I can use it on my locked-down work computer). The UI hasn't changed in 20 years. Newer file-formats can be played after adding plug-ins. I've put together a whole bunch of Portable Apps, and nowadays I first try to find a portable version of an app I need before a non-portable version.
- stallmanite 7y ago
  100% concur re: portable apps. It’s a better way to live from an end-user standpoint in my experience. Portableapps.com and librekey both are excellent
kzrdude 7y ago
Now, what would be amazing would be if we had found the Antikythera mechanism so intact that it could be reconstructed perfectly. And then we'd check everything it could do, and what kinds of drawbacks or errors the construction had!
lalalandland 7y ago
The general purposeness of computers is the reason for complexity. We use the same systems to do highly secure and critical business transaction as high performance simulation, playing and fun. The conveniency of not having to switch systems when doing different tasks is adding a lot of complexity. Special purpose hardware can by its nature of not being general, be much simpler and omit a lot of the security and complexity. But it's much less convenient and much less flexible.
arketyp 7y ago
The repeal of Moore's law will be a blessing in disguise, I think. Not only will programmers need to get clever in a traditional sense but, also, a new era of specialized hardware will require a more intimate understanding of the bits and pieces. I'm optimistic.
- philwelch 7y ago
  I’m actually looking forward to the end of Moore’s Law. Instead of keeping up with a rapidly shifting landscape of new and creative ways of wasting CPU cycles, we can maybe build things that have a chance of lasting a hundred years.
  - pixl97 7y ago
    Even after Moore's we are going to have a shifting landscape of security changes in processors that has been neglected for some time.
- RachelF 7y ago
  Compared to the 1990's Moore's law stopped really making CPUs faster around 10 years ago. We now have more cores, but getting performance gains from parallelism can be hard. I think the real reason software may be more buggy is that good quality software takes too long to develop. There is a big commercial advantage in being first to market. Constant updates over the web also mean that you don't need to test so much before shipping. You can always send a patch later. This was different when software was shipped on disks.
lolc 7y ago
Wow he has a rosy picture of the past. I don't see where he gets the five nines from. He doesn't even quote anybody on it. Most of the examples he gives would have been zero nines back in the day because they were not available at all! Wikipedia for example has one nine availability in my life. Because when I sleep my phone is still on.
roenxi 7y ago
The risk is that things 'just work' for extended periods of time and the maintainers are optimised out of the system because they aren't needed in the short term. My personal guess at why civilisations can collapse so slowly (100s of years for the Romans, for example) is that the people who maintain the political systems do too good a job, and so the safeguards are forgotten. For example, after WWII the Europeans learned some really scary lessons about privacy. The Americans enjoyed greater peace and stability, so the people with privacy concerns are given less air time in places like Silicon Valley or Washington. The two-step process at work here is that when things are working, standards slip and the proper response to problems are forgotten. Then when things don't work, people don't know what to do and the system degrades.
- nabla9 7y ago
  Sean Carroll's podcast Episode 37: Edward Watts on the End of the Roman Republic and Lessons for Democracy has good discussion about this (there is also transcript) https://www.preposterousuniverse.com/podcast/2019/03/11/episode-37-edward-watts-on-the-end-of-the-roman-republic-and-lessons-for-democracy/ https://www.preposterousuniverse.com/podcast/2019/03/11/epis... Basically there are norms and unwritten understanding, deeply understood ideas about what is not acceptable. Rulers don't push their power to full extent. Then someone comes along and starts to push them and gradually what is acceptable changes.
js8 7y ago
It seems to me that there is a cultural problem that deep expertise is not valued, because it is difficult to understand and to get somebody "flexible" is easier. I was just at a workshop about https://en.wikipedia.org/wiki/Design_thinking https://en.wikipedia.org/wiki/Design_thinking. The whole premise was that you don't actually need to hire an (expensive, inflexible) expert, who understands how something is done, but rather what you need to do is to "observe" an expert. But imagine what happens when everybody does that! Everybody gets rid of their experts, assuming that the client (who they are supposed to provide the service for) has the actual expertise. And they are assuming the same about their clients and so on. The end result is complete disregard for expertise. So expertise is a positive externality, in an economic sense. Nobody is incentivized to keep it more than neccessary. This leads to losses over time.
- pixl97 7y ago
  This is very common in industry with boom/bust cycles. Lots of experts in the boom, they leave when the bust comes, then when the next boom comes there are lots of problems expanding said processes quickly because of lack of expertise.
jmiskovic 7y ago
Great talk. I agree that sw is on the decline. You can see it in your OS, on the web, everywhere. Robust products are replaced with 'modern' crappy redesigns. We are surprised if the thing still works after 5 years. I don't agree on his conclusions. The real source of problem is that now we have maybe x100000 more software than we had in 70s. It's that many more programmers, so not just the 1% smartest greybeards as before. We need more abstractions, and yes, they will run slower and have their issues. Also, not everybody is sitting at the top of hierarchy of abstractions. Some roll up their sleeves and work on JIT runtimes and breakthrough DB algorithms. All those blocks of software need to communicate with the platform and between them. IMO the way out is open source. Open platforms, open standards, open policies. Every time I found a good piece of code in company's huge codebase, it was open source library. Every time. You have to open up to external world to produce well engineered piece of software. The lack of financial models for open source is the obstacle. We should work on making simple and robust software profitable.
- imiric 7y ago
  > The real source of problem is that now we have maybe x100000 more software than we had in 70s. It's that many more programmers, so not just the 1% smartest greybeards as before. The greybeards from the 70s weren't much smarter than today's programmers. They were the same curious hackers from today, with the advantage of being born in the right place at the right time, when the technology was still developing, so they were forced to build their own tools and operating systems. > We need more abstractions, and yes, they will run slower and have their issues. I disagree, and side with Jon Blow on this: abstractions (if done well) create the illusion of simplicity and more often that not hide the apparent complexity of lower levels. Sometimes this complexity is indeed too difficult to work with, but often it's the problem itself that needs to be simplified instead of creating an abstraction layer on top. I think as an industry we've failed to make meaningful abstractions while educating new programmers on the lower level functionality. A lot of today's programmers learned on Python, PHP, Ruby, JavaScript, etc., which are incredibly complex tools by themselves. And only a minority of those will end up going back and really learning the fundamentals in the same way hackers in the 60s and 70s did. > IMO the way out is open source. Open platforms, open standards, open policies. Agreed. But education and simplification are also crucial.
soup10 7y ago
bravo, great talk Jon. but real talk can you make another Braid already
glandium 7y ago
He briefly mentions the Boeing 737 MAX issue, but he understates the problem. Sure there was a software problem, but the urderlying issue was the whole notion that everything can be "fixed" (worked around) by software. That it's fine to make changes to the plane aerodynamics and compensate with software so that it would seemingly act like the previous model.
- paulkon 7y ago
  I'm reminded of this explanation that was posted on HN couple months back https://twitter.com/trevorsumner/status/1106934362531155974 https://twitter.com/trevorsumner/status/1106934362531155974 Seems like it was a physical engineering and business problem and software was added to (inadequately) compensate.
Illniyar 7y ago
At what point in the past were our programs stable, robust and just worked? Perhaps it was before my time, but DOS and windows (3.11, 95) would crash, constantly. Blue screens of death, infinite loops that would just freeze my computer, memory leaks that would cause my computer to stop working after a day. I now expect my computer to stay on for months without issues. I expect to be able to put it to sleep and open it in the same state it was. I expect that if a website or a program errors, my OS simply shrugs and closes it. I expect my OS to be robust enough so that if I enter a usb or download a file I'm not playing Russian roulette that it might contain a virus that would destroy my computer. In the past I would close my computer at the end of every day because otherwise it will simply crash some time in the night. I would run de fragmentation at least once a month. Memory errors and disk errors were common, and the OS had no idea how to overcome it. Crashes were so common, you just shrugged and learned to save often.
- sdfjkl 7y ago
  > At what point in the past were our programs stable, robust and just worked? DOS was rock solid, at least around the era of DR-DOS. DESQView 386 was absolutely stable too. The BBS software I ran on them in those days was a wobbly piece of shit though. I also recall Borland's Turbo Pascal compiler and the accompanying text-mode IDE being ultra reliable. After DOS I've used OS/2 which was also extremely stable, although suffered from limited hard- and software availability. Mac OS X used to be rock solid too in the heydays of the Powerbooks and earlier Intel Macbooks. Every now and then there were hardware design flaws though, and now the quality of both soft- and hardware seems to have taken a tragic turn for the worse. You still do play Russian roulette whenever you plug a USB device into your computer, see "USB Rubber Ducky".
  - messe 7y ago
    > DOS was rock solid DOS was rock solid because it did nothing. Many programs, particularly games and anything that did networking, didn't even use DOS interfaces—they bypassed them entirely and worked with either the BIOS or hardware directly. There was no memory protection, no multitasking, and, on a higher level, no permissions nor sandboxing. So while maybe it "just worked", I wouldn't call it robust.
cheschire 7y ago
This whole presentation feels like a great candidate for a meta analysis on the effects of recency bias on analysis.
std_throwaway 7y ago
At my workplace when I ask about why some process parameters are this way it usually leads to a dead end where the people who know are long gone and those who should know don't know the essentials. Everything is kind of interconnected and errors show up months later so you can't really change anything on any machine until the machine as a whole breaks and needs to be replaced. Then you try to get it to work somehow and those parameters are then set forever.
new4thaccount 7y ago
Reducing all this complexity is partly why I'm hoping the Red Language project can succeed where Rebol failed. Of course you can't do everything, but a good full-stack language could cover perhaps 80% of software needs using well written DSLs. The simple fact that we have so many languages targeting the same thing is a waste and duplicative effort (Java, C#, Kotlin, Scala, Clojure, F#...etc) for business apps and (Python, Matlab, Julia, R, and Fortran) for data science and scientific programming. Also systems languages like (C, C++, Ada, Rust). On one side it is good to have purpose built languages, but on another it puts a big barrier to entry. Note that I'm advocating for abstractions, but far fewer languages. Yes, abstractions add complexity, but actually make the code more readable. I shudder to think of humanity having to maintain and support ever increasing levels of software.
- d_burfoot 7y ago
  I absolutely agree that we actually need fewer languages. The languages we have today really are good enough for the vast majority of programming work. To the extent that they fall short, the solution is to either improve the language or to build good libraries for it.
  - benc666 7y ago
    Many programming languages start out as 'experiments' by their creators, to combine or extend the capabilities and attributes of some prior ones. That seems like a very healthy evolutionary approach to me.
fallingfrog 7y ago
Not to pick on Microsoft specifically too much, but I remember seeing the hello world program for windows 3.1 for the first time and thinking, “this is not looking good.” And I was right.
d_burfoot 7y ago
I loved this talk, and in particular the point about programmers being forced to learn trivia instead of deep knowledge. I just started a new job at a big tech company, and I've spent a whole week so far trying to figure out how to use the build tool. The frustrating part is that most of the software modules my team is working on aren't very complicated. The complexity comes from pulling in all sorts of 3rd party libraries and managing their transitive dependencies.
alexashka 7y ago
There is no preventing a collapse. Try and enjoy the ride :)
PorterDuff 7y ago
I had a few cocktails and thought about a few points made in the video. It seems to me that there are two different notions here that are being conflated: . A rotting of knowledge over time. . A variant of Moore's Law. In this case, the idea that the value of technology, in a particular area, has a decreasing value on the margin. It's kind of like the notions you see in cliodynamics, that there are a few interacting sine waves (or some other function) in mass human behavior. I suppose that the main concept of importance is how it all might mess with your own personal situation. Personally, I think that the West is in decline, but that doesn't have a whole lot to do with the quality of software on internet websites.
potrarch 7y ago
There is a relationship between quantity of functionality and bugginess. Even with the most demanding testing, bugs will remain. The question is, as more and more software permeates our lives, will the accumulation of unfixable bugs ultimately overwhelm us. Can we build an AI that can clean enough of the bugs out of all of our software, including it's own, for our civilization to survive?
adverbly 7y ago
I remember seeing the foundationDB distributed systems testing video for the first time, and being blown away with what it takes to build robust software. Worth a view if you haven't seen it. https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc Would love to see more things in this direction, but, I agree that the market doesn't want it. Most users will gladly accept an infrequent bug for an earlier release, or lower cost version of a product.

231 points by dmit 7y ago

I was with him until the middle. Lack of inter-generational knowledge transfer doesn't cut it. Most of the people who rolled this stuff are still alive. And as for the whipper-snappers: people don't get very far writing programming languages/video games/operating systems without knowing their stuff. The real boogeyman is feature combinatorics. When making a tightly-integrated product (which people tend to expect these days), adding "just" one new feature (when you already have 100 of them) means touching several (if not all 100) things. Take OpenBSD for example: When you have a volunteer project by nerds for nerds, prioritizing getting it right (over having the fastest benchmark or feature-parity with X) is still manageable. Bring that into a market scenario (where buyers have a vague to non-existent understanding of what they're even buying), and we get what we get. Software companies live and die by benchmark and feature parity, and as long as it crashes and frustrates less than the other guy's product, the cash will keep coming in.

houseinthewoods 7y ago
heh thx for editing out the weird cynical part ;) moved from phone to computer to argue but it was too late
- jstewartmobile 7y ago
  My bad
microcolonel 7y ago
I tend to agree that OpenBSD is hitting the spot a lot better, but the problem I have is that there's not enough momentum that it keeps up with hardware releases. They had been maintaining kernel drivers for AMD GPUs for a while, but it seems they stopped updating regularly. I now own no hardware from the last decade that OpenBSD can get accelerated graphics on, and I need accelerated graphics to power the displays that allow me to be productive (by showing me enough information at once that I can understand what I'm doing). I was having a conversation with somebody the other day about a privacy concern they were addressing, where a company was offering to monitor cell signals for some retail analytics purpose; and it was genuinely surprising to them that mobile phones broadcast and otherwise leak information that can be used to fingerprint the device. I think it's rather shocking the amount of ignorance people allow themselves to have when it comes to things like this. Furthermore, the way she was talking about it, it seems she thought it was the responsibility of basically anyone but the owners of these devices to consider things like this, or even ask the questions that would tell you something like this exists.
TeMPOraL 7y ago
> When making a tightly-integrated product (which people tend to expect these days) Do they? It was my impression that the recent evolution of user-facing software (i.e. the web, mostly) was about less integation due to reduced scope and capabilities of any single piece of software. > adding "just" one new feature (when you already have 100 of them) means touching several (if not all 100) things. This sounds true on first impression, but I'm not sure how true it really is. Consider that I could start rewriting this as "adding 'just' one new program when you already have 100 of them installed on your computer"... and it doesn't make sense anymore. A feature to a program is like a program to OS, and yet most software doesn't involve extensive use, or changes, of the operating system. The most complex and feature-packed software I've seen (e.g. 3D modelling tools, Emacs, or hell, Windows or Linux) doesn't trigger combinatorial explosion; every new feature is developed almost in isolation from all others, and yet tight integration is achieved.
- z3t4 7y ago
  Programs can often be extended with new features using the plugin pattern.
  - pixl97 7y ago
    And plugins can include logic that reacts in completely unexpected ways with existing plugins. Testing all possible scenarios can be near impossible.
    - majkinetor 7y ago
      And this is actually more rule then exception - once you have more then 2 plugins, the chance of plugins colliding or not allowing updates of the main software in the future are more or less norm. Plugin minimization is mandatory IMO.
      - z3t4 7y ago
        Plugins should not depend on other plugins. Sometimes you have to move functionality to core.
std_throwaway 7y ago
> Lack of inter-generational knowledge transfer doesn't cut it. Most of the people who rolled this stuff are still alive. I think he means generations in terms of the workplace/politics where you can have a generation change every few years. Meaning that most old guys go and new guys come. Technically you could ask the old guys because most are still alive but it doesn't happen for a lot of different reasons.

austincheney 7y ago

As a JavaScript developer I strongly resonate with the quote at 14:50 into the video. In summary all of the silicon industrys' chips at the time were full of defects and often the same defects between various vendors. The industry was completely aware of this. The problem is that the original generation of chips were designed by old guys who figured it out. The current generation of chips (at that time) was designed by youngsters working in the shadow of the prior generation and not asking the right questions because they were not aware of what those questions were. A decade ago JavaScript developers had little or no trouble working cross browser, writing small applications, and churning out results that work reasonably well very quickly. It isn't that cross browser compatibility had been solved, far from it, but that you simply worked to the problem directly and this was part of regular testing. That older generation did not have the benefit of helpful abstractions like jQuery or React. They had to know what the APIs were and they worked to them directly. The biggest problem with this is that there weren't many people who could do this work well. Then shortly after the helpful abstractions appeared and suddenly there was an explosion of competent enough developers, but many of these good enough developers did not and cannot work without their favorite abstractions. These abstractions impose a performance penalty, increase product size, impose additional maintenance concerns, and complicate requirements. The ability to work below the abstractions is quickly becoming lost knowledge. Many commercial websites load slower now than they did 20 years ago despite radical increases in connection speeds. To the point of the video this loss of knowledge is not static and results in degrading quality over time that is acceptable to later generations of developers who don't know the proper questions to ask.

shams93 7y ago
Well I came from the pre jQuery era but the whole industry is obsessed with react. There are tons of tools available web components are becoming standard, ultimately using native apis is far more performent but it's also more standard and consistent and easier than the old days to write pure vanilla js.
- austincheney 7y ago
  I have heard people say this in justification of their framework, library, abstraction, whatever. The reality is that, from my experience, I can easily replace maybe 8 other JavaScript developers. I am not saying that out of arrogance or some uniformed guess at my radical superiority. I am saying this out of experience. It isn't because I am smart or a strong programmer. This has proven true for me, because when writing vanilla JS I am the bottleneck and the delay. I am not waiting on tools, performance delays, code complexity, or anything else. If there is bad code its because I put it there and I am at fault, and so I have nothing to blame but myself. Knowing this, and I mean as a non-cognitive emotional certainty, means I can solve the problem immediately with the shortest effort necessary or it isn't getting solved ever and that changes my priority of effort. It also means not tolerating a slow shitty product since you are fully in control (at fault). When people go through project management training I tell them there are only two kinds of problems: natural disasters and bad human decisions. When you can remove blame from the equation the distance between you and the problem/solution becomes immensely shorter.

magicbuzz 7y ago

He obviously uses Windows for his anecdotal examples, but I don’t think you can point the finger at Windows specifically. I think it’s consistent throughout OSes. I see regressions in iOS since the earlier versions, as well as in Linux and the applications I use. My IDE stopped providing menus. It’s open source so I just shrug and track the issue in Github.

earenndil 7y ago

People don't care about five 9s anymore? It's not as important as it was (I assume—I wasn't really around at the time), but cloud providers definitely advertise their number of 9s.

throwaway2048 7y ago
Good thing they have regular service outages that mysteriously don't quality for SLA (or show up on dashboards)

earenndil 7y ago

I challenge his assertion around 32:50 that something is lost. I've done assembly programming. C programming. I might venture to say I'm pretty good at it. I even dabbled a bit in baremetal programming, was going to make my own OS, but lost interest. Wanna know why? Take a look at this[1] article. Yep. If, on x86, you want to know what memory you're allowed to access; how much of it and where it is, there is literally no good, or standard way to do that. "Well," you (or jon blow) might say, "just use grub (or another multiboot bootloader), it'll give you the memory map." But wait, wasn't that what we were trying to avoid? If you do this, you'll say "I'm smart, I'm sparing myself the effort," but really there is a loss of capability, you don't really know where these BIOS calls are going, what the inner workings of this bootloader are, and something is lost there. This is a bit of a contrived and exaggerated example, but it serves to prove my point which is that these things really do scale linearly: you give up the same amount you get back by going up a layer of abstraction (in understanding/productivity, not talking about performance yet). Low-level programming languages aren't more productive than high-level programming languages. Low-level programmers are more productive than high-level ones because it takes more discipline to get good at low-level programming so the ones that make it in low-level programming are likely to be more skilled or, at least, to have acquired more skill. Think about the story of mel[2]. Does anyone honestly think, with any kind of conviction, that mel would have been less productive had he programmed in python and not thought about how machine instruction would be loaded? As I've mentioned, I have done, and gotten reasonably good at, low-level programming, and yet my current favourite language is perl6. A language that is about as far from the cpu as it gets, on a par with javascript or haskell. Why? Because nothing is lost. Nothing is lost, and quite a lot is gained. There are things I can do with perl6 that I cannot do with c—but, of course, the reverse is also true. And I think that jon blow's perspective is rather coloured by his profession—game development—where performance is important and it really does pay, sometimes, to think about how your variables are going in memory. He has had, I'm sure, negative interactions with proponents of dynamic languages, because he sees their arguments as (maybe that's what their arguments are, I don't know) "c is useless, javascript is good enough for everything." Maybe the people who truly think that have lost something, but I do not think that mel, or jon blow, or I, would lose much by using perl6 instead of c where perl6 is sufficient. 1: https://wiki.osdev.org/Detecting_Memory_(x86) https://wiki.osdev.org/Detecting_Memory_(x86) 2: http://www.catb.org/~esr/jargon/html/story-of-mel.html http://www.catb.org/~esr/jargon/html/story-of-mel.html

cryptica 7y ago

I think it's because companies always try to commoditize software developers but it doesn't work. You can't replace a good software developer with 10 mediocre ones plus thousands of unit tests. The only way to become a good software developer is with experience.

mike00632 7y ago

It seems like Jonathan Blow completely forgot about the blue screen of death and how common it was.

deleted 7y ago
[deleted]

reactspa 7y ago

Fantastic talk. Portable Apps on Windows are a hedge against some of the angst he describes. (E.g. the part about updates changing a lot of things around or causing failure-to-launch problems). E.g., I still use WinAmp to play mp3's. It's a portable version, doesn't need installation (so I can use it on my locked-down work computer). The UI hasn't changed in 20 years. Newer file-formats can be played after adding plug-ins. I've put together a whole bunch of Portable Apps, and nowadays I first try to find a portable version of an app I need before a non-portable version.

stallmanite 7y ago
100% concur re: portable apps. It’s a better way to live from an end-user standpoint in my experience. Portableapps.com and librekey both are excellent

kzrdude 7y ago

Now, what would be amazing would be if we had found the Antikythera mechanism so intact that it could be reconstructed perfectly. And then we'd check everything it could do, and what kinds of drawbacks or errors the construction had!

lalalandland 7y ago

The general purposeness of computers is the reason for complexity. We use the same systems to do highly secure and critical business transaction as high performance simulation, playing and fun. The conveniency of not having to switch systems when doing different tasks is adding a lot of complexity. Special purpose hardware can by its nature of not being general, be much simpler and omit a lot of the security and complexity. But it's much less convenient and much less flexible.

arketyp 7y ago

The repeal of Moore's law will be a blessing in disguise, I think. Not only will programmers need to get clever in a traditional sense but, also, a new era of specialized hardware will require a more intimate understanding of the bits and pieces. I'm optimistic.

philwelch 7y ago
I’m actually looking forward to the end of Moore’s Law. Instead of keeping up with a rapidly shifting landscape of new and creative ways of wasting CPU cycles, we can maybe build things that have a chance of lasting a hundred years.
- pixl97 7y ago
  Even after Moore's we are going to have a shifting landscape of security changes in processors that has been neglected for some time.
RachelF 7y ago
Compared to the 1990's Moore's law stopped really making CPUs faster around 10 years ago. We now have more cores, but getting performance gains from parallelism can be hard. I think the real reason software may be more buggy is that good quality software takes too long to develop. There is a big commercial advantage in being first to market. Constant updates over the web also mean that you don't need to test so much before shipping. You can always send a patch later. This was different when software was shipped on disks.

lolc 7y ago

Wow he has a rosy picture of the past. I don't see where he gets the five nines from. He doesn't even quote anybody on it. Most of the examples he gives would have been zero nines back in the day because they were not available at all! Wikipedia for example has one nine availability in my life. Because when I sleep my phone is still on.

roenxi 7y ago

The risk is that things 'just work' for extended periods of time and the maintainers are optimised out of the system because they aren't needed in the short term. My personal guess at why civilisations can collapse so slowly (100s of years for the Romans, for example) is that the people who maintain the political systems do too good a job, and so the safeguards are forgotten. For example, after WWII the Europeans learned some really scary lessons about privacy. The Americans enjoyed greater peace and stability, so the people with privacy concerns are given less air time in places like Silicon Valley or Washington. The two-step process at work here is that when things are working, standards slip and the proper response to problems are forgotten. Then when things don't work, people don't know what to do and the system degrades.

nabla9 7y ago
Sean Carroll's podcast Episode 37: Edward Watts on the End of the Roman Republic and Lessons for Democracy has good discussion about this (there is also transcript) https://www.preposterousuniverse.com/podcast/2019/03/11/episode-37-edward-watts-on-the-end-of-the-roman-republic-and-lessons-for-democracy/ https://www.preposterousuniverse.com/podcast/2019/03/11/epis... Basically there are norms and unwritten understanding, deeply understood ideas about what is not acceptable. Rulers don't push their power to full extent. Then someone comes along and starts to push them and gradually what is acceptable changes.

js8 7y ago

It seems to me that there is a cultural problem that deep expertise is not valued, because it is difficult to understand and to get somebody "flexible" is easier. I was just at a workshop about https://en.wikipedia.org/wiki/Design_thinking https://en.wikipedia.org/wiki/Design_thinking. The whole premise was that you don't actually need to hire an (expensive, inflexible) expert, who understands how something is done, but rather what you need to do is to "observe" an expert. But imagine what happens when everybody does that! Everybody gets rid of their experts, assuming that the client (who they are supposed to provide the service for) has the actual expertise. And they are assuming the same about their clients and so on. The end result is complete disregard for expertise. So expertise is a positive externality, in an economic sense. Nobody is incentivized to keep it more than neccessary. This leads to losses over time.

pixl97 7y ago
This is very common in industry with boom/bust cycles. Lots of experts in the boom, they leave when the bust comes, then when the next boom comes there are lots of problems expanding said processes quickly because of lack of expertise.

jmiskovic 7y ago

Great talk. I agree that sw is on the decline. You can see it in your OS, on the web, everywhere. Robust products are replaced with 'modern' crappy redesigns. We are surprised if the thing still works after 5 years. I don't agree on his conclusions. The real source of problem is that now we have maybe x100000 more software than we had in 70s. It's that many more programmers, so not just the 1% smartest greybeards as before. We need more abstractions, and yes, they will run slower and have their issues. Also, not everybody is sitting at the top of hierarchy of abstractions. Some roll up their sleeves and work on JIT runtimes and breakthrough DB algorithms. All those blocks of software need to communicate with the platform and between them. IMO the way out is open source. Open platforms, open standards, open policies. Every time I found a good piece of code in company's huge codebase, it was open source library. Every time. You have to open up to external world to produce well engineered piece of software. The lack of financial models for open source is the obstacle. We should work on making simple and robust software profitable.

imiric 7y ago
> The real source of problem is that now we have maybe x100000 more software than we had in 70s. It's that many more programmers, so not just the 1% smartest greybeards as before. The greybeards from the 70s weren't much smarter than today's programmers. They were the same curious hackers from today, with the advantage of being born in the right place at the right time, when the technology was still developing, so they were forced to build their own tools and operating systems. > We need more abstractions, and yes, they will run slower and have their issues. I disagree, and side with Jon Blow on this: abstractions (if done well) create the illusion of simplicity and more often that not hide the apparent complexity of lower levels. Sometimes this complexity is indeed too difficult to work with, but often it's the problem itself that needs to be simplified instead of creating an abstraction layer on top. I think as an industry we've failed to make meaningful abstractions while educating new programmers on the lower level functionality. A lot of today's programmers learned on Python, PHP, Ruby, JavaScript, etc., which are incredibly complex tools by themselves. And only a minority of those will end up going back and really learning the fundamentals in the same way hackers in the 60s and 70s did. > IMO the way out is open source. Open platforms, open standards, open policies. Agreed. But education and simplification are also crucial.

soup10 7y ago

bravo, great talk Jon. but real talk can you make another Braid already

glandium 7y ago

He briefly mentions the Boeing 737 MAX issue, but he understates the problem. Sure there was a software problem, but the urderlying issue was the whole notion that everything can be "fixed" (worked around) by software. That it's fine to make changes to the plane aerodynamics and compensate with software so that it would seemingly act like the previous model.

paulkon 7y ago
I'm reminded of this explanation that was posted on HN couple months back https://twitter.com/trevorsumner/status/1106934362531155974 https://twitter.com/trevorsumner/status/1106934362531155974 Seems like it was a physical engineering and business problem and software was added to (inadequately) compensate.

Illniyar 7y ago

At what point in the past were our programs stable, robust and just worked? Perhaps it was before my time, but DOS and windows (3.11, 95) would crash, constantly. Blue screens of death, infinite loops that would just freeze my computer, memory leaks that would cause my computer to stop working after a day. I now expect my computer to stay on for months without issues. I expect to be able to put it to sleep and open it in the same state it was. I expect that if a website or a program errors, my OS simply shrugs and closes it. I expect my OS to be robust enough so that if I enter a usb or download a file I'm not playing Russian roulette that it might contain a virus that would destroy my computer. In the past I would close my computer at the end of every day because otherwise it will simply crash some time in the night. I would run de fragmentation at least once a month. Memory errors and disk errors were common, and the OS had no idea how to overcome it. Crashes were so common, you just shrugged and learned to save often.

sdfjkl 7y ago
> At what point in the past were our programs stable, robust and just worked? DOS was rock solid, at least around the era of DR-DOS. DESQView 386 was absolutely stable too. The BBS software I ran on them in those days was a wobbly piece of shit though. I also recall Borland's Turbo Pascal compiler and the accompanying text-mode IDE being ultra reliable. After DOS I've used OS/2 which was also extremely stable, although suffered from limited hard- and software availability. Mac OS X used to be rock solid too in the heydays of the Powerbooks and earlier Intel Macbooks. Every now and then there were hardware design flaws though, and now the quality of both soft- and hardware seems to have taken a tragic turn for the worse. You still do play Russian roulette whenever you plug a USB device into your computer, see "USB Rubber Ducky".
- messe 7y ago
  > DOS was rock solid DOS was rock solid because it did nothing. Many programs, particularly games and anything that did networking, didn't even use DOS interfaces—they bypassed them entirely and worked with either the BIOS or hardware directly. There was no memory protection, no multitasking, and, on a higher level, no permissions nor sandboxing. So while maybe it "just worked", I wouldn't call it robust.

cheschire 7y ago

This whole presentation feels like a great candidate for a meta analysis on the effects of recency bias on analysis.

std_throwaway 7y ago

At my workplace when I ask about why some process parameters are this way it usually leads to a dead end where the people who know are long gone and those who should know don't know the essentials. Everything is kind of interconnected and errors show up months later so you can't really change anything on any machine until the machine as a whole breaks and needs to be replaced. Then you try to get it to work somehow and those parameters are then set forever.

new4thaccount 7y ago

Reducing all this complexity is partly why I'm hoping the Red Language project can succeed where Rebol failed. Of course you can't do everything, but a good full-stack language could cover perhaps 80% of software needs using well written DSLs. The simple fact that we have so many languages targeting the same thing is a waste and duplicative effort (Java, C#, Kotlin, Scala, Clojure, F#...etc) for business apps and (Python, Matlab, Julia, R, and Fortran) for data science and scientific programming. Also systems languages like (C, C++, Ada, Rust). On one side it is good to have purpose built languages, but on another it puts a big barrier to entry. Note that I'm advocating for abstractions, but far fewer languages. Yes, abstractions add complexity, but actually make the code more readable. I shudder to think of humanity having to maintain and support ever increasing levels of software.

d_burfoot 7y ago
I absolutely agree that we actually need fewer languages. The languages we have today really are good enough for the vast majority of programming work. To the extent that they fall short, the solution is to either improve the language or to build good libraries for it.
- benc666 7y ago
  Many programming languages start out as 'experiments' by their creators, to combine or extend the capabilities and attributes of some prior ones. That seems like a very healthy evolutionary approach to me.

fallingfrog 7y ago

Not to pick on Microsoft specifically too much, but I remember seeing the hello world program for windows 3.1 for the first time and thinking, “this is not looking good.” And I was right.

d_burfoot 7y ago

I loved this talk, and in particular the point about programmers being forced to learn trivia instead of deep knowledge. I just started a new job at a big tech company, and I've spent a whole week so far trying to figure out how to use the build tool. The frustrating part is that most of the software modules my team is working on aren't very complicated. The complexity comes from pulling in all sorts of 3rd party libraries and managing their transitive dependencies.

alexashka 7y ago

There is no preventing a collapse. Try and enjoy the ride :)

PorterDuff 7y ago

I had a few cocktails and thought about a few points made in the video. It seems to me that there are two different notions here that are being conflated: . A rotting of knowledge over time. . A variant of Moore's Law. In this case, the idea that the value of technology, in a particular area, has a decreasing value on the margin. It's kind of like the notions you see in cliodynamics, that there are a few interacting sine waves (or some other function) in mass human behavior. I suppose that the main concept of importance is how it all might mess with your own personal situation. Personally, I think that the West is in decline, but that doesn't have a whole lot to do with the quality of software on internet websites.

potrarch 7y ago

There is a relationship between quantity of functionality and bugginess. Even with the most demanding testing, bugs will remain. The question is, as more and more software permeates our lives, will the accumulation of unfixable bugs ultimately overwhelm us. Can we build an AI that can clean enough of the bugs out of all of our software, including it's own, for our civilization to survive?

adverbly 7y ago

I remember seeing the foundationDB distributed systems testing video for the first time, and being blown away with what it takes to build robust software. Worth a view if you haven't seen it. https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc Would love to see more things in this direction, but, I agree that the market doesn't want it. Most users will gladly accept an infrequent bug for an earlier release, or lower cost version of a product.