6 ms·

The problem is that both hardware and software are garbage. Spectre/Meldown & friends are just the tip of an iceberg. We have layers & layers of indirection/ab

The problem is that both hardware and software are garbage. Spectre/Meldown & friends are just the tip of an iceberg. We have layers & layers of indirection/abstraction everywhere. We have hardware that lies and tells you that it has certain properties when in reality it doesn't (example: sector sizes in hard drives/NVMs, processors still pretending that they behave like PDP-11), we have hardware that is flat out broken. We try to fix those issues in software. But in the software, we have another dump of workarounds, dependencies, abstractions with a sprinkle of backward compatibility. We are now creating "minimalist" applications with a fraction of functionality of the software from 30 years ago but using so many layers that total amount of code used to make it work is many orders of magnitude larger than what we had back then. I know that most of the programmers did not work with systems where it's very, very easy to debug the whole stack and you can learn it in a short period but it's amazing when you have knowledge about EVERY part of the system in your head. There are some good things going on (like strive to replace C with something which has similar performance characteristics but without its flaws) but it's not enough. Here are two things worth watching: https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk - Jonathan Blow - Preventing the Collapse of Civilization https://www.youtube.com/watch?v=t9MjGziRw-c https://www.youtube.com/watch?v=t9MjGziRw-c - Computers Barely Work - Interview with Linux Legend Greg Kroah-Hartman

scroot 6y ago
Simplicity and good design take lots of time and money. Our culture is not truly ready to make these kinds of investments in the manner required. Why would they? There is a whole universe of FOSS out there upon which anyone can cheaply create "working" software. If your goals are short term (quarterly earnings, looking only a year or two down the road) this is "good enough." Worse, that FOSS foundation is typically filled to the brim with complexity. We have created a computing culture that is premised on pushing the extremes of the teletype model of computing, and tacking what customers think they want on top of it. We have good alternate examples from the past (Oberon, Lisp machines, Hypercard, Smalltalk systems, etc). How often does the new generation of computing people get exposed to these ideas?
- de_watcher 6y ago
  No, FOSS/proprietary is orthogonal to that. You can find simple and performant software. The problem is in something else.
- cjfd 6y ago
  I don't think FOSS is the problem either. I think much complexity is required because everything is expected to live on the web and also/thereby is expected to be client/server. As soon as one wants to do more than just display a document on the web one discovers that its architecture is not very suitable for doing anything else besides displaying a document. A badly formatted document at that..... Also the expectation of client-server communication is a big driver of complexity. As soon as one has that we have network communication, serialization and so on. I.e., stuff that is on the large side of things to write oneself. Of course, with the web one more or less has a client-server architecture by default.
- zozbot234 6y ago
  > Worse, that FOSS foundation is typically filled to the brim with complexity. Really? In my experience, FOSS tends to be a lot simpler and more streamlined than non-free software with comparable functionality.
  - na85 6y ago
    I agree with 'de_watcher that FOSS/encumbered is an orthogonal axis to complexity/simplicity. Lots of FOSS software is excessively complex (the systemd ecosystem of shovelware comes immediately to mind) and lots of FOSS is simple. Similarly there are untold thousands of overcomplicated/overengineered proprietary suites and of course it's hard for a graphical application to get simpler than notepad.exe.
- asdfman123 6y ago
  And what software developers consider "simplicity and good design" often comes across to other people as "I have no idea what to do and looking at this literally gives me anxiety to the point that I want to avoid it."
darepublic 6y ago
Well is the solution more tech (like improving on C language) or is it making sure we have robust non technical fallbacks as a civilization
trimbo 6y ago
> The problem is that both hardware and software are garbage. I think it's incredible that, in my lifetime, computers went from giant mainframes with dedicated hard-line terminals to always-connected supercomputers in everyone's pocket, worldwide. Furthermore, anyone can use the internet to learn how to program. Maybe that's garbage compared to some mythical ideal but in terms of impact on the world it's incredible. > I know that most of the programmers did not work with systems where it's very, very easy to debug the whole stack and you can learn it in a short period but it's amazing when you have knowledge about EVERY part of the system in your head. Well, you can tell from the above that I was around then. I started programming with a single manual and the Beagle Brothers "Peeks, Pokes and Pointers" cheatsheet[1]. People forget that the software itself had to do much less than it does today. Here's just one angle to judge: security. We did not have a worldwide interconnected network with so many people trying to steal data from it. We all used rsh and packets were flying around in cleartext, no problem. But today, all software will have to incorporate TLS. And far fewer people built that simpler software. EA's first titles were written by one or two people. Now a typical EA title has hundreds of people working on it. Things will get better than where they are today. In the future, the industry will have invest more money in "10X" productivity and reliability improvements. Eventually, I think that will happen as productivity continues to slow on large codebases. [1] - https://downloads.reactivemicro.com/Apple%20II%20Items/Documentation/Books/Beagle%20Brothers%20-%20Peeks,%20Pokes%20and%20Pointers.png https://downloads.reactivemicro.com/Apple%20II%20Items/Docum...
- Nextgrid 6y ago
  And yet, even doing something as basic as a smart doorbell requires a backend server somewhere (essentially a mainframe) and all kinds of NAT hole-punching, UPnP and proprietary push notifications despite the supercomputer in your pocket technically being able to listen for incoming TCP or UDP packets from the doorbell directly. The "supercomputer" processing power is also being wasted on all kinds of malicious and defective-by-design endeavors such as ads, analytics, etc (install any mainstream app and look at the network traffic, 90% of it will be for analytics and can be blocked with no ill effects). Despite the supercomputers being 10x faster than the early ones (back in the iPhone 3G days) we somehow lost the ability to render a 60fps non-stuttering UI despite modern UIs being less rich and consisting mostly of whitespace. > anyone can use the internet to learn how to program I think there used to be a "golden age" of this where the resources were all available for free and at the same time the stacks were manageable (think basic PHP deployed on shared hosting or Ruby or Django) where as nowadays it is considered "wrong" if you don't use Kubernetes, 10 microservices and 100+ NPM packages just to serve a "Hello world" page to a browser.
matheusmoreira 6y ago
> we have hardware that is flat out broken Reading Linux driver code is very informative. Sometimes hardware just doesn't do what is expected and the driver must try and fix it up so that user space can have a reasonable interface. A simple example: /* The ITE8595 always reports 0 as value for the rfkill button. Luckily * it is the only button in its report, and it sends a report on * release only, so receiving a report means the button was pressed. */ if (usage->hid == HID_GD_RFKILL_BTN) { input_event(input, EV_KEY, KEY_RFKILL, 1); input_sync(input); input_event(input, EV_KEY, KEY_RFKILL, 0); input_sync(input); return 1; }
- Dunedan 6y ago
  It's even more interesting when you can infer why its broken. Let's take the NVMe controller in the MacBook Pro 2016 and later for example: That controller is not properly detected by Linux and needs a quirk [1] to be identified by its PCI device id instead. Why is that? Well, Linux usually detects NVMe devices based on their PCI class. The class for NVMe devices is 0x010802. So guess what the Apple controller provides as class id: 0x018002. If you have to compare the ids twice to notice what's different, you're not alone. My guess is that this subtle difference is just a human error made by an Apple engineer, which wasn't caught during QA and macOS simply works around it as well or doesn't use PCI class anyway. So for the same reason software isn't perfect, hardware (or the firmware powering that hardware) is neither. [1]: https://github.com/torvalds/linux/blob/b791d1bdf9212d944d749a5c7ff6febdba241771/drivers/nvme/host/pci.c#L3126 https://github.com/torvalds/linux/blob/b791d1bdf9212d944d749... [2]: https://lists.infradead.org/pipermail/linux-nvme/2017-February/008351.html https://lists.infradead.org/pipermail/linux-nvme/2017-Februa...
taneq 6y ago
This is kind of like someone living during the Crusades yelling metallurgy is shiiiiit.
BruceEel 6y ago
> https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk - Jonathan Blow - Preventing the Collapse of Civilization Priceless, thanks for sharing.
ecf 6y ago
I too sometimes have the same feeling that the software stack we have today is just a tower of cards waiting to crumble. How long do you think it would take to get back to where we are if everything was scrapped and we restarted with the initial binary -> assembly jump?
MaxBarraclough 6y ago
You might enjoy this 2016 article by Chuck Moore (the Forth guy). His position is pretty extreme, as he dismisses static typing as needless complexity, thinks even C is too elaborate, and he doesn't touch on web technologies, but still worth a read. [0] Also, mandatory link to the Software Disenchantment article. [1] [0] https://web.archive.org/web/20160311002141/http://colorforth.com/1percent.html https://web.archive.org/web/20160311002141/http://colorforth... [1] https://tonsky.me/blog/disenchantment/ https://tonsky.me/blog/disenchantment/
pwdisswordfish2 6y ago
Jonathan Blow: "Software has been freeriding on hardware." True or false? If I make the code that does a particular task smaller, faster, less resource-intensive, then I am not freeriding. He says people do not reference the "five nines" anymore. True? I do not work in the industry anymore. I had no idea this has disappeared. That is really sad. "Developer time" versus "user time". What is more important? Are they equally important? Recently someone posted a video of a recent talk from Margo Seltzer. She said users, e.g., in the Physics Department, do not care about computer languages and these things that computer scientists and programmers think are so important. They care about how fast the program runs. That's all. "Make it go faster". The incentives seem backwards. We pay programmers more today to do less than they did in the past. There is a lot of "busy work" going on.
bcrosby95 6y ago
I find it interesting that your two links seem to contradict each other.
bob1029 6y ago
It would seem there may be a lack of appreciation for how powerful a modern x86 CPU actually is. Even when you apply every side-channel mitigation in the book, these processors are incredibly powerful when used appropriately. Somehow x86 is being branded as this inferior compute facility, always at odds with the GPU/ARM/et.al., and I feel it mostly boils down to shitty software engineering more than anything else. I believe a lot of this can be attributed to rampant, unmitigated abuse of the NUMA architecture exposed by the x86 stack, as well as a neglect for the parallelism offered by CPUs of the last 3-4 generations. Most developers are not following principles which align with what the hardware can actually do, rather, they are following principles which align with what academia/cult-wisdom says you should do. These two worlds are vastly at odds when it comes to keeping your L1/2/3 caches as hot as possible and keeping all of those cores busy with useful work. For example, most software engineers would tell you that busy waiting is a horrible practice, but when you step back and pay attention to the hardware reality, you have 32+ threads to burn, why not put your ultra-high-performance timer as a high-priority process on one of those threads and let it check trigger conditions in a tight loop? You can get timing errors measured in 10s of nanoseconds with this approach, and it's dead simple. Now you only have 31 threads remaining. 30 if you really want to make sure that timer runs smooth. Running at 3+ ghz, that one thread could service an incredible number of events per second. There are new ways to think about how we build things given newer hardware capabilities. I feel a lot of the high-performance software revolution is going to come out of some ideas that have been floating around in fintech. Frameworks like the LMAX Disruptor (and the ideology behind it) can serve as the foundation for a UI framework capable of transacting tens of millions of aggregate user events per second and with peak latencies measured in microseconds. I have personally started to dabble in this area, and the results after just a few weekends have been very encouraging. You would be surprised what even high level languages (Java/C#) are capable of when their use is well-aligned with the hardware.
sharkjacobs 6y ago
This doesn't seem to have anything to do with the article. Am I wrong to wish that the top voted comment was by someone who read the article before they posted?
ponker 6y ago
In spite of all this "garbage" I carry around a $500 machine which fits in my pants pocket and give me directions to anywhere I can think of, a live video call (!) with my friends or family, a vast trove of knowledge about millions of different topics, and a camera that basically matches my $3000 DSLR from six years ago. And these are available whenever and wherever I want, so I can do this video call at 2am on a mountaintop if I want. So... I love this garbage.

scroot 6y ago

Simplicity and good design take lots of time and money. Our culture is not truly ready to make these kinds of investments in the manner required. Why would they? There is a whole universe of FOSS out there upon which anyone can cheaply create "working" software. If your goals are short term (quarterly earnings, looking only a year or two down the road) this is "good enough." Worse, that FOSS foundation is typically filled to the brim with complexity. We have created a computing culture that is premised on pushing the extremes of the teletype model of computing, and tacking what customers think they want on top of it. We have good alternate examples from the past (Oberon, Lisp machines, Hypercard, Smalltalk systems, etc). How often does the new generation of computing people get exposed to these ideas?

de_watcher 6y ago
No, FOSS/proprietary is orthogonal to that. You can find simple and performant software. The problem is in something else.
cjfd 6y ago
I don't think FOSS is the problem either. I think much complexity is required because everything is expected to live on the web and also/thereby is expected to be client/server. As soon as one wants to do more than just display a document on the web one discovers that its architecture is not very suitable for doing anything else besides displaying a document. A badly formatted document at that..... Also the expectation of client-server communication is a big driver of complexity. As soon as one has that we have network communication, serialization and so on. I.e., stuff that is on the large side of things to write oneself. Of course, with the web one more or less has a client-server architecture by default.
zozbot234 6y ago
> Worse, that FOSS foundation is typically filled to the brim with complexity. Really? In my experience, FOSS tends to be a lot simpler and more streamlined than non-free software with comparable functionality.
- na85 6y ago
  I agree with 'de_watcher that FOSS/encumbered is an orthogonal axis to complexity/simplicity. Lots of FOSS software is excessively complex (the systemd ecosystem of shovelware comes immediately to mind) and lots of FOSS is simple. Similarly there are untold thousands of overcomplicated/overengineered proprietary suites and of course it's hard for a graphical application to get simpler than notepad.exe.
asdfman123 6y ago
And what software developers consider "simplicity and good design" often comes across to other people as "I have no idea what to do and looking at this literally gives me anxiety to the point that I want to avoid it."

darepublic 6y ago

Well is the solution more tech (like improving on C language) or is it making sure we have robust non technical fallbacks as a civilization

trimbo 6y ago

> The problem is that both hardware and software are garbage. I think it's incredible that, in my lifetime, computers went from giant mainframes with dedicated hard-line terminals to always-connected supercomputers in everyone's pocket, worldwide. Furthermore, anyone can use the internet to learn how to program. Maybe that's garbage compared to some mythical ideal but in terms of impact on the world it's incredible. > I know that most of the programmers did not work with systems where it's very, very easy to debug the whole stack and you can learn it in a short period but it's amazing when you have knowledge about EVERY part of the system in your head. Well, you can tell from the above that I was around then. I started programming with a single manual and the Beagle Brothers "Peeks, Pokes and Pointers" cheatsheet[1]. People forget that the software itself had to do much less than it does today. Here's just one angle to judge: security. We did not have a worldwide interconnected network with so many people trying to steal data from it. We all used rsh and packets were flying around in cleartext, no problem. But today, all software will have to incorporate TLS. And far fewer people built that simpler software. EA's first titles were written by one or two people. Now a typical EA title has hundreds of people working on it. Things will get better than where they are today. In the future, the industry will have invest more money in "10X" productivity and reliability improvements. Eventually, I think that will happen as productivity continues to slow on large codebases. [1] - https://downloads.reactivemicro.com/Apple%20II%20Items/Documentation/Books/Beagle%20Brothers%20-%20Peeks,%20Pokes%20and%20Pointers.png https://downloads.reactivemicro.com/Apple%20II%20Items/Docum...

Nextgrid 6y ago
And yet, even doing something as basic as a smart doorbell requires a backend server somewhere (essentially a mainframe) and all kinds of NAT hole-punching, UPnP and proprietary push notifications despite the supercomputer in your pocket technically being able to listen for incoming TCP or UDP packets from the doorbell directly. The "supercomputer" processing power is also being wasted on all kinds of malicious and defective-by-design endeavors such as ads, analytics, etc (install any mainstream app and look at the network traffic, 90% of it will be for analytics and can be blocked with no ill effects). Despite the supercomputers being 10x faster than the early ones (back in the iPhone 3G days) we somehow lost the ability to render a 60fps non-stuttering UI despite modern UIs being less rich and consisting mostly of whitespace. > anyone can use the internet to learn how to program I think there used to be a "golden age" of this where the resources were all available for free and at the same time the stacks were manageable (think basic PHP deployed on shared hosting or Ruby or Django) where as nowadays it is considered "wrong" if you don't use Kubernetes, 10 microservices and 100+ NPM packages just to serve a "Hello world" page to a browser.

matheusmoreira 6y ago

> we have hardware that is flat out broken Reading Linux driver code is very informative. Sometimes hardware just doesn't do what is expected and the driver must try and fix it up so that user space can have a reasonable interface. A simple example: /* The ITE8595 always reports 0 as value for the rfkill button. Luckily * it is the only button in its report, and it sends a report on * release only, so receiving a report means the button was pressed. */ if (usage->hid == HID_GD_RFKILL_BTN) { input_event(input, EV_KEY, KEY_RFKILL, 1); input_sync(input); input_event(input, EV_KEY, KEY_RFKILL, 0); input_sync(input); return 1; }

Dunedan 6y ago
It's even more interesting when you can infer why its broken. Let's take the NVMe controller in the MacBook Pro 2016 and later for example: That controller is not properly detected by Linux and needs a quirk [1] to be identified by its PCI device id instead. Why is that? Well, Linux usually detects NVMe devices based on their PCI class. The class for NVMe devices is 0x010802. So guess what the Apple controller provides as class id: 0x018002. If you have to compare the ids twice to notice what's different, you're not alone. My guess is that this subtle difference is just a human error made by an Apple engineer, which wasn't caught during QA and macOS simply works around it as well or doesn't use PCI class anyway. So for the same reason software isn't perfect, hardware (or the firmware powering that hardware) is neither. [1]: https://github.com/torvalds/linux/blob/b791d1bdf9212d944d749a5c7ff6febdba241771/drivers/nvme/host/pci.c#L3126 https://github.com/torvalds/linux/blob/b791d1bdf9212d944d749... [2]: https://lists.infradead.org/pipermail/linux-nvme/2017-February/008351.html https://lists.infradead.org/pipermail/linux-nvme/2017-Februa...

taneq 6y ago

This is kind of like someone living during the Crusades yelling metallurgy is shiiiiit.

BruceEel 6y ago

> https://www.youtube.com/watch?v=pW-SOdj4Kkk https://www.youtube.com/watch?v=pW-SOdj4Kkk - Jonathan Blow - Preventing the Collapse of Civilization Priceless, thanks for sharing.

ecf 6y ago

I too sometimes have the same feeling that the software stack we have today is just a tower of cards waiting to crumble. How long do you think it would take to get back to where we are if everything was scrapped and we restarted with the initial binary -> assembly jump?

MaxBarraclough 6y ago

You might enjoy this 2016 article by Chuck Moore (the Forth guy). His position is pretty extreme, as he dismisses static typing as needless complexity, thinks even C is too elaborate, and he doesn't touch on web technologies, but still worth a read. [0] Also, mandatory link to the Software Disenchantment article. [1] [0] https://web.archive.org/web/20160311002141/http://colorforth.com/1percent.html https://web.archive.org/web/20160311002141/http://colorforth... [1] https://tonsky.me/blog/disenchantment/ https://tonsky.me/blog/disenchantment/

pwdisswordfish2 6y ago

Jonathan Blow: "Software has been freeriding on hardware." True or false? If I make the code that does a particular task smaller, faster, less resource-intensive, then I am not freeriding. He says people do not reference the "five nines" anymore. True? I do not work in the industry anymore. I had no idea this has disappeared. That is really sad. "Developer time" versus "user time". What is more important? Are they equally important? Recently someone posted a video of a recent talk from Margo Seltzer. She said users, e.g., in the Physics Department, do not care about computer languages and these things that computer scientists and programmers think are so important. They care about how fast the program runs. That's all. "Make it go faster". The incentives seem backwards. We pay programmers more today to do less than they did in the past. There is a lot of "busy work" going on.

bcrosby95 6y ago

I find it interesting that your two links seem to contradict each other.

bob1029 6y ago

It would seem there may be a lack of appreciation for how powerful a modern x86 CPU actually is. Even when you apply every side-channel mitigation in the book, these processors are incredibly powerful when used appropriately. Somehow x86 is being branded as this inferior compute facility, always at odds with the GPU/ARM/et.al., and I feel it mostly boils down to shitty software engineering more than anything else. I believe a lot of this can be attributed to rampant, unmitigated abuse of the NUMA architecture exposed by the x86 stack, as well as a neglect for the parallelism offered by CPUs of the last 3-4 generations. Most developers are not following principles which align with what the hardware can actually do, rather, they are following principles which align with what academia/cult-wisdom says you should do. These two worlds are vastly at odds when it comes to keeping your L1/2/3 caches as hot as possible and keeping all of those cores busy with useful work. For example, most software engineers would tell you that busy waiting is a horrible practice, but when you step back and pay attention to the hardware reality, you have 32+ threads to burn, why not put your ultra-high-performance timer as a high-priority process on one of those threads and let it check trigger conditions in a tight loop? You can get timing errors measured in 10s of nanoseconds with this approach, and it's dead simple. Now you only have 31 threads remaining. 30 if you really want to make sure that timer runs smooth. Running at 3+ ghz, that one thread could service an incredible number of events per second. There are new ways to think about how we build things given newer hardware capabilities. I feel a lot of the high-performance software revolution is going to come out of some ideas that have been floating around in fintech. Frameworks like the LMAX Disruptor (and the ideology behind it) can serve as the foundation for a UI framework capable of transacting tens of millions of aggregate user events per second and with peak latencies measured in microseconds. I have personally started to dabble in this area, and the results after just a few weekends have been very encouraging. You would be surprised what even high level languages (Java/C#) are capable of when their use is well-aligned with the hardware.

sharkjacobs 6y ago

This doesn't seem to have anything to do with the article. Am I wrong to wish that the top voted comment was by someone who read the article before they posted?

ponker 6y ago

In spite of all this "garbage" I carry around a $500 machine which fits in my pants pocket and give me directions to anywhere I can think of, a live video call (!) with my friends or family, a vast trove of knowledge about millions of different topics, and a camera that basically matches my $3000 DSLR from six years ago. And these are available whenever and wherever I want, so I can do this video call at 2am on a mountaintop if I want. So... I love this garbage.