6 ms·

[caution old man yells at clouds] There are so many comments I started to write, re-wrote, deleted, etc. I have largely the same view of the complexity hell-sc

[caution old man yells at clouds] There are so many comments I started to write, re-wrote, deleted, etc. I have largely the same view of the complexity hell-scape we're creating. I can't access the vendor binary blob software that has to run to bring up my CPU and system devices. I can't comprehend the 20 million+ lines of kernel running on it. I can't comprehend what instructions I'm running because the ISA spans several volumes. It's not like the 6502 - which you could keep in your head. I know there are all sorts of subsystems that can take over at any time, like the BMC. The drive firmware is free to ignore some of my requests, or lie to me about completing status. When I do write code in C, I can't guarantee that the disassembled code resembles my actual code (I got schooled in that by clang, again, recently). And the CPU isn't obligated to run it in that order. All this was in pursuit of slightly more performance. And what do we do with that computing substrate? We add layer upon layer of abstraction. To the point where we can talk about server consolidation for a hot 5 minutes after a new generation comes out. But then, five minutes later, we need even more servers. C is too slow to write in. Let's move to Java. Java's too slow. Let's move to Node... but that's not good, let's have a whole infrastructure to transpile JavaScript to JavaScript. Only 12 gigs of dependencies from 53,000 random repos in Github. Write the back end in Java... no Python... no whole programs are too slow... go with web functions. Then a whole other virtualized infrastructure so we can tolerate the loss of a data center, availability zone, or even a Geo. Can we run it in space??? What if the earth blows up? How can we track peoples' browsing if all the world's data centers are destroyed, unless we also run it in space? Sigh...

throwaway14356 4y ago
the 6502 was not perfect but great for its time. If we were to design it today we could probably leave stuff out. Use multiple bytes per instruction to address the large memory, a few more accumulators and it wil do just fine. Then we bake x of them onto a chip with y custom roms besides it and write everything from scratch. Some operations will be more complicated than ever, some things will be slow but most complexity is gone. As a bonus it will be absurdly cheap to make.
- dspillett 4y ago
  > it will be absurdly cheap to make Being cheap was one of the key reasons why the 6502, despite its faults†, was great for its time. It was much cheaper to buy than the similar competition (8600, 8080), IIRC by a factor of five in common cases. ---- [†] single true register‡ (plus a pair of more limited indexing registers, three if you count the stack pointer), and a few notable bugs like off-spec wrapping if indirect jump reads from a location at a page end (xxff) which I once banged my head against. [‡] mitigated a bit by rapid access to page 0
  - throwaway14356 4y ago
    The part where you wrote something and it would run on any c64 exactly the same way. A few hacks aside, you could get some work done and it was done! Today the update circus feels like jenga.
ip26 4y ago
Layers of abstraction aren’t automatically bad. Sometimes abstraction even adds performance. There do seem to be a few layers that are responsible for more performance loss and complexity than others, however. For example, Python is great but runs anywhere from 100-100,000 slower than C, depending on where you look. Investment in some of these layers has outsized importance.
- phoehne 4y ago
  100% agree. As long as the trade off between performance and new failure states is tractable to analyze. My worry is with enough layering, that analysis becomes intractable. Not that you need to account for every imaginable failure (what happens if magic is real and a wizard casts a spell on DIMM 3?). But when things go south, and as a stint as an SRE I have come to realize we are all on a giant South-bound train, you have to perform that analysis.
  - ip26 4y ago
    If each layer is well built and cleanly separated, in theory you can just raise or lower to the appropriate layer. The trick, however, becomes acquiring visibility to the layer of interest.
    - phoehne 4y ago
      Layers are not as cleanly isolated as we like to think. Take buffered IO as an example. With unbuffered IO with a dumb drive, a lot of programs run slower. However, what unbuffered IO does is give you immediate feedback about the completion of the IO. I said write 64 bytes. It either writes 64 bytes, or fails. When I add buffering, I now have a new possible failure state where my request to write data completed, but the data is sitting in the buffer. If the data were really important (say a database) I add a write barrier like a 'sync' call. It's one more thing I need to be aware of. Bufferring adds more ways to fail. Then we add an additional twist. Our disk controller is really clever and has its own local cache so I can write 64 bytes, send it to the controller, have the controller say "we're good", but not have to wait for the data to be written to the platter. With the dumb drive, that wasn't the case. I had immediate feedback. But now we have a new failure state and I have to account for that as well. Maybe I have to put battery backup on the controller. Maybe I need a new primitive to really flush the data. And I might also have new errors where the controller accepts the data, attempts to write it to disk, but the disk is dying and fails to write. So my program lost data and the only way I know about it is to look in an unrelated log for drive failures. Unfortunately no one thought to check or change the card's battery backup in 3 years... This isn't just a caching problem. Let's say you take some functionality and move it from a library to a service on the network. Now you have to account for all sorts of other issues related to network failures that just weren't part of the base case for local resources. I can no longer just make a call, I have to think about things like retries, or what I need to do if the service is unavailable. Possibly, I need to percolate up new errors to drive my code into behaviors like caching the data locally until I can send it. My debugging also gets more complicated, like 1) is the server alive (ping is often blocked so we use Test-NetConnection to do an NS lookup, ping, and TCP connection test), 2) Is the address routable? 3) Is my network connection still valid (e.g. did my local switch meet its maker)? 4) Do I have correct certificates (e.g. we can't complete the SSL handshake)? 5) Was there a protocol change (e.g. Team A is sending {"foo": 1} but Team B is expecting {"foo": "1"}, and so on. It isn't a layering thing like well constructed, SOLID code, it's changing the assumptions and conditions you have for all sorts of states you could be in. It's even further compounded with complex services where service A depends on B, C, and D, and D is failing. But now the data in B and C don't reflect what's in D, which failed to update. Your code becomes much more complex to handle these additional failure scenarios. Your search space for possible failures is orders of magnitude larger.

throwaway14356 4y ago

the 6502 was not perfect but great for its time. If we were to design it today we could probably leave stuff out. Use multiple bytes per instruction to address the large memory, a few more accumulators and it wil do just fine. Then we bake x of them onto a chip with y custom roms besides it and write everything from scratch. Some operations will be more complicated than ever, some things will be slow but most complexity is gone. As a bonus it will be absurdly cheap to make.

dspillett 4y ago
> it will be absurdly cheap to make Being cheap was one of the key reasons why the 6502, despite its faults†, was great for its time. It was much cheaper to buy than the similar competition (8600, 8080), IIRC by a factor of five in common cases. ---- [†] single true register‡ (plus a pair of more limited indexing registers, three if you count the stack pointer), and a few notable bugs like off-spec wrapping if indirect jump reads from a location at a page end (xxff) which I once banged my head against. [‡] mitigated a bit by rapid access to page 0
- throwaway14356 4y ago
  The part where you wrote something and it would run on any c64 exactly the same way. A few hacks aside, you could get some work done and it was done! Today the update circus feels like jenga.

ip26 4y ago

Layers of abstraction aren’t automatically bad. Sometimes abstraction even adds performance. There do seem to be a few layers that are responsible for more performance loss and complexity than others, however. For example, Python is great but runs anywhere from 100-100,000 slower than C, depending on where you look. Investment in some of these layers has outsized importance.

phoehne 4y ago
100% agree. As long as the trade off between performance and new failure states is tractable to analyze. My worry is with enough layering, that analysis becomes intractable. Not that you need to account for every imaginable failure (what happens if magic is real and a wizard casts a spell on DIMM 3?). But when things go south, and as a stint as an SRE I have come to realize we are all on a giant South-bound train, you have to perform that analysis.
- ip26 4y ago
  If each layer is well built and cleanly separated, in theory you can just raise or lower to the appropriate layer. The trick, however, becomes acquiring visibility to the layer of interest.
  - phoehne 4y ago
    Layers are not as cleanly isolated as we like to think. Take buffered IO as an example. With unbuffered IO with a dumb drive, a lot of programs run slower. However, what unbuffered IO does is give you immediate feedback about the completion of the IO. I said write 64 bytes. It either writes 64 bytes, or fails. When I add buffering, I now have a new possible failure state where my request to write data completed, but the data is sitting in the buffer. If the data were really important (say a database) I add a write barrier like a 'sync' call. It's one more thing I need to be aware of. Bufferring adds more ways to fail. Then we add an additional twist. Our disk controller is really clever and has its own local cache so I can write 64 bytes, send it to the controller, have the controller say "we're good", but not have to wait for the data to be written to the platter. With the dumb drive, that wasn't the case. I had immediate feedback. But now we have a new failure state and I have to account for that as well. Maybe I have to put battery backup on the controller. Maybe I need a new primitive to really flush the data. And I might also have new errors where the controller accepts the data, attempts to write it to disk, but the disk is dying and fails to write. So my program lost data and the only way I know about it is to look in an unrelated log for drive failures. Unfortunately no one thought to check or change the card's battery backup in 3 years... This isn't just a caching problem. Let's say you take some functionality and move it from a library to a service on the network. Now you have to account for all sorts of other issues related to network failures that just weren't part of the base case for local resources. I can no longer just make a call, I have to think about things like retries, or what I need to do if the service is unavailable. Possibly, I need to percolate up new errors to drive my code into behaviors like caching the data locally until I can send it. My debugging also gets more complicated, like 1) is the server alive (ping is often blocked so we use Test-NetConnection to do an NS lookup, ping, and TCP connection test), 2) Is the address routable? 3) Is my network connection still valid (e.g. did my local switch meet its maker)? 4) Do I have correct certificates (e.g. we can't complete the SSL handshake)? 5) Was there a protocol change (e.g. Team A is sending {"foo": 1} but Team B is expecting {"foo": "1"}, and so on. It isn't a layering thing like well constructed, SOLID code, it's changing the assumptions and conditions you have for all sorts of states you could be in. It's even further compounded with complex services where service A depends on B, C, and D, and D is failing. But now the data in B and C don't reflect what's in D, which failed to update. Your code becomes much more complex to handle these additional failure scenarios. Your search space for possible failures is orders of magnitude larger.