7 ms·

This is the fundamental premise that object oriented thinking was created to solve. You cannot understand a sufficiently advanced system from both an overview

This is the fundamental premise that object oriented thinking was created to solve. You cannot understand a sufficiently advanced system from both an overview and granular view. You have to either see the overview and trust the granular objects, or specialize in one granular object and then trust your adjacent objects. Its a problem written about in detail by Marcus Aurelius in Meditations when he comments on running an empire. He notes that the most difficult part is not having information in a timely fashion at scale -- you can either have timely information about a single place, or out of date information about the entire empire. They key part in both is trust and responsibility. You must trust that granular objects can take responsibility for themselves. If each one requires knowledge of the entire system to function, then the weight of that will overburden any system over time.

Elof 10y ago
I think there is an important art in recognizing signal over noise and being pragmatic when considering how to design a system. I think you're right about trust, but I would add in a focus on fundamentals/first principles over design.
pdkl95 10y ago
> If each one requires knowledge of the entire system The general problem is the growing number of dependencies. James Burke in "Connections" warned that civilization is an interconnected web of technology traps[1]. We survive as long as everything works (or "mostly" works), but there is a risk of cascade failure. More recently, Dan Geer warned[2] about the same type of problem: In the last couple of years, I've found that institutions that I more or less must use [...] no longer accept paper letter they each only accept digital delivery of such instructions. This means that each of them has created a critical dependence on an Internet swarming with men in the middle and, which is more, they have doubtlessly given up their own ability to fall back to what worked for a century before. [...] Everything in meatspace we give over to cyberspace replaces dependencies that are local and manageable with dependencies that are certainly not local and I would argue much less manageable because they are much less secure. I say that because the root cause of risk is dependence, and most especially dependence on expectations of system state. [...] Accommodating old methods and Internet rejectionists preserves alternate, less complex, more durable means and therefore bounds dependence. Bounding dependence is *the* core of rational risk management. [1] https://www.youtube.com/watch?v=lKELMR6wACw https://www.youtube.com/watch?v=lKELMR6wACw [2] http://geer.tinho.net/geer.blackhat.6viii14.txt http://geer.tinho.net/geer.blackhat.6viii14.txt
- nradov 10y ago
  It might have been reasonable to maintain backup paper workflows years ago when those paper workflows already existed and institutions were just starting to offer digital alternatives. But new institutions and workflows have arisen that never had paper alternatives in the first place. And it's unreasonable to expect institutions to build and test paper alternatives just as a backup. No one's going to pay for that, regardless of potential consequences.
  - pdkl95 10y ago
    I agree that new institutions that never had pre-internet systems are a harder problem. > it's unreasonable to expect institutions to build and test paper alternatives just as a backup. Some type of backup is necessary for anything critically important. From my previous [2]: In sum, as a matter of policy everything that is officially categorized as a critical infrastructure must conclusively show how it can operate in the absence of the Internet. Depending on a single source for mission critical features has been popular since at least the dot com era. Maybe some people can live with the risk that e.g. Twitter can shut their business down at any time by banning their API key, but some institutions need to be reliable. In the past, this need for reliability lead to the agreement where Intel licensed AMD to second-source[3] the 8086 and other parts. [3] https://en.wikipedia.org/wiki/Second_source https://en.wikipedia.org/wiki/Second_source
    - int_19h 10y ago
      Some type of backup is undeniably useful, but why does it have to be paper? The problem with paper vs electronic is that they're largely incompatible, requiring manual labor for interoperability. A backup system should be as drop-in as possible, so for digital systems, it should be another digital system.
- kansface 10y ago
  I don't think it matters too much that most institutions no longer accept paper since paper is just the external interface. You get no more reliability if they did when all critical processes rely on technology. In other words, you aren't walking your check to the IRS, the postal service couldn't deliver it, your bank couldn't cash it even if they did, and WTF would the IRS do with a billion checks?
mjevans 10y ago
You've hit the nail on the head with the trust and responsibility. There does need to be overall planning and co-ordination; but solutions cannot be dictated at a large scale. Instead /goals/ must be dictated, strive for X, prevent Y, co-operate with Z to prevent unwanted unintended consequences. Larger governing bodies need to provide a basic framework, the minimum standard and boundary for the system's operation. Possibly a framework for re-adjusting or outright re-initiating failed layers they contain. They should then provide both the goals and responsibilities to their contained units, but not an express direction on how the task is to be handled. This allows for adjusting and reacting at a more finely grained level which also reduces the time needed to reach consensus and adjust. I would like to provide an example. It's true of the metro area that I live in, and I suspect it is true of others as well. There appears to be a complete disconnect between landing sources of jobs, providing places for workers to live, and reliable transit to/from jobs. In particular suburban sprawl from hell exists (yet is confined vaguely within outlying urban growth boundaries); a result that is a misuse of resources and ensures that traffic is a nightmare. What's worse, departments of the government, such as the DoT, are so focused that they can only see trying to use their own specific hammer on the problem, instead of working with other departments to reach an actual solution. In the containers system, the DoT would work with some urban planning / their own measurement to show current and projected traffic needs, as well as the budget required for tolerating them. Their containing layer, reacting to the massive budget shortfall properly, would pass directives that require localities to increase the density of 'quality' housing and lower the housing prices, and 'make the housing attractive to families'. The localities would then react and modify the environment, leading to an actual solution to the issue. Government is a complex system, just like an engineer of any discipline should recognize, and it needs a massive overhaul.
- panic 10y ago
  Instead /goals/ must be dictated, strive for X, prevent Y, co-operate with Z to prevent unwanted unintended consequences. I don't think it's that simple. The "goals" of a city are the goals of the people living in it, which are as complicated as the city itself. Your example is actually a great illustration of how large-scale decision making can go wrong: the sprawl met someone's goals, or it wouldn't have happened, but it didn't match the needs of the actual people living in the city. In general, how do you know the goals you're dictating are the right ones? A better focus would be on ways for individuals and communities within cities to organize, communicate their needs and get resources to solve their unique problems.
haimez 10y ago
I think that this is more a natural observation about distributed systems that object orientation. Neither precludes the other, but the root of the problem is the inherent necessity and limitation that comes from distributing information and decision nodes, and less from leveraging hierarchy to "get shit done".
brazzledazzle 10y ago
I've met micro-managers that actually defend micro-management but even if you think it's a viable way to actually get what you want from your reports this is an objective fundamental problem with it as a management style. You simply can't scale it to any useful degree because any kind of scale requires trust and delegation. Once you pass that magic threshold the people that report to you will take advantage of any slack you give because micro-managers don't inspire or create loyalty. They will also actively avoid making decisions because that's the manager's job and they don't want them all over their ass for being autonomous. At that point you're a manager with no time to think about the long term and you're in firefighting mode constantly. And I can't say I have any sympathy. You reap what you sow.
acjohnson55 10y ago
Really, this is the idea behind every paradigm for programming. It's all about creating rules, structures, and abstractions to compartmentalize software horizontally and vertically (so to speak) and formalize relationships between the pieces.
dilemma 10y ago
I think this is the insight underlying Tokyo's zoning system (linked here a few weeks back) that keeps the city so eclectic and alive, and Toyota's production system that seemingly no one can replicate though its been studied for decades, with support from Toyota. It seems to me that in Toyota, a first, basic process is designed centrally by someone who is an authority in that specific area, but realizing that the top of the pyramid lacks the on the ground information necessary for continuous improvement, it pushes agency to change the process down to the bottom of the pyramid, to the people who make up the process. That is how Toyota is able to evolve their process and achieve yearly gains in efficiency of around 10%. There's no great genius other than in the insight that the boss can't do the job of his subordinates, that knowledge is situational. And that is probably why companies fail to adapt the TPS - because they do not trust and even despise people on the floor.

Elof 10y ago

I think there is an important art in recognizing signal over noise and being pragmatic when considering how to design a system. I think you're right about trust, but I would add in a focus on fundamentals/first principles over design.

pdkl95 10y ago

> If each one requires knowledge of the entire system The general problem is the growing number of dependencies. James Burke in "Connections" warned that civilization is an interconnected web of technology traps[1]. We survive as long as everything works (or "mostly" works), but there is a risk of cascade failure. More recently, Dan Geer warned[2] about the same type of problem: In the last couple of years, I've found that institutions that I more or less must use [...] no longer accept paper letter they each only accept digital delivery of such instructions. This means that each of them has created a critical dependence on an Internet swarming with men in the middle and, which is more, they have doubtlessly given up their own ability to fall back to what worked for a century before. [...] Everything in meatspace we give over to cyberspace replaces dependencies that are local and manageable with dependencies that are certainly not local and I would argue much less manageable because they are much less secure. I say that because the root cause of risk is dependence, and most especially dependence on expectations of system state. [...] Accommodating old methods and Internet rejectionists preserves alternate, less complex, more durable means and therefore bounds dependence. Bounding dependence is *the* core of rational risk management. [1] https://www.youtube.com/watch?v=lKELMR6wACw https://www.youtube.com/watch?v=lKELMR6wACw [2] http://geer.tinho.net/geer.blackhat.6viii14.txt http://geer.tinho.net/geer.blackhat.6viii14.txt

nradov 10y ago
It might have been reasonable to maintain backup paper workflows years ago when those paper workflows already existed and institutions were just starting to offer digital alternatives. But new institutions and workflows have arisen that never had paper alternatives in the first place. And it's unreasonable to expect institutions to build and test paper alternatives just as a backup. No one's going to pay for that, regardless of potential consequences.
- pdkl95 10y ago
  I agree that new institutions that never had pre-internet systems are a harder problem. > it's unreasonable to expect institutions to build and test paper alternatives just as a backup. Some type of backup is necessary for anything critically important. From my previous [2]: In sum, as a matter of policy everything that is officially categorized as a critical infrastructure must conclusively show how it can operate in the absence of the Internet. Depending on a single source for mission critical features has been popular since at least the dot com era. Maybe some people can live with the risk that e.g. Twitter can shut their business down at any time by banning their API key, but some institutions need to be reliable. In the past, this need for reliability lead to the agreement where Intel licensed AMD to second-source[3] the 8086 and other parts. [3] https://en.wikipedia.org/wiki/Second_source https://en.wikipedia.org/wiki/Second_source
  - int_19h 10y ago
    Some type of backup is undeniably useful, but why does it have to be paper? The problem with paper vs electronic is that they're largely incompatible, requiring manual labor for interoperability. A backup system should be as drop-in as possible, so for digital systems, it should be another digital system.
kansface 10y ago
I don't think it matters too much that most institutions no longer accept paper since paper is just the external interface. You get no more reliability if they did when all critical processes rely on technology. In other words, you aren't walking your check to the IRS, the postal service couldn't deliver it, your bank couldn't cash it even if they did, and WTF would the IRS do with a billion checks?

mjevans 10y ago

You've hit the nail on the head with the trust and responsibility. There does need to be overall planning and co-ordination; but solutions cannot be dictated at a large scale. Instead /goals/ must be dictated, strive for X, prevent Y, co-operate with Z to prevent unwanted unintended consequences. Larger governing bodies need to provide a basic framework, the minimum standard and boundary for the system's operation. Possibly a framework for re-adjusting or outright re-initiating failed layers they contain. They should then provide both the goals and responsibilities to their contained units, but not an express direction on how the task is to be handled. This allows for adjusting and reacting at a more finely grained level which also reduces the time needed to reach consensus and adjust. I would like to provide an example. It's true of the metro area that I live in, and I suspect it is true of others as well. There appears to be a complete disconnect between landing sources of jobs, providing places for workers to live, and reliable transit to/from jobs. In particular suburban sprawl from hell exists (yet is confined vaguely within outlying urban growth boundaries); a result that is a misuse of resources and ensures that traffic is a nightmare. What's worse, departments of the government, such as the DoT, are so focused that they can only see trying to use their own specific hammer on the problem, instead of working with other departments to reach an actual solution. In the containers system, the DoT would work with some urban planning / their own measurement to show current and projected traffic needs, as well as the budget required for tolerating them. Their containing layer, reacting to the massive budget shortfall properly, would pass directives that require localities to increase the density of 'quality' housing and lower the housing prices, and 'make the housing attractive to families'. The localities would then react and modify the environment, leading to an actual solution to the issue. Government is a complex system, just like an engineer of any discipline should recognize, and it needs a massive overhaul.

panic 10y ago
Instead /goals/ must be dictated, strive for X, prevent Y, co-operate with Z to prevent unwanted unintended consequences. I don't think it's that simple. The "goals" of a city are the goals of the people living in it, which are as complicated as the city itself. Your example is actually a great illustration of how large-scale decision making can go wrong: the sprawl met someone's goals, or it wouldn't have happened, but it didn't match the needs of the actual people living in the city. In general, how do you know the goals you're dictating are the right ones? A better focus would be on ways for individuals and communities within cities to organize, communicate their needs and get resources to solve their unique problems.

haimez 10y ago

I think that this is more a natural observation about distributed systems that object orientation. Neither precludes the other, but the root of the problem is the inherent necessity and limitation that comes from distributing information and decision nodes, and less from leveraging hierarchy to "get shit done".

brazzledazzle 10y ago

I've met micro-managers that actually defend micro-management but even if you think it's a viable way to actually get what you want from your reports this is an objective fundamental problem with it as a management style. You simply can't scale it to any useful degree because any kind of scale requires trust and delegation. Once you pass that magic threshold the people that report to you will take advantage of any slack you give because micro-managers don't inspire or create loyalty. They will also actively avoid making decisions because that's the manager's job and they don't want them all over their ass for being autonomous. At that point you're a manager with no time to think about the long term and you're in firefighting mode constantly. And I can't say I have any sympathy. You reap what you sow.

acjohnson55 10y ago

Really, this is the idea behind every paradigm for programming. It's all about creating rules, structures, and abstractions to compartmentalize software horizontally and vertically (so to speak) and formalize relationships between the pieces.

dilemma 10y ago

I think this is the insight underlying Tokyo's zoning system (linked here a few weeks back) that keeps the city so eclectic and alive, and Toyota's production system that seemingly no one can replicate though its been studied for decades, with support from Toyota. It seems to me that in Toyota, a first, basic process is designed centrally by someone who is an authority in that specific area, but realizing that the top of the pyramid lacks the on the ground information necessary for continuous improvement, it pushes agency to change the process down to the bottom of the pyramid, to the people who make up the process. That is how Toyota is able to evolve their process and achieve yearly gains in efficiency of around 10%. There's no great genius other than in the insight that the boss can't do the job of his subordinates, that knowledge is situational. And that is probably why companies fail to adapt the TPS - because they do not trust and even despise people on the floor.