4 ms·
I work on the design of novel federated systems for adverse environments, like the Internet, where you have to consider the full range of threats to system beha
by jandrewrogers 6y ago
I work on the design of novel federated systems for adverse environments, like the Internet, where you have to consider the full range of threats to system behavior, functionality, and economy. There is a pretty long list of theoretical trade offs and hard design problems that do not have practical solutions today, which makes it an interesting research area.
Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes:
How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization.
How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly.
How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state?
Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale.
tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.
- Santosh83 6y agoOn the other hand, only if we plunge in will we learn to swim. The solutions will come when there is enough at stake to solve them, but right now since almost no one is using decentralisation/federation at scale on the consumer side, no one has the motivation to tackle the problems you have mentioned. Waiting for a perfect solution to arise before jumping in will ensure it will never happen.