3 ms·

ZeroTier looks awesome and that's a great post too. I'd dispute the point about the CAP theorem slightly. Thing is, your data - Little Data - is naturally cen

by urbit 13y ago

ZeroTier looks awesome and that's a great post too. I'd dispute the point about the CAP theorem slightly. Thing is, your data - Little Data - is naturally centralized. It doesn't have to be centralized in the same center as everyone else's data though! Then it becomes Big Data, where it's having this giant privacy-violating orgy with everyone else's data all day long. The amount of traffic that your own Little Data has to serve, unless you're a celebrity in which case you can pay for serious hosting, is never going to require a giant cluster of replicated servers. So CAP just isn't that much of an issue.

api 13y ago
It's been obvious to me for a long time that the firewall and NAT are the primary causes of Internet centralization. I think they're a far greater factor than the CAP theorem or protocol / programming model difficulties. For whatever reason, I seem unable to get other people to see what I see here. Most people seem to just not get it. I mean seriously... if nodes cannot easily contact each other horizontally then of course everything evolves toward a super-centralized model with large central groups of nodes acting as intermediaries. What part of that is hard to understand?!? Thanks for the props. A new alpha release of ZeroTier is coming soon, and then it's going into beta with downloadable installers and other nice things. ZT1 is not going to decentralize the 'net, but it does create a lab where people can play with such things. (And it's an interesting VPN alternative for decentralized orgs too.)
- unimpressive 13y ago
  >I seem unable to get other people to see what I see here. Most people seem to just not get it. I've thought this for a long time. I was even going to cover it in a "computer issues for regular people" book. EDIT: Cover it in one, not as one, there's a lot more material than that.
  - api 13y ago
    I think a big barrier is the netsec defense in depth / reduce surface area dogma. There's this massive cargo cult of the firewall in security circles, and if you suggest removing a firewall everyone freaks out and calls you an idiot. Of course most malware and other attacks today bypass the firewall using "pull" based vectors like HTTP and e-mail, but try telling people that. Remotely exploitable "pushable" vulnerabilities are rare these days on stock OSes too, but again try telling people that. And by firewall in this context I am referring to middle-box firewalls, not local firewalls. The latter are under the control of a box's user/OS and so can easily be opened to permit lateral communication. Middleboxes are the structural culprit here.
- urbit 13y ago
  I would argue that spam is an even bigger one. Basically, the Internet has spam because identity isn't a limited resource. IP addresses are kind of a limited resource, but they're not really the property of the person using/abusing them, so a blacklist doesn't inflict precise targeted damage, can't be made too draconian, and is easy to evade in lots of ways. And everything above the IP level is unlimited. If there's an unlimited supply of identities, you can't tell the difference between a new customer and an old enemy. You want the first to have positive default reputation and the second to have infinite negative reputation. People in the personal cloud community often point to email as proof that spam can be solved. Yes - but spam was solved in email because email already existed in an spam-free Internet. On the Internet we have, there's a much easier solution to the fact that any new protocol which is successful starts to attract spammers. The solution is: stop using the protocol. Google turned off XMPP federation for this very reason. Basically in an orc-infested environment, you can't have your own cute little bungalow in the cloud. You gotta have an apartment in a giant fortified castle in the cloud. Having a limited supply of identities, in which identities are (a) property and (b) property you control cryptographically (Bitcoin style, "allodial title"), makes it easy to make spamming not pay, once the price of an identity is greater than the profit a spammer can earn by burning it. And it does not require a central governance authority, or even a central reputation authority. (Reputation authorities shouldn't be built into any system, because if they abuse their own reputations the consequences are insanely dire.) NAT is a problem, but there are lots of ways to tunnel around it. Which all suck, of course, but...
  - api 13y ago
    NAT can be tunneled around, but there's two problems with that: (1) Every NAT traversal protocol is unique, so there is no interoperability between different apps. The power of IP lies in the fact that it's a lingua franca-- anything can open TCP or send UDP. But anything cannot speak BitTorrent-DHT or Skype or whatever. So there's no potential for exponential growth in capability by tying disparate things together. Firewalls and NAT kill protocol interoperability. (2) NAT traversal is hard. I know cause I just did it. It's a pain in the rear, and I'm still going to have to build port 80 HTTP tunneling into ZeroTier for that 0.1% of users who cannot use UDP or tunnel through their NAT. So you have to implement NAT-t + a proxy service for everything. As far as spam goes, you're right. I should have mentioned that. But there are lots of strategies for dealing with spam. Why can't we have a few million small castles instead of one big one, for example? BBSes each had sysops who would kick off abusive users. There are also cryptographic things like hash-cash, Bitcoin economies, trust matrices, etc. These are complex but if we could engineer something good here then it could eventually be packaged into a friendly library that programmers could use without having to understand all the devilish details. The fact is that we did build a decentralized many-to-many Internet. Then we broke it with firewalls and NAT.
- bct 13y ago
  Another critical factor is that centralization is the easiest way to turn a profit.
  - api 13y ago
    There are other ways though: sell the software itself, sell services built around something that is itself distributed, etc. But in general you are right. Most Internet business models revolve around monetizing a flow of traffic, and to do that the traffic has to flow through you.

by urbit 13y ago

api 13y ago

It's been obvious to me for a long time that the firewall and NAT are the primary causes of Internet centralization. I think they're a far greater factor than the CAP theorem or protocol / programming model difficulties. For whatever reason, I seem unable to get other people to see what I see here. Most people seem to just not get it. I mean seriously... if nodes cannot easily contact each other horizontally then of course everything evolves toward a super-centralized model with large central groups of nodes acting as intermediaries. What part of that is hard to understand?!? Thanks for the props. A new alpha release of ZeroTier is coming soon, and then it's going into beta with downloadable installers and other nice things. ZT1 is not going to decentralize the 'net, but it does create a lab where people can play with such things. (And it's an interesting VPN alternative for decentralized orgs too.)

unimpressive 13y ago
>I seem unable to get other people to see what I see here. Most people seem to just not get it. I've thought this for a long time. I was even going to cover it in a "computer issues for regular people" book. EDIT: Cover it in one, not as one, there's a lot more material than that.
- api 13y ago
  I think a big barrier is the netsec defense in depth / reduce surface area dogma. There's this massive cargo cult of the firewall in security circles, and if you suggest removing a firewall everyone freaks out and calls you an idiot. Of course most malware and other attacks today bypass the firewall using "pull" based vectors like HTTP and e-mail, but try telling people that. Remotely exploitable "pushable" vulnerabilities are rare these days on stock OSes too, but again try telling people that. And by firewall in this context I am referring to middle-box firewalls, not local firewalls. The latter are under the control of a box's user/OS and so can easily be opened to permit lateral communication. Middleboxes are the structural culprit here.
urbit 13y ago
I would argue that spam is an even bigger one. Basically, the Internet has spam because identity isn't a limited resource. IP addresses are kind of a limited resource, but they're not really the property of the person using/abusing them, so a blacklist doesn't inflict precise targeted damage, can't be made too draconian, and is easy to evade in lots of ways. And everything above the IP level is unlimited. If there's an unlimited supply of identities, you can't tell the difference between a new customer and an old enemy. You want the first to have positive default reputation and the second to have infinite negative reputation. People in the personal cloud community often point to email as proof that spam can be solved. Yes - but spam was solved in email because email already existed in an spam-free Internet. On the Internet we have, there's a much easier solution to the fact that any new protocol which is successful starts to attract spammers. The solution is: stop using the protocol. Google turned off XMPP federation for this very reason. Basically in an orc-infested environment, you can't have your own cute little bungalow in the cloud. You gotta have an apartment in a giant fortified castle in the cloud. Having a limited supply of identities, in which identities are (a) property and (b) property you control cryptographically (Bitcoin style, "allodial title"), makes it easy to make spamming not pay, once the price of an identity is greater than the profit a spammer can earn by burning it. And it does not require a central governance authority, or even a central reputation authority. (Reputation authorities shouldn't be built into any system, because if they abuse their own reputations the consequences are insanely dire.) NAT is a problem, but there are lots of ways to tunnel around it. Which all suck, of course, but...
- api 13y ago
  NAT can be tunneled around, but there's two problems with that: (1) Every NAT traversal protocol is unique, so there is no interoperability between different apps. The power of IP lies in the fact that it's a lingua franca-- anything can open TCP or send UDP. But anything cannot speak BitTorrent-DHT or Skype or whatever. So there's no potential for exponential growth in capability by tying disparate things together. Firewalls and NAT kill protocol interoperability. (2) NAT traversal is hard. I know cause I just did it. It's a pain in the rear, and I'm still going to have to build port 80 HTTP tunneling into ZeroTier for that 0.1% of users who cannot use UDP or tunnel through their NAT. So you have to implement NAT-t + a proxy service for everything. As far as spam goes, you're right. I should have mentioned that. But there are lots of strategies for dealing with spam. Why can't we have a few million small castles instead of one big one, for example? BBSes each had sysops who would kick off abusive users. There are also cryptographic things like hash-cash, Bitcoin economies, trust matrices, etc. These are complex but if we could engineer something good here then it could eventually be packaged into a friendly library that programmers could use without having to understand all the devilish details. The fact is that we did build a decentralized many-to-many Internet. Then we broke it with firewalls and NAT.
bct 13y ago
Another critical factor is that centralization is the easiest way to turn a profit.
- api 13y ago
  There are other ways though: sell the software itself, sell services built around something that is itself distributed, etc. But in general you are right. Most Internet business models revolve around monetizing a flow of traffic, and to do that the traffic has to flow through you.