5 ms·

Urbit: The Internet failed (slides and video)

41 points by urbit 13y ago

Gravityloss 13y ago
Is the link right?
- urbit 13y ago
  Yes. It's the second talk on the list. The other talks are great too! Watch them all!
api 13y ago
http://blog.zerotier.com/post/58157836374/op-ed-internet-centralization-is-not-a-conspiracy http://blog.zerotier.com/post/58157836374/op-ed-internet-cen...
- urbit 13y ago
  ZeroTier looks awesome and that's a great post too. I'd dispute the point about the CAP theorem slightly. Thing is, your data - Little Data - is naturally centralized. It doesn't have to be centralized in the same center as everyone else's data though! Then it becomes Big Data, where it's having this giant privacy-violating orgy with everyone else's data all day long. The amount of traffic that your own Little Data has to serve, unless you're a celebrity in which case you can pay for serious hosting, is never going to require a giant cluster of replicated servers. So CAP just isn't that much of an issue.
  - api 13y ago
    It's been obvious to me for a long time that the firewall and NAT are the primary causes of Internet centralization. I think they're a far greater factor than the CAP theorem or protocol / programming model difficulties. For whatever reason, I seem unable to get other people to see what I see here. Most people seem to just not get it. I mean seriously... if nodes cannot easily contact each other horizontally then of course everything evolves toward a super-centralized model with large central groups of nodes acting as intermediaries. What part of that is hard to understand?!? Thanks for the props. A new alpha release of ZeroTier is coming soon, and then it's going into beta with downloadable installers and other nice things. ZT1 is not going to decentralize the 'net, but it does create a lab where people can play with such things. (And it's an interesting VPN alternative for decentralized orgs too.)
    - unimpressive 13y ago
      >I seem unable to get other people to see what I see here. Most people seem to just not get it. I've thought this for a long time. I was even going to cover it in a "computer issues for regular people" book. EDIT: Cover it in one, not as one, there's a lot more material than that.
      - api 13y ago
        I think a big barrier is the netsec defense in depth / reduce surface area dogma. There's this massive cargo cult of the firewall in security circles, and if you suggest removing a firewall everyone freaks out and calls you an idiot. Of course most malware and other attacks today bypass the firewall using "pull" based vectors like HTTP and e-mail, but try telling people that. Remotely exploitable "pushable" vulnerabilities are rare these days on stock OSes too, but again try telling people that. And by firewall in this context I am referring to middle-box firewalls, not local firewalls. The latter are under the control of a box's user/OS and so can easily be opened to permit lateral communication. Middleboxes are the structural culprit here.
    - urbit 13y ago
      I would argue that spam is an even bigger one. Basically, the Internet has spam because identity isn't a limited resource. IP addresses are kind of a limited resource, but they're not really the property of the person using/abusing them, so a blacklist doesn't inflict precise targeted damage, can't be made too draconian, and is easy to evade in lots of ways. And everything above the IP level is unlimited. If there's an unlimited supply of identities, you can't tell the difference between a new customer and an old enemy. You want the first to have positive default reputation and the second to have infinite negative reputation. People in the personal cloud community often point to email as proof that spam can be solved. Yes - but spam was solved in email because email already existed in an spam-free Internet. On the Internet we have, there's a much easier solution to the fact that any new protocol which is successful starts to attract spammers. The solution is: stop using the protocol. Google turned off XMPP federation for this very reason. Basically in an orc-infested environment, you can't have your own cute little bungalow in the cloud. You gotta have an apartment in a giant fortified castle in the cloud. Having a limited supply of identities, in which identities are (a) property and (b) property you control cryptographically (Bitcoin style, "allodial title"), makes it easy to make spamming not pay, once the price of an identity is greater than the profit a spammer can earn by burning it. And it does not require a central governance authority, or even a central reputation authority. (Reputation authorities shouldn't be built into any system, because if they abuse their own reputations the consequences are insanely dire.) NAT is a problem, but there are lots of ways to tunnel around it. Which all suck, of course, but...
urbit 13y ago
A direct link to the video: http://www.youtube.com/watch?v=6S8JFoT6BEM http://www.youtube.com/watch?v=6S8JFoT6BEM
- brian_cloutier 13y ago
  Near the end of the talk someone asks you how to delete data if the state of the machine is just a function of the network packets it has received. I have to admit I didn't really understand your answer: > [When you try to forget data] you're making a subset of this computer that will not compute when it tries to use this information. Could you elaborate a little on what that means, and how you'd apply it in practice. If I wanted to not just make data inaccessible but actually purge it from my machine, how would I do so?
  - urbit 13y ago
    So, you're doing exactly the same thing when you press ^C on an event - what happens is, the event fails to compute. It doesn't go into your log. It doesn't affect your state. It never happened. Packet caused an infinite loop? You never got that packet. And so on. If an event depends on data that's been deleted, that event never happened. But, at the Unix level (which can generate any events it wants), we can generate another event which reports the error. So it's not like things fall silently down the hole, either. (For the same reason, you can also get the stack trace of the infinite loop that was interrupted by your ^C.)
mbrubeck 13y ago
The thing that got me interested in Urbit when it was first announced back in 2010 was reading Van Jacobson's "Networking Named Content" paper [1] and related work from PARC, referenced in this blog post [2]. As someone who has spent 15 years now working on web development, browser development, and web standards, I feel the Internet really needs some form of "named data" networking as its next big step. I don't know if Urbit is the way it's going to happen, but it'll happen somehow. Maybe we'll evolve browsers and HTTP into a named data network instead. (On the one hand, it would be orders of magnitude easier than building an entire new model of computing from scratch... on the other hand, you don't get to throw out the whole ball of mud if you just keep building on top of it.) Anyway, I encourage everyone to read the Van Jacobson paper or watch the tech talk. It points out some assumptions that are so baked into our network architectures that I had never really thought about them. And it gives some context for these slides' claim that "the internet has failed," by pointing to a working alternate system that's a better fit for some common cases. The link from the Urbit blog to the Van Jacobson talk is broken since Google Video was shut down, but you can now find it on YouTube [3]. [1] http://www.parc.com/publication/2318/networking-named-content.html http://www.parc.com/publication/2318/networking-named-conten... [2] http://moronlab.blogspot.com/2010/01/urbit-functional-programming-from.html http://moronlab.blogspot.com/2010/01/urbit-functional-progra... [3] http://www.youtube.com/watch?v=oCZMoY3q2uM http://www.youtube.com/watch?v=oCZMoY3q2uM
- pdonis 13y ago
  the Internet really needs some form of "named data" networking Don't we already have that? URIs are names for data. I had this same reaction when I read about "Content-Centric Networking" back when it first came out. Maybe we'll evolve browsers and HTTP into a named data network They already are. We don't need to evolve them so much as use them in the way HTTP was originally designed to be used: you retrieve and (if you have the privileges) act on data by addressing appropriate HTTP verbs and payloads to its URI.
  - urbit 13y ago
    URIs are names for data - but they're not referentially transparent names. Basically, the inspiration for URIs is Unix filesystems, whereas it should have been Git repositories. It's easy to build mutable resources on top of an immutable namespace. Just put the date, change number, or label, in the name. But once you stop being referentially transparent, you can never recover that virginity. There was sort of an attempt to make Cache-Control headers work in HTTP, but they really don't (I speak as a recovering browser developer), and there's just no substitute for everything having an infinite lifetime.
mkhalil 13y ago
Initially by the title, I thought this was going to be another overly-dramatic simplification of problem article. Turns out to be a good one. I was thinking about this a lot lately.
sz4kerto 13y ago
It's a shame that as a developer, I need to see these stuff (like this Urbit fantasy) to remind myself about why I really started to tinker with computers.
- urbit 13y ago
  Fantasy? Laugh while you can, monkey-boy! :-)
platz 13y ago
Around the same time I heard of Urbit, I was listening to Don Stewart talk on a podcast (Haskell Cast episode #2) about using Haskell to avoid running an entire OS in the cloud; only the necessary code to perform the necessary operations to integrate with other systems was necessary. He was using Xen to run a GHC process which he describes as a "microkernel environment". Xen + Language Runtime (ghc) + user functions = the whole stack, and this boots in milliseconds, using only 1 or 2 meg of memory. What's interesting is that this is a technique that he's using in development today. I didn't read all the Urbit documentation, but the concepts between Don Stewart's approach and Urbit seem to share some similarities with eliminating OS bloat. Some folks have claimed Urbit is as much art at practical, but Don is already doing practical things like this; unless I'm mistaken about what Urbit is about (I'm sure it does a lot of other things differently as well).
- urbit 13y ago
  I can argue with Haskell but I would never argue with teh awesome that is Don Stewart. Urbit runs on Linux right now - but you're right, it would be very cool to run directly on the hypervisor, and it's totally possible for a self-contained system like this one. Personally I think Haskell never should have tried to be a language for producing Unix system calls - the impedance mismatch between FP and imperative OS is just too great.
  - asciilifeform 13y ago
    I'd go one step further and argue that: no sane language should be a language for producing Unix system calls. The impedance mismatch between sanity and imperative OS (and imperative hardware) is too great.
- wmf 13y ago
  Or you could skip the hypervisor and use containers.
- helloTree 13y ago
  You are talking about halvm I think - is it still in use or is it a dead project? The github project page does not seem to have seen commits for a while, but maybe it just ... works. And can you please point me to the talk?

41 points by urbit 13y ago

Gravityloss 13y ago

Is the link right?

urbit 13y ago
Yes. It's the second talk on the list. The other talks are great too! Watch them all!

api 13y ago

http://blog.zerotier.com/post/58157836374/op-ed-internet-centralization-is-not-a-conspiracy http://blog.zerotier.com/post/58157836374/op-ed-internet-cen...

urbit 13y ago
ZeroTier looks awesome and that's a great post too. I'd dispute the point about the CAP theorem slightly. Thing is, your data - Little Data - is naturally centralized. It doesn't have to be centralized in the same center as everyone else's data though! Then it becomes Big Data, where it's having this giant privacy-violating orgy with everyone else's data all day long. The amount of traffic that your own Little Data has to serve, unless you're a celebrity in which case you can pay for serious hosting, is never going to require a giant cluster of replicated servers. So CAP just isn't that much of an issue.
- api 13y ago
  It's been obvious to me for a long time that the firewall and NAT are the primary causes of Internet centralization. I think they're a far greater factor than the CAP theorem or protocol / programming model difficulties. For whatever reason, I seem unable to get other people to see what I see here. Most people seem to just not get it. I mean seriously... if nodes cannot easily contact each other horizontally then of course everything evolves toward a super-centralized model with large central groups of nodes acting as intermediaries. What part of that is hard to understand?!? Thanks for the props. A new alpha release of ZeroTier is coming soon, and then it's going into beta with downloadable installers and other nice things. ZT1 is not going to decentralize the 'net, but it does create a lab where people can play with such things. (And it's an interesting VPN alternative for decentralized orgs too.)
  - unimpressive 13y ago
    >I seem unable to get other people to see what I see here. Most people seem to just not get it. I've thought this for a long time. I was even going to cover it in a "computer issues for regular people" book. EDIT: Cover it in one, not as one, there's a lot more material than that.
    - api 13y ago
      I think a big barrier is the netsec defense in depth / reduce surface area dogma. There's this massive cargo cult of the firewall in security circles, and if you suggest removing a firewall everyone freaks out and calls you an idiot. Of course most malware and other attacks today bypass the firewall using "pull" based vectors like HTTP and e-mail, but try telling people that. Remotely exploitable "pushable" vulnerabilities are rare these days on stock OSes too, but again try telling people that. And by firewall in this context I am referring to middle-box firewalls, not local firewalls. The latter are under the control of a box's user/OS and so can easily be opened to permit lateral communication. Middleboxes are the structural culprit here.
  - urbit 13y ago
    I would argue that spam is an even bigger one. Basically, the Internet has spam because identity isn't a limited resource. IP addresses are kind of a limited resource, but they're not really the property of the person using/abusing them, so a blacklist doesn't inflict precise targeted damage, can't be made too draconian, and is easy to evade in lots of ways. And everything above the IP level is unlimited. If there's an unlimited supply of identities, you can't tell the difference between a new customer and an old enemy. You want the first to have positive default reputation and the second to have infinite negative reputation. People in the personal cloud community often point to email as proof that spam can be solved. Yes - but spam was solved in email because email already existed in an spam-free Internet. On the Internet we have, there's a much easier solution to the fact that any new protocol which is successful starts to attract spammers. The solution is: stop using the protocol. Google turned off XMPP federation for this very reason. Basically in an orc-infested environment, you can't have your own cute little bungalow in the cloud. You gotta have an apartment in a giant fortified castle in the cloud. Having a limited supply of identities, in which identities are (a) property and (b) property you control cryptographically (Bitcoin style, "allodial title"), makes it easy to make spamming not pay, once the price of an identity is greater than the profit a spammer can earn by burning it. And it does not require a central governance authority, or even a central reputation authority. (Reputation authorities shouldn't be built into any system, because if they abuse their own reputations the consequences are insanely dire.) NAT is a problem, but there are lots of ways to tunnel around it. Which all suck, of course, but...

urbit 13y ago

A direct link to the video: http://www.youtube.com/watch?v=6S8JFoT6BEM http://www.youtube.com/watch?v=6S8JFoT6BEM

brian_cloutier 13y ago
Near the end of the talk someone asks you how to delete data if the state of the machine is just a function of the network packets it has received. I have to admit I didn't really understand your answer: > [When you try to forget data] you're making a subset of this computer that will not compute when it tries to use this information. Could you elaborate a little on what that means, and how you'd apply it in practice. If I wanted to not just make data inaccessible but actually purge it from my machine, how would I do so?
- urbit 13y ago
  So, you're doing exactly the same thing when you press ^C on an event - what happens is, the event fails to compute. It doesn't go into your log. It doesn't affect your state. It never happened. Packet caused an infinite loop? You never got that packet. And so on. If an event depends on data that's been deleted, that event never happened. But, at the Unix level (which can generate any events it wants), we can generate another event which reports the error. So it's not like things fall silently down the hole, either. (For the same reason, you can also get the stack trace of the infinite loop that was interrupted by your ^C.)

mbrubeck 13y ago

The thing that got me interested in Urbit when it was first announced back in 2010 was reading Van Jacobson's "Networking Named Content" paper [1] and related work from PARC, referenced in this blog post [2]. As someone who has spent 15 years now working on web development, browser development, and web standards, I feel the Internet really needs some form of "named data" networking as its next big step. I don't know if Urbit is the way it's going to happen, but it'll happen somehow. Maybe we'll evolve browsers and HTTP into a named data network instead. (On the one hand, it would be orders of magnitude easier than building an entire new model of computing from scratch... on the other hand, you don't get to throw out the whole ball of mud if you just keep building on top of it.) Anyway, I encourage everyone to read the Van Jacobson paper or watch the tech talk. It points out some assumptions that are so baked into our network architectures that I had never really thought about them. And it gives some context for these slides' claim that "the internet has failed," by pointing to a working alternate system that's a better fit for some common cases. The link from the Urbit blog to the Van Jacobson talk is broken since Google Video was shut down, but you can now find it on YouTube [3]. [1] http://www.parc.com/publication/2318/networking-named-content.html http://www.parc.com/publication/2318/networking-named-conten... [2] http://moronlab.blogspot.com/2010/01/urbit-functional-programming-from.html http://moronlab.blogspot.com/2010/01/urbit-functional-progra... [3] http://www.youtube.com/watch?v=oCZMoY3q2uM http://www.youtube.com/watch?v=oCZMoY3q2uM

pdonis 13y ago
the Internet really needs some form of "named data" networking Don't we already have that? URIs are names for data. I had this same reaction when I read about "Content-Centric Networking" back when it first came out. Maybe we'll evolve browsers and HTTP into a named data network They already are. We don't need to evolve them so much as use them in the way HTTP was originally designed to be used: you retrieve and (if you have the privileges) act on data by addressing appropriate HTTP verbs and payloads to its URI.
- urbit 13y ago
  URIs are names for data - but they're not referentially transparent names. Basically, the inspiration for URIs is Unix filesystems, whereas it should have been Git repositories. It's easy to build mutable resources on top of an immutable namespace. Just put the date, change number, or label, in the name. But once you stop being referentially transparent, you can never recover that virginity. There was sort of an attempt to make Cache-Control headers work in HTTP, but they really don't (I speak as a recovering browser developer), and there's just no substitute for everything having an infinite lifetime.

mkhalil 13y ago

Initially by the title, I thought this was going to be another overly-dramatic simplification of problem article. Turns out to be a good one. I was thinking about this a lot lately.

sz4kerto 13y ago

It's a shame that as a developer, I need to see these stuff (like this Urbit fantasy) to remind myself about why I really started to tinker with computers.

urbit 13y ago
Fantasy? Laugh while you can, monkey-boy! :-)

platz 13y ago

Around the same time I heard of Urbit, I was listening to Don Stewart talk on a podcast (Haskell Cast episode #2) about using Haskell to avoid running an entire OS in the cloud; only the necessary code to perform the necessary operations to integrate with other systems was necessary. He was using Xen to run a GHC process which he describes as a "microkernel environment". Xen + Language Runtime (ghc) + user functions = the whole stack, and this boots in milliseconds, using only 1 or 2 meg of memory. What's interesting is that this is a technique that he's using in development today. I didn't read all the Urbit documentation, but the concepts between Don Stewart's approach and Urbit seem to share some similarities with eliminating OS bloat. Some folks have claimed Urbit is as much art at practical, but Don is already doing practical things like this; unless I'm mistaken about what Urbit is about (I'm sure it does a lot of other things differently as well).

urbit 13y ago
I can argue with Haskell but I would never argue with teh awesome that is Don Stewart. Urbit runs on Linux right now - but you're right, it would be very cool to run directly on the hypervisor, and it's totally possible for a self-contained system like this one. Personally I think Haskell never should have tried to be a language for producing Unix system calls - the impedance mismatch between FP and imperative OS is just too great.
- asciilifeform 13y ago
  I'd go one step further and argue that: no sane language should be a language for producing Unix system calls. The impedance mismatch between sanity and imperative OS (and imperative hardware) is too great.
wmf 13y ago
Or you could skip the hypervisor and use containers.
helloTree 13y ago
You are talking about halvm I think - is it still in use or is it a dead project? The github project page does not seem to have seen commits for a while, but maybe it just ... works. And can you please point me to the talk?