9 ms·

This Could be Big: Decentralized Web Standard Under Development by W3C

136 points by kerben 15y ago

swombat 15y ago
As the comments point out, this is not a generic decentralised web standard to get around ICE and the like, but just a specification for p2p audio/video/etc communications for online video conferencing and so on. Not as big as I hoped.
- mcrider 15y ago
  That's a shame. I was just thinking yesterday how great it would be to be able to host web pages in a P2P fashion (after all, people are building an economy of the bittorrent infrastructure). Sites like WikiLeaks would get a great advantage of this sort of protocol.
  - bxr 15y ago
    It sounds like you're looking for the Freenet project. http://en.wikipedia.org/wiki/Freenet http://en.wikipedia.org/wiki/Freenet
- fragsworth 15y ago
  But if you enable p2p communication between javascript clients, there should be nothing stopping you from implementing that kind of stuff within the browser.
phlux 15y ago
I wonder how easily man-in-the-middle attacks via node spoofing would be. You masquerade as a node by re-hosting their content and you capture any other client that accesses your proxy of that information.
- sp332 15y ago
  Oh, it's simple! All we need is for each node to sign their content with a private key. Then everyone can check the validity of the content with the originating node's public key, even if it comes from some untrusted intermediary. Hm, now all we need is a centralized server to distribute the public keys securely. But that's a lot of keys (one for every node), so let's use a distributed system... oh wait :)
  - aristus 15y ago
    All p2p tamper-proofing schemes reduce to the Byzantine Generals Problem. There exist working algorithms, and the math was figured out a long time ago. There are two wrinkles, though: BGP solutions are kind of slow. Worse, it is impossible for any BGP system to function securely if more than 1/3 of the nodes act in bad faith.
  - akdubya 15y ago
    Good background from the bitcoin people: "A number of Byzantine Generals each have a computer and want to attack the King's wi-fi by brute forcing the password, which they've learned is a certain number of characters in length. Once they stimulate the network to generate a packet, they must crack the password within a limited time to break in and erase the logs, lest they be discovered. They only have enough CPU power to crack it fast enough if a majority of them attack at the same time. They don't particularly care when the attack will be, just that they agree. It has been decided that anyone who feels like it will announce an attack time, which we'll call the "plan", and whatever plan is heard first will be the official plan. The problem is that the network is not instantaneous, and if two generals announce different plans at close to the same time, some may hear one first and others hear the other first. They use a proof-of-work chain to solve the problem. Once each general receives whatever plan he hears first, he sets his computer to solve a difficult hash-based proof-of-work problem that includes the plan in its hash. The proof-of-work is difficult enough that with all of them working at once, it's expected to take 10 minutes before one of them finds a solution and broadcasts it to the network. Once received, everyone adjusts the hash in their proof-of-work computation to include the first solution, so that when they find the next proof-of-work, it chains after the first one. If anyone was working on a different plan, they switch to this one, because its proof-of-work chain is now longer. After about two hours, the plan should be hashed by a chain of 12 proofs-of-work. Every general, just by verifying the difficulty of the proof-of-work chain, can estimate how much parallel CPU power per hour was expended on it and see that it must have required the majority of the computers to produce in the allotted time. At the least, most of them had to have seen the plan, since the proof-of-work is proof that they worked on it. If the CPU power exhibited by the proof-of-work is sufficient to crack the password, they can safely attack at the agreed time." http://www.bitcoin.org/byzantine.html http://www.bitcoin.org/byzantine.html
codemechanic 15y ago
Tonido (http://www.tonido.com http://www.tonido.com) is a pioneer in this space. They have invented the model much before opera unit. The cool thing is that Tonido also provides a decentralized openid to end users. Firefox should buy Tonido. It will change the industry if it happens and the way people share information. I probably may be little ahead. If you think deeply it will make sense.
- zyfo 15y ago
  Do you work for Tonido? If you do, please provide a disclaimer. What you are doing is borderline spamming. See his comments for yourself [0] If I'm wrong, I apologize profusely in advance. EDIT: Parent does work for Tonido [1]. This is spamming. 0: http://news.ycombinator.com/threads?id=codemechanic http://news.ycombinator.com/threads?id=codemechanic 1: http://news.ycombinator.com/item?id=643833 http://news.ycombinator.com/item?id=643833
  - fragsworth 15y ago
    Even if you were wrong, I don't think you need to apologize. Almost all of his comments were trying to promote something rather than provide meaningful discussion.
    - codemechanic 15y ago
      It is very relevant to the discussion. If you are not sure check the Tonido architecture.
  - codemechanic 15y ago
    Yes I work for Tonido. There is nothing wrong in saying we do the same stuff. If u think Tonido is irrelevant to the discussion that is your prerogative.
    - zyfo 15y ago
      There's something wrong with blatant self-promotion. Especially considering that you: - don't use a disclaimer - pretend you're not involved in the project by talking about "They" It's a cheap way of trying to game the community to get users.
EGreg 15y ago
freenet and other services already do this. You can already use your browser to browse freenet, if you get the freenet program. They recommend using Chrome in Incognito mode for maximum privacy. And it is impervious to DNS takedowns and you can even set up a darknet. It's used in China a lot. Also Perfect Dark is used. They operate on distributed hash tables. The problem is that without a central server, the only way you can connect to the hive is by hoping one of the last known hosts is still up. It also needs to use heuristics for routing.
- xtacy 15y ago
  Yes, there are a ton of other overlays that do p2p. The advantage of a browser is that no one needs to download/install any software to avail p2p features.
  - EGreg 15y ago
    Well yes, if browser makers create their own version of freenet built in, that would be true.
- fauigerzigerk 15y ago
  The problem I have with freenet is that I don't want to blindly store or transfer other people's encrypted data. I think, if we take some of the powers away from the government, it is all the more important that we are able to make a judgement on the kind of content and communications that we want to support. No single entity should be able to control all access to information, not even a democratic government. But turning the tables completely and make everyone help anyone spread any kind of information can't be the solution. I'm totally aware of the dilemma we're in. Knowing what other people transfer over our machines puts us in a position of "must make judgement and be liable". Not knowing puts us in the position of "cannot make a judgement even if we want to". It's just a difficult problem.
  - EGreg 15y ago
    If you don't store other's people data, who will? If those people store their own data, how will you get it if they disconnect their computer from the network, or it goes down? And more importantly, if their computer is the only place to get the data, then how do you make the "host" of the data untraceable?
benihana 15y ago
>Opera is always several years ahead of its time Except in adoption rates.
- billybob 15y ago
  What do you mean? Opera had Opera's 2011 adoption rates way back in 2009.
codemechanic 15y ago
Freedombox foundation is trying to achieve the same end result.
haberman 15y ago
The W3C is broken: it has spent the last 10 years standardizing ideas instead of existing practice. This is totally backwards and leads to standards that are too complicated, unrealistic, and in many cases not needed at all. The W3C was originally created to standardize HTML, which was already being used by many vendors and users but in incompatible ways. That is exactly the right situation for creating a standard. It leads to standards that are realistic and motivated by a demonstrated need. Unfortunately, almost everything else the W3C has ever done has happened in the opposite direction: in response to an idea or a perceived need, some people theorize about the best way to solve the problem and then write a document that a bunch of vendors are supposed to then implement from scratch. This is how we ended up with the XML stack, which was designed to solve the data interchange problem but ended up being a disaster of complexity, inefficiency, and ad hoc implementations. Even the case of CSS (which has been quite successful) is sub-optimal IMO, because it didn't choose to standardize the existing practice of how people were using tables for layout. The CSS box model makes it stupidly difficult to do things that are trivial with table-based layouts, like a a three column layout (which is considered a "holy grail" by even CSS advocates: http://www.alistapart.com/articles/holygrail/ http://www.alistapart.com/articles/holygrail/). CSS could have used a table-like layout model that makes it easy to arrange <div>s into rows and columns, allowing a smooth upgrade from people who were using the <table> tag. Instead they invented something new that was much more difficult to design for, creating an unnecessary tension between web standards advocates and people who just wanted to get things done. Standards should codify and refine existing practice, not attempt to invent new things.
- jerf 15y ago
  "CSS could have used a table-like layout model that makes it easy to arrange <div>s into rows and columns, allowing a smooth upgrade from people who were using the <table> tag." Like this? http://www.w3.org/TR/CSS2/tables.html http://www.w3.org/TR/CSS2/tables.html
  - haberman 15y ago
    A lot like that, yes, but that was not introduced until CSS2 which is so big and complex that they've spent almost 15 years correcting it (CSS 2.1 keeps bouncing back and forth between "Working Draft" and "Candidate Recommendation"). Despite being first defined in CSS2 (1998) CSS tables were not supported in IE for 10 years (IE8, 2009), and while some of the blame for this is surely Microsoft's, with a standard as big and complex as CSS it's hard to expect implementations to get it right for a long time. To avoid this disaster CSS3 is "modular" and currently has over 40 modules defined! This is madness. The more complicated things get, the more difficult it is to bring the web to new devices or to innovate in the browser space.
jamii 15y ago
I've been thinking about this sort of thing a lot lately. Sugar (http://www.sugarlabs.org/ http://www.sugarlabs.org/) get a lot of things right in terms of p2p user interaction. If you want to, say, edit a document with a friend you just click invite and Sugar will handle everything else. It will even send them a copy of the editor app if they don't have it installed. I think developing small-scale p2p apps (eg IM, 1-1 audio/video, multiplayer-editors ala etherpad) can be made a lot easier than it is today. My rough plan of attack is to use erl-telehash (https://github.com/jamii/erl-telehash https://github.com/jamii/erl-telehash) for addressing and NAT traversal with something similar to bloom (http://www.bloom-lang.net/ http://www.bloom-lang.net/) for coordination / logging / debugging. Add libraries for at-least-once messaging, leader election and operational transform. Maybe piggyback on chromeos or android to get secure p2p app installation. I also had some thoughts about the CALM hypothesis (http://www.bloom-lang.net/calm/ http://www.bloom-lang.net/calm/) which I haven't seen mentioned in the literature. A monotonic bloom program is one in which every delivery order for a given set of messages results in the same state. For many protocols what I actually care about is that every possible end state is equivalent, for some protocol-specific notion of equivalence. For example, for leader election all I care about is that every end state should have exactly one leader. Monotonic programs are easy to model check and I think explicitly stating the desired equivalence relations will reduce the state space explosion at points of order. It might be possible to get good results from a very simple/naive model checker by exploiting this. Not hugely related to the article, but its been on my mind a lot lately.
protagonist_h 15y ago
Flash supports peer-to-peer communication since Flash Player 10 using RTMFP (Real Time Media Flow Protocol).
- Fargren 15y ago
  Isn't that still in beta?
  - protagonist_h 15y ago
    No, in fact it's been available in FP10 since October 2008.
omouse 15y ago
Less corporate/company crap, more independent ideas please. Things like Freenet, GNUNet, etc. have the right idea for decentralization, same with more specific projects like Appleseed, StatusNet, etc.

136 points by kerben 15y ago

As the comments point out, this is not a generic decentralised web standard to get around ICE and the like, but just a specification for p2p audio/video/etc communications for online video conferencing and so on. Not as big as I hoped.

mcrider 15y ago
That's a shame. I was just thinking yesterday how great it would be to be able to host web pages in a P2P fashion (after all, people are building an economy of the bittorrent infrastructure). Sites like WikiLeaks would get a great advantage of this sort of protocol.
- bxr 15y ago
  It sounds like you're looking for the Freenet project. http://en.wikipedia.org/wiki/Freenet http://en.wikipedia.org/wiki/Freenet
fragsworth 15y ago
But if you enable p2p communication between javascript clients, there should be nothing stopping you from implementing that kind of stuff within the browser.

phlux 15y ago

I wonder how easily man-in-the-middle attacks via node spoofing would be. You masquerade as a node by re-hosting their content and you capture any other client that accesses your proxy of that information.

sp332 15y ago
Oh, it's simple! All we need is for each node to sign their content with a private key. Then everyone can check the validity of the content with the originating node's public key, even if it comes from some untrusted intermediary. Hm, now all we need is a centralized server to distribute the public keys securely. But that's a lot of keys (one for every node), so let's use a distributed system... oh wait :)
- aristus 15y ago
  All p2p tamper-proofing schemes reduce to the Byzantine Generals Problem. There exist working algorithms, and the math was figured out a long time ago. There are two wrinkles, though: BGP solutions are kind of slow. Worse, it is impossible for any BGP system to function securely if more than 1/3 of the nodes act in bad faith.
- akdubya 15y ago
  Good background from the bitcoin people: "A number of Byzantine Generals each have a computer and want to attack the King's wi-fi by brute forcing the password, which they've learned is a certain number of characters in length. Once they stimulate the network to generate a packet, they must crack the password within a limited time to break in and erase the logs, lest they be discovered. They only have enough CPU power to crack it fast enough if a majority of them attack at the same time. They don't particularly care when the attack will be, just that they agree. It has been decided that anyone who feels like it will announce an attack time, which we'll call the "plan", and whatever plan is heard first will be the official plan. The problem is that the network is not instantaneous, and if two generals announce different plans at close to the same time, some may hear one first and others hear the other first. They use a proof-of-work chain to solve the problem. Once each general receives whatever plan he hears first, he sets his computer to solve a difficult hash-based proof-of-work problem that includes the plan in its hash. The proof-of-work is difficult enough that with all of them working at once, it's expected to take 10 minutes before one of them finds a solution and broadcasts it to the network. Once received, everyone adjusts the hash in their proof-of-work computation to include the first solution, so that when they find the next proof-of-work, it chains after the first one. If anyone was working on a different plan, they switch to this one, because its proof-of-work chain is now longer. After about two hours, the plan should be hashed by a chain of 12 proofs-of-work. Every general, just by verifying the difficulty of the proof-of-work chain, can estimate how much parallel CPU power per hour was expended on it and see that it must have required the majority of the computers to produce in the allotted time. At the least, most of them had to have seen the plan, since the proof-of-work is proof that they worked on it. If the CPU power exhibited by the proof-of-work is sufficient to crack the password, they can safely attack at the agreed time." http://www.bitcoin.org/byzantine.html http://www.bitcoin.org/byzantine.html

codemechanic 15y ago

Tonido (http://www.tonido.com http://www.tonido.com) is a pioneer in this space. They have invented the model much before opera unit. The cool thing is that Tonido also provides a decentralized openid to end users. Firefox should buy Tonido. It will change the industry if it happens and the way people share information. I probably may be little ahead. If you think deeply it will make sense.

zyfo 15y ago
Do you work for Tonido? If you do, please provide a disclaimer. What you are doing is borderline spamming. See his comments for yourself [0] If I'm wrong, I apologize profusely in advance. EDIT: Parent does work for Tonido [1]. This is spamming. 0: http://news.ycombinator.com/threads?id=codemechanic http://news.ycombinator.com/threads?id=codemechanic 1: http://news.ycombinator.com/item?id=643833 http://news.ycombinator.com/item?id=643833
- fragsworth 15y ago
  Even if you were wrong, I don't think you need to apologize. Almost all of his comments were trying to promote something rather than provide meaningful discussion.
  - codemechanic 15y ago
    It is very relevant to the discussion. If you are not sure check the Tonido architecture.
- codemechanic 15y ago
  Yes I work for Tonido. There is nothing wrong in saying we do the same stuff. If u think Tonido is irrelevant to the discussion that is your prerogative.
  - zyfo 15y ago
    There's something wrong with blatant self-promotion. Especially considering that you: - don't use a disclaimer - pretend you're not involved in the project by talking about "They" It's a cheap way of trying to game the community to get users.

EGreg 15y ago

freenet and other services already do this. You can already use your browser to browse freenet, if you get the freenet program. They recommend using Chrome in Incognito mode for maximum privacy. And it is impervious to DNS takedowns and you can even set up a darknet. It's used in China a lot. Also Perfect Dark is used. They operate on distributed hash tables. The problem is that without a central server, the only way you can connect to the hive is by hoping one of the last known hosts is still up. It also needs to use heuristics for routing.

xtacy 15y ago
Yes, there are a ton of other overlays that do p2p. The advantage of a browser is that no one needs to download/install any software to avail p2p features.
- EGreg 15y ago
  Well yes, if browser makers create their own version of freenet built in, that would be true.
fauigerzigerk 15y ago
The problem I have with freenet is that I don't want to blindly store or transfer other people's encrypted data. I think, if we take some of the powers away from the government, it is all the more important that we are able to make a judgement on the kind of content and communications that we want to support. No single entity should be able to control all access to information, not even a democratic government. But turning the tables completely and make everyone help anyone spread any kind of information can't be the solution. I'm totally aware of the dilemma we're in. Knowing what other people transfer over our machines puts us in a position of "must make judgement and be liable". Not knowing puts us in the position of "cannot make a judgement even if we want to". It's just a difficult problem.
- EGreg 15y ago
  If you don't store other's people data, who will? If those people store their own data, how will you get it if they disconnect their computer from the network, or it goes down? And more importantly, if their computer is the only place to get the data, then how do you make the "host" of the data untraceable?

benihana 15y ago

>Opera is always several years ahead of its time Except in adoption rates.

billybob 15y ago
What do you mean? Opera had Opera's 2011 adoption rates way back in 2009.

codemechanic 15y ago

Freedombox foundation is trying to achieve the same end result.

haberman 15y ago

The W3C is broken: it has spent the last 10 years standardizing ideas instead of existing practice. This is totally backwards and leads to standards that are too complicated, unrealistic, and in many cases not needed at all. The W3C was originally created to standardize HTML, which was already being used by many vendors and users but in incompatible ways. That is exactly the right situation for creating a standard. It leads to standards that are realistic and motivated by a demonstrated need. Unfortunately, almost everything else the W3C has ever done has happened in the opposite direction: in response to an idea or a perceived need, some people theorize about the best way to solve the problem and then write a document that a bunch of vendors are supposed to then implement from scratch. This is how we ended up with the XML stack, which was designed to solve the data interchange problem but ended up being a disaster of complexity, inefficiency, and ad hoc implementations. Even the case of CSS (which has been quite successful) is sub-optimal IMO, because it didn't choose to standardize the existing practice of how people were using tables for layout. The CSS box model makes it stupidly difficult to do things that are trivial with table-based layouts, like a a three column layout (which is considered a "holy grail" by even CSS advocates: http://www.alistapart.com/articles/holygrail/ http://www.alistapart.com/articles/holygrail/). CSS could have used a table-like layout model that makes it easy to arrange <div>s into rows and columns, allowing a smooth upgrade from people who were using the <table> tag. Instead they invented something new that was much more difficult to design for, creating an unnecessary tension between web standards advocates and people who just wanted to get things done. Standards should codify and refine existing practice, not attempt to invent new things.

jerf 15y ago
"CSS could have used a table-like layout model that makes it easy to arrange <div>s into rows and columns, allowing a smooth upgrade from people who were using the <table> tag." Like this? http://www.w3.org/TR/CSS2/tables.html http://www.w3.org/TR/CSS2/tables.html
- haberman 15y ago
  A lot like that, yes, but that was not introduced until CSS2 which is so big and complex that they've spent almost 15 years correcting it (CSS 2.1 keeps bouncing back and forth between "Working Draft" and "Candidate Recommendation"). Despite being first defined in CSS2 (1998) CSS tables were not supported in IE for 10 years (IE8, 2009), and while some of the blame for this is surely Microsoft's, with a standard as big and complex as CSS it's hard to expect implementations to get it right for a long time. To avoid this disaster CSS3 is "modular" and currently has over 40 modules defined! This is madness. The more complicated things get, the more difficult it is to bring the web to new devices or to innovate in the browser space.

jamii 15y ago

I've been thinking about this sort of thing a lot lately. Sugar (http://www.sugarlabs.org/ http://www.sugarlabs.org/) get a lot of things right in terms of p2p user interaction. If you want to, say, edit a document with a friend you just click invite and Sugar will handle everything else. It will even send them a copy of the editor app if they don't have it installed. I think developing small-scale p2p apps (eg IM, 1-1 audio/video, multiplayer-editors ala etherpad) can be made a lot easier than it is today. My rough plan of attack is to use erl-telehash (https://github.com/jamii/erl-telehash https://github.com/jamii/erl-telehash) for addressing and NAT traversal with something similar to bloom (http://www.bloom-lang.net/ http://www.bloom-lang.net/) for coordination / logging / debugging. Add libraries for at-least-once messaging, leader election and operational transform. Maybe piggyback on chromeos or android to get secure p2p app installation. I also had some thoughts about the CALM hypothesis (http://www.bloom-lang.net/calm/ http://www.bloom-lang.net/calm/) which I haven't seen mentioned in the literature. A monotonic bloom program is one in which every delivery order for a given set of messages results in the same state. For many protocols what I actually care about is that every possible end state is equivalent, for some protocol-specific notion of equivalence. For example, for leader election all I care about is that every end state should have exactly one leader. Monotonic programs are easy to model check and I think explicitly stating the desired equivalence relations will reduce the state space explosion at points of order. It might be possible to get good results from a very simple/naive model checker by exploiting this. Not hugely related to the article, but its been on my mind a lot lately.

protagonist_h 15y ago

Flash supports peer-to-peer communication since Flash Player 10 using RTMFP (Real Time Media Flow Protocol).

Fargren 15y ago
Isn't that still in beta?
- protagonist_h 15y ago
  No, in fact it's been available in FP10 since October 2008.

omouse 15y ago

Less corporate/company crap, more independent ideas please. Things like Freenet, GNUNet, etc. have the right idea for decentralization, same with more specific projects like Appleseed, StatusNet, etc.