9 ms·

Peer-To-peer Architecture Could Save Streaming Video

76 points by ericaeb 12y ago

jacquesm 12y ago
We had peer to peer streaming 18 years ago. It does not scale for larger audiences, so if it stays on the 'skype' level then it will do fine, otherwise you're going to need some kind of redistribution network to handle the fan-out. If you're willing to tolerate lag and decreasing fidelity you can also use peer-to-peer relaying but this only works for a very limited number of hops.
- latch 12y ago
  I'm also skeptical. Off the top of my head, if I had to make this work, I might try two things: 1 - While streaming normally from the CDN, also preload segments from peers. Complexity and client-bloat aside, you don't lose much trying (assuming the client has the bandwidth) 2 - Peering with folk that appear to be close. I feel that doing it by network address or ip->geo lookup might work well.
  - Rodi 12y ago
    Hey, founder of the p2p service described in the article here (www.streamroot.io) This is exactly what we are doing :) We have have a hybrid p2p model that fetches video segments from others peers as long as it can, and switches back to CDN on urgency cases. And our tracker uses geoIp data, ISP and some network topology elements to connect the best candidates together
dracolytch 12y ago
So... I'm conflicted about this. Sure, on one hand peer-to-peer streaming could solve some of the problems she's talking about, so long as there's a large enough audience for each video. Services like Steam have used P2P very effectively. At times it can break down to a very frustrating level quickly, so you'll still want CDNs around to act as seeds. It also doesn't work for live broadcasts which is becoming more common with the likes of Twitch. Also, the current technologies being discussed (marketed?) would require a plugin to work. I don't think this will fly until a browser-native solution is engineered.
- higherpurpose 12y ago
  > so long as there's a large enough audience for each video Completely a non-issue. The providers would "seed" all the videos from their own servers, too, so when there aren't enough peers seeding it, the bulk of the streaming would be offered by the provider. Using the Pareto principle, something like 80 percent of the videos would be "long-tail" with few to no seeds (other than from the provider), but only use 20 percent of the video traffic (so the provider's burden is greatly reduced anyway). The other 20 percent videos would represent 80 percent of the traffic, and those are the videos that would be helped most by P2P streaming.
dexen 12y ago
A whole article on p2p video streaming, and yet Popcorn Time [1] is curiously absent. The flagship program that proves p2p movie watching is feasible, even comfortable, when delivered through BitTorrent doesn't get a mention. In a way the program already saved streaming video already. We can only speculate whether the omission is conscious attempt at avoiding linking to risque programs, or does that indicate the article is copy-pasted press release of content providers. [1] https://en.wikipedia.org/wiki/Popcorn_Time https://en.wikipedia.org/wiki/Popcorn_Time ; https://popcorntime.io/ https://popcorntime.io/ seems to be the leading fork.
- rubiquity 12y ago
  That is probably due to this article being about WebRTC and not Torrents.
- Rodi 12y ago
  The main difference with the solution the article describes and Popcorn time is that the former can be used by the online broadcaster for their own videos and integrates seamlessly in their webpage, whereas for Popcorn time you need to install the soft, and can only download torrents.
jasode 12y ago
>Instead of one endpoint pulling down a single unique stream, the P2P approach allows simultaneous users to exchange video segments among themselves rather than each connecting to a server to do so. Issues: -- residential/mobile internet service has asymmetric upload/download speeds. -- ISPs are suspicious of heavy uploads originating from residential homes and could be flagged as "servers" in violation of ISP's TOS or reclassified as "business tier pricing" thereby increasing the billing amount. -- too much latency for live mega events (World Cup, etc) because of asymmetry mentioned above. (Your neighbor tweets that Spain scored the winning goal before your P2P stream received the last packets showing that it happened.) Latency for bittorrent is ok, but for live megaevents, no. I don't think P2P down to residential is realistic. However, P2P between commercial entities looks more workable. If Akamai projects that their CDNs can't handle the next mega event, they contract with Amazon AWS CDNs to handle some of the spillover bandwidth. And/or work with ISPs (Comcast, FiOS edge servers) to act as CDNs. Yes, it involves competitors partnering with each other but this looks more feasible than coordinating residential internet service to deliver the extra bandwidth. As an analogy, residential neighbors don't transfer supplemental electricity to each other but power stations in Canada might supply extra peak electricity to power stations in New York.
- rakoo 12y ago
  > I don't think P2P down to residential is realistic. However, P2P between commercial entities looks more workable. The nice thing with P2P is that both model are not mutually exclusive. You can have a single peer that is effectively served by Akamai's CDN, and you can have another single peer that is served by you behind your poor ISP line; the protocol will work the same and automatically select the one that offers the better bandwith -- so people will in practice fetch from Akamai's CDN when it has capacity, and automatically switch to anybody else when Akamai alone isn't enough. You don't even need any logic, it will be handled automatically.
- higherpurpose 12y ago
  You're thinking backwards of the problem. The ISPs don't offer high upload speeds because there's no demand for it. If there was, they would be forced to raise those speeds (in a competitive market, at least). Google Fiber offers symmetrical 1Gbps for download and upload speeds.
natar 12y ago
I'm not sure if that's going to work when more and more people are using mobile internet. Sure, we can all hope for real mobile flatrates but p2p videodistribution will add more traffic the networks … wait, no it wouldn't, would it?
tootie 12y ago
I don't understand the benefit of P2P when CDNs like Akamai are already distributing load across their massive network of servers.
- sylvinus 12y ago
  A smaller Akamai bill?
  - justincormack 12y ago
    Will I see the cost saving of handing over my bandwidth?
KaiserPro 12y ago
Sorry, but no. For a p2p system to work you need to have a nearest neighbour finder(please wait 5 minutes whilst we probe your network) you then need to have a realtime "rationilser" that will go through and re-route the live stream according to how many peers in one segment are connected to another. Then, you need a number of backup peers for when your current master peer disappears. Then you have to deal with partition, and also make sure that NAT doesn't get in the way. Also you'll need to block mobile phones as well. the best part is, you'll be forced to use HVEC as the usable upload is around 2 megabits. That means a stream encoded at 750k (to allow uploading to two other users) or half that if you want 4 other peers. So no, P2P is not the answer. Iterate again.
- sylvinus 12y ago
  Did you read the article? Your points are irrelevant because it's not 100% P2P, they say they reach ~58% of P2P use, while the rest comes from regular CDNs.
  - KaiserPro 12y ago
    yes, I read the article, I also note that they were using a 1 meg stream. They didn't describe the jitter, dropout rate, cache misses and eventual fall back. For example was that 58% peak? was that 58% attempted p2p use? what was the threshold? did the users notice? what happens when peers go away? does that mean that create burst loads on the cache master? What is the success rate when they stream 4meg, 8meg? or a bitrate that changes significantly? for example going from a semi static image to fast moving sports (think transition from commentator studio to live action) Having dealt with these types of systems in the bad old days (think 2007-9) I can tell you that they suck hard. All that engineering work required to patch over a under resourced and terribly unreliable transport, when many a pre-built system with SLAs already exists
- alecco 12y ago
  That can be cached if there's only one system. And BitTorrent DHT takes about 30s to start finding peers. That's a very old type of DHT, a new protocol could prioritize finding closer neighbors.
diydsp 12y ago
Note that this article claiming online video streaming needs to be saved is on a WebRTC marketing site "Powered by Technology Marketing Corp" advocating a competing technology. I don't think online video streaming needs to be saved. > There’s a clear event horizon where delivery overhead outstrips even CDN capacity. If it's so clear, please enlighten me: All I see is network architecture making ever bolder jumps to increase capacity... Look at Amazon's multiple locations, look at fiber-to-the-home, look at the server farms around the country. Look at the bold instantiation of cell towers becoming ever-denser. I think if the public demands 150 million individual 10Mbps streams, it will be built. Not that WebRTC isn't cool for what it is, but a soccer game isn't a volcano (10 people punting a ball/data around like WebRTC isn't the same as a massive one-way structure of energy/data like a Volcano).
est 12y ago
P2p video streaming has been used in China for over a decade now. from earliest synacast to Flash p2p
williamcotton 12y ago
WebTorrent is a streaming torrent client for node.js and the browser. https://github.com/feross/webtorrent https://github.com/feross/webtorrent It can stream video torrents into a <video> tag (webm (vp8, vp9) or mp4 (h.264))

76 points by ericaeb 12y ago

We had peer to peer streaming 18 years ago. It does not scale for larger audiences, so if it stays on the 'skype' level then it will do fine, otherwise you're going to need some kind of redistribution network to handle the fan-out. If you're willing to tolerate lag and decreasing fidelity you can also use peer-to-peer relaying but this only works for a very limited number of hops.

latch 12y ago
I'm also skeptical. Off the top of my head, if I had to make this work, I might try two things: 1 - While streaming normally from the CDN, also preload segments from peers. Complexity and client-bloat aside, you don't lose much trying (assuming the client has the bandwidth) 2 - Peering with folk that appear to be close. I feel that doing it by network address or ip->geo lookup might work well.
- Rodi 12y ago
  Hey, founder of the p2p service described in the article here (www.streamroot.io) This is exactly what we are doing :) We have have a hybrid p2p model that fetches video segments from others peers as long as it can, and switches back to CDN on urgency cases. And our tracker uses geoIp data, ISP and some network topology elements to connect the best candidates together

dracolytch 12y ago

So... I'm conflicted about this. Sure, on one hand peer-to-peer streaming could solve some of the problems she's talking about, so long as there's a large enough audience for each video. Services like Steam have used P2P very effectively. At times it can break down to a very frustrating level quickly, so you'll still want CDNs around to act as seeds. It also doesn't work for live broadcasts which is becoming more common with the likes of Twitch. Also, the current technologies being discussed (marketed?) would require a plugin to work. I don't think this will fly until a browser-native solution is engineered.

higherpurpose 12y ago
> so long as there's a large enough audience for each video Completely a non-issue. The providers would "seed" all the videos from their own servers, too, so when there aren't enough peers seeding it, the bulk of the streaming would be offered by the provider. Using the Pareto principle, something like 80 percent of the videos would be "long-tail" with few to no seeds (other than from the provider), but only use 20 percent of the video traffic (so the provider's burden is greatly reduced anyway). The other 20 percent videos would represent 80 percent of the traffic, and those are the videos that would be helped most by P2P streaming.

dexen 12y ago

A whole article on p2p video streaming, and yet Popcorn Time [1] is curiously absent. The flagship program that proves p2p movie watching is feasible, even comfortable, when delivered through BitTorrent doesn't get a mention. In a way the program already saved streaming video already. We can only speculate whether the omission is conscious attempt at avoiding linking to risque programs, or does that indicate the article is copy-pasted press release of content providers. [1] https://en.wikipedia.org/wiki/Popcorn_Time https://en.wikipedia.org/wiki/Popcorn_Time ; https://popcorntime.io/ https://popcorntime.io/ seems to be the leading fork.

rubiquity 12y ago
That is probably due to this article being about WebRTC and not Torrents.
Rodi 12y ago
The main difference with the solution the article describes and Popcorn time is that the former can be used by the online broadcaster for their own videos and integrates seamlessly in their webpage, whereas for Popcorn time you need to install the soft, and can only download torrents.

jasode 12y ago

>Instead of one endpoint pulling down a single unique stream, the P2P approach allows simultaneous users to exchange video segments among themselves rather than each connecting to a server to do so. Issues: -- residential/mobile internet service has asymmetric upload/download speeds. -- ISPs are suspicious of heavy uploads originating from residential homes and could be flagged as "servers" in violation of ISP's TOS or reclassified as "business tier pricing" thereby increasing the billing amount. -- too much latency for live mega events (World Cup, etc) because of asymmetry mentioned above. (Your neighbor tweets that Spain scored the winning goal before your P2P stream received the last packets showing that it happened.) Latency for bittorrent is ok, but for live megaevents, no. I don't think P2P down to residential is realistic. However, P2P between commercial entities looks more workable. If Akamai projects that their CDNs can't handle the next mega event, they contract with Amazon AWS CDNs to handle some of the spillover bandwidth. And/or work with ISPs (Comcast, FiOS edge servers) to act as CDNs. Yes, it involves competitors partnering with each other but this looks more feasible than coordinating residential internet service to deliver the extra bandwidth. As an analogy, residential neighbors don't transfer supplemental electricity to each other but power stations in Canada might supply extra peak electricity to power stations in New York.

rakoo 12y ago
> I don't think P2P down to residential is realistic. However, P2P between commercial entities looks more workable. The nice thing with P2P is that both model are not mutually exclusive. You can have a single peer that is effectively served by Akamai's CDN, and you can have another single peer that is served by you behind your poor ISP line; the protocol will work the same and automatically select the one that offers the better bandwith -- so people will in practice fetch from Akamai's CDN when it has capacity, and automatically switch to anybody else when Akamai alone isn't enough. You don't even need any logic, it will be handled automatically.
higherpurpose 12y ago
You're thinking backwards of the problem. The ISPs don't offer high upload speeds because there's no demand for it. If there was, they would be forced to raise those speeds (in a competitive market, at least). Google Fiber offers symmetrical 1Gbps for download and upload speeds.

natar 12y ago

I'm not sure if that's going to work when more and more people are using mobile internet. Sure, we can all hope for real mobile flatrates but p2p videodistribution will add more traffic the networks … wait, no it wouldn't, would it?

tootie 12y ago

I don't understand the benefit of P2P when CDNs like Akamai are already distributing load across their massive network of servers.

sylvinus 12y ago
A smaller Akamai bill?
- justincormack 12y ago
  Will I see the cost saving of handing over my bandwidth?

KaiserPro 12y ago

Sorry, but no. For a p2p system to work you need to have a nearest neighbour finder(please wait 5 minutes whilst we probe your network) you then need to have a realtime "rationilser" that will go through and re-route the live stream according to how many peers in one segment are connected to another. Then, you need a number of backup peers for when your current master peer disappears. Then you have to deal with partition, and also make sure that NAT doesn't get in the way. Also you'll need to block mobile phones as well. the best part is, you'll be forced to use HVEC as the usable upload is around 2 megabits. That means a stream encoded at 750k (to allow uploading to two other users) or half that if you want 4 other peers. So no, P2P is not the answer. Iterate again.

sylvinus 12y ago
Did you read the article? Your points are irrelevant because it's not 100% P2P, they say they reach ~58% of P2P use, while the rest comes from regular CDNs.
- KaiserPro 12y ago
  yes, I read the article, I also note that they were using a 1 meg stream. They didn't describe the jitter, dropout rate, cache misses and eventual fall back. For example was that 58% peak? was that 58% attempted p2p use? what was the threshold? did the users notice? what happens when peers go away? does that mean that create burst loads on the cache master? What is the success rate when they stream 4meg, 8meg? or a bitrate that changes significantly? for example going from a semi static image to fast moving sports (think transition from commentator studio to live action) Having dealt with these types of systems in the bad old days (think 2007-9) I can tell you that they suck hard. All that engineering work required to patch over a under resourced and terribly unreliable transport, when many a pre-built system with SLAs already exists
alecco 12y ago
That can be cached if there's only one system. And BitTorrent DHT takes about 30s to start finding peers. That's a very old type of DHT, a new protocol could prioritize finding closer neighbors.

diydsp 12y ago

Note that this article claiming online video streaming needs to be saved is on a WebRTC marketing site "Powered by Technology Marketing Corp" advocating a competing technology. I don't think online video streaming needs to be saved. > There’s a clear event horizon where delivery overhead outstrips even CDN capacity. If it's so clear, please enlighten me: All I see is network architecture making ever bolder jumps to increase capacity... Look at Amazon's multiple locations, look at fiber-to-the-home, look at the server farms around the country. Look at the bold instantiation of cell towers becoming ever-denser. I think if the public demands 150 million individual 10Mbps streams, it will be built. Not that WebRTC isn't cool for what it is, but a soccer game isn't a volcano (10 people punting a ball/data around like WebRTC isn't the same as a massive one-way structure of energy/data like a Volcano).

est 12y ago

P2p video streaming has been used in China for over a decade now. from earliest synacast to Flash p2p

williamcotton 12y ago

WebTorrent is a streaming torrent client for node.js and the browser. https://github.com/feross/webtorrent https://github.com/feross/webtorrent It can stream video torrents into a <video> tag (webm (vp8, vp9) or mp4 (h.264))