5 ms·

This is what Feross and I have been working on the past few weeks. It's been a lot of fun. Here are a few of the things I learned: * Electron is surprisingly n

by dcposch 11y ago

This is what Feross and I have been working on the past few weeks. It's been a lot of fun. Here are a few of the things I learned: * Electron is surprisingly nice to work with. It is simple, pragmatic, and thorough. System tray integrations, desktop notifications, launcher progress bar in OSX, installers for all three platforms, seamless auto updaters for Mac and Windows, a crash reporter, and so on --- all built in. Many of the little things that an app needs to feel native and professional have clean cross-platform APIs. * Shout out to Cheng Zhao, aka zcbenz. He's incredibly prolific and seems to be nearly single-handedly developing the Electron project. Every single page of Electron docs or code I've looked at on Github, the latest commit on it is zcbenz. * Peer-to-peer is moving very fast. WebRTC support in browsers was flaky and buggy until very recently. Even today, Chrome has a few big bugs left ( eg https://bugs.chromium.org/p/chromium/issues/detail?id=392651 https://bugs.chromium.org/p/chromium/issues/detail?id=392651 ) and WebRTC is not yet supported in Web Workers ( https://bugs.chromium.org/p/chromium/issues/detail?id=302019 https://bugs.chromium.org/p/chromium/issues/detail?id=302019 ). But things are improving quickly. Edge is getting Data Channel support soon. Safari is almost certainly gonna get WebRTC support as well -- it's already in WebKit, just not enabled in Safari yet. Tons of cool new uses of the web are just at the edge of becoming possible. I'm really excited about the way WebTorrent Desktop bridges the WebTorrent and BitTorrent networks. If enough people end up running it, BitTorrent in the browser will really happen. Projects like the Internet Archive can make huge files available for immediate streaming on their website, without paying for bandwidth. It's going to be interesting times.

terhechte 11y ago
Just wondering, because it is not explicitly mentioned in the docs, does it only stream or does it also seed the downloaded chunks?
- feross 11y ago
  It seeds, like all good torrent clients.
- dcposch 11y ago
  Yes. All torrent clients I know, including WebTorrent Desktop, seed while downloading. WebTorrent Desktop keeps seeding after you've finished downloading a file, and keeps running in the system tray after you close the window unless you explicitly tell it to quit. We want to be good citizens and keep the swarms healthy.
  - terhechte 11y ago
    Absolutely. I'm just asking because in certain jurisdictions seeding (illegal content) will lead to huge fines, and not mentioning this on the homepage might lead people to believe that they can safely use this software. This is especially worrying since "streaming" is oftentimes understood as a read-only operation which cannot be pursued by law enforcement.
    - douche 11y ago
      The general rule of thumb that seems to keep you under the radar is to limit your upload rate. Most clients have an easy option to do this - although apparently this is non-trivial in this client[1]? [1] https://github.com/feross/webtorrent/issues/163 https://github.com/feross/webtorrent/issues/163
educar 11y ago
I didn't realize zcbenz was involved. He was also a big contributor to node-webkit.
dcposch 11y ago
One more: * It's surprisingly fun and liberating working on a completely noncommercial project. I have no problem at all with commercial software -- especially if it's free of ads and "microtransactions" and is simply sold to the user for a clear up-front price. That said, it's fun just shipping something that anybody can work on, with no boss, no formal user requirements, no worries about whether the app will sell or whether the client will pay you. The only goal here is to make something we're proud of and which users enjoy.
- feross 11y ago
  And... one more: * It's really fun to share project ownership with all the contributors. From CONTRIBUTING.md: > Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project. (https://github.com/feross/webtorrent-desktop/blob/master/CONTRIBUTING.md#project-governance https://github.com/feross/webtorrent-desktop/blob/master/CON...) The first time I witnessed this in action was on the LevelUp project (https://github.com/Level/levelup https://github.com/Level/levelup). Since the days of the iojs fork, Node.js itself has adopted the same model. Sharing commit access liberally is awesome for project health, because: - The project can move forward even when the project owner is busy. - It empowers contributors. It makes them feel like owners, because, well, they are. They end up contributing a lot more code. - In the worst cast, it's Git, so bad commits can be backed out. But I've NEVER had to do this, or ever seen anyone else need to do it.
cloudjacker 11y ago
hey there! One feature that XBMC (now called Kodi) had since 2004 but I haven't seen anywhere else, is the ability to play the contents of rar files. So you could download movies and never have to bother unzipping them. It worked for multipart rars too. The lore is that they implemented a custom rar algorithm but anyway, that was 14 years ago and its an open source project so I should see this somewhere else by now! Since you guys are doing something over the torrent network where these kind of video releases are common (multipart rar, reminiscent of the release group's newsgroup roots), could you implement this functionality too: When streaming, discover the sequence of the rar files in the torrent, prioritize the file downloads by rar number, and play the video file within?
- ferongr 11y ago
  This is an incredibly niche feature today because no releases come packaged in archives. Furthermore, to my knowledge you can't really "stream" archives.
  - jaaames 11y ago
    They still do come packaged directly from scene releases. Smaller private torrent trackers have these releases. Public trackers usually contain just the media files so people don't have to unzip. But I agree, playing from archives would be a great feature.
  - ChrisClark 11y ago
    I'd say the majority still are split into rars for the initial scene release but by the time it comes to the public trackers, and even many private trackers, someone has already unrared it for you.
- cgb_ 11y ago
  I've implemented the same thing in a Usenet/NZB streaming project a few years ago - it's actually trivial since the multi-part rars are uncompressed, so the raw bytes of inner-content are there ready to be read. Just have to handle RAR headers/trailers. Seeking worked too since Usenet posts were commonly made up of 700kb posts, so if a seek occurred, you prioritize retrieving the required posts where the user seeked to, to fill the media player buffer. Code has never been released for obvious reasons :)
k_lander 11y ago
Is Adaptive Bitrate Streaming possible?
- dspillett 11y ago
  I would have thought not. Well, it would be possible but only if the video format in the torrent allowed for this is a way compatible with the way torrent transfers are split into blocks - i.e. it has a block structure that lines up with the torrent block structure such that block X is enough to play a given set of frames at the lowest quality, block X+1 for better quality, and so forth. The blocks wouldn't have to be sequential: as torrent transfers are essentially random access you could have the content required for lowest quality display at the start of the file and the extra data further on, similar to how FLIF supports almost immediate low quality with progressive improvement as the rest of the file arrives (http://flif.info/ http://flif.info/ - see the video demo part way down that page, or https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ for a more interactive look at how it copes with truncated files). I'm not aware of any video formats that are arranged this way though. Those that are designed for adaptive streaming have a more dynamic structure that would not align well with the torrent protocols blocks.
- mbilker 11y ago
  Adaptive Bitrate Streaming works by having multiple copies of the source video at different bitrates, resolutions, and audio qualities. Torrents are usually one file, which would not allow Adaptive Bitrate Streaming.

by dcposch 11y ago

terhechte 11y ago

Just wondering, because it is not explicitly mentioned in the docs, does it only stream or does it also seed the downloaded chunks?

feross 11y ago
It seeds, like all good torrent clients.
dcposch 11y ago
Yes. All torrent clients I know, including WebTorrent Desktop, seed while downloading. WebTorrent Desktop keeps seeding after you've finished downloading a file, and keeps running in the system tray after you close the window unless you explicitly tell it to quit. We want to be good citizens and keep the swarms healthy.
- terhechte 11y ago
  Absolutely. I'm just asking because in certain jurisdictions seeding (illegal content) will lead to huge fines, and not mentioning this on the homepage might lead people to believe that they can safely use this software. This is especially worrying since "streaming" is oftentimes understood as a read-only operation which cannot be pursued by law enforcement.
  - douche 11y ago
    The general rule of thumb that seems to keep you under the radar is to limit your upload rate. Most clients have an easy option to do this - although apparently this is non-trivial in this client[1]? [1] https://github.com/feross/webtorrent/issues/163 https://github.com/feross/webtorrent/issues/163

educar 11y ago

I didn't realize zcbenz was involved. He was also a big contributor to node-webkit.

dcposch 11y ago

One more: * It's surprisingly fun and liberating working on a completely noncommercial project. I have no problem at all with commercial software -- especially if it's free of ads and "microtransactions" and is simply sold to the user for a clear up-front price. That said, it's fun just shipping something that anybody can work on, with no boss, no formal user requirements, no worries about whether the app will sell or whether the client will pay you. The only goal here is to make something we're proud of and which users enjoy.

feross 11y ago
And... one more: * It's really fun to share project ownership with all the contributors. From CONTRIBUTING.md: > Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project. (https://github.com/feross/webtorrent-desktop/blob/master/CONTRIBUTING.md#project-governance https://github.com/feross/webtorrent-desktop/blob/master/CON...) The first time I witnessed this in action was on the LevelUp project (https://github.com/Level/levelup https://github.com/Level/levelup). Since the days of the iojs fork, Node.js itself has adopted the same model. Sharing commit access liberally is awesome for project health, because: - The project can move forward even when the project owner is busy. - It empowers contributors. It makes them feel like owners, because, well, they are. They end up contributing a lot more code. - In the worst cast, it's Git, so bad commits can be backed out. But I've NEVER had to do this, or ever seen anyone else need to do it.

cloudjacker 11y ago

hey there! One feature that XBMC (now called Kodi) had since 2004 but I haven't seen anywhere else, is the ability to play the contents of rar files. So you could download movies and never have to bother unzipping them. It worked for multipart rars too. The lore is that they implemented a custom rar algorithm but anyway, that was 14 years ago and its an open source project so I should see this somewhere else by now! Since you guys are doing something over the torrent network where these kind of video releases are common (multipart rar, reminiscent of the release group's newsgroup roots), could you implement this functionality too: When streaming, discover the sequence of the rar files in the torrent, prioritize the file downloads by rar number, and play the video file within?

ferongr 11y ago
This is an incredibly niche feature today because no releases come packaged in archives. Furthermore, to my knowledge you can't really "stream" archives.
- jaaames 11y ago
  They still do come packaged directly from scene releases. Smaller private torrent trackers have these releases. Public trackers usually contain just the media files so people don't have to unzip. But I agree, playing from archives would be a great feature.
- ChrisClark 11y ago
  I'd say the majority still are split into rars for the initial scene release but by the time it comes to the public trackers, and even many private trackers, someone has already unrared it for you.
cgb_ 11y ago
I've implemented the same thing in a Usenet/NZB streaming project a few years ago - it's actually trivial since the multi-part rars are uncompressed, so the raw bytes of inner-content are there ready to be read. Just have to handle RAR headers/trailers. Seeking worked too since Usenet posts were commonly made up of 700kb posts, so if a seek occurred, you prioritize retrieving the required posts where the user seeked to, to fill the media player buffer. Code has never been released for obvious reasons :)

k_lander 11y ago

Is Adaptive Bitrate Streaming possible?

dspillett 11y ago
I would have thought not. Well, it would be possible but only if the video format in the torrent allowed for this is a way compatible with the way torrent transfers are split into blocks - i.e. it has a block structure that lines up with the torrent block structure such that block X is enough to play a given set of frames at the lowest quality, block X+1 for better quality, and so forth. The blocks wouldn't have to be sequential: as torrent transfers are essentially random access you could have the content required for lowest quality display at the start of the file and the extra data further on, similar to how FLIF supports almost immediate low quality with progressive improvement as the rest of the file arrives (http://flif.info/ http://flif.info/ - see the video demo part way down that page, or https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ for a more interactive look at how it copes with truncated files). I'm not aware of any video formats that are arranged this way though. Those that are designed for adaptive streaming have a more dynamic structure that would not align well with the torrent protocols blocks.
mbilker 11y ago
Adaptive Bitrate Streaming works by having multiple copies of the source video at different bitrates, resolutions, and audio qualities. Torrents are usually one file, which would not allow Adaptive Bitrate Streaming.