5 ms·

The peer-to-peer storage and sharing of video files isn't the hard part: it's the ability to search for and discover content that makes YouTube compelling in my

by mikece 7y ago

The peer-to-peer storage and sharing of video files isn't the hard part: it's the ability to search for and discover content that makes YouTube compelling in my opinion. Are any of these p2p YouTube replacements presenting a compelling way to search/discover content in a distributed way? Or if the search index is centralized, do any of them have a compelling model for staying funded without selling everyone's search/catalog information to data brokers?

kauffj 7y ago
You could look at LBRY (https://lbry.tech https://lbry.tech https://lbry.com https://lbry.com https://beta.lbry.tv https://beta.lbry.tv). LBRY is similar to PeerTube in open and decentralization, but all content metadata is written to a blockchain, which means everyone/anyone can access the index. This blockchain can then be searched (https://github.com/lbryio/lighthouse https://github.com/lbryio/lighthouse) or extracted to SQL (https://github.com/lbryio/chainquery https://github.com/lbryio/chainquery).
- deleted 7y ago
  [deleted]
- olah_1 7y ago
  Sadly, LBRY isn't able to track view counts. Which makes organic, network effects really difficult. Maybe they could implement a naive "view tracking" by having the client do small proof of work when they interact with content? Similar to voting in NotaBug? https://github.com/notabugio/notabug https://github.com/notabugio/notabug
jasode 7y ago
> Are any of these p2p YouTube replacements presenting a compelling way to search/discover content in a distributed way? Being "distributed" over p2p/federated architecture opposes the end-user convenience of search/discovery/ranking/recommendations because of speed-of-light limitations. I wrote a previous comment about this: https://news.ycombinator.com/item?id=17578332 https://news.ycombinator.com/item?id=17578332 Also, the previous last reply by posting2fast of a partial-centralized server doesn't really replace Youtube because his proposed idea creates new problems of spam videos and untrusted/fake videos. E.g. the central index of metadata says "www.johndoehomeserver.com" has a tutorial video for Algebra but when you actually stream the video from "johndoehomeserver.com", you get a spam video for Viagra instead of math instruction. Therefore, users will naturally gravitate toward the centralized servers that have both the metadata and the actual video content. This emergent group behavior of preferences would end up recreating another "Youtube"-like clone. p2p architecture and torrents works well for things like pirated Photoshop or ripped Marvel Avengers movies because the users already have the content's title _preloaded_ in their brain and therefore a centralized index for discovery/serendipity of unknown content isn't necessary.
- deleted 7y ago
  [deleted]
- tiborsaas 7y ago
  It does not necessarily oppose it. All there needs to be done is to expose a static, daily generated JSON file that contains all videos on the instance. This has nothing to do with the speed of light. Anyone then could build a search index and build a good search experience. To combat spam, instances should reveal up/downvotes to indicate quality, I guess your fake math video would not get much love from the community.
  - jasode 7y ago
    >This has nothing to do with the speed of light. Please take extra care to correctly parse what I actually wrote in response to the gp. Yes, speed-of-light is still a limitation based on the gp's constraint of "search/discovery in a _distributed_ way" which means the search algorithm avoids central servers and loops through a bunch of remote p2p nodes to parse a bunch of exposed JSON manifest files. If instead, the search algorithm loops through data in a cached index server, that's no longer "search in a distributed way" that the gp was originally wondering about. That's the particular point I was responding to. >Anyone then could build a search index and build a good search experience. Now, as to the issue with that "cache index server" that pre-parses the JSON files... The cache server that also contains the actual video data will naturally attract the most users because when they hit the "play" button on their smartphone, the video starts immediately instead of waiting or suffering stuttering from somebody's flakey home video server. So, the index server with the "good experience" as perceived by users will be the one that also includes the actual videos -- basically acts as a CDN -- and this emergent behavior of user preferences defeats the decentralized ideals of p2p video. We see that p2p of things like illegal software already works and is proven. However, p2p of mainstream videos has massive technical hurdles that oppose how typical users like to discover content and play them with immediate gratification.
    - SamBam 7y ago
      You're using "search/discovery in a distributed way" in a very literal manner. I interpreted the question as being one much more meaningful to the vast majority of users: "can a user search for content across the various distributed servers?" That's all an end-user cares about. Indexing videos once a day (or once an hour or whatever) would be very feasible. Indeed, different servers could create their own indexes, and some might be better at sorting for relevance than others.
dmos62 7y ago
Well that's the real problem when you start competing with Google and the like. They're data behemoths. As long as you don't have as much data collected, they'll have a significant advantage. The solution to this is really simple and hardly conceivable at the same time: make competitively important data public. In this case, that's anonymized video browsing/viewing data. Market competition would flourish, the big guys would lose the monopoly.
- pault 7y ago
  But then who will be incentivized to gather the data?
  - dmos62 7y ago
    Anyone who needs more, I guess? Or, is the premise that gathering data is resource intensive (like innovation and patents for example)? These aren't big problems, I think. The problem is that the people who hold the monopolies on data at the moment are also the people who are extremely powerful lobyists.
- boomboomsubban 7y ago
  >In this case, that's anonymized video browsing/viewing data So the view count publicly available and something like the Google search analytics already available? Personalized data offers a huge competitive advantage.
  - dmos62 7y ago
    Personalized (as in per user) and anonymized are not mutually exclusive.
    - boomboomsubban 7y ago
      Sure, but non-anonymized personalized data still provides a competitive advantage.
arkh 7y ago
> discover content Discover heavily sponsored content from content farms. As an experiment, even with an old account, start just browsing the content Youtube highlights. You will soon end with a recommendation page full of shit with 500K+ views using the same template.
- glennpratt 7y ago
  I don't understand your point; garbage in, garbage out. I have a very old account and browse suggestions using my brain, not randomly. While not perfect, almost every recommendation right now looks like something I could watch. Bicycling, civil engineering, cat toys, and weird metal music mashups, which are all similar to things I intentionally watched, but haven't watched.
velcrovan 7y ago
> Are any of these p2p YouTube replacements presenting a compelling way to search/discover content in a distributed way? Ideally we’d see blogs return as the curated content mediators.

by mikece 7y ago

kauffj 7y ago

You could look at LBRY (https://lbry.tech https://lbry.tech https://lbry.com https://lbry.com https://beta.lbry.tv https://beta.lbry.tv). LBRY is similar to PeerTube in open and decentralization, but all content metadata is written to a blockchain, which means everyone/anyone can access the index. This blockchain can then be searched (https://github.com/lbryio/lighthouse https://github.com/lbryio/lighthouse) or extracted to SQL (https://github.com/lbryio/chainquery https://github.com/lbryio/chainquery).

deleted 7y ago
[deleted]
olah_1 7y ago
Sadly, LBRY isn't able to track view counts. Which makes organic, network effects really difficult. Maybe they could implement a naive "view tracking" by having the client do small proof of work when they interact with content? Similar to voting in NotaBug? https://github.com/notabugio/notabug https://github.com/notabugio/notabug

jasode 7y ago

> Are any of these p2p YouTube replacements presenting a compelling way to search/discover content in a distributed way? Being "distributed" over p2p/federated architecture opposes the end-user convenience of search/discovery/ranking/recommendations because of speed-of-light limitations. I wrote a previous comment about this: https://news.ycombinator.com/item?id=17578332 https://news.ycombinator.com/item?id=17578332 Also, the previous last reply by posting2fast of a partial-centralized server doesn't really replace Youtube because his proposed idea creates new problems of spam videos and untrusted/fake videos. E.g. the central index of metadata says "www.johndoehomeserver.com" has a tutorial video for Algebra but when you actually stream the video from "johndoehomeserver.com", you get a spam video for Viagra instead of math instruction. Therefore, users will naturally gravitate toward the centralized servers that have both the metadata and the actual video content. This emergent group behavior of preferences would end up recreating another "Youtube"-like clone. p2p architecture and torrents works well for things like pirated Photoshop or ripped Marvel Avengers movies because the users already have the content's title _preloaded_ in their brain and therefore a centralized index for discovery/serendipity of unknown content isn't necessary.

deleted 7y ago
[deleted]
tiborsaas 7y ago
It does not necessarily oppose it. All there needs to be done is to expose a static, daily generated JSON file that contains all videos on the instance. This has nothing to do with the speed of light. Anyone then could build a search index and build a good search experience. To combat spam, instances should reveal up/downvotes to indicate quality, I guess your fake math video would not get much love from the community.
- jasode 7y ago
  >This has nothing to do with the speed of light. Please take extra care to correctly parse what I actually wrote in response to the gp. Yes, speed-of-light is still a limitation based on the gp's constraint of "search/discovery in a _distributed_ way" which means the search algorithm avoids central servers and loops through a bunch of remote p2p nodes to parse a bunch of exposed JSON manifest files. If instead, the search algorithm loops through data in a cached index server, that's no longer "search in a distributed way" that the gp was originally wondering about. That's the particular point I was responding to. >Anyone then could build a search index and build a good search experience. Now, as to the issue with that "cache index server" that pre-parses the JSON files... The cache server that also contains the actual video data will naturally attract the most users because when they hit the "play" button on their smartphone, the video starts immediately instead of waiting or suffering stuttering from somebody's flakey home video server. So, the index server with the "good experience" as perceived by users will be the one that also includes the actual videos -- basically acts as a CDN -- and this emergent behavior of user preferences defeats the decentralized ideals of p2p video. We see that p2p of things like illegal software already works and is proven. However, p2p of mainstream videos has massive technical hurdles that oppose how typical users like to discover content and play them with immediate gratification.
  - SamBam 7y ago
    You're using "search/discovery in a distributed way" in a very literal manner. I interpreted the question as being one much more meaningful to the vast majority of users: "can a user search for content across the various distributed servers?" That's all an end-user cares about. Indexing videos once a day (or once an hour or whatever) would be very feasible. Indeed, different servers could create their own indexes, and some might be better at sorting for relevance than others.

dmos62 7y ago

Well that's the real problem when you start competing with Google and the like. They're data behemoths. As long as you don't have as much data collected, they'll have a significant advantage. The solution to this is really simple and hardly conceivable at the same time: make competitively important data public. In this case, that's anonymized video browsing/viewing data. Market competition would flourish, the big guys would lose the monopoly.

pault 7y ago
But then who will be incentivized to gather the data?
- dmos62 7y ago
  Anyone who needs more, I guess? Or, is the premise that gathering data is resource intensive (like innovation and patents for example)? These aren't big problems, I think. The problem is that the people who hold the monopolies on data at the moment are also the people who are extremely powerful lobyists.
boomboomsubban 7y ago
>In this case, that's anonymized video browsing/viewing data So the view count publicly available and something like the Google search analytics already available? Personalized data offers a huge competitive advantage.
- dmos62 7y ago
  Personalized (as in per user) and anonymized are not mutually exclusive.
  - boomboomsubban 7y ago
    Sure, but non-anonymized personalized data still provides a competitive advantage.

arkh 7y ago

> discover content Discover heavily sponsored content from content farms. As an experiment, even with an old account, start just browsing the content Youtube highlights. You will soon end with a recommendation page full of shit with 500K+ views using the same template.

glennpratt 7y ago
I don't understand your point; garbage in, garbage out. I have a very old account and browse suggestions using my brain, not randomly. While not perfect, almost every recommendation right now looks like something I could watch. Bicycling, civil engineering, cat toys, and weird metal music mashups, which are all similar to things I intentionally watched, but haven't watched.

velcrovan 7y ago

> Are any of these p2p YouTube replacements presenting a compelling way to search/discover content in a distributed way? Ideally we’d see blogs return as the curated content mediators.