6 ms·

Reposting an old comment of mine here just to see if it can get an interesting conversation going: "I was daydreaming recently after revamping and dialing in m

by dchuk 3y ago

Reposting an old comment of mine here just to see if it can get an interesting conversation going: "I was daydreaming recently after revamping and dialing in my home media server really well (Plex plus all of the *Arr services plus torrents automatically downloading shows and hosting a large movie library)… Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood, powered by people either contributing storage space or content or both, where no one ever hosts any in-tact media and it’s all encrypted enough where each storage provider is basically hosting useless shards of content, and then some client side software can put it all together on demand and serve it up to apps like Plex in a way Plex doesn’t even know it’s not a local file? Assuming everyone has decent bandwidth and there’s enough contributed storage for redundancy to power the torrent network, you could basically have an unlimited and resilient media library… There’s probably a way to craft an incentive scheme in there for both content submitters and storage providers, probably a decent use case for crypto honestly so it’s all decentralized. At this point, it seems like content capture and uploading to the torrent world is basically “solved”, and now it’s all about ensuring the network and storage health is there to preserve everything in perpetuity. There’s so many redundant torrents out there, should be possible to merge it all in a giant media library mesh network right?" https://news.ycombinator.com/item?id=35213039 https://news.ycombinator.com/item?id=35213039

yebyen 3y ago
Is it possible to maintain such a project? I thought it was called popcorn-time, but earlier I remember MythTV. Both projects have seemingly been hobbled from general use, the main idea is to watch whatever you want, when you want, and it doesn't have advertising in it. (If the original source recording had it, then it could be removed.) Maybe it's overly dramatic to say I think this should be a human right, but maybe it isn't? I don't need advertising in my life, I think.
- cortesoft 3y ago
  If it becomes a human right to be able to watch any content for free and with no advertising, how would there be any money to produce content?
  - yebyen 3y ago
    I didn't say for free, I said without advertising. So I guess we gotta sell all our souls... or haggle?
    - mrtweetyhack 3y ago
      [dead]
  - tenebrisalietum 3y ago
    People will pay creators to make content they want.
  - chii 3y ago
    > how would there be any money to produce content? the same way content on youtube gets produced today. But this argument is a hyperbole. The content gets paid for, and some people will pay - convenience, access etc. The rest can free ride off those people. So far, it has worked.
    - bryanrasmussen 3y ago
      convenience and access exist because it is not a human right to watch whatever you want when you want it. if there were such a right free content would be convenient and accessible.
  - geraldhh 3y ago
    idk but most content cherished by humanity was created for propaganda (religion) and/or by poor artists
- totetsu 3y ago
  Yes popcorn prioritised downloading the torrent chunks in order so you could start playing the video while it was still downloading
  - yebyen 3y ago
    There is still Soda Player, which works as well but without the torrent catalogue built in.
- dchuk 3y ago
  Hypothetically, you could setup (maybe with crypto) a rewards system for people who contribute content, add storage to the network, do transcoding, etc, and then allow other people who don't want to do any of that to simply pay for accessing the mesh network. That would open up a flow of funds that the development organization could take a percentage of. That being said, it's likely just a lawsuit honeypot at that point, so mostly a daydream :)
deleted 3y ago
[deleted]
JasonFruit 3y ago
Isn't that basically Freenet?
- toyg 3y ago
  My reaction exactly, "this sounds a lot like Freenet". Although I wouldn't be ideologically opposed to the idea of getting another stab at the concept of FN, I think it's imperative for people interested in doing that to look at the history and challenges of that project. Even if we could get over the scalability issues (as the technology landscape has undoubtedly changed since then - "free" storage, larger bandwidth, faster processors... - we'd still have to wrangle with the unavoidable truth that nobody wants to risk hosting CP. I think you need to solve that problem, before you look at building such a system.
qersist3nce 3y ago
Can someone tell me what is the appeal of *arr apps? I'm not a hoarder or professional movie/show watcher, why should I set up what is basically a glorified "periodic metadata downloader with a tiles UI"? I tested radarr and readarr recently. Didn't find out why people use them. Like libgen was better than readarr and I couldn't even select the encoding/file size of a movie title in radarr (there are some predefined profiles and you can define yours too but I expected something waay too simpler to do this. Like if you select fhd, it may download a very large file. You have to manually set the bitrate slider for all profiles, but then you may risk losing some available item if the release is very new and not encoded. I mean the UX was confusing) And I see people host yet another tiles UI (Plex/Jellyfin) on top of that just to show a folder structure with pretty thumbnails? Each one of these apps is usually written in C#, taking ~300MB of RAM mind you... What would help me is simply a search box that crawls all available torrents and presents rows with filtering based on size and encoding. I tested prowlarr for this but the UX was poor. I must be missing something very obvious here.
- somehnguy 3y ago
  I don’t want to download tv shows or movies when they come out. I don’t even know their schedules. Just add them in *arr and they appear in Plex automagically as they come out. For TV shows specifically this is a huge time saver. Plus they let less technical family users add and manage media on their own. Regarding not being able to select encoding, etc - you setup profiles and it automatically chooses the best release for based on the criteria you set. Again, it’s one less thing to think about.
  - qersist3nce 3y ago
    how do you control the quality and encoding of the show? And why is there a separate *arr app for every content? (music album, tv, movie, book, subtile(!),...)
    - wincy 3y ago
      It’s weird to me too. I consume a decent amount of media but if I can’t think “oh yeah I want to watch this particular thing” I just don’t download it and it falls out of my brain. Which I’m fine with. Foundation was like that, and His Dark Materials. I guess I would have finished them if they downloaded automatically but they were kinda lame. I guess I just consume media more haphazardly than most people. The whole point of my PLEX is only exactly what I’ve requested shows up.
lucb1e 3y ago
I think an idea of mine is related: index clips from various sources to put together an entire movie. Or index quotes posted to various forums or quote websites to reassemble the original book. No one hosts the full content, all is fair use, and the index is also meaningless by itself. Though I suppose you'd get a similar conviction as The Pirate bay which also hosts no copyright infringing content. Your idea would be a bit more 'direct' in that the parts are hosted by people actively contributing to have a copyrighted work in full rather than linking to random third parties. A lot more reliable, at least until it catches on and contributors start receiving desist notices.
- CPLX 3y ago
  If that’s the purpose of the system then all the clips and quotes are not fair use.
  - bryanrasmussen 3y ago
    well the clips and quotes can be fair use if they are gathered from around the world, but the service itself would be infringing in such a situation.
- TechSquidTV 3y ago
  Isnt this just Usenet?
temphn3728 3y ago
I believe this was more for storing content, and not streaming, but this was presented at defcon a few years ago. Project: https://github.com/seantmalone/HiveMind https://github.com/seantmalone/HiveMind Presentation: https://m.youtube.com/watch?v=GAq86wfeC9E https://m.youtube.com/watch?v=GAq86wfeC9E
aftbit 3y ago
aka Napster
- dchuk 3y ago
  Sort of. It would be more like firing up Netflix, but it has every piece of content ever in it. No choosing to download anything, no real notion of a personal library of content (maybe favoriting or something for UX I guess)...just a infinite mesh of content, powered by a distributed network, with economics involved to incentivize everyone to either consume the content or help host the network.
  - ipaddr 3y ago
    Kodi addons
voynich 3y ago
I had a similar idea, at least in terms of resilience. It was, basically, to compress each piece of content just so that the compressed version would be theoretically uncensorable. Jon Lech Johansen did this with DeCSS back in the day [1], and the compressed version of the program was a prime number, which gave it a sort of "untouchable" quality. Obviously, doing this for much larger content (i.e. movies and general videos) would be a challenge, and this technique might not be the best choice. Still an interesting concept, though. [1]: https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes
- chii 3y ago
  > which gave it a sort of "untouchable" quality. while true, the actual reason it's untouchable is that the companies cannot realistically sue everyone - it's too costly, and they cannot recover the funds.
- zo1 3y ago
  What you guys seem to be describing is "Freenet", but that project has been around forever and relatively obscure. These days, unfortunately, I wouldn't touch it with a 10ft pole due to safety concerns. https://en.wikipedia.org/wiki/Freenet https://en.wikipedia.org/wiki/Freenet "Freenet is a peer-to-peer platform for censorship-resistant, anonymous communication. It uses a decentralized distributed data store to keep and deliver information" From what I remember, it basically takes content and distributes "anonymous" and encrypted chunks of it across various members on the network. Content stays active by being replicated, and replication happens proportionally based on popularity. So content that never gets used basically disappears.
  - fsflover 3y ago
    Does Freenet have torrents? I2P does.
whamlastxmas 3y ago
Copyright holders would still track the IPs that these shards are coming from and sue/threaten them for copyright infringement. 5 seconds of Mulan compressed and sitting in isolation is still illegal to distribute
- cjblomqvist 3y ago
  Is that really true? I thought it was fine to use small snippets of stuff?
  - etskinner 3y ago
    It's fine to use small snippets, but it specifically has to be for commentary, satire, derivative works, etc. The term you're looking for is 'fair use'
    - whamlastxmas 3y ago
      Which is still infringement technically, fair use is a defense for infringement
  - whamlastxmas 3y ago
    Even distributing a single still image or frame is infringement
6510 3y ago
Someone made a desktop application much like that, it was wonderful, truly wonderful... until I started chatting with the developer about not keeping semi-permanent logs of who [previously] hosted what. I had hoped to solve the puzzle but he started unpublishing his application immediately. I think simpler things could help more. I would like to see 1) a client that (in stead of just seeding) periodically checks how rare the rare blocks are on a torrent then seed just those blocks. 2) attempt to extend the tit for tad by looking for similar titles and file names (or some other formula) if insufficient seeds are available. So you might have [parts of] foo1 and foo3 and are looking for [parts of] foo2. foo2 is served by someone looking for [parts of] foo3. One could even ask to help seed something they are uninterested in and/or unfamiliar with. It does seem less of a "crime" than getting involved in permanently seeding the motherload.
thomastjeffery 3y ago
The problem is moderation. Any time you make a new media library hosting platform, it attracts exactly the people who need it most. The answer is good metadata and categorization, but that requires active moderation. Moderation is the single most expensive part of the internet, and most of it is being done for free. Big tech like Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it. -- I think the best solution is a web of trust. Instead of using a tracker/site like TPB (and its moderation) to introduce trust, let each user make whatever authoritative claims they want (this torrent is good, that one is corrupted, and this other one has a virus), and let other users back those claims via gpg signature. Instead of forum moderators, you get trust curators.
- __MatrixMan__ 3y ago
  I agree that webs of trust are the way forward. Power corrupts, and any set of curators will eventually fall to corruption, but the nice thing about having software that lets you explicitly trust people (instead of implicitly trusting the platform) is that the switching costs are low. You don't have to wait for somebody who is both tech savvy enough to build a competing platform and also trustworthy enough to curate the content, you just need the trustworthy part. You just twiddle your trust settings and you've got a fresh start. I also think that the who-trusts-whom-in-what-category dataset would be incredibly useful. For instance, you can follow the edges until you get a cycle, and that cycle is going to be at the heart of a community of experts. Or if you're having trouble seeing eye to eye with someone you can consult the trust graph and find a mediator, or if they're not to be found then you'll find thought leaders who exemplify the disagreement. You'll have to be hygenic about who you trust though, and that's not the lazy path forward. So culturally I think any project of this sort is going to face an uphill battle.
- pojzon 3y ago
  > Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it Its the other way around. They try to automate and fail miserably after spending billions and fallback to manual moderation by farms in asian countries coz its cheaper.
trevor-e 3y ago
If I'm following correctly, I believe this is basically what Sia does, although not optimized to be used directly as a media server (or maybe it could?). https://sia.tech/ https://sia.tech/
agilob 3y ago
There was such a thing round 2007-2010. People on forums were opening free email accounts, then a desktop app was splitting files into chunks of predefined size and uploading them as emails. Individually each chunk was an encrypted chuink of 7z or zip archive, so it made no sense on its own, but the client was building a repository of email account and email IDs. This index was then shared with other people on a forum and you could download Shrek. https://forums.freebsd.org/threads/peer-2-mail-file-sharing-technology-using-mailboxes.66118/ https://forums.freebsd.org/threads/peer-2-mail-file-sharing-... https://www.ygoow.com/index.php?lang=en https://www.ygoow.com/index.php?lang=en
shzhdbi09gv8ioi 3y ago
> Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood We actually had bittorrent-enabled video players in the early days of bittorrent. There seems to be some options active today, too. Google suggested frostwire.com
hummerbliss 3y ago
Please can you tell me if you are using any package managers for *arr services like swizzin/onebox ?

by dchuk 3y ago

yebyen 3y ago

Is it possible to maintain such a project? I thought it was called popcorn-time, but earlier I remember MythTV. Both projects have seemingly been hobbled from general use, the main idea is to watch whatever you want, when you want, and it doesn't have advertising in it. (If the original source recording had it, then it could be removed.) Maybe it's overly dramatic to say I think this should be a human right, but maybe it isn't? I don't need advertising in my life, I think.

cortesoft 3y ago
If it becomes a human right to be able to watch any content for free and with no advertising, how would there be any money to produce content?
- yebyen 3y ago
  I didn't say for free, I said without advertising. So I guess we gotta sell all our souls... or haggle?
  - mrtweetyhack 3y ago
    [dead]
- tenebrisalietum 3y ago
  People will pay creators to make content they want.
- chii 3y ago
  > how would there be any money to produce content? the same way content on youtube gets produced today. But this argument is a hyperbole. The content gets paid for, and some people will pay - convenience, access etc. The rest can free ride off those people. So far, it has worked.
  - bryanrasmussen 3y ago
    convenience and access exist because it is not a human right to watch whatever you want when you want it. if there were such a right free content would be convenient and accessible.
- geraldhh 3y ago
  idk but most content cherished by humanity was created for propaganda (religion) and/or by poor artists
totetsu 3y ago
Yes popcorn prioritised downloading the torrent chunks in order so you could start playing the video while it was still downloading
- yebyen 3y ago
  There is still Soda Player, which works as well but without the torrent catalogue built in.
dchuk 3y ago
Hypothetically, you could setup (maybe with crypto) a rewards system for people who contribute content, add storage to the network, do transcoding, etc, and then allow other people who don't want to do any of that to simply pay for accessing the mesh network. That would open up a flow of funds that the development organization could take a percentage of. That being said, it's likely just a lawsuit honeypot at that point, so mostly a daydream :)

deleted 3y ago

[deleted]

JasonFruit 3y ago

Isn't that basically Freenet?

toyg 3y ago
My reaction exactly, "this sounds a lot like Freenet". Although I wouldn't be ideologically opposed to the idea of getting another stab at the concept of FN, I think it's imperative for people interested in doing that to look at the history and challenges of that project. Even if we could get over the scalability issues (as the technology landscape has undoubtedly changed since then - "free" storage, larger bandwidth, faster processors... - we'd still have to wrangle with the unavoidable truth that nobody wants to risk hosting CP. I think you need to solve that problem, before you look at building such a system.

qersist3nce 3y ago

Can someone tell me what is the appeal of *arr apps? I'm not a hoarder or professional movie/show watcher, why should I set up what is basically a glorified "periodic metadata downloader with a tiles UI"? I tested radarr and readarr recently. Didn't find out why people use them. Like libgen was better than readarr and I couldn't even select the encoding/file size of a movie title in radarr (there are some predefined profiles and you can define yours too but I expected something waay too simpler to do this. Like if you select fhd, it may download a very large file. You have to manually set the bitrate slider for all profiles, but then you may risk losing some available item if the release is very new and not encoded. I mean the UX was confusing) And I see people host yet another tiles UI (Plex/Jellyfin) on top of that just to show a folder structure with pretty thumbnails? Each one of these apps is usually written in C#, taking ~300MB of RAM mind you... What would help me is simply a search box that crawls all available torrents and presents rows with filtering based on size and encoding. I tested prowlarr for this but the UX was poor. I must be missing something very obvious here.

somehnguy 3y ago
I don’t want to download tv shows or movies when they come out. I don’t even know their schedules. Just add them in *arr and they appear in Plex automagically as they come out. For TV shows specifically this is a huge time saver. Plus they let less technical family users add and manage media on their own. Regarding not being able to select encoding, etc - you setup profiles and it automatically chooses the best release for based on the criteria you set. Again, it’s one less thing to think about.
- qersist3nce 3y ago
  how do you control the quality and encoding of the show? And why is there a separate *arr app for every content? (music album, tv, movie, book, subtile(!),...)
  - wincy 3y ago
    It’s weird to me too. I consume a decent amount of media but if I can’t think “oh yeah I want to watch this particular thing” I just don’t download it and it falls out of my brain. Which I’m fine with. Foundation was like that, and His Dark Materials. I guess I would have finished them if they downloaded automatically but they were kinda lame. I guess I just consume media more haphazardly than most people. The whole point of my PLEX is only exactly what I’ve requested shows up.

lucb1e 3y ago

I think an idea of mine is related: index clips from various sources to put together an entire movie. Or index quotes posted to various forums or quote websites to reassemble the original book. No one hosts the full content, all is fair use, and the index is also meaningless by itself. Though I suppose you'd get a similar conviction as The Pirate bay which also hosts no copyright infringing content. Your idea would be a bit more 'direct' in that the parts are hosted by people actively contributing to have a copyrighted work in full rather than linking to random third parties. A lot more reliable, at least until it catches on and contributors start receiving desist notices.

CPLX 3y ago
If that’s the purpose of the system then all the clips and quotes are not fair use.
- bryanrasmussen 3y ago
  well the clips and quotes can be fair use if they are gathered from around the world, but the service itself would be infringing in such a situation.
TechSquidTV 3y ago
Isnt this just Usenet?

temphn3728 3y ago

I believe this was more for storing content, and not streaming, but this was presented at defcon a few years ago. Project: https://github.com/seantmalone/HiveMind https://github.com/seantmalone/HiveMind Presentation: https://m.youtube.com/watch?v=GAq86wfeC9E https://m.youtube.com/watch?v=GAq86wfeC9E

aftbit 3y ago

aka Napster

dchuk 3y ago
Sort of. It would be more like firing up Netflix, but it has every piece of content ever in it. No choosing to download anything, no real notion of a personal library of content (maybe favoriting or something for UX I guess)...just a infinite mesh of content, powered by a distributed network, with economics involved to incentivize everyone to either consume the content or help host the network.
- ipaddr 3y ago
  Kodi addons

voynich 3y ago

I had a similar idea, at least in terms of resilience. It was, basically, to compress each piece of content just so that the compressed version would be theoretically uncensorable. Jon Lech Johansen did this with DeCSS back in the day [1], and the compressed version of the program was a prime number, which gave it a sort of "untouchable" quality. Obviously, doing this for much larger content (i.e. movies and general videos) would be a challenge, and this technique might not be the best choice. Still an interesting concept, though. [1]: https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes

chii 3y ago
> which gave it a sort of "untouchable" quality. while true, the actual reason it's untouchable is that the companies cannot realistically sue everyone - it's too costly, and they cannot recover the funds.
zo1 3y ago
What you guys seem to be describing is "Freenet", but that project has been around forever and relatively obscure. These days, unfortunately, I wouldn't touch it with a 10ft pole due to safety concerns. https://en.wikipedia.org/wiki/Freenet https://en.wikipedia.org/wiki/Freenet "Freenet is a peer-to-peer platform for censorship-resistant, anonymous communication. It uses a decentralized distributed data store to keep and deliver information" From what I remember, it basically takes content and distributes "anonymous" and encrypted chunks of it across various members on the network. Content stays active by being replicated, and replication happens proportionally based on popularity. So content that never gets used basically disappears.
- fsflover 3y ago
  Does Freenet have torrents? I2P does.

whamlastxmas 3y ago

Copyright holders would still track the IPs that these shards are coming from and sue/threaten them for copyright infringement. 5 seconds of Mulan compressed and sitting in isolation is still illegal to distribute

cjblomqvist 3y ago
Is that really true? I thought it was fine to use small snippets of stuff?
- etskinner 3y ago
  It's fine to use small snippets, but it specifically has to be for commentary, satire, derivative works, etc. The term you're looking for is 'fair use'
  - whamlastxmas 3y ago
    Which is still infringement technically, fair use is a defense for infringement
- whamlastxmas 3y ago
  Even distributing a single still image or frame is infringement

6510 3y ago

Someone made a desktop application much like that, it was wonderful, truly wonderful... until I started chatting with the developer about not keeping semi-permanent logs of who [previously] hosted what. I had hoped to solve the puzzle but he started unpublishing his application immediately. I think simpler things could help more. I would like to see 1) a client that (in stead of just seeding) periodically checks how rare the rare blocks are on a torrent then seed just those blocks. 2) attempt to extend the tit for tad by looking for similar titles and file names (or some other formula) if insufficient seeds are available. So you might have [parts of] foo1 and foo3 and are looking for [parts of] foo2. foo2 is served by someone looking for [parts of] foo3. One could even ask to help seed something they are uninterested in and/or unfamiliar with. It does seem less of a "crime" than getting involved in permanently seeding the motherload.

thomastjeffery 3y ago

The problem is moderation. Any time you make a new media library hosting platform, it attracts exactly the people who need it most. The answer is good metadata and categorization, but that requires active moderation. Moderation is the single most expensive part of the internet, and most of it is being done for free. Big tech like Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it. -- I think the best solution is a web of trust. Instead of using a tracker/site like TPB (and its moderation) to introduce trust, let each user make whatever authoritative claims they want (this torrent is good, that one is corrupted, and this other one has a virus), and let other users back those claims via gpg signature. Instead of forum moderators, you get trust curators.

__MatrixMan__ 3y ago
I agree that webs of trust are the way forward. Power corrupts, and any set of curators will eventually fall to corruption, but the nice thing about having software that lets you explicitly trust people (instead of implicitly trusting the platform) is that the switching costs are low. You don't have to wait for somebody who is both tech savvy enough to build a competing platform and also trustworthy enough to curate the content, you just need the trustworthy part. You just twiddle your trust settings and you've got a fresh start. I also think that the who-trusts-whom-in-what-category dataset would be incredibly useful. For instance, you can follow the edges until you get a cycle, and that cycle is going to be at the heart of a community of experts. Or if you're having trouble seeing eye to eye with someone you can consult the trust graph and find a mediator, or if they're not to be found then you'll find thought leaders who exemplify the disagreement. You'll have to be hygenic about who you trust though, and that's not the lazy path forward. So culturally I think any project of this sort is going to face an uphill battle.
pojzon 3y ago
> Facebook and Google are making themselves incompatible with traditional moderation, which is why they keep trying (and failing) to automate it Its the other way around. They try to automate and fail miserably after spending billions and fallback to manual moderation by farms in asian countries coz its cheaper.

trevor-e 3y ago

If I'm following correctly, I believe this is basically what Sia does, although not optimized to be used directly as a media server (or maybe it could?). https://sia.tech/ https://sia.tech/

agilob 3y ago

There was such a thing round 2007-2010. People on forums were opening free email accounts, then a desktop app was splitting files into chunks of predefined size and uploading them as emails. Individually each chunk was an encrypted chuink of 7z or zip archive, so it made no sense on its own, but the client was building a repository of email account and email IDs. This index was then shared with other people on a forum and you could download Shrek. https://forums.freebsd.org/threads/peer-2-mail-file-sharing-technology-using-mailboxes.66118/ https://forums.freebsd.org/threads/peer-2-mail-file-sharing-... https://www.ygoow.com/index.php?lang=en https://www.ygoow.com/index.php?lang=en

shzhdbi09gv8ioi 3y ago

> Shouldn’t it be possible to create a “virtual media library” that is actually BitTorrent under the hood We actually had bittorrent-enabled video players in the early days of bittorrent. There seems to be some options active today, too. Google suggested frostwire.com

hummerbliss 3y ago

Please can you tell me if you are using any package managers for *arr services like swizzin/onebox ?