4 ms·

The problem is not search, it's curation. Any system that accepts everything will be overrun with things you do not want (fakes, very illegal things), and so

The problem is not search, it's curation. Any system that accepts everything will be overrun with things you do not want (fakes, *very* illegal things), and so there needs to be some authority that determines what is allowed. At that point, you are centralized, and hosting a website doesn't significantly hurt you. This is also the problem with all distributed social networks. In the end, your options are formal centralization, and informal centralization, because absolutely nobody wants to live in true decentralization.

throwawayiddqd2 3y ago
Most of the time when I search for something, I sort by more seeders first. Some type of signature can be used too.
- ryandrake 3y ago
  Sorting by number of seeders will probably get you a file that is fast to download because of the number of seeders, but it will rarely get you quality. Let's go take a stroll over to a popular public torrent site and search for, say, a recent superhero movie. I see 100 (!) results for that movie's name. 49 of them have zero seeders at all. I don't know what even is the point. 29 of the results have one seeder. So already, 78% of the results are pure crap. Let's look at the top result with 338 seeders: File is 3GB, H.264 video, 1080p, but with a crappy stereo AAC audio encoding... arrggh why?? Number 2 result with 84 seeders: 1.43GB, H.264 video, 720p, no word on the quality of the audio encoding. Even more worthless. Number 3 result with 17 seeders: HEVC format, 2160p, audio streams include TrueHD Atmos 7.1, DTS-HD, Dolby Digital 5.1, stereo, and three non-English language streams. But, with an eye-watering download size of 61GB. Holy shit! Nice, but wow, what a download. You have to go a few more down the list to find a good balance of high quality video and audio encoding, but with a reasonable file size. By that point you're in the single digit number of seeders. Don't get me wrong, it's great to have a few choices and quality trade-offs. I guess there's someone out there who doesn't care about the stereo audio because they watch their movies with laptop speakers. But 100 results, with the vast majority of them either unseeded, poorly-seeded, or flawed in some way. I agree with OP: You definitely want some curation, not just search!
  - fknorangesite 3y ago
    > arrggh why?? Because you are vastly overestimating how much most people actually notice or care about this.
  - hirako2000 3y ago
    I often pick the 0 seeders results if there is even a slight chance of better quality, and quite often there is. 0 seeders now doesn't mean forever. Some seeders only hook to the network once in a while, long enough for a few leechers to fetch it all. If the content is in demand enough then a few seeders may be left and suddenly a 0 seeder search result becomes an attractive one. Even curating is complicated for the human eye, so good luck automating that.
    - saiya-jin 3y ago
      Yes it sometimes works but usually for smaller stuff like ebooks or music. You dont want to download gigabytes if not more from 1 spotty seeder, the chances it wont take days of 24h online and source being obline too are tiny. Sometimes disconnect from site seeder count is bigger, and say 1 seeder mentioned is actually 15. But that was never rarbg's case, it was reliable and dependable like no other similar service. It also had IMBD rating so I could quickly weed out not so great stuff and focus on well rated ones, it generated maybe 60-80 movies a day with my filter applied, so quite a stream. After 2 week vacation, catching up took some time. I used to, including today, to just go there every day or two and check whats new with my predefined filter. Often great movies that I never heard about before, old and new alike, took 2 mins to get 1080p x265 variant. Glad I've still managed to download that highly rated turkish movie this morning.
      - hirako2000 3y ago
        It is sad they didn't take the care to open source their stuff with whatever doc may be floating in their chats, for a resurrect
  - o1y32 3y ago
    Great example. Some private trackers have strict rules about quality, e.g. at most one 720p and at most one 1080p (probably 1080p and 4k nowadays), bitrate must be above a certain threshold, whichever was published first stays, unless there is an "official" version published by the same people maintaining the website. The exact opposite of decentralized torrent sites, and works really well. Takes a lot of work, obviously.
- pessimizer 3y ago
  > I sort by more seeders first. But this couldn't be easier to game. All you have to do to be a seed is to report that you are a seed. You don't even have to send any data.
tracker1 3y ago
It's pretty much impossible to support Anonymous + Distributed + Free from poison/injection attacks. Not to mention, you still largely need at least some known points of entry into such a system.
Paul-E 3y ago
Has anyone tried to decentralize curation? Here is straw man proposal, similar to cert chains and webs of trust: Say I'm a "curator". I say on HN/Reddit/Discord "here is my key hash 'p2pcuration:185da2bc59167692f596404fd83235f9bcb4e107b041f2e6e8d972da6dba00b7'". Any user that clicks the link or copies it into the search app adds the key to the trusted user list. With my private key I can sign torrents after I download them myself, which would mark the torrent as "good". When anyone who has added my key searches, the system searches for a corresponding signature from me as well. If a signature is found, the UI can chose to elevate that result. The system could be extended so that signers could also sign other keys, expanding the trust network. This system doesn't need to be run or maintained by each user. It could be served through a webui that can be run locally or shared with a small community. Migrating the interface to a new host would just require moving the config and keys.
- Tuna-Fish 3y ago
  What happens when you do this is that a few or one people in the system become trusted above the others, network effects do their thing, and the whole thing becomes informally centralized around them.
BlueTemplar 3y ago
Some examples trying to deal with this involving Mastodon (years before the Twitter Musk debacle) : Mastodon is big in Japan, and the reason why is uncomfortable ["child porn"] https://news.ycombinator.com/item?id=15053064 https://news.ycombinator.com/item?id=15053064 Mastodon and the challenges of abuse in a federated system https://news.ycombinator.com/item?id=17894684 https://news.ycombinator.com/item?id=17894684 How the biggest decentralized social network is dealing with its ["]Nazi["] problem https://news.ycombinator.com/item?id=20429465 https://news.ycombinator.com/item?id=20429465 Trump’s new social media platform found using Mastodon code https://news.ycombinator.com/item?id=29079404 https://news.ycombinator.com/item?id=29079404
abwizz 3y ago
> because absolutely nobody wants to live in true decentralization. oh, some ppl do indeed want to. thanks to encryption they can. but i agree that the likes of kazaa, edk and freenet are not for most ppl.
- jasonfarnon 3y ago
  I seem to remember kazaa being extremely popular (especially after napster was gone) and moreover centralized. The client was local. gnutella sounds like what you're describing?

The problem is not search, it's curation. Any system that accepts everything will be overrun with things you do not want (fakes, *very* illegal things), and so

throwawayiddqd2 3y ago

Most of the time when I search for something, I sort by more seeders first. Some type of signature can be used too.

ryandrake 3y ago
Sorting by number of seeders will probably get you a file that is fast to download because of the number of seeders, but it will rarely get you quality. Let's go take a stroll over to a popular public torrent site and search for, say, a recent superhero movie. I see 100 (!) results for that movie's name. 49 of them have zero seeders at all. I don't know what even is the point. 29 of the results have one seeder. So already, 78% of the results are pure crap. Let's look at the top result with 338 seeders: File is 3GB, H.264 video, 1080p, but with a crappy stereo AAC audio encoding... arrggh why?? Number 2 result with 84 seeders: 1.43GB, H.264 video, 720p, no word on the quality of the audio encoding. Even more worthless. Number 3 result with 17 seeders: HEVC format, 2160p, audio streams include TrueHD Atmos 7.1, DTS-HD, Dolby Digital 5.1, stereo, and three non-English language streams. But, with an eye-watering download size of 61GB. Holy shit! Nice, but wow, what a download. You have to go a few more down the list to find a good balance of high quality video and audio encoding, but with a reasonable file size. By that point you're in the single digit number of seeders. Don't get me wrong, it's great to have a few choices and quality trade-offs. I guess there's someone out there who doesn't care about the stereo audio because they watch their movies with laptop speakers. But 100 results, with the vast majority of them either unseeded, poorly-seeded, or flawed in some way. I agree with OP: You definitely want some curation, not just search!
- fknorangesite 3y ago
  > arrggh why?? Because you are vastly overestimating how much most people actually notice or care about this.
- hirako2000 3y ago
  I often pick the 0 seeders results if there is even a slight chance of better quality, and quite often there is. 0 seeders now doesn't mean forever. Some seeders only hook to the network once in a while, long enough for a few leechers to fetch it all. If the content is in demand enough then a few seeders may be left and suddenly a 0 seeder search result becomes an attractive one. Even curating is complicated for the human eye, so good luck automating that.
  - saiya-jin 3y ago
    Yes it sometimes works but usually for smaller stuff like ebooks or music. You dont want to download gigabytes if not more from 1 spotty seeder, the chances it wont take days of 24h online and source being obline too are tiny. Sometimes disconnect from site seeder count is bigger, and say 1 seeder mentioned is actually 15. But that was never rarbg's case, it was reliable and dependable like no other similar service. It also had IMBD rating so I could quickly weed out not so great stuff and focus on well rated ones, it generated maybe 60-80 movies a day with my filter applied, so quite a stream. After 2 week vacation, catching up took some time. I used to, including today, to just go there every day or two and check whats new with my predefined filter. Often great movies that I never heard about before, old and new alike, took 2 mins to get 1080p x265 variant. Glad I've still managed to download that highly rated turkish movie this morning.
    - hirako2000 3y ago
      It is sad they didn't take the care to open source their stuff with whatever doc may be floating in their chats, for a resurrect
- o1y32 3y ago
  Great example. Some private trackers have strict rules about quality, e.g. at most one 720p and at most one 1080p (probably 1080p and 4k nowadays), bitrate must be above a certain threshold, whichever was published first stays, unless there is an "official" version published by the same people maintaining the website. The exact opposite of decentralized torrent sites, and works really well. Takes a lot of work, obviously.
pessimizer 3y ago
> I sort by more seeders first. But this couldn't be easier to game. All you have to do to be a seed is to report that you are a seed. You don't even have to send any data.

tracker1 3y ago

It's pretty much impossible to support Anonymous + Distributed + Free from poison/injection attacks. Not to mention, you still largely need at least some known points of entry into such a system.

Paul-E 3y ago

Has anyone tried to decentralize curation? Here is straw man proposal, similar to cert chains and webs of trust: Say I'm a "curator". I say on HN/Reddit/Discord "here is my key hash 'p2pcuration:185da2bc59167692f596404fd83235f9bcb4e107b041f2e6e8d972da6dba00b7'". Any user that clicks the link or copies it into the search app adds the key to the trusted user list. With my private key I can sign torrents after I download them myself, which would mark the torrent as "good". When anyone who has added my key searches, the system searches for a corresponding signature from me as well. If a signature is found, the UI can chose to elevate that result. The system could be extended so that signers could also sign other keys, expanding the trust network. This system doesn't need to be run or maintained by each user. It could be served through a webui that can be run locally or shared with a small community. Migrating the interface to a new host would just require moving the config and keys.

Tuna-Fish 3y ago
What happens when you do this is that a few or one people in the system become trusted above the others, network effects do their thing, and the whole thing becomes informally centralized around them.

BlueTemplar 3y ago

Some examples trying to deal with this involving Mastodon (years before the Twitter Musk debacle) : Mastodon is big in Japan, and the reason why is uncomfortable ["child porn"] https://news.ycombinator.com/item?id=15053064 https://news.ycombinator.com/item?id=15053064 Mastodon and the challenges of abuse in a federated system https://news.ycombinator.com/item?id=17894684 https://news.ycombinator.com/item?id=17894684 How the biggest decentralized social network is dealing with its ["]Nazi["] problem https://news.ycombinator.com/item?id=20429465 https://news.ycombinator.com/item?id=20429465 Trump’s new social media platform found using Mastodon code https://news.ycombinator.com/item?id=29079404 https://news.ycombinator.com/item?id=29079404

abwizz 3y ago

> because absolutely nobody wants to live in true decentralization. oh, some ppl do indeed want to. thanks to encryption they can. but i agree that the likes of kazaa, edk and freenet are not for most ppl.

jasonfarnon 3y ago
I seem to remember kazaa being extremely popular (especially after napster was gone) and moreover centralized. The client was local. gnutella sounds like what you're describing?