4 ms·

The masses need to pick a new site to invest their energies in, rather than scatter at random to lots of different sites! Any particular one out there that's be

by skippy 14y ago

The masses need to pick a new site to invest their energies in, rather than scatter at random to lots of different sites! Any particular one out there that's better than the rest?

deleted 14y ago
[deleted]
troels 14y ago
As someone who hasn't used usenet for downloading content, why is it that it seems that a few central indexing services are the way people access this medium? Wouldn't it be possible - and preferable - to distribute the index through p2p? That way, there would be no single point of failure.
- nwh 14y ago
  The index would be insanely huge. A lot of these indexing sites are used for their API. Dedicated applications like SickBeard automatically compile lists of wanted files (movies, TV rips and music) and use them to search against the provider. The provider returns a file (NZB) that contains references to hundreds of encoded files hosted on newsgroups. Here's a breakdown of one NZB, for a single 3.88GB file. Inside are 74 RAR files, and 15 additional PAR files containing parity for recovering corrupted data. Each of these 89 files is split into 37 parts of 1.4MB for ~3200 pieces total. These pieces are then encoded with yENC (similar to base64) and then uploaded to the newsgroup. Every single one of these yENC pieces is referenced in the NZB file, making for a total of 400KB. Just because of this, the NZB indexs are enormous; hundreds of gigabytes, probably terabytes in some cases.
  - morsch 14y ago
    There's no need for everybody to serve the entire index. Set up a DHT and let interested parties exchange the NZBs. All problems in computer science can be solved by another level of indirection.
    - trotsky 14y ago
      it's a reputational and data quality issue as well. completely decentralized content distribution doesn't usually work unless there is some kind of curation, there are just too many motivations for disruption. it's not an impossible problem, but long ago ceased being mostly about the technical occupation of shipping bits between nodes.
      - pyre 14y ago
        You could always have sites that only serve up indexes of NZB files (e.g. "NZB file => SHA-256"). Then you could use these to determine which NZB files to trust. I would assume that distributing a "NZB => Checksum" hash would be a lot smaller/easier than running an entire site on top of a terabyte of NZB files.
    - rektide 14y ago
      DebTorrent was to be a set of BitTorrent extensions to allow the Debian archive to be syndicated as one ever-growing torrent. Seems similar to the use case here, and free from the problem we have now which is the huge indexes getting hit by takedown notices. http://wiki.debian.org/DebTorrent http://wiki.debian.org/DebTorrent
  - asdfaoeu 14y ago
    You can always just upload the nzb file to Usenet and reference that.
    - nwh 14y ago
      That's usually where the indexing sites get them from in the first place. The issue comes when you're a client that's not indexing continuously; downloading billions of headers to try and find a particular NZB is a mammoth task.
      - pyre 14y ago
        > downloading billions of headers to try and find a > particular NZB is a mammoth task This is true. Even a ~10 of years ago, downloading just the headers on some of the newsgroups shot my cache directory into the gigabytes (for a single newsgroup).
      - foobarqux 14y ago
        Why aren't there subgroups with just nzbs?
- Wicher 14y ago
  Such a project exists: Spotnet http://en.wikipedia.org/wiki/Spotnet http://en.wikipedia.org/wiki/Spotnet The searchable information (and comments and such) is distributed through Usenet itself.
slaxo 14y ago
I'd give nzb.su a try. I started using them a few weeks ago as an auxiliary when I was having issues with nzbmatrix. In my short few weeks of using it, the API seems better than nzbmatrix, but the community / comments on files is near nonexistent. If you're only looking for something to hook up to sickbeard / couchpotato though, it should do the job just fine.
- Down_n_Out 14y ago
  Can't sign up for nzb.su at the moment so it seems, together with a lot of others out there ... Nice, but not really.

by skippy 14y ago

The masses need to pick a new site to invest their energies in, rather than scatter at random to lots of different sites! Any particular one out there that's better than the rest?

deleted 14y ago

[deleted]

troels 14y ago

As someone who hasn't used usenet for downloading content, why is it that it seems that a few central indexing services are the way people access this medium? Wouldn't it be possible - and preferable - to distribute the index through p2p? That way, there would be no single point of failure.

nwh 14y ago
The index would be insanely huge. A lot of these indexing sites are used for their API. Dedicated applications like SickBeard automatically compile lists of wanted files (movies, TV rips and music) and use them to search against the provider. The provider returns a file (NZB) that contains references to hundreds of encoded files hosted on newsgroups. Here's a breakdown of one NZB, for a single 3.88GB file. Inside are 74 RAR files, and 15 additional PAR files containing parity for recovering corrupted data. Each of these 89 files is split into 37 parts of 1.4MB for ~3200 pieces total. These pieces are then encoded with yENC (similar to base64) and then uploaded to the newsgroup. Every single one of these yENC pieces is referenced in the NZB file, making for a total of 400KB. Just because of this, the NZB indexs are enormous; hundreds of gigabytes, probably terabytes in some cases.
- morsch 14y ago
  There's no need for everybody to serve the entire index. Set up a DHT and let interested parties exchange the NZBs. All problems in computer science can be solved by another level of indirection.
  - trotsky 14y ago
    it's a reputational and data quality issue as well. completely decentralized content distribution doesn't usually work unless there is some kind of curation, there are just too many motivations for disruption. it's not an impossible problem, but long ago ceased being mostly about the technical occupation of shipping bits between nodes.
    - pyre 14y ago
      You could always have sites that only serve up indexes of NZB files (e.g. "NZB file => SHA-256"). Then you could use these to determine which NZB files to trust. I would assume that distributing a "NZB => Checksum" hash would be a lot smaller/easier than running an entire site on top of a terabyte of NZB files.
  - rektide 14y ago
    DebTorrent was to be a set of BitTorrent extensions to allow the Debian archive to be syndicated as one ever-growing torrent. Seems similar to the use case here, and free from the problem we have now which is the huge indexes getting hit by takedown notices. http://wiki.debian.org/DebTorrent http://wiki.debian.org/DebTorrent
- asdfaoeu 14y ago
  You can always just upload the nzb file to Usenet and reference that.
  - nwh 14y ago
    That's usually where the indexing sites get them from in the first place. The issue comes when you're a client that's not indexing continuously; downloading billions of headers to try and find a particular NZB is a mammoth task.
    - pyre 14y ago
      > downloading billions of headers to try and find a > particular NZB is a mammoth task This is true. Even a ~10 of years ago, downloading just the headers on some of the newsgroups shot my cache directory into the gigabytes (for a single newsgroup).
    - foobarqux 14y ago
      Why aren't there subgroups with just nzbs?
Wicher 14y ago
Such a project exists: Spotnet http://en.wikipedia.org/wiki/Spotnet http://en.wikipedia.org/wiki/Spotnet The searchable information (and comments and such) is distributed through Usenet itself.

slaxo 14y ago

I'd give nzb.su a try. I started using them a few weeks ago as an auxiliary when I was having issues with nzbmatrix. In my short few weeks of using it, the API seems better than nzbmatrix, but the community / comments on files is near nonexistent. If you're only looking for something to hook up to sickbeard / couchpotato though, it should do the job just fine.

Down_n_Out 14y ago
Can't sign up for nzb.su at the moment so it seems, together with a lot of others out there ... Nice, but not really.