3 ms·

Why doesn't HF support BitTorrent? I know about hf-torrent and hf_transfer, but those aren't nearly as accessible as a link in the web UI.

by sowbug 8mo ago

embedding-shape 8mo ago
> Why doesn't HF support BitTorrent? Harder to track downloads then. Only when clients hit the tracker would they be able to get download states, and forget about private repositories or the "gated" ones that Meta/Facebook does for their "open" models. Still, if vanity metrics wasn't so important, it'd be a great option. I've even thought of creating my own torrent mirror of HF to provide as a public service, as eventually access to models will be restricted, and it would be nice to be prepared for that moment a bit better.
- sowbug 8mo ago
  I thought of the tracking and gate questions, too, when I vibed up an HF torrent service a few nights ago. (Super annoying BTW to have to download the files just to hash the parts, especially when webseeds exist.) Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. It's a bit like any legalization question -- the black market exists anyway, so a regulatory framework could bring at least some of it into the sunlight.
  - embedding-shape 8mo ago
    > Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. But that'll only stop a small part, anyone could share the infohash and if you're using the dht/magnet without .torrent files or clicks on a website, no one can count those downloads unless they too scrape the dht for peers who are reporting they've completed the download.
    - sowbug 8mo ago
      Right, but that's already happening today. That's the black-market point.
    - fc417fc802 8mo ago
      > unless they too scrape the dht for peers who are reporting they've completed the download. Which can be falsified. Head over to your favorite tracker and sort by completed downloads to see what I mean.
- taminka 8mo ago
  most of the traffic is probably from open weights, just seed those, host private ones as is
- homarp 8mo ago
  how are all the private trackers tracking ratios?
- jimbob45 8mo ago
  Wouldn’t it still provide massive benefits if they could convince/coerce their most popular downloaded models to move to torrenting?
  - intrasight 8mo ago
    Benefit to you, but great downside to the three letter agencies that inject their goods into these models.
- Barbing 8mo ago
  That would be a very nice service. I think folks might rely on it for a number of reasons, including that we'll want to see how biases changed over time. What got sloppier, shillier...

Why doesn't HF support BitTorrent? I know about hf-torrent and hf_transfer, but those aren't nearly as accessible as a link in the web UI.

by sowbug 8mo ago

Why doesn't HF support BitTorrent? I know about hf-torrent and hf_transfer, but those aren't nearly as accessible as a link in the web UI.

embedding-shape 8mo ago

> Why doesn't HF support BitTorrent? Harder to track downloads then. Only when clients hit the tracker would they be able to get download states, and forget about private repositories or the "gated" ones that Meta/Facebook does for their "open" models. Still, if vanity metrics wasn't so important, it'd be a great option. I've even thought of creating my own torrent mirror of HF to provide as a public service, as eventually access to models will be restricted, and it would be nice to be prepared for that moment a bit better.

sowbug 8mo ago
I thought of the tracking and gate questions, too, when I vibed up an HF torrent service a few nights ago. (Super annoying BTW to have to download the files just to hash the parts, especially when webseeds exist.) Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. It's a bit like any legalization question -- the black market exists anyway, so a regulatory framework could bring at least some of it into the sunlight.
- embedding-shape 8mo ago
  > Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. But that'll only stop a small part, anyone could share the infohash and if you're using the dht/magnet without .torrent files or clicks on a website, no one can count those downloads unless they too scrape the dht for peers who are reporting they've completed the download.
  - sowbug 8mo ago
    Right, but that's already happening today. That's the black-market point.
  - fc417fc802 8mo ago
    > unless they too scrape the dht for peers who are reporting they've completed the download. Which can be falsified. Head over to your favorite tracker and sort by completed downloads to see what I mean.
taminka 8mo ago
most of the traffic is probably from open weights, just seed those, host private ones as is
homarp 8mo ago
how are all the private trackers tracking ratios?
jimbob45 8mo ago
Wouldn’t it still provide massive benefits if they could convince/coerce their most popular downloaded models to move to torrenting?
- intrasight 8mo ago
  Benefit to you, but great downside to the three letter agencies that inject their goods into these models.
Barbing 8mo ago
That would be a very nice service. I think folks might rely on it for a number of reasons, including that we'll want to see how biases changed over time. What got sloppier, shillier...