7 ms·

BitTorrent Records

104 points by Uncle_Sam 16y ago

phamilton 16y ago
I think the Rocks Avalanche Installer ( http://www.rocksclusters.org/rocks-doc/papers/two-pager/paper.pdf http://www.rocksclusters.org/rocks-doc/papers/two-pager/pape... ) is worthy of mention. Any time there is a power outage in our HPC cluster, we have 1000 machines all downloading their new image using bittorrent via Avalanche. We are a small HPC shop. Places like LLNL or TACC have much bigger clusters probably have 10s of thousands of nodes that use Avalanche.
- jacquesm 16y ago
  > Any time there is a power outage in our HPC cluster I don't like the sound of that, how often do you have power outages ?
  - phamilton 16y ago
    Twice in the last year. Our infrastructure is all on UPS, but we decided not to do so for our compute cluster. Additional hardware was deemed a better way to spend our budget. In the event of an outage, people lose the job that was currently running. All their data is safe. They just need to relaunch the job and they are back in business. Most jobs take just a few days, so it really isn't that big of a deal. Either users wait in a queue for an extra day every time they want to run a job, or once or twice a year they have to resubmit a job that died in a power outage. We chose the latter.
    - maukdaddy 16y ago
      Excellent tradeoff in terms of cost/benefit. More companies should re-evaluate their need for 5 9s and instead look at cases where they can gain efficiencies by doing LESS. Does every single company really need a huge UPS system and backup generators?
jmillikin 16y ago
My first thought was "I wonder if their Largest Torrent category counts the lossless Touhou torrent?[1]". Turns out they didn't, though some commenters mention it. It measures in at 848GB (790 GiB), which is incredible considering its content (indie video-game soundtrack remixes). The companion album-art torrent[2] is an additional 20GB. [1] http://www.nyaatorrents.org/?page=torrentinfo&tid=167750 http://www.nyaatorrents.org/?page=torrentinfo&tid=167750 [2] http://www.nyaatorrents.org/?page=torrentinfo&tid=167751 http://www.nyaatorrents.org/?page=torrentinfo&tid=167751
deleted 16y ago
[deleted]
Groxx 16y ago
Largest is 746 gb? Bull. Largest active, perhaps (ignoring the Touhou one mentioned elsewhere), but I've encountered 1.5 tb and greater. Maybe it's been dropped from a few trackers?
- jmillikin 16y ago
  He's only counting public trackers.
  - Groxx 16y ago
    They were public. If I remember correctly, they were on some of the bigger ones as well.
marcamillion 16y ago
Am I the only one that read the title of this post and immediately thought: "BitTorrent Record Label - Oh my Gosh, that's BRILLIANT!" Then was disappointed at the fact that it is just records for file sizes?
- mfukar 16y ago
  That's how I read it too. Wishful thinking, I suppose. :(

104 points by Uncle_Sam 16y ago

I think the Rocks Avalanche Installer ( http://www.rocksclusters.org/rocks-doc/papers/two-pager/paper.pdf http://www.rocksclusters.org/rocks-doc/papers/two-pager/pape... ) is worthy of mention. Any time there is a power outage in our HPC cluster, we have 1000 machines all downloading their new image using bittorrent via Avalanche. We are a small HPC shop. Places like LLNL or TACC have much bigger clusters probably have 10s of thousands of nodes that use Avalanche.

jacquesm 16y ago
> Any time there is a power outage in our HPC cluster I don't like the sound of that, how often do you have power outages ?
- phamilton 16y ago
  Twice in the last year. Our infrastructure is all on UPS, but we decided not to do so for our compute cluster. Additional hardware was deemed a better way to spend our budget. In the event of an outage, people lose the job that was currently running. All their data is safe. They just need to relaunch the job and they are back in business. Most jobs take just a few days, so it really isn't that big of a deal. Either users wait in a queue for an extra day every time they want to run a job, or once or twice a year they have to resubmit a job that died in a power outage. We chose the latter.
  - maukdaddy 16y ago
    Excellent tradeoff in terms of cost/benefit. More companies should re-evaluate their need for 5 9s and instead look at cases where they can gain efficiencies by doing LESS. Does every single company really need a huge UPS system and backup generators?

jmillikin 16y ago

My first thought was "I wonder if their Largest Torrent category counts the lossless Touhou torrent?[1]". Turns out they didn't, though some commenters mention it. It measures in at 848GB (790 GiB), which is incredible considering its content (indie video-game soundtrack remixes). The companion album-art torrent[2] is an additional 20GB. [1] http://www.nyaatorrents.org/?page=torrentinfo&tid=167750 http://www.nyaatorrents.org/?page=torrentinfo&tid=167750 [2] http://www.nyaatorrents.org/?page=torrentinfo&tid=167751 http://www.nyaatorrents.org/?page=torrentinfo&tid=167751

deleted 16y ago

[deleted]

Groxx 16y ago

Largest is 746 gb? Bull. Largest active, perhaps (ignoring the Touhou one mentioned elsewhere), but I've encountered 1.5 tb and greater. Maybe it's been dropped from a few trackers?

jmillikin 16y ago
He's only counting public trackers.
- Groxx 16y ago
  They were public. If I remember correctly, they were on some of the bigger ones as well.

marcamillion 16y ago

Am I the only one that read the title of this post and immediately thought: "BitTorrent Record Label - Oh my Gosh, that's BRILLIANT!" Then was disappointed at the fact that it is just records for file sizes?

mfukar 16y ago
That's how I read it too. Wishful thinking, I suppose. :(