3 ms·

Have you got any real evidence behind your first paragraph?

by dash2 2y ago

michaelt 2y ago
I don't follow gwern's work closely. But I do know he created an enormous dataset of anime images used to train machine learning and generative AI models [1]. Hosting large datasets is moderately expensive - and it's full of NSFW stuff, so he's probably not having his employer or his college host it. Easy for someone on a six-figure salary, difficult for a person on $12k/year. Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff. [1] https://gwern.net/danbooru2021 https://gwern.net/danbooru2021
- esdf 2y ago
  Hosting large datasets can be expensive but the hosting for the danbooru datasets was not. It's "only" a few terabytes in size. A previous release was 3.4TB, so the latest is probably some hundreds of GB, to a TB~, in size larger. The download was hosted on a hetzner IP, which is a provider known for cheap servers. You can pay them $50/m for a server with "unmetered" 1gigabit up/down network + 16TB of disks. $600 a year would not be difficult.
  - itake 2y ago
    $600 a year is a lot of money..
  - gwern 2y ago
    I think people tend to wildly overestimate how expensive hosting a large dataset has to be, because of cloud. If you only need a few terabytes, you can rent from Hetzner for more like <$20/month. Maybe <$10 at this point, if you are patient. And you can use the server for other things too, since it's a fixed cost, like your website or encrypted backups or other files you host. I spent a while figuring this out before I decided to do Danbooru20xx, to make sure it was very cheap. Given how my website has grown in size, and how absurdly exorbitant cloud bandwidth and object/hard drive space is, the Danbooru dataset was practically free at the margin! > Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff. Some people are, some people aren't. I'm not. (I have intellectual sympathy with EA, but not emotional.)
freejazz 2y ago
I think it would be more odd to take someone with such a shtick at their word, but skepticism abounds either way.
Quinzel 2y ago
Do people need evidence for an opinion?

Have you got any real evidence behind your first paragraph?

by dash2 2y ago

Have you got any real evidence behind your first paragraph?

I don't follow gwern's work closely. But I do know he created an enormous dataset of anime images used to train machine learning and generative AI models [1]. Hosting large datasets is moderately expensive - and it's full of NSFW stuff, so he's probably not having his employer or his college host it. Easy for someone on a six-figure salary, difficult for a person on $12k/year. Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff. [1] https://gwern.net/danbooru2021 https://gwern.net/danbooru2021

esdf 2y ago
Hosting large datasets can be expensive but the hosting for the danbooru datasets was not. It's "only" a few terabytes in size. A previous release was 3.4TB, so the latest is probably some hundreds of GB, to a TB~, in size larger. The download was hosted on a hetzner IP, which is a provider known for cheap servers. You can pay them $50/m for a server with "unmetered" 1gigabit up/down network + 16TB of disks. $600 a year would not be difficult.
- itake 2y ago
  $600 a year is a lot of money..
- gwern 2y ago
  I think people tend to wildly overestimate how expensive hosting a large dataset has to be, because of cloud. If you only need a few terabytes, you can rent from Hetzner for more like <$20/month. Maybe <$10 at this point, if you are patient. And you can use the server for other things too, since it's a fixed cost, like your website or encrypted backups or other files you host. I spent a while figuring this out before I decided to do Danbooru20xx, to make sure it was very cheap. Given how my website has grown in size, and how absurdly exorbitant cloud bandwidth and object/hard drive space is, the Danbooru dataset was practically free at the margin! > Also, I thought these lesswrong folks were all about "effective altruism" and "earning to give" and that stuff. Some people are, some people aren't. I'm not. (I have intellectual sympathy with EA, but not emotional.)

freejazz 2y ago

I think it would be more odd to take someone with such a shtick at their word, but skepticism abounds either way.

Quinzel 2y ago

Do people need evidence for an opinion?