nee1r

4 ms·

1. 31.▲
   by nee1r 1y ago

   We did want more pictures!! Recently bought a Sony A7III to capture more fun moments like this. We're working on pretraining computer action models from the ground up—hence the pretraining data cluster. We're a public benefit corp
2. 32.▲
   by nee1r 1y ago

   Thanks for helping!!!
3. 33.▲
   by nee1r 1y ago

   Around 2-5 hours/month, mostly powercycling the servers and replacing hard drives
4. 34.▲
   by nee1r 1y ago

   We reached out to almost every colocation space in SF/some in Fremont to get quotes. There wasn't a difference between the quote price and what we ended up paying, though we did negotiate terms + one-time costs.
5. 35.▲
   by nee1r 1y ago

   Thanks!! :)
6. 36.▲
   by nee1r 1y ago

   Definitely much less redundancy, this was definitely a tradeoff we made for pretraining data and cost.
7. 37.▲
   by nee1r 1y ago

   True, this is a large reason why we chose to have the datacenter a couple blocks away from the office.
8. 38.▲
   by nee1r 1y ago

   We're currently 1/1 for the recent 4.3 magnitude earthquake (though if SF crumbles we might lose data)
9. 39.▲
   by nee1r 1y ago

   Thanks! Lots of hard work went into them.
10. 40.▲
    by nee1r 1y ago

    tips hat back
11. 41.▲
    by nee1r 1y ago

    Already filled up and looking to possibly copy and paste :)
12. 42.▲
    by nee1r 1y ago

    True, though this is specifically for pretraining data (S3 wouldn't sell us used disk + no DR storage).
13. 43.▲
    by nee1r 1y ago

    We want to get darkfiber from the datacenter to the office. I love 100Gbps
14. 44.▲
    by nee1r 1y ago

    yeah, exactly! we have a 100G uplink, and then we use nginx secure links that we then just curl from the machines using HTTP. (funnily HTTPS adds overhead so we just pre-sign URLs)
15. 45.▲
    Building the heap: racking 30 petabytes of hard drives for pretraining(si.inc)

    412 points by nee1r 1y ago|274 comments