5 ms·

I recently did a comparison of cloud storage options: - If you want "personal storage" (no programmatic access) the options are generally around $5 per TB/mont

I recently did a comparison of cloud storage options: - If you want "personal storage" (no programmatic access) the options are generally around $5 per TB/month - but that's a lot of drag-and-dropping and praying the connection doesn't die during transfer. - If you want object storage (think S3) the cheapest is $5 per TB/month, the average is $10 per TB/month, and the "high end" is $20 per TB/month, with extra costs for bandwidth ranging from $10 to $120 per TB/month. Honestly, just store less crap. Marie Condo your digital life. Does it not spark joy? Have you not looked at it in the last 6 months? Does it not serve a useful purpose, such as tax records? Ditch it. Even if you want to keep some pictures/video, either print a copy in original-quality, or compress/downscale it. I took a 3 minute video on my phone and it's 425MB. And it was still grainy! I used to download two-hour movies that were 725MB and looked like a DVD! errrrrrr... I mean, I heard about a guy that did that.

SilverRed 5y ago
>And it was still grainy! Because the grain is not from video compression but because the camera sensor wasn't receiving enough light so it boosted the ISO to compensate which causes grain. Also the fact that phones have to live encode video which is very non optimal while movies are shot uncompressed and then a powerful CPU can spend as long as it needs for a perfect compression.
- deathanatos 5y ago
  The CPU in a real machine makes a huge difference: the last video I shot on my phone went from 3.9M → 1.6M, ~59% compressed. (With no noticable quality loss, but the original isn't winning any Oscars…)
  - SilverRed 5y ago
    And your phone is already pretty well off. Cheap dashcams spit out awful quality video with large file sizes since they don't have the benefit of expensive encoder chips.
    - barrkel 5y ago
      OTOH with the unambitious encoding parameters they are very easy to cut with ffmpeg, or at least the MP4s produced by my helmet camera are. I mean, the cheaper the hardware, the fewer frames it'll be able to buffer, so I-frames should come pretty regularly. I wrote a very simple video editor: video playback, some scrubbing controls on the arrow keys to jump around, and space bar to mark / unmark segments to keep (capturing timestamps). Run ffmpeg to cut the bits out then move source video to a different directory, ready to be deleted. This cut down on my video volume 100x, while I still retained a few old commute videos, as a keepsake for that time in my life.
- GrayShade 5y ago
  Also, noisier video will probably compress worse.
killjoywashere 5y ago
This becomes exhausting. Like, it becomes its own workload. There are some decent workflow tools for iPad that make it tolerable to work on while vegging out in the evening or on a plane, but it's definitely a chore to Marie Condo your digital life. And we haven't even gotten to the fragmentation of life as you move, move jobs, etc.
- fridif 5y ago
  Agreed. $10 for some TBs is much cheaper than the time I spend deciding if I will need something or not. Very similar to just buying a few more items at the grocery than I normally would. My time is so incredibly important that it's not worth $20 in wasted items to have to potentially drive back there if I've missed something. If I don't have as much time as possible to spend on working/innovating/personal health and family, my income source will dry up and then I'm in dire straits ;)
- freddie_mercury 5y ago
  Yeah, a few times I've tried to prune my collection of photos/ripped-mp3s/movies/whatever. I've always given up after I'm 1 hour in and still have like 98% of the collection to go. It is kinda like how the old Yahoo-organized-like-a-library died out in favor of the Google-just-search-everything approach. Unless you're triaging as you go (which still takes tons of time; do people really go through 1,000 photos after their 3 week holiday to Iceland and keep just the 15-20 best?) it is easy to build up an insurmountable backlog where the only viable option is just "delete everything". I actually did that with my old collection of ripped-from-CD mp3s and just resigned myself to streaming anything I really cared about. But you can't exactly do that with "family photos".
nixpulvis 5y ago
I just bought a pretty high quality 4TB internal HDD for like $120. Compress an archive and that can hold quite a lot for quite a while. Next, I might need to build a little blueray changer to burn massive quantities of optical media.
throwaway81523 5y ago
Scaleway.com C14 cold archive is $2/TB/month with multi-hour retrieval time, like AWS Glacier. Retrieval uploads to a normal S3 object store that costs more but you don't have to leave it there long. Outbound bandwidth from the S3 store is .01/GB but alternatively I think you can retrieve through a Scaleway hourly VPS and then the bandwidth is essentially free.
berkes 5y ago
The problem with digital storage is the immense difference in filesize. Removing those tax-forms isn't going to help you at all in freeing space, they are probably compressed XML or just PDFs. Yet, for example, that single slideshow for granma's 90th birthday gobbles up 95% of all the diskspace all your presentations combined use. It is far more effective to hunt those large files and go through those only. I have largest.sh and baobab for that, but there are numerous other (GUI) tools for this. Less fun, though. But after seeing my wife spend an entire afternoon sifting through her files (look! I already deleted 22 GB) and me going: did you see the 'Photoshoot 2012-04-01 waalkant (kopie)' full of Raws? That is 210GB. Can it go? done! Then I realised we need better tools and educate people a little here.
- rambambram 5y ago
  This is cool. I triggered on 'waalkant', then I saw your topbanner on Twitter, now I think I've been looking at your house last year from the back of a garbage truck a couple of times. But on topic: indeed, chase these big duplicate files first!
  - berkes 5y ago
    Wow, you probably have the best garbage-truck-run around here, then. Well, it probably is a lot of hanging and little garbage-hauling!
    - rambambram 5y ago
      It was a perfect summer indeed. I called it a 'paid vacation'. Wave to my ex-colleagues in the white truck!
plater 5y ago
What many people forget with these x$/month is that you will have to pay this for the rest of your life. That's why I have chosen an option where you pay a higher fee once (about 3 years cost) and then it does not cost anything going forward. Another issue with storing all these photos for your whole life is of course that once you die, what happens? Who will take charge of all these photos? Who will continue to pay? Who will clean up and take ownership of it? Or will they just be lost once you are dead and all this money for what? The third issue is of course that the more random unsorted pictures we have the less we want to look at it because 90% are bad and only 10% are good. It's important to clean up just after taking the photos.
- XorNot 5y ago
  Head too far down this path of thought and you hit an existential crisis pretty quick though: "why are you storing photos?" "who will look at them?" "will anyone be looking at them in 200 years? 100? 50? 10?" On a long enough timescale we all die and are probably forgotten. I've actually spent a decent amount of time resisting the urge to just delete all my old photos (I have things going back to early high school - taken on a Sony Mavica that used floppy disks). There's a very liberating feeling to the idea of just having no history at all - probably similar to the appeal of the idea of having all your works crumble to dust when you die, which is also something I've seen people refer to at different times. But, since I haven't done that yet, and since managing an on-site server or disks is it's own stress (i.e. eventually my house will burn down or flood and I'll lose everything...so what was the point?), then paying that $x/month is basically the price I pay to defer having to really think about it.
  - msh 5y ago
    Unless I am dead I feel pretty confident that I want to see my photos and videos of my kids growing up in 50 years time (it would make me 90)!
    - throwaway290232 5y ago
      That's why I recommend physical prints. Hopefully you would put them somewhere where you actually look at them from time to time. Or put them in a safe deposit box (assuming you trust those). The problem of "what to keep" isn't even digital-specific. In my youth I probably bought two dozen disposable cameras, had the pictures printed, even paid for the CD-ROMs when they became available. Where are those pictures now? Who knows! That's why I like Marie's message: it's not about throwing away junk, but keeping what you love close to you.

SilverRed 5y ago

>And it was still grainy! Because the grain is not from video compression but because the camera sensor wasn't receiving enough light so it boosted the ISO to compensate which causes grain. Also the fact that phones have to live encode video which is very non optimal while movies are shot uncompressed and then a powerful CPU can spend as long as it needs for a perfect compression.

deathanatos 5y ago
The CPU in a real machine makes a huge difference: the last video I shot on my phone went from 3.9M → 1.6M, ~59% compressed. (With no noticable quality loss, but the original isn't winning any Oscars…)
- SilverRed 5y ago
  And your phone is already pretty well off. Cheap dashcams spit out awful quality video with large file sizes since they don't have the benefit of expensive encoder chips.
  - barrkel 5y ago
    OTOH with the unambitious encoding parameters they are very easy to cut with ffmpeg, or at least the MP4s produced by my helmet camera are. I mean, the cheaper the hardware, the fewer frames it'll be able to buffer, so I-frames should come pretty regularly. I wrote a very simple video editor: video playback, some scrubbing controls on the arrow keys to jump around, and space bar to mark / unmark segments to keep (capturing timestamps). Run ffmpeg to cut the bits out then move source video to a different directory, ready to be deleted. This cut down on my video volume 100x, while I still retained a few old commute videos, as a keepsake for that time in my life.
GrayShade 5y ago
Also, noisier video will probably compress worse.

killjoywashere 5y ago

This becomes exhausting. Like, it becomes its own workload. There are some decent workflow tools for iPad that make it tolerable to work on while vegging out in the evening or on a plane, but it's definitely a chore to Marie Condo your digital life. And we haven't even gotten to the fragmentation of life as you move, move jobs, etc.

fridif 5y ago
Agreed. $10 for some TBs is much cheaper than the time I spend deciding if I will need something or not. Very similar to just buying a few more items at the grocery than I normally would. My time is so incredibly important that it's not worth $20 in wasted items to have to potentially drive back there if I've missed something. If I don't have as much time as possible to spend on working/innovating/personal health and family, my income source will dry up and then I'm in dire straits ;)
freddie_mercury 5y ago
Yeah, a few times I've tried to prune my collection of photos/ripped-mp3s/movies/whatever. I've always given up after I'm 1 hour in and still have like 98% of the collection to go. It is kinda like how the old Yahoo-organized-like-a-library died out in favor of the Google-just-search-everything approach. Unless you're triaging as you go (which still takes tons of time; do people really go through 1,000 photos after their 3 week holiday to Iceland and keep just the 15-20 best?) it is easy to build up an insurmountable backlog where the only viable option is just "delete everything". I actually did that with my old collection of ripped-from-CD mp3s and just resigned myself to streaming anything I really cared about. But you can't exactly do that with "family photos".

nixpulvis 5y ago

I just bought a pretty high quality 4TB internal HDD for like $120. Compress an archive and that can hold quite a lot for quite a while. Next, I might need to build a little blueray changer to burn massive quantities of optical media.

throwaway81523 5y ago

Scaleway.com C14 cold archive is $2/TB/month with multi-hour retrieval time, like AWS Glacier. Retrieval uploads to a normal S3 object store that costs more but you don't have to leave it there long. Outbound bandwidth from the S3 store is .01/GB but alternatively I think you can retrieve through a Scaleway hourly VPS and then the bandwidth is essentially free.

berkes 5y ago

The problem with digital storage is the immense difference in filesize. Removing those tax-forms isn't going to help you at all in freeing space, they are probably compressed XML or just PDFs. Yet, for example, that single slideshow for granma's 90th birthday gobbles up 95% of all the diskspace all your presentations combined use. It is far more effective to hunt those large files and go through those only. I have largest.sh and baobab for that, but there are numerous other (GUI) tools for this. Less fun, though. But after seeing my wife spend an entire afternoon sifting through her files (look! I already deleted 22 GB) and me going: did you see the 'Photoshoot 2012-04-01 waalkant (kopie)' full of Raws? That is 210GB. Can it go? done! Then I realised we need better tools and educate people a little here.

rambambram 5y ago
This is cool. I triggered on 'waalkant', then I saw your topbanner on Twitter, now I think I've been looking at your house last year from the back of a garbage truck a couple of times. But on topic: indeed, chase these big duplicate files first!
- berkes 5y ago
  Wow, you probably have the best garbage-truck-run around here, then. Well, it probably is a lot of hanging and little garbage-hauling!
  - rambambram 5y ago
    It was a perfect summer indeed. I called it a 'paid vacation'. Wave to my ex-colleagues in the white truck!

plater 5y ago

What many people forget with these x$/month is that you will have to pay this for the rest of your life. That's why I have chosen an option where you pay a higher fee once (about 3 years cost) and then it does not cost anything going forward. Another issue with storing all these photos for your whole life is of course that once you die, what happens? Who will take charge of all these photos? Who will continue to pay? Who will clean up and take ownership of it? Or will they just be lost once you are dead and all this money for what? The third issue is of course that the more random unsorted pictures we have the less we want to look at it because 90% are bad and only 10% are good. It's important to clean up just after taking the photos.

XorNot 5y ago
Head too far down this path of thought and you hit an existential crisis pretty quick though: "why are you storing photos?" "who will look at them?" "will anyone be looking at them in 200 years? 100? 50? 10?" On a long enough timescale we all die and are probably forgotten. I've actually spent a decent amount of time resisting the urge to just delete all my old photos (I have things going back to early high school - taken on a Sony Mavica that used floppy disks). There's a very liberating feeling to the idea of just having no history at all - probably similar to the appeal of the idea of having all your works crumble to dust when you die, which is also something I've seen people refer to at different times. But, since I haven't done that yet, and since managing an on-site server or disks is it's own stress (i.e. eventually my house will burn down or flood and I'll lose everything...so what was the point?), then paying that $x/month is basically the price I pay to defer having to really think about it.
- msh 5y ago
  Unless I am dead I feel pretty confident that I want to see my photos and videos of my kids growing up in 50 years time (it would make me 90)!
  - throwaway290232 5y ago
    That's why I recommend physical prints. Hopefully you would put them somewhere where you actually look at them from time to time. Or put them in a safe deposit box (assuming you trust those). The problem of "what to keep" isn't even digital-specific. In my youth I probably bought two dozen disposable cameras, had the pictures printed, even paid for the CD-ROMs when they became available. Where are those pictures now? Who knows! That's why I like Marie's message: it's not about throwing away junk, but keeping what you love close to you.