5 ms·

Ever tried committing a 100meg file to a Git repository? It'll peg your CPU to 100% for 45 seconds. I like the idea, but Git isn't a great general blob-store fo

Ever tried committing a 100meg file to a Git repository? It'll peg your CPU to 100% for 45 seconds. I like the idea, but Git isn't a great general blob-store for this kind of project.

nantes 15y ago
This may well be true, but I also don't generally store 100MB files in my (free ~3GB) Dropbox account. So, while it might not work for all needs, it seems to actually meet my Dropbox usage pretty well.
- bravura 15y ago
  I use Dropbox to store my media (photos, videos, and music). I love it. Each time I have a new laptop, I know that all my media will be intact, and I don't need to invest any time to migrate it. Media storage is, I believe, a common Dropbox use case. And Dropbox's transparent uploads, which seem to automatically detect and throttle when I am using bandwidth, are a great feature that I haven't seen in any open-source competitor.
  - nantes 15y ago
    I think one of the reasons I have never considered using Dropbox for media storage is that, given three teenage daughters, I don't think I could afford a big enough Dropbox account to store all of the media we consume and generate. Also, one of my favorite applications of Dropbox is having all of our important documents in one central location (whether this is a wise move or not... ). This makes accessing documents on the fly from my iPhone a breeze. Having media files in there as well, would, I think destroy my antique iPhone 3G! You are right on the money about the background uploading though.
    - pronoiac 15y ago
      Having media files in there as well, would, I think destroy my antique iPhone 3G! The iPhone app doesn't sync everything - it will download (and separately, cache) only the files you request.
      - nantes 15y ago
        Good to know, thanks!
carmen 15y ago
actually one time du -s was surprising - i'd committed a 700 mb iso without realizing it since no unusual delays occured. the 'append only' nature of git makes it unsuitable for this application imo. a sliding expiration/history window size depending on avilable resources is ideal
antics 15y ago
First off, it's not going to "peg your CPU to 100%" for 45 seconds. I just tried a 600 meg file on my dumpy Macbook Pro. 15 seconds tops, and it was never noticeable or prohibitive. Besides, committing is not the bottle neck, it's add, when the hash and gzip are done. And then, besides all that, the major problem with committing blobs to git is that git will store the blob once for each (potentially small) revision that you make. That's not space efficient, but the git community knows about it, and is actively trying to fix it.

Ever tried committing a 100meg file to a Git repository? It'll peg your CPU to 100% for 45 seconds. I like the idea, but Git isn't a great general blob-store for this kind of project.

nantes 15y ago

This may well be true, but I also don't generally store 100MB files in my (free ~3GB) Dropbox account. So, while it might not work for all needs, it seems to actually meet my Dropbox usage pretty well.

bravura 15y ago
I use Dropbox to store my media (photos, videos, and music). I love it. Each time I have a new laptop, I know that all my media will be intact, and I don't need to invest any time to migrate it. Media storage is, I believe, a common Dropbox use case. And Dropbox's transparent uploads, which seem to automatically detect and throttle when I am using bandwidth, are a great feature that I haven't seen in any open-source competitor.
- nantes 15y ago
  I think one of the reasons I have never considered using Dropbox for media storage is that, given three teenage daughters, I don't think I could afford a big enough Dropbox account to store all of the media we consume and generate. Also, one of my favorite applications of Dropbox is having all of our important documents in one central location (whether this is a wise move or not... ). This makes accessing documents on the fly from my iPhone a breeze. Having media files in there as well, would, I think destroy my antique iPhone 3G! You are right on the money about the background uploading though.
  - pronoiac 15y ago
    Having media files in there as well, would, I think destroy my antique iPhone 3G! The iPhone app doesn't sync everything - it will download (and separately, cache) only the files you request.
    - nantes 15y ago
      Good to know, thanks!

carmen 15y ago

actually one time du -s was surprising - i'd committed a 700 mb iso without realizing it since no unusual delays occured. the 'append only' nature of git makes it unsuitable for this application imo. a sliding expiration/history window size depending on avilable resources is ideal

antics 15y ago

First off, it's not going to "peg your CPU to 100%" for 45 seconds. I just tried a 600 meg file on my dumpy Macbook Pro. 15 seconds tops, and it was never noticeable or prohibitive. Besides, committing is not the bottle neck, it's add, when the hash and gzip are done. And then, besides all that, the major problem with committing blobs to git is that git will store the blob once for each (potentially small) revision that you make. That's not space efficient, but the git community knows about it, and is actively trying to fix it.