6 ms·
The Dropbox terabyte conundrum
- huhtenberg 12y agoThe actual link - http://sixcolors.com/post/2014/09/the-dropbox-terabyte-conundrum/ http://sixcolors.com/post/2014/09/the-dropbox-terabyte-conun...
- jude- 12y ago(Disclaimer: tooting my own horn) It sounds like you want my PhD thesis project. Link: https://github.com/jcnelson/syndicate https://github.com/jcnelson/syndicate
- PaulHoule 12y agoDo you live in Italy or are you stuck in the U.S. where it would take few months for most people to upload a TB of data to dropbox?
- harryh 12y agoItaly has an average connection speed of 5.2Mbps (47th in the world), the US is ranked 12th at 10.5 Mbps - about 2x Italy. Just so you know. http://www.akamai.com/dl/akamai/akamai-soti-q114.pdf http://www.akamai.com/dl/akamai/akamai-soti-q114.pdf
- suprjami 12y agoAre you seriously so well-trained by Apple that you think you need to pay for software to run a local LAN Dropbox clone like rsync/btsync/ownCloud?
- coldtea 12y agoWhat does Apple have to do with anything? The same problem exists on Windows and Linux. And, no, I don't want to run and manage a local "LAN Dropbox clone". It's all about the integration with different mobile and desktop apps and services, the remote backup, and that you don't have to manage it.
- ivyirwin 12y agoI think you've articulated a great point about the current trend in the Cloud services and fast SSD personal computers. I have personally stayed away from the large storage plans at services like Dropbox because, as you point out, they would fill my entire 128GB Macbook Air instantly. But that seems to be the common theme with laptops these days – blazing fast hard drives at the cost of larger storage. I actually think iTunes Match does a good job of solving this problem, on a media storage basis. I have my whole library "uploaded" to iTunes, but then I can selectively download and listen to songs on any device when I need them, even to my 8GB phone. Now my music library doesn't clog up my computer or phone or tablet, etc. If dropbox found a similar model, I would sign up for that in an instant.
- nihaody 12y agoThey basically need this feature: http://windows.microsoft.com/en-US/windows-8/onedrive-online-available-offline http://windows.microsoft.com/en-US/windows-8/onedrive-online... Totally agree though. I'm not going to pay for a TB of storage until I can moves files I don't need on my SSD anymore (ie RAW photos I've developed) into Dropbox and have them removed from my computer seamlessly. Sooooo easy to implement, just upload the file. When it's done, run an md5 on the local and the cloud, if same then delete local.
- techrat 12y agoI've been using my Dropbox space as a way to turn my systems into a thin(ner) client. I install portable (ini and not registry based) Windows apps into d:\Dropbox that I know can be run in Wine and run them in Wine on my Linux systems from ~/Dropbox Best part of it is that UI changes done on one system remain consistent when using other machines. Completely took the pain out of a format/reinstall, too. Trying to constrain Home to my Dropbox folder also reawakened an old skill: Space discipline. With drive space exploding and SSDs still being relatively constrained, I had forgotten how much crap I tended to hoard and not sort through until I was feeling pinched again thanks to my Dropbox quota.
- lelandbatey 12y agoThis is also what I do, but almost the inverse: I use Cygwin on Windows to get all the same applications (I do everything other than browse the web in my terminal) as I would on Linux, then use Dropbox to keep everything the same across machines. Each of my machines is configured (almost) exactly the same, and all the media/projects/work is available on all my computers at the same time. I also experience the forced "space discipline" and would add that I also have to have a better organizational structure in general. I'm constantly "re-balancing the tree" that is my directory structure, re-defining and re-grouping my media into ever finer groups in an effort to stop things from accumulating in digital "junk drawers". On a different note, I disagree with the suggestion of having a "virtual filesystem" that is the entire Dropbox service. I feel that doesn't fit with what Dropbox is: a folder that syncs. I don't want to have to worry about the physical location of my files, which I most definitely would have to do based on my internet speed (which is as good as it can get right now).
- Osmium 12y agoYeah. I have this problem too. I want to upload a ~ 200 GB photo library so I have access to it on the go now that Everpix has shut down, but I can't do it from my main machine because the drive just isn't big enough. Selective sync seems to be the only option. I hoped iCloud Drive would solve this: iCloud Drive supports similar capacities, but obviously iPhones can't store the whole Drive so presumably it's been designed with this in mind, but it seems like the OS X version still syncs the entire thing (at least I haven't been able to find any information to the contrary).
- deleted 12y ago[deleted]
- davidu 12y agoI have this exact problem. The original reason I use Dropbox is now the thing that annoys me about trying to just store all my files there in the cloud. It's like I need a ~/Dropbox and a ~/DropboxCloud where the ~/DropboxCloud is just virtual, not real disk space on my HDD.
- jmathai 12y agoBitcasa's Infinite Drive is exactly what you're describing for ~/DropboxCloud. Unfortunately it looks like they've changed focus to the enterprise. https://bitcasa.com https://bitcasa.com
- daveloyall 12y agoI don't know anything about dropbox, but based on the types of things the author spends time thinking about, I think s/he might find http://git-annex.branchable.com/assistant/ http://git-annex.branchable.com/assistant/ useful.
- angersock 12y agoSo, the proposed solution is basically to regress to FTP. That's why my old startup offered FTP with no frills: it just works.
- ramLlama 12y agoThis is a very interesting problem, espeicially since consumer computing devices quickly shunted to small storage space when SSD's arrived. A few years ago, a laptop with 500GB of hard drive space was rather small! Nowadays, a 500GB SSD is top-tier. Granted, SSD capacity is quickly rising, but with the combination of small computing like phones and tablets, space is at a premium. Funnily enough, this problem has been solved before! If you look at distributed file systems, especially something like Coda (http://en.wikipedia.org/wiki/Coda_(file_system) http://en.wikipedia.org/wiki/Coda_(file_system)), they are designed to make the local computer a "thin client for storage". Basically, local storage is used as a cache for the main copy on the server, and this behavior is transparent to user applications via the FS driver. Dropbox uses a user-space program to sync the files and cannot intercept system calls. As such, they had no choice but to synchronize all of it rather than bringing things in "on-demand" and releasing local copies that are not used. Nowadays, this can easily be done via FUSE driver in Linux. I do not know if something like FUSE exists on other platforms though. Overall, I personally love the idea of a local machine being a "cache" for the server copy. However, the technical challenges are greater than a simple mirroring scheme, and there may be UX issues as well.
- btown 12y agoDokan for Windows (last updated 2011) and MacFUSE for OS X do exist, but they're not battle-tested, to say the least. It wouldn't be wise for Dropbox to stake its reputation on such unreliable software.
- jrochkind1 12y agoMacFuse worked well when it was first released, but it has not been maintained in years and has no maintainers and does not really work with any recent OSX versions at the moment. It looks like there might be something called OSXFuse which might be more recent and maintained, but I haven't looked at it, it also might not be.
- hemancuso 12y agoOSXFuse is very well battle tested. We use it in ExpanDrive. CBFS by eldos is a commercial userspace filesystem lib for windows. We also use that.
- joshstrange 12y agoI have the same problem. My MBP right now only has a 256GB SSD in it and so I can only fit about 1/8 or less of my DB on it once I load up other data/apps. I had to create a "Camera Uploads Archive" folder and move all my pictures to that then turn on selective sync for it because the "Camera Uploads" folder got way to big to keep on my MBP. I really wish I could still see the "Camera Uploads Archive" and if I tried to open or use one of the files it would download it until after I had finished with it. I'd love to be able to browser all my picture but I just don't have the space.
- spindritf 12y agoI don't think they want you to use 1000GB of storage so the fact that you won't is a feature. Terabyte limit essentially means "unlimited but don't do anything silly." Dropbox has a really good Linux CLI client so I'm sure there are people who will manage to bump up against that limit.
- mbesto 12y agoJust use Selective Sync: https://www.dropbox.com/en/help/175 https://www.dropbox.com/en/help/175
- tonyhb 12y agoI don't know if you read the post, but he says: > Right now the only way to do that would be to put something in the box, then turn off Selective Sync everywhere for that folder. (Or upload files via Dropbox’s web interface, to a folder that’s been unchecked on all your devices via Selective Sync.) > It’s not ideal. Selective Sync wasn’t meant to be used in that way… but for that matter, neither was Dropbox. I think his gripe is the core premise of dropbox — file syncing — isn't the same as 1TB backup.
- baudehlo 12y agoIt really helps to read the article - this is discussed.
- deleted 12y ago[deleted]
- cmkrnl 12y agoCamlistore (https://camlistore.org/ https://camlistore.org/) is the answer. Here's hoping it becomes just as easy to use as Dropbox someday.
- hemancuso 12y agoShameless plug for my product, ExpanDrive http://www.expandrive.com/expandrive http://www.expandrive.com/expandrive Mount Dropbox as a network drive. Smart local cache. Access the data on demand without syncing the repo in first. Also supports gdrive, s3, sftp, onedrive, box and more. Makes a 1TB account make a lot more sense if you only have a 128GB SSD. Use selective sync with the primary client to only sync a portion of your Dropbox. Then use ExpanDrive to offload the rest and access it as needed.
- yid 12y ago> Shameless plug for my product, ExpanDrive Please take this as a compliment, but I didn't realize you needed to "plug" ExpanDrive. I sort of assumed that it was produced by a large company. Anyway, thank you for a wonderful piece of software, and for continually developing it.
- marban 12y agoAny arguments over https://itunes.apple.com/us/app/airfile-cloud-manager-for/id747595327?mt=12 https://itunes.apple.com/us/app/airfile-cloud-manager-for/id... + Any plans for an iOS version?
- hemancuso 12y agoYeah ExpanDrive is a full network filesystem. Air file is like a drag and drop ftp client. iOS version is almost done. Tried to get it out for iOS 8 launch, but didn't quite get there.
- deleted 12y ago[deleted]
- lelandbatey 12y agoOn the product page, please list your price more clearly. It was non-trivial to figure out what your product actually cost. A walkthrough of my thoughts: 1. He said product, so it's probly not free. I wonder how much it costs? 2. Scrolls through the whole page I don't see a breakdown of price. Hmmmm 3. ctrl-f "$" Ahh, I see it says upgrade pricing is $24.99. Wait, does it cost that much, or is that a special "upgrade only" price? 4. looks around on the page more Ahhh, there's a store link, I hope it lists the price there! 5. Ok, so the price is $50 for a single-user license. Why was that so hard to find? I don't mean to be rude with these remarks; your product seems perfectly relevant and it's awesome if it works as you say. But as a customer, I found that a bit confusing and I thought you as the business owner may want some feedback about one particular users experience.
- alimoeeny 12y agoI personally like https://bitcasa.com https://bitcasa.com in this regard. They have this feature that you can choose a "directory" to be a bottomless storage that is not actually on your device, it is on the cloud for you when you need it.
- bbx 12y agoAh! Same kind of user (paying customer for years) and same issues (too much space on my Dropbox, not enough on my drive). My Macbook Air is the only computer I use, both at home and at work. Its SSD is 60 GB. Last month, Dropbox upgraded my account (for free) up to 1 TB. Selective Sync? It's not optimal. We use several shared folders at work. Some of them become rapidly heavy without me noticing, unless I sense my laptop slowing down and find the culprit folder (with Diskwave) and unselect it from my sync. Plus: any (non-empty) shared folder you join will automatically sync on all your devices. It should be opt-in, not opt-out. Offloading massive files? I use the (very good) web interface. It's a two-step process though: upload via the browser, and then delete your file. It's usually a one-off event, but is not what Dropbox is usually about: being seamless. Remember Doug's pitch to YC? The level of abstraction where you just hit "Save" and you're done? That's what's still missing with one-way transfers to the cloud.
- cortesoft 12y agoJust because the author uses a machine that doesn't have that big a hard drive doesn't meant that everyone else does. I use dropbox on a computer with 15TB of HD space.... so having a 1TB drop box is great. Saying drop box should not offer the bigger size just because he doesn't have that much hard drive space is a very limited view.
- jaimebuelta 12y agoI'd say that 15TB is not a common case. Most of the hard drives installed in new computers these days are less than 2TB, and less than 1TB on laptops (which are very popular). At the moment Dropbox has only a (free) plan of 2GB and another of 1TB. I think this is something that needs to be managed by Dropbox, if they want users to be happy.
- deleted 12y ago[deleted]
- dsl 12y agoYour Dropbox is limited to the free space on your computer with the least amount of storage. I have a Super Mega Death Rig with multi-TB of storage too... I also use an old Toughbook with a few GB of space free when I travel to the third world.
- vader1 12y agoWuala has supported this for as long as I can remember, it's called "WualaDrive": a network mount of your entire file archive.
- ryankshaw 12y agoI just want to point out that spacemonkey (spacemonkey.com) handles this very elegantly. spacemonkey was designed from the beginning to handle storing much more data than you'd ever be able to store on your laptop hard drive. The way it does this is it actually mounts itself as a FUSE filesystem and keeps it's own LRU cache of your spacemonkey data on the drive. If you access a recently created or accessed file it will just load from the disk cache. If it detects that you are running out of hard drive space it will automatically prune the stuff that hasn't been accessed in a while (so spacemonkey never causes you to have an "Out of disk space" error). you still see it though, it tells you its size, metadata etc. and if you try to open one of those files it will grab it from the network transparently behind the scene for you it just might add a little bit of latency (as if you were accessing something on a network share). In 2014 you should not have to be thinking about what to "selectively sync" the service you use should just figure it out for you (and let you customize it if you need to) disclamer: I used to work for spacemonkey but do not any more, although I think they make a great product and are a super smart team.
- gregd 12y agoSpaceMonkey was recently acquired by Vivint, Inc. "The Space Monkey team is excited to announce that the company has been acquired by Vivint, Inc." I don't hold out any hope that SpaceMonkey as a product will continue on, or if this was more or less an acquihire.
- alenlpeacock 12y agoCo-founder of Space Monkey here. I can assure you this was not an acquihire. At the highest levels of the org from CEO and down, Vivint believes in the Space Monkey vision. We're super excited to have their backing, and think you will be excited to see where this goes from here.
- abroncs 12y agoI never bought a pro account because, like the OP says, I consider Dropbox for smaller things. 100 GB was too much then, and now 1TB is ridiculous. I would have liked a smaller tier (20-25 GB at the most), but I guess they can't really make money if people actually use up their space. oh well..
- 1123581321 12y agoYou can get that much space with referrals. I maxed my referrals out with a $40 Adwords campaign a few years ago and it's been a good investment.
- mark_l_watson 12y agoI don't understand the point of the article that 1 terabyte of storage is not worthwhile if your laptop has a small disk drive. I use a MacBook Air with only 128GB of storage. I use "selective syncing" to not keep most of my Dropbox files on my laptop. If I need anything not synced to my laptop, then I choose it to be synced, wait a short while, and then it is available locally.
- orenmazor 12y agoyou can do this pretty easily with bittorrent sync
- ShaunK 12y agoI've often wished Dropbox's selective sync was managed by folder rather than by client. It would be more helpful to me to mark folders to not automatically sync account wide than to have to choose and re-choose those folders every time I setup Dropbox. I feel like that would potentially be helpful in this use case.
- CPAhem 12y agoThe selective sync is one way to manage the larger storage space, and do it effectively.
- vex 12y agoWhine whine whine, I have a mac, whine whine whine get a computer with actual expandable storage
- objclxt 12y agoLike a Chromebook, right?
- honestcoyote 12y agoSeems like Dropbox has already solved this problem on mobile devices. Using the Dropbox client on my Nexus phone or on the iPad, I can read or edit a file, and then mark it for offline use which keeps it in local storage, or just leave it alone, where I assume it's eventually deleted from the cache. It's pretty transparent and intuitive. I would think this could be done on desktop / laptop systems, maybe via an optional client, with all files set initially to a no-sync value.
- cr4zy 12y agoI just store large files I don't want synced in Google Drive and don't run the Google Drive client. Dropbox selective sync is a huge pain when setting up a new machine since it doesn't recognize yet unsynced folders and files.
- dwg 12y agoI'm surprised to see so many people saying they have the same "problem". You don't have to use the extra space. You had less space before and the problem didn't exist, and now suddenly it does?? It makes no sense. 1. It's a good thing for the many people who DO have more space on their computer. 2. It's a good thing because it leaves room for growth. 3. It's a good thing because it makes the service more useful for power users. hit "Save" and you're done? Newsflash: it STILL works that way!!
- jrochkind1 12y agoThe "problem" is that they WANT to use the extra remote storage, with a UI as good as DropBox's and consolidated with the other stuff they have on DropBox, but can't because they don't have enough local space or don't want to sync all their remote storage to every machine. It is true that in some cases they may not have realized they had this desire until ExpanDrive gave them enough storage to think about it, but they do now.
- ycombosnator 12y agoI think this limitation is actually fundamental to Dropbox's (and Google/etc) pricing model. Only a small subset of users can fully utilize the space they pay for, which means the remainder of users are more profitable. If the average user consumed 1TB, or if pricing was by usage rather than tiered, I'd expect the price per GB to go up.
- laacz 12y agoThe only reason I use Dropbox is because it works beyond any type of firewall (be it simple HTTP proxy or socks). Others tend not to.
- kyriakos 12y agoThis is exactly the problem Microsoft solved with OneDrive. You can choose what is kept only online and what is also offline. In windows at least its integration goes as far as keeping lower resolution versions of your photos offline for previewing purposes and seamlessly downloading the full-res when you try to open it in say Photoshop. Unfortunately as nice as this sounds in practice its extremely slow (at least for us in Europe). I am facing the same problem with hubiC which has a 10GB plan only and unless you have some crazy raid configuration on your desktop there is no way you can fill it up unless you use the web interface exclusively.