7 ms·
Engineering Dropbox Transfer: Making simple even simpler
- nicksundin 6y agoAuthor here. Happy to answer questions!
- godzillabrennus 6y agoGreat work on this. I had to send 100GB files last year (1TB in total) and it was extremely difficult to send them as-is. Most clients that upload data choked while attempting it. Had to break the files up into smaller ones using a compression tool. It actually increased the file size overall but the transfer could then be facilitated. Hope to see you guys add a feature like DocSend next. It’s like a missing puzzle piece from a utility perspective. I have no idea who downloads a shared file once a url is generated. Would be nice to request emails and get alerts when it’s downloaded.
- aroch 6y agoIf you use 7zip on Windows or just split/cat on macOS/Linux, you can split without compressing or archiving, saving you unnecessary CPU cycles (and space).
- jdxcode 6y agoYou gotta wonder how much easier it would be to put a stamp on a letter with an SD card in it sometimes
- deleted 6y ago[deleted]
- raun1 6y agoNo Syncthing, rsync, or multi-connection FTPS/SFTP with split archives?
- nicksundin 6y agoHi! Sending groups of files was definitely one of the focus areas here so hope Transfer can be helpful in this role. Re: Tracking content, we actually do provide this in Transfer. If you specify a set of email recipients, rather than copying a shared link, we provide a few tracking features for you. 1) Record who has viewed or downloaded your content. 2) Remind your recipient if the content you sent is about to expire. 3) (Optionally) notify you to let you know when your recipient has downloaded your content.
- psadri 6y agoWhat was the most technically challenging part of the project for you? And do you do any tricks for handling failures during very large downloads or uploads (Eg ranges)
- nicksundin 6y agoThe most technically challenging part was probably wiring the storage & sharing side up to not behave like a traditional Dropbox shared folder or shared link (e.g. not take up storage quota or sync down to your machine). There were certainly a long tail of internal reliability work to get that operating smoothly (there are many internal services to work with here). In terms of tricks at handling failures: The first (and probably most important) thing is finding the failures. Bug bashing sessions and also just reading failure logs can help catch the trickier transient problems our integration or unit tests do not catch. In terms of mitigating them, retries and correct error handlers in code are essential (e.g. Promise.catch case should not be ignored!). Retries on the client side are made a bit easier as we chunk uploads into 4mb blocks so any upload errors of actual content can be retried piecemeal.
- WJW 6y agoWhy is there so much talk about hypotheses and market validation in the article when the product is a straight up clone of WeTransfer almost down to the name? For those who don't know: compare the visual design of https://www.dropbox.com/transfer https://www.dropbox.com/transfer with the design of https://wetransfer.com/ https://wetransfer.com/. Design and location of key elements are super similar.
- DandyDev 6y agoThe author is happy to answer questions unless they're somewhat embarrassing
- yjftsjthsd-h 6y agoThe author answered the question, and did so before you remarked on it.
- nicksundin 6y agoRight, this is a really good question. Dropbox Transfer achieves the same purpose as WeTransfer with a few differentiators. Unpacking this a bit, there are two parts here: 1) For a company with an established sharing model (shared folder, live-updating links), a departure or addition to this needs to be thoughtfully orchestrated. Moving to snapshotting content, gathering content inside and outside your Dropbox, and enforcing expiration in this tool is a very different paradigm. 2) However, specifically because we are also a storage provider, we can provide a much faster sending experience for them as we can remove the upload step, which is especially important for folks sending really large files. We have a large user base storing their content in Dropbox, this is a net new added value/functionality for them.
- deleted 6y ago[deleted]
- txcwpalpha 6y agoEven if the internal discussions were "other services do this and we want a piece of that pie", no company is going to come out in a product launch announcement and directly say "another service is doing this and we want a piece of that pie". And in this case, I would still hope they did do market validation internally, even if it is just a copy of wetransfer. Just because some other company is doing something doesn't automatically mean it's a good fit for your company to do it, too. As for the design, it's pretty hard to say that "a generic file selection panel in the middle of a page and a Sign Up button at the top right of the page where it typically is on most sites" is a "straight up clone". The color schemes are similar, but that's just because the main color branding of both companies happens to be light blue.
- rkagerer 6y agoUser here. Just want to say: 1) Thanks for your hard work advocating for us and for that interesting writeup, and 2) This was never a problem I needed solved until Dropbox killed the Public folder four years ago. It took exactly two clicks to create a direct link I could send to anyone, and it just worked.
- pkamb 6y ago“Send with Transfer...” pollutes the context menu of every folder on my Mac, not just in the Dropbox folder. Is there a way to remove that menu item? https://apple.stackexchange.com/questions/389415/remove-dropbox-send-with-transfer-item-from-finder-context-menu https://apple.stackexchange.com/questions/389415/remove-drop...
- butz 6y agoSame issue on Windows. If you are adding new context items automatically, please have decency to add option to remove them, without hacking the registry. Also, please do not add it back in later update, which happen quite often.
- pmccarren 6y agoThis is awesome! Firefox Send[0] is another very similar product in the space. Excited for more file transfer players! - [0] https://send.firefox.com https://send.firefox.com
- newscracker 6y agoAnd Firefox Send has much higher limits too, for free. Without a Firefox Account, the user can upload up to 1GB. And with a Firefox Account, the limit is 2.5GB. Dropbox Transfer, in comparison, provides (just) 100MB in the free tier.
- knjoy 6y agoYea but it doesn't seem like Firefox Send lets you right click on any file anywhere on my computer and just right click to Transfer. The uploading with Dropbox Transfer (through the desktop client) is super reliable too -- I have Time Warner internet and when it invariably craps out, my upload auto-resumes. Looks like this here: https://www.dropbox.com/t/jJNe5IBPrezaw7aE https://www.dropbox.com/t/jJNe5IBPrezaw7aE
- knjoy 6y agoAs an existing Dropbox user, Firefox Send or WeTransfer is not as good a product for me. Because I already have the Dropbox client installed for file storage, I don't need to visit any websites to upload to Transfer. I can literally just right click on any file anywhere on my laptop and click "Transfer" -- then it automatically pops up, I can add more files into it, rename it, etc. and then just send an email or copy link. I screenshot how it looks here https://www.dropbox.com/t/jJNe5IBPrezaw7aE https://www.dropbox.com/t/jJNe5IBPrezaw7aE (and it's so easy I find myself just automatically use it). What I really like about it is that it sends a copy of the file and my grandma won't just delete my photos like the good ol' Dropbox links.
- kyawzazaw 6y agoIs visiting send.firefox.com that much of a friction?
- tananaev 6y agoPlease make Dropbox Transfer email notifications configurable. I use it a lot and I don't want to be spammed with emails.
- nicksundin 6y agoThis is something we started providing recently (based on user feedback we heard, like yours). There's now a "let me know when someone downloads" checkbox when you're making a Transfer (both on Desktop & web)
- toomuchtodo 6y agoCan you also support "Let me know when the download completes successfully?" I assume this is easy if the recipient is adding to their own Dropbox account, or downloading to their client using a Dropbox client, but may be slightly more tricky with a browser download. EDIT: In the event my comment was ambiguous, I meant these notifications should be configurable by the sender. Essentially "delivery confirmation" for bits.
- nicksundin 6y agoWe're definitely looking into ways to make the recipient side smoother. Stay tuned ;)
- tananaev 6y agoI see it now. Thanks for adding this feature. It would also be nice to control other notifications. For example, an email notification for transfer creation doesn't really make sense in 99% of the cases. And I think ideally all those controls should be in the main Dropbox settings (notifications tab). Or at least defaults should be there.
- leetrout 6y agoRelated I’ve been wearing out webwormhole.io to send large files to coworkers. Not a 1:1 here but very useful tool in the same space.
- jrochkind1 6y ago> helped us break away from preconceived notions based on what is easy and incremental to build on top of the Dropbox stack. In my experience, this is one of the biggest challenges for developers involved in product design, expressed succinctly there. And to try to step out of this is why it's so important to have product owners/managers who are NOT technical staff, and who technical staff doesn't unduly influence. When you've invested your time and energy in building a hammer, you really want to figure out how to treat anything as a nail.
- yjftsjthsd-h 6y ago> When you've invested your time and energy in building a hammer, you really want to figure out how to treat anything as a nail. And to be fair, this is reasonable, right up until it isn't. When every tool costs onboarding and maintenance effort, it makes sense to leverage as few tools as possible to the greatest effect - but much as "everything should be made as simple as possible, but not simpler", you want the fewest tools that will do the job well, but sometimes that is N+1, and a new tool will pay for itself.
- jrochkind1 6y agoAbsolutely. But it's really hard to tell when it is and isn't reasonable, when it comes to UX. I find that the only thing you can do is try to have peopel who aren't developers and don't even know what's "easy to build incrementally on top of the existing stack" determining the appropriate UX for the customer/market/business need. Then the developers can say "OK, but if we did it like THIS, it would be a lot cheaper to implement because we can incrementally add to what we've got", and the product manager/owner can push back "Eh, that's not going to meet the market need", "OK, but we can't afford it" -- it's a negotiation. But in my experience you really have to have someone who isn't a developer at all standing up for the user need, for anyone who will be involved in the implementation, no matter how smart and user-centered, it's just too tempting to decide that the thing that can be met with the incremental change to existing stack is "nearly as good" no matter what, when it's so much easier and more elegant under the hood.
- yahyaheee 6y agoNot sure why it still stakes hours to days to upload a video from my iPhone.
- ykevinator 6y agoFirefox send (send.firefox.com) is an excellent unknown product and I highly recommend it
- gaogao 6y agoThis was a great writeup! I loved start to finish view, that didn't get bogged down in technical details.
- faitswulff 6y agoDoes anyone know if there's a CLI tool to transfer files? Ideally I'd want to give it a file as an input and then have it return a shareable URL that transfers the file directly from my computer to whoever clicks the URL.
- dguido 6y agoyep, https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole
- faitswulff 6y agoI did look at magic-wormhole, but it's difficult for non-tech people to use since, IIUC, you need to use the command line.
- deleted 6y ago[deleted]
- ColanR 6y agoSo you do or you don't want a CLI tool?
- faitswulff 6y agoI mean, I said right in my first comment that it should output a shareable URL.
- pornel 6y agoFirefox Send has a CLI: https://github.com/timvisee/ffsend https://github.com/timvisee/ffsend
- timvisee 6y agoThanks for sharing, dev here. It's an unofficial tool, but fully featured for sure. Supports link shortening or QR code generation as well.
- dguido 6y agoFirefox Send, SendSafely, and Magic Wormhole are all end-to-end encrypted. https://send.firefox.com/ https://send.firefox.com/ https://www.sendsafely.com/ https://www.sendsafely.com/ https://github.com/warner/magic-wormhole https://github.com/warner/magic-wormhole https://webwormhole.io/ https://webwormhole.io/ (magic wormhole over WebRTC!) This seems like table-stakes for a modern file transfer system. Accepting unencrypted files and storing them temporarily in the clear on your own servers seems like it only introduces tons of additional risks without much gain.
- zaroth 6y agoIf I might ask a lazy question - do any of these work with basic (e.g easily available on a lightweight container) command line tools on the terminal? I assume not since they must all be Javascript-based.
- pezdeath 6y agoFound this: https://github.com/timvisee/ffsend https://github.com/timvisee/ffsend
- timvisee 6y agoThanks for linking, dev here. Yes, perfectly usable for automation!
- prakis 6y agoffsend excellent tool. Ffsend is inspiration for me to make 'ele' https://github.com/prakis/helloele https://github.com/prakis/helloele Ele is a command line tool works with your own cloud. Main difference from ffsend and other cmd tools is no need to share links, files are referred by file-indexes.
- timvisee 6y agoSounds cool! Will check it out later today.
- errantmind 6y agoI uninstalled Dropbox and moved my files elsewhere because there was NO WAY to disable the annoying notifications asking me to 'upgrade' my account (when my account was almost full). My whole family used Dropbox with paid accounts and I'm actively moving them elsewhere because of this.
- inopinatus 6y agoThe client is now full of all kinds of irritating nagware to enable this and that unnecessary feature, even though I’m a paying customer already. Doing so whilst ignoring the design language of the OS is a hallmark of applications arrogantly deluded of their own importance. The Dropbox client UX now puts me in mind of 90s-era RealPlayer, which surely ranks as one of the worst software products of all time. I do not appreciate intrusive micromanagement and passive-aggressive interdepartmental memos from humans, let alone from my file sync utility, and when this occurs separately on every god-damn device and even after a time machine recovery, I find myself verbally abusing it, and the product managers that spawned it, aloud in the most vulgar language. Not good, Dropbox.
- new2628 6y agoThe web interface also went downhill from its initial simplicity. Hitting the download button for a pdf file in my own dropbox folder (99.99% of my use case) has become a whack-a-mole, with a constant danger of accidentally connecting some app with the document, entering some kind of crazy collaborative editing/commenting mode, updating my "profile", installing a dropbox app, getting notified of something, or activating some kind of sharing feature, none of which I ever used or wanted.
- dbxerthrowaway 6y agoCan you elaborate on what you mean by nagware? Are they OS notifications, "ads" in the tray or the Dropbox app?
- inopinatus 6y agoI'm mostly a macOS user, so by this I'm referring to the litany of prompts I have to dismiss in the menu bar drop-down. I just opened it up on this machine and behold, there's a new one, bizarrely hoping I might connect my calendar (no thanks). Fun fact: I mostly option-click the Dropbox icon to force it back to system style. The UX inconsistency and unwelcome intrusions otherwise just interfere with whatever I'm actually trying to do (usually, pause syncing).
- deleted 6y ago[deleted]
- Eric_WVGG 6y agowow, that was a very exhaustively informative explanation of how to clone WeTransfer
- dilandau 6y agoIn which someone shits out 5000 words about statically serving files over http... I mean ffs. Is there even a hint of self awareness?
- deleted 6y ago[deleted]