james_cowling

8 ms·

31.
by james_cowling 10y ago
Just a caveat I'll add is that you have to work really very hard to build a system with equivalent/better durability and availability guarantees as the big cloud storage providers. It's typically not possible to be significan
32.
by james_cowling 10y ago
Yep looks like we may have had slightly different goals here. Another one of our design requirements was that we wanted to be able to move data around within the system (repair operations, rebalancing, encoding, compaction etc) without deal
33.
by james_cowling 10y ago
It's been a very long time since I read the GFS paper but it's similar in many ways - there are only so many ways to design a large-scale blob store and they all share a lot of the same design elements. If I remember correctly GFS
34.
by james_cowling 10y ago
Technically the different copies of the buckets aren't "identical" since they may get populated in unserialized ways (they may store data in different orders) but yes they all store the same set of blocks. Note that I just de
35.
by james_cowling 10y ago
Yup, as @jamwt says we directly manage the block device. That said, up until somewhat recently our data really was written into 1GB extents stored as files on an XFS filesystem. This mostly worked fine but we switched away from this so that
36.
by james_cowling 10y ago
Hey folks, this is James from Dropbox. As with the last post ( https://news.ycombinator.com/item?id=11282948 ) a few of us will check in from time to time to see if there are any questions. Apologies for taking so long to get
37.
by james_cowling 11y ago
Lazy answer but we'll blog about this in the next month. On a high level it's variable-sized blocks packed into 1GB extents, which are then aggregated into volumes and erasure coded across a set of disks on different machines&#x2F
38.
by james_cowling 11y ago
No filesystem - the rust codebase directly handles the disk layout and scheduling. Our previous version of the storage layer ran on top of XFS which was probably a bad choice since we ran into a few XFS bugs along the way, but nothing serio
39.
by james_cowling 11y ago
Haha, who am I to argue with a story like that?
40.
by james_cowling 11y ago
These are standard-width rackmount chassis btw. They're 1.5x depth but otherwise it's standard 4U. The dimensions given are just an approximation.
41.
by james_cowling 11y ago
Btw. that was a typo, I meant over "half" a terabit of data transfer. You get into bigger numbers than this when you start talking about aggregate traffic.
42.
by james_cowling 11y ago
We use a variant on Reed-Solomon coding that's optimized for lower reconstruction cost. This is similar to Local Reconstruction Codes but a design/implementation of our own. The data placement isn't RAID. We encode aggregated
43.
by james_cowling 11y ago
Couldn't agree with this more. One of the things that's not immediately apparent in an initiative like this is the huge amount of effort required to provision, deploy and manage a system of this scale, across a whole bunch of diff
44.
by james_cowling 11y ago
We have a big fleet of what we call "backfill nodes" which handle the transfer but the most difficult component of the transfer is just the network capacity. We worked pretty closely with Amazon here since we have a long-standing
45.
by james_cowling 11y ago
I probably wasn't sufficiently clear in my other message re Golang: - Python is our primary development language for most stuff at Dropbox. - Mobile development and UI code use whatever is appropriate for each platform. - Go is the pri
46.
by james_cowling 11y ago
We were pushing over a terabit of data transfer at peak, so 4-5PB per day.
47.
by james_cowling 11y ago
I think @jamwt did a pretty good job of explaining this in https://news.ycombinator.com/item?id=11283688 . At a high level it's really a memory thing. There are a lot of great things about Rust but we're also mostl
48.
by james_cowling 11y ago
Yep we're going to get at least a few actual technical blog posts online in the coming month. We haven't got around to writing them yet tho so feel free to surface any requests :) Will most likely start with the following: 1. Over
49.
by james_cowling 11y ago
To be fair to Backblaze this level of storage density is really only possible with recent advances in disk technology (higher densities, SMR storage, etc). Also not everyone wants to be packing a petabyte into a box. At that level of densit
50.
by james_cowling 11y ago
This is most accurate. A couple of comments: > Dropbox still uses AWS for its European customers. We haven't publicly launched EU storage yet but will be doing so later in the year. > Dropbox hired a bunch of engineers from Faceb
51.
by james_cowling 11y ago
Yup, Dropbox Infra is mostly a Go shop. It's our primary development language and we don't plan to switch off any time soon. We're always about the right tool for the job tho, and there are definitely use cases where Rust mak
52.
by james_cowling 11y ago
Very possibly :) Drop me an email at james@dropbox.com.