jamwt

10 ms·

91.
by jamwt 11y ago
The difference to get even higher is our system supports host-managed SMR disks that come in 10T and 14T sizes.
92.
by jamwt 11y ago
Not yet, but we will very soon. We'll be using Rust as a library with a C ABI, and calling it via CGO.
93.
by jamwt 11y ago
> Dropbox rewrote Magic Pocket in Golang, and then rewrote it again in Rust, to fit on their custom built machines. Actually, full disclosure, we really just rewrote a couple of components in Rust. Most of Magic Pocket (the distributed
94.
by jamwt 11y ago
Good questions, let me try to tackle them one by one. > The article makes a brief mention of Go causing issues with RAM usage. Was this due to large heap usage, or was it a problem of GC pressure/throughput/latency? If the form
95.
by jamwt 11y ago
Both companies control the technology that most impacts their business. Dropbox stores quite a bit more data than Netflix. Data storage is our business. Ergo, we control storage hardware. On the other hand, Netflix pushes quite a bit more
96.
by jamwt 11y ago
> Oh dear I didn't realise Dropbox had invested all of that time and money moving into their own data centre. We've had our own datacenters for a very long time; this project just represents moving "blob storage" into
97.
by jamwt 11y ago
Hi HN! A couple of us from the Magic Pocket software team are around to answer questions if anyone has some.
98.
by jamwt 11y ago
Pretty sure there would be a fork. Many large tech companies (Google, Dropbox for two) are deeply invested in Python 2, and some of us already have some modifications to CPython. Python 2 is Too Big to Fail.
99.
by jamwt 11y ago
You would definitely split that project up into multiple crates at the moment. 100KLOC would take an eternity to compile. My guess would be 15 to 45 minutes for a release build, depending on things like how much you were using generics.
100.
by jamwt 11y ago
A better example is the timeless: err := f1() if err != nil {..} // have to do this or compiler error err = f2() // oops, forgot to actually do anything about it! Now, granted, `go vet` helps with t
101.
by jamwt 11y ago
> Ah good, now we're talking specifics. Well, specifics within a hypothetical. Hypotheticals are always game to to discuss. :-) > Does REPORT ZONES not point to this conventional section always? Yep, REPORT ZONES includes the c
102.
by jamwt 11y ago
Wow, very cool! So, candidate for offloading that's really commonly considered is "scrubbing". Essentially, protecting against passive bit flips by checking your data against your own checksums. If you have some N TB of data
103.
by jamwt 11y ago
Nope, that was just for this example; custom enclosures and larger disks can push quite a bit higher than this.
104.
by jamwt 11y ago
Host Managed (or Host Aware) SMR disks are fine. You need to do your own zone management using ZAC/ZBC, and throw out your filesystem. The disks all have a conventional section near LBA 0 for metadata management (e.g. indices). Usual
105.
by jamwt 11y ago
I can't edit the parent anymore, but it has been fairly pointed out that my tone of piggybacking on Backblaze's great blog post is a shameful bit of thunder-stealing. The technical community owes a great debt to them for their tr
106.
by jamwt 11y ago
Okay, fair point. Given empirical evidence wrt my blogging discipline (and the fact that I probably can't), it's unlikely that blog post will ever happen. I'll leave this thread alone because there's some useful technic
107.
by jamwt 11y ago
Hmm, maybe I was being tone deaf? I've repeatedly tried to state in subsequent children in this thread that it seems as though Backblaze made the right set of tradeoffs given their previous gen, and their time to recoup their investmen
108.
by jamwt 11y ago
> You merely state that it's just not state of the art, and to come work at dropbox. That's about all I can do. However... > This is what I mean by being disingenuous. I'm don't think disingenuous is what I'm
109.
by jamwt 11y ago
You can do a lot more than 60, and you can get a lot bigger than 6T. > It's disingenuous to call this not "state of the art" when it's really quite close, and obviously is meeting a backblaze design goal. So, I never
110.
by jamwt 11y ago
Hi Gleb! > Where are you buying 40 TB hard drives? I'd love to get some ;-) Me too. :-) Not 40TB, but larger than 4T and quite a few more of them. > But I believe these are one of, if not the, lowest cost storage servers per GB
111.
by jamwt 11y ago
Worth pointing out (for those of you not in the storage game) that these appliances, while cool, are pretty far from state of the art. We're (Dropbox) packing up to an order of magnitude more storage into 4U. And costs aren't cl
112.
by jamwt 11y ago
Hi! Dropbox two account works great. It's specifically designed to keep your personal Dropbox completely separate (and unscrewup-able) from your company assets. We built it because people had been intermingling personal Dropboxes wit
113.
by jamwt 11y ago
Usual disclaimer, I am not an official Dropbox spokesperson, and these are not necessarily company stances. 1. The best answer I can give you is here: https://www.dropbox.com/en/help/28 The background behind it is
114.
by jamwt 11y ago
I'm pretty excited about this one. We've been using Paper internally at Dropbox (under a few different codenames) for more than a year, and we now run almost all our collaboration through it--design docs, specs, etc. I'm on
115.
by jamwt 11y ago
I think that line of thinking ends up not applying to rust terribly much. I know there's been several articles lately about broader applications of rust, and then "is it worth it?" type responses, concerns about "manual
116.
by jamwt 11y ago
Dropbox has something futures-based ala Finagle that's built on mio. We'll be open sourcing it sometime soon-ish.
117.
by jamwt 11y ago
Although, servers have billions of users. ~Everyone on the Internet is a user of Linux servers.
118.
by jamwt 11y ago
Not true, Riak was completely, 100% correct with siblings enabled, which is Totally Expected behavior for a standard Dynamo-style system. The criticism is that last-write-wins is the default, and that's a fair criticism--that's no
119.
by jamwt 11y ago
We use conduit at Dropbox as well, but mostly for PEP 333 type reasons ("universal" request/response types and a pluggable coupling layer between handlers and servers.)
120.
by jamwt 11y ago
Dropbox's block storage system is written in go, and we're pushing terabit+ bandwidth around in it; I'd say we're in the running. :-)

More ›