mjb

13 ms·

91.
by mjb 3y ago
Yes, it's super interesting when systems move into the regime where stale information becomes worse than no information at all. Part of what makes best-of-2 interesting is how long it resists falling into this pit, but it does eventual
92.
by mjb 3y ago
Doing consistent reads this way is a fairly old technique, at least back to the 80s (I can dig up some papers if folks are interested). Spanner, rather famously, uses this range approach, but a good number of other systems are based on simi
93.
by mjb 3y ago
I was reacting to this: > I am not sure if talking about anything else in the context of distributed databases is reasonable. There's a whole world in distributed databases, and I suspect you'd agree that there's a lot of
94.
by mjb 3y ago
That is one way (and a good one) of classifying consistency models and there relationship to isolation levels. But it's an incomplete one (e.g. there are linearizable variants of snapshot and repeatable read that exist that are not cap
95.
by mjb 3y ago
Completely agree. In-region (e.g. at the scale of a US state), CP seems like a clear winner. For more geo-distributed latency becomes challenging, and in environments like IoT and mobile unreliable connectivity becomes challenging. If you&#
96.
by mjb 3y ago
As database builders and users, we’ve made talking about systems a lot harder on ourselves by conflating the ideas of replication, active-active, atomic commitment, and concurrency control. - Replication is a technique used to achieve highe
97.
by mjb 3y ago
Yes! There are a lot of benefits to the 'everything is a file' philosophy, but files aren't very good APIs for a structured data. The perfect example is the stuff in /proc and /sys. Making it all 'just a file&#
98.
by mjb 3y ago
And that Worth worked primarily on systems that are many times smaller (maybe many orders of magnitude smaller) than today's biggest systems. It's no surprise, then, that the successful techniques are different. The right techniqu
99.
by mjb 3y ago
I love this, and have similar feelings about PRISM. I hope recent work to add probabilistic properties to TLA+ and P provide a route towards doing this kind of work with languages that are easier to use. While this example is a bit silly, t
100.
by mjb 3y ago
Right. Even a machine like the Tormach 440, which is pretty close to the minimal steel-cutting machine, contains nearly 200kg of cast iron. That, plus lead screws, motors, etc needed to move it around add up quickly. A while back I looked i
101.
by mjb 3y ago
Completely agree. It's a huge trap, and an easy one to fall into, especially if you're not super careful about which echo chambers you spend time in. My advice is to be thoughtful about your career and understand your company and
102.
by mjb 3y ago
Aurora Limitless Database is based on our own investments in database-optimized virtualization (Caspian), in scale-out log-first database storage (Grover)[2], and in a custom approach to cross-shard transactions that makes use of the high-q
103.
by mjb 3y ago
Yes,distributed systems problems and multi threading problems tend to have very similar solutions. Contention is the critical factor indeed - if there's no coordination needed, then we can keep scaling forever.
104.
by mjb 3y ago
Phased array, heat sensors, no turntable, uses beam forming to get food uniformly hot.
105.
by mjb 3y ago
> Operations always felt like “the things around the Thing” - a supporting cast that is important to making something happen. In the short term, it can feel that way. But in the longer term, at least in the part of the industry I'm
106.
by mjb 3y ago
> The problem with POSIX file APIs are the semantics in the presence of multiple concurrent writers. That sure is a problem with the file APIs, but far from the only one. Other issues include weird asynchrony, weird scatter/gather,
107.
by mjb 3y ago
Well, yeah. But (DAMON and friends aside), Linux doesn't handle that non-zero marginal memory cost well today.
108.
by mjb 3y ago
The other interesting detail here is the memory sizing problem. If I can consume all my RAM with caches and buffers, how much RAM do I need? The answer (as always) depends on what you're optimizing for. For performance, bigger is bette
109.
by mjb 3y ago
There's some really interesting little details here. Linux, by default, is making the very reasonable assumption that the marginal cost of converting empty physical memory into caches and buffers is very near zero. This is fundamentall
110.
by mjb 3y ago
I don't think most people would disagree with that. If you want to build a database, for curiosity, or to solve a particular problem, or just because it seems fun, go ahead. But know that making a real production database tends to be h
111.
by mjb 3y ago
2PC (or atomic commitment more generally) is needed for sharded/partitioned systems with different data on each node. In these systems, each node gets a vote on whether a transaction should be allowed to commit. Replication, making mul
112.
by mjb 3y ago
TLA+ also uses that convention (along with the temporal logic operators [] and <>, and some other unusual conventions like #=).
113.
by mjb 3y ago
You can't do better than At Least Once if you're having side effects outside the database, so it's not clear that SQS's weaker semantics have any practical effect.
114.
by mjb 3y ago
This should be safe under SI (other than the ABA issue, which isn't even fixed with serializable). The update forces a W-W conflict, which is sufficient to make the behavior serializable under SI (and therefore, I think but am not sure
115.
by mjb 3y ago
The system described here (if I understand correctly) could store sets if care is taken to make the set writes commutative. The CRDT literature contains several solutions to this problem.
116.
by mjb 3y ago
CAP's A isn't a very useful thing at all. There's nothing stopping a database from being both Consistent and (common-sense definition) Available on the majority side of a network partition. It just can't be available on
117.
by mjb 3y ago
Roughly, edges can roll over, break off, or wear down. Honing helps with the first, but not the other two. Which one happens depends on geometry, steel, and what you've been cutting.
118.
by mjb 3y ago
I don't think we need to descend to this level of mysticism when discussing knives. If you want a knife that is a carefully crafted beautiful object, hand-made by a craftsman with a long legacy and great skill, then feel free to buy a
119.
by mjb 3y ago
"Eternal September" has big "I can't believe they let those people into the country club" energy. Gatekeeping is bad. It might feel good when you're on the right side of the gate. It might even feel righteous
120.
by mjb 3y ago
I suspect you could make the receivers weigh single digit grams, depending on the band.

More ›