_vvhw

17 ms·

121.
by _vvhw 5y ago
> The most popular siren song in all of modern technology is the obsession with making all the cores share the work. For a vast majority of problems that operate on shared state in a serialized fashion (i.e. anything a bank would do), co
122.
by _vvhw 5y ago
This is an incredible reference that's helped tremendously, time and again, with designing TigerBeetle [1]. Especially slide 80 about the CPU's Line Fill Buffer: the bandwidth to a single core is limited by LFB entries and i
123.
by _vvhw 5y ago
> In hindsight, data logic should be in the database itself. This is the reason we are creating TigerBeetle [1] at Coil, as an open source distributed financial accounting database, with the double entry logic and financial invariants en
124.
Database functions to wrap logic and SQL queries(sive.rs)
4 points by _vvhw 5y ago|1 comments
125.
by _vvhw 5y ago
How do these NVMe SSDs fare when setting the FUA or Force Unit Access bit for write through on Linux (O_DIRECT | O_DSYNC) instead of F_FULLFSYNC on macOS? I imagine that different firmware machinery would be activated for FUA, and knowing w
126.
Wasm4nia(jerwuqu.itch.io)
3 points by _vvhw 5y ago|0 comments
127.
by _vvhw 5y ago
> Also note that 3rd party drives are known to ignore F_FULLFSYNC SQLite, MySQL et al. [1] fall back to `fsync()` if F_FULLFSYNC fails, in order to cover this case of 3rd party or external drives. [1] https://twitter.com/
128.
by _vvhw 5y ago
Fantastic thread. The history is also interesting. It's not that "macOS cheats", but that it sincerely inherited the status quo of many years, then tried to go further by adding F_FULLFSYNC. However, Linux since got better, l
129.
by _vvhw 5y ago
> There can only be one main node at one time, hence no latency/clock errors/multi leader fail-over/PAXOS/voting/stale keys/exotic application specific conflict resolution logic. This is not accurate. You ca
130.
by _vvhw 5y ago
It's interesting how the proper handling of local storage faults is now also recognized to be all the more critical for global replicated systems—that local faults do propagate across distributed systems. For example, when not using O_
131.
by _vvhw 5y ago
It's a huge pleasure, I'm glad you enjoyed it. Diving into those fsync details at the time was a lot of fun! If you have time at some point to dig in more, we also did a Zig SHOWTIME intro on TigerBeetle that you might like ( http
132.
by _vvhw 5y ago
Thanks, see you there!
133.
by _vvhw 5y ago
Thanks! Have you read: Can Applications Recover from fsync Failures? — https://www.usenix.org/system/files/atc20-rebello.pdf The paper shows that for databases requiring durability and redo logging, O_DIRECT is a
134.
by _vvhw 5y ago
Thanks to the author for an excellent post! > I have to caveat this post with the fact that we are ignoring large swathes of failure modes. I work on a database [1] that has a strict storage fault model, and so I thought I'd add a f
135.
by _vvhw 5y ago
> At scale this adds up to millions of dollars. Buffer bleeds like Heartbleed cost the industry hundreds of millions of dollars. Also, to be fair, Dan's post is about checked arithmetic in hot loops, i.e. the data plane, which as I&
136.
by _vvhw 5y ago
> Do you have examples of this actually causing security problems in Rust Do we need them though? I don't believe we need to go through a Heartbleed moment for Rust before we realize that checked arithmetic is just a good idea. We s
137.
by _vvhw 5y ago
> This article has a core point which is good: “in Rust the default is safe, and you have to opt-in to unsafety, but in C++ the default is unsafe, and you have to opt-in to safety”. On the subject of safe defaults, just to correct that R
138.
by _vvhw 5y ago
> Of course there are limits to WUFFS, that's why it isn't a general purpose language. You shouldn't implement these distributed protocols in it for the same reason toothpaste isn't a good engine lubricant, you delibe
139.
by _vvhw 5y ago
Thanks, good catch!
140.
by _vvhw 5y ago
Yes, although as s_gourichon said, if there's any object pool reuse of these buffers then the potential for bleeds comes back. So, for example, you could still have a bleed in Node.js, even if you're safely allocating them as zero
141.
by _vvhw 5y ago
What I find interesting about buffer bleeds like Heartbleed (and also Cloudbleed) as memory exploits in general, is that: 1. they surprise everyone by often being mere underflows into an allocated buffer (for example, any buffer where you f
142.
How to Prevent the Next Heartbleed (2020)(dwheeler.com)
36 points by _vvhw 5y ago|12 comments
143.
by _vvhw 5y ago
> Something like WUFFS is exactly what we should be using for Wrangling Untrusted File Formats as it says in the name, even if you've decided to do that in a distributed system. No, I was saying earlier that there are limits to WUFF
144.
by _vvhw 5y ago
> As for integer overflows, I don’t think they are nearly as big a security concern in a memory safe language with bounds checking. Feel free to correct me though. See HeartBleed, CloudBleed, all buffer underflows resulting in buffer ble
145.
by _vvhw 5y ago
> If dealing with potentially hostile data, Zig certainly isn't more appropriate than Rust in my opinion, try maybe WUFFS. Thanks! Great recommendation on WUFFS! And completely agreed, it's also easy to turn on checked arithmet
146.
by _vvhw 5y ago
I don't know, people like Mitchell Hashimoto and Tobi Lütke are taking Zig seriously for systems programming. Coil are also investing in writing a new distributed financial database for Zig—considering the trajectory of the language an
147.
by _vvhw 5y ago
Well worth doing, I've learned a lot about coding from Loris.
148.
by _vvhw 5y ago
> Checked arithmetic is a much bigger performance hit than most people expect. You're right about the branching cost. I believe there's a better way to solve that than disabling checked arithmetic everywhere. This comes out of
149.
by _vvhw 5y ago
> Modular arithmetic is perfectly well-defined. Yes (and thanks for the link!), I was in fact thinking more of this non-UB case (not signed overflow UB) as an example of where it's clearly defined as wraparound but can be chained in
150.
by _vvhw 5y ago
I think the comparison should be more nuanced and holistic than only memory safety, if it is to be a discussion on security and not theater. For example, to get the conversation started, how do both languages compare in terms of checked int

More ›