ryanworl

20 ms·

151.
by ryanworl 8y ago
FoundationDB does not use buffered IO, so no.
152.
by ryanworl 8y ago
Transparent compression would also be a useful feature here. On a lot of workloads it would result in the replication cost being more like 1.5x than the 3x you’d typically deploy FDB with, especially if you ran zstd training in the backgrou
153.
by ryanworl 8y ago
I may be misunderstanding the documentation, but it appears MariaDB system versioned tables don't do any garbage collection at all (it stores all versions forever), and requires reading over (skipping) old versions at runtime.
154.
by ryanworl 8y ago
The implementation of these kinds of features is a lot easier than one might think if you use multi-versioned storage. You delay garbage collection of old versions of rows until the configured cut off (or never if you want to go to any time
155.
by ryanworl 8y ago
The ones I'm referring to are in Manhattan and Philadelphia.
156.
by ryanworl 8y ago
Trader Joe's uses a single line (in every one I've ever been to) and it is wonderful. Even a line wrapping the entire store will only take 10-15 minutes because they staff appropriately and you're almost always moving. You al
157.
by ryanworl 8y ago
Microsoft and Google do not collectively hire that many people and are not who I am talking about. “Brain-teaser” type questions are explicitly not work-sample tests and, yes, have no correlation with job performance.
158.
by ryanworl 8y ago
BigQuery is probably the best price/effort ratio for this.
159.
by ryanworl 8y ago
You’re getting too philosophical here for something that isn’t that complicated. This isn’t related to power dynamics. Asking someone if they can flip burgers is obviously not as effective as asking them to flip a burger and seeing if they
160.
by ryanworl 8y ago
I don’t have an opinion on whether or not it is done synchronously or asynchronously and I hope I didn’t imply work-sample = take home test.
161.
by ryanworl 8y ago
It does always amaze me that literally what you learn in HR 101 in an undergraduate management curriculum is controversial. The r^2 of a ton of different methods for predicting job performance is something large companies are highly incenti
162.
by ryanworl 8y ago
This technique is just more IO parallelism at the physical layer due to higher concurrency while submitting IO, correct? Since NVMe and new SSDs don't hit peak throughput until very high queue depths this doesn't surprise me.
163.
by ryanworl 8y ago
I may be misinterpreting what you mean by a 1-1 relationship, but the documentation for Cloud Spanner, Cockroach, and the old FoundationDB SQL layer all use a similar schema to the one I described in their examples. Additionally, the F1 pap
164.
by ryanworl 8y ago
https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mappin... this was in another comment. https://emsal.me/blog/5 this blog post has a good introduction to their implementation of interleaved t
165.
by ryanworl 8y ago
Hard disagree. This is the only way in a distributed, sorted KV store to get any semblance of data locality. If Cockroach and Spanner didn't do this, they would constantly be doing 2PC for modifying data that is related but stored on d
166.
by ryanworl 8y ago
The CockroachDB blog post on this topic is a good summary. There is an additional trick that isn't directly KV related, but is important in a distributed environment when using a KV storage engine. When defining a hierarchy of tables,
167.
by ryanworl 8y ago
No, I am not aware of any storage engine that provides that out of the box. The techniques are very tied into what your query processing engine can do and expects the data to look like. For example, do you materialize tuples immediately, or
168.
by ryanworl 8y ago
For analytics workloads, your best bet is using compression techniques that let you do operations on the data without decompressing it. A good example is dictionary encoding a set of sorted string keys so you can preform prefix queries by d
169.
by ryanworl 8y ago
It would be a fine choice, (and the document layer is) but the most performance sensitive parts of the entire system when you take it as a whole are already written in Flow. Writing it in C++ entirely would take a lot longer.
170.
by ryanworl 8y ago
FoundationDB was not open source before it was acquired by Apple. They had a free version you could download and use, though.
171.
by ryanworl 8y ago
https://github.com/apple/foundationdb/tree/master/flow
172.
by ryanworl 8y ago
FDB itself is written in Flow, which is 98% C++ with a few extra keywords thrown on top that gets compiled to actual C++. This gives them actor model concurrency in C++ that is completely deterministic by generating very ugly code that uses
173.
by ryanworl 8y ago
FDB does not automatically index your data, but you can write a layer (like this one) to index your data. In a transaction, you write a key like “users/1” with a value of “bob” and then write another key like “users/bob/1” wi
174.
by ryanworl 8y ago
The best way I can describe FoundationDB is it is like a file system. You can do just about whatever you’d like with files and a file system, in theory. You can implement just about any data model you can dream up in FDB. But the current st
175.
by ryanworl 8y ago
If you’re referring to my comment, by “not possible”, I meant “subject to the 5s transaction duration limit.” etcd and ZK could be implemented on top of FDB.
176.
by ryanworl 8y ago
FoundationDB is about as honest and up-front about its limitations and flaws as any system I've ever seen. Have you had a negative experience with it you can share?
177.
by ryanworl 8y ago
Are you using FDB at Olark? (Saw it in your profile)
178.
by ryanworl 8y ago
* Schema evolution and schema management. Keeping schema state updated transactionally for a single, high-traffic table is complicated. An FDB core feature I requested could make this easier. Basically managing the lifecycle of adding, buil
179.
by ryanworl 8y ago
It would be tough to implement every SQL construct on top of what is there today. I could explain why (I have tried and presumably ran into the same issues they did described in various areas in the docs and the paper), but the docs give an
180.
by ryanworl 8y ago
Congrats to the team at Apple for getting this released! They have had a busy few months with getting the document layer released, the FDB Summit, and now the record layer.

More ›