arjunnarayan

9 ms·

31.
by arjunnarayan 6y ago
The argument is that without stronger consistency guarantees you can't do joins between two streams (or even something like argmax over a single stream, since it splits the stream into two subcomputations, which then have to be joined
32.
Eventual Consistency isn’t for Streaming(materialize.io)
165 points by arjunnarayan 6y ago|49 comments
33.
by arjunnarayan 6y ago
Jay Kreps is an extremely well read man; the reference is surely ironic. Enterprise middleware, after all, is and always has been a bureaucratic rigmarole. The nightmare has existed since the first ETL job was written, but by democratizing
34.
by arjunnarayan 6y ago
I think a 5-second summary of the difference is that Materialize is Oracle FAST REFRESH for almost* all of SQL, as opposed to just those very limited cases where Oracle's FAST REFRESH works. Oracle's docs have entire features in o
35.
Data Denormalization is Broken (2016)(hackernoon.com)
1 points by arjunnarayan 6y ago|0 comments
36.
by arjunnarayan 6y ago
Thanks for the vote of confidence! One thing: We're not marketing it in the OLAP space. Our existing users very much are building new applications. Initially we went for the metaphor of "what if you could keep complex SQL queries
37.
by arjunnarayan 6y ago
This is not the original Differential Privacy paper, this is an invited talk (I agree with you that it remains a great read). However, the original paper is "Calibrating Noise to Sensitivity in Private Data Analysis"[1] with four
38.
by arjunnarayan 6y ago
The trigger was the "let's look at this" phrase, not the "Join zoom" phrase. This is similar to when you say "where are you": it prompts the other person to share location (also on Android). This article i
39.
What consistency guarantees should you expect from your streaming data platform?(materialize.io)
6 points by arjunnarayan 7y ago|0 comments
40.
by arjunnarayan 7y ago
Agreed! We plan to contribute to the Debezium project over the next months along these lines. We’ll have more to say on this shortly!
41.
by arjunnarayan 7y ago
1. Comparing numbers from one system (Cockroach) that adheres to strict durability requirements to another that does not (MemSQL) is apples to oranges, especially, as you point out, you see a 2x performance hit when you impose that requirem
42.
by arjunnarayan 7y ago
This benchmark is pretty ridiculous for the following reasons: 1. Their database is run in asynchronous durability mode. 2. They specifically do the one thing that TPC-C says you shouldn't do, which is get really high throughput on a s
43.
by arjunnarayan 8y ago
Excellent question. There's a CockroachDB blog post about that: https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mappin...
44.
by arjunnarayan 8y ago
Unfortunately most of the systems that build what you're describing are closed source (e.g. Snowflake, Microsoft SQL Server, Vertica, Teradata). There isn't an open-source project that does all of those things.
45.
by arjunnarayan 8y ago
Apache Arrow is your best bet, but it's still very much a new project without a lot of the things you're looking for.
46.
by arjunnarayan 8y ago
(author of the blog post here) I'd second ryanworl's comment that the rabbit hole goes much deeper than just storing things in a column oriented disk or in-memory format like Parquet or Arrow. That's just the first step. To g
47.
A Brief History of Log Structured Merge Trees(ristret.com)
3 points by arjunnarayan 8y ago|0 comments
48.
Reporter might have to rollback Aurora story(ristret.com)
1 points by arjunnarayan 8y ago|0 comments
49.
by arjunnarayan 8y ago
Maybe some Republicans should troll Bernie back by proposing THE A rrest T he T radition of A crostics I n the N ames of D efective E rroneous R egulations ACT
50.
by arjunnarayan 8y ago
> Postgres is a notable exception Did you have good experiences with postgres' serializable mode? When I tried to do some TPC-C benchmarking with postgres set explicitly in serializable mode, it would fall over almost instantly (rea
51.
by arjunnarayan 8y ago
TPC-C's intent is to make a tradeoff between two competing goals: benchmarking total data stored and processing transactions on the stored data. It's one point in the spectrum of possibilities, but one that has withstood scrutiny
52.
by arjunnarayan 8y ago
Horizontally scaling TPC-C with sharding is nontrivial, as the TPC-C specification requires cross-shard transactions. This is the intent of the TPC-C benchmark: real world use cases seldom shard perfectly. CockroachDB scales to 30 nodes in
53.
by arjunnarayan 8y ago
Author here. The point of certification is to ensure transparency. CockroachDB is open-source. Our entire benchmarking - tools, scripts, are also open source, and the results are from runs on public cloud providers. You can replay our entir
54.
by arjunnarayan 9y ago
A short note that the total cost of that SPARC cluster was $30 million. You're not misreading those numbers, but it requires a little context. We're focusing today on our improvements over CockroachDB 1.1, using a small-ish cluste
55.
by arjunnarayan 9y ago
I don't think it's true to claim that TPC-C is obsolete and subsumed by TPC-E. They are both different OLTP benchmarks, with different characteristics. TPC-C is more write heavy, TPC-E is far more read heavy. It's true that T
56.
by arjunnarayan 9y ago
Short answer: 3 n1-highcpu-16 GCE VMs with Local SSDs attached. We're working on a complete disclosure document, with comprehensive reproduction steps to replicate all our numbers. This document should be out in a couple of weeks. We w
57.
by arjunnarayan 9y ago
CockroachDB is not (yet) ready for use on OLAP-style workloads. Our performance work has focused on OLTP workloads so far. That said, we do great on OLTP joins (which is a stressed in the TPC-C workload).
58.
A History of Database Transaction Histories(ristret.com)
9 points by arjunnarayan 9y ago|0 comments
59.
by arjunnarayan 9y ago
Start by talking to this person's manager. You have prior experience in this situation, which means you would be an effective mentor for getting this person out of this situation. However, such a relationship should begin with, be appr
60.
by arjunnarayan 9y ago
Reposting my comment from when this came up on our 1.0 launch: IIRC, in the case that aphyr refers to for these specific numbers, the reads are scans that span multiple shards[1], while the writes are writes to single shards. [1] even thoug

More ›