rescrv

9 ms·

61.
Cassandra, MongoDB, and HyperDex Performance(hyperdex.org)
18 points by rescrv 14y ago|8 comments
62.
by rescrv 14y ago
When benchmarking HyperDex we found something similar. Cassandra is better at writes than reads by a surprising amount. Given Facebook's elaborate caching architecture (which likely pushed for such an inversion) this makes sense.
63.
by rescrv 14y ago
Checkout HyperDex. We just released a new version, and it is indeed consistent as nodes join and fail.
64.
by rescrv 14y ago
Consistency is a safety property. HyperDex, Hbase and BigTable all provide linearizability, which has a well-defined meaning. Cassandra does not, and most of its descriptions of consistency only refer to the behavior of the system, and not
65.
by rescrv 14y ago
Check out HyperDex, Hbase and BigTable for systems that provide better guarantees than "eventually."
66.
by rescrv 14y ago
I bet you found the thread on their mailing list. I was the one who ran the benchmark and it was one core vs. one core. In the other benchmarks we had multiple machines, but not the Redis benchmark. We clarified after they speculated that a
67.
by rescrv 14y ago
First, HyperDex is well-optimized C++ code. And second, they take issue with the standard benchmark we used to evaluate both systems. While we recognize that Redis may have an impedance mismatch with the benchmark, we were hesitant to creat
68.
by rescrv 14y ago
1. In the current release it's possible to completely shutdown the cluster. You cannot just cut power, but it's relatively painless to do otherwise. Checkout http://hyperdex.org/doc/06.faults/#shutting-down-and-restori... for more deta
69.
by rescrv 14y ago
You don't need all machines in the same data center. Latency between machines does have effects on performance, so it is faster in a single data center.
70.
by rescrv 14y ago
We love Python and it makes for easy-to-read code snippets! Internally we do make use of the fact that LevelDB sorts objects by key.
71.
by rescrv 14y ago
Thanks for your time. I'm the lead dev on HyperDex, so I'd be happy to address your points. - A SEARCH is simply a mapping from secondary attributes to objects and it was designed for your use case. - We're still adding server-side atomic
72.
by rescrv 14y ago
EC2 is just one platform. Although many people point to EC2 as the gold standard, it's worth noting that Google and Facebook both avoid virtualized servers. On the other hand, EC2 is about leveraging spare capacity and intentionally overs
73.
HyperDex, a consistent, fault-tolerant key-value store, hits version 1.0.rc1(groups.google.com)
55 points by rescrv 14y ago|20 comments
74.
by rescrv 14y ago
TrueTime measures a point in time plus/minus an error interval. The API allows a program to query for when a time/error pair has definitely passed. Each transaction acquires locks using 2-phase locking. At commit time, the time/error are re
75.
by rescrv 14y ago
Thanks for the positive feedback. We've been working on improving the documentation (and soon, the website). Your feedback certainly helps.
76.
by rescrv 14y ago
We have complete Java bindings ( https://github.com/rescrv/HyperDex/commit/1c4e98d88517a63bf0... ). Does this help for Clojure? Our base bindings are in C. Any language which can wrap C can make bindings.
77.
by rescrv 14y ago
Most of the decent providers will help you improve failure independence. I know first hand that Linode is very accommodating for this. I mention independent failures because f=3 is sufficient for this scenario for most. If there is correl
78.
by rescrv 14y ago
HyperDex performs best in a single-datacenter environment. You are right that synchronous replication between datacenters can (and likely will) be slow. There's much work on this (my favorites are Walter and COPS from SOSP 11) that is on-g
79.
by rescrv 14y ago
f is the number of failures that the system can tolerate. Typical systems can tolerate f failures with either (f+1) or (2f + 1) replicas. HyperDex can tolerate f failures with f+1 replicas. This failure threshold is per region of the hype
80.
by rescrv 14y ago
We have a demo of an earlier version deployed on http://gibbr.org/ . We currently are not releasing the source, but we may make an example application using the same design we use in the real app.
81.
by rescrv 14y ago
Just a further note: Although HyperDex is multithreaded by default, when benchmarking it against Redis, we disabled all but one thread from serving network requests. Edit: reword for clarity.
82.
by rescrv 14y ago
We have worked with LinkedIn to use HyperDex for some of their analytics work.
83.
by rescrv 14y ago
We have a full 14 pages ( http://hyperdex.org/papers/hyperdex.pdf ) describing the tradeoffs that went into the design and where the system's strengths and weaknesses lie. Our statements that it is faster derive directly from our observatio
84.
by rescrv 14y ago
CAP, as stated, is a tautology. If you want to understand this better, please read our description of the CAP theorem ( http://hyperdex.org/extras/ ). As formulated by Gilbert and Lynch, the CAP theorem really says, "If you must always re
85.
by rescrv 14y ago
Memcache is a caching solution that stores binary blobs. HyperDex stores data persistently and offers a wide variety of types such as lists, sets, and maps. Redis offers many datastructures as well, but it has a limited architecture. If y
86.
by rescrv 14y ago
The code is available on GitHub. If the benchmark is flawed, help us fix it. Here are some ways it could be flawed: * MongoDB and Cassandra were set to default consistency levels. They are not configured to wait for the majority of replic
87.
HyperDex v0.4.0 Released with More Features and Better Performance(groups.google.com)
3 points by rescrv 14y ago|0 comments
88.
HyperDex adds bindings for Node.js(github.com)
3 points by rescrv 14y ago|0 comments
89.
by rescrv 14y ago
We're actively working on a project to make it have a more-flexible schema. It's definitely a useful thing to have.
90.
HyperDex adds rich datastructures similar to Redis(hyperdex.org)
15 points by rescrv 14y ago|5 comments

More ›