rescrv

13 ms·

91.
by rescrv 15y ago
To tolerate f failures, we need to have f+1 replicas of the data. This implies increased storage, but the f parameter is entirely up to the application. Record insertion affects all replicas of the data, so latency increases with replicati
92.
by rescrv 15y ago
The homepage ( http://hyperdex.org/ ) has more information, although the best source for technical information would be the technical report we've put together ( http://hyperdex.org/papers/hyperdex.pdf ) Edit: Link to paper and not out-dat
93.
by rescrv 15y ago
Thanks for pointing out the paper. If the paper doesn't cover a question, I'm happy to answer (primary dev/paper author here) it via email, or here on HN.
94.
by rescrv 15y ago
HyperDex provides strongly consistent GET/PUT operations and fast searches over data. It uses a new object placement technique to facilitate these searches, and as a result, a search may contact a very small subset of hosts in the system.
95.
by rescrv 15y ago
It's the former. I'd rather not sacrifice our strong consistency by using replicas to serve GET requests.
96.
by rescrv 15y ago
The chains heal in a similar fashion to chain replication ( http://www.cs.cornell.edu/home/rvr/papers/osdi04.pdf ). There will be no dangling insertions.
97.
by rescrv 15y ago
Just a simple typo. I've fixed it. Thanks for pointing this out.
98.
by rescrv 15y ago
Reads and writes for a given key always go to a single node. As you add more machines with the same workload, you are handling fewer keys per machine. The flip side is, if the number of keys per machine stays constant, more machines direc
99.
by rescrv 15y ago
I'm not sure exactly what you're looking for, but Berkeley DB is embedded in your application on a single host, while HyperDex is distributed. It sounds like that's exactly what you want. What do you use exclusive locks for?
100.
by rescrv 15y ago
We use many lock-free datastructures which rely heavily upon the x86_64 architecture. Further, the expanded virtual address space enables us to mmap everything. All network traffic is packed, and in network byte order.
101.
by rescrv 15y ago
In this case,the node should report its own failure. It must be able to do so, otherwise it will be deemed faulty by the entity it reports to.
102.
by rescrv 15y ago
Both systems were single process per host. I agree that Redis does not have good support for "E", so anyone looking at the results should look at A-D,F. Both systems performed entirely out of main memory to avoid touching disk. The Redis be
103.
by rescrv 15y ago
HyperDex utilizes value-dependent chains for replication. Updates move forward in the chains, while acknowledgements flow in reverse. To issue a PUT or a GET, the client contacts the head of the chain responsible for the object it is modif
104.
by rescrv 15y ago
The packages are built for Ubuntu 11.10. What version of Ubuntu are you running?
105.
by rescrv 15y ago
We compared it to Redis as it is one of the many key-value/document-database systems that provide the high throughput and low latency properties that make it comparable to HyperDex. We are expanding the systems we benchmark against, aiming
106.
by rescrv 15y ago
The only scenario in which the operation has "at most once semantics" is when the node the client is directly communicating with fails. No other failure is visible to the clients. Furthermore, every failure scenario provides the following
107.
by rescrv 15y ago
Yes! We've released HyperDex under the 3-clause BSD license. You can get binaries and source from the downloads page ( http://hyperdex.org/download/ ).
108.
by rescrv 15y ago
What you said was right on. I just wanted to add a few things. The coordinator is only involved for recovering from failures, so the cluster can still serve requests until server (non-coordinator) nodes start failing too. I would also add
109.
by rescrv 15y ago
HyperDex uses value-dependent chaining, which offers fault tolerance properties similar to those provided by chain replication ( http://www.cs.cornell.edu/home/rvr/papers/osdi04.pdf ). A single node failure will be recovered from quickly wi
110.
by rescrv 15y ago
HyperDex is different. ZigZag focuses on data visualization. HyperDex provides you the same key-value interface of other systems, while also providing an efficient search primitive. Check out http://hyperdex.org/tutorial/ for examples of
111.
by rescrv 15y ago
I've uploaded a simple tutorial ( http://hyperdex.org/tutorial/ ) on how to get started. The tutorial covers everything from installation through to fully functioning sample code. If you have a minute to check it out, I'd appreciate any fe