snewman

43 ms·

421.
by snewman 14y ago
We're double-checking, but it looks like all of the EBS volumes were 100GB. It would be interesting to see the impact of using 1 TB volumes.
422.
by snewman 14y ago
Thanks. I did put in quite a bit of work in finding the best ways to present the data, which depending on how you count is around five- or six-dimensional. I'll see whether we can work larger RAID configurations into a follow-up.
423.
by snewman 14y ago
Yes, the differences between graphs were fairly substantial. You can get a sense of the EBS vs. ephemeral difference by looking at the "Throughput by Threadcount" graphs (about 1/3 of the way through the post). Briefly, EBS has much higher
424.
by snewman 14y ago
Thanks! $/iops between storage types is the subject of the "Cost Effectiveness" charts near the top of the post, but I suspect I'm not quite catching your meaning. What do you mean by $/iops/storage?
425.
by snewman 14y ago
Yes -- unfortunately, Amazon announced provisioned IOPs approximately the day after we finished the benchmarking work. It took a while to find time to write up the results. We'll very likely do a followup post to test provisioned IOPs. If a
426.
by snewman 14y ago
OP here. We'd like this work to be a useful resource - everyone benefits when there's more / better information about how these complex cloud systems perform in real life. So please comment with suggestions, questions, or any other feedback
427.
EC2 I/O(blog.scalyr.com)
220 points by snewman 14y ago|26 comments
428.
by snewman 14y ago
It's important to understand what's behind the protests - the movie is basically just a pretext: http://www.nytimes.com/2012/09/16/opinion/sunday/douthat-its...
429.
by snewman 14y ago
Car sickness tends to happen when you're focusing on the interior of the car. That's what leads to the disconnect between your visual field and your mechanical sense of motion. I expect this is the main reason that drivers are rarely affect
430.
by snewman 14y ago
Yes #4 could be problematic if not implemented carefully, but it is addressing a real problem. Parallel evolution is commonplace in the tech industry. In the current system, if two (or three, or twenty) companies legitimately and independen
431.
by snewman 14y ago
Yeah, it's pretty hard. Microsoft's spec (for the OOXML formats) is over 6,000 pages long, and extremely hard to follow by all accounts. And no, that's not a 10-page specification followed by 5,990 pages of examples. The Office apps have in
432.
by snewman 14y ago
I understand that to start something new you need to focus -- but, I'd also love to see Peninsula meetings. Maybe if there's enough interest it can happen.
433.
by snewman 15y ago
Where did you see the mention of SSDs?
434.
by snewman 15y ago
A few questions / comments (as a prospective client): 1. Do you have any thoughts or guidelines as to project size? Any lower or upper limits? (I sometimes have projects as small as a few hours' work for someone with the right background --
435.
by snewman 15y ago
A key point here is the tradeoff between energy density (how much energy the battery or capacitor can hold) and power density (how quickly the energy can be released). I'm not an expert, but just working from the graph in the article, the b
436.
by snewman 15y ago
Yes! As a hobby, I've been running long-term benchmarks on a handful of cloud services, for exactly the reason you suggest -- performance varies quite a bit over time. You can browse through some of the data at ( http://amistrongeryet.com/d
437.
by snewman 15y ago
It's all about tradeoffs and acceptable risk. A bank, an e-mail provider, and a game analytics tracking system all have different levels of risk tolerance. (Protip: delivery services already lose way more than one in 10 million packages, fo
438.
by snewman 15y ago
To be precise, I would argue this is actually a CA system. "C" because there are consistency guarantees, which are upheld even in the face of failures / partition. "A" because the system will continue making progress even after a node failu
439.
by snewman 15y ago
It's a bit of a middle ground. Yes, the replication is synchronous, which impacts availability. However, the master can remove a failed replica from the chain fairly quickly. In principle, with proper tuning, a node failure would merely cau
440.
by snewman 15y ago
Yes, good advice. But you're likely to find some large grey areas, and also some things that the ToS seems to forbid but seem reasonable / ethical, and lots of people are doing. If you avoid all of those grey and pseudo-black areas, you're
441.
by snewman 15y ago
In principle, of course you are correct. In practice, however, this viewpoint ignores a key fact of modern life. The ToS for most services nowadays are extremely draconian and one-sided. The vast majority of the time, service providers don'
442.
by snewman 15y ago
The birthday paradox kicks in when you have a set of objects, and a collision between any pair is interesting. Here, only a collision with 0 is interesting. Suppose that the sequence of values you have after each ack is A, B, C, D, E (five
443.
by snewman 15y ago
Each new value is written to the journal. Later, when the journal is compacted, the value will be written to an SSTable. Each subsequent compaction involving that value causes the value to be written (copied) yet again. So over time, a 100-
444.
by snewman 15y ago
Though sometimes it can seem that way. :)
445.
by snewman 15y ago
The headline says "active users" -- by which they might mean, users who have actively logged into the site in the last 30 days.
446.
by snewman 15y ago
> Riak has the ability to select keys for processing via various queries, including range of the key. Based on the resources I can find online, any select-by-range operation in Riak requires broadcasting to all nodes (or at least enough
447.
by snewman 15y ago
True, it's pay-for-capacity and that's worse for the user. On the flip side, the constant factor seems to be about 20x cheaper (caveat: this is based on my personal experience with SimpleDB; since Amazon doesn't seem to explain how "box usa
448.
by snewman 15y ago
Capacity Units strike me as a big improvement (for the user) over Machine Hours, because it's very clear how a given usage pattern will translate into Capacity Units. I can predict how many Capacity Units I'll need. I've gotten badly burned
449.
by snewman 15y ago
As you and others have pointed out, hosting + SSDs + synchronous replication across availability zones counts for a lot. If DynamoDB lives up to the hype, it could be a huge step forward in the world of "don't have to think about it" data s
450.
by snewman 15y ago
FWIW, this doesn't especially match my experience at Google. (I was an engineer there for four years, ending about a year and a half ago. I doubt things have changed drastically since then.) Most of the descriptions about the environment an

More ›