27 ms·

Call me maybe: Aerospike

383 points by joshrotenberg 11y ago

jack9 11y ago
The minimum useful licensing is in the tens of thousands of dollars. For the SLA they offer, it makes sense. Many well known Ad Serving companies utilize Aerospike (at a fraction of the cost of their previous solutions). It's very impressive result, per machine, from an operational standpoint.
- deleted 11y ago
  [deleted]
yellowapple 11y ago
When a product claims to have 100% uptime, I immediately cringe, knowing full well that they're probably full of bovine manure. Good read.
- saryant 11y ago
  Same with "exactly once delivery".
- grogers 11y ago
  My analog to this is when someone claims that partitions are rare so you can ignore them, any product they make is very likely to lose data during partitions.
- peterwwillis 11y ago
  https://www.youtube.com/watch?v=pjvQFtlNQ-M https://www.youtube.com/watch?v=pjvQFtlNQ-M
stephen_mcd 11y ago
The rotated A in the Aerospike logo reminds me of a system that's fallen over, and now you can't unsee it: http://www.aerospike.com/ http://www.aerospike.com/
- drivers99 11y ago
  Trying to read aerospike's front page to figure out what it is reminds me of this: http://shouldiuseacarousel.com/ http://shouldiuseacarousel.com/
- oska 11y ago
  "A system"? Not sure what you mean by that. To me, it was immediately apparent that they are making the whole word Aerospike look like a rocket, a motif they repeat through their home-page.
  - simoncion 11y ago
    Worker 1: "The server has fallen over." Worker 2: "I'll go restart it." See also: The frequent interchangeability of the words "system" and "computer".
- AnDowNS 11y ago
  My co-worker glanced at my screen and thought I was reading about erospike, which has a decidedly different feel, especially given the phallic imagery on the page.
- curun1r 11y ago
  Really? When I look at it I only see an eye (viewed in profile) with the legs of the A being eye lashes.
SchizoDuckie 11y ago
Oh wow, so this is not ACID, not "Eventual Consistency", but "Eventual Inconsistency"
DAddYE 11y ago
Awesome read and still quite impressed by Aerospike.
- misframer 11y ago
  What is impressive about being somewhat misleading and cutting corners to gain performance?
  - DAddYE 11y ago
    As said by Aphyr this product is ideal in the ad-tech. Of course is misleading but good engs will test products to see if it fit their scenario without basing decision on marketing slides. Yes, ideally they should be more clear and I bet after this article they will.
    - rgbrenner 11y ago
      As said by Aphyr this product is ideal in the ad-tech No, Aphyr said the data loss is ok for ad tracking and analytics because it doesn't matter. That's very different. And if that's the case, then why make those claims.. they could just as easily give accurate info to their customers, and the customer could decide if that fits their case. Instead they claim something very difficult (if not impossible), and let their customers find out it's not true (possibly after it's too late, and they've already lost valuable data).
- Dylan16807 11y ago
  It's easy to make something that's fast and doesn't always work. Give a coder a day and they can make a high speed, replicated, key-value store that loses a little bit of data during partitions. Heck, that could be a single homework assignment in an algorithm or distributed system class.
Ankhers 11y ago
I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is happening. When dealing with monetary values, these fluctuations are very bad for us. Needless to say, we have begun migrating away from Aerospike.
- smt88 11y ago
  Yikes. What made you choose Aerospike in the first place?
  - Ankhers 11y ago
    Sadly, a requirement of the application is that it needs to be backward compatible with the previous system. I'm sure many of you know, this leads to quite a few issues.
- obituary_latte 11y ago
  Can you add any detail to this anecdote? It's interesting and important, but detail might help steer others appropriately. What kind of inconsistency? What kind of fluctuation?
  - Ankhers 11y ago
    I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then continuing up. Rinse repeat. (e.g., 158 -> 160 -> 158 -> 170 -> 175 -> 173 -> 180) The second I found was when trying to debugging the previous issue. I noticed the counters were essentially cycling. (e.g., 158 -> 160 -> 170 -> 158 -> 160 -> 170). This just repeated for the duration we watched the counters (approximately five minutes). Please note that I used small numbers here. The counters I was monitoring were in the hundred millions, and I saw decrements in average between 2-3k.
elchief 11y ago
If you're big enough to require something like Aerospike, you're rich enough to build something like F1.
- vosper 11y ago
  I dunno, there are ad-tech companies with less than 20 engineers who're processing millions of events per minute on their endpoints, and trying to do various things with that data. This from personal experience. Of course, I would love if someone gave me the mandate to go out and build something like F1...
- rbetts 11y ago
  You couldn't even hire a couple of experienced engineers let alone come close to affording to build F1. There are several zeros in the cost difference.
- nemothekid 11y ago
  I can think of quite a few companies that are big enough to require something like Aerospike, but wouldn't be willing to outfit all their machines with a GPS receiver.
- peterwwillis 11y ago
  Even if your company has 20 billion in cash, if your department isn't called "sales" or "marketing", good luck getting the budget for that. And honestly, if you have money, it's a lot simpler and less risky to just hire 20 DBAs and programmers to build a database application that can handle that kind of operation. Low latency, network-partition-resistant, high-performance database applications are not a new thing.
tootie 11y ago
Is there any reason he's never tried to analyze a "classic" RDBMS like Oracle or SQL Server? I have to imagine they'd clobber a lot of this hipster technology.
- samkone 11y ago
  Lol He can't because Oracle licensing forbids to do this kind of work and make the results public.
  - tootie 11y ago
    Thank for a real answer. Do you have a source for that assertion. Is Microsoft the same?
- electrum 11y ago
  He tested PostgreSQL: https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres
  - threeseed 11y ago
    I don't understand how this is comparable though. All of other databases were tested in clustered mode. Why not PostgreSQL as well ?
    - rsynnott 11y ago
      I don't think postgres has a first-party clustering system.
    - Jweb_Guru 11y ago
      Postgres doesn't have a builtin clustered mode, or claim to be totally available (and also, incidentally, doesn't guarantee serializability for hot standby replication targets, which is prominently stated in a bright red box that says "warning" in the documentation on serializable isolation, i.e. the first place you would look). It claims to be CP in a single-node configuration (which it is) and aphyr tested it on those claims. Remember, Jepsen isn't about proving that a database can't violate CAP, nor is it trying to say "all databases are crap." It's about verifying the marketing claims and then determining whether there are any mitigating strategies. The fact that many databases make unreasonable marketing claims is unfortunate, but certainly not a requirement. It's also worth posing this question in reverse: what would happen if these distributed databases were tested in a single-cluster configuration? As noted in the most recent article on Elasticsearch, many of them (e.g. Elasticsearch, Cassandra, and Riak) acknowledge writes before fsync and can therefore lose data due to issues like `kill -9`, power loss, and other exceptional conditions, while Postgres doesn't. For a single-node database this robustness is very important, while he argues that it isn't as important for a distributed one. Because these databases aren't designed to be used as single nodes, aphyr didn't substantially ding them for that. Again, what's important is whether the database does what its documentation says it does when used as its documentation says it should be used.
deleted 11y ago
[deleted]
yetanother415 11y ago
Can't say I'm surprised. Their CTO Brian has got to be one of the most obnoxious, arrogant neckbeards I've ever had the misfortune of working with. Though in the valley these days unbridled and unwarranted confidence in your product is often an asset.
theVirginian 11y ago
"schemaless" nope
pkaye 11y ago
Fast, functional or reliable... pick two.
- me_again 11y ago
  ITYM consistent, available or partition-tolerant...
  - Jweb_Guru 11y ago
    ...but you can't pick CA. Something Aerospike didn't realize. (And nope, sorry, I'm completely uninterested in your anecdotes about how you haven't personally lost data when [1] there's a clear data loss scenario highlighted in the post, [2] Aerospike actively recommend services like EC2 and GCE that routinely partition, and [3] there are people in this thread who have experienced the same problems).
gmagnusson 11y ago
I've used Aerospike at scale (approx 1MM tx per second) in private network, and smaller loads in cloud. I have always found it to be fast, reliable and extremely easy to operate (upgrade, modify cluster members, etc) w/o any downtime or interruption. It is a critical tool in my toolbox. I also have found their support and engineering team to be excellent. I admire the work that Aphyr does - though at the end of the day, I need to build systems that work for the problem I'm trying to solve (and I have to choose from real things that are available). Aerospike isn't the solution to every storage problem, and if you are choosing technology based on marketing material, you're probably going to be disappointed. These technologies in general are trying to address really hard problems and design and architecture is the art of balancing tradeoffs. Nothing is going to be perfect. Yet.
- nemothekid 11y ago
  While people tend to use Aphyr's posts as anti-nosql-cannon-fodder, I don't think he has ever advocated for anyone to straight up not use a database or used any of these results to show that no one should use NoSQL ever. These seem like highly detailed Github Issues (infact the recent elasticsearch was a GH issue-turned-blog post), and these issue are brought to attention so that they could be fixed - not to slander the name of the company (and when they are, everyone benefits). IIRC, even after finding these bugs were published he continued to use elasticsearch. Given how hard these problems are and how difficult they can be to reproduce, these writeups seem to be the most appropriate way to highlight these issues. That said, if I was an aerospike user I'd be happier knowing this issue exists, someone has debugged it, and supported a detailed report about rather than being called in at 3am and discovering our data is funky.
  - draven 11y ago
    In addition the testing methodology (the blog post) and the code (on github) are available so the tests can be reproduced to validate any enhancement or bugfix. Those blog posts are also a great at debunking marketing claims.
- ploxiln 11y ago
  "though at the end of the day" "nothing is perfect" ... Aerospike makes blatantly ridiculous promises in their high-level descriptions of their database. That makes our jobs harder (before Aphyr makes them easier) because we don't know what Aerospike is actually good at or exactly what kinds of data loss potential we need to architect our systems around. Isn't it kind of annoying that some technical projects bolster their popularity/ecosystem with very fancy websites and impressive/competitive claims, but to really do your job right you have to throw all that away? The best you can do is try to get a sense from the reports of others who have tried something (and may or may not have been rigorous in their evaluation) so you can pick good candidates to even put through trials. (so again, thanks Aphyr)
bketelsen 11y ago
Many kudos to Stripe for funding this. Truly a great gift to the community.
- mdellabitta 11y ago
  I'm imagining the job interview. "We want to sponsor your distributed database research, but not your Barbie animated GIF production."
cschneid 11y ago
What is the 3 color chart that I've seen posted in several of these articles? I get that it breaks down the different CAP combinations with the things that it allows/implies. But is there a good breakdown of all the terms and what they mean?
- dwetterau 11y ago
  Here is a paper [0] out of Berkeley that explains the chart you're asking about (the chart appears on page 8). For more information on each isolation level you might have to refer to the cited works or other sources. [0] http://db.cs.berkeley.edu/papers/vldb14-hats.pdf http://db.cs.berkeley.edu/papers/vldb14-hats.pdf
  - cschneid 11y ago
    Thank you! That's perfect.
manigandham 11y ago
We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has much tighter timings and tolerances in exchange for providing much higher performance in certain situations and definitely has one of the best SSD focused storage systems I've come across. If you don't have a high performance network interconnect between nodes, then there will be more issues with Aerospike since it relies on that more than some other system that use Paxos for all writes (like aphyr mentions). We run several TB's of data accessed at 100k+ TPS including very fine grained counters and everything works. And yes, we run on the cloud in AWS and SoftLayer and have yet to have major problems with the proper network setup. Btw, there is a comment below from the current CTO of AppNexus, one of the companies that pioneered real-time bidding for digital ads and runs several million auctions per second on one of the biggest ad exchanges available. They were the first customer for Aerospike and from everything I've learned from their team, it works really well for them, and they definitely are not happy to just "lose" data however insignificant it might seem. Volume changes everything and even a fraction of a percent will add up. We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage, the key is being aware of all the technical requirements and the environment you're deploying in. I think the real major issue here that people seem upset with are the general claims and marketing information. I can't speak to all that and there are definitely some things like 100% uptime which do seem overly confident, but this is true of every single technology vendor out there unfortunately. I'm not saying Aerospike is any better or worse as a company but marketing material only goes so far and it would surprise me if further research wasn't done for any mission critical system.
- teraflop 11y ago
  > There is no perfect database out there, all of them will fail with network partitions. Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise? The "crazy revelation" for me was not that Aerospike's software is, like everything else, subject to the CAP theorem. It's that they apparently think it's awesome to claim that it isn't, and charge tens of thousands of dollars for their product on that justification.
acqq 11y ago
I was wondering what is going on, with the titles of this form appearing on HN and now browsing the blog, it appears that the author is the fan of the "Call me maybe" titled posts (a lot of them there, and then also here!). From what I understand, this phrase, as used in the titles by him, seem to mean to him something like the "review of" something or the "comment on" something. For what it's worth.
- beberlei 11y ago
  It is a pun on this song https://www.youtube.com/watch?v=fWNaR-rxAic https://www.youtube.com/watch?v=fWNaR-rxAic
  - acqq 11y ago
    Apart from author being obviously inspired by the song I admit I don't see any connection or anything worth naming that "the pun." Maybe it's just me.
    - chipsy 11y ago
      Making a phone call is an asynchronous event, and as the song suggests, sometimes you give someone your number but they never call back. With any distributed system in real world conditions, a similar situation arises where a request doesn't get handled or is lost along the way. </explainer>
      - acqq 11y ago
        So he uses the phrase instead of "distributed systems?" (shrug)
    - lmm 11y ago
      It's explained at the top of the original post: https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-a...

383 points by joshrotenberg 11y ago

jack9 11y ago

The minimum useful licensing is in the tens of thousands of dollars. For the SLA they offer, it makes sense. Many well known Ad Serving companies utilize Aerospike (at a fraction of the cost of their previous solutions). It's very impressive result, per machine, from an operational standpoint.

deleted 11y ago
[deleted]

yellowapple 11y ago

When a product claims to have 100% uptime, I immediately cringe, knowing full well that they're probably full of bovine manure. Good read.

saryant 11y ago
Same with "exactly once delivery".
grogers 11y ago
My analog to this is when someone claims that partitions are rare so you can ignore them, any product they make is very likely to lose data during partitions.
peterwwillis 11y ago
https://www.youtube.com/watch?v=pjvQFtlNQ-M https://www.youtube.com/watch?v=pjvQFtlNQ-M

stephen_mcd 11y ago

The rotated A in the Aerospike logo reminds me of a system that's fallen over, and now you can't unsee it: http://www.aerospike.com/ http://www.aerospike.com/

drivers99 11y ago
Trying to read aerospike's front page to figure out what it is reminds me of this: http://shouldiuseacarousel.com/ http://shouldiuseacarousel.com/
oska 11y ago
"A system"? Not sure what you mean by that. To me, it was immediately apparent that they are making the whole word Aerospike look like a rocket, a motif they repeat through their home-page.
- simoncion 11y ago
  Worker 1: "The server has fallen over." Worker 2: "I'll go restart it." See also: The frequent interchangeability of the words "system" and "computer".
AnDowNS 11y ago
My co-worker glanced at my screen and thought I was reading about erospike, which has a decidedly different feel, especially given the phallic imagery on the page.
curun1r 11y ago
Really? When I look at it I only see an eye (viewed in profile) with the legs of the A being eye lashes.

SchizoDuckie 11y ago

Oh wow, so this is not ACID, not "Eventual Consistency", but "Eventual Inconsistency"

DAddYE 11y ago

Awesome read and still quite impressed by Aerospike.

misframer 11y ago
What is impressive about being somewhat misleading and cutting corners to gain performance?
- DAddYE 11y ago
  As said by Aphyr this product is ideal in the ad-tech. Of course is misleading but good engs will test products to see if it fit their scenario without basing decision on marketing slides. Yes, ideally they should be more clear and I bet after this article they will.
  - rgbrenner 11y ago
    As said by Aphyr this product is ideal in the ad-tech No, Aphyr said the data loss is ok for ad tracking and analytics because it doesn't matter. That's very different. And if that's the case, then why make those claims.. they could just as easily give accurate info to their customers, and the customer could decide if that fits their case. Instead they claim something very difficult (if not impossible), and let their customers find out it's not true (possibly after it's too late, and they've already lost valuable data).
Dylan16807 11y ago
It's easy to make something that's fast and doesn't always work. Give a coder a day and they can make a high speed, replicated, key-value store that loses a little bit of data during partitions. Heck, that could be a single homework assignment in an algorithm or distributed system class.

Ankhers 11y ago

I currently work for a company that uses Aerospike quite heavily. In the past couple weeks, we have begun to notice data inconsistencies in our counters. We are seeing fluctuations in the data, despite having no decrement operations. We have the enterprise edition of Aerospike, allowing us to be in constant contact with their support team and developers. A couple weeks later, and we still have no idea why this is happening. When dealing with monetary values, these fluctuations are very bad for us. Needless to say, we have begun migrating away from Aerospike.

smt88 11y ago
Yikes. What made you choose Aerospike in the first place?
- Ankhers 11y ago
  Sadly, a requirement of the application is that it needs to be backward compatible with the previous system. I'm sure many of you know, this leads to quite a few issues.
obituary_latte 11y ago
Can you add any detail to this anecdote? It's interesting and important, but detail might help steer others appropriately. What kind of inconsistency? What kind of fluctuation?
- Ankhers 11y ago
  I'm current working in AdTech. We are using counters to keep track of a lot of things that are important to us (e.g., reasons for not bidding on a given campaign, money spent within a given transaction, number of bid requests we get by exchange, etc). I personally have found two different, yet very similar, data fluctuations. The first I found when debugging an issue. I noticed the counters going up, dip down, then continuing up. Rinse repeat. (e.g., 158 -> 160 -> 158 -> 170 -> 175 -> 173 -> 180) The second I found was when trying to debugging the previous issue. I noticed the counters were essentially cycling. (e.g., 158 -> 160 -> 170 -> 158 -> 160 -> 170). This just repeated for the duration we watched the counters (approximately five minutes). Please note that I used small numbers here. The counters I was monitoring were in the hundred millions, and I saw decrements in average between 2-3k.

elchief 11y ago

If you're big enough to require something like Aerospike, you're rich enough to build something like F1.

vosper 11y ago
I dunno, there are ad-tech companies with less than 20 engineers who're processing millions of events per minute on their endpoints, and trying to do various things with that data. This from personal experience. Of course, I would love if someone gave me the mandate to go out and build something like F1...
rbetts 11y ago
You couldn't even hire a couple of experienced engineers let alone come close to affording to build F1. There are several zeros in the cost difference.
nemothekid 11y ago
I can think of quite a few companies that are big enough to require something like Aerospike, but wouldn't be willing to outfit all their machines with a GPS receiver.
peterwwillis 11y ago
Even if your company has 20 billion in cash, if your department isn't called "sales" or "marketing", good luck getting the budget for that. And honestly, if you have money, it's a lot simpler and less risky to just hire 20 DBAs and programmers to build a database application that can handle that kind of operation. Low latency, network-partition-resistant, high-performance database applications are not a new thing.

tootie 11y ago

Is there any reason he's never tried to analyze a "classic" RDBMS like Oracle or SQL Server? I have to imagine they'd clobber a lot of this hipster technology.

samkone 11y ago
Lol He can't because Oracle licensing forbids to do this kind of work and make the results public.
- tootie 11y ago
  Thank for a real answer. Do you have a source for that assertion. Is Microsoft the same?
electrum 11y ago
He tested PostgreSQL: https://aphyr.com/posts/282-call-me-maybe-postgres https://aphyr.com/posts/282-call-me-maybe-postgres
- threeseed 11y ago
  I don't understand how this is comparable though. All of other databases were tested in clustered mode. Why not PostgreSQL as well ?
  - rsynnott 11y ago
    I don't think postgres has a first-party clustering system.
  - Jweb_Guru 11y ago
    Postgres doesn't have a builtin clustered mode, or claim to be totally available (and also, incidentally, doesn't guarantee serializability for hot standby replication targets, which is prominently stated in a bright red box that says "warning" in the documentation on serializable isolation, i.e. the first place you would look). It claims to be CP in a single-node configuration (which it is) and aphyr tested it on those claims. Remember, Jepsen isn't about proving that a database can't violate CAP, nor is it trying to say "all databases are crap." It's about verifying the marketing claims and then determining whether there are any mitigating strategies. The fact that many databases make unreasonable marketing claims is unfortunate, but certainly not a requirement. It's also worth posing this question in reverse: what would happen if these distributed databases were tested in a single-cluster configuration? As noted in the most recent article on Elasticsearch, many of them (e.g. Elasticsearch, Cassandra, and Riak) acknowledge writes before fsync and can therefore lose data due to issues like `kill -9`, power loss, and other exceptional conditions, while Postgres doesn't. For a single-node database this robustness is very important, while he argues that it isn't as important for a distributed one. Because these databases aren't designed to be used as single nodes, aphyr didn't substantially ding them for that. Again, what's important is whether the database does what its documentation says it does when used as its documentation says it should be used.

deleted 11y ago

[deleted]

yetanother415 11y ago

Can't say I'm surprised. Their CTO Brian has got to be one of the most obnoxious, arrogant neckbeards I've ever had the misfortune of working with. Though in the valley these days unbridled and unwarranted confidence in your product is often an asset.

theVirginian 11y ago

"schemaless" nope

pkaye 11y ago

Fast, functional or reliable... pick two.

me_again 11y ago
ITYM consistent, available or partition-tolerant...
- Jweb_Guru 11y ago
  ...but you can't pick CA. Something Aerospike didn't realize. (And nope, sorry, I'm completely uninterested in your anecdotes about how you haven't personally lost data when [1] there's a clear data loss scenario highlighted in the post, [2] Aerospike actively recommend services like EC2 and GCE that routinely partition, and [3] there are people in this thread who have experienced the same problems).

gmagnusson 11y ago

I've used Aerospike at scale (approx 1MM tx per second) in private network, and smaller loads in cloud. I have always found it to be fast, reliable and extremely easy to operate (upgrade, modify cluster members, etc) w/o any downtime or interruption. It is a critical tool in my toolbox. I also have found their support and engineering team to be excellent. I admire the work that Aphyr does - though at the end of the day, I need to build systems that work for the problem I'm trying to solve (and I have to choose from real things that are available). Aerospike isn't the solution to every storage problem, and if you are choosing technology based on marketing material, you're probably going to be disappointed. These technologies in general are trying to address really hard problems and design and architecture is the art of balancing tradeoffs. Nothing is going to be perfect. Yet.

nemothekid 11y ago
While people tend to use Aphyr's posts as anti-nosql-cannon-fodder, I don't think he has ever advocated for anyone to straight up not use a database or used any of these results to show that no one should use NoSQL ever. These seem like highly detailed Github Issues (infact the recent elasticsearch was a GH issue-turned-blog post), and these issue are brought to attention so that they could be fixed - not to slander the name of the company (and when they are, everyone benefits). IIRC, even after finding these bugs were published he continued to use elasticsearch. Given how hard these problems are and how difficult they can be to reproduce, these writeups seem to be the most appropriate way to highlight these issues. That said, if I was an aerospike user I'd be happier knowing this issue exists, someone has debugged it, and supported a detailed report about rather than being called in at 3am and discovering our data is funky.
- draven 11y ago
  In addition the testing methodology (the blog post) and the code (on github) are available so the tests can be reproduced to validate any enhancement or bugfix. Those blog posts are also a great at debunking marketing claims.
ploxiln 11y ago
"though at the end of the day" "nothing is perfect" ... Aerospike makes blatantly ridiculous promises in their high-level descriptions of their database. That makes our jobs harder (before Aphyr makes them easier) because we don't know what Aerospike is actually good at or exactly what kinds of data loss potential we need to architect our systems around. Isn't it kind of annoying that some technical projects bolster their popularity/ecosystem with very fancy websites and impressive/competitive claims, but to really do your job right you have to throw all that away? The best you can do is try to get a sense from the reports of others who have tried something (and may or may not have been rigorous in their evaluation) so you can pick good candidates to even put through trials. (so again, thanks Aphyr)

bketelsen 11y ago

Many kudos to Stripe for funding this. Truly a great gift to the community.

mdellabitta 11y ago
I'm imagining the job interview. "We want to sponsor your distributed database research, but not your Barbie animated GIF production."

cschneid 11y ago

What is the 3 color chart that I've seen posted in several of these articles? I get that it breaks down the different CAP combinations with the things that it allows/implies. But is there a good breakdown of all the terms and what they mean?

dwetterau 11y ago
Here is a paper [0] out of Berkeley that explains the chart you're asking about (the chart appears on page 8). For more information on each isolation level you might have to refer to the cited works or other sources. [0] http://db.cs.berkeley.edu/papers/vldb14-hats.pdf http://db.cs.berkeley.edu/papers/vldb14-hats.pdf
- cschneid 11y ago
  Thank you! That's perfect.

manigandham 11y ago

We use Aerospike heavily. It works just fine. I'm constantly surprised by the general tone of comments on posts like these as if it's some crazy revelation that this software still obeys the fundamental laws of distributed systems. There is no perfect database out there, all of them will fail with network partitions. Aerospike was designed to work in clusters that are very close together, often the same rack. It has much tighter timings and tolerances in exchange for providing much higher performance in certain situations and definitely has one of the best SSD focused storage systems I've come across. If you don't have a high performance network interconnect between nodes, then there will be more issues with Aerospike since it relies on that more than some other system that use Paxos for all writes (like aphyr mentions). We run several TB's of data accessed at 100k+ TPS including very fine grained counters and everything works. And yes, we run on the cloud in AWS and SoftLayer and have yet to have major problems with the proper network setup. Btw, there is a comment below from the current CTO of AppNexus, one of the companies that pioneered real-time bidding for digital ads and runs several million auctions per second on one of the biggest ad exchanges available. They were the first customer for Aerospike and from everything I've learned from their team, it works really well for them, and they definitely are not happy to just "lose" data however insignificant it might seem. Volume changes everything and even a fraction of a percent will add up. We trust Aerospike because it's been hardened by lots of much much larger companies with very high production usage, the key is being aware of all the technical requirements and the environment you're deploying in. I think the real major issue here that people seem upset with are the general claims and marketing information. I can't speak to all that and there are definitely some things like 100% uptime which do seem overly confident, but this is true of every single technology vendor out there unfortunately. I'm not saying Aerospike is any better or worse as a company but marketing material only goes so far and it would surprise me if further research wasn't done for any mission critical system.

teraflop 11y ago
> There is no perfect database out there, all of them will fail with network partitions. Some of them will fail in a way that keeps your data safe, others will fail in a way that preserves uptime but gives you temporarily inconsistent data. Aerospike apparently does neither. Why is it unreasonable to expect them not to falsely claim otherwise? The "crazy revelation" for me was not that Aerospike's software is, like everything else, subject to the CAP theorem. It's that they apparently think it's awesome to claim that it isn't, and charge tens of thousands of dollars for their product on that justification.

acqq 11y ago

I was wondering what is going on, with the titles of this form appearing on HN and now browsing the blog, it appears that the author is the fan of the "Call me maybe" titled posts (a lot of them there, and then also here!). From what I understand, this phrase, as used in the titles by him, seem to mean to him something like the "review of" something or the "comment on" something. For what it's worth.

beberlei 11y ago
It is a pun on this song https://www.youtube.com/watch?v=fWNaR-rxAic https://www.youtube.com/watch?v=fWNaR-rxAic
- acqq 11y ago
  Apart from author being obviously inspired by the song I admit I don't see any connection or anything worth naming that "the pun." Maybe it's just me.
  - chipsy 11y ago
    Making a phone call is an asynchronous event, and as the song suggests, sometimes you give someone your number but they never call back. With any distributed system in real world conditions, a similar situation arises where a request doesn't get handled or is lost along the way. </explainer>
    - acqq 11y ago
      So he uses the phrase instead of "distributed systems?" (shrug)
  - lmm 11y ago
    It's explained at the top of the original post: https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions https://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-a...