14 ms·
Aurora - New MySQL-Compatible Database Engine
- skj 12y agooh neat, AWS following GCP instead of the other way around.
- curiousDog 12y agoWhat is GCP?
- sciurus 12y agoGoogle Cloud Platform.
- loco77 12y agoIn his defence even googling GCP doesn't return that as a result on the first page. I was searching for the same thing.
- samstave 12y agoGCP is largely used as the acronym for "Good Clinical Practice" and has been around a heck of a lot longer than Google.
- spacemanmatt 12y agoMost acronyms (especially TLAs) get overloaded daily. Being first means next to nothing, in this space.
- arthursilva 12y agocare to explain?
- kapilvt 12y agoCloudSQL on gce is mysql on bigtable afaicr, its been around for a few years. EDIT: actually it looks like cloudsql might actually be a mysql.. also on gce they max out 16gb of ram and 100gb of storage
- loco77 12y agoAurora max looks like it's 64TB, that's 640 times larger, which I guess is what you're hinting at.
- deleted 12y ago[deleted]
- loco77 12y agoYour profile says you work at Google...
- skj 12y agoI certainly do. On cloud no less. The reality is that Amazon got to market several years before Google, so GCP had some catching up to do. That trend is changing, and this is one indication.
- loco77 12y agoIf you're going to post your subjective opinion on something involving your employer vs a competitor it's best to include a disclaimer in future.
- skj 12y agoWhat was the subjective opinion? "neat"? The rest was factual.
- loco77 12y agoThe subjective opinion was about "following" when Amazon has had relational database services for years, and the latest engine they say was under development for 3 years, not exactly something in response to goog. Irrespective of that you shouldn't be posting as joe public when in fact you are a Google employee playing cheerleader for Google products on threads about your competitors products. Keep it classy.
- skj 12y agoWe're not talking about a relational database, we're talking about a mysql-wire-compatible database. Google BigQuery has been around for a while too. Adding "disclaimer: googler" to every post I make on HN seems pretty obnoxious to both me and anyone reading. I just didn't feel that "neat" + a fact qualified. Clearly opinions on that differ, and I'll probably just post less in the future.
- state_machine 12y ago“Aurora" is already the name of a cloud infrastructure management product, an open-souce apache project no less: http://aurora.incubator.apache.org/ http://aurora.incubator.apache.org/ Using an existing name for a product in a similar space is just confusing and hurts everyone.
- cr3ative 12y agoTo be fair, I just searched for "aurora software" and "aurora open source" and got nothing for that project on the first page of results. A lot of the good names are already taken; there's going to be overlap, and they're very different products.
- covi 12y agoName-clashing with an Apache project really surprises me. Also, consider the fact that Aurora stems from Mesos, which supposedly Amazon (cloud; although DB people are doing this one) folks have to know.
- t1m 12y agoMichael Stonebraker created a database called Aurora more than 10 years ago. http://cs.brown.edu/research/aurora/ http://cs.brown.edu/research/aurora/
- elwell 12y agoAlso, the first thing I thought of when I saw the word Aurora was the 2012 shooting in Colorado. That's sad that the word is associated with that, but also a facet of the naming consideration.
- loco77 12y ago4 times faster than MySQL on the same platform, how did they pull that off?
- ericfrenkiel 12y agousing SSDs looks like
- wmf 12y agoTokuDB claims to be 20x faster, so Amazon must be slacking. :-) I don't vouch for any of these claims, but MySQL is certainly not the ultimate in DB performance.
- sciurus 12y agoHere is what they claim in the FAQ: "Amazon Aurora delivers significant increases over MySQL performance by tightly integrating the database engine with an SSD-based virtualized storage layer purpose-built for database workloads, reducing writes to the storage system, minimizing lock contention and eliminating delays created by database process threads. Our tests with SysBench show that Amazon Aurora delivers over 500,000 SELECTs/sec and 100,000 updates/sec, five times higher than MySQL running the same benchmark on the same hardware."
- 12y ago
- zippergz 12y agoHas anyone been able to find specific pricing on this? All of the language I've seen about it has been vague. One of the things that's kept me from using RDS for toy projects is that it basically amounts to running another instance on top of what I already have. For a "real" project, that's a drop in the bucket, but for toys and prototypes, it can be a significant chunk of the cost (so I usually end up just running my own db on the same instance as the web server for those projects).
- himanshuy 12y agoWhat is the use of a cloud relational database service? I thought everybody is going for Document Based databases.
- wmf 12y agoIf your data is fundamentally relational you're better off putting it into a relational DB. http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/ http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...
- ranman 12y agoJust want to point out that the headline is sensational and there are many good use cases for document stores.
- pessimizer 12y agoThe headline doesn't mention document stores, it mentions MongoDB.
- ranman 12y agoWhich (as a document store) has plenty of use cases... and even when you read the article it's mostly about how they didn't understand what they were doing going into things... It's not a compelling argument. "But this stuff wasn’t obvious at all. The MongoDB docs tell you what it’s good at, without emphasizing what it’s not good at. That’s natural. All projects do that. But as a result, it took us about six months, a lot of user complaints, and a lot of investigation to figure out that we were using MongoDB the wrong way." The film/actor/career example they wanted to do could have been solved pretty easily with a document that had an _id for an actors collection and a name... that's just the document paradigm... I'm a huge fan of both postgres and mongodb and I know that's not a popular opinion around HN. I just get tired of seeing clickbait headlines being cited as the criticism within is correct.
- marktangotango 12y ago
- sciurus 12y agoMore product details: http://aws.amazon.com/rds/aurora/details http://aws.amazon.com/rds/aurora/details and Frequently Asked Questions: http://aws.amazon.com/rds/aurora/faqs/ http://aws.amazon.com/rds/aurora/faqs/
- deleted 12y ago[deleted]
- slik33 12y agoWhy mysql? Why not postgres engine?
- deleted 12y ago[deleted]
- ahachete 12y agoNo one is saying that Aurora is built on MySQL, I'd personally like to know :) It's the protocol (drivers) compatibility what has been claimed so far. I'd agree with you from a technical perspective it would have been better to offer Postgres compatibility, rather than MySQL. But I believe the intent is to target both a bigger market (as of today) and challenge Oracle.
- BenoitP 12y ago> tables that I create in Amazon Aurora must use the InnoDB engine It hasn't been claimed, but the article is filled with MySQL comparisons and references. I would not be surprised if it was a MySQL fork.
- gdulli 12y agoThis is likely for the people who for whatever reason just aren't willing to switch to Postgres.
- spacemanmatt 12y agoThere are a lot of apps with complete ecosystems that aren't going to port to PostgreSQL anytime soon. Wordpress is the poster-child for this pattern. Yes, you can bring it up on PostgreSQL, and some modules support it, but from what I've heard, it's pretty spotty and most of the popular modules do not support it.
- dubcanada 12y agoEven though PostgreSQL may be better. MySQL is still significantly more popular. That's why.
- 12y ago
- frozenport 12y agoDoes this introduce more latency compared to running a LAMPs stack on EC2 or does it target larger databases? My database is about 300 megabytes.
- Xorlev 12y agoYou probably aren't the target audience if your database fits in RAM.
- frozenport 12y agoYou can see notable changes in performance if your database gets fragmented, and tweaking the .my.cnf script seems like a massive distraction from my actual work. Having somebody handle these and improve performance for long queries would be wonderful.
- jcampbell1 12y agoWith a database that small, performance shouldn't ever be noticeable. Are you running the DB on EBS? 1) If you are having trouble with tweaking my.cnf, give the perl script `mysqltuner` a go. It is pretty good. 2) The biggest improvement would be moving the storage to SSDs. I personally prefer Linode to DO, but both are cheap, and good. My linode servers can do full text non-blocking backups at about 100MB per second, so depending on your outage recovery requirement, you may be able to get away with a simple cron job.
- vidarh 12y agoI you manage to have "long queries" with a 300MB dataset, you're doing something seriously wrong unless your definition of "long" is very different from mine. Or you're running on harware from the 90's. Run "explain" on all your queries and review your indexes to begin with. That dataset is so small that there should also be no need to be continuously tweaking the config file. 10 minutes looking at a performance guide, and ensuring you have enough ram to load everything into cache should be enough to get your configuration to a "good enough" shape - there's just nothing reasonable you'll be doing with a 300MB dataset that should need anything but getting the very basics of the config right to perform decently.
- jasondc 12y agoI think the problem AWS is seeing is how they are being commoditized (e.g. you can just run your database on the cheapest hosting provider), so profits will move towards 0. They will need to add more services like this (Aurora, DynamoDB, etc.) to ensure AWS isn't a commodity (and you can't just easily switch to DigitalOcean).
- Cieplak 12y agoI think that the AWS vendor lock-in won't come from single features such as databases from specific vendors, but rather the ecosystem of Route53, elastic load balancers, S3, EBS, AMIs, Glacier, RDS, CloudFormation templates, SQS, security groups, libraries like boto, auto scaling groups, and everything else besides instance provisioning. Of course you can implement most of these yourself, e.g., HAProxy for ELBs, rabbitmq for SQS, but all that requires application code changes and significantly more configuration management.
- freshflowers 12y agoSo far I've been able to avoid vendor lock-in on the code level (besides S3, but the S3 API is simple and has become pretty much a standard for object storage). On the infrastructure level, the "lock-in" part is not so much the technology, all of that is relatively easy to replace. But it would take me two additional FTE's to configure and manage everything AWS offers as a service ourselves. But that applies to a lot of things these days, AWS just happens to be a one stop shop for a growing range of commodity services.
- vishnugupta 12y ago> but all that requires application code changes and significantly more configuration management. And not to talk about making them highly available, scalable, having to continuously monitor and so on takes time, effort and involves significant opportunity costs. Imagine business folks telling engineers about an upcoming promotion campaign that bring in 3X more load. Outside of AWS (or cloud provided service) one has to sit and plan for scaling up all the individual components (rabbitmq, haproxy etc.,) which really becomes pain after a few iterations.
- andr 12y agoReading between the lines, it sounds like they've rewritten InnoDB's storage layer and made it multi-server aware w/ consensus.
- dschiptsov 12y agoSo, this is a Xen VPS (Amazon uses Xen as far as I remeber) with a MySQL 5.6 database, with a custom storage engine that stores data in some Dynamo-based storage. Any code could access it via MySQL protocol (bindings to libmysqlclient.so). Fine.
- jakozaur 12y agoI appreciate your technical insight, but glueing those technologies together is far from trivial. Making it easily to manage and reliable is even harder. I trust your best intention, but even Dropbox was dismissed on HN as user-friendly rsync.
- dschiptsov 12y agoOK, perhaps they wrote a MySQL-Dynamo (or whatever it is) wire-protocol translation engine, and do not use MySQL server. But custom storage engine looks like most natural way. We will see it shortly.)
- arnehormann 12y agoMariaDB enabled Cassandra as a storage engine: https://mariadb.com/kb/en/mariadb/documentation/storage-engines/cassandra/cassandra-storage-engine-overview/ https://mariadb.com/kb/en/mariadb/documentation/storage-engi...
- bcantrill 12y ago(Disclaimer: I work for an AWS competitor.) To me, there is a very interesting contrast to be had between this announcement and Microsoft's announcement: it feels like Microsoft is discovering the business value of being open at the same time that Amazon is living in the time warp of proprietary everything. Has Microsoft internalized that open source is (or can be) a differentiator in the cloud? Amazon is clearly still oblivious to it -- and it will be very interesting to see if this service generates fear of vendor lock-in...
- chuhnk 12y agoI have to agree with this. Amazon sees the competitive landscape and their move is to attempt to create some form of vendor lock-in by providing the services that their customers have asked for or are probably hosting themselves. This is by no means a bad move and will result in holding on to a lot more customers as well as creating new revenue streams from those services. For someone like myself however, it completely turns me off the AWS path. I will use AWS as a compute resource, as well as any other cloud provider. They will fight a battle for profit while compute costs continue to come down allowing me to reap the benefits while building a cloud agnostic stack.
- EGreg 12y agoOf course Amazon wants vendor lock-in. Their whole business model is about being a marketplace. Do you want them to open-source their shopping cart software? And in Amazon Web Services, the biggest thing Amazon has to fear is the OpenStack initiative producing a platform that lots of little hosts can host. At the same time, Amazon is stock market time bomb that will either start extracting monopoly rents and draw regulatory ire, or whose stock will drop, because of the amount of revenue multiples. That said ... this is the way it nearly always works. Microsoft invented .NET over 10 years ago, and has kept it proprietary, leveraging the dominance of Windows, extracting rents. Now that Mobile is the new hotness and Microsoft needs to catch up, it goes open source. Google released Android for the same reason. In short ... the companies that devote a lot of resources to R&D usually like to recoup the costs, but eventually PLATFORMS get opened. Frankly, the world benefits more when this happens sooner (Netscape open sources its codebase, the Web is designed, etc.) but the engine of progress in capitalism is still speculative investment, and that needs to pay off for years before the public at large can benefit, if investors are to continue investing. Once we switch to crowdfunding, however, this model of capitalism and risktaking may be disrupted as well.
- deleted 12y ago[deleted]
- LeonD 12y agoIs a relational workload that requires four-nines-plus availability like Amazon is touting really something an organization would be willing to run in the cloud? Especially since that category includes a lot of personally identifiable data that legally can't be trusted to a third party. Wonder what kind of use cases they're aiming for.
- techdebt5112 12y agopeople should be roughly as comfortable with this as with operating mysql on EC2 or using RDS, and I think that's quite common.
- fortpoint 12y agoAmazon's environments meet PCI standards and there are folks with PCI Level 1 companies running their infrastructure in AWS and storing credit card data there.
- AaronFriel 12y agoOuch. This pricing is pretty rough for SaaS sold on the premise of cloud scalability. At $200/month for the entry level, their lowest price is many times what the cheapest geo-replicated "SQL engine as a service" from Google or Microsoft is. I'm not sure how the performance differs, but I am guessing theirs are no slouches. Microsoft offers "SQL Database" geo-replicated for as low as $15/mo., and it scales up from there. Not sure about performance, but it would be apples to oranges (MySQL versus SQL Server) and difficult to compare. I wonder what the TPC numbers are, but apparently the TPC organization doesn't allow publishing that yet. Google offers "Google Cloud SQL", also geo-replicated, and their cheapest pricing is between $10 and $18 dollars a month.
- gaadd33 12y agoI don't think this is targeted towards those who only need to store a small amount of data and have very low performance needs. If you are are already spending $500/mo on an RDS instance, then it sounds like this would be a great solution. If you are spending $10/mo on a micro instance for your database server, I don't think this is targeted towards you.
- kidambisrinivas 12y agoThe google cloud SQL pricing[1] for a comparable specs tier (D32) is 35$/day (~900+) which is way expensive compared to aws db.r3.large [1] https://cloud.google.com/sql/pricing https://cloud.google.com/sql/pricing
- ohyesyodo 12y agoIt's not SQL Server, it's SQL Database. There are many large differences between the two. A 50GB database with 10GB of RAM usage and 2 cores costs 700USD/Mo with SQL Database. This is replicated twice within DC. Double that price if you want georedundancy. Their entry level costs less but is also unusable for most real applications
- donmaq 12y agoI'm interested in transaction support for e-commerce. Magento users can experience significant performance issues on MySQL, and the upper tier are always interested in more RDBMS performance, eg via NewSQL solutions. So what can Aurora do for that workload? Do the support multi-table transactions and referential integrity across all 3 Availability Zones? Similarly, they mentioned Durability targets; what's their targets for Consistency (ie ACID).
- i_have_to_speak 12y agoThis is beginning to feel like how it was when Intel was competing with AMD -- evolutionary improvements at first, which just kept on coming, increasing in size and impact, until AMD just faded out of the high-perf server market.