7 ms·
This release shows the different philosophies of Google vs Amazon in an interesting way. Google prefers building advanced systems that let you do things "the o
by jedberg 10y ago
This release shows the different philosophies of Google vs Amazon in an interesting way.
Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable.
Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it.
I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction.
I've always said that Google is technically better than AWS, but no one will ever know because they don't have a strong sales team to go and show people.
This release only solidifies that point.
- hueving 10y ago>and release products that let you do things "the old way" while hiding the details in an abstraction. However, by 'abstracting' this away, you're not being forced to think about failure domains. If there is ever a massive country-wide connectivity break to the wider Internet (feasible for lots of people inside censored countries), you'll be pretty pissed when you can't use the DB services for your servers in the Google-local datacenter that you still have connectivity to because it can't get quorum.
- jedberg 10y agoExactly my point. I would say I personally prefer the Amazon way of forcing you to think about these things.
- boulos 10y agoI'd encourage you to read the F1 paper and the Spanner paper if you haven't already. The big thing that stood out to me is that Google started that way (with BigTable) where you'd roll your own availability and transactions. It didn't go well. That's the motivation behind both Spanner and F1: take the experience of how painful it is to do transactions on a Regional or Global level, and never make individual teams do it again. I see it a bit like "Don't roll your own crypto". Clearly some people are exempted from it, but you better be able to tell me why you get an exception. Disclosure: I work on Google Cloud and want you to pay for Spanner :).
- dietrichepp 10y agoCloud Spanner is currently a regional service, not a global service. So you would only lose availability for failures within the region.
- inopinatus 10y agoPeople used to make similar comparisons between the Russian and American space programs.
- jedberg 10y agoOh yeah? Which was which in that comparison? I'm not familiar with that.
- WorldMaker 10y agoPresumably a reference to the classic "Pencil myth": Americans when faced with a need to write things down spent millions of dollars inventing a low gravity ballpoint pen and Russians just used a pencil. As cutesy of a sentiment as it is, it's also full of misconceptions. The pens were invented by an American corporation that wanted better pens to sell in general (a smoother flow in a pen, regardless of gravity/orientation, is a better pen), and they saw a good opportunity to market the pen to NASA for use in space. Both NASA and the Russians used pencils in space, but the problems with pencils is the flakes can pollute an environment pretty quickly in low gravity and the pens turned out to be a much better solution. (So far as I've heard, every space agency these days buys similar pens.)
- inopinatus 10y agoYou presume wrong. Pens are not a space program. I meant the differences in design philosophy that permeate aerospace engineering on both sides. Russian, built ugly but for strength and longevity. American, built for high capability with finesse and finer tolerances. The emergent properties of these different principles explain why Soyuz is still a preferred launch vehicle, but it was the Americans who got to the moon and operated the STS. Amazon is more like the Russians: built in the knowledge that things fail, but less magical as a result. Google is more like the Americans: remarkable technology, you just need a herd of geniuses to run it.
- WorldMaker 10y agoSorry if you think I made the wrong presumption, but I've seen the "pencil myth" trotted out many times as a "definitive" example of precisely the dichotomy that you have spelled out. (A low gravity ballpoint pen is high capability that requires finesse and fine tolerances; a pencil is ugly but strong and typically exhibits longevity... and so forth.) It is an interesting analogy this dichotomy you see in the design philosophies (both between the space programs and the mega-corporations), but perhaps my point, if I were attempting a point, is to beware of false dichotomies.
- takeda 10y agoI'm not sure if what you said applies, they have severe restrictions and spanner offers subset of MySQL functionality which is already bare compared to other databases. Changes can be done by primary key only, so it almost feels like a KV store that can do joins... I don't think it's easy to port existing applications to use it and in the end you will still need to accommodate shortcomings in your application.
- jedberg 10y agoThat's a fair assessment. But I'm assuming they will make it do more "SQL" things in the future. I could be wrong though. Either way, they are trying to abstract away having to think about eventual consistency with this offering.
- u320 10y agoThe "old way" was sacrificing functionality such as transactions and joins to get scalability (BigTable, DynamoDB). Google tried that a decade ago and found it lacking, this is why Spanner exists in the first place.
- dietrichepp 10y agoThis isn't entirely accurate. BigTable was Google's earlier cloud database, and it's certainly non-traditional, and you have to build your application without traditional consistency guarantees, the way you describe. Spanner doesn't exactly hide the details, but it lets you make transactions that span multiple shards. You still eat the cost of the transaction, you're just free from having to implement it at the application level, which is a more difficult and error-prone way of doing things. The bottom line is that if you need consistency, it needs to be implemented somewhere in your stack. If you don't need consistency (analytics workloads come to mind) then you have more flexibility with your database. Disclosure: Google employee, reconstructing what I know from published information.
- q3k 10y agoSame for GAE, or now GKE (basically microservices before microservices were a thing) vs. EC2. GAE was pushing a fairly non-traditional architectural model while others were just trying to provide cloud-based VMs. Discolsure: Also a Google employee, also reconstructing.
- user5994461 10y agoAmazon: Create usual services and sell them. Google: Make unique products that push the boundaries of what was previously thought possible. Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain. Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business.
- jedberg 10y agoYeah that's all pretty accurate too. :)
- aanm1988 10y ago> Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business. Which Amazon totally didn't have to do with their firehose of cash?
- user5994461 10y agoGoogle has to support Google, Youtube and many of the most resources intensive services in existence on Earth. They needed to be "efficient enough" to operate that, meaning incredibly efficient. Amazon runs nothing, it's an outsourcing firm. They needed to make services "good enough" to be sold. If a service is somewhat inefficient, it just charges the clients more to cover the costs. Technologies reflect the business they were created in.
- dcosson 10y agoI would phrase it as: AWS prioritizes building blocks that support very high throughput and avoid leaky abstractions at all costs, and they're happy to push forward as long as these criteria are met. IMO they really succeed at this goal. Minus specific bugs that they're generally good about acknowledging, their services reliably do what they say they're going to. And they definitely solve a lot of problems for you, even if sometimes you're still required to get further into the weeds than you might want. I'll buy that Google Cloud is better at questioning underlying assumptions and sometimes succeeds in releasing higher-level abstractions than AWS without any leakiness (a great example of this now being Spanner vs Aurora). It also feels to me that with releases like this Google is leveraging the full value of their own experiences running their services, and seems to be more advanced than amazon in some areas so this has a lot of value, whereas AWS seems to build a broader range of products with a specific customer in mind which is not necessarily themselves (e.g. all of their move your on-prem stuff to the cloud helpers). If you consider Spanner vs Dynamo, it definitely matches up as Google wrapping the old way and Amazon forcing a new way (though to be fair, Dynamo was released 5 years earlier). But on the other hand considering Spanner vs Aurora, Amazon is the one embracing the old way with full MySQL & Postgres compatibility whereas Spanner sounds like a pretty dramatically different subset of SQL in not supporting insert and update statements. It's a very reasonable compromise for basically getting to ignore the CAP theorem, but it is a a significant difference that every developer will have to learn.
- cavisne 10y agoSpanner doesn't use SQL for writes it seems, so its still a significant rewrite for legacy applications (especially ones that dont use an ORM). The thing thats really different here is Google are basically saying, heres this awesome system, yes it has obvious risks from partitioning, we are going to stake our reputation on those partitions not happening. In contrast AWS are saying, this is DynamoDb, its really limiting but because of those limitations it should be pretty reliable as long as you write your application correctly. It will be interesting to see if Microsoft and Amazon have to follow Google's lead here.
- nickpsecurity 10y ago"I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction." No, they said this in the F1 RDBMS or Spanner papers. They originally did the NoSQL, eventual-consistency type of stuff. This had app developers required to do a lot of work to avoid problems that model can create. Apparently, even their bright people had enough problems with it that they decided to eliminate or simplify that situation with stronger consistency. Took some brilliant engineering but now they have a database easy to use as old model with advantages of newer ones. If anything, they learned some hard lessons with a good solution to them. Now, they're offering it to others. I was hoping they'd do this instead of keep it internal only. F1 and Spanner are amazing tech that could benefit many companies.