4 ms·

A good friend told me that he felt that Google Cloud and AWS had "a severe lack of imagination". Imagine what you could do if you didn't even assume a process

by codemac 10y ago

A good friend told me that he felt that Google Cloud and AWS had "a severe lack of imagination". Imagine what you could do if you didn't even assume a process model? All app state just resident in memory, but magically persisted? Who needs object storage, re-invent the pointer! We could have lived in the future, now it seems we're permanently wed to the past.

mschuster91 10y ago
> Imagine what you could do if you didn't even assume a process model? All app state just resident in memory, but magically persisted? Who needs object storage, re-invent the pointer! Take your usual Java, NodeJS or Ruby payload, enjoy your memory leaks eating up your space.
count 10y ago
We're starting to get to the point where these giants can innovate like that. It wasn't 3 years ago that 'nobody serious' was 'trusting' cloud providers like AWS/GCE with anything important. This is still the very early days, as evidenced by the ridiculous growth numbers being posted YoY.
gtaylor 10y ago
> A good friend told me that he felt that Google Cloud and AWS had "a severe lack of imagination". I don't know about that. Google Container Engine (hosted Kubernetes) is actually pretty awesome and imaginative. It's feeling like GCP's niche is going to have a sizable containerization element. If you browse around their docs, you'll find that GKE/containers have started creeping into the examples for seemingly unrelated services. They're not just dipping a toe in. More generally, I feel like GCP's container strategy is just leagues ahead of AWS' at this point. While this article was thin on substance, ECS is definitely difficult to set up and maintain. If I'm going to go through all of that trouble, I might as well run my own Kubernetes or Mesos setup and not be locked into ECS.
- _asummers 10y ago
  I know you can deploy Kubernetes on AWS, though I have not tried myself. What, if you have tried it, is it lacking from the GCP version?
  - gtaylor 10y ago
    It's not that it's lacking at all, but on GCP I just click a few buttons and Google spins a Kubernetes cluster up for me. They handle my node OS upgrades for me. They've also got a really slick in-place rolling cluster upgrade setup. The primary difference on AWS is that you are doing all of this yourself instead. This isn't bad for everyone, but I've got the option of having Google do it in addition to doing it myself on GCP. As a bonus, Kubernetes abstracts a lot of inter-provider differences away. I can move between providers more easily, or even run across multiple providers (cluster federation). ECS is severely limited and slow to evolve, by comparison. They got to market first, but their overall container strategy is now inferior to GCP's. They're still ahead in many other ways, though. I just feel like containers are GCP's sweet spot.
    - boulos 10y ago
      Kubernetes was out well before ECS. And technically we launched GKE in public Alpha before ECS (which was a week or so later at re:Invent). If you mean before we called it a GA service, then you're right (but the team wanted it to be based on what they considered Kubernetes 1.0).
  - lobster_johnson 10y ago
    Google has been unusually/surprisingly environment-agnostic when it comes to Kubernetes, and it has ample support for AWS: It will read EC2 metadata (instace tags, AZ placement), it will automatically provision route tables and ELBs, and can attach EBS volumes. As such, it integrates almost as well with AWS as with GCP. It's telling that Google has lots of experience with containers, because GCE is a smoother environment to deploy Kubernetes in. Perhaps the most visible is the one-IP-per-pod/one-IP-per-service design requirements of Kubernetes, which fits directly into GCE's super-flexible IP address space (an amazing, modern L3 SDN), whereas on AWS you have to work around AWS's creaky, old-hat VPC system and its positively stone-age security group system. As a result, a lot of people are resorting to overlay solutions like Flannel, Weave and Calico (overlays on top of overlays!). GCE's networking, in short, is amazing, and AWS's isn't. GKe's automatic cluster setup is also impressive. GKE will create a master for you and you just tell it how many nodes you want. Disappointingly, the Kubernetes dashboard is not integrated into the GCE admin, but it's likely that they're working on a new one for that. Because of the lack of a UI, though, GKE feels a bit unfinished right now. That said, it's obvious from even a casual glance that GCP as a whole is a lot less mature than AWS. The SDK situation is a mess compared to AWS. Google's complicated style of authentication is particularly onerous on developers. Google's services feel less nailed down than AWS; for example, CloudSQL exists as two (incompatible, afaik) versions, and there are multiple APIs that are marked as beta-quality (Google is slowly adopting gRPC throughout their system as an alternative to REST APIs). Some of their services offer odd design decisions; for example, Pub/Sub is clearly a clone of SQS, down to its more annoying features (no proper fanout support, and plenty of latency), but it also innovates on it by supporting "realtime" delivery... which for some reason only supports push events. Go figure. Overall, Google Cloud Platform feels a bit half-hearted, unpolished and ramshackle in many places, even though clearly there's a lot that is technically solid. This impression is partly due to the messiness of the documentation. As with Kubernetes itself, it seems Google likes to developer faster than they can document. I do think that GCP will, in time, be much better than AWS. They are clearly innovating in many ways that Amazon isn't, especially in containerization.
deleted 10y ago
[deleted]
nikanj 10y ago
I heartily recommend this essay http://scholar.harvard.edu/files/mickens/files/thenightwatch.pdf http://scholar.harvard.edu/files/mickens/files/thenightwatch... , for gems like "Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis."
- codemac 10y ago
  I love James Mickens with all my heart. All of it.
jacques_chester 10y ago
> Imagine what you could do if you didn't even assume a process model? We have that world, it's called single-process apps. And it's awful from the point of view of security, scalability and disaster recovery. > All app state just resident in memory, but magically persisted? You need transactions or this ends unhappily. Some languages truly grok transactional updates to state. Most do not. In the meantime, you've rate limited the entire system to the slowest component. > We could have lived in the future, now it seems we're permanently wed to the past. Your friend overlooks that extremely intelligent people have looked into these things. They usually had extremely important disadvantages, which sufficiently offset the advantages that mere momentum kept the majority approaches as the majority approaches. Your friend also seems to have left it as an exercise for the reader on how one is meant to deal with distributed systems. The short answer is: it is hard, and trying to create the seamless illusion that the network doesn't exist hasn't really panned out. The speed of light is a cruel limit.

by codemac 10y ago

mschuster91 10y ago

> Imagine what you could do if you didn't even assume a process model? All app state just resident in memory, but magically persisted? Who needs object storage, re-invent the pointer! Take your usual Java, NodeJS or Ruby payload, enjoy your memory leaks eating up your space.

count 10y ago

We're starting to get to the point where these giants can innovate like that. It wasn't 3 years ago that 'nobody serious' was 'trusting' cloud providers like AWS/GCE with anything important. This is still the very early days, as evidenced by the ridiculous growth numbers being posted YoY.

gtaylor 10y ago

> A good friend told me that he felt that Google Cloud and AWS had "a severe lack of imagination". I don't know about that. Google Container Engine (hosted Kubernetes) is actually pretty awesome and imaginative. It's feeling like GCP's niche is going to have a sizable containerization element. If you browse around their docs, you'll find that GKE/containers have started creeping into the examples for seemingly unrelated services. They're not just dipping a toe in. More generally, I feel like GCP's container strategy is just leagues ahead of AWS' at this point. While this article was thin on substance, ECS is definitely difficult to set up and maintain. If I'm going to go through all of that trouble, I might as well run my own Kubernetes or Mesos setup and not be locked into ECS.

_asummers 10y ago
I know you can deploy Kubernetes on AWS, though I have not tried myself. What, if you have tried it, is it lacking from the GCP version?
- gtaylor 10y ago
  It's not that it's lacking at all, but on GCP I just click a few buttons and Google spins a Kubernetes cluster up for me. They handle my node OS upgrades for me. They've also got a really slick in-place rolling cluster upgrade setup. The primary difference on AWS is that you are doing all of this yourself instead. This isn't bad for everyone, but I've got the option of having Google do it in addition to doing it myself on GCP. As a bonus, Kubernetes abstracts a lot of inter-provider differences away. I can move between providers more easily, or even run across multiple providers (cluster federation). ECS is severely limited and slow to evolve, by comparison. They got to market first, but their overall container strategy is now inferior to GCP's. They're still ahead in many other ways, though. I just feel like containers are GCP's sweet spot.
  - boulos 10y ago
    Kubernetes was out well before ECS. And technically we launched GKE in public Alpha before ECS (which was a week or so later at re:Invent). If you mean before we called it a GA service, then you're right (but the team wanted it to be based on what they considered Kubernetes 1.0).
- lobster_johnson 10y ago
  Google has been unusually/surprisingly environment-agnostic when it comes to Kubernetes, and it has ample support for AWS: It will read EC2 metadata (instace tags, AZ placement), it will automatically provision route tables and ELBs, and can attach EBS volumes. As such, it integrates almost as well with AWS as with GCP. It's telling that Google has lots of experience with containers, because GCE is a smoother environment to deploy Kubernetes in. Perhaps the most visible is the one-IP-per-pod/one-IP-per-service design requirements of Kubernetes, which fits directly into GCE's super-flexible IP address space (an amazing, modern L3 SDN), whereas on AWS you have to work around AWS's creaky, old-hat VPC system and its positively stone-age security group system. As a result, a lot of people are resorting to overlay solutions like Flannel, Weave and Calico (overlays on top of overlays!). GCE's networking, in short, is amazing, and AWS's isn't. GKe's automatic cluster setup is also impressive. GKE will create a master for you and you just tell it how many nodes you want. Disappointingly, the Kubernetes dashboard is not integrated into the GCE admin, but it's likely that they're working on a new one for that. Because of the lack of a UI, though, GKE feels a bit unfinished right now. That said, it's obvious from even a casual glance that GCP as a whole is a lot less mature than AWS. The SDK situation is a mess compared to AWS. Google's complicated style of authentication is particularly onerous on developers. Google's services feel less nailed down than AWS; for example, CloudSQL exists as two (incompatible, afaik) versions, and there are multiple APIs that are marked as beta-quality (Google is slowly adopting gRPC throughout their system as an alternative to REST APIs). Some of their services offer odd design decisions; for example, Pub/Sub is clearly a clone of SQS, down to its more annoying features (no proper fanout support, and plenty of latency), but it also innovates on it by supporting "realtime" delivery... which for some reason only supports push events. Go figure. Overall, Google Cloud Platform feels a bit half-hearted, unpolished and ramshackle in many places, even though clearly there's a lot that is technically solid. This impression is partly due to the messiness of the documentation. As with Kubernetes itself, it seems Google likes to developer faster than they can document. I do think that GCP will, in time, be much better than AWS. They are clearly innovating in many ways that Amazon isn't, especially in containerization.

deleted 10y ago

[deleted]

nikanj 10y ago

I heartily recommend this essay http://scholar.harvard.edu/files/mickens/files/thenightwatch.pdf http://scholar.harvard.edu/files/mickens/files/thenightwatch... , for gems like "Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis."

codemac 10y ago
I love James Mickens with all my heart. All of it.

jacques_chester 10y ago

> Imagine what you could do if you didn't even assume a process model? We have that world, it's called single-process apps. And it's awful from the point of view of security, scalability and disaster recovery. > All app state just resident in memory, but magically persisted? You need transactions or this ends unhappily. Some languages truly grok transactional updates to state. Most do not. In the meantime, you've rate limited the entire system to the slowest component. > We could have lived in the future, now it seems we're permanently wed to the past. Your friend overlooks that extremely intelligent people have looked into these things. They usually had extremely important disadvantages, which sufficiently offset the advantages that mere momentum kept the majority approaches as the majority approaches. Your friend also seems to have left it as an exercise for the reader on how one is meant to deal with distributed systems. The short answer is: it is hard, and trying to create the seamless illusion that the network doesn't exist hasn't really panned out. The speed of light is a cruel limit.