6 ms·
In my company, we were aware about the potential honeypots in each cloud and we developed our product from the first commit to be deployed on 3 (!) clouds: AWS,
by yonixw 5y ago
In my company, we were aware about the potential honeypots in each cloud and we developed our product from the first commit to be deployed on 3 (!) clouds: AWS, Azure, IBM.
And while we made it work by sticking to the least common denominator which was FaaS/IaaS (Lambda, S3, API GW, K8s). It was certainly not easy. We also ignored tools that could've helped us greatly only against a single cloud in order to be multi cloud.
The conclusion after 2 years for us is kind of not that exciting.
[1] AWS is the most mature one, Azure is best suited for Microsoft products and Old Enterprise features. And IBM is best if you use only K8s.
[2] Each cloud has a lot of unique closed code features that are amazing for certain use cases ( Such as Athena for S3 in AWS or Cloud Run in GCP). But relaying on them means you are trapped in that cloud. Looking back, Athena could have simplified our specific solution if we were only on AWS.
[3] Moving between clouds, given shared features, is possible, but is definitely not a couple clicks or couple of Jenkins jobs away. Moving between clouds is a full time job. Finding how to do that little VM thing you did in AWS, now in Azure, will take time and learning. And moving between AWS IAM and Azure AD permission? time, time and time.
[4] Could we have known which cloud is best for us before? No. Only after developing our product we know exactly which cloud would offer us the most suited set of features. Not to mention different credits and discount we got as a startup.
Hope this helps.
- tinco 5y agoWhy did you feel IBM is best when you only use K8S? Our platform fully K8S and I'm looking to somewhere where we get more performance per buck, probably not IBM but I'm surprised it's even in the list. Do they have an extra nice K8S dashboard or something?
- OliverGilan 5y agoSecond this question. Never heard of IBM being exceptional for k8s. Curious to know what makes OP say it’s so good.
- nijave 5y agoMaybe they're using Openshift (OKD). Openshift has a bunch of value add features they add a template marketplace and turn it into a hybrid PaaS with some other multitenancy management bits afaik
- jhickok 5y agoOKD is slightly different than the OpenShift (OCP) provided on IBM Cloud. OpenShift is on all clouds but IIRC OpenShift on IBM Cloud receives OCP updates first.
- yonixw 5y agoHi, I was in an IBM Startup Accelerator. I just got the feeling that they were pushing for it hardcore. Gave us startup credit, free training and free premium support for K8s. So If you want to go with K8s and you are a Startup, that is the best offering I experienced.
- nijave 5y agoNot sure where you're currently at and considering moving to, but GKE has been much simpler than EKS. Not sure about cost but it'll likely save some operations time (auto scaling is a single check box, no IAM, scaling controllers, etc)
- bavell 5y agoIME GKE on GCP is top-tier if you want a painless managed k8s offering. I've been running my business on it (solo founder) since ~2017 with minimal fuss. Hosting a couple dozen WP sites and some bespoke webapps/apis.
- mjb152 5y agoI'd throw in stability, IAM, storage and the management plane.
- milesward 5y agoCloud Run works on all three major clouds, and VMWare, and Bare Metal. No lock-in here.
- yonixw 5y agoDo you mean GCP Cloud Run [1] ? I would love to have it on AWS and Azure if you have a link to share. Or do you mean it's possible, but through different services on each cloud? [1] https://cloud.google.com/run https://cloud.google.com/run
- ctippett 5y agoI assume the OP is referring to Knative[0] which is the framework powering Cloud Run behind the scenes. [0] https://knative.dev/ https://knative.dev/
- milesward 5y agoNo, Knative Serving, via Anthos fleets.
- milesward 5y agoRuns in Anthos: https://cloud.google.com/anthos/run/docs/install https://cloud.google.com/anthos/run/docs/install
- nijave 5y agoAthena isn't fully closed source. It's a customized, hosted version of Presto originally built by Facebook. Apache Drill is in a similar space to Athena and can query unstructured or semi structured data like object/S3
- yonixw 5y agoYou are correct, But as I understand, They get "RAW" file access to the S3 hard disk (or the equivalent) in their solution. So no matter what solution I might spawn, it will always be slower and more expensive. Maybe I should have said "closed internal access features"
- bushbaba 5y agoDoubtful. I've personally had projects pull many hundreds of gigabits per second of s3 throughput. How you architect and design has a large influence to your analytics performance.
- dijit 5y ago“Many hundreds” Last time I talked to a technical person at AWS the limit was 5GBits. Wonder what you’re doing differently. Perhaps that changed.
- dastbe 5y agoyou were quoted a per-object rate.
- nijave 5y agoThere's another benchmark somewhere showing S3 can max out a 100Gbps instance. https://github.com/dvassallo/s3-benchmark https://github.com/dvassallo/s3-benchmark Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data
- wallfacer120 5y agoThank you for this very useful answer!
- gitgud 5y agoAlso cold start times for serverless differ greatly between those cloud providers. AWS is < 1 second, whereas Google Cloud is 5-10 seconds
- spyspy 5y agoThis is pretty misinformed. Each provider has multiple “serverless” offerings and cold start time has much to do with your specific application and what it’s doing on start up.
- topicseed 5y agoNone of my workloads (5x Cloud Run services, 10's of Functions) have anywhere near 5s cold starts. More like 2s with network latency.
- mshenfield 5y agoThanks for acknowledging how much harder this is when you use a cloud-specific feature. Modifying your codebase to migrate off some cloud specific service seems like it would be by far the hardest part of switching clouds.
- gregmac 5y agoWas it a business or technical decision to do multi-cloud? Did you run simultaneously in 3 clouds? Can you explain the setup? If not, did you do just run on each for a while to test, or have a reason to switch? This is probably an impossible question to answer, but: were the savings/benefits of doing this actually worth the engineering costs involved in the end? Eg even if you chose what turned out to be the most expensive, worst option, would the company ultimately have been in a better place by having engineering focused on building things to increase customer value instead?
- yonixw 5y ago> Was it a business or technical decision to do multi-cloud? > Did you run simultaneously in 3 clouds? Can you explain the setup? The solution itself could be running on a single cloud. But we work in the finance sector and targeting highly regulated clients. And we got a tip very early on, that each client could ask for deployment on their cloud account that is monitored by them. Which will probably be AWS or Azure. Today we know only some require that. So it helped somewhat. > were the savings/benefits of doing this actually worth the engineering costs involved in the end? Like you said, very hard to know. In our case, we had a DevOps Cloud guy working a full time job, so, it was not noticeable. Reason being, Probably, [1] Because although he had problems to solve on all clouds, clouds deployments eventually get stable enough, So pressure was spread. [2] Although all clouds still need constant maintenance, it's a-synchronic (you can't plan ahead when AWS EKS K8s will force a new version), so pressure was spread out and it never stopped client feature building. But who knows, maybe for other architectures or a bigger company, it would have become noticeable.