10 ms·
Dear Google Cloud: Your Deprecation Policy Is Killing You (2020)
- more_corn 3y agoDepreciation of Google products is baked into the company promotion process. They will always betray you. It is in their nature.
- jay_kyburz 3y agoSo great to hear somebody like Yegge voice this. I'm spending a lot of my weekend hours racing against the clock to keep some old app engine apps online. I wish I had started in 2020 :) I've learnt my lesson and will be sticking with boring technology I can take anywhere in future. Node and Postgres for me from now on.
- junon 3y agoThis is why, as a systems designer, I refuse to use "wrapper" services proved by cloud providers. If it doesn't have a shell where individual services can be run using my own choice of stack, I don't like it - for this exact reason.
- asah 3y agohow do y'all look at systems like AWS Aurora, GCP Spanner, etc which offer big speedups and cost savings, but lock you into the platform in practice? this isn't academic for me - I've got a service that's on vanilla RDS PostgreSQL but growing fast, and I'll soon need to decide whether to move to something cheaper/faster but proprietary or blow my budget trying to stay generic...
- zmgsabst 3y agoDrive-by response: I’d ask two questions — how possible is it to isolate the code which is proprietary? …is there a third option which allows not blowing your budget, but remaining generic? Naively, isolating the proprietary interface in a wrapper in your code minimizes changes should you need to move — and blowing your budget isn’t a long term solution. So does it come down to the proprietary option or fail?
- scarface_74 3y agoThis would only come from someone who has never done or managed a migration at scale. There are so many moving pieces doing any migration at scale, your code is the least of the problems. I have seen it take nearly a year to migrate hundreds of generic VMs with VM hosted databases from on prem to cloud. You can’t get more generic than that. Yes, this was with AWS Professional Services (where I use to work) doing much of the work.
- zmgsabst 3y agoYawn, insulting others experience to make your point is trite. Perhaps you could justify your point rather than attack other people and appeal to authority?
- scarface_74 3y agoThere is so much involved with any migration at scale - data migration, regression testing, network changes even where a company many have a physical connection to their cloud provider, security, compliance , training, and the list goes on.
- zmgsabst 3y agoOkay — and? How is any of that relevant to what I said? You’re ignoring that maintaining your code without modifying your data access (changing only the wrapper) significantly simplifies virtually everything you listed - with the exception of network topology changes.
- scarface_74 3y agoBecause in the grand scheme of things “your code” and even changing your code is just a minuscule part of what it takes if a migration did ever happen. So the question remains, have you ever been a part of a large scale migration? Have you ever been in the room when the planning was happening?
- scarface_74 3y agoPlease tell how Aurora - which is just MySql and Postgres with a better underlying storage engine - “locks you in”?
- jpdb 3y agoI agree with you, but I think the idea is that the underlying storage engine affords you functionality that you can't mimic yourself / elsewhere and therefore your capabilities to leave are limited. You're always "locked in" to some degree and it's an almost worthless thing to invest into if you never actually need to actually migrate off.
- athrun 3y agoYou just have to realise that given enough scale everything turns proprietary anyway. The difference being whether you are internally maintaining your special snowflake, or else paying for a solution maintained for you by a 3rd party.
- ravivyas 3y agoNote: Post is from 2020
- rwmj 3y agoThe only thing that has changed is more stuff has been deprecated. I got an email from Google only last week that something in Analytics is changing (don't care - I stopped using it ages ago when it broke because of some earlier deprecation).
- deleted 3y ago[deleted]
- gtirloni 3y agoJust a rant about having to react to things that the author didn't want to react to. But these are facts of life. I find GCP policies pretty reasonable. I was never caught off guard by them.
- zinekeller 3y ago> I find GCP policies pretty reasonable. I was never caught off guard by them. For some individuals, sure. But other people and especially businesses don't really want change for change's sake (and if possible want to have a more-or-less set-and-forget thing), and the article clearly explains why it is better with AWS if that's the goal.
- tight-ship 3y agoI think this is a good point. Maybe GCP is failing at communicating why things are changing. As a customer, I'd be fine with GCP doing things to keep their costs down so GCP pricing stays the same or decreases. But I understand this is hard to quantify outside GCP and requires a bit of faith (which I accept as inherent, since full transparency is impossible).
- gorjusborg 3y agoI am guessing that you either aren't using it for business, or if you are, you are very small. The author's take is brutal, but totally on-point. The cloud business is about asking businesses to use your system as a foundation. Google doesn't take that seriously enough.
- busterarm 3y agoWhen you have 30 engineering teams all using different parts of the platform and your Product team is crushing your whole org with their aggressive roadmap, you will no longer feel like GCP policies are reasonable. When everything takes 20% longer to ship than you planned because every other week some random team(s) is forced to do unplanned, immediate rewrites by a vendor, that vendor starts to look like a dick and also a huge liability to your business. Also Google's own documentation is full of grenades. Just yesterday I found in their GKE docs something they tell you is the default and you must use it...except it doesn't apply to clusters created a few versions ago, even if they're updated, because they didn't roll their change out to existing clusters -- just new ones created after their change. Also google's docs reflect their deprecation policy -- there is no document versioning. You can always and only see latest.
- garganzol 3y agoThis story resonates with me so much. An avid cloud platforms user in the past, now I just use Dockerfile + fly.io combo. Dockerfile gives you a total freedom without vendor lock-ins and forceful deprecations. I have never been calmer and happier before that switch. Now, only I decide when to upgrade - and usually it is as easy as editing a text file pointing to a new package version. IMHO, the only thing cloud providers are genuinely useful for is a bottomless data storage. But even that aspect is now isolated by Rclone in all my apps since migrating off the cloud. Apps just talk with the file system, which is then abstracted to a cloud storage of choice by Rclone.
- scarface_74 3y agoYes, I’m sure major corporations are going to trust their entire enterprise to fly.io and all they need is Docker and storage.
- VHRanger 3y agoI mean, you could still avoid vendor lockin by having proper build scripts and deploying directly to EC2 instances and their equivalent. The lockin happens when DevOps gets lazy and start using a bunch of cloudy functions.
- scarface_74 3y agoAnd now you’re going to spend more money on cloud just to treat it like an overpriced collocation center without any of the benefits of moving to it. Have you ever done a large migration at scale? I have - even moving over a lot of VM hosted services involves your project management organization, regression testing, networking, firewall, training, security, working with outside vendors, and the list goes on. The average enterprise uses 254 SaaS services. I have never in 5 years heard any CTO of any company of any size worry about “lock in” when considering their cloud choices. Source: 2 years as an architect responsible for integrating acquired companies before a private equity owned company went public, two years leading the “application modernization” effort at a startup that exited 10x revenue, 3 years working at AWS in the Professional Services department and now working at a smaller consulting shop. And I got Amazoned a couple of months ago. I have no particular loyalty toward Amazon
- unicornmama 3y agoWe noped-out or Google cloud because of this. It wasn’t hard to drive home the argument after they rug pulled Google Domains.
- atomicfiredoll 3y agoWhile my team and I were already getting progressively more skeptical of Google products for all the often retread reasons, the Google Domains issue in particular solidified my team's feelings as well into a single example as well. I know there are SLAs, that Domains wasn't necessarily targeted at the same audience, etc. But, I'm guessing lots of folks got burned by it and I don't know how to justify trusting Google at this point. Given their struggles with their reputation and cloud offering, that situation and how they handled it still blows my mind.
- adrr 3y agoThat was a poor decision. Every other cloud provider has domain support. Just left my CTO role at a public company to do a another startup, GCP wasnt even in the running. It was between Azure and AWS. Also the first company i didnt pick GSuite or whatever they are calling it now because they fail to innovate on it. Office365 has been beating it in feature launches like adding an AI copilot and AI generated meeting summaries. Spend less effort renaming products and more time building features. Google needs to fire their leadership. They will be the next IBM if they dont.
- zdragnar 3y agoI don't want innovation in GSuite. I want email and calendar and video calls and maybe the occasional docs. Of the companies I've worked for, half had GSuite and half had office. Personally, I'll take GSuite any and every time.
- znpy 3y agohad gsuite in a job, and office365 at another job. I'd be okay with office365 if teams wasn't such a shitty piece of software. slack so far has been the best corporate chat experience.
- roenxi 3y ago> Successful long-lived open systems owe their success to building decades-long micro-communities around extensions/plugins, also known as a marketplace. I’ve ranted about Platforms before, and how important they are, and how Google has never once in their entire corporate history ever really understood what goes into making a successful open Platform, not counting Android or Chrome. That one statement deserves its own essay in the context of Chrome and Firefox. It'll be interesting for anyone around in 30 years to look back and see how both those browsers are doing against the Emacs web browsers just to score the thinking.
- bradleyjg 3y agoGoogle engineers just don’t seem to value back compat in their bones. I’ve read they have tools and processes internally that let them make breaking changes and globally find / fix callers. That’s cool, but the rest of don’t have that. No one likes getting an email that says “we’ve changed the way X works, you have until Y to change your code or it will break.” Not even if Y is a year from now. A rule that says no breaks ever is probably not optimal either but it’s a big deal and should be treated as such. Maybe require a VP sign off or something. Also: keeping your product in beta for a decade is cute but unconvincing.
- renegade-otter 3y agoGoogle culture in general seems to be "this is not fun or a very profitable, so we are just going to, like, not do it anymore". Fundamentally it comes from the fact that they don't have to do any of it anyway. Anyone using AdWords knows that it's a money-printing machine. Google is thousands of fresh CS graduates who want to do "cool green-field stuff" but not really the day-to-day grind we are all part of and what we call "software engineering".
- UncleMeat 3y agoI think that there are competing cultures, and ultimately only one of them has the power to fire people. I'm currently losing a multi-year long siege to kill a project with significant external visibility almost entirely because of organizational structure. A series of reorgs and new VPs have ended up such that our VP doesn't give a crap about our project and the VP that does doesn't have a great relationship with our VP. My experience is that people really do become proud of the systems that they build and that deleting them is something they avoid. This isn't coming from junior engineers wanting to do green field stuff. This is coming from VPs who swoop in and say "what I care about is this metric and if you aren't pushing on this metric then you are fired." There are aggravating factors. Regulation sometimes means that core infrastructure needs to change out from under you, so the "it is stable and just needs a small amount of ongoing maintenance" doesn't always persist. I think that Google's monorepo also contributes to this. It has a lot of huge benefits but it also means that "keep the old version of library X around" isn't really possible.
- andrewstuart 3y agoGoogle is not interested.
- bloppe 3y agoI think python would've been in more trouble if it never made the transition to version 3. You probably shouldn't be building massive apps in Python in the first place.
- f33d5173 3y agoA major critique of the python transition is that they forced you to port your code instead of supporting python 2 code alongside python 3.
- morkalork 3y agoAs someone who only started using python in earnest after 3 got its legs, I appreciate that decision.
- D13Fd 3y agoYeah that's the boat I'm in. I started using Python for production projects after the switch. It sucks for those who came before, but it's just fine for those who came after, and I'm not sure it hurt the platform as much as this article suggests. Plus, they clearly learned their lesson after that switch, which is still talked about in 2023.
- crote 3y agoPython 3 was released in 2008. Python 2.7 was supported until 2020. They supported Python 2 alongside Python 3 for 12 years. They provided an automatic conversion tool. They provided libraries which made it possible to run the same code on both 2 & 3. The transition sucked, but they did everything they could to make it as smooth as possible.
- ufmace 3y agoMy impression is rather different. The automatic conversion tools and compatibility libraries were only provided several years and versions into the v3 transition because the community was demanding it so hard and most users refused to do anything with v3 without it. Most of it probably wouldn't have been necessary at all, or would have been there on day 1, if they had made backwards compatibility the priority instead of going hard on the everyone rewrite everything to the new v3 style right away. I'm sure they didn't want to support Python 2 for 12 years but were instead basically forced to because the transition was so difficult for so many major projects.
- fidotron 3y agoThe Android team did need to be told. There was a lot of internal dev pressure to abandon backwards compat (by which I actually mean reproducing bugs) even between immediate revisions. At one point their (at the time brilliant) dev relations baited me into telling them “if the targetSdk setting isn’t for this exact problem then what is it for?” and never heard of them thinking of making such changes again. The backstory was one product I had become responsible for was reliant on a bug in the inheritance implementation of the earlier VMs. I was stunned to say the least. That product happened to be FIFA which made my case somewhat easier.
- deleted 3y ago[deleted]
- Havoc 3y agoYep. Been navigating the firestore / datastore maze and even though it mostly works I still can’t quite tell how the bits relate to each other. There is firestore on fire base side. And then on gcp side also firestore. Except it’s called datastore there. And comes in native flavour and datastore mode. And then there is firestore compatibility with native mode datastore. But also datastore in datastore mode. And they all have different documentation pages and APIs and pricing and honestly wtf google
- vrosas 3y agoI can understand the confusion. A brief history lesson. Firebase: a paas that google bought that included a real-time nosql db (also casually called firebase) Datastore: googles OG serverless nosql db that was tightly baked into appengine as part of their own paas After buying firebase google decided the db implementation was shit and set out to write their own. In typical google fashion they picked the most confusing name and called it firestore. Rather than maintain 2 different nosql databases google quietly broke datastore free from appengine and migrated everyone’s data to firestore. But to maintain compatibility they kept both sets of APIs so now you have Firestore native (to get people to move off firebase) and firestore in datastore mode (for anyone who was using the og datastore). The 2 APIs still have specific features and access patterns hence the differences in billing. You can access and manage firebase both from the GCP console and the Firebase console but it’s the same db, same as how firebase storeage is really just GCS. So that adds another layer of confusion. Hope this helps.
- fnord77 3y agosame complaints with Java Spring
- jvolkman 3y agoI had this experience with AWS earlier this week. ======== Hello, This is a reminder to update your policies to avoid changes to your access to AWS Billing, Cost Management and Account consoles. Our records indicate that you are still using retired actions to access these consoles. If your policies are not updated with new actions by December 11, 2023, your users’ access to the AWS billing, Cost Management, and Account consoles will be affected. The policies that need to be updated to include the new fine-grained actions are listed in the “Affected Resources” tab of the AWS Health Dashboard in the “Policy | Policy Name | Policy ARN | Type” format. To help you with the migration, we have published a mapping between old and new actions in our user guide [1]. If you need to update policies across multiple member accounts in your organization, we have built bulk policy migration scripts to help you update all policies quickly and securely from your management account. See the bulk policy migration scripts user guide [2] for more information. You can find a detailed guide on how and which policies you need to update on our blog [3] and definitions of new IAM actions in Cost Management [4] and Billing [5] user guides. AWS will not be able to grant further exceptions after December 11, 2023, so we strongly recommend that you act promptly to migrate your policies to new actions. If you have more questions or need help make updates to your policies, please contact AWS Support [6]. Sincerely, Amazon Web Services ======== Not only did they send me this email, they sent me 10 copies - one for each account we have. Pages and pages of documentation, steps to follow, Cloud Formation stacks to create, Python scripts to run, etc. I'm still not completely sure what the problem was or what is supposedly improved, or if I would've actually lost access to anything, but hopefully the emails stop now.
- mh- 3y agoI've been getting these too, and it all feels very messy and out of character for AWS. If automatable changes are required.. pop up a dialog and ask me to confirm them?
- coredog64 3y agoThe automation all happens on your side of the control plane. AWS isn’t going to update IAM roles for any customer. The closest you get is that roles can consume managed policies, and those could change but you know that going in.
- dielll 3y agorecently Youtube shows you a pop up that says they banned adblockers and even suggest that you pay for Youtube Premium. However, in my country if you click the Youtube Premium link, it says YouTube Premium is not available in your country" I don't think Google takes their non ad products seriously. They only launch their products in few countries, the teams use them for promotion then abandon the products
- Joker_vD 3y agoWell, you better watch those ads then — or else.
- deleted 3y ago[deleted]
- biorach 3y agoYegge classic: > ...GNU Emacs, which is a sort of hybrid between Windows Notepad, a monolithic-kernel operating system, and the International Space Station
- anytime5704 3y agoIt’s disheartening to read all the comments agreeing with this post. I actually worked on GCP for several years and we always made sure our customer facing API changes were backwards compatible. The decision to deprecate or “sunset” a feature was not taken lightly and needed to be announced O(years) ahead of time. Apparently, my team was an exception rather than the norm.
- VladTheImpalor 3y agoIt's concerning to read your comment on this post, since it suggests you were unaware of all these issues people were facing with Google cloud. Was there not much cross talk or discussion of what issues the customers were experiencing with the platform as a whole?
- anytime5704 3y agoGCP is a gigantic organization and I was a cog in a machine. I was aware that people criticized GCP for various reasons, but there wasn’t much I could do beyond my team/product area.
- fenesiistvan 3y agoI don't agreee with the Android related tougths. Altrough they don't deprecate API's, permission related things are changed too often. If you have any non-trivial app, you have to make changes every year to remain in Google Play and the rules are awkard. My latest pain: I have a voip phone app in the store and last time I had to add a big popup before touching the contact list, informing the users the purpose (which is obvious with a phone app). Also the app description now contains that "this app might upload your phone numbers", altrough it only list it for the user, so they can select a contact and call it. There are many more idiot thing, but I stop here because I already start to become angry, thinkig about the many wasted hours at their stupidity.
- ravedave5 3y agoNo cloud provider is immune to this, Azure dropping support for MariaDB is going to cost us a lot of money in migrations. It's a pretty terrible migration at that.
- nunez 3y ago> I’m not actively developing on AWS, so I don’t have as much of a sense for how often they sunset APIs that they have previously dangled alluringly before unwitting developers. But I have a suspicion it’s nowhere near as often as happens at Google, and I believe wholeheartedly that this source of constant friction, and frustration, in GCP, is one of the biggest factors holding it back. AWS very rarely deprecates APIs. The last API they deprecated (that I can recall) was EC2-Classic, and that was after 15 years of service AND having a vastly better alternative (EC2 with default VPCs) that almost everyone was already using. In fact, AWS S3 is another example of a well-built, time-tested platform that passes the backwards compatibility test. AWS S3 spun up in 2006 or thereabouts. Even though the version of their most-current API schema is from *2012*, IIRC, you can still use code that you wrote back in 2006 to create stuff into buckets and store them. Same with EC2 VPCs and core networking. > Go ahead, I dare you. Follow the link and click the button. Choose “yes” to get all the default parameters and deploy the cluster to your Google Cloud project. Haha, joke’s on you; it doesn’t work. None of that shit works. It’s never tested, starts bit-rotting the minute they roll it out, and it wouldn’t surprise me if over half the click-to-deploy “solutions” (now we understand the air quotes) don’t work at all. It’s a completely embarrassing dark alley that you don’t want to wander down. Azure 100% does the same thing with their Marketplace AMIs, and I'm almost certain that AWS has a ton of quickstart guides that straight-up don't work. My guess is that a Solutions Architect or Presales Engineer wrote the solution for a small handful of customers to use, then published it for "impact" and completely forgot about it to move onto the next big problem to solve. (Guilty as charged.) However, if someone contacted Azure and AWS after running the broken quickstart asking for help, if they have enough money for support, they will absolutely get an SA to help them get up and running. Not sure about GCP. I suppose it's probably the same. > Larry & Sergey got rid of all the unhealthy snacks by 2006 though. I will always tell people about how when I worked there in Corp Eng in 2015 (worst job ever, unfortunately), all of the doors on their (many) fridges had large frosted glass films covering their bottom halves. The top half had "heatlhy" soft drinks and juices, along with seltzers and other low-calorie beverages. The bottom half? Good ol' fashioned American sodas...including diet, zero-calorie sodas! You could go absolutely crazy gorging yourself on their infinite frozen yogurt and snacks (they definitely 10000% had unhealthy snacks when I was there) twice/day, but good luck finding the Diet Dr Pepper, I guess?
- rldjbpin 3y agopersonal anecdote: used dialogflow (when it got rebranded from api.ai after getting acquired by Google) to create a chatbot in 2018. funny to track how it got placed inside Google's AI offerings since. i may have missed a few, but it got moved inside their GCP suite of offerings, and now lost somewhere in Vertex GenAI offerings probably. advancements happen and i understand that. but when your entire platform to achieve x changes drastically within a span of a couple years, how can we consider creating stable systems like of the past with XaaS model?
- anuraaga 3y agoDon't think I see much in terms of new content here so to add an anecdote about a deprecation post-dating the OP: My wife and I created an app using the GCP Maps Platform for Unity (or some name similar to it, it was definitely a GCP project) 2-3 years ago. It was for an exhibition, not a long term project though there were aspirations to maybe do something like that. But exhibition ended, and a year or so later the API was sunsetted. Fast forward to just last week, phones have changed since then, and she wanted to rebuild the app to demo it. I had to say nope, API's gone... But we're lucky to have not actually gone forward with trying to build a business on this despite there definitely must have been people that did and had to migrate to self hosting by force. Still I use GCP, the UX compared to AWS is just much better, both the APIs (i.e. Terraform) and console (no region!!!). I do know the typical Googler personality and suspect server features like Cloud Run are safe while anything too close to application like analytics is at risk and I'll generally avoid it. I do wonder where auth falls on this spectrum, I'd like to avoid auth0 etc since just too expensive, but it is a bit hard to trust after seeing a shutdown of an app feature.