5 ms·
This is because neither AWS nor Azure use referential integrity in any of their "cloud scale" databases. For example, Azure uses some hideous JavaScript-based d
by jiggawatts 4y ago
This is because neither AWS nor Azure use referential integrity in any of their "cloud scale" databases. For example, Azure uses some hideous JavaScript-based document DB where things like renames, moves, and deletes are hit & miss at best. A never-ending whack-a-mole of bugs and issues.
Remember boys and girls: Being "cloud scale" means data corruption and referential integrity violation!
- orf 4y agoI don’t think it is - the issue is that the Batch service needs to assume your role in order to clean up associated cluster resources like auto scaling groups. If the role is deleted, it can’t do this.
- jiggawatts 4y agoThat's what referential integrity violation means: an essential related piece of information can be deleted without the parent/using object also being deleted.
- orf 4y agoSure, in an abstract sense maybe, but in a cross-service context it’s got nothing to do with their “cloud scale databases”. In fact, being able to delete a role without removing any associated resources is a feature, not a bug. And how would you even ensure referential integrity in this case - you would achieve the same effect by modifying the assume role policy but keeping the role around. You could craft a policy that only allows Batch to assume the role on a Tuesday for example.
- XorNot 4y agoWedging a cloud resource so you can't delete it is always a bug.
- orf 4y agoYes, in the service that provides the AWS resource. Because it didn’t handle the fact that the role it’s using might be deleted or otherwise rendered un-assumable for a variety of different reasons at any point in time. Which is a feature. Not a bug.
- TedDoesntTalk 4y agoI’m curious how you see this as a feature when it can get you into a very expensive and unresolvable situation; a AWS resource can’t be deleted and is running up costs. You’re at the mercy of AWS support.
- bergenty 4y agoIt’s not a feature but how do you solve this? You can’t block role deletion because it has a resource associated with it, that would be a functional nightmare.
- speedgoose 4y agoA very classic way of doing would be to prevent deleting a role that are resource tightly associated to it by default. If someone really wants to delete the role anyway, you can provide a feature to do that and delete associated ressources in cascade.
- TedDoesntTalk 4y ago"Cascading deletes"
- bergenty 4y agoIt’s like you didn’t even read what I wrote. That way sucks, especially if you have hundreds of allocated resources. Cascade delete is not a realistic option here either. Imagine someone leaves the company without notice. All their resources need to be deleted?
- efxhoy 4y agoNot even redshift enforces uniqueness on their primary keys or referential integrity on their foreign keys. That was a fun finding out…
- formerkrogemp 4y ago> That was a fun finding out… Not my idea of a good time, but whatever floats your boat. Hopefully you had a backup?
- thayne 4y agoThat was almost certainly sarcasm.
- deathanatos 4y agoNot only referential integrity, but also not supporting Read-Your-Writes[1]. Cloud scale! Nothing like, 1. Create resource. Success. 2. Attempt to use/reference first resource in another resource/call: Failure: Referenced resource does not exist. Odd. 3. Create resource again? Failure! Resource already exists. Our scripts have so many retry loops and arbitrary pauses mixed in to account for garbage like this. Distinguishing "did the call fail?" from "or is the system just lost in the land of eventual inconsistency?" ugh. And yeah. shout out to Azure AAD where I can have a role assignment that is granting permissions to "unknown". We call them ghosts. [1]: https://jepsen.io/consistency/models/read-your-writes https://jepsen.io/consistency/models/read-your-writes
- jiggawatts 4y ago> Our scripts have so many retry loops and arbitrary pauses mixed in This is unfortunately your fault, not Azure's. Their API is explicitly designed around this eventual consistency and weak references, so your client side must take this into account. Typical bash, Python, or PowerShell scripts are the Wrong approach with a capital W, and you will forever be tearing your hair out if you persist on using them. (Or any similar imperative deployment mechanism) The only robust method is ARM Templates, or better yet, Bicep templates[1]. The latter simply compile down to ARM, so they're essentially equivalent but terser and with nicer tab-complete. Compared to scripts, templates have key advantages: 1. Built-in incremental / differential deploy capability. A partially deployed template can be simply redeployed[2] to "fix it up", without requiring client-side logic for every corner case. 2. Can deploy multiple changes that would fail if deployed step-by-step. For example, App Gateway can have intermediate configurations that won't validate on the way to a valid final configuration. This is madness to unravel with scripts. Templates generally just take you to the final configuration in one step. 3. Inherently parallel. Anything that can be deployed concurrently will be. Anything. No need to write complex and error-prone parallel loops on the client side! 4. Largely immune to temporary failures like missing reads after writes.[3] The template engine has a built-in retry loop for most (all?) fallible steps. You'll see it has "failed"... and then "succeeded" anyway. [1] https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview https://docs.microsoft.com/en-us/azure/azure-resource-manage... [2] Most of the time. All resources should be idempotent to redeployment, but many aren't because this is not mechanically enforced. IMHO, this is just shoddy, shoddy engineering and everyone involved should be ashamed. Being nearly idempotent is like being nearly pregnant. [3] You still need a few tricks up your sleeve for robust deployments. Anything outside of ARM, such as Azure AD groups and RBAC tend to be a PITA. Generally you want to wrap your deployments in a script that takes the object GUID of the created group and feed that into the template. That works, because the GUID can be used even if the full object is not fully replicated around yet.