5 ms·

Policy Engines: Open Policy Agent vs. AWS Cedar vs. Google Zanzibar

78 points by gemanor 3y ago

jreynoldsdev 3y ago
What I still struggle to understand with these systems is they seem great for single resource authorization, but how do you perform bulk queries? For example, a user wants to query all blogs they have access to (assuming there are large amounts of them), does that require separate authorization logic in the DB?
- random3 3y ago
  From short scans of the papers, at least with Zanzibar, AFAIK you can define entities and relations (think groups of users and directories) and infer rights based on those. I'm assuming Zanzibar backs the actual Goolge 360 document sharing so presumably it would scale for that use-case.
  - RandomBK 3y ago
    The google paper refers to the existence of some 'permissions-aware index' (paraphrasing) that's used to answer range queries like this, but doesn't cover how this index would work. I know various Zanzibar implementations have exposed APIs to solve this problem, but I still don't have a great intuitive understanding of how they work beyond 'push the ACL logic into the data layer', which brings us back to a pre-zanzibar world.
- jen20 3y ago
  Zanzibar in particular is designed to be able to answer the question "what can this user access?", or "who can access resource X?" as well as "can user Y access resource Y?". This article from OSO [1] explains how, with references to tweets from Lea Kissner (one of the authors of the paper and implementors) which are unfortunately less useful now that Twitter threads have been vandalised. [1]: https://www.osohq.com/post/zanzibar https://www.osohq.com/post/zanzibar
- bfeynman 3y ago
  What do you mean separate authorization logic? There are many layers to auth and usually they act as interceptors in request that go very fast. If you have blanket permissions to list, you are able to list resources you have access to... that's trivial. However `Blog` resources might have explicit deny policies on them as well, so yes those are also evaluated. Not sure how else you'd expect it to work sans caching like current state of resources and access.
  - ahoka 3y ago
    Yes, you need to consider authorization at every layer. You can blanket deny a lot of things in a midlayer, but sooner or later you need to start interpreting business logic to do the rest.
- turtles3 3y ago
  This, especially when you combine it with pagination, filtering and ordering requirements. Zanzibar implementations (eg. Spicedb, keto etc) offer functionality for listing resources accessible to a given principal, but as far as I can see none have a coherent solution for filtering and ordering. The only solution I can see to this is as you suggest maintaining a shadow copy of the relationships in your db so you can answer the question with a regular SQL query. This obviously comes with a lot of headaches, and is the sole factor preventing us from adopting one of these systems, so I really hope I'm wrong about this.
- gemanor 3y ago
  Data filtering implementation has different approaches among the mentioned policy engine. For OPA, custom Rego code could return the allowed data, and the caching mechanism will ensure its consistency and reliability. For Zanzibar, since the policy derived from the data relations, data filtering is using is an internal part of the paper. I recommend the following article for more information about policy as code and policy as data to understand the context better - https://www.permit.io/blog/zanzibar-vs-opa https://www.permit.io/blog/zanzibar-vs-opa
- jzelinskie 3y ago
  Full disclosure: I'm a maintainer of SpiceDB, the most mature open source project inspired by Zanzibar For this exact use case, SpiceDB created two APIs not available in Zanzibar: LookupSubjects and LookupResources. For other scenarios, there's also a BulkCheck API to performing many checks with less request overhead. The sibling comment here is correct that there isn't filtering/sorting available in SpiceDB yet. Additionally, there are folks using SpiceDB today by replicating denormalized checks back into their database (e.g. Postgres) or search index (e.g. Elastic) so that you can filter them natively. This is the combination of the aforementioned Lookup APIs with our Watch API. While this strategy requires moving parts, it is necessary beyond a particular scale which is well beyond the point at which policy engines typically fall over. While I'm biased, I do find this article somewhat misleading when describing Zanzibar-inspired systems; it presents opinion without any evidence or examples to justify the claim and concludes it as fact, but that might be because they're leaning on their previous article. Zanzibar is novel because it is fundamentally designed to be ran at the edge and solves the difficult problem of keeping the view of data at the edge consistent. This article conveniently leaves out how other systems get data to the edge while still keeping it consistent for their authorization logic. Latency is also brought up, but we recently managed to scale SpiceDB to >1M requests per second with 100B relationships while maintaining a 5ms p95 measured at the client application[0]. The claim that you absolutely need a service to run a Zanzibar system is a provably false claim based on the number of clusters in the wild running SpiceDB or Ory's Keto project. [0]: https://authzed.com/blog/google-scale-authorization https://authzed.com/blog/google-scale-authorization
  - orweis 3y ago
    Jimmy I truly think you're awesome (And so is SpiceDB), but the irony here stands out: "it presents opinion without any evidence or examples to justify the claim and concludes it as fact" You mean stuff like: 1) "SpiceDB, the most mature open source project inspired by Zanzibar" (though I'd vouch for that one) 2) " it is necessary beyond a particular scale which is well beyond the point at which policy engines typically fall over." 3) "Zanzibar is novel because it is fundamentally designed to be ran at the edge" 4) "we recently managed to scale SpiceDB to >1M requests per second with 100B relationships while maintaining a 5ms p95 measured at the client application" - you should bundle that statement with you need to set it up within your own VPC for it to be fair. 5) "The claim that you absolutely need a service to run a Zanzibar system is a provably false claim based on the number of clusters in the wild running SpiceDB or Ory's Keto project" - how many clusters? :) Re: "This article conveniently leaves out how other systems get data to the edge while still keeping it consistent for their authorization logic" The article actually does mention OPAL [0] [0]: https://www.permit.io/blog/introduction-to-opal https://www.permit.io/blog/introduction-to-opal
orweis 3y ago
Founder of Permit.io here- cool that this article grabbed some love. For those of you not sure which is the best from the article- Permit combines all 3 together. - OPA/REGO or Cedar at the edge, for quick efficient and zero latency policies - And Zanzibar at the cloud control plane to manage the overall picture and relationships
tptacek 3y ago
I thought this was a pretty weak writeup. I'm somewhat familiar with Zanzibar and less familiar with OPA or Cedar, and the coverage of Zanzibar was odd and superficial. Zanzibar is a large-scale distributed system whose motivation is handling intricately related sets of ACLs while avoiding vulnerabilities that come from disrespecting causal ordering. One big "con" to using it is that it's an internal Google system! If you're going to compare it to open-source policy engines, the sane thing to do would be to pick one of the open-source Zanzibar-inspired systems, and compare that.
- orweis 3y ago
  In Zanziabr - The article refers to OSS implementations like SpiceDB or Ory. It's a follow-up to a more in depth article (1), trying to be a lighter read starting point. - 1: https://www.permit.io/blog/zanzibar-vs-opa https://www.permit.io/blog/zanzibar-vs-opa
  - tptacek 3y ago
    It refers to Zanzibar as a "graphical" system, which I think was the first thing that snagged me on this. Your post does too; I assume this is a language snag? "Graphical" doesn't connote "graph-based" in American idiom, but rather "visual". I don't think your writeup really captures OPA vs. Zanzibar especially well either, for the reasons given by the SpiceDB person upthread. It just sort of defines away the problem Zanzibar is trying to solve, while claiming that Zanzibar-type systems aren't deployable at the edge --- which is pretty clearly not true?
    - orweis 3y ago
      Re: "Graphical" - I can see how that would have that effect :) To be fair it doesn't really say that, it reads:"Graph-based authorization systems utilize a graphical representation to illustrate relationships between users and resources" Still, I think Daniel (post author) could have picked better phrasing - I'll ask him to change it. > "while claiming that Zanzibar-type systems aren't deployable at the edge" For most companies it's extremely impractical; and for a developer (Audience of this article) that simply wants to add performant permissions to their without embarking on a whole devops adventure it's as good as so.
mdaniel 3y ago
Since the team seems to be hanging out in the comments: it appears the article was written out of order and "ReBAC" was used without the parenthetical definition in the OPA part but only defined down in the "Advantages of Google Zanzibar" section. I did what I usually do on an unusual acronym and "right-click, search" but .. that went poorly. The actual definition of "Relationship-based Access Control (ReBAC)" did show up about 4 results in, so fine, but it would be cheaper for the reader if you were to pull that upward to "define on first use" I would pay $1 if it was clarified about how that differs from RBAC or ABAC but maybe I just haven't gone through enough blogs to already know
- orweis 3y ago
  Hi! Fair point. We got two articles coming this month: RBAC vs ReBAC, and RBAC vs ReBAC vs ABAC - we'll post those here / in the article itself when ready. For now, in short: RBAC (Role based) is a simple identity to role to permission mapping. ABAC (Attribute based) maps conditions on attributes to to permissions (technically can implement anything - mostly used for things like time based, quotas, location, etc.) ReBAC (relationship based) maps relations between identities and resources to permissions (e.g. if a user is related as an owner to a folder, and the folder contains a file, the user is the owner of the file) - commonly used for resource and organization hierarchies
academia_hack 3y ago
It's a pity the state of the art here is still so dire for basic use cases. Things like managing group access to files in directories, or giving people field-level permissions to specific columns on specific database records. All of these languages (OPA, Cedar) are nifty but assume you have a whole team of devops engineers dedicated to replicating slices of your database state and getting them to the policy engine at the right time without discrepancies. Likewise, systems like Permit.io, OPAL, Topaz and SpiceDB claim to solve this problem, but they do it by adding an extra API call to everything your application does and then potentially having a permissions database which is in a divergent state from your real application. I was really hopeful about Oso for this since it is a policy engine that operates over the applications data as a library (rather than a service) but the development on the actual library appears basically abandoned in favor of their SaaS and it's currently full of weird quirks, bugs and outdated dependencies - especially around the filtering tasks (e.g. show me all the blog posts a user is allowed to read). I'd happily pay for a Oso library that worked, but it seems like they've hyper-focused on SaaS. Really would love to see one of these languages solve the data access layer for regular applications, everyone immediately seems to run to kubernetes at Google scale when I'd kill for a nice way to write permissions that can sit on the same server as my application and do the job reasonably well.

78 points by gemanor 3y ago

What I still struggle to understand with these systems is they seem great for single resource authorization, but how do you perform bulk queries? For example, a user wants to query all blogs they have access to (assuming there are large amounts of them), does that require separate authorization logic in the DB?

random3 3y ago
From short scans of the papers, at least with Zanzibar, AFAIK you can define entities and relations (think groups of users and directories) and infer rights based on those. I'm assuming Zanzibar backs the actual Goolge 360 document sharing so presumably it would scale for that use-case.
- RandomBK 3y ago
  The google paper refers to the existence of some 'permissions-aware index' (paraphrasing) that's used to answer range queries like this, but doesn't cover how this index would work. I know various Zanzibar implementations have exposed APIs to solve this problem, but I still don't have a great intuitive understanding of how they work beyond 'push the ACL logic into the data layer', which brings us back to a pre-zanzibar world.
jen20 3y ago
Zanzibar in particular is designed to be able to answer the question "what can this user access?", or "who can access resource X?" as well as "can user Y access resource Y?". This article from OSO [1] explains how, with references to tweets from Lea Kissner (one of the authors of the paper and implementors) which are unfortunately less useful now that Twitter threads have been vandalised. [1]: https://www.osohq.com/post/zanzibar https://www.osohq.com/post/zanzibar
bfeynman 3y ago
What do you mean separate authorization logic? There are many layers to auth and usually they act as interceptors in request that go very fast. If you have blanket permissions to list, you are able to list resources you have access to... that's trivial. However `Blog` resources might have explicit deny policies on them as well, so yes those are also evaluated. Not sure how else you'd expect it to work sans caching like current state of resources and access.
- ahoka 3y ago
  Yes, you need to consider authorization at every layer. You can blanket deny a lot of things in a midlayer, but sooner or later you need to start interpreting business logic to do the rest.
turtles3 3y ago
This, especially when you combine it with pagination, filtering and ordering requirements. Zanzibar implementations (eg. Spicedb, keto etc) offer functionality for listing resources accessible to a given principal, but as far as I can see none have a coherent solution for filtering and ordering. The only solution I can see to this is as you suggest maintaining a shadow copy of the relationships in your db so you can answer the question with a regular SQL query. This obviously comes with a lot of headaches, and is the sole factor preventing us from adopting one of these systems, so I really hope I'm wrong about this.
gemanor 3y ago
Data filtering implementation has different approaches among the mentioned policy engine. For OPA, custom Rego code could return the allowed data, and the caching mechanism will ensure its consistency and reliability. For Zanzibar, since the policy derived from the data relations, data filtering is using is an internal part of the paper. I recommend the following article for more information about policy as code and policy as data to understand the context better - https://www.permit.io/blog/zanzibar-vs-opa https://www.permit.io/blog/zanzibar-vs-opa
jzelinskie 3y ago
Full disclosure: I'm a maintainer of SpiceDB, the most mature open source project inspired by Zanzibar For this exact use case, SpiceDB created two APIs not available in Zanzibar: LookupSubjects and LookupResources. For other scenarios, there's also a BulkCheck API to performing many checks with less request overhead. The sibling comment here is correct that there isn't filtering/sorting available in SpiceDB yet. Additionally, there are folks using SpiceDB today by replicating denormalized checks back into their database (e.g. Postgres) or search index (e.g. Elastic) so that you can filter them natively. This is the combination of the aforementioned Lookup APIs with our Watch API. While this strategy requires moving parts, it is necessary beyond a particular scale which is well beyond the point at which policy engines typically fall over. While I'm biased, I do find this article somewhat misleading when describing Zanzibar-inspired systems; it presents opinion without any evidence or examples to justify the claim and concludes it as fact, but that might be because they're leaning on their previous article. Zanzibar is novel because it is fundamentally designed to be ran at the edge and solves the difficult problem of keeping the view of data at the edge consistent. This article conveniently leaves out how other systems get data to the edge while still keeping it consistent for their authorization logic. Latency is also brought up, but we recently managed to scale SpiceDB to >1M requests per second with 100B relationships while maintaining a 5ms p95 measured at the client application[0]. The claim that you absolutely need a service to run a Zanzibar system is a provably false claim based on the number of clusters in the wild running SpiceDB or Ory's Keto project. [0]: https://authzed.com/blog/google-scale-authorization https://authzed.com/blog/google-scale-authorization
- orweis 3y ago
  Jimmy I truly think you're awesome (And so is SpiceDB), but the irony here stands out: "it presents opinion without any evidence or examples to justify the claim and concludes it as fact" You mean stuff like: 1) "SpiceDB, the most mature open source project inspired by Zanzibar" (though I'd vouch for that one) 2) " it is necessary beyond a particular scale which is well beyond the point at which policy engines typically fall over." 3) "Zanzibar is novel because it is fundamentally designed to be ran at the edge" 4) "we recently managed to scale SpiceDB to >1M requests per second with 100B relationships while maintaining a 5ms p95 measured at the client application" - you should bundle that statement with you need to set it up within your own VPC for it to be fair. 5) "The claim that you absolutely need a service to run a Zanzibar system is a provably false claim based on the number of clusters in the wild running SpiceDB or Ory's Keto project" - how many clusters? :) Re: "This article conveniently leaves out how other systems get data to the edge while still keeping it consistent for their authorization logic" The article actually does mention OPAL [0] [0]: https://www.permit.io/blog/introduction-to-opal https://www.permit.io/blog/introduction-to-opal

orweis 3y ago

Founder of Permit.io here- cool that this article grabbed some love. For those of you not sure which is the best from the article- Permit combines all 3 together. - OPA/REGO or Cedar at the edge, for quick efficient and zero latency policies - And Zanzibar at the cloud control plane to manage the overall picture and relationships

tptacek 3y ago

I thought this was a pretty weak writeup. I'm somewhat familiar with Zanzibar and less familiar with OPA or Cedar, and the coverage of Zanzibar was odd and superficial. Zanzibar is a large-scale distributed system whose motivation is handling intricately related sets of ACLs while avoiding vulnerabilities that come from disrespecting causal ordering. One big "con" to using it is that it's an internal Google system! If you're going to compare it to open-source policy engines, the sane thing to do would be to pick one of the open-source Zanzibar-inspired systems, and compare that.

orweis 3y ago
In Zanziabr - The article refers to OSS implementations like SpiceDB or Ory. It's a follow-up to a more in depth article (1), trying to be a lighter read starting point. - 1: https://www.permit.io/blog/zanzibar-vs-opa https://www.permit.io/blog/zanzibar-vs-opa
- tptacek 3y ago
  It refers to Zanzibar as a "graphical" system, which I think was the first thing that snagged me on this. Your post does too; I assume this is a language snag? "Graphical" doesn't connote "graph-based" in American idiom, but rather "visual". I don't think your writeup really captures OPA vs. Zanzibar especially well either, for the reasons given by the SpiceDB person upthread. It just sort of defines away the problem Zanzibar is trying to solve, while claiming that Zanzibar-type systems aren't deployable at the edge --- which is pretty clearly not true?
  - orweis 3y ago
    Re: "Graphical" - I can see how that would have that effect :) To be fair it doesn't really say that, it reads:"Graph-based authorization systems utilize a graphical representation to illustrate relationships between users and resources" Still, I think Daniel (post author) could have picked better phrasing - I'll ask him to change it. > "while claiming that Zanzibar-type systems aren't deployable at the edge" For most companies it's extremely impractical; and for a developer (Audience of this article) that simply wants to add performant permissions to their without embarking on a whole devops adventure it's as good as so.

mdaniel 3y ago

Since the team seems to be hanging out in the comments: it appears the article was written out of order and "ReBAC" was used without the parenthetical definition in the OPA part but only defined down in the "Advantages of Google Zanzibar" section. I did what I usually do on an unusual acronym and "right-click, search" but .. that went poorly. The actual definition of "Relationship-based Access Control (ReBAC)" did show up about 4 results in, so fine, but it would be cheaper for the reader if you were to pull that upward to "define on first use" I would pay $1 if it was clarified about how that differs from RBAC or ABAC but maybe I just haven't gone through enough blogs to already know

orweis 3y ago
Hi! Fair point. We got two articles coming this month: RBAC vs ReBAC, and RBAC vs ReBAC vs ABAC - we'll post those here / in the article itself when ready. For now, in short: RBAC (Role based) is a simple identity to role to permission mapping. ABAC (Attribute based) maps conditions on attributes to to permissions (technically can implement anything - mostly used for things like time based, quotas, location, etc.) ReBAC (relationship based) maps relations between identities and resources to permissions (e.g. if a user is related as an owner to a folder, and the folder contains a file, the user is the owner of the file) - commonly used for resource and organization hierarchies

academia_hack 3y ago

It's a pity the state of the art here is still so dire for basic use cases. Things like managing group access to files in directories, or giving people field-level permissions to specific columns on specific database records. All of these languages (OPA, Cedar) are nifty but assume you have a whole team of devops engineers dedicated to replicating slices of your database state and getting them to the policy engine at the right time without discrepancies. Likewise, systems like Permit.io, OPAL, Topaz and SpiceDB claim to solve this problem, but they do it by adding an extra API call to everything your application does and then potentially having a permissions database which is in a divergent state from your real application. I was really hopeful about Oso for this since it is a policy engine that operates over the applications data as a library (rather than a service) but the development on the actual library appears basically abandoned in favor of their SaaS and it's currently full of weird quirks, bugs and outdated dependencies - especially around the filtering tasks (e.g. show me all the blog posts a user is allowed to read). I'd happily pay for a Oso library that worked, but it seems like they've hyper-focused on SaaS. Really would love to see one of these languages solve the data access layer for regular applications, everyone immediately seems to run to kubernetes at Google scale when I'd kill for a nice way to write permissions that can sit on the same server as my application and do the job reasonably well.