6 ms·

Yes, Kerberos seems complex but in practice it's not the technology being a problem but usually organizational problems. But one big benefit that Kerberos has o

Yes, Kerberos seems complex but in practice it's not the technology being a problem but usually organizational problems. But one big benefit that Kerberos has over certificates is that almost every big company has a central Active Directory and almost every user already has Kerberos tickets. The same is not true for certificates which (in my subjective experience from securing tens of clusters) always cause issues. I wonder what you mean by "operational complexity" considering that once you have the principals created there's basically no overhead.... Or to put it differently: Can you elaborate on which problem you're trying to solve?

smegel 9y ago
> The same is not true for certificates which (in my subjective experience from securing tens of clusters) always cause issues. Yeah, getting a new client certificate issued for a system account is like trying to squeeze blood out of stone.
- jamesblonde 9y ago
  We ship the certificates out transparently to users using Yarn. Yarn keeps them as private local resources and cleans them up for us.
jamesblonde 9y ago
Actually, we use LDAP. If you have an LDAP account, you're authenticated. The use of the certs is transparent to clients - it's wrapped in library calls. The problem we're addressing is multi-tenancy for users on Hadoop. If you have sensitive data and want to restrict access to it, you need at a minimum dynamic roles or attribute-based access controls. With Kerberos, you have a single ID, a single role. You can't revoke privileges from users, give him/her access to a dataset, then restore privileges. We introduced dynamic roles by introducing a new abstraction - a project. You get a project-specific username - and a new cert for every project-specific username. Unlike Kerberos with the centralized KDC, cert generation can be scaled out with intermediate servers. This way, you can give a project-specific user access to sensitive data in a project - and the user can't export the data, as that user only has privileges for resources within the project. You can still share datasets across projects using either ACLs or group privileges in HDFS.
- puzzle 9y ago
  Rigidity when it comes to tenancy and lack of groups were two of the reasons why Google went with its own LOAS instead of Kerberos for the production environment. The even bigger one was performance.
  - jamesblonde 9y ago
    Yes, the Kerberos KDC is a big bottleneck. In contrast, you can scale out TLS/SSL cert generation with intermediate CAs.
- lars_francke 9y ago
  Thank you for the extra explanations. But I'm afraid I still don't understand. With Kerberos you have a single Id, yes but that is augmented by groups coming from LDAP and RBAC systems like Sentry & Ranger. Also having another system issuing certificates automatically is a no-go in lots of companies and especially security conscious industries. I'm not saying that there's not a market for what you're developing - I'm merely trying to understand and I have a hard time. Your homepage, I'm afraid though, is useless at conveying any information about what you're actually doing.
  - jamesblonde 9y ago
    Sentry and Ranger offer RBAC, but not dynamic roles or attribute-based access control. Dynamic roles for us are: if you are a data scientist working on a dataset, when working on that dataset, we give you the minimum roles needed to work on that dataset. If you normally have privileges to write to some other directory outside that subtree in HDFS, we take it away while you are active in the project. When you change projects, your access rights change. Sentry and Ranger are horrendous. Best practice for them in HDFS is to change permissions for subtrees in Hive/Impala to '000' so that sysadmins know that an external system is doing access control here. They do not offer unified access control, rather fragmented access control. In our case, you see a different username in hdfs for each of your projects. I agree the homepage sucks - it's on our todo list :)
  - jamesblonde 9y ago

smegel 9y ago

> The same is not true for certificates which (in my subjective experience from securing tens of clusters) always cause issues. Yeah, getting a new client certificate issued for a system account is like trying to squeeze blood out of stone.

jamesblonde 9y ago
We ship the certificates out transparently to users using Yarn. Yarn keeps them as private local resources and cleans them up for us.

jamesblonde 9y ago

Actually, we use LDAP. If you have an LDAP account, you're authenticated. The use of the certs is transparent to clients - it's wrapped in library calls. The problem we're addressing is multi-tenancy for users on Hadoop. If you have sensitive data and want to restrict access to it, you need at a minimum dynamic roles or attribute-based access controls. With Kerberos, you have a single ID, a single role. You can't revoke privileges from users, give him/her access to a dataset, then restore privileges. We introduced dynamic roles by introducing a new abstraction - a project. You get a project-specific username - and a new cert for every project-specific username. Unlike Kerberos with the centralized KDC, cert generation can be scaled out with intermediate servers. This way, you can give a project-specific user access to sensitive data in a project - and the user can't export the data, as that user only has privileges for resources within the project. You can still share datasets across projects using either ACLs or group privileges in HDFS.

puzzle 9y ago
Rigidity when it comes to tenancy and lack of groups were two of the reasons why Google went with its own LOAS instead of Kerberos for the production environment. The even bigger one was performance.
- jamesblonde 9y ago
  Yes, the Kerberos KDC is a big bottleneck. In contrast, you can scale out TLS/SSL cert generation with intermediate CAs.
lars_francke 9y ago
Thank you for the extra explanations. But I'm afraid I still don't understand. With Kerberos you have a single Id, yes but that is augmented by groups coming from LDAP and RBAC systems like Sentry & Ranger. Also having another system issuing certificates automatically is a no-go in lots of companies and especially security conscious industries. I'm not saying that there's not a market for what you're developing - I'm merely trying to understand and I have a hard time. Your homepage, I'm afraid though, is useless at conveying any information about what you're actually doing.
- jamesblonde 9y ago
  Sentry and Ranger offer RBAC, but not dynamic roles or attribute-based access control. Dynamic roles for us are: if you are a data scientist working on a dataset, when working on that dataset, we give you the minimum roles needed to work on that dataset. If you normally have privileges to write to some other directory outside that subtree in HDFS, we take it away while you are active in the project. When you change projects, your access rights change. Sentry and Ranger are horrendous. Best practice for them in HDFS is to change permissions for subtrees in Hive/Impala to '000' so that sysadmins know that an external system is doing access control here. They do not offer unified access control, rather fragmented access control. In our case, you see a different username in hdfs for each of your projects. I agree the homepage sucks - it's on our todo list :)
- jamesblonde 9y ago