3 ms·
Actually, we use LDAP. If you have an LDAP account, you're authenticated. The use of the certs is transparent to clients - it's wrapped in library calls. The p
by jamesblonde 9y ago
Actually, we use LDAP. If you have an LDAP account, you're authenticated. The use of the certs is transparent to clients - it's wrapped in library calls.
The problem we're addressing is multi-tenancy for users on Hadoop. If you have sensitive data and want to restrict access to it, you need at a minimum dynamic roles or attribute-based access controls. With Kerberos, you have a single ID, a single role. You can't revoke privileges from users, give him/her access to a dataset, then restore privileges.
We introduced dynamic roles by introducing a new abstraction - a project. You get a project-specific username - and a new cert for every project-specific username. Unlike Kerberos with the centralized KDC, cert generation can be scaled out with intermediate servers. This way, you can give a project-specific user access to sensitive data in a project - and the user can't export the data, as that user only has privileges for resources within the project. You can still share datasets across projects using either ACLs or group privileges in HDFS.
- puzzle 9y agoRigidity when it comes to tenancy and lack of groups were two of the reasons why Google went with its own LOAS instead of Kerberos for the production environment. The even bigger one was performance.
- jamesblonde 9y agoYes, the Kerberos KDC is a big bottleneck. In contrast, you can scale out TLS/SSL cert generation with intermediate CAs.
- lars_francke 9y agoThank you for the extra explanations. But I'm afraid I still don't understand. With Kerberos you have a single Id, yes but that is augmented by groups coming from LDAP and RBAC systems like Sentry & Ranger. Also having another system issuing certificates automatically is a no-go in lots of companies and especially security conscious industries. I'm not saying that there's not a market for what you're developing - I'm merely trying to understand and I have a hard time. Your homepage, I'm afraid though, is useless at conveying any information about what you're actually doing.
- jamesblonde 9y agoSentry and Ranger offer RBAC, but not dynamic roles or attribute-based access control. Dynamic roles for us are: if you are a data scientist working on a dataset, when working on that dataset, we give you the minimum roles needed to work on that dataset. If you normally have privileges to write to some other directory outside that subtree in HDFS, we take it away while you are active in the project. When you change projects, your access rights change. Sentry and Ranger are horrendous. Best practice for them in HDFS is to change permissions for subtrees in Hive/Impala to '000' so that sysadmins know that an external system is doing access control here. They do not offer unified access control, rather fragmented access control. In our case, you see a different username in hdfs for each of your projects. I agree the homepage sucks - it's on our todo list :)
- jamesblonde 9y agoBtw, I gave the talk after you at the Dataworks Summit in Munich. Nice talk!