5 ms·

Out of interest, what are the security issues you see with Big Data stacks?

• TallGuyShort 12y ago
  Most of the tools are written under the assumption that you're behind a firewall or airgap that will do all your security for you, and the controls are very weak and poorly enforced in the younger projects. Even in the mature ones, there's hasn't really been a cohesive model of security across the stack that meets the needs of stricter organizations. That's improving - I'm doing some work with Apache Sentry (incubating) that lets you apply more sophisticated security policies to fine-grained data in Hive, Impala and Solr (and in the future, hopefully many others). There are also projects like Apache Knox that, as I understand it, is a gateway for your cluster - so taking the idea of having an external tool protect your cluster, but at least one that understands what's happening in the cluster and handles authentication, etc. Multi-tenancy is also in it's infancy for Hadoop - but I'd expect as more of the projects start integrating better with YARN's resource management this will also be improving in the very near future.