4 ms·

Because it absolutely wasn't, and still isn't, necessary. Full stop. ElasticSearch speaks HTTP. Why would I want it to reimplement HTTP authentication when I ca

Because it absolutely wasn't, and still isn't, necessary. Full stop. ElasticSearch speaks HTTP. Why would I want it to reimplement HTTP authentication when I can place it behind a reverse proxy that's already integrated with my company's AAA and has been doing nothing but HTTP for oh, three decades or so? Separation of concerns. This is how this goes, and I've seen the pattern followed with several infrastructure services, even here in this HN thread right now: 1) A bug report comes in to AwesomeDB. "You do not have authentication support. This is bad. I'm going to yell about it on social media and make fun of you until you can accept a username and password, because everything in existence should write its own authentication code. Even MySQL takes passwords, dude, get with 2017." 2) A rough version is implemented by an intern, because authorization is way less sexy than hyperdimensional bloom filter quantum hash tables to speed up random reads. The intern then departs back to beer pong and nobody at AwesomeDB now understands the security infrastructure of their product. 3A) A mid-level executive, seeing the announcement, instructs me that I no longer have to keep AwesomeDB behind my 'stupid' reverse proxy + ACL solution and we can finally use native authentication and be really secure! 3B) Alternatively and more reastically, someone at Qualys adds "authentication is not turned on in AwesomeDB" to their scanning product and I get nastygrams from a contractor clicking "Scan" and copying and pasting the results for each of my fifty AwesomeDB instances. 4) During testing, I capture the credentials in plain text in about three minutes because it's a database. Why on earth would it terminate TLS? If we are down the 3B path above: Ha! Your move, security team! 5) Another bug report is filed against AwesomeDB to terminate TLS. Equal shaming is performed, because every service in the world needs to link in OpenSSL and terminate TLS, because that's never been a problem or backfired in any way. Oh yeah, and everyone has a different mechanism for handling certificates, and most don't bother to consider revocation. (Just like the broader point I'm making about auth, stop adding TLS support to your systems. Please. If you speak TCP I can wrap you in something uniform that does a much better job anyway. You know how I know your integration is shitty? Because OpenSSL is hopelessly undocumented and the APIs even more so. You Googled and read a guide. How about instead, I can drop a pretty awesome TLS listener in front of you, potentially even in hardware! Whoo!) 6) TLS support is added to AwesomeDB. Cool! Now we can really be secure! (But nobody implements client certificates, so we still have to do the username/password shared secret business. And alternate DH parameters? What are those? They weren't covered in the guide, so AwesomeDB hardcoded a couple.) At this point, most people would put credentials directly in their repo. I'd sit around making the Vault argument, but nobody would listen. We have now traded my ACLs and authentication stack for security on our private GitHub repos. I set to work automating changing all of our infrastructure passwords every time an intern leaves the company. We still secure the network, of course, but now there's special strings of characters in existence to work around a competent network security strategy that already existed. 7) As a growing company we introduce LDAP and Kerberos. Oh wait, AwesomeDB and its home brewed authentication stack do not support LDAP because what startup needs LDAP? 8) Another bug report is filed against AwesomeDB for "LDAP support." A bunch of startup engineers spend several months trying to understand the guts of LDAP (I feel your pain) and, worse, the cornucopia of vendor shenanigans in that space, and set the bug report to "holy shit, someday." A rough version that works with one exact schema and hopelessly avoids being just right for my needs might ship, or might not. 9) I redeploy my perfectly functional reverse proxy solution which has supported LDAP since the mid 90s and wire it up to Google OAuth for bonus points (getting rid of passwords for all corporate systems overnight! Neat!) while half the world pointlessly fights for every infrastructure service that exists to basically link in an operating system's worth of encryption and authentication support just to be a document store or search server. Given the choice I would very much rather secure my network or use well-established software to secure an infrastructure service. Asking every database vendor in existence to reimplement this stuff is a bad idea. People who blindly advocate for this sort of work, then paper over it with "you can't trust the network" without really supporting that statement, just generate busy work for companies that write infrastructure services. If you can't trust your network, what business do you have operating a production service that maintains user data? Do you really think having ElasticSearch do its own HTTP basic is more secure than any other solution? They at least had the JVM/netty/whatever to fall back on, where a lot of these infrastructure services have had to write their own authentication. There is a middle ground here and a lot of advocacy, both on HN and elsewhere, that misses the forest for the trees. Of course, none of this applies to MongoDB, because they were like sure, the world needs another bespoke binary protocol. You absolutely should be keeping your data on a secured network, though, and opaque strings of characters for system A to log in to system B might seem like defense in depth, but they end up in your source code, another system, or the working set of your executing process, and they are routinely captured by active adversaries. So. Having a mode where an infrastructure service assumes a connection is legitimate and just services the client is important. Some of us have a little more confidence in our security strategy than others and feel that we know what we are doing. Wiring that mode up to a 0.0.0.0 bind, by default, is the bug. Oh yeah, almost forgot: 10) All that custom authentication in AwesomeDB that you had an intern write becomes a CVE factory and starts reflecting on the security of the product. Womp womp.

bogomipz 10y ago
>"Why would I want it to reimplement HTTP authentication when I can place it behind a reverse proxy that's already integrated with my company's AAA and has been doing nothing but HTTP for oh, three decades or so?" Because getting RBAC right for individual indexes in Elasticsearch is not necessarily trivial. Example, I have multiple groups and users. And lets say I want to give some users access to these indexes but not these other ones and this group can access these indexes read-only and this group can add or delete but not access the admin endpoints etc. Just "putting it behind your company's proxy" doesn't work out of the box. And integrating this on Elasticsearch client Nodes - nodes where node.data is set to false and node.master is set to false, would have been very logical. Also not every company has AAA with full LDAP integration, i.e any early stage start up that consists of a small handful of people wearing many hats because some of which are not their area of expertise. Lots of these types of companies are also using Elasticsearch for full text indexing.
- jsmthrowaway 10y ago
  There's a difference between username/password gatekeeping security of an infrastructure service, which my post was quite clearly discussing, and fine-grained RBAC in the data plane. One is itself data within the database, while the other is gatekeeping to the database; they have vastly different impetuses, design constraints, and security strategy. Conflating the two is wrong and, yes, many services themselves conflate the two which reduces the flexibility of both. Why not let my system do the AA and tell the backend what user it is so that RBAC can be implemented? I haven't used Shield but I'm guessing this is difficult to separate; in a number of other systems it is. The key to your office is not nearly the same as the key to each drawer of your filing cabinet. Each can be shared independently and serve different purposes. You might decide to never use your office key, for example, because of the armed linebacker in your lobby. In my experience, most of the effort to make an infrastructure service speak authentication is intended for gatekeeping and RBAC comes later. ElasticSearch obviously went all the way with it. This whole conversation about infrastructure authorization tends to make people forget what the actual threat vectors are, which is really the subtext of my thesis here. Every company has at least a subset of AAA, and hand waving it away as "eh, early startup stuff" is also dangerous. Putting stuff behind Google SSO is nice for startups these days and, boom, AA-sorta-A.

bogomipz 10y ago

>"Why would I want it to reimplement HTTP authentication when I can place it behind a reverse proxy that's already integrated with my company's AAA and has been doing nothing but HTTP for oh, three decades or so?" Because getting RBAC right for individual indexes in Elasticsearch is not necessarily trivial. Example, I have multiple groups and users. And lets say I want to give some users access to these indexes but not these other ones and this group can access these indexes read-only and this group can add or delete but not access the admin endpoints etc. Just "putting it behind your company's proxy" doesn't work out of the box. And integrating this on Elasticsearch client Nodes - nodes where node.data is set to false and node.master is set to false, would have been very logical. Also not every company has AAA with full LDAP integration, i.e any early stage start up that consists of a small handful of people wearing many hats because some of which are not their area of expertise. Lots of these types of companies are also using Elasticsearch for full text indexing.

jsmthrowaway 10y ago
There's a difference between username/password gatekeeping security of an infrastructure service, which my post was quite clearly discussing, and fine-grained RBAC in the data plane. One is itself data within the database, while the other is gatekeeping to the database; they have vastly different impetuses, design constraints, and security strategy. Conflating the two is wrong and, yes, many services themselves conflate the two which reduces the flexibility of both. Why not let my system do the AA and tell the backend what user it is so that RBAC can be implemented? I haven't used Shield but I'm guessing this is difficult to separate; in a number of other systems it is. The key to your office is not nearly the same as the key to each drawer of your filing cabinet. Each can be shared independently and serve different purposes. You might decide to never use your office key, for example, because of the armed linebacker in your lobby. In my experience, most of the effort to make an infrastructure service speak authentication is intended for gatekeeping and RBAC comes later. ElasticSearch obviously went all the way with it. This whole conversation about infrastructure authorization tends to make people forget what the actual threat vectors are, which is really the subtext of my thesis here. Every company has at least a subset of AAA, and hand waving it away as "eh, early startup stuff" is also dangerous. Putting stuff behind Google SSO is nice for startups these days and, boom, AA-sorta-A.