4 ms·

Unsecured Elasticsearch servers have been implicated in multiple breaches in recent months [1][2]. Since this post is an "In depth guide to running Elasticsearc

Unsecured Elasticsearch servers have been implicated in multiple breaches in recent months [1][2]. Since this post is an "In depth guide to running Elasticsearch in production,” it should prominently include information related to security and configuration. With tools like these where there is a learning curve for new users, security can end up treated as an afterthought, leading to these kinds of breaches. 1. https://www.pandasecurity.com/mediacenter/news/billion-consumers-data-breach-elasticsearch/ https://www.pandasecurity.com/mediacenter/news/billion-consu... 2. https://thedefenceworks.com/blog/250-million-microsoft-records-exposed-in-another-elasticsearch-server-related-breach/ https://thedefenceworks.com/blog/250-million-microsoft-recor... Edited for clarity

cloakandswagger 7y ago
This guide is clearly intended to focus on the ops-side of ElasticSearch. No one is being irresponsible, you're basically just complaining that the article was written about one topic instead of another. Notice how it also doesn't talk about system architecture, load balancers, disaster recovery, etc? It's because the author chose to focus the post on cluster configuration. The topic of security could be its own standalone writeup and I highly doubt that its omission is an endorsement for running an ES cluster totally exposed and unsecured.
- tmpz22 7y ago
  The argument is that you can't have an in-depth production guide to Elasticsearch without a section on security. "Production" should be "secure". A better title would be "optimizing Elasticsearch performance in production" or something of the sort.
  - isbvhodnvemrwvn 7y ago
    To be honest I think if you're responsible for running production systems, it would be a no-brainer to run everything as closed up as it gets, with only access from servers which actually need it.
    - judge2020 7y ago
      Yet we see security breaches caused by trivial misconfigurations and bad (or no) firewall setups. Chances are, people building these systems aren't accustomed to security-first deployment and will use and bookmark a guide like this to properly set up instances, rarely if ever going back to the docs or looking at other guides.
      - rhizome 7y ago
        Chances are, people building these systems aren't accustomed to security-first deployment and will use and bookmark a guide like this to properly set up instances Or they aren't given the time, running on ASAP-brand project management and/or pushing the POC to prod.
- rumanator 7y ago
  > This guide is clearly intended to focus on the ops-side of ElasticSearch. What's your point? The ops side of anything also covers security. In fact, you cannot have ops without effective security.
  - jann 7y ago
    I can't answer for cloakandswagger, but GPs comment sounded to me like this blog post is missing something essential because it doesn't talk about security. This isn't an expensive course on setting up the perfect ES cluster in production. As someone who is currently planning to set up a substantial ES cluster, I'm very grateful for someone to write up their learnings in such a compact overview.
    - rhizome 7y ago
      The antonym for "insecure" is not "perfect."
bobjordan 7y ago
An ES stack is fairly easy to get up and running in a development environment with docker-compose. But, not so much with a secure production installation. After going down the path of trying to get production up and running with security, I found Open Distro for Elasticsearch [1] to be very helpful. https://opendistro.github.io/for-elasticsearch/ https://opendistro.github.io/for-elasticsearch/
- DmitryOlshansky 7y ago
  Indeed OpenDistro is a great option to have. Currently using ODFE Security plugin with X-Pack basic (minus X-Pack security module). Still waiting on these guys to review my perf patch though: https://github.com/opendistro-for-elasticsearch/security/pull/198 https://github.com/opendistro-for-elasticsearch/security/pul...
  - _msw_ 7y ago
    Disclosure: I work for AWS, but not directly on Open Distro I'm sorry about the lack of engagement on that PR, Dmitry. Let me see if I can get some attention on it.
    - DmitryOlshansky 7y ago
      Thanks, I appreciate that. I tried reaching out on forums and was assured they’d get to it. I’m used to things going slow in OpenSource so not too worried at this point. Would be lovely to upstream it though, as I’ve patched 2 versions now and not looking forward to continue doing so ;) Plus I’d be glad to contribute more things based on our experience running ODFE, I’m very intrigued by other upcoming plugins in ODFE repos as well.
      - erudite 7y ago
        Hey DmitryOlshansky, I am one of the PMs and we discussed this PR internally. Our engineer will connect with you soon. Apologize for the delay but our team was working on getting the release out. Thanks
caro_douglos 7y ago
While we’re at it let’s touch on how vulnerable nginx is because port 80 is open. /s
- tomnipotent 7y ago
  There's a huge difference between an HTTP server listening on a port with no obvious "map" of what's available, an ES server that can quickly be explored and its contents exfiltrated with no prior knowledge of the content.
skinnyarms 7y ago
I just wanted to point out that Elastic has made some changes in the last year or so that help with security like... * [Making security bits available with the ("free") basic license](https://www.elastic.co/blog/security-for-elasticsearch-is-now-free https://www.elastic.co/blog/security-for-elasticsearch-is-no...) * [Releasing Kubernetes Operators with security enabled by default](https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html https://www.elastic.co/guide/en/cloud-on-k8s/current/index.h...) This has the effect of making most "getting started" guide setups more secure by default, which is good. Unfortunately this is a new change and those bits are not in the Apache licensed core offering, but it's still a big improvement IMHO.

cloakandswagger 7y ago

This guide is clearly intended to focus on the ops-side of ElasticSearch. No one is being irresponsible, you're basically just complaining that the article was written about one topic instead of another. Notice how it also doesn't talk about system architecture, load balancers, disaster recovery, etc? It's because the author chose to focus the post on cluster configuration. The topic of security could be its own standalone writeup and I highly doubt that its omission is an endorsement for running an ES cluster totally exposed and unsecured.

tmpz22 7y ago
The argument is that you can't have an in-depth production guide to Elasticsearch without a section on security. "Production" should be "secure". A better title would be "optimizing Elasticsearch performance in production" or something of the sort.
- isbvhodnvemrwvn 7y ago
  To be honest I think if you're responsible for running production systems, it would be a no-brainer to run everything as closed up as it gets, with only access from servers which actually need it.
  - judge2020 7y ago
    Yet we see security breaches caused by trivial misconfigurations and bad (or no) firewall setups. Chances are, people building these systems aren't accustomed to security-first deployment and will use and bookmark a guide like this to properly set up instances, rarely if ever going back to the docs or looking at other guides.
    - rhizome 7y ago
      Chances are, people building these systems aren't accustomed to security-first deployment and will use and bookmark a guide like this to properly set up instances Or they aren't given the time, running on ASAP-brand project management and/or pushing the POC to prod.
rumanator 7y ago
> This guide is clearly intended to focus on the ops-side of ElasticSearch. What's your point? The ops side of anything also covers security. In fact, you cannot have ops without effective security.
- jann 7y ago
  I can't answer for cloakandswagger, but GPs comment sounded to me like this blog post is missing something essential because it doesn't talk about security. This isn't an expensive course on setting up the perfect ES cluster in production. As someone who is currently planning to set up a substantial ES cluster, I'm very grateful for someone to write up their learnings in such a compact overview.
  - rhizome 7y ago
    The antonym for "insecure" is not "perfect."

bobjordan 7y ago

An ES stack is fairly easy to get up and running in a development environment with docker-compose. But, not so much with a secure production installation. After going down the path of trying to get production up and running with security, I found Open Distro for Elasticsearch [1] to be very helpful. https://opendistro.github.io/for-elasticsearch/ https://opendistro.github.io/for-elasticsearch/

DmitryOlshansky 7y ago
Indeed OpenDistro is a great option to have. Currently using ODFE Security plugin with X-Pack basic (minus X-Pack security module). Still waiting on these guys to review my perf patch though: https://github.com/opendistro-for-elasticsearch/security/pull/198 https://github.com/opendistro-for-elasticsearch/security/pul...
- _msw_ 7y ago
  Disclosure: I work for AWS, but not directly on Open Distro I'm sorry about the lack of engagement on that PR, Dmitry. Let me see if I can get some attention on it.
  - DmitryOlshansky 7y ago
    Thanks, I appreciate that. I tried reaching out on forums and was assured they’d get to it. I’m used to things going slow in OpenSource so not too worried at this point. Would be lovely to upstream it though, as I’ve patched 2 versions now and not looking forward to continue doing so ;) Plus I’d be glad to contribute more things based on our experience running ODFE, I’m very intrigued by other upcoming plugins in ODFE repos as well.
    - erudite 7y ago
      Hey DmitryOlshansky, I am one of the PMs and we discussed this PR internally. Our engineer will connect with you soon. Apologize for the delay but our team was working on getting the release out. Thanks

caro_douglos 7y ago

While we’re at it let’s touch on how vulnerable nginx is because port 80 is open. /s

tomnipotent 7y ago
There's a huge difference between an HTTP server listening on a port with no obvious "map" of what's available, an ES server that can quickly be explored and its contents exfiltrated with no prior knowledge of the content.

skinnyarms 7y ago

I just wanted to point out that Elastic has made some changes in the last year or so that help with security like... * [Making security bits available with the ("free") basic license](https://www.elastic.co/blog/security-for-elasticsearch-is-now-free https://www.elastic.co/blog/security-for-elasticsearch-is-no...) * [Releasing Kubernetes Operators with security enabled by default](https://www.elastic.co/guide/en/cloud-on-k8s/current/index.html https://www.elastic.co/guide/en/cloud-on-k8s/current/index.h...) This has the effect of making most "getting started" guide setups more secure by default, which is good. Unfortunately this is a new change and those bits are not in the Apache licensed core offering, but it's still a big improvement IMHO.