4 ms·

Anatomy of an Elasticsearch Cluster: Part I

67 points by ronaknnathani 10y ago

sciurus 10y ago
The elasticsearch documentation in this area is actually not bad, e.g. https://www.elastic.co/guide/en/elasticsearch/guide/2.x/distributed-cluster.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/dist... https://www.elastic.co/guide/en/elasticsearch/guide/2.x/distributed-search.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/dist... https://www.elastic.co/guide/en/elasticsearch/guide/2.x/inside-a-shard.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/insi...
- whenwillitstop 10y ago
  It's actually kind of terrible. It leaves out a ton of the caveats and detailed information. Once you start using elastic search heavily, the docs are almost useless.
  - true_religion 10y ago
    What kind of caveats? I run a cluster in production and have little trouble with it.
hobs 10y ago
I like this post, but it took me an extra 20 seconds to realize the DO NOT EQUAL signs were being used to equate things. Analogy to relational database terms Elasticsearch Index <> Database Types <> Tables Properties <> Schema
- ronaknnathani 10y ago
  Ah! Good catch. I shall update that.
bogomipz 10y ago
Does anyone have any insight on insightdataengineering.com? Is this just a "hacker school" job placement agency? They say its free but nothing is free. I've become very skeptical about these. 7 weeks is barely enough time to really understand the nuances of any distributed datastore let alone a number of them in a pipeline.
- ddrum001 10y ago
  I'm from Insight - our program is kept free for our Fellows because the companies that we partner with pay for the Fellowship. Rather than classes, we believe that the best way for advanced engineers to learn the detailed nuances of these tools is to use them, so the concept is to learn them by building a data platform. You're right that it's really difficult to understand distributed systems in a few weeks, but our Fellows already have several years of programming, so they have been able to pick up a general understanding quite quickly.
  - bogomipz 10y ago
    The site is somewhat lacking in any kind of syllabus or any details of how past fellows(not really sure what that means, in context of a hacker school, although I am familiar with the term in the context of IBM or Sun from days past) What are some past projects that attendees built?
    - ddrum001 10y ago
      We don't have a fixed syllabus since we don't operate like a school per se. Instead, the Fellows choose a project and have to make engineering tradeoffs about which tools to use. With that said, most of the past participants focused on distributed systems like Hadoop, Spark, Flink, Kafka, and NoSQL DBs. You can check out past projects on the blog (http://insightdataengineering.com/blog/ http://insightdataengineering.com/blog/) and the past Fellows are here (http://www.insightdataengineering.com/fellows.html http://www.insightdataengineering.com/fellows.html)

67 points by ronaknnathani 10y ago

The elasticsearch documentation in this area is actually not bad, e.g. https://www.elastic.co/guide/en/elasticsearch/guide/2.x/distributed-cluster.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/dist... https://www.elastic.co/guide/en/elasticsearch/guide/2.x/distributed-search.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/dist... https://www.elastic.co/guide/en/elasticsearch/guide/2.x/inside-a-shard.html https://www.elastic.co/guide/en/elasticsearch/guide/2.x/insi...

whenwillitstop 10y ago
It's actually kind of terrible. It leaves out a ton of the caveats and detailed information. Once you start using elastic search heavily, the docs are almost useless.
- true_religion 10y ago
  What kind of caveats? I run a cluster in production and have little trouble with it.

hobs 10y ago

I like this post, but it took me an extra 20 seconds to realize the DO NOT EQUAL signs were being used to equate things. Analogy to relational database terms Elasticsearch Index <> Database Types <> Tables Properties <> Schema

ronaknnathani 10y ago
Ah! Good catch. I shall update that.

bogomipz 10y ago

Does anyone have any insight on insightdataengineering.com? Is this just a "hacker school" job placement agency? They say its free but nothing is free. I've become very skeptical about these. 7 weeks is barely enough time to really understand the nuances of any distributed datastore let alone a number of them in a pipeline.

ddrum001 10y ago
I'm from Insight - our program is kept free for our Fellows because the companies that we partner with pay for the Fellowship. Rather than classes, we believe that the best way for advanced engineers to learn the detailed nuances of these tools is to use them, so the concept is to learn them by building a data platform. You're right that it's really difficult to understand distributed systems in a few weeks, but our Fellows already have several years of programming, so they have been able to pick up a general understanding quite quickly.
- bogomipz 10y ago
  The site is somewhat lacking in any kind of syllabus or any details of how past fellows(not really sure what that means, in context of a hacker school, although I am familiar with the term in the context of IBM or Sun from days past) What are some past projects that attendees built?
  - ddrum001 10y ago
    We don't have a fixed syllabus since we don't operate like a school per se. Instead, the Fellows choose a project and have to make engineering tradeoffs about which tools to use. With that said, most of the past participants focused on distributed systems like Hadoop, Spark, Flink, Kafka, and NoSQL DBs. You can check out past projects on the blog (http://insightdataengineering.com/blog/ http://insightdataengineering.com/blog/) and the past Fellows are here (http://www.insightdataengineering.com/fellows.html http://www.insightdataengineering.com/fellows.html)