4 ms·

TBH sounds like growing pains, I would imagine with the might of amazon engineering behind the hard-fork it will eventually surpass the original. Question is w

TBH sounds like growing pains, I would imagine with the might of amazon engineering behind the hard-fork it will eventually surpass the original. Question is when??

__blockcipher__ 7y ago
Don’t bet on it. AWS Elasticsearch is really as bad as this article indicates.
- lflux 7y ago
  Agreed, this article captures my experience accurately as well.
- tedivm 7y ago
  Yeah, we ran a minor upgrade to our cluster earlier this week and it knocked out the entire cluster for over two hours, and we were getting AWS specific errors regarding hostname headers that we've never gotten before. I managed to get a developer advocate on twitter to lend us a hand, but if I had actually waited on support things would likely still be down. Unfortunately that seems more like a trend with AWS- there are a lot of new services that feel like they're 80% ready for production, but which are being sold as complete solutoins.
  - journalctl 7y ago
    Finishing things isn’t cool or sexy, don’tchaknow! Gotta get that MVP out the door and move onto the next project. It’s fine, you can iterate on it after re:Invent!
bifrost 7y ago
Its been this bad for a while, I think its just a stalled project inside of AWS or the people who work on it think it functions as designed...
nwsm 7y ago
That's a weird assumption to make. AWS Elasticsearch is focused on providing minimum necessary functionality to sell instances. Opensource Elasticsearch is focused on improving Elasticsearch.
- greatjack613 7y ago
  Thats a valid point. But don't you think the more features it has the more instances of it will sell?
  - nwsm 7y ago
    Generally yes, but they can wait for Elasticsearch to implement them and port them over with less time/money spent. I don't think they have a ton of incentive to surpass Elasticsearch, but just to lag behind at some reasonable rate
bryanrasmussen 7y ago
Is the might of Amazon engineering really behind it though, or is that might behind a bunch of different things of varying importance. How much might can Amazon really bring to bear on any one project?
- lmeyerov 7y ago
  ES revenue can cover a lot of devs. But rephrasing: is ES the logging and visibility solution they are betting on, or is ES an 'ok to be 2nd for folks insisting on ES' and they plan to win with other cloud-native log/doc analytics tooling? (Their crocodile smile on ethical open source practices through better operational excellence reads even thinner when seeing stuff like this tho.)
  - time0ut 7y ago
    They've definitely been positioning it as the preferred solution. Our SA steered us towards it as a general search solution over their seemingly abandoned cloud search offering. The (also abandoned) AWS landing zone has an addon to stand up an ELK stack for you and no alternative afaik.
    - lmeyerov 7y ago
      Really not obvious to me... just as MS released internal Kusto as a product, and Google did BigQuery, I've been curious if some internal more integrated Amazon tool would surface. E.g., see the direction of CloudWatch to go from metrics queries to log queries (https://aws.amazon.com/about-aws/whats-new/2019/07/cloudwatch-logs-insights-adds-cross-log-group-querying/ https://aws.amazon.com/about-aws/whats-new/2019/07/cloudwatc...) and how GuardDuty is marching towards a modern SIEM. What the cloud vendors can do vs. splunk/elk/cisco makes it feel more like a "when" not "if". Zero judgement here on technical quality, this is just the nature of big tech co's. Do most AWS teams use ES, or something else? Is CloudWatch Insights on a scaling & parity path?
tjholowaychuk 7y ago
Haven't used many AWS services have you? :D hahah
- bifrost 7y ago
  Remember the EBS garbage fest a couple years ago?
juliansimioni 7y ago
I would have believed that. I was told by someone that "Amazon has a practice of launching early [with bugs] and then iterating" But all the things in this article were true when I tried the Amazon Elasticsearch Service in 2016, so if they haven't fixed it in 3 years, what make you think they'll fix it soon?

TBH sounds like growing pains, I would imagine with the might of amazon engineering behind the hard-fork it will eventually surpass the original. Question is when??

__blockcipher__ 7y ago

Don’t bet on it. AWS Elasticsearch is really as bad as this article indicates.

lflux 7y ago
Agreed, this article captures my experience accurately as well.
tedivm 7y ago
Yeah, we ran a minor upgrade to our cluster earlier this week and it knocked out the entire cluster for over two hours, and we were getting AWS specific errors regarding hostname headers that we've never gotten before. I managed to get a developer advocate on twitter to lend us a hand, but if I had actually waited on support things would likely still be down. Unfortunately that seems more like a trend with AWS- there are a lot of new services that feel like they're 80% ready for production, but which are being sold as complete solutoins.
- journalctl 7y ago
  Finishing things isn’t cool or sexy, don’tchaknow! Gotta get that MVP out the door and move onto the next project. It’s fine, you can iterate on it after re:Invent!

bifrost 7y ago

Its been this bad for a while, I think its just a stalled project inside of AWS or the people who work on it think it functions as designed...

nwsm 7y ago

That's a weird assumption to make. AWS Elasticsearch is focused on providing minimum necessary functionality to sell instances. Opensource Elasticsearch is focused on improving Elasticsearch.

greatjack613 7y ago
Thats a valid point. But don't you think the more features it has the more instances of it will sell?
- nwsm 7y ago
  Generally yes, but they can wait for Elasticsearch to implement them and port them over with less time/money spent. I don't think they have a ton of incentive to surpass Elasticsearch, but just to lag behind at some reasonable rate

bryanrasmussen 7y ago

Is the might of Amazon engineering really behind it though, or is that might behind a bunch of different things of varying importance. How much might can Amazon really bring to bear on any one project?

lmeyerov 7y ago
ES revenue can cover a lot of devs. But rephrasing: is ES the logging and visibility solution they are betting on, or is ES an 'ok to be 2nd for folks insisting on ES' and they plan to win with other cloud-native log/doc analytics tooling? (Their crocodile smile on ethical open source practices through better operational excellence reads even thinner when seeing stuff like this tho.)
- time0ut 7y ago
  They've definitely been positioning it as the preferred solution. Our SA steered us towards it as a general search solution over their seemingly abandoned cloud search offering. The (also abandoned) AWS landing zone has an addon to stand up an ELK stack for you and no alternative afaik.
  - lmeyerov 7y ago
    Really not obvious to me... just as MS released internal Kusto as a product, and Google did BigQuery, I've been curious if some internal more integrated Amazon tool would surface. E.g., see the direction of CloudWatch to go from metrics queries to log queries (https://aws.amazon.com/about-aws/whats-new/2019/07/cloudwatch-logs-insights-adds-cross-log-group-querying/ https://aws.amazon.com/about-aws/whats-new/2019/07/cloudwatc...) and how GuardDuty is marching towards a modern SIEM. What the cloud vendors can do vs. splunk/elk/cisco makes it feel more like a "when" not "if". Zero judgement here on technical quality, this is just the nature of big tech co's. Do most AWS teams use ES, or something else? Is CloudWatch Insights on a scaling & parity path?

tjholowaychuk 7y ago

Haven't used many AWS services have you? :D hahah

bifrost 7y ago
Remember the EBS garbage fest a couple years ago?

juliansimioni 7y ago

I would have believed that. I was told by someone that "Amazon has a practice of launching early [with bugs] and then iterating" But all the things in this article were true when I tried the Amazon Elasticsearch Service in 2016, so if they haven't fixed it in 3 years, what make you think they'll fix it soon?