4 ms·

I'm not an elastic person, but I can shed some light on this: You're holding it wrong. It's not a bug. You have multiple config files in one directory - if you

I'm not an elastic person, but I can shed some light on this: You're holding it wrong. It's not a bug. You have multiple config files in one directory - if you do that, all those files are combined to one, that means that each event gets handed to each of your individual outputs - multiplying the message. See https://www.elastic.co/guide/en/logstash/current/command-line-flags.html https://www.elastic.co/guide/en/logstash/current/command-lin... Feel free to hop on the IRC if you have further questions, there's usually somebody qualified to answer.

clebio 11y ago
I appreciate the help, though isn't the point of `/etc//conf.d` directories generally that you have multiple config files? This is a common idiom that other packages handle correctly (differently?). I have hopped on the logstash IRC at times to ask about some of this, though I guess not this exact item. In fact, there's a different (well-known issue) that the init script for logstash has the config path hard-coded: https://botbot.me/freenode/logstash/2015-11-17/?msg=54338903&page=7 https://botbot.me/freenode/logstash/2015-11-17/?msg=54338903... There's also the problem that logstash (and forwarder) doesn't seem to let me do anything useful with the file names. I could work around that, sure, but it would be nice to have meaningful file names (not the "ls." thing that LS uses). Syslog-ng, for comparison, gives you a lot of control of that.
- clebio 11y ago
  (I've never gotten used to HN's non-Markdown markup.)
- Xylakant 11y ago
  > isn't the point of `/etc//conf.d` directories generally that you have multiple config files? Yes certainly. It's totally fine to place multiple config files there, I do as well. I split up my configs in the various outputs, inputs etc. It's just that logstash combines them to a single pipeline and does not run a pipeline per config file. Nginx doesn't run a webserver per config file either :). It's certainly something that's unexpected and could be much better documented, but alas, I'm just a user :) (and I do agree, your issue could have been handled much better, especially since it's not actually a bug)
  - clebio 11y ago
    Awesome, thanks for the clarification. This might help (if I ever go back to using logstash at this point!). Yeah, my point was more that they accepted my issue, but there's no action and there are more than 600+ other open issues. Seems Elastic is too busy branding and pushing breaking changes to their APIs. I do sincerely appreciate your clarifications and comments on this one, though.
clebio 11y ago
Oh, well, interesting that I just notice your HN username is also that IRC username. I suppose you're the one that replied to me there! Thank you!
- Xylakant 11y ago
  likely, I tend to use the same name all over the internet :) You're welcome.

clebio 11y ago

I appreciate the help, though isn't the point of `/etc//conf.d` directories generally that you have multiple config files? This is a common idiom that other packages handle correctly (differently?). I have hopped on the logstash IRC at times to ask about some of this, though I guess not this exact item. In fact, there's a different (well-known issue) that the init script for logstash has the config path hard-coded: https://botbot.me/freenode/logstash/2015-11-17/?msg=54338903&page=7 https://botbot.me/freenode/logstash/2015-11-17/?msg=54338903... There's also the problem that logstash (and forwarder) doesn't seem to let me do anything useful with the file names. I could work around that, sure, but it would be nice to have meaningful file names (not the "ls." thing that LS uses). Syslog-ng, for comparison, gives you a lot of control of that.

clebio 11y ago
(I've never gotten used to HN's non-Markdown markup.)
Xylakant 11y ago
> isn't the point of `/etc//conf.d` directories generally that you have multiple config files? Yes certainly. It's totally fine to place multiple config files there, I do as well. I split up my configs in the various outputs, inputs etc. It's just that logstash combines them to a single pipeline and does not run a pipeline per config file. Nginx doesn't run a webserver per config file either :). It's certainly something that's unexpected and could be much better documented, but alas, I'm just a user :) (and I do agree, your issue could have been handled much better, especially since it's not actually a bug)
- clebio 11y ago
  Awesome, thanks for the clarification. This might help (if I ever go back to using logstash at this point!). Yeah, my point was more that they accepted my issue, but there's no action and there are more than 600+ other open issues. Seems Elastic is too busy branding and pushing breaking changes to their APIs. I do sincerely appreciate your clarifications and comments on this one, though.

clebio 11y ago

Oh, well, interesting that I just notice your HN username is also that IRC username. I suppose you're the one that replied to me there! Thank you!

Xylakant 11y ago
likely, I tend to use the same name all over the internet :) You're welcome.