5 ms·
Very nice & all, but... MongoDB? MongoDB is very popular, but all the (limited) criticisms of it seem to related to insert performance once the dataset it too
by nl 15y ago
Very nice & all, but... MongoDB?
MongoDB is very popular, but all the (limited) criticisms of it seem to related to insert performance once the dataset it too big to fit in RAM.
Normally the easy-of-development arguments make up for that, but log files is one of those areas that has a tendency to expand quickly beyond any expectations.
There is a reason why most companies are using HDFS and/or Cassandra for structured log file storage.
- kzk_mover 15y agofluentd's greatest advantage is, it's written in Ruby. So, it's really easy to write the plugin for any datastores. This is HDFS (Hoop, the HDFS REST gateway) plugin. * https://github.com/tagomoris/fluent-plugin-hoop https://github.com/tagomoris/fluent-plugin-hoop And also, Cassandra plugin are now in under development. * https://github.com/tomitakazutaka/fluent-plugin-cassandra https://github.com/tomitakazutaka/fluent-plugin-cassandra You can see the user contributed plugin list here. * https://github.com/tomitakazutaka/fluent-plugin-cassandra https://github.com/tomitakazutaka/fluent-plugin-cassandra
- seany 15y agoIf you were going from logs -> hdfs, flume would be a much better chose imho