We built a similar system log-searching system using SenseiDB at LinkedIn. Splunk was outrageously expensive. It turns out that lucene based systems are pretty

We built a similar system log-searching system using SenseiDB at LinkedIn. Splunk was outrageously expensive. It turns out that lucene based systems are pretty good at information retrieval and aren't shackled with all the OLTP requirements most databases have.