At FOSDEM the talk on loki was described as a modern version of what you and I do with syslog servers mine come in to an anycast IP on the network, one file pe

At FOSDEM the talk on loki was described as a modern version of what you and I do with syslog servers mine come in to an anycast IP on the network, one file per host, the syslog stamps the receive time at the start in "y-m-d-h-m-s+0000" format, in a y/m/d directory struture, bzip2 after a few days I have a few scripts which I use to parse the logs and pull reports out (BGP drops/recover times for example), but most of the time tail/cat/sort/grep/cut/etc does the job. Where I differ is I use perl rather than awk. Sure it doesn't scale to millions of terrabytes a second of minable personal information or whatever the average modern LAMP stack generates, but it currently records about 15G a day from 400 different devices just fine.