3 ms·
Seq seems to be targeting application logs, where JSON (and jsonl) is a common format and it makes sense.
by tomnipotent 5y ago
Seq seems to be targeting application logs, where JSON (and jsonl) is a common format and it makes sense.
- jsnell 5y agoThere's three data formats involved: 1. The one used for ingesting data. 2. The one used for storage. 3. The one used for exporting data. There is no reason for all three formats to be the same, and there are good reason for them not to be: the actual use cases are requiring very different properties from the data format. Since you'd expect the average log record to be ingested once, processed a gazillion times, and returned as output <<1 times, any data conversion costs would get amortized over a lot of processing. Even minor efficiency gains in the at-rest storage format should pay off quickly. Your answer is addressing format 1: the clients are using JSON anyway, so it makes sense for the ingestion format to be JSON. That's making the bet that any data conversion costs can't be paid back over the lifetime of the data. But they are already not using the raw client input as the storage format! At a minimum they're validating / minifying / canonicalizing the JSON, so they're already paying the cost of doing a format conversion. (But all of this is obvious, and clearly the people writing this service are smart and would have thought of it. So it seems like there's a non-obvious answer for using minified JSON for storage at rest.)