>You could probably pass the data through a gzip filter before storage That p much voids all the benefit of using SQLite. >And to be honest, if you need to se

>You could probably pass the data through a gzip filter before storage That p much voids all the benefit of using SQLite. >And to be honest, if you need to search through or index JSON blobs in a database you need to reconsider your design. This is absolutely intentional the log contains information to manually re-play certain events in case of an fatal error mostly for debug purpose or performance benchmarks. This means 99.99% of the data will never be needed and eventually will be discarded. But when I need to re-play data I need to be able to query for exactly the data I need. Each JSON "blob" (which is actually store as TEXT in SQLite) has a few additional fields to quickly filter off 95% or more of unwanted data. The rest SQLite simply reads completely to separate wanted and unwanted entries further. With a DB size around 100GB I get the data I need by reading 5GB or less. That's perfectly fine for my use case even if I would let the DB grow quite a lot larger it would be just fine.