I've built a columnar in-memory data transformation engine [1] and wrote [2] about the need in common columnar data format to avoid re-compression. The problem

I've built a columnar in-memory data transformation engine [1] and wrote [2] about the need in common columnar data format to avoid re-compression. The problem I have with the Apache projects is that they all require strongly typed columns (if I'm not missing something). In our case, the app is actively used for processing spreadsheets and sometimes XML files which requires supporting mixed data types (e.g. timestamps and strings) in a column. Another issue is storing 128-bit decimals. They are more preferable for financial calculations, but are not supported in the Apache projects. So may be we need a forth standard for columnar data representation. Or expand the existing ones to make them more versatile. [1] http://easymorph.com/in-memory-engine.html http://easymorph.com/in-memory-engine.html [2] http://bi-review.blogspot.ca/2015/06/the-world-needs-open-source-columnar.html http://bi-review.blogspot.ca/2015/06/the-world-needs-open-so...