4 ms·
I've no idea of what your doc requirements are, but if you don't have any real plans for the data, stick it in a cost effective format that requires little to n
by thisone 10y ago
I've no idea of what your doc requirements are, but if you don't have any real plans for the data, stick it in a cost effective format that requires little to no maintenance but that can be read by multiple systems. Say parquet or avro for example.
I assume you don't have someone/some system 24/7 pouring over the data so you could look into hosted notebooks like databricks, qubole (and gleaning from people I met at a recent conference any of the bazillion that are about to launch), or host zeppelin yourself or use AWS EMR.
And don't lose your source data for that moment when you do figure out what you want to do with the data and you realise you need to reformat it.