3 ms·
Can anybody speak to their experience using sqlite for data analysis purposes? Am I wrong in thinking it's just plain old row oriented storage and not somethin
by baconner 12y ago
Can anybody speak to their experience using sqlite for data analysis purposes?
Am I wrong in thinking it's just plain old row oriented storage and not something more aggregate oriented?
- mbrzusto 12y agoI use it quite a bit for data analysis, in particular for user-defined functions with python. Being able to explore your data (via SQL and it's powerful syntax) in addition to functions and aggregators that I define, is REALLY useful. You could do the whole thing in python (and data imports), but adding the SQL part in is so much easier than building dictionaries and filtering, sorting etc...
- k2enemy 12y agoI use sqlite to store all of my small datasets. Minimally, I think of it as a replacement for zipped CSV files. But it also has the added benefit of a relational structure and SQL. It is super easy to access from julia, R, python, etc, so instead of importing a CSV and manipulating the data, I find it a lot easier to connect to the sqlite database and use SQL for the a lot of the joining and manipulating.
- baconner 12y agoI see. So the benefit is mainly in having more relational structure than in actually crunching numbers. Makes sense.