3 ms·
So far I've only used this format for tiny datasets smaller than 10k root entries (and around ~1M nodes). And even then, before I use Python/R/etc to analyze th
by breck 8y ago
So far I've only used this format for tiny datasets smaller than 10k root entries (and around ~1M nodes). And even then, before I use Python/R/etc to analyze things, I transform it into a normalized table form exactly like you've shown above.
I think the tree form is great for the smaller datasets that need to be edited or reviewed manually, not bulk machine generated data. For one project I have thousands of files, each with data stored in a tree, all stored in a git repo with many editors, sort of like a data wiki.
It also works for the "presentation thing", as you say. You can use a table-backed storage system but present to the user a tree, and when they edit a node you propagate the change to the appropriate row.
Finally, some editing plugins can give you good highlighting and more context if you create a grammar file.
Source: have been tinkering with this stuff for a couple years.