3 ms·

I'm approaching the nested datastructure playlist problem by having playlists be nothing more than a special tag in the existing sqlite table. But then you have

by zdkl 9y ago

I'm approaching the nested datastructure playlist problem by having playlists be nothing more than a special tag in the existing sqlite table. But then you have to reckon with explosive growth when dealing with that table, especially with (potentially ephemeral) machine generated tags. Anecdotaly, tagging attributes and relationships on a large library is a good way to DIY big-ish data, as my HD learned.

peterburkimsher 9y ago
What size is the big-ish data that you're talking about? iTunes slows down noticeably over 30k songs. My Facebook friends database is small by comparison (under 6000), but if I want to include all their Liked Pages, the database size increases to 223,783. The nested structure is really important to me. For example, my friend David is in the lists: Taiwan, Christians, Health & Medicine, KGCC Church, KTL Christian student group. The KGCC and KTL playlists are inside the Taiwan folder. I can use Smart Playlists to sort by the year when I added friends, but I want to manually manage some playlists as well. For example, if I go on holiday to New Zealand and add a new friend, then they belong in the New Zealand list, not the Taiwan list, even though the date added is during my time living in Taiwan. Some playlists are not in any country folder e.g. male/female, Limited Profile. The highly nested nature of the database means Excel isn't suitable, and my brief look at SQL tells me that it would quickly become a mess of tables. There must be some kind of database program that supports nested structure, but I haven't found it yet, and that's why I'm asking here.
- dragonwriter 9y ago
  > The highly nested nature of the database means Excel isn't suitable, and my brief look at SQL tells me that it would quickly become a mess of tables. There must be some kind of database program that supports nested structure, but I haven't found it yet, and that's why I'm asking here. Nested structure is easy to represent in SQL, and doesn't take a lot of tables. It can be a little bit awkward to query, and it tends to be inefficient to query, but with a smallish DB of on the order of hundreds of thousands of records with a not really deeply nested structure, that's probably not a big concern in practice. Graph Databases may be a better solution; they should handle the nesting more simply.
  - peterburkimsher 9y ago
    ArangoDB looks like the most popular Graph Database program. The internal data structure model looks fine, and probably quite efficient, but I can't find a GUI that shows playlist folders.
- zdkl 9y ago
  My partialy annotated library of ~200k items & 6M tags is nearing 1Tb disk space total with items tagged a couple hundred times each on average. I feel the mess of tables pain right now, but cannot see a more appropriate classification system than (arbitrarily super- or sub-tagged) tag collections to express the arbitrarily nested, potentially recursive relations. Same as parent, if anyone has a brilliant idea, I'm taking. Edit: See http://www.opencypher.org http://www.opencypher.org

by zdkl 9y ago

peterburkimsher 9y ago

What size is the big-ish data that you're talking about? iTunes slows down noticeably over 30k songs. My Facebook friends database is small by comparison (under 6000), but if I want to include all their Liked Pages, the database size increases to 223,783. The nested structure is really important to me. For example, my friend David is in the lists: Taiwan, Christians, Health & Medicine, KGCC Church, KTL Christian student group. The KGCC and KTL playlists are inside the Taiwan folder. I can use Smart Playlists to sort by the year when I added friends, but I want to manually manage some playlists as well. For example, if I go on holiday to New Zealand and add a new friend, then they belong in the New Zealand list, not the Taiwan list, even though the date added is during my time living in Taiwan. Some playlists are not in any country folder e.g. male/female, Limited Profile. The highly nested nature of the database means Excel isn't suitable, and my brief look at SQL tells me that it would quickly become a mess of tables. There must be some kind of database program that supports nested structure, but I haven't found it yet, and that's why I'm asking here.

dragonwriter 9y ago
> The highly nested nature of the database means Excel isn't suitable, and my brief look at SQL tells me that it would quickly become a mess of tables. There must be some kind of database program that supports nested structure, but I haven't found it yet, and that's why I'm asking here. Nested structure is easy to represent in SQL, and doesn't take a lot of tables. It can be a little bit awkward to query, and it tends to be inefficient to query, but with a smallish DB of on the order of hundreds of thousands of records with a not really deeply nested structure, that's probably not a big concern in practice. Graph Databases may be a better solution; they should handle the nesting more simply.
- peterburkimsher 9y ago
  ArangoDB looks like the most popular Graph Database program. The internal data structure model looks fine, and probably quite efficient, but I can't find a GUI that shows playlist folders.
zdkl 9y ago
My partialy annotated library of ~200k items & 6M tags is nearing 1Tb disk space total with items tagged a couple hundred times each on average. I feel the mess of tables pain right now, but cannot see a more appropriate classification system than (arbitrarily super- or sub-tagged) tag collections to express the arbitrarily nested, potentially recursive relations. Same as parent, if anyone has a brilliant idea, I'm taking. Edit: See http://www.opencypher.org http://www.opencypher.org