4 ms·

I can't find a really important detail on the website or github - and haven't been able to easily find it - how is the underlying data stored? It looks like it

by ydant 5y ago

I can't find a really important detail on the website or github - and haven't been able to easily find it - how is the underlying data stored? It looks like it might be a custom format, which might make sense for the project, but is unfortunate for a broader accessible knowledge-base and programmable integration - plain text allows you "grep", sqlite allows you SQL. The discussion of this being an org-mode mindset kind of GUI definitely appeals to me. The arbitrary hierarchy of data/knowledge in org-mode works really well for me. I tried a bunch of note-taking apps until org-mode, and it was the second one that just "worked" for me (Trilium Notes and Obsidian were the previous best options for different reasons). But... I'm an Emacs convert just for the purposes of org-mode, and I'm finding a lot of Emacs specific quirks and issues that I don't really want to spend time fixing - but customizing until the quirks are your own preferred quirks seems to be the entire culture of Emacs.

quag 5y ago
The data is stored in a custom binary file format. Parts of the format are zlib deflated. There is documentation for the format, but it is wrong and outdated. I had to reverse engineer the format from the C++ code.
- Aardappel 5y ago
  I am not aware that it is outdated. If you found inaccuracies, please open an issue, or better a yet a PR for the file https://github.com/aardappel/treesheets/blob/master/TS/docs/file_format_spec.txt https://github.com/aardappel/treesheets/blob/master/TS/docs/...
  - quag 5y ago
    By the way! Thanks for Treesheets I love how naturally it helps you when thinking through a topic. I built a Python treesheets cts file to HTML converter so I could quickly see my treesheets along side other notes in my notebooks. Issue created: https://github.com/aardappel/treesheets/issues/185 https://github.com/aardappel/treesheets/issues/185 My (partial) parser in Python: https://gist.github.com/quag/e219f69670cd395d4a59a392557df284 https://gist.github.com/quag/e219f69670cd395d4a59a392557df28... If people would like the current format documented and simple test files to parse please let me know. It’s a 1-2 day task to get it to the point were I wouldn’t have had to read the code to write a parser.
    - Aardappel 5y ago
      Thanks!
Aardappel 5y ago
Yes, it is a custom binary format, for speed and compactness. It does however export to many formats, xml, html, text, csv etc. There's even an example JSON exporter for the scripting language. More could be added. There's not really a standard tree file format that would allow storage that can interop with other formats. Yes, there's of course JSON and XML, but you'd end up with a file with so many custom fields it is not interoperable. And these formats are extremely inefficient compared to the current binary format. The binary format is also able to store images inline.
klysm 5y ago
I’m honestly not sure how I would even start to try to cram this into SQLite in a way that wasn’t awful.

by ydant 5y ago

quag 5y ago

The data is stored in a custom binary file format. Parts of the format are zlib deflated. There is documentation for the format, but it is wrong and outdated. I had to reverse engineer the format from the C++ code.

Aardappel 5y ago
I am not aware that it is outdated. If you found inaccuracies, please open an issue, or better a yet a PR for the file https://github.com/aardappel/treesheets/blob/master/TS/docs/file_format_spec.txt https://github.com/aardappel/treesheets/blob/master/TS/docs/...
- quag 5y ago
  By the way! Thanks for Treesheets I love how naturally it helps you when thinking through a topic. I built a Python treesheets cts file to HTML converter so I could quickly see my treesheets along side other notes in my notebooks. Issue created: https://github.com/aardappel/treesheets/issues/185 https://github.com/aardappel/treesheets/issues/185 My (partial) parser in Python: https://gist.github.com/quag/e219f69670cd395d4a59a392557df284 https://gist.github.com/quag/e219f69670cd395d4a59a392557df28... If people would like the current format documented and simple test files to parse please let me know. It’s a 1-2 day task to get it to the point were I wouldn’t have had to read the code to write a parser.
  - Aardappel 5y ago
    Thanks!

Aardappel 5y ago

Yes, it is a custom binary format, for speed and compactness. It does however export to many formats, xml, html, text, csv etc. There's even an example JSON exporter for the scripting language. More could be added. There's not really a standard tree file format that would allow storage that can interop with other formats. Yes, there's of course JSON and XML, but you'd end up with a file with so many custom fields it is not interoperable. And these formats are extremely inefficient compared to the current binary format. The binary format is also able to store images inline.

klysm 5y ago

I’m honestly not sure how I would even start to try to cram this into SQLite in a way that wasn’t awful.