5 ms·

Since Joplin's developer decided to put all Markdown files into a database and thus give up all the advantages of a flat file system, I have uninstalled Joplin.

by Ringz 3y ago

Since Joplin's developer decided to put all Markdown files into a database and thus give up all the advantages of a flat file system, I have uninstalled Joplin. In my view, the last discussion in Joplin's forum on this topic further discredited the developer.

burnished 3y ago
That seems an odd take given the content of the discussion: https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-now/17003/77 https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-... Especially since it looks like a solution via plugin was identified? Thanks for bringing it up though, I went for spicy programmer drama but instead found a somewhat interesting and incredibly polite disagreement. I am curious about what happened that they have such a solid opinion about acceptable use of file metadata.
- MezzoDelCammin 3y ago
  Thanks for bringing this one up. It's almost strange to see a civilized discussion about something that clearly polarizes a lot of people in the thread. Makes me wonder exactly what kind of social dynamic results in a bunch of people actually talking through their differences, instead of the these days usual trench warfare...
- fauigerzigerk 3y ago
  >incredibly polite disagreement Not sure I agree. Here's one comment by the developer (laurent): "Ok, there's so much wrong in what you write about that it's hard to know where to start. I guess it would help if you had a clue about what you're talking about." https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-now/17003/10 https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-...
  - wiseowise 3y ago
    Ouch.
  - laurent123456 3y ago
    I'm not proud of that answer but to be fair there are also more balanced views in this thread. It's also an old thread and some of what's in there is no longer relevant. For example someone recently created a prototype of the app that saves the notes to disk, and although we can't merge his work as it is, we plan to investigate and see whether it can be somehow integrated. It's one of these features that, all things being equal, we'd rather have than not have, but it's obviously not trivial to add it when the app wasn't originally designed for it.
    - fauigerzigerk 3y ago
      I completely understand. It wasn't my intention to blow this out of proportion. It's a single expression of frustration that every developer can relate to.
  - burnished 3y ago
    Sure - but it is a departure in tone and sandwiched by more normal and productive comments (from that dev). It isnt ideal but it is also a pretty normal and minor human mistake (growing frustrated and then short to be specific). And the previous comments were claiming that this project's data was 'proprietary' and not 'open' and the situation was inexcusable, basically because they assumed a database was this big scary thing (as opposed to an open book). That is a frustrating experience, and yeah maybe indicates that the person telling you how to write software doesn't know what they are talking about (for the folks following along at home, those frivolous cries of foul play were distracting from the real need they had for plain text over a database, someone else describes it quite succinctly a little later)
    - fauigerzigerk 3y ago
      Sure, it's no big deal and it's completely understandable. I just wouldn't characterise it as "incredibly polite". I think file system vs database storage is very much about trade-offs. Unfortunately the debate descends too quickly into who is right and who is wrong. It's always difficult for developers to talk to users who often really don't know what they are talking about. I know that very well. The only thing you can do as a developer (if you have time and patience) is to translate what users say into the closest thing that does make sense in terms of trade-offs. But nobody is always patient.
      - burnished 3y ago
        Oh! That makes sense, I'm a little jealous then. To me the way that discussion did not crash and burn due to a little friction was incredible
  - darkwater 3y ago
    I applause Laurent's answer. I'm so tired of self-entitled users that don't contribute anything and have no skin in the game and just complain "this should be done as I say because I'm smart". Yes, obviously sometimes maintainer can get something bad and a better solution is easily available, but as app complexity goes up, it's unfair (to not say naive or a worse word) for users assume that developers are clueless and hadn't thought about the problem space before taking the design decision.
tovej 3y ago
Well, if what you need is a flat file system, the json-as-database approach that penx is taking isn't really an improvement. I'm interested in what advantages you have lost. Joplin allows you to export your entire notebook as a directory of markdown files (with front matter), and from a performance, reliability, and replication perspective, a database seems like a much better choice.
- Ringz 3y ago
  I won a lot and didn't lose anything. Obsidian plus a few plugins beats Joplin on every level. And I can edit all notes without redundancy by exporting from a Joplin database with nvim, iA Writer, find, grep, cat and millions of other tools.
  - PurpleRamen 3y ago
    > Obsidian plus a few plugins beats Joplin on every level. Not really. Obsidian has its shares of problems too, and most of them originate from using Markdown. Markdown is a freeform text-format, and works very well for writing text, but it really sucks for data and structured content. Most plugins and features in that area are very brittle and overspecialized, working only well enough in their specific use case. And gosh, Obsidian has really a huge amount of plugins for data-handling. At some point, it was so bad that there were multiple competing task-plugins which broke each other just because they had different formatting for dates.
    - rubslopes 3y ago
      > And gosh, Obsidian has really a huge amount of plugins for data-handling. At some point, it was so bad that there were multiple competing task-plugins which broke each other just because they had different formatting for dates. Depends on how you see it. I think it's great that they have a free market of open-sourced plug-ins that you can donate to. I'm actually impressed of how rarely they break each other.
      - Ringz 3y ago
eviks 3y ago
The linked discussion also lists some huge limitations of the flat markdown file system, how did he discredit himself?
- jauntywundrkind 3y ago
  I loved the TinaCMS blog post on how they still use flat files as their source of truth, but have a level.js database that powers a bunch of their functionality. https://tina.io/blog/Tina-CMS--Leveljs/ https://tina.io/blog/Tina-CMS--Leveljs/ You can blow up the db & regenerate it very quickly. All the data is still flat files. But the advanced functions harvest & are derived from the source of truth. Seems like a perfect compromise. I haven't done it yet but I have a git-auto-cimmit project I intend to use with logseq, that'll help me better keep a record of changes over time. That ability to have a flat file source of truth feels hugely hugely empowering to me, and I have a hard time imagining having anything else.
  - eviks 3y ago
    You can blow one database (level), but not the second one (git) which you also need to sync, and this doesn't cover the issues of changing the data layout, git isn't always great in tracking those (doesn't store renames). Then also all your data is duplicated in a git db and a flat file db (they also don't support smartphones?)
    - jauntywundrkind 3y ago
      Yeah you still need a source of truth, duh. I'm not sure what you are talking about vis a vie "changing the data layout". You can do whatever you want, it's just flat files. Git is great at tracking file renames. Trying to say data is duplicated is basically wrong, as far as I'm concerned. There's a materialized view, which very few people I think would argue breaks the idea of data-nornamization. There is only one data source, the other is a generated view which can be regarded as ephemeral data. I sympathize a lot with the parents perspective that Joplin switching to database backed world is a pain. I think the appeal of well managed flat files, controlled by git, is much more directly manageable & has much broader tooling. I appreciate attempts to raise dissent against this system, but none of your points carry much water for me personally. It strikes me as pretty weird having these litany of basically wrong non-issues to swat away?

by Ringz 3y ago

burnished 3y ago

That seems an odd take given the content of the discussion: https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-now/17003/77 https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-... Especially since it looks like a solution via plugin was identified? Thanks for bringing it up though, I went for spicy programmer drama but instead found a somewhat interesting and incredibly polite disagreement. I am curious about what happened that they have such a solid opinion about acceptable use of file metadata.

MezzoDelCammin 3y ago
Thanks for bringing this one up. It's almost strange to see a civilized discussion about something that clearly polarizes a lot of people in the thread. Makes me wonder exactly what kind of social dynamic results in a bunch of people actually talking through their differences, instead of the these days usual trench warfare...
fauigerzigerk 3y ago
>incredibly polite disagreement Not sure I agree. Here's one comment by the developer (laurent): "Ok, there's so much wrong in what you write about that it's hard to know where to start. I guess it would help if you had a clue about what you're talking about." https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-now/17003/10 https://discourse.joplinapp.org/t/why-im-leaving-joplin-for-...
- wiseowise 3y ago
  Ouch.
- laurent123456 3y ago
  I'm not proud of that answer but to be fair there are also more balanced views in this thread. It's also an old thread and some of what's in there is no longer relevant. For example someone recently created a prototype of the app that saves the notes to disk, and although we can't merge his work as it is, we plan to investigate and see whether it can be somehow integrated. It's one of these features that, all things being equal, we'd rather have than not have, but it's obviously not trivial to add it when the app wasn't originally designed for it.
  - fauigerzigerk 3y ago
    I completely understand. It wasn't my intention to blow this out of proportion. It's a single expression of frustration that every developer can relate to.
- burnished 3y ago
  Sure - but it is a departure in tone and sandwiched by more normal and productive comments (from that dev). It isnt ideal but it is also a pretty normal and minor human mistake (growing frustrated and then short to be specific). And the previous comments were claiming that this project's data was 'proprietary' and not 'open' and the situation was inexcusable, basically because they assumed a database was this big scary thing (as opposed to an open book). That is a frustrating experience, and yeah maybe indicates that the person telling you how to write software doesn't know what they are talking about (for the folks following along at home, those frivolous cries of foul play were distracting from the real need they had for plain text over a database, someone else describes it quite succinctly a little later)
  - fauigerzigerk 3y ago
    Sure, it's no big deal and it's completely understandable. I just wouldn't characterise it as "incredibly polite". I think file system vs database storage is very much about trade-offs. Unfortunately the debate descends too quickly into who is right and who is wrong. It's always difficult for developers to talk to users who often really don't know what they are talking about. I know that very well. The only thing you can do as a developer (if you have time and patience) is to translate what users say into the closest thing that does make sense in terms of trade-offs. But nobody is always patient.
    - burnished 3y ago
      Oh! That makes sense, I'm a little jealous then. To me the way that discussion did not crash and burn due to a little friction was incredible
- darkwater 3y ago
  I applause Laurent's answer. I'm so tired of self-entitled users that don't contribute anything and have no skin in the game and just complain "this should be done as I say because I'm smart". Yes, obviously sometimes maintainer can get something bad and a better solution is easily available, but as app complexity goes up, it's unfair (to not say naive or a worse word) for users assume that developers are clueless and hadn't thought about the problem space before taking the design decision.

tovej 3y ago

Well, if what you need is a flat file system, the json-as-database approach that penx is taking isn't really an improvement. I'm interested in what advantages you have lost. Joplin allows you to export your entire notebook as a directory of markdown files (with front matter), and from a performance, reliability, and replication perspective, a database seems like a much better choice.

Ringz 3y ago
I won a lot and didn't lose anything. Obsidian plus a few plugins beats Joplin on every level. And I can edit all notes without redundancy by exporting from a Joplin database with nvim, iA Writer, find, grep, cat and millions of other tools.
- PurpleRamen 3y ago
  > Obsidian plus a few plugins beats Joplin on every level. Not really. Obsidian has its shares of problems too, and most of them originate from using Markdown. Markdown is a freeform text-format, and works very well for writing text, but it really sucks for data and structured content. Most plugins and features in that area are very brittle and overspecialized, working only well enough in their specific use case. And gosh, Obsidian has really a huge amount of plugins for data-handling. At some point, it was so bad that there were multiple competing task-plugins which broke each other just because they had different formatting for dates.
  - rubslopes 3y ago
    > And gosh, Obsidian has really a huge amount of plugins for data-handling. At some point, it was so bad that there were multiple competing task-plugins which broke each other just because they had different formatting for dates. Depends on how you see it. I think it's great that they have a free market of open-sourced plug-ins that you can donate to. I'm actually impressed of how rarely they break each other.
    - Ringz 3y ago

eviks 3y ago

The linked discussion also lists some huge limitations of the flat markdown file system, how did he discredit himself?

jauntywundrkind 3y ago
I loved the TinaCMS blog post on how they still use flat files as their source of truth, but have a level.js database that powers a bunch of their functionality. https://tina.io/blog/Tina-CMS--Leveljs/ https://tina.io/blog/Tina-CMS--Leveljs/ You can blow up the db & regenerate it very quickly. All the data is still flat files. But the advanced functions harvest & are derived from the source of truth. Seems like a perfect compromise. I haven't done it yet but I have a git-auto-cimmit project I intend to use with logseq, that'll help me better keep a record of changes over time. That ability to have a flat file source of truth feels hugely hugely empowering to me, and I have a hard time imagining having anything else.
- eviks 3y ago
  You can blow one database (level), but not the second one (git) which you also need to sync, and this doesn't cover the issues of changing the data layout, git isn't always great in tracking those (doesn't store renames). Then also all your data is duplicated in a git db and a flat file db (they also don't support smartphones?)
  - jauntywundrkind 3y ago
    Yeah you still need a source of truth, duh. I'm not sure what you are talking about vis a vie "changing the data layout". You can do whatever you want, it's just flat files. Git is great at tracking file renames. Trying to say data is duplicated is basically wrong, as far as I'm concerned. There's a materialized view, which very few people I think would argue breaks the idea of data-nornamization. There is only one data source, the other is a generated view which can be regarded as ephemeral data. I sympathize a lot with the parents perspective that Joplin switching to database backed world is a pain. I think the appeal of well managed flat files, controlled by git, is much more directly manageable & has much broader tooling. I appreciate attempts to raise dissent against this system, but none of your points carry much water for me personally. It strikes me as pretty weird having these litany of basically wrong non-issues to swat away?