3 ms·

Is kdbx broken or has it been causing data loss? I've been using KeepassXC as long as it has existed and no issues for me thus far. If kdbx is not problematic

by Bender 9mo ago

Is kdbx broken or has it been causing data loss? I've been using KeepassXC as long as it has existed and no issues for me thus far. If kdbx is not problematic for it's intended use then I think moving to SQLite just makes it more hacker friendly which I have no need for. I have no need for other applications extending the use of my passwords. If anything, maybe give people the option to export to SQLite and then use that going forward but keep it entirely optional.

ctoth 9mo ago
I don't want to assume you didn't read the article, but this isn't really about the database engine. It's about the shadow schema that has grown up around the format. The database switch would serve as a flag day to unify things. It won't be a permanent fix, nothing at this scale ever is, and we'll probably need another migration in a few decades. Still worth doing.
- Bender 9mo ago
  I read it and I get it but I have never run into problems with KP using its existing schema. The only reason I could see this debate making sense would be if third parties want to integrate with it otherwise it works perfectly fine. And sure if it's worth doing then make it optional like I said. Have an export as SQLite, MySQL, Postgres, Oracle, DB2 but keep it optional given there is no need otherwise. This is solving a perceived problem that does not exist or creating a problem where there isn't one. As for scale this app is for one person doing one query at a time or saving one password at a time. One person will not be saving millions of passwords or if for some reason they are then they can export to a format that an enterprise solution could take it's place. People have written team / company password managers that can use Oracle, MySQL and Postgres that can track and audit user and team password changes. This is not something that should ever be expected of an individual personal password manager like KeepassXC. This is just my take but I would rather have the maintainers of KP focus on bugs, usability issues, quality of life enhancements which they have been doing great thus far in my opinion. Forcing a schema migration is just asking for trouble and potentially causing bugs that turn some people away from using it or cause people to lose data if they do not have snapshots of their database. Or if forcing a schema change make for damn sure there are many backups in each format and encourage the users to back up all the files to many external encrypted drives and store some of the encrypted devices off-site. Something everyone should be doing regardless.
  - wink 9mo ago
    I've been syncing my kdbx files with syncthing for many years and I have had a couple of inexplicable conflicts. Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know. In any case, the only 2 moving parts were KeePassXC (I think, not sure when I moved on from KeePass) and syncthing. But most importantly, it was easily fixed every time, no broken file - which points me more into a syncthing diverge. Otherwise, no, apparently no issues with kdx itself, or I wouldn't have been able to merge these problems. (which brings me to my pet peeve, a good working merge is missing...)
    - Bender 9mo ago
      Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know. File locks or attempting to overwrite and open database would be a problem regardless of schema format or what application is holding the database file open for writing. This should be expected behavior and one would hope the app would detect this and seqfault / core dump without corrupting the original / previous version of the database file.
      - wink 9mo ago
        Yes, I just felt I had to specify further besides "I encountered problems that didn't seem to be inherent problems" but "no problems" would have been dishonest.
- WorldMaker 9mo ago
  I think that's where I see the most concerns with this proposal: XML is already an extensible format (that's what the X stands for) and XML schema changes should be simpler than needing to run SQL migrations (alter/drop/etc). A switch to SQL doesn't guarantee that schema extensions better align as standards or allow for easier schema modifications. I think it more extends the risk that schema components get ossified and harder to extend. That problem that the current schema doesn't have enough ways to declare custom memory-protected fields outside of user-facing attributes seems just as likely to be replicated and maybe worsened in an SQL schema. Changing the database engine doesn't fix the architecture problems nor schema migration problems. It's certainly a good time to reevaluate the architecture problems and the schema migration problems. But the huge caution I'd suggest here would be look at the ossification complaints about the current XML schema and expect SQL migrations to be worse and plan for worse multi-schema operations and intenser ossification. (Especially because these files are expected to live in a large multi-vendor ecosystem, SQLite schema migration management is going to be much worse than any XML schema management.)
wps 9mo ago
Exporting to a relational db is beside the point. You can very easily just export a CSV file and double click it in DB Browser and have something working. Or you could export the XML document and write a quick importer for it. But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements? All the benefits that I was referring to are for daily usage of a KeePass client, not for a one off export.
- Bender 9mo ago
  But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements? None at all. That is why I am saying it should be entirely optional so most people can just ignore it and those wanting the alternate schema can get their feature request fulfilled.
- kantselovich 9mo ago
  I've glanced over the article - I think using SQLCypher makes sense if it benefits Keepass project internally, meaning makes it easier to implement new features or fix bugs. I do not see how it would benefits end users in any significant way. Sure, you can look inside the database, how many people need that? Dumping the database to CSV is not a good backup, schema changes over time, what was dumped from one version of the app would not work for importing into another version if schema changes. Backup it needs a versioned schema format, which would actually look like KDBX format if implemented in XML :-)

by Bender 9mo ago

ctoth 9mo ago

I don't want to assume you didn't read the article, but this isn't really about the database engine. It's about the shadow schema that has grown up around the format. The database switch would serve as a flag day to unify things. It won't be a permanent fix, nothing at this scale ever is, and we'll probably need another migration in a few decades. Still worth doing.

Bender 9mo ago
I read it and I get it but I have never run into problems with KP using its existing schema. The only reason I could see this debate making sense would be if third parties want to integrate with it otherwise it works perfectly fine. And sure if it's worth doing then make it optional like I said. Have an export as SQLite, MySQL, Postgres, Oracle, DB2 but keep it optional given there is no need otherwise. This is solving a perceived problem that does not exist or creating a problem where there isn't one. As for scale this app is for one person doing one query at a time or saving one password at a time. One person will not be saving millions of passwords or if for some reason they are then they can export to a format that an enterprise solution could take it's place. People have written team / company password managers that can use Oracle, MySQL and Postgres that can track and audit user and team password changes. This is not something that should ever be expected of an individual personal password manager like KeepassXC. This is just my take but I would rather have the maintainers of KP focus on bugs, usability issues, quality of life enhancements which they have been doing great thus far in my opinion. Forcing a schema migration is just asking for trouble and potentially causing bugs that turn some people away from using it or cause people to lose data if they do not have snapshots of their database. Or if forcing a schema change make for damn sure there are many backups in each format and encourage the users to back up all the files to many external encrypted drives and store some of the encrypted devices off-site. Something everyone should be doing regardless.
- wink 9mo ago
  I've been syncing my kdbx files with syncthing for many years and I have had a couple of inexplicable conflicts. Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know. In any case, the only 2 moving parts were KeePassXC (I think, not sure when I moved on from KeePass) and syncthing. But most importantly, it was easily fixed every time, no broken file - which points me more into a syncthing diverge. Otherwise, no, apparently no issues with kdx itself, or I wouldn't have been able to merge these problems. (which brings me to my pet peeve, a good working merge is missing...)
  - Bender 9mo ago
    Most probable reason would be that I ended up editing on the wrong machine while being offline or having it open on another one, so user error. But I don't know. File locks or attempting to overwrite and open database would be a problem regardless of schema format or what application is holding the database file open for writing. This should be expected behavior and one would hope the app would detect this and seqfault / core dump without corrupting the original / previous version of the database file.
    - wink 9mo ago
      Yes, I just felt I had to specify further besides "I encountered problems that didn't seem to be inherent problems" but "no problems" would have been dishonest.
WorldMaker 9mo ago
I think that's where I see the most concerns with this proposal: XML is already an extensible format (that's what the X stands for) and XML schema changes should be simpler than needing to run SQL migrations (alter/drop/etc). A switch to SQL doesn't guarantee that schema extensions better align as standards or allow for easier schema modifications. I think it more extends the risk that schema components get ossified and harder to extend. That problem that the current schema doesn't have enough ways to declare custom memory-protected fields outside of user-facing attributes seems just as likely to be replicated and maybe worsened in an SQL schema. Changing the database engine doesn't fix the architecture problems nor schema migration problems. It's certainly a good time to reevaluate the architecture problems and the schema migration problems. But the huge caution I'd suggest here would be look at the ossification complaints about the current XML schema and expect SQL migrations to be worse and plan for worse multi-schema operations and intenser ossification. (Especially because these files are expected to live in a large multi-vendor ecosystem, SQLite schema migration management is going to be much worse than any XML schema management.)

wps 9mo ago

Exporting to a relational db is beside the point. You can very easily just export a CSV file and double click it in DB Browser and have something working. Or you could export the XML document and write a quick importer for it. But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements? All the benefits that I was referring to are for daily usage of a KeePass client, not for a one off export.

Bender 9mo ago
But genuinely what benefit does it provide other than allowing me to hack at my unencrypted db with sql statements? None at all. That is why I am saying it should be entirely optional so most people can just ignore it and those wanting the alternate schema can get their feature request fulfilled.
kantselovich 9mo ago
I've glanced over the article - I think using SQLCypher makes sense if it benefits Keepass project internally, meaning makes it easier to implement new features or fix bugs. I do not see how it would benefits end users in any significant way. Sure, you can look inside the database, how many people need that? Dumping the database to CSV is not a good backup, schema changes over time, what was dumped from one version of the app would not work for importing into another version if schema changes. Backup it needs a versioned schema format, which would actually look like KDBX format if implemented in XML :-)