3 ms·

Downloading the resulting files is a really strong argument against the original "dash encoding" as proposed, because if I try to download the example /legislat

Downloading the resulting files is a really strong argument against the original "dash encoding" as proposed, because if I try to download the example /legislators/legislator_terms-/1.json path I'll end up with a file named `1.json` on my filesystem even though I'm download the file corresponding to the whole `legislator_terms/1` table. Or perhaps if I fetch a URL for the table `foo.json` and I don't add my own extension, I'll be viewing a URL ending in the component `foo-.json` even though it's not JSON data. Percent-encoding really would be the right solution here if only middleware would leave it alone instead of decoding and re-encoding it. In the absence of that, whatever encoding scheme is used must ensure the results do not leave any slashes or periods in the resulting string. As you've pointed out, other encodings (using the base dash character) can still preserve lexicographic ordering so there seems to be no reason to retain the actual period and slash characters.

simonw 5y ago
This is an excellent point that I had not considered, thanks! I'm actually leaning towards a slightly different scheme now that would address this, where I use regular percentage encoding but with the hyphen character instead of the percentage character. I'll write an update if I decide to go with that approach.
- kragen 5y ago
  I look forward to reading it! Your blog is always a pleasure.
- lilyball 5y ago
  Are you intentionally trying to preserve lexicographic ordering? If so, doing percent-encoding-but-with-dash is reasonable and pretty easy to explain. If you don't care about lexicographic ordering then I would recommend picking a different character than dash. Even if Datasette's particular set of tables don't use dash in their name, if you're hoping other people will use this same scheme then using a more esoteric character might be nice. Using &/$/! would still preserve ordering relative to alphabetic characters and are not commonly found in URL paths.
kragen 5y ago
That's a really important point, one I should have thought of. So maybe these changes would be advantageous even in Simon's intended use case. Probably characters that aren't * and | could be friendlier for filenames. I don't know that preserving lexicographical ordering is important, though it might be handy if you're storing things in LevelDB or something. I just have a soft spot for homomorphisms over the free monoid.

simonw 5y ago

This is an excellent point that I had not considered, thanks! I'm actually leaning towards a slightly different scheme now that would address this, where I use regular percentage encoding but with the hyphen character instead of the percentage character. I'll write an update if I decide to go with that approach.

kragen 5y ago
I look forward to reading it! Your blog is always a pleasure.
lilyball 5y ago
Are you intentionally trying to preserve lexicographic ordering? If so, doing percent-encoding-but-with-dash is reasonable and pretty easy to explain. If you don't care about lexicographic ordering then I would recommend picking a different character than dash. Even if Datasette's particular set of tables don't use dash in their name, if you're hoping other people will use this same scheme then using a more esoteric character might be nice. Using &/$/! would still preserve ordering relative to alphabetic characters and are not commonly found in URL paths.

kragen 5y ago

That's a really important point, one I should have thought of. So maybe these changes would be advantageous even in Simon's intended use case. Probably characters that aren't * and | could be friendlier for filenames. I don't know that preserving lexicographical ordering is important, though it might be handy if you're storing things in LevelDB or something. I just have a soft spot for homomorphisms over the free monoid.