4 ms·
Looks like a good idea overall. Getting a format like this widely accepted would be tricky. Good tools that use it would certainly help. A few comments/quibble
by ggchappell 13y ago
Looks like a good idea overall. Getting a format like this widely accepted would be tricky. Good tools that use it would certainly help.
A few comments/quibbles concerning the format itself:
(1) It think the realities of the modern world are such that you'll need to embrace Unicode a bit more warmly and deal robustly with files containing non-ASCII characters. (Some of those ordinary text editors you want to use will do UTF-8 transparently.)
I would suggest:
- Officially specifying UTF-8 as the default character encoding, to be used if no character encoding is specified with some other mechanism.
- Allowing specification of a different encoding within the file format. (You already have an HTTP-style header; you could use a Content-Type line to specify encoding in the same way HTTP does.)
(2) You want to be editable using a standard text editor, but you allow null-terminated data as a type. You can't do both of those.
(3) You're using "," as a separator with lower precedence than ";". That goes against common usage in both programming languages and English text. Also your header, following the HTTP convention, is "key: value".
Putting those together, doesn't it make more sense to do something like this:
Field_name:Type,Type; 2nd_field_name:Type
Etc.
(The above looks more natural to me, anyway.)
(4) Your escaping looks odd. The explanation about awk doesn't quite do it for me. Why the strings of "$" and "%"?
To put it differently, why not use escapes in a more traditional style, something like the following?
; %;
, %,
: %:
% %%
- jimktrains2 13y agoI wanted the format to degrade to CSV which is why I chose the ordering I did. As for the escaping that's because I wanted to be able to escape a comma without using a comma. I want to be able to use sed or awk without having to worry a field having a comma. The pascal and null terminated values are for binary datatypes, which wouldn't be editable in a text editor (by muggles anyway). That said I'm not sure why I thought storing a string in a binary field was a sane idea. I've been debating using the header to store field metadata. I do agree that it'll be cleaner. Also, it would alleviate most of the issues with the odd seperator precedence. I also agree that it should default to utf8 friendly strings by default. I'll change that in the spec. Thanks for your feedback; I greatly appreciate it.
- jimktrains2 13y agoI think I do like using the headers to define fields better: http://jimkeener.com/posts/TDFrev3 http://jimkeener.com/posts/TDFrev3 That allows it to fall back to being a simple CSV if all the headers are removed. It looks a lot better and just becomes more readable I think. I also simplified the types a bit (most noticeably removed the null-terminated and Pascal strings).