5 ms·

I wish! > It's 2019, "someone else has had this problem." Sure, and that format only support 98% of what I need, and they're not interested in adding one new

by ken 7y ago

I wish! > It's 2019, "someone else has had this problem." Sure, and that format only support 98% of what I need, and they're not interested in adding one new feature, or no longer available to maintain the format. Now I have a choice: write my own from scratch, or extend theirs in an unofficial way. Or maybe they are willing and able to make a "1.1" version of their file format, and now 10 other programs that support the 1.0 file format won't support my files consistently, since they were never designed or tested for it. As your LTU link suggests, in many cases: > your solution belongs in a library, not a language Sounds great, but what's a "library", in the world of file formats? It would be fantastic if every file format were extensible in all the ways that were useful to me, but they're not.

defanor 7y ago
> Sounds great, but what's a "library", in the world of file formats? I don't quite understand the linked programming language design advice (i.e., if existence of SKI is a sufficient argument to discard languages based on cleaner models and fewer axioms, it seems that any language can be discarded on step 4 because there are Turing complete languages already), but I think a sensible course of action that would fit the "language versus library" analogy is to use an existing serialization format and/or data model, only defining its schema/ontology (if there is no suitable one). Similarly to separation of layers (particularly presentation and application ones, and perhaps down to transport layer, given the article's advice to include checksums) in network protocols.
- ken 7y ago
  I like that interpretation, though existing file formats don't tend to be implemented with such layers. At best it will make documentation a little bit simpler, but won't help with my implementation. There's no straightforward way, in any language/library/system I've used, to be able to say "take the lexer from file format XYZ, and I'm going to change the data model it produces" -- or vice versa. In theory I thought that's supposed to be what inheritance is about but nobody writes that way, and object-oriented programming isn't magic pixie dust you can sprinkle on any design to make it automatically reusable.
  - carapace 7y ago
    That would be cool. I think some of the PEG parser generators (and Prolog DCGs) can work like that, if you're careful.
  - defanor 7y ago
    > "take the lexer from file format XYZ, and I'm going to change the data model it produces" Apparently we've imagined slightly different things (and with the sibling comment too; perhaps I should have provided examples at once). What I had in mind is rather standard formats and markup languages, which are already used in practice (though not always): such as XML (SVG, OO XML, XHTML being based on it) and JSON; BER and others for standard encoding rules; RDF for a generic data model with various serialization options. That is, not taking a lexer from another format's implementation, but using a standard one. Likewise with integrity checks: many formats already rely on integrity being guaranteed by file systems and network protocols, and don't include checksums; a similar story with compression, and with encryption (both of which some formats would still redefine and include, but often they can be applied separately as well, or at least using an existing standard/algorithm/library).

by ken 7y ago

defanor 7y ago

> Sounds great, but what's a "library", in the world of file formats? I don't quite understand the linked programming language design advice (i.e., if existence of SKI is a sufficient argument to discard languages based on cleaner models and fewer axioms, it seems that any language can be discarded on step 4 because there are Turing complete languages already), but I think a sensible course of action that would fit the "language versus library" analogy is to use an existing serialization format and/or data model, only defining its schema/ontology (if there is no suitable one). Similarly to separation of layers (particularly presentation and application ones, and perhaps down to transport layer, given the article's advice to include checksums) in network protocols.

ken 7y ago
I like that interpretation, though existing file formats don't tend to be implemented with such layers. At best it will make documentation a little bit simpler, but won't help with my implementation. There's no straightforward way, in any language/library/system I've used, to be able to say "take the lexer from file format XYZ, and I'm going to change the data model it produces" -- or vice versa. In theory I thought that's supposed to be what inheritance is about but nobody writes that way, and object-oriented programming isn't magic pixie dust you can sprinkle on any design to make it automatically reusable.
- carapace 7y ago
  That would be cool. I think some of the PEG parser generators (and Prolog DCGs) can work like that, if you're careful.
- defanor 7y ago
  > "take the lexer from file format XYZ, and I'm going to change the data model it produces" Apparently we've imagined slightly different things (and with the sibling comment too; perhaps I should have provided examples at once). What I had in mind is rather standard formats and markup languages, which are already used in practice (though not always): such as XML (SVG, OO XML, XHTML being based on it) and JSON; BER and others for standard encoding rules; RDF for a generic data model with various serialization options. That is, not taking a lexer from another format's implementation, but using a standard one. Likewise with integrity checks: many formats already rely on integrity being guaranteed by file systems and network protocols, and don't include checksums; a similar story with compression, and with encryption (both of which some formats would still redefine and include, but often they can be applied separately as well, or at least using an existing standard/algorithm/library).