8 ms·
You know what I want? Schemas. And clear error messages. I want to know beforehand what I can put in a config file and I want a fast and hard failure if what I
by oblio 6y ago
You know what I want? Schemas. And clear error messages.
I want to know beforehand what I can put in a config file and I want a fast and hard failure if what I put in there is not good.
And this should be implemented at the file format parser level, with hooks for apps to add on top of the default behavior, so that every app that implements this format gets these things almost for free.
- already_exists 6y agoLike in Windows where you configure by clicking check boxes that can get disabled if invalid, with tooltips explaining what they do, additional help if you press F1, etc. ? It would be nice if we had such tools.
- pas 6y agoThere's JSONSchema, and there are GUIs for handling/inspecting them.
- zmj 6y agoProtobufs have a text representation.
- andrewg 6y agoYes indeed - it's actually pretty nice. You just define a message for your configuration schema: message Config { repeated Server server = 1; } message Server { string address = 1; int32 port = 2; bool standby = 3; } And then you use the text representation in a config file: # main instance server { address: "127.0.0.1" port: 4567 } # backup instance server { address: "127.0.0.1" port: 9876 standby: true } And load it into a message instance: Config config; google::protobuf::TextFormat::ParseFromString(input, &config);
- kortex 6y agoWow, I use pb's a ton and didn't know this. I'd upvote this twice if I could! It looks oddly like HCL. I wonder...
- atombender 6y agoUnfortunately, it is undocumented and has no formal spec, and this appears to be intentional, with no plans for improvement: https://github.com/protocolbuffers/protobuf/issues/3755 https://github.com/protocolbuffers/protobuf/issues/3755.
- quietbritishjim 6y agoAs of protobuf 3 they also have a canonical JSON representation, which you can access from all the supported languages.
- vlovich123 6y agoHaven’t you described cap’n’proto, protobuf, thrift, flatbuffers etc? I know cap’n’proto also has fantastic support for using the schema for config files. You can just compile any constant as a stand-alone serialized message that you mmap into your code in a safe way. It can’t do complex math and things (at least yet) but you can express lists, dictionaries, and reference other constants, so as a config file replacement I love it. I’ve also found the format to be far more regular and consistent than you get with things like text protobuf (you’re still using the schema language instead of another format)
- quietbritishjim 6y ago> You can just compile any constant as a stand-alone serialized message that you mmap into your code in a safe way. Are you suggesting using a binary format for your config files? I think most people would find that more trouble than a decent text format. > ... than you get with things like text protobuf You can just use protobuf's canonical JSON representation (thought the lack of ability to use comments is annoying).
- LibertyBeta 6y agoI think the parent is typing to say that the data is stored in a map which is read to a proto, etc. Kinda like what GPRC does over HTTP. Which kinda makese sense. The schema gives you a great idea of what "should be", and the typing/errors/etc are understood by the host language.
- vlovich123 6y agoYou store your configuration as plain text in your repository and whatnot. When it comes to deployment you just compile it to a binary file. Cap’n’proto also has plain text and JSON serialization formats if you really want to have your deployed config file be directly human-editable and deserialize from that. I was just noting a very cool feature of having your config written in cap’n’proto and it’s what Cloudflare uses to maintain a bunch of config internally if I read Kenton’s allusions to it correctly.
- deleted 6y ago
- michaelmior 6y agoJSON schemas are available for a number of JSON/YML config formats from JSON Schema Store[0] [0] https://www.schemastore.org/json/ https://www.schemastore.org/json/
- pookeh 6y agoWe had that a decade ago. It was called XML and XML Schema. All IDEs support it. JSON was a huge step backwards in the name of simplicity. And now when we are going to add similar functionality to JSON, something else is going to come out in the name of simplicity (like NestedText).
- thaumaturgy 6y agoMagento is a popular codebase that made XML-based configuration a fundamental part of its architecture. The results were terrible and caused numerous headaches and countless hours lost to trying to troubleshoot inscrutable configuration issues. The Magento 2 codebase began a shift away from XML for configuration, although it still uses some. There may be room for an argument that Magento did XML badly (it did many things badly), but I don't believe I've ever seen XML done well.
- vbezhenar 6y agoI love XML configuration in Spring.
- imtringued 6y agoI don't get it. The @Configuration and @Bean annotations are at least 100 times more readable and powerful than whatever garbage people used to write into their xml files to define beans. 20 lines of xml are often equivalent to like 8 lines of Java and each of those Java lines is shorter than the xml equivalent. Repeating closing tags is not very interesting.
- m463 6y agoI think XML sort of failed simplicity. In a minute I can read and write json from most languages I use. In the same amount of time, I'm still wondering if I should use a tag or attribute in xml. cdata? expat? It's not that xml isn't a good technology. It's that it's not appropriate for general use, especially in comparison to simpler alternatives.
- RobIII 6y ago> You know what I want? Schemas. I can see this work perfectly fine in typed languages like C#: `NestedText.Deserialize<T>("nestedtext")` where the deserialize method handles the actual mapping of nested text objects to `T` by providing the deserializer a class / classes that handles the string -> scalar(s) mapping for the given T. That would, sort of, function as a Schema. I think the only thing, from glancing over the project, that would need to be supported to make this really useful is nested lists/dictionaries. I don't see how this can be done but maybe I'm missing it.
- setr 6y agoYou can always do that, defining the schema in the client to produce sensible checks, even with JSON. The problem is that wherever the spec is underspecified is another place where two different clients can deserialize differently, and both be correct. And the problem with stringly typed systems is that everything is underspecified
- rudolph9 6y agoCheck out https://cuelang.org/ https://cuelang.org/
- verdverm 6y agoCame here to say the same, Cuelang is by far the best config system and paradigm I have tried. All else seems so last century, though Cuelang has its foundation in NLP systems from last century :]
- vitiral 6y agoNever seen this, it's awesome! Might be an improvement over jsonnet, which was my favorite approach
- tootie 6y agoYou want XML from 15 years ago? Yes, me too. Schemas and includes.
- oblio 6y agoI've used XML. I don't want namespaces, I don't want the verbosity, I don't want entities, I don't want the security vulnerabilities. I should have mentioned that I want something simple and readable.
- fn1 6y agoSlightly off-topic, but yes, having fail-fast deserialisation is great. I wrote a json/kotlin-serialisation library once and purposely restricted some json-features to achieve that: 1. Fields can arrive in any order - this is standard 2. Field names are matched case-insensitively - so keyA and keya are the same, because who would use two variables differing only by case. Serialization keeps the original casing of the name. 3. Missing fields throw an error. if they are nullable, they have to be explicitly set to null - so that you can be sure the serialization side upgraded to the latest version of a protocol if a field was added, and things don't just work by chance. 4. Nullable strings are not coerced to empty strings or anything like it. Kotlin is null-safe, so if it's a string, it has to be "". If it's, for whatever reason, a nullable string, you can set it to null. 5. Enums are also serialized case-insensitively - so you an write "keyA": "eNumVaLuE" if you want - typos should not break the code here, no on would you two enums differing only by case. IIRC booleans could also be TRUE, tRuE, truE etc. (but NOT t or f, or yes or no, or 0 or 1 or empty). 6. Superfluous properties are silently ignored. These rules were a great tradeoff for quick development, mixing languages and having fail-fast behavior with a stable protocol. (https://medium.com/@fabianzeindl/generated-json-serialisation-for-kotlin-4470dabe40b8 https://medium.com/@fabianzeindl/generated-json-serialisatio...)