6 ms·
A Critique of the Cap'n Proto Schema Language (2019)
- CoastalCoder 3y ago2019
- signa11 3y agoyes ?
- eesmith 3y agoKnowing that it's from a few years ago gives helpful information to the potential reader. 1) Someone may have read the essay before, remembered the point, but forgotten the exact title. It has, after all, been posted on HN twice before. Simply reading the comment that it's from 4 years ago may be enough to confirm that it's the same article, as it did with me. While the essay has the date right at the top, in general there is no consistency about where people post a date, or if there is one at all. I know that I sometimes skim the HN comments first rather than follow a page link because then I don't need to worry about cookie notices, requests to subscribe to a newsletter, etc. This essay does not have those, but enough do that it's a general nuisance. 2) Software changes over time. Cap'n Proto's Schema language may have changed since then - perhaps as a response to this essay - making this essay of historic interest only. While that is not likely the case, it is an example of how having the date may provide helpful information. This is useful enough that it is accepted practice to add the date to older information. For example, the previous two HN submissions to this essay both include (2019) in the title. See https://news.ycombinator.com/from?site=zenhack.net https://news.ycombinator.com/from?site=zenhack.net
- gjvc 3y agoKnowing that it's from a few years ago gives helpful information to the potential reader. because it's so difficult to find the date of publication before people waste their precious time goofing off from their overpaid jobs.
- orhmeh09 3y agoThe submission guidelines contain no such allowance. They specify a very limited set of cases where the title of a submission can be modified from the web page title.
- CoastalCoder 3y agoThanks. I wasn't aware that putting a date in the title might violate the guidelines.
- eesmith 3y agoIt does not. You can see dang, one of the moderators, write: "It's the convention on HN to add the year to a title when the article isn't from this year." at https://news.ycombinator.com/item?id=27867435 https://news.ycombinator.com/item?id=27867435 , write "On HN, it's the convention to add the year when a story is more than a year and a bit old." at https://news.ycombinator.com/item?id=16306830 https://news.ycombinator.com/item?id=16306830 and write "It's just the convention to add the year in parens (and for commenters to mention when we missed one)." at https://news.ycombinator.com/item?id=8244765 https://news.ycombinator.com/item?id=8244765 . Found via https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22dang%22%20%22add%20the%20year%22&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... . If orhmeh09 has issue with how to interpret the submission guidelines, I suggest contacting hn@ycombinator.com to resolve the issue.
- CoastalCoder 3y agoThanks for clarifying the rules. In retrospect, my original comment was probably too terse. It's not obvious if I was suggesting that "(2019)" be added to the title, or if I was claiming that the article was outdated for some reason. That's my fault.
- orhmeh09 3y agoThe guidelines are here. It's unreasonable to expect people to search for them in a comment. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- CoastalCoder 3y agoYeah, that ^^^^ was pretty much my reasoning. Thanks for articulating it so clearly. Honestly I didn't expect my original comment to be controversial.
- scrame 3y agodoes that invalidate the criticisms?
- gary_0 3y agoIt's common on HN to note the date for reposts. The mods will usually add the year to the title if it's not included. So that's probably all that CoastalCoder was signaling.
- orhmeh09 3y agoUsers are not advised to do this themselves. The submission guidelines contain no such allowance. They specify a very limited set of cases where the title of a submission can be modified from the web page title.
- eesmith 3y agoThis is a long-established convention to include the year if the article is a year and a bit old. See dang, one of the moderators, affirm this multiple times at https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=dang%20convention%20%22the%20year%22&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... , most recently 17 days ago.
- orhmeh09 3y agoThe guidelines are here. It's unreasonable to expect people to search for them in a comment. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- eesmith 3y agoGuidelines are not strict rules, as Captain Barbossa pointed out, but let's assume they are. Since moderators have promoted this practice for at least a decade, it's much more likely your interpretation of the guidelines which is wrong. Any moderately good rules lawyer would point out the guideline is "please use the original title" not "please use only the original title" nor the even stronger "use only the original title", which turns the polite request into a requirement. Adding the year is not editorializing. Nor is adding "[video]" or "[pdf]", nor even adding the conference name and year as https://news.ycombinator.com/item?id=37216895 https://news.ycombinator.com/item?id=37216895 did 11 hours ago. Since there appears to be a conflict between your understanding of the guidelines and that of the moderators, you should take it up with them at hn@ycombinator.com in hopes of resolving the issue. After all, the guidelines say "If you're worried about abuse, email hn@ycombinator.com and we'll look at the data." ;)
- hlieberman 3y agoRest in peace, Ian, my friend.
- muxator 3y agoFor context: it seems that the author passed away on last July https://www.winchesteruu.org/2023/07/25/joys-sorrows-and-transitions/ https://www.winchesteruu.org/2023/07/25/joys-sorrows-and-tra...
- gadders 3y agohttps://us.firenews.video/us-news/boston-man-dead-art-festival-vermont-necn/ https://us.firenews.video/us-news/boston-man-dead-art-festiv...
- tom_ 3y ago"Last" in this context refers to the year. Last July would be July 2022; July 2023 is this July. July 2023 will become last July at the start of 2024.
- muxator 3y agoThank you. Sorry for my bad English. "This July" from now on :)
- appleflaxen 3y agoOMG. I'm so sorry to see this news! RIP zenhack
- ardel95 3y agoI think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above performance.
- signa11 3y agohmm, i don’t understand how schema-languages can do anything about that though. afterall, you just serialize/de-serialize based on provided inputs, GIGO if you will. safety w.r.t bad/malicious inputs should be a ‘higher level’ concern afaik.
- jiggawatts 3y agoSafety means: garbage in, error out.
- signa11 3y agoright, and it should not be left to the serialization layer for that.
- afiori 3y agoit depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.
- seangrogg 3y agoIt feels like a check against an input size of 10MB is something you would do well before deserialization, no?
- klabb3 3y agoGreat post, digestible without experience in capn proto specifics. I really like that they talk in terms of “features carrying their own weight” and survey real world code to see what cases features solve for, and how (surprisingly little) certain features are actually used. A side effect of dropping features when designing for code generation (in this case) is that it makes things more concise for everyone else as well, including the users. There are generally no easy answers, but as a principle I think the burden of proof for new features should be high. “Oh it’s nice” is not enough, unless it’s trivial syntactic sugar. It should really be at least a strong need and ideally multiple needs per feature, so to say.
- posix86 3y agoPremature optimization is the root of all evil - again. Applies to performance and features. Unless you can proove (not formally, but for yourself) that feature will make things better, don't implement it.
- jffhn 3y ago>Cap’n Proto unions are not first class types. Instead, they are fields of structs In some binary data description format, I didn't include unions because they are just a special case of what I called "conditional sequences". But then I added them, because it is very difficult or impossible to figure out whether a set of conditional sequences corresponds to a union (all must have same size, which is only known at encoding/decoding, and only one must be present), which you need to know when converting into more restrictive formats that only support unions.
- umvi 3y agoWhy do all these serialization frameworks like protobufs, flatbuffers, capnproto, etc. have bespoke schema languages? Why not just use JSON-based schema so that you don't need custom parsers for it? It would definitely make metaprogramming easier as well.
- kentonv 3y agoBecause JSON-based schemas are excessively verbose making them painful to read and write, to the point where really nobody wants to do so. Consider: struct Person { name @0 :Text; age @1 :UInt16; } vs.: "declarations": [ { "name": "Person", "kind": "struct", "fields": [ { "name": "name", "type": "text", "ordinal": 0 }, { "name": "age", "type": "uint16", "ordinal": 1 } ] } ]
- zamalek 3y agoAlso JSON schemas can represent the exact same type in multiple ways, meaning some form of normalization is required prior.
- thayne 3y agoAvro uses json for its schema language.
- never_inline 3y agoBut also has a schema language, and a avro.reflect which can construct schema from java class. Avro schema in Json is very unreadable in my experience. Another bad thing is something like ["int", "null"] gets formatted over 2 lines by standard JSON formatters, making it even harder to read.
- thayne 3y ago> But also has a schema language, Really? I admittedly don't know a whole lot about avro, but everything I've been able to find about avro schemas uses json. Edit: I found it: https://avro.apache.org/docs/1.11.1/idl-language/ https://avro.apache.org/docs/1.11.1/idl-language/