4 ms·
A rant I've wanted to do for a while: We're at the point now where it would be useful to be able to express ABIs and FFI in a language-agnostic manner without
by jcranmer 9y ago
A rant I've wanted to do for a while:
We're at the point now where it would be useful to be able to express ABIs and FFI in a language-agnostic manner without resorting to C. C lacks the capability to describe fairly widespread functionality: things like fixed-size integer types [0], pointer lifetimes, arrays with lengths, functions with bound environments, variable arguments [1]. Keeping distinctions between binary strings and textual strings would be useful for languages, and tracking how to deallocate pointers would be beneficial as well. An ABI that allows systems with different GCs to pass GC information to each other would be amazing, if challenging.
Unfortunately, we're sort of in a catch-22. The multilingual interoperation has to go through C, because that's the only standard that exists. The people who write the standards look at the situation and see no reason to go beyond C because no one's proposing an alternative.
[0] Sort of. There is <stdint.h>, but C tends to fundamentally think in terms of char/short/int/long/long long, and there have been issues with trying to work out what [u]int64_t maps to.
[1] Not in C's broken sense of "there are more arguments after this point, but you get to guess how many." Rather, I'm thinking of a more Java sense of "here's the list of extra arguments the caller gave me."
- adrianratnapala 9y agoI mostly agree. The caveat is that we can still express things like pointer lifetimes and even arrays-with-lengths in English. E.g. libc is full of arrays-with-lengths, implemented as separate params. Don't get me wrong: I think it is great that more powerful languages automate and check those things. But it is also right for ABIs to be two-steps behind so that they don't tie themselves to a single language. And here is the real problem with ABIs and C -- C is a single language. Ages ago C was a decent lingua franca, because C programs translated to machine code in pretty easy to guess ways, and the job of an ABI was just to formalise those guesses. But nowadays we have aggressive optimisers, expansive definitions of undefined behaviour, clever parameter passing conventions, abstract "memory models" defining concurrency rules and all kinds of other stuff that make C something more than just an obvious abstraction of real-world machines.
- mjevans 9y agoWhat sort of properties would you consider a "textual string" to have that a "binary string" would lack? Offhand I'd assume you mean some of the following, but I'm curious if /my/ assumption about your needs is correct and complete. (Also, note that some of these may be implied by values elsewhere in the specification, it isn't strictly a data structure.) * flag:isValidated * targetEncoding * flag:isNormalized * targetNormalization * lengthMemory (not including the implicit size of the type's metadata) * lengthCodepoints (distinct encoding elements) * length(Some word meaning complete display elements, but not the actual width, height, etc.) Do you also need actual random access to individual codepoints/"full characters"? Or is the assumption that if such a scan is in progress the entire sequence or major parts of it are being fully-scanned? What if, within a given function call*, the underlying storage could be referenced with zero-copying and substrings made? (On exiting that context they'd be copied if retained.) It might be useful to have library iterators that split on say, a single given character, segments up to a maximum memory size of N (and scanning backwards for the last "full character" or at least complete rune). Of course, I believe that, by /default/ a program should perform /binary/ interaction with files including standard in/out/err. If there is a process for converting such binary input/output it should assume and produce data in UTF-8 (in NFC normalization) (when converting to/from "text" types) as a default, which should be easy to globally over-ride or set specifically for a given open file (and change mid use). Of course I don't normally need to care about the encoding of data directly; libraries with parsers and writers typically do that as well as other encoding/decoding for that format for me. Everything else tends to be copy / compare and do not change, or making filenames (mostly append bits of "should all be UTF-8"). Actual, hard core, manipulation of encoded streams seems far more like a composition editing issue (A specific editor where default input dialogs are insufficient) or something that underlying libraries need to worry about.
- hsivonen 9y agoIn practice, use cases don't justify strings precalculating their number of code points or grapheme clusters. Pixel width isn't a property of a string alone, since it depends on font and the container to render it in (for line breaks). So a string should know its number of code units. Validity information should be part of the type system and not runtime flags. I.e. you have a different type for arbitrary bytes and for valid UTF-8. (See IDNA for why normalization guarantee in infra over time is problematic.)
- pcwalton 9y agoI mean, what you describe is basically Microsoft COM. But COM ended up being too heavyweight to get much traction outside Windows.
- flukus 9y ago> C lacks the capability to describe fairly widespread functionality: things like fixed-size integer types [0], pointer lifetimes, arrays with lengths, functions with bound environments, variable arguments [1]. Keeping distinctions between binary strings and textual strings would be useful for languages, and tracking how to deallocate pointers would be beneficial as well. An ABI that allows systems with different GCs to pass GC information to each other would be amazing, if challenging. Adding some of those features would mean that every language has to support them and that includes c if you wanted to gain any traction. Part of the reason c is the interop standard is because it lacks those features, the less features you have the easy it is for the rest of the world to inter operate with you.
- pjmlp 9y agoC is the interop standard on OSes written in C, mostly following UNIX ideas. It isn't the interop standard on OSes that have decided to follow other path, namely Windows (COM/.NET), Android (Java), ChromeOS (JavaScript), IBM z (Language Environment), IBM i (ILE), Unisys ClearPath (MCP).
- pjmlp 9y agoThat is COM as mentioned by pcwalton. Other systems would be IBM z and IBM i language environments. The only place where this doesn't catch on are UNIX clones.