4 ms·

I've discussed a similar concern in another hackernews post, though there I suggested improvements to the C language itself since it's the one standard everyone

I've discussed a similar concern in another hackernews post, though there I suggested improvements to the C language itself since it's the one standard everyone is somewhat forced to follow. The problem with such an ABI is getting people to use it, there's no single entity with enough power to enforce it. I've thought of tackling the problem in another direction, making a simple C-like language (but slightly higher level) which uses the C ABI but with proper support for slices/strings/interfaces/errors/etc. and automatically produces wrapper code for mainstream languages (C++, Java, C#, Swift). The wrappers wouldn't be idiomatic code in the host language, but they'd be directly usable, whereas right now wrapping a "C" library (even when produced by a higher level language) requires recovering all of the semantic information in an additional, library-dependent high-level wrapper. For example, some SMT solvers expose quite nice C APIs, but to use them in say... Python, you'd need to use ctypes and a lot of low level code, which is why they go out of their way to expose wrapper APIs. Some solvers like Yices2 have really low level Python wrappers, which mostly take care of the memory management, unpacking lists into a pointer and a length, and string conversions. There's no reason that couldn't be automated with a slightly higher-level than C language.

estebank 5y ago
Focusing on the "you have now N + 1 standards" problem, I think following the Language Server strategy is the way to go: you don't force people to support it, you focus on specifying an API that support a handful of representative bridges (let's say Python <-> Swift <-> Java <-> Haskell), with a builtin method to evolve and then let people realize that supporting this new ABI immediately brings support for every other language that supports it in an idiomatic-ish way.
- sirwhinesalot 5y ago
  Yes, the wrapper generators would not be built into the compiler, they're just little pieces of software that generate the relevant code (anyone could develop one in their own language in a couple of days). The language exists merely as a way of directly interfacing with the ABI, a way to get the libraries out the door so interest in the ABI exists (same as how VSCode made LSP popular). The problem with doing Python <-> Swift is that you cannot (and should not) try to expose the entirety of Python and Swift to either side. So you need an "extern ABI" solution to expose your Swift code, except this is not built in to the Swift compiler so now you are in trouble. If you go the external route you end up with SWIG, which is something you only use when you have no other option (CVC4 moved away from it for example). Having a language that directly supports the ABI sidesteps this issue. All code written in the language works with the ABI directly so you get the wrappers for free without thinking about how to implement them. A lot of libraries are written in C directly for this very reason! Once it becomes popular enough, languages would implement "extern ABI" directly, and can now talk to each other over it without the middleman. Same as how a compiler with LSP support can be used in editors that are not VSCode.
pjmlp 5y ago
> The problem with such an ABI is getting people to use it, there's no single entity with enough power to enforce it. There is on Apple (XPC), Microsoft (COM/UWP) and Google (JNI, Android IPC, Fuchsia IDL), IBM (TIMI / Language Environments). The problem are UNIX FOSS clones that lack such overlords.
- sirwhinesalot 5y ago
  Those OS solutions aren't cross platform (obviously) so they're only used reluctantly for the smallest amount of code necessary to interface with the platform. Microsoft went out of their way to support WinRT in Rust because sure as heck nobody else was going to. That said, if IPC solutions count (which to me they don't), then the Linux standard would be D-Bus.
  - pjmlp 5y ago
    For me they count, because it is a way to improve overall application security. icrosoft went out of their way to support WinRT in Rust, because they want to use Rust themselves, it wasn't out of love. Same applies with Google's support for Android IPC and Fuchsia IDL for Rust. Naturally the platforms with headless management like UNIX FOSS clones will never adopt those solutions at scale, hence why nobody cares about D-BUS.
    - sirwhinesalot 5y ago
      Oh don't get me wrong, for platform APIs IPC is probably the way to go. Microkernels are built entirely around IPC after all. But I don't want to have to start a separate process and serialize data across a socket to call functions in every library that's packaged with my software, it's too expensive. Even if the same IPC approach is used in-process and we accept the serialization costs (which is fine), why would library X use this approach and not another one? Specially if it needs to also run on macOS? Different IPC solutions per platform? Nah. My point about Microsoft is that, had they not supported Rust themselves, it's likely nobody would have bothered with Rust projection for WinRT. That shows that even the OS devs lack the "strength" to enforce these things. C is the glue everyone uses and will continue to be so for the foreseeable future.

estebank 5y ago

Focusing on the "you have now N + 1 standards" problem, I think following the Language Server strategy is the way to go: you don't force people to support it, you focus on specifying an API that support a handful of representative bridges (let's say Python <-> Swift <-> Java <-> Haskell), with a builtin method to evolve and then let people realize that supporting this new ABI immediately brings support for every other language that supports it in an idiomatic-ish way.

sirwhinesalot 5y ago
Yes, the wrapper generators would not be built into the compiler, they're just little pieces of software that generate the relevant code (anyone could develop one in their own language in a couple of days). The language exists merely as a way of directly interfacing with the ABI, a way to get the libraries out the door so interest in the ABI exists (same as how VSCode made LSP popular). The problem with doing Python <-> Swift is that you cannot (and should not) try to expose the entirety of Python and Swift to either side. So you need an "extern ABI" solution to expose your Swift code, except this is not built in to the Swift compiler so now you are in trouble. If you go the external route you end up with SWIG, which is something you only use when you have no other option (CVC4 moved away from it for example). Having a language that directly supports the ABI sidesteps this issue. All code written in the language works with the ABI directly so you get the wrappers for free without thinking about how to implement them. A lot of libraries are written in C directly for this very reason! Once it becomes popular enough, languages would implement "extern ABI" directly, and can now talk to each other over it without the middleman. Same as how a compiler with LSP support can be used in editors that are not VSCode.

pjmlp 5y ago

> The problem with such an ABI is getting people to use it, there's no single entity with enough power to enforce it. There is on Apple (XPC), Microsoft (COM/UWP) and Google (JNI, Android IPC, Fuchsia IDL), IBM (TIMI / Language Environments). The problem are UNIX FOSS clones that lack such overlords.

sirwhinesalot 5y ago
Those OS solutions aren't cross platform (obviously) so they're only used reluctantly for the smallest amount of code necessary to interface with the platform. Microsoft went out of their way to support WinRT in Rust because sure as heck nobody else was going to. That said, if IPC solutions count (which to me they don't), then the Linux standard would be D-Bus.
- pjmlp 5y ago
  For me they count, because it is a way to improve overall application security. icrosoft went out of their way to support WinRT in Rust, because they want to use Rust themselves, it wasn't out of love. Same applies with Google's support for Android IPC and Fuchsia IDL for Rust. Naturally the platforms with headless management like UNIX FOSS clones will never adopt those solutions at scale, hence why nobody cares about D-BUS.
  - sirwhinesalot 5y ago
    Oh don't get me wrong, for platform APIs IPC is probably the way to go. Microkernels are built entirely around IPC after all. But I don't want to have to start a separate process and serialize data across a socket to call functions in every library that's packaged with my software, it's too expensive. Even if the same IPC approach is used in-process and we accept the serialization costs (which is fine), why would library X use this approach and not another one? Specially if it needs to also run on macOS? Different IPC solutions per platform? Nah. My point about Microsoft is that, had they not supported Rust themselves, it's likely nobody would have bothered with Rust projection for WinRT. That shows that even the OS devs lack the "strength" to enforce these things. C is the glue everyone uses and will continue to be so for the foreseeable future.