4 ms·
You are correct. When you develop a FFI, you solve many of the same issues that are solved in RPC (remote procedure calls). When you convert Lisp objects to C
by kazinator 5y ago
You are correct. When you develop a FFI, you solve many of the same issues that are solved in RPC (remote procedure calls).
When you convert Lisp objects to C objects and vice versa across the call, this is very similar to marshalling and unmarshalling remote procedure call arguments.
The different representations and memory management of the objects are such that the two domains are often like different machines.
If you use FFI in reverse, you could use it to implement RPC. So that is to say, you can register a FFI callback in some C code which will pass some structure into the Lisp image. It gets converted to a Lisp structure. But what is that? Marshaling. The Lisp code could pass it over a network as an S-expression (further marshalling). The other side receives it, converts it to a Lisp structure, and then calls a FFI function into C code, where that becomes a C structure. Thus a RPC was achieved, leveraged by FFI.
The minimum requirements to legitimize the use of "foreign function interface" is that one domain activates a call in another domain, communicating data flows, which undergo a change in data representation.
The consideration of different programming languages is a red herring! Two different languages could have a very similar native representation. For instance, if we have Pascal and C modules directly calling each other because they use the same calling conventions and
data representations like "record of two 32 bit integers", that is not FFI. C calling assembly routines and vice versa isn't FFI.
"Foreign" doesn't so much mean "Foreign language" but "Foreign run-time/VM/data model.". Under the hood it might all be x86 machine code, which is the same language!
Typically FFI denotes the coupling of some managed environment with something closer to the metal, but other situations/combinations can fit the concept.
(Claimer: I've authored a well-developed FFI, as well as more than one remote procedure implementations, with data-description-driven marshaling/unmarshaling of binary objects. So you don't have to take anything here with any salt.)
- Kinrany 5y agoThere should be an interface description language that works equally well for FFI and RPC. It could be agnostic of the transport mechanism and the type system used to describe messages, and merely enumerate possible methods in a hierarchical (and composable) way.
- kazinator 5y agoWhere we see a difference between the two is that FFIs are designed to be able to directly call all sorts of uncooperative code which is oblivious to FFI, without adding any wrapper code to it. FFI languages have a lot of detail because of that, for describing binary types, calling conventions, memory management ownership and such. (You can use the binary types system of the FFI even if you aren't calling any foreign functions; e.g. to parse a header in a binary file.) Whereas in RPC, we usually control both sides of the call. If there is third party code, we will glue to that from the server-side RPC stub. Even if this is a RPC based on a lower level language like C, it doesn't have to have the generality of describing any C type or function; you design your RPC API to the limitations of the RPC system. E.g. if, say, it cannot pass structures by value, you don't do that in the API. If it doesn't handle bit fields in structures, you likewise don't feature such a thing in your API.
- gavinray 5y agoThere is, it's called COM, and Windows as an operating system and ecosystem is built on top of it. (Though COM as a specification is not limited to Windows) The language you use for defining the types is creatively named "Interface Definition Language (IDL)", and the "MIDL" compiler converts the interface definitions into C & C++ types and objects which can be used. Both via in-process/inter-process function calls, or via RPC/DCOM (Distributed COM). https://en.wikipedia.org/wiki/Component_Object_Model#DCE/RPC_and_MSRPC https://en.wikipedia.org/wiki/Component_Object_Model#DCE/RPC... "As a cross-language component model, COM relies on an interface definition language, or IDL, to describe the objects and associated functions. The COM IDL is based heavily on the feature-rich DCE/RPC IDL, with object-oriented extensions. Microsoft's own implementation of DCE/RPC, known as MSRPC, is heavily used as the primary inter-process communication mechanism for Windows NT services and internal components, making it an obvious choice of foundation." "DCOM (Distributed COM) extended the reach of COM from merely supporting a single user with separate applications communicating on the Windows desktop, to activating objects running under different security contexts, and on different machines across the network. With this were added necessary features for configuring which users have authority to create, activate and call objects, for identifying the calling user, as well as specifying required encryption for security of calls." If you want to learn more about this (it's an incredibly powerful concept that has a wide range of uses and is very much still used today), my recommendation would be: 1. The book: "Essential COM (1997)" by Don Box, one of the architects of COM at Microsoft in the 90s https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465 2. The online tutorial "COM in plain C", which is one of the only (it might even be, like, THE only) resources for learning how COM actually works, since generally you use C++ frameworks to interact with it and they cover up all the details. https://www.codeproject.com/Articles/13601/COM-in-plain-C --------- Disclaimer: I'm no expert in COM myself, I'm also working through these same learning materials. I have a strong interest in FFI/Interop between languages and so have been working through things like "What is an ABI, conceptually?", understanding the C++ Itanium ABI, reading about vtable's, manually emulating C++ classes and class-inheritance with C structs of function pointers (vtables) etc. COM is foundational here so it's the thing on my list atm. Hope you find this useful!