4 ms·
The big question is "Would you want to use this in a mission-critical project?" Emscripten lets you write webapps in C, for example, but outside of sharing com
by nostrademons 10y ago
The big question is "Would you want to use this in a mission-critical project?" Emscripten lets you write webapps in C, for example, but outside of sharing common libraries across server/Android/iOS/web clients, few people use it.
Pretty much every professional job I've had used polyglot programming of some sort - when I was in financial software it was mixed Java/C/Fortran numerics, when I founded my first startup it was polyglot Javascript/ActionScript, and at Google it was a large server that was first Python/C++ and then Java/C++. The boundary between languages is always problematic. And the reason for that is because you choose memory layout to make certain trade-offs around the access patterns for that data. Do you inline data structures in a vector, or chase pointers? Do objects carry their type information with them so you can manipulate them dynamically, or do they throw it away at compile-time for greater efficiency? Do you get O(1) string indexing or native UTF-8? Do you allocate on the stack or the heap? Do you copy, borrow, move, or COW?
And then when you go to switch representations to call into another language, you pay a cost to convert all the data structures you might be touching. In many cases, that cost could be more than you saved by using optimized representations in the first place.
- thomaswue 10y agoThe idea of Truffle language interoperability is to avoid switching the representation at the language boundary. Objects carry their type information with them that includes the semantics on how to access their properties. This also allows Truffle to fake up the existence of an object without actually performing allocations (e.g., a table in raw buffer format can be interpreted as an array of objects). This clear separation of logical and physical layout enables efficient data representation in the context of higher level languages that typically suffer from pointer chasing and object header overheads.
- nostrademons 10y agoRight, and my point is that this won't work unless you dictate a common memory layout for all Truffle objects. And if you do that, then you lose the ability to make trade-offs about object representation that depend upon access patterns that only the programmer can know. The whole reason we have different programming languages is because not all domains of computation face the same trade-offs.
- thomaswue 10y agoIt works without common memory layout between Truffle languages. It even simplifies the ability to use diverse physical memory layouts within the same language. The programmer specifies the logical layout based on the semantics of a language. The runtime decides how to map this logical layout onto the physical hardware. It can take into account the trade-offs the programmer decided to choose.
- nostrademons 10y agoRight, and my point is that the reason people continue to use C++ or Rust over JVM languages is because there are some use-cases where the JVM's decision about how to map the logical layout onto physical memory causes unacceptably high memory usage and/or cache misses, or prevents them from taking advantage of clever serialization formats (Cap'n Proto, for example, loses much of its performance benefits on the JVM without the use of sun.misc.unsafe). Will Truffle allow the programmer to override Graal's decisions on this and manually specify memory layout? And if so, how are conversions between different language formats specified? If you already buy the JVM's premise of "just let the compiler do it, we'll figure out the most efficient representation", then this is a non-issue. But there are still programmers out there who believe that Rust or C++ or Go or CPython or whatever presents a better memory layout for the tasks that they wish to accomplish, or language designers who think they can do better than all of the above, and these are the users that Graal+Truffle is trying to win over. What's the story for them?
- thomaswue 10y agoThe default when running static languages like C++ or Rust or Go via Sulong is to stick to the memory layout chosen by the programmer. There is no conversion to typical JVM memory layouts. Nor does running on the JVM has major restrictions for the framework (we are working on a version that does not run on the JVM in the SubstrateVM project). What Graal+Truffle allows is for library writers and language designers to invent new representations without any restrictions on the layout and have them interact with the rest of the system. We want programmers to use the best language for the task. And even combine multiple languages within one program. Foreign objects can be passed around freely as parameters and local variables, but there are restrictions when building combined data structures - e.g., if you have a highly optimized native data structure, it is not possible to install a pointer to a JavaScript object in it without performance loss.
- deleted 10y ago[deleted]
- trishume 10y agoThat's exactly how Truffle's cross-language-interface works though. Instead of paying the high cost of conversion, data stays in its existing representation and it is the interface code that is recompiled to fit. An example is JRuby+Truffle where in its C extensions, pointers to Ruby objects act to the C code as if they are pointers to MRI data structures, but behind the scenes Graal compiles code that accesses them like the JRuby objects that they are. The C-memory to Java GC transition doesn't work as well, but you can get around that by implementing C-memory using the Java GC like the Managed-C paper does, alternative implementations only have to keep the same behaviour, not the same underlying data representation.
- nostrademons 10y agoInteresting. What happens if the native memory layout doesn't support a feature that the extension code requires? For example, what if a JRuby+Truffle script calls into a C library, which allocates a struct and invokes a Ruby callback with it, which then wants to access the fields via reflection? C doesn't normally include type information with its structs, so how would the Ruby code know the memory layout of the object? What if it's not the C extension in your project that does the allocation, but a closed-source third-party library that your C code calls into? Do you need to recompile all source code to generate the appropriate typemaps?
- thomaswue 10y agoYes. Sulong assumes that at least the LLVM bitcode including debug information is available. It does not work with third party machine code binaries where no source information is available.