4 ms·

Foreign Function Interface is one of the key advantages what C# had over Java (e.g. against huge APIs like Android or macOS). I am really curious how this JEP t

by oaiey 3y ago

Foreign Function Interface is one of the key advantages what C# had over Java (e.g. against huge APIs like Android or macOS). I am really curious how this JEP turns out.

MaxBarraclough 3y ago
For anyone reading, here's a link to the page with detail on this project: https://openjdk.org/jeps/454 https://openjdk.org/jeps/454
soulbadguy 3y ago
The FFI is also an artifact of CLR type system and the ability of have proper stack value based types.
- grishka 3y ago
  They will eventually bring value types to Java with Project Valhalla.
  - layer8 3y ago
    I wonder how long it will take. We’ve been waiting for ten years already.
    - kaba0 3y ago
      One design iteration already has a working, implemented branch available, it’s just one of the hardest designs to get right, to remain backwards compatible, to have it work nice with generics and all the other features, etc. So once they actually settle on a design (which they seem to have closed on quite a lot in the last couple of years), it might not take that long to happen.
    - ivan_gammel 3y ago
      Looks like we may see a first preview in Java 23, i.e. already this year.
  - soulbadguy 3y ago
    Noting that Valhalla value types are about guaranteeing object value identity with memory layout and/or flattening supposedly deriving from that. Modeling unmanaged sequence of bytes within the type system (like receiving a typed buffer from un managed code) might still be challenging
- int_19h 3y ago
  Not just value types, but also: 1. Pointers to the same (and not just references to objects). 2. Unions (via explicit-layout structs) 3. Function pointers. 4. C-style varargs. To be fair, not all of these have been exposed in C# historically even though CLR had them all along. Most notably, unmanaged function pointers took over 20 years. And since most people look at CLR through the prism of C#, they aren't necessarily aware of these features. Still, one way or the other, we're at the point where pretty much any C program can be transpiled to C# in a one-to-one mapping of almost every construct (setjmp/longjmp has to be emulated with exceptions), and a very similar performance profile of the resulting code.
  - neonsunset 3y ago
    Honorable but important mentions: 5. Stackalloc (C alloca), fixed buffers in structs and inline arrays 6. ref T and Span<T>, which act just like &mut T and &mut [T] in Rust (with the same syntax). They are used in both advanced and most basic APIs alike to provide zero-cost wrapping and/or slicing of arbitrary memory (managed heap, stack, NativeMemory.Alloc'd) e.g. You can receive byte* and length from FFI and construct a (ReadOnly)Span<byte> from them. That span then can be passed to almost every method that used to work with arrays only during .NET Framework days.
  - soulbadguy 3y ago
    Indeed. The CLR from its inception implemented a lot of features for helping native inter-op. I would guess that msft did make the effort simply because of their large legacy c/c++/win32 code base. It's a shame really that msft stewardship of the .net/clr was so lacking. Of all the modern virtual machine clr us pretty much up there
    - pjmlp 3y ago
      Because it was yet another all-in, where all future Microsoft software was going to be .NET based, but WinDev/Office never played ball. We just have to go back in time and see all those Visual Studio.NET shows on surviving documents, or how Ext-VOS (what became .NET) was being done at MSR. We could have had something like Android, Inferno/Limbo in terms of .NET usage across the Windows layers, but it always bumps into the love for COM and C++, which incendently is how many Longhorn ideas were remade in Vista and continues to this day, using COM instead of .NET.
- merb 3y ago
  Sadly records are not value types, maybe Java will do this better, but I doubt it. Valhalla is probably years away.
neonsunset 3y ago
Small correction: FFI in Android land is abysmal, to get decent performance when accessing all kinds of system APIs you almost always end up having to write helpers in Java, to try to alleviate the pain of JNI as much as possible (it is still significant). Interop with iOS OTOH is a breeze, some of the APIs may require newer Swift Library Evolution ABI (direct calls to Swift exprots, without going through C/objC) but .NET is getting it in .NET 9 (the first and only platform to have it at the moment of release).
- pron 3y ago
  Calling Android "Java" has always been a stretch. Not only has Android never been compatible with any version of Java, the divergence between Android and Java is only growing. JDK 22 is Java; Android is Android.
  - neonsunset 3y ago
    That is true, for all intents and purposes however the glue code to (insufficiently) reduce marshalling and FFI overhead is written in Java the language, even if it does not use OpenJDK.
    - pron 3y ago
      Okay, but we're not talking about OpenJDK but about Java, a specification -- not an implementation like the OpenJDK JDK -- that Android has never conformed to and from which it is only growing further apart. I.e. the OpenJDK JDK is an implementation of Java; Android is not (there certainly may be some Java code that happens to behave the same way on Android, but it is not, and has never been, the case for Java code generally). The relationship between Java and Android is similar to that between Linux and Unix -- lots of intended similarities, but there shouldn't be an expectation that a new Linux feature appears in, say, AIX or vice-versa.
      - pjmlp 3y ago
        At least they were forced to accept how much Android ecosystem depends on Java and now Java 17 LTS is the latest supported version, minus the stuff they never supported like Swing and Java 2D.
adra 3y ago
I started to tinker with this new API last week. I definitely think it's a lot better than needing to self compile a lot of the bindings yourself which will certainly bring more from the community into the feature than were here before. That said, you really need to write a bunch of boiler-plate to implement useful things in java-like paradigms in the library today. Just as an example, I wrote up a sane windows _getcwd() stub and it's like 30 lines to implement correctly (properly freeing the buffer when I'm down with it) My pet project is to see if I can write automatic stubs around calls based on published API info and sources, so that at least some of that platform binding lift isn't just for those with non-trivial knowledge of C. Well, that's the hope anyways.
- schoenobates 3y ago
  I've been quite keen on the new FFI too - apologies if this is old news, but have you tried the [0] jextract stuff for some of the boilerplate gubbins? [0]: https://github.com/openjdk/jextract https://github.com/openjdk/jextract
  - adra 3y ago
    No! Thanks for the callout! That would certainly help on my path. My main goal was to insulate calls to avoid overly complicated reference counting when integrating large native surfaces into as Java program, but certainly an important aspect is getting the function signatures to target in the first place. I threw something bad together to work sort of like this (testing using proton), but I'm more than happy to bin it for something that just-works.
- ptx 3y ago
  Have you looked at Microsoft's Win32 API metadata package [0]? They're using it to generate C# and Rust bindings, and other people have targeted other languages. [0] https://github.com/microsoft/win32metadata https://github.com/microsoft/win32metadata
ackfoobar 3y ago
It worked great for me. I had long wanted to rewrite a Python lib in Kotlin, since the Python part of the lib is slow, and it does not support multi-threading. But the ugliness of JNI stopped me. Then I tried FFI in Java 21, with jextract it was amazing. I wasn't aware of FFI at first, but the finalized version of virtual threads made me check the Java release notes.
zbentley 3y ago
It's definitely exciting and a big step forward from JNI/JNA. However, I am a little concerned about the heavy emphasis on library unloading in the JEP. Unloading is famously fraught and unsafe for wide swaths of common libraries ... or maybe I'm overfitting due to recent trauma[1] and everything will be fine. 1. https://gist.github.com/thomcc/8c4a8003cf1a282949a539e899cb45aa https://gist.github.com/thomcc/8c4a8003cf1a282949a539e899cb4...

by oaiey 3y ago

Foreign Function Interface is one of the key advantages what C# had over Java (e.g. against huge APIs like Android or macOS). I am really curious how this JEP turns out.

MaxBarraclough 3y ago

For anyone reading, here's a link to the page with detail on this project: https://openjdk.org/jeps/454 https://openjdk.org/jeps/454

soulbadguy 3y ago

The FFI is also an artifact of CLR type system and the ability of have proper stack value based types.

grishka 3y ago
They will eventually bring value types to Java with Project Valhalla.
- layer8 3y ago
  I wonder how long it will take. We’ve been waiting for ten years already.
  - kaba0 3y ago
    One design iteration already has a working, implemented branch available, it’s just one of the hardest designs to get right, to remain backwards compatible, to have it work nice with generics and all the other features, etc. So once they actually settle on a design (which they seem to have closed on quite a lot in the last couple of years), it might not take that long to happen.
  - ivan_gammel 3y ago
    Looks like we may see a first preview in Java 23, i.e. already this year.
- soulbadguy 3y ago
  Noting that Valhalla value types are about guaranteeing object value identity with memory layout and/or flattening supposedly deriving from that. Modeling unmanaged sequence of bytes within the type system (like receiving a typed buffer from un managed code) might still be challenging
int_19h 3y ago
Not just value types, but also: 1. Pointers to the same (and not just references to objects). 2. Unions (via explicit-layout structs) 3. Function pointers. 4. C-style varargs. To be fair, not all of these have been exposed in C# historically even though CLR had them all along. Most notably, unmanaged function pointers took over 20 years. And since most people look at CLR through the prism of C#, they aren't necessarily aware of these features. Still, one way or the other, we're at the point where pretty much any C program can be transpiled to C# in a one-to-one mapping of almost every construct (setjmp/longjmp has to be emulated with exceptions), and a very similar performance profile of the resulting code.
- neonsunset 3y ago
  Honorable but important mentions: 5. Stackalloc (C alloca), fixed buffers in structs and inline arrays 6. ref T and Span<T>, which act just like &mut T and &mut [T] in Rust (with the same syntax). They are used in both advanced and most basic APIs alike to provide zero-cost wrapping and/or slicing of arbitrary memory (managed heap, stack, NativeMemory.Alloc'd) e.g. You can receive byte* and length from FFI and construct a (ReadOnly)Span<byte> from them. That span then can be passed to almost every method that used to work with arrays only during .NET Framework days.
- soulbadguy 3y ago
  Indeed. The CLR from its inception implemented a lot of features for helping native inter-op. I would guess that msft did make the effort simply because of their large legacy c/c++/win32 code base. It's a shame really that msft stewardship of the .net/clr was so lacking. Of all the modern virtual machine clr us pretty much up there
  - pjmlp 3y ago
    Because it was yet another all-in, where all future Microsoft software was going to be .NET based, but WinDev/Office never played ball. We just have to go back in time and see all those Visual Studio.NET shows on surviving documents, or how Ext-VOS (what became .NET) was being done at MSR. We could have had something like Android, Inferno/Limbo in terms of .NET usage across the Windows layers, but it always bumps into the love for COM and C++, which incendently is how many Longhorn ideas were remade in Vista and continues to this day, using COM instead of .NET.
merb 3y ago
Sadly records are not value types, maybe Java will do this better, but I doubt it. Valhalla is probably years away.

neonsunset 3y ago

Small correction: FFI in Android land is abysmal, to get decent performance when accessing all kinds of system APIs you almost always end up having to write helpers in Java, to try to alleviate the pain of JNI as much as possible (it is still significant). Interop with iOS OTOH is a breeze, some of the APIs may require newer Swift Library Evolution ABI (direct calls to Swift exprots, without going through C/objC) but .NET is getting it in .NET 9 (the first and only platform to have it at the moment of release).

pron 3y ago
Calling Android "Java" has always been a stretch. Not only has Android never been compatible with any version of Java, the divergence between Android and Java is only growing. JDK 22 is Java; Android is Android.
- neonsunset 3y ago
  That is true, for all intents and purposes however the glue code to (insufficiently) reduce marshalling and FFI overhead is written in Java the language, even if it does not use OpenJDK.
  - pron 3y ago
    Okay, but we're not talking about OpenJDK but about Java, a specification -- not an implementation like the OpenJDK JDK -- that Android has never conformed to and from which it is only growing further apart. I.e. the OpenJDK JDK is an implementation of Java; Android is not (there certainly may be some Java code that happens to behave the same way on Android, but it is not, and has never been, the case for Java code generally). The relationship between Java and Android is similar to that between Linux and Unix -- lots of intended similarities, but there shouldn't be an expectation that a new Linux feature appears in, say, AIX or vice-versa.
    - pjmlp 3y ago
      At least they were forced to accept how much Android ecosystem depends on Java and now Java 17 LTS is the latest supported version, minus the stuff they never supported like Swing and Java 2D.

adra 3y ago

I started to tinker with this new API last week. I definitely think it's a lot better than needing to self compile a lot of the bindings yourself which will certainly bring more from the community into the feature than were here before. That said, you really need to write a bunch of boiler-plate to implement useful things in java-like paradigms in the library today. Just as an example, I wrote up a sane windows _getcwd() stub and it's like 30 lines to implement correctly (properly freeing the buffer when I'm down with it) My pet project is to see if I can write automatic stubs around calls based on published API info and sources, so that at least some of that platform binding lift isn't just for those with non-trivial knowledge of C. Well, that's the hope anyways.

schoenobates 3y ago
I've been quite keen on the new FFI too - apologies if this is old news, but have you tried the [0] jextract stuff for some of the boilerplate gubbins? [0]: https://github.com/openjdk/jextract https://github.com/openjdk/jextract
- adra 3y ago
  No! Thanks for the callout! That would certainly help on my path. My main goal was to insulate calls to avoid overly complicated reference counting when integrating large native surfaces into as Java program, but certainly an important aspect is getting the function signatures to target in the first place. I threw something bad together to work sort of like this (testing using proton), but I'm more than happy to bin it for something that just-works.
ptx 3y ago
Have you looked at Microsoft's Win32 API metadata package [0]? They're using it to generate C# and Rust bindings, and other people have targeted other languages. [0] https://github.com/microsoft/win32metadata https://github.com/microsoft/win32metadata

ackfoobar 3y ago

It worked great for me. I had long wanted to rewrite a Python lib in Kotlin, since the Python part of the lib is slow, and it does not support multi-threading. But the ugliness of JNI stopped me. Then I tried FFI in Java 21, with jextract it was amazing. I wasn't aware of FFI at first, but the finalized version of virtual threads made me check the Java release notes.

zbentley 3y ago

It's definitely exciting and a big step forward from JNI/JNA. However, I am a little concerned about the heavy emphasis on library unloading in the JEP. Unloading is famously fraught and unsafe for wide swaths of common libraries ... or maybe I'm overfitting due to recent trauma[1] and everything will be fine. 1. https://gist.github.com/thomcc/8c4a8003cf1a282949a539e899cb45aa https://gist.github.com/thomcc/8c4a8003cf1a282949a539e899cb4...