3 ms·

Regarding seamless FFI, I suspect that part of the JNI friction was intentional, out of fear that programmers would use FFI as a crutch. Erlang's BEAM's nif[0]

by KMag 6y ago

Regarding seamless FFI, I suspect that part of the JNI friction was intentional, out of fear that programmers would use FFI as a crutch. Erlang's BEAM's nif[0] interface is the best I've seen in mainstream runtimes for FFI. You write an Erlang/Elixir/etc. function, and loading a native library may swap out the function body. This allows for workarounds in Erlang/Elixir/etc. in case of version skew in your native libraries. In my day job, I work with a domain-specific language with a similar FFI (actually, 3 different FFIs, one similar to nifs), and it's very handy to have a fallback that does things with /proc/, calling external command line binaries, slower versions of FFI functions, etc. (This allows the native code release schedule to be decoupled from the domain-specific language code changes.) The syntax isn't the same, but a Java-like syntax would be // getpid is built in, but it could be done something like this // look for XNativeGetPID() in mylib.so or mylib.dll depending on OS @native_replace("XNativeGetPID", "mylib") public static int getpid() { // Throw an exception if you want to force // native implemementation return Integer.parse(Runtime.system( "/bin/grep pid /proc/self/status | /bin/sed 's/..." )); } [0] http://erlang.org/doc/man/erl_nif.html http://erlang.org/doc/man/erl_nif.html

pjmlp 6y ago
It was, Mark Reinhold has a couple of talks where he acknowledges it and why new replacement (Panama) was being designed. By the way, it is now in preview.
- KMag 6y ago
  Great, and thanks for the confirmation. Do you have any idea how easy it is to provide a Java fallback on platforms where a native implementation isn't available? (I find it's often possible to call out to a command line tool as a workaround for things I'm trying to do in native code.)
  - pjmlp 6y ago
    Well, it is going to be the same as in any other language, you are the one that needs to take care of that fallback exists. To read more about Panama, https://jdk.java.net/panama/ https://jdk.java.net/panama/ Check Foreign-Memory Access API and Foreign Linker API related entries.
  - kaba0 6y ago
    With Panama, you get a really low level API and first have to load in the shared library - where you can easily handle this case. Though I’m not sure I would want this behavior, or at most for really optional deps; ideally software should be “installed” in an ideal environment.
    - KMag 6y ago
      > ideally software should be “installed” in an ideal environment. Native implementations with non-native fallbacks can be an important decoupling mechanism. I used to be a developer for a Linux desktop application with a peak of something like 60 million unique users over a trailing 3 month window. We had Windows, OS X, RPM, and dpkg installers with native libraries, but people also ran our software on FreeBSD, Gentoo, etc. It's handy to have gradual enhancement/graceful degradation in non-ideal installs where the native API's you'd like either don't exist or are buggy and shouldn't be used. In my day job, we sometimes have a business need to iterate new features at a faster tempo than our native code release schedule, so it's nice to have a similar sort of fallback implementation.
      - whoisthemachine 6y ago
        I'm not very familiar with Java's native interface, but you could potentially implement the native and non-native approaches as separate classes with a common interface, and choose to inject the correct dependency to the calling code at runtime by detecting OS features?
        KMag 6y ago
        It clearly can be done, perhaps by using reflection and catching a class loading exception, but it's pretty common boilerplate that should optionally be handled by the language runtime's FFI. Let's say you want to have a native-looking prompt window via wxWidgets, and fall back to Swing if wx isn't installed. Or, you want to use some particular optimized native BLAS library if it's installed, or fall back to doing the linear algebra in Java if it's not available. In these cases, and many others, "detecting OS features" involves checking LD_LIBRARY_PATH (or PATH on Win32) for libraries, checking they're for the correct architecture, etc. The OS already provides dlopen() or its equivalent for this purpose. Properly detecting the OS and emulating the OS's algorithm for finding native libraries is just wasteful when the OS already exposes the functionality. As an added bonus, an FFI that replaces a method body allows someone who doesn't have the C/C++/Rust/FORTRAN handy, or can't read it, to see roughly what the native code should be doing.

by KMag 6y ago

pjmlp 6y ago

It was, Mark Reinhold has a couple of talks where he acknowledges it and why new replacement (Panama) was being designed. By the way, it is now in preview.

KMag 6y ago
Great, and thanks for the confirmation. Do you have any idea how easy it is to provide a Java fallback on platforms where a native implementation isn't available? (I find it's often possible to call out to a command line tool as a workaround for things I'm trying to do in native code.)
- pjmlp 6y ago
  Well, it is going to be the same as in any other language, you are the one that needs to take care of that fallback exists. To read more about Panama, https://jdk.java.net/panama/ https://jdk.java.net/panama/ Check Foreign-Memory Access API and Foreign Linker API related entries.
- kaba0 6y ago
  With Panama, you get a really low level API and first have to load in the shared library - where you can easily handle this case. Though I’m not sure I would want this behavior, or at most for really optional deps; ideally software should be “installed” in an ideal environment.
  - KMag 6y ago
    > ideally software should be “installed” in an ideal environment. Native implementations with non-native fallbacks can be an important decoupling mechanism. I used to be a developer for a Linux desktop application with a peak of something like 60 million unique users over a trailing 3 month window. We had Windows, OS X, RPM, and dpkg installers with native libraries, but people also ran our software on FreeBSD, Gentoo, etc. It's handy to have gradual enhancement/graceful degradation in non-ideal installs where the native API's you'd like either don't exist or are buggy and shouldn't be used. In my day job, we sometimes have a business need to iterate new features at a faster tempo than our native code release schedule, so it's nice to have a similar sort of fallback implementation.
    - whoisthemachine 6y ago
      I'm not very familiar with Java's native interface, but you could potentially implement the native and non-native approaches as separate classes with a common interface, and choose to inject the correct dependency to the calling code at runtime by detecting OS features?
      - KMag 6y ago
        It clearly can be done, perhaps by using reflection and catching a class loading exception, but it's pretty common boilerplate that should optionally be handled by the language runtime's FFI. Let's say you want to have a native-looking prompt window via wxWidgets, and fall back to Swing if wx isn't installed. Or, you want to use some particular optimized native BLAS library if it's installed, or fall back to doing the linear algebra in Java if it's not available. In these cases, and many others, "detecting OS features" involves checking LD_LIBRARY_PATH (or PATH on Win32) for libraries, checking they're for the correct architecture, etc. The OS already provides dlopen() or its equivalent for this purpose. Properly detecting the OS and emulating the OS's algorithm for finding native libraries is just wasteful when the OS already exposes the functionality. As an added bonus, an FFI that replaces a method body allows someone who doesn't have the C/C++/Rust/FORTRAN handy, or can't read it, to see roughly what the native code should be doing.