6 ms·

I get that this is mostly for fun, but this is as good a place as any to bring up an issue I rarely see discussed with any post about a new language. Well-esta

I get that this is mostly for fun, but this is as good a place as any to bring up an issue I rarely see discussed with any post about a new language. Well-established languages have tons of widely used, highly vetted libraries implementing functionality that doesn’t exist in the new language and would be totally impractical for an individual to implement themselves. For example, if I’m doing scientific computing, I need a good linear algebra library like Eigen or Armadillo (or Numpy/PyTorch/Tensorflow), all of which would be impossible for me to implement myself. Therefore, for a new language to catch on, it needs a good foreign function interface. Yet FFIs are almost never brought up whenever new languages are discussed on HN. (Again, I realize that this is just a fun project and widespread adoption is not a goal.)

tibordp 4y ago
Totally agreed about FFI. I wanted to make it easy to interop with C code and write expressive bindings. Check for example the language bindings to LLVM's C API (fairly low level) and Tree-Sitter which is used internally (a bit higher level bindings) https://github.com/tibordp/alumina/tree/master/libraries/llvm https://github.com/tibordp/alumina/tree/master/libraries/llv... https://github.com/tibordp/alumina/blob/master/libraries/tree_sitter.alu https://github.com/tibordp/alumina/blob/master/libraries/tre... I think UFCS makes it quite nice for bindings, since external C functions can be used as if they were methods if the object is passed as the first parameter. So in many cases there might not even be a need to write wrapper structs for bindings that feel native. Of course, it's still a manual process and since Alumina is just a compiler and stdlib for now (no llibrary ecosystem, no compiler driver), it's a bit cumbersome. But I like the approach Rust has with bindgen and cc crates, to automatically create bindings for C and C++ code.
- girvo 4y ago
  UFCS making C bindings a hell of a lot nicer is one of the things I love about Nim, so I’m happy to see another language try it!
danias 4y ago
We are building a new language that is not for fun and we have thought of this important issue. We solve it with a Package Ports and Package Adapters. The Bitloops Language (BL) is a transpiled language with support for only TypeScript at the moment. Nonetheless, if it picks up we will be adding support for Java, C#, and C++ so that people can leverage their investments in their existing packages. https://github.com/bitloops/bitloops-language https://github.com/bitloops/bitloops-language
aliqot 4y ago
Libraries are a farce. Stdlib or die. I 100% mean this and live this.
- Asraelite 4y ago
  That's a strange take. Do you mean that you would rather call external binaries directly, like Bash does, or that you would rather re-implement all functionality you need, however complex?
  - aliqot 4y ago
    To me, there is value in the process and craft. It started as a one year challenge to use only stdlib in all projects, for work and personal code, and became a way of life. Though it may not work for everyone, or all languages, or all skillsets, it was a revealing experience. In retrospect, it made sense given that in my community we make our own furniture, instruments, tools, foods from whole items. Code seemed like the next logical step. During travels, once there was a man who had a word in his language for this. He said it was an aphorism loosely translatable to "the beauty of struggle". As he explained it, this is the positive benefit gained in return for the time and effort to do something as an act of appreciation of the craft, and how the value in the experience surpasses the debt of time and sweat. To put it in a more modern and eastern philosphical context, think of it like Kata. https://en.wikipedia.org/wiki/Kata https://en.wikipedia.org/wiki/Kata
    - Art9681 4y ago
      I too suffer from obessively overcoming challenges. I will say this, in a snarky and friendly way. I hope you understand. If developing with high level abstractions wasn't challenging enough, your vision was too small. Surely you can think of a problem to solve that even Copilot would be of little use. Imagine what you could build if you got a little more help from third party libs and a little more ambition?
      - aliqot 4y ago
        I appreciate you for relating your experience. It's not ambition or challenge that is lacking, we are what's commonly called 'Amish'. This is just how we do things. https://en.wikipedia.org/wiki/Simple_living https://en.wikipedia.org/wiki/Simple_living https://en.wikipedia.org/wiki/Nonconformity_to_the_world https://en.wikipedia.org/wiki/Nonconformity_to_the_world I'm sure this creates more questions than it answers, however, I hope it lends some context as to how we've both arrived at our own respective happy milieus.
culi 4y ago
What's an example of a language with "good" FFI?
- davedx 4y ago
  Rust's seems pretty good. I've been doing some Rust/C interop and it works very smoothly
- zyx_tony 4y ago
  Nim is quite enjoyable to use. I do a decent amount of scientific computing, interfacing with various c/cpp libs
- williamstein 4y ago
  Calling C functions from zig is very nice, partly because zig uses llvm to parse C header files and uses that information.
- beagle3 4y ago
  LuaJIT has a pretty great FFi which was later ported to standard Lua; Python’s ctypes provides a decent FFI. Nim, Zig and Rust do too, but they have semantics much closer to C, so it’s almost free (especially when they can all use llvm directly for code generation, and Nim’s preferred way is to compile through C in the first place)
- WalterBright 4y ago
  With the D programming language, to interface with C's stdio.h: import stdio; void main() { printf("hello from D!\n"); } is all that's necessary, as D has a built-in C compiler that reads stdio.h, compiles it, and presents its interface to the D code.
  - cercatrova 4y ago
    I hear about D often on HN but I never see it being used in the wild so to speak. When I looked at it, it seemed, like C++, a hodgepodge of features built up over time rather than being a more deliberately focused language. I hate to compare and contrast but Rust comes to mind as one of the latter, probably because it has a somewhat more intensive and extensive RFC process for adding new features.
belmarca 4y ago
We've been investigating coupling syntactic and low-level FFIs [1]. Our most recent work will be published at the upcoming Scheme Workshop [2], where we integrate Gambit Scheme with CPython. We share the concerns you have and have had quite good success with the Scheme<->Python interface! The syntactic interface makes it really enjoyable. [1]: https://zenodo.org/record/4711425 https://zenodo.org/record/4711425 [2]: The paper should be up in a few days I suppose.

tibordp 4y ago

Totally agreed about FFI. I wanted to make it easy to interop with C code and write expressive bindings. Check for example the language bindings to LLVM's C API (fairly low level) and Tree-Sitter which is used internally (a bit higher level bindings) https://github.com/tibordp/alumina/tree/master/libraries/llvm https://github.com/tibordp/alumina/tree/master/libraries/llv... https://github.com/tibordp/alumina/blob/master/libraries/tree_sitter.alu https://github.com/tibordp/alumina/blob/master/libraries/tre... I think UFCS makes it quite nice for bindings, since external C functions can be used as if they were methods if the object is passed as the first parameter. So in many cases there might not even be a need to write wrapper structs for bindings that feel native. Of course, it's still a manual process and since Alumina is just a compiler and stdlib for now (no llibrary ecosystem, no compiler driver), it's a bit cumbersome. But I like the approach Rust has with bindgen and cc crates, to automatically create bindings for C and C++ code.

girvo 4y ago
UFCS making C bindings a hell of a lot nicer is one of the things I love about Nim, so I’m happy to see another language try it!

danias 4y ago

We are building a new language that is not for fun and we have thought of this important issue. We solve it with a Package Ports and Package Adapters. The Bitloops Language (BL) is a transpiled language with support for only TypeScript at the moment. Nonetheless, if it picks up we will be adding support for Java, C#, and C++ so that people can leverage their investments in their existing packages. https://github.com/bitloops/bitloops-language https://github.com/bitloops/bitloops-language

aliqot 4y ago

Libraries are a farce. Stdlib or die. I 100% mean this and live this.

Asraelite 4y ago
That's a strange take. Do you mean that you would rather call external binaries directly, like Bash does, or that you would rather re-implement all functionality you need, however complex?
- aliqot 4y ago
  To me, there is value in the process and craft. It started as a one year challenge to use only stdlib in all projects, for work and personal code, and became a way of life. Though it may not work for everyone, or all languages, or all skillsets, it was a revealing experience. In retrospect, it made sense given that in my community we make our own furniture, instruments, tools, foods from whole items. Code seemed like the next logical step. During travels, once there was a man who had a word in his language for this. He said it was an aphorism loosely translatable to "the beauty of struggle". As he explained it, this is the positive benefit gained in return for the time and effort to do something as an act of appreciation of the craft, and how the value in the experience surpasses the debt of time and sweat. To put it in a more modern and eastern philosphical context, think of it like Kata. https://en.wikipedia.org/wiki/Kata https://en.wikipedia.org/wiki/Kata
  - Art9681 4y ago
    I too suffer from obessively overcoming challenges. I will say this, in a snarky and friendly way. I hope you understand. If developing with high level abstractions wasn't challenging enough, your vision was too small. Surely you can think of a problem to solve that even Copilot would be of little use. Imagine what you could build if you got a little more help from third party libs and a little more ambition?
    - aliqot 4y ago
      I appreciate you for relating your experience. It's not ambition or challenge that is lacking, we are what's commonly called 'Amish'. This is just how we do things. https://en.wikipedia.org/wiki/Simple_living https://en.wikipedia.org/wiki/Simple_living https://en.wikipedia.org/wiki/Nonconformity_to_the_world https://en.wikipedia.org/wiki/Nonconformity_to_the_world I'm sure this creates more questions than it answers, however, I hope it lends some context as to how we've both arrived at our own respective happy milieus.

culi 4y ago

What's an example of a language with "good" FFI?

davedx 4y ago
Rust's seems pretty good. I've been doing some Rust/C interop and it works very smoothly
zyx_tony 4y ago
Nim is quite enjoyable to use. I do a decent amount of scientific computing, interfacing with various c/cpp libs
williamstein 4y ago
Calling C functions from zig is very nice, partly because zig uses llvm to parse C header files and uses that information.
beagle3 4y ago
LuaJIT has a pretty great FFi which was later ported to standard Lua; Python’s ctypes provides a decent FFI. Nim, Zig and Rust do too, but they have semantics much closer to C, so it’s almost free (especially when they can all use llvm directly for code generation, and Nim’s preferred way is to compile through C in the first place)
WalterBright 4y ago
With the D programming language, to interface with C's stdio.h: import stdio; void main() { printf("hello from D!\n"); } is all that's necessary, as D has a built-in C compiler that reads stdio.h, compiles it, and presents its interface to the D code.
- cercatrova 4y ago
  I hear about D often on HN but I never see it being used in the wild so to speak. When I looked at it, it seemed, like C++, a hodgepodge of features built up over time rather than being a more deliberately focused language. I hate to compare and contrast but Rust comes to mind as one of the latter, probably because it has a somewhat more intensive and extensive RFC process for adding new features.

belmarca 4y ago

We've been investigating coupling syntactic and low-level FFIs [1]. Our most recent work will be published at the upcoming Scheme Workshop [2], where we integrate Gambit Scheme with CPython. We share the concerns you have and have had quite good success with the Scheme<->Python interface! The syntactic interface makes it really enjoyable. [1]: https://zenodo.org/record/4711425 https://zenodo.org/record/4711425 [2]: The paper should be up in a few days I suppose.