12 ms·

Jank is C++

289 points by Jeaye 1y ago

superdisk 1y ago
Cool stuff for sure, I've been brainstorming making a language that has some of the same characteristics as Jank. I'm jelly that you took the opportunity to work full time on this for a year, wish I could do the same!
jekwoooooe 1y ago
Clojure syntax (or clojure like, whatever this is) is easily the worst I’ve ever seen. (How [do you guys] (live) like this) it’s just awful. One could say it’s jack
nanomonkey 1y ago
Ferret is another Clojure implementation in C, that has a decent C/C++ interface: https://nakkaya.com/2017/06/24/ferret-lisp-ffi-notes/ https://nakkaya.com/2017/06/24/ferret-lisp-ffi-notes/ I'm curious if the author of jank was aware of Ferret, and what was found lacking.
sesm 1y ago
So, if I import a C++ library in Jank, all the internal memory allocations/deallocation will go through Jank's GC? How is this implemented? And what if a C++ library relies on some 3rd party allocator library?
almostgotcaught 1y ago
i commented on reddit (and got promptly downvoted) but since i think jank's author is around here (and hopefully is receptive to constructive criticism): the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. there's no reason to do this except that libclang currently doesn't support any other way. that's not jank's fault but it could be "fixed" in libclang. at a minimum you could use https://github.com/llvm/llvm-project/blob/main/clang/lib/CodeGen/CodeGenTypes.h https://github.com/llvm/llvm-project/blob/main/clang/lib/Cod... to emit the code based on clang ast. at a maximum would be to use something like https://github.com/Mr-Anyone/abi https://github.com/Mr-Anyone/abi or this if/when it comes to fruition https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-library/84554 https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-... to generate ABI compliant calls/etc for cpp libs. note, i say all this with maximum love in my heart for a language that would have first class cpp interop - i would immediately become jank's biggest proponent/user if its cpp interop were robust. EDIT: for people wanting/needing receipts, you can skim through https://github.com/compiler-research/CppInterOp/blob/main/lib/CppInterOp/CppInterOp.cpp#L2742-L2756 https://github.com/compiler-research/CppInterOp/blob/main/li...
- wk_end 1y ago
  > the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. So, I agree that this sounds janky as heck. My question is: besides sounding janky as heck, is there something wrong with this? Is it slow/unreliable?
  - almostgotcaught 1y ago
    i mean it's as prone to error as any other thing that relies on string munging. it's probably not that much slower than the alternative i proposed - because the trampolines/wrappers are jitted and then reused - but it's just not robust enough that i would ever imagine building a prod system on top of it (eg using cppyy in prod) let alone baking it into my language/runtime.
    - refulgentis 1y ago
      The delta between the title and the content gave me extreme pause, thanks for sharing that there's, uh, worse problems. I'm a bit surprised I've seen two articles about jank here the last 2 days if these are exemplars of the technical approach and communication style. Seems like that wouldn't be enough to get on people's radars.
      - actionfromafar 1y ago
        Given how the world works, that might mean we will all sit and curse Jank instead of cursing Node. :)
      - Jeaye 1y ago
        Which particular delta between the title and the content gave you extreme pause?
        refulgentis 1y ago
        It said "jank is C++", which I assumed would be explaining that jank compiles down to C++ or something similar, i.e. there is a layer of abstraction between jank and C++, but it effectively "works like" C++. On re-read, I recognize where it is used in the article: "jank is C++. There is no runtime reflection, no guess work, and no hints. If the compiler can't find a member, or a function, or a particular overload, you will get a compiler error." I assume other interop scenarios don't pull this off*, thus it is distinctive. Additionally, I'm not at all familiar with Clojure, sadly, but it also sounds like there's some special qualities there ("I think that this is an interesting way to start thinking about jank, Clojure, and static types") Now I'll riff and just write out the first 3-5 titles that come to mind with that limited understanding: - Implementing compile-time verifiable C++ interop in jank - Sparks of C++ interop: jank, Clojure, & verifying interop before runtime - jank's progress on C++ interop - Safe C++ interop lessons from jank * for example, I write a lot of Dart day to day and rely on Dart's "FFI" implementation to call C++, which now that I'm thinking about, only works because there's a code generator that creates "Dart headers" (my term) for the C++ libraries. I could totally footgun and call arbitrary functions that don't exist.
xxr 1y ago
These recursive initialism PL names are getting out of hand /s
- Jeaye 1y ago
  I've pondered this for a while and I have no idea how jank is a recursive acronym. What're you seeing that I'm not?
  - eurleif 1y ago
    Jank's A Native Klojure? :)
  - xxr 1y ago
    It’s a joke (hence the “/s”) on the “[PL name] is [words beginning with the rest of the letters of the Pl name]” snowclone. However as time approaches infinity I’m sure it will get a recursive backronym.
johnnyjeans 1y ago
I'm not surprised to see that Jank's solution to this is to embed LLVM into their runtime. I really wish there was a better way to do this. There are a lot of things I don't like about C++, and close to the top of the list is the lack of standardization for name-mangling, or even a way mangle or de-mangle names at compile-time. Sepples is a royal pain in the ass to target for a dynamic FFI because of that. It would be really nice to have some way to get symbol names and calling semantics as constexpr const char* and not have to deal with generating (or writing) a ton of boilerplate and extern "C" blocks. It's absolutely possible, but it's not low-hanging fruit so the standards committee will never put it in. Just like they'll never add a standardized equivalent for alloca/VLAs. We're not allowed to have basic, useful things. Only more ways to abuse type deduction. Will C++26 finally give us constexpr dynamic allocations? Will compilers ever actually implement one of the three (3) compile-time reflection standards? Stay tuned to find out!
- kmeisthax 1y ago
  [dead]
- kazinator 1y ago
  > embed LLVM into their runtime That comically reads like "embed a blue whale into your hammock".
- Someone 1y ago
  I would think name mangling is out of scope for a programming language definition, more so for C and C++, which target running on anything under the sun, including systems that do not have libraries, do not have the concept of shared libraries or do not have access to function names at runtime. > It would be really nice to have some way to get symbol names and calling semantics Again, I think that’s out of scope for a programming language. Also, is it even possible to have a way to describe low level calling semantics for any CPU in a way such that a program can use that info? The target CPU may not have registers or may not have a stack, may have multiple types of memory, may have segmented memory, etc.
- almostgotcaught 1y ago
  > LLVM into their runtime they're not embedding LLVM - they're embedding clang. if you look at my comment below, you'll see LLVM is not currently sufficient. > [C++] is a royal pain in the ass to target for a dynamic FFI because of that name mangling is by the easiest part of cpp FFI - the hard part is the rest of the ABI. anyone curious can start here https://github.com/rust-lang/rust-bindgen/issues/778 https://github.com/rust-lang/rust-bindgen/issues/778
dmoy 1y ago
From two days ago: https://news.ycombinator.com/item?id=44482273 https://news.ycombinator.com/item?id=44482273
papichulo2023 1y ago
Recently I tried D lang and was surprise with the nice interop with C++ (the language in general feels pretty good), Carbon is nowhere to be seen and havent tried Swift's yet. I hope this is a good one.
- actionfromafar 1y ago
  Shedskin lang has excellent integration with C++.
  - almostgotcaught 1y ago
    shedskin isn't actively developed ... or at least it wasn't for like 10 years https://github.com/shedskin/shedskin/graphs/contributors https://github.com/shedskin/shedskin/graphs/contributors
    - actionfromafar 1y ago
      It suddenly sprung to life and gained Python 3 compatilibity, which makes it much more interesting than before despite it's ten year hiatus in Python 2 land.
- Imustaskforhelp 1y ago
  Now ofc jank already has gotten C++ support but if I may ask, if it had, let's say gotten D lang support, then would that have been easier/more doable/practical?
Mathnerd314 1y ago
Ok so jank is Clojure but with C++/LLVM runtime rather than JVM. So already all of its types are C++ types, that presumably makes things a lot easier. Basically it just uses libclang / CppInterOp to get the corresponding LLVM types and then emits a function call. https://github.com/jank-lang/jank/blob/interop/compiler%2Bruntime/src/cpp/jank/codegen/llvm_processor.cpp#L1663 https://github.com/jank-lang/jank/blob/interop/compiler%2Bru...
Jach 1y ago
Neat project, I can only marvel at your ability to deal with such madness. But it would be nice to have better C++ interop in higher level languages, there's some useful C++ code out there. I also appreciate the brief mention of Clasp, as I was immediately thinking of it as I was reading through.
netbioserror 1y ago
I used Clojure back in the day and use Nim at work these days. Linking in to C is trivially easy in Nim. Happy to see this working for jank, but C++ is...such a nightmare target. Any chance of Jank eventually settling on reference counting? It checks so many boxes in my book: Simple, predictable, few edge cases, fast. I guess it really just depends on how much jank programs thrash memory, I remember Clojure having a lot of background churn.
- Jeaye 1y ago
  I started with reference counting, but the amount of garbage Clojure programs churn out ends up bogging everything down unless a GC is used. jank's GC will change, going forward, and I want jank to grow to support optional affine typing, but the Clojure base is likely always going to be garbage collected.
  - fnordsensei 1y ago
    For a novice, could you elaborate the difference that GC does? Naively, it seems like the only difference would be whether you pay the deallocation fee immediately or later on. Is there less of a problem when done in bulk if the volume of trash to collect is high enough?
    - software-is-art 1y ago
      GCs typically fall into two categories: 1. Reference counting - tracks how many references point to each object. When references are added or removed, the count is updated. When it hits zero, the object is freed immediately. This places overhead on every operation that modifies references. 2. Mark and sweep - objects are allocated in heap regions managed by the GC. Periodically the GC traces from roots (stack, globals) to find all live objects, then frees the rest. Usually generational: new objects in a nursery/gen0 are collected frequently, survivors are promoted to older generations collected less often. In general reference counting is favoured for predictable latency because you’re cleaning up incrementally as you go. Total memory footprint is similar to manual memory management with some overhead for counting refs. The cost is lower throughput as every reference change requires bookkeeping (see Swift ARC for a good example). Mark and sweep GCs are favoured for throughput as allocations and reference updates have zero overhead - you just bump a pointer to allocate. When collection does occur it can cause a pause, though modern concurrent collectors have greatly reduced this (see Java G1GC or .NET for good examples). Memory footprint is usually quite a bit larger than manual management. In the case of Clojure which in addition to being a LISP also uses immutable data structures, there is both object churn and frequent changes to the object graph. This makes throughput a much larger concern than a less allocation heavy language - favouring mark and sweep designs.
YuriNiyazov 1y ago
A long long time ago, at ClojureConj 2014, I asked Rich Hickey whether a cpp-based clojure was possible, and his answer was "well, the primary impediment there is a lack of a garbage collector". There were a lot of conversations going on at the same time, so I didn't get an opportunity to "delve" into it, but: 1. does that objection make sense? 2. How does jank approach that hurdle.
- ethan_smith 1y ago
  Jank likely uses a combination of LLVM's garbage collection support (GC intrinsics) and smart pointers, similar to how Clasp implemented GC for Common Lisp on C++.
- bertmuthalaly 1y ago
  It's the first section in the article - "I have implemented manual memory management via cpp/new and cpp/delete. This uses jank's GC allocator (currently bdwgc), rather than malloc, so using cpp/delete isn't generally needed. However, if cpp/delete is used then memory collection can be eager and more deterministic. The implementation has full bdwgc support for destructors as well, so both manual deletion and automatic collection will trigger non-trivial destructors."
- Jeaye 1y ago
  A GC is nowhere near the most difficult part of this. In 2014, there was no viable technology for JIT compiling C++, and very little technology for JIT compiling native code in general.
- bluGill 1y ago
  In the artical - it garbage collects always but if you call delete the garbage collector will be more agressive about cleaning that up.
caim 1y ago
That's great! Interop with C++ is such a complex task. Congratss on your work! It's definitely not an easy thing. I've always wondered what is the best way to interact with C++ template instantiation while keeping performance. For a static language, you'd probably need to translate your types to C++ during compilation, ask Clang/GCC/MSVC to compile the generated C++ file, and then link the final result. And finally, pray to the computer gods that name mangiling was done right.

289 points by Jeaye 1y ago

Cool stuff for sure, I've been brainstorming making a language that has some of the same characteristics as Jank. I'm jelly that you took the opportunity to work full time on this for a year, wish I could do the same!

jekwoooooe 1y ago

Clojure syntax (or clojure like, whatever this is) is easily the worst I’ve ever seen. (How [do you guys] (live) like this) it’s just awful. One could say it’s jack

nanomonkey 1y ago

Ferret is another Clojure implementation in C, that has a decent C/C++ interface: https://nakkaya.com/2017/06/24/ferret-lisp-ffi-notes/ https://nakkaya.com/2017/06/24/ferret-lisp-ffi-notes/ I'm curious if the author of jank was aware of Ferret, and what was found lacking.

sesm 1y ago

So, if I import a C++ library in Jank, all the internal memory allocations/deallocation will go through Jank's GC? How is this implemented? And what if a C++ library relies on some 3rd party allocator library?

almostgotcaught 1y ago

i commented on reddit (and got promptly downvoted) but since i think jank's author is around here (and hopefully is receptive to constructive criticism): the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. there's no reason to do this except that libclang currently doesn't support any other way. that's not jank's fault but it could be "fixed" in libclang. at a minimum you could use https://github.com/llvm/llvm-project/blob/main/clang/lib/CodeGen/CodeGenTypes.h https://github.com/llvm/llvm-project/blob/main/clang/lib/Cod... to emit the code based on clang ast. at a maximum would be to use something like https://github.com/Mr-Anyone/abi https://github.com/Mr-Anyone/abi or this if/when it comes to fruition https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-library/84554 https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-... to generate ABI compliant calls/etc for cpp libs. note, i say all this with maximum love in my heart for a language that would have first class cpp interop - i would immediately become jank's biggest proponent/user if its cpp interop were robust. EDIT: for people wanting/needing receipts, you can skim through https://github.com/compiler-research/CppInterOp/blob/main/lib/CppInterOp/CppInterOp.cpp#L2742-L2756 https://github.com/compiler-research/CppInterOp/blob/main/li...

wk_end 1y ago
> the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. So, I agree that this sounds janky as heck. My question is: besides sounding janky as heck, is there something wrong with this? Is it slow/unreliable?
- almostgotcaught 1y ago
  i mean it's as prone to error as any other thing that relies on string munging. it's probably not that much slower than the alternative i proposed - because the trampolines/wrappers are jitted and then reused - but it's just not robust enough that i would ever imagine building a prod system on top of it (eg using cppyy in prod) let alone baking it into my language/runtime.
  - refulgentis 1y ago
    The delta between the title and the content gave me extreme pause, thanks for sharing that there's, uh, worse problems. I'm a bit surprised I've seen two articles about jank here the last 2 days if these are exemplars of the technical approach and communication style. Seems like that wouldn't be enough to get on people's radars.
    - actionfromafar 1y ago
      Given how the world works, that might mean we will all sit and curse Jank instead of cursing Node. :)
    - Jeaye 1y ago
      Which particular delta between the title and the content gave you extreme pause?
      - refulgentis 1y ago
        It said "jank is C++", which I assumed would be explaining that jank compiles down to C++ or something similar, i.e. there is a layer of abstraction between jank and C++, but it effectively "works like" C++. On re-read, I recognize where it is used in the article: "jank is C++. There is no runtime reflection, no guess work, and no hints. If the compiler can't find a member, or a function, or a particular overload, you will get a compiler error." I assume other interop scenarios don't pull this off*, thus it is distinctive. Additionally, I'm not at all familiar with Clojure, sadly, but it also sounds like there's some special qualities there ("I think that this is an interesting way to start thinking about jank, Clojure, and static types") Now I'll riff and just write out the first 3-5 titles that come to mind with that limited understanding: - Implementing compile-time verifiable C++ interop in jank - Sparks of C++ interop: jank, Clojure, & verifying interop before runtime - jank's progress on C++ interop - Safe C++ interop lessons from jank * for example, I write a lot of Dart day to day and rely on Dart's "FFI" implementation to call C++, which now that I'm thinking about, only works because there's a code generator that creates "Dart headers" (my term) for the C++ libraries. I could totally footgun and call arbitrary functions that don't exist.

xxr 1y ago

These recursive initialism PL names are getting out of hand /s

Jeaye 1y ago
I've pondered this for a while and I have no idea how jank is a recursive acronym. What're you seeing that I'm not?
- eurleif 1y ago
  Jank's A Native Klojure? :)
- xxr 1y ago
  It’s a joke (hence the “/s”) on the “[PL name] is [words beginning with the rest of the letters of the Pl name]” snowclone. However as time approaches infinity I’m sure it will get a recursive backronym.

johnnyjeans 1y ago

I'm not surprised to see that Jank's solution to this is to embed LLVM into their runtime. I really wish there was a better way to do this. There are a lot of things I don't like about C++, and close to the top of the list is the lack of standardization for name-mangling, or even a way mangle or de-mangle names at compile-time. Sepples is a royal pain in the ass to target for a dynamic FFI because of that. It would be really nice to have some way to get symbol names and calling semantics as constexpr const char* and not have to deal with generating (or writing) a ton of boilerplate and extern "C" blocks. It's absolutely possible, but it's not low-hanging fruit so the standards committee will never put it in. Just like they'll never add a standardized equivalent for alloca/VLAs. We're not allowed to have basic, useful things. Only more ways to abuse type deduction. Will C++26 finally give us constexpr dynamic allocations? Will compilers ever actually implement one of the three (3) compile-time reflection standards? Stay tuned to find out!

kmeisthax 1y ago
[dead]
kazinator 1y ago
> embed LLVM into their runtime That comically reads like "embed a blue whale into your hammock".
Someone 1y ago
I would think name mangling is out of scope for a programming language definition, more so for C and C++, which target running on anything under the sun, including systems that do not have libraries, do not have the concept of shared libraries or do not have access to function names at runtime. > It would be really nice to have some way to get symbol names and calling semantics Again, I think that’s out of scope for a programming language. Also, is it even possible to have a way to describe low level calling semantics for any CPU in a way such that a program can use that info? The target CPU may not have registers or may not have a stack, may have multiple types of memory, may have segmented memory, etc.
almostgotcaught 1y ago
> LLVM into their runtime they're not embedding LLVM - they're embedding clang. if you look at my comment below, you'll see LLVM is not currently sufficient. > [C++] is a royal pain in the ass to target for a dynamic FFI because of that name mangling is by the easiest part of cpp FFI - the hard part is the rest of the ABI. anyone curious can start here https://github.com/rust-lang/rust-bindgen/issues/778 https://github.com/rust-lang/rust-bindgen/issues/778

dmoy 1y ago

From two days ago: https://news.ycombinator.com/item?id=44482273 https://news.ycombinator.com/item?id=44482273

papichulo2023 1y ago

Recently I tried D lang and was surprise with the nice interop with C++ (the language in general feels pretty good), Carbon is nowhere to be seen and havent tried Swift's yet. I hope this is a good one.

actionfromafar 1y ago
Shedskin lang has excellent integration with C++.
- almostgotcaught 1y ago
  shedskin isn't actively developed ... or at least it wasn't for like 10 years https://github.com/shedskin/shedskin/graphs/contributors https://github.com/shedskin/shedskin/graphs/contributors
  - actionfromafar 1y ago
    It suddenly sprung to life and gained Python 3 compatilibity, which makes it much more interesting than before despite it's ten year hiatus in Python 2 land.
Imustaskforhelp 1y ago
Now ofc jank already has gotten C++ support but if I may ask, if it had, let's say gotten D lang support, then would that have been easier/more doable/practical?

Mathnerd314 1y ago

Ok so jank is Clojure but with C++/LLVM runtime rather than JVM. So already all of its types are C++ types, that presumably makes things a lot easier. Basically it just uses libclang / CppInterOp to get the corresponding LLVM types and then emits a function call. https://github.com/jank-lang/jank/blob/interop/compiler%2Bruntime/src/cpp/jank/codegen/llvm_processor.cpp#L1663 https://github.com/jank-lang/jank/blob/interop/compiler%2Bru...

Jach 1y ago

Neat project, I can only marvel at your ability to deal with such madness. But it would be nice to have better C++ interop in higher level languages, there's some useful C++ code out there. I also appreciate the brief mention of Clasp, as I was immediately thinking of it as I was reading through.

netbioserror 1y ago

I used Clojure back in the day and use Nim at work these days. Linking in to C is trivially easy in Nim. Happy to see this working for jank, but C++ is...such a nightmare target. Any chance of Jank eventually settling on reference counting? It checks so many boxes in my book: Simple, predictable, few edge cases, fast. I guess it really just depends on how much jank programs thrash memory, I remember Clojure having a lot of background churn.

Jeaye 1y ago
I started with reference counting, but the amount of garbage Clojure programs churn out ends up bogging everything down unless a GC is used. jank's GC will change, going forward, and I want jank to grow to support optional affine typing, but the Clojure base is likely always going to be garbage collected.
- fnordsensei 1y ago
  For a novice, could you elaborate the difference that GC does? Naively, it seems like the only difference would be whether you pay the deallocation fee immediately or later on. Is there less of a problem when done in bulk if the volume of trash to collect is high enough?
  - software-is-art 1y ago
    GCs typically fall into two categories: 1. Reference counting - tracks how many references point to each object. When references are added or removed, the count is updated. When it hits zero, the object is freed immediately. This places overhead on every operation that modifies references. 2. Mark and sweep - objects are allocated in heap regions managed by the GC. Periodically the GC traces from roots (stack, globals) to find all live objects, then frees the rest. Usually generational: new objects in a nursery/gen0 are collected frequently, survivors are promoted to older generations collected less often. In general reference counting is favoured for predictable latency because you’re cleaning up incrementally as you go. Total memory footprint is similar to manual memory management with some overhead for counting refs. The cost is lower throughput as every reference change requires bookkeeping (see Swift ARC for a good example). Mark and sweep GCs are favoured for throughput as allocations and reference updates have zero overhead - you just bump a pointer to allocate. When collection does occur it can cause a pause, though modern concurrent collectors have greatly reduced this (see Java G1GC or .NET for good examples). Memory footprint is usually quite a bit larger than manual management. In the case of Clojure which in addition to being a LISP also uses immutable data structures, there is both object churn and frequent changes to the object graph. This makes throughput a much larger concern than a less allocation heavy language - favouring mark and sweep designs.

YuriNiyazov 1y ago

A long long time ago, at ClojureConj 2014, I asked Rich Hickey whether a cpp-based clojure was possible, and his answer was "well, the primary impediment there is a lack of a garbage collector". There were a lot of conversations going on at the same time, so I didn't get an opportunity to "delve" into it, but: 1. does that objection make sense? 2. How does jank approach that hurdle.

ethan_smith 1y ago
Jank likely uses a combination of LLVM's garbage collection support (GC intrinsics) and smart pointers, similar to how Clasp implemented GC for Common Lisp on C++.
bertmuthalaly 1y ago
It's the first section in the article - "I have implemented manual memory management via cpp/new and cpp/delete. This uses jank's GC allocator (currently bdwgc), rather than malloc, so using cpp/delete isn't generally needed. However, if cpp/delete is used then memory collection can be eager and more deterministic. The implementation has full bdwgc support for destructors as well, so both manual deletion and automatic collection will trigger non-trivial destructors."
Jeaye 1y ago
A GC is nowhere near the most difficult part of this. In 2014, there was no viable technology for JIT compiling C++, and very little technology for JIT compiling native code in general.
bluGill 1y ago
In the artical - it garbage collects always but if you call delete the garbage collector will be more agressive about cleaning that up.

caim 1y ago

That's great! Interop with C++ is such a complex task. Congratss on your work! It's definitely not an easy thing. I've always wondered what is the best way to interact with C++ template instantiation while keeping performance. For a static language, you'd probably need to translate your types to C++ during compilation, ask Clang/GCC/MSVC to compile the generated C++ file, and then link the final result. And finally, pray to the computer gods that name mangiling was done right.