5 ms·

i commented on reddit (and got promptly downvoted) but since i think jank's author is around here (and hopefully is receptive to constructive criticism): the Cp

i commented on reddit (and got promptly downvoted) but since i think jank's author is around here (and hopefully is receptive to constructive criticism): the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. there's no reason to do this except that libclang currently doesn't support any other way. that's not jank's fault but it could be "fixed" in libclang. at a minimum you could use https://github.com/llvm/llvm-project/blob/main/clang/lib/CodeGen/CodeGenTypes.h https://github.com/llvm/llvm-project/blob/main/clang/lib/Cod... to emit the code based on clang ast. at a maximum would be to use something like https://github.com/Mr-Anyone/abi https://github.com/Mr-Anyone/abi or this if/when it comes to fruition https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-library/84554 https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-... to generate ABI compliant calls/etc for cpp libs. note, i say all this with maximum love in my heart for a language that would have first class cpp interop - i would immediately become jank's biggest proponent/user if its cpp interop were robust. EDIT: for people wanting/needing receipts, you can skim through https://github.com/compiler-research/CppInterOp/blob/main/lib/CppInterOp/CppInterOp.cpp#L2742-L2756 https://github.com/compiler-research/CppInterOp/blob/main/li...

wk_end 1y ago
> the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. So, I agree that this sounds janky as heck. My question is: besides sounding janky as heck, is there something wrong with this? Is it slow/unreliable?
- almostgotcaught 1y ago
  i mean it's as prone to error as any other thing that relies on string munging. it's probably not that much slower than the alternative i proposed - because the trampolines/wrappers are jitted and then reused - but it's just not robust enough that i would ever imagine building a prod system on top of it (eg using cppyy in prod) let alone baking it into my language/runtime.
  - refulgentis 1y ago
    The delta between the title and the content gave me extreme pause, thanks for sharing that there's, uh, worse problems. I'm a bit surprised I've seen two articles about jank here the last 2 days if these are exemplars of the technical approach and communication style. Seems like that wouldn't be enough to get on people's radars.
    - actionfromafar 1y ago
      Given how the world works, that might mean we will all sit and curse Jank instead of cursing Node. :)
    - Jeaye 1y ago
      Which particular delta between the title and the content gave you extreme pause?
      - refulgentis 1y ago
        It said "jank is C++", which I assumed would be explaining that jank compiles down to C++ or something similar, i.e. there is a layer of abstraction between jank and C++, but it effectively "works like" C++. On re-read, I recognize where it is used in the article: "jank is C++. There is no runtime reflection, no guess work, and no hints. If the compiler can't find a member, or a function, or a particular overload, you will get a compiler error." I assume other interop scenarios don't pull this off*, thus it is distinctive. Additionally, I'm not at all familiar with Clojure, sadly, but it also sounds like there's some special qualities there ("I think that this is an interesting way to start thinking about jank, Clojure, and static types") Now I'll riff and just write out the first 3-5 titles that come to mind with that limited understanding: - Implementing compile-time verifiable C++ interop in jank - Sparks of C++ interop: jank, Clojure, & verifying interop before runtime - jank's progress on C++ interop - Safe C++ interop lessons from jank * for example, I write a lot of Dart day to day and rely on Dart's "FFI" implementation to call C++, which now that I'm thinking about, only works because there's a code generator that creates "Dart headers" (my term) for the C++ libraries. I could totally footgun and call arbitrary functions that don't exist.
Jeaye 1y ago
Hey! I'm here and receptive. I completely agree that Clang could solve this by actually supporting my use case. Unfortunately, Clang is very much designed for standalone AOT compilation, not intertwined with another IR generating mechanism. Furthermore, Clang struggles to handle some errors gracefully which can get it into a bad state. I have grown jank's fork of CppInterOp quite significantly, in the past quarter, with the full change list being here: https://gist.github.com/jeaye/f6517e52f1b2331d294caed70119f15f https://gist.github.com/jeaye/f6517e52f1b2331d294caed70119f1... Hoping to get all of this upstreamed, but it's a lot of work that is not high priority for me right now. I think, based on my experience in the guts of CppInterOp, that the largest issue is not the C++ code generation. Basically any code generation is some form of string building. You linked to a part of CppInterOp which is constructing C++ functions. What's _actually_ wrong with that, in terms of robustness? The strings are generated not based on arbitrary user input, but based on Clang QualTypes and Decls. i.e. you need valid Clang values to actually get there anyway. Given that the ABI situation is an absolute mess, and that jank is already using Clang's JIT C++ compiler, I think this is a very viable solution. However, in terms of robustness, I go back to Clang's error handling, lack of grace, and poor tooling for use cases like this. Based on my experience, _that_ is what will cause robustness issues. Please don't take my response as unreceptive or defensive. I really do appreciate the discussion and if I'm saying something wrong, or if you want to explain further, please do. For alternatives, you linked to https://github.com/Mr-Anyone/abi https://github.com/Mr-Anyone/abi which is 3 months old and has 0 stars (and so I assume 0 users and 0 years of battle testing). You also linked to https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-library/84554 https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-... which I agree would be great, _if/when it becomes available_. So, out of all of the options, I'll ask clearly and sincerely: is there really a _better_ option which exists today? CppInterOp is an implementation detail of jank. If we can replace C++ string generation with more IR generation and a portable ABI mechanism, _and_ if Clang can provide the sufficient libraries to make it so that I don't need to rely on C++ strings to be certain that my template specializations get the correct instantiation, I am definitely open to replacing CppInterOp. From all I've seen, we're not there yet.
- almostgotcaught 1y ago
  > which is 3 months old and has 0 stars (and so I assume 0 users and 0 years of battle testing) ah my bad i meant to link to this one https://github.com/scrossuk/llvm-abi https://github.com/scrossuk/llvm-abi which inspired the gsoc. > is there really a _better_ option which exists today? today the "best in class" approach is swift's which fully (well tries to) model cpp AST and do what i suggested (emitting code directly): https://github.com/swiftlang/swift/blob/c09135b8f30c0cec8f5f1fee7e09e55184164ef2/lib/IRGen/GenCall.cpp#L2335-L2359 https://github.com/swiftlang/swift/blob/c09135b8f30c0cec8f5f...

wk_end 1y ago

> the CppInterOp approach to cpp interop is completely janky (no pun intended). the approach literally string munges cpp and then parses/interprets it to emit ABI compliant calls. So, I agree that this sounds janky as heck. My question is: besides sounding janky as heck, is there something wrong with this? Is it slow/unreliable?

almostgotcaught 1y ago
i mean it's as prone to error as any other thing that relies on string munging. it's probably not that much slower than the alternative i proposed - because the trampolines/wrappers are jitted and then reused - but it's just not robust enough that i would ever imagine building a prod system on top of it (eg using cppyy in prod) let alone baking it into my language/runtime.
- refulgentis 1y ago
  The delta between the title and the content gave me extreme pause, thanks for sharing that there's, uh, worse problems. I'm a bit surprised I've seen two articles about jank here the last 2 days if these are exemplars of the technical approach and communication style. Seems like that wouldn't be enough to get on people's radars.
  - actionfromafar 1y ago
    Given how the world works, that might mean we will all sit and curse Jank instead of cursing Node. :)
  - Jeaye 1y ago
    Which particular delta between the title and the content gave you extreme pause?
    - refulgentis 1y ago
      It said "jank is C++", which I assumed would be explaining that jank compiles down to C++ or something similar, i.e. there is a layer of abstraction between jank and C++, but it effectively "works like" C++. On re-read, I recognize where it is used in the article: "jank is C++. There is no runtime reflection, no guess work, and no hints. If the compiler can't find a member, or a function, or a particular overload, you will get a compiler error." I assume other interop scenarios don't pull this off*, thus it is distinctive. Additionally, I'm not at all familiar with Clojure, sadly, but it also sounds like there's some special qualities there ("I think that this is an interesting way to start thinking about jank, Clojure, and static types") Now I'll riff and just write out the first 3-5 titles that come to mind with that limited understanding: - Implementing compile-time verifiable C++ interop in jank - Sparks of C++ interop: jank, Clojure, & verifying interop before runtime - jank's progress on C++ interop - Safe C++ interop lessons from jank * for example, I write a lot of Dart day to day and rely on Dart's "FFI" implementation to call C++, which now that I'm thinking about, only works because there's a code generator that creates "Dart headers" (my term) for the C++ libraries. I could totally footgun and call arbitrary functions that don't exist.

Jeaye 1y ago

Hey! I'm here and receptive. I completely agree that Clang could solve this by actually supporting my use case. Unfortunately, Clang is very much designed for standalone AOT compilation, not intertwined with another IR generating mechanism. Furthermore, Clang struggles to handle some errors gracefully which can get it into a bad state. I have grown jank's fork of CppInterOp quite significantly, in the past quarter, with the full change list being here: https://gist.github.com/jeaye/f6517e52f1b2331d294caed70119f15f https://gist.github.com/jeaye/f6517e52f1b2331d294caed70119f1... Hoping to get all of this upstreamed, but it's a lot of work that is not high priority for me right now. I think, based on my experience in the guts of CppInterOp, that the largest issue is not the C++ code generation. Basically any code generation is some form of string building. You linked to a part of CppInterOp which is constructing C++ functions. What's _actually_ wrong with that, in terms of robustness? The strings are generated not based on arbitrary user input, but based on Clang QualTypes and Decls. i.e. you need valid Clang values to actually get there anyway. Given that the ABI situation is an absolute mess, and that jank is already using Clang's JIT C++ compiler, I think this is a very viable solution. However, in terms of robustness, I go back to Clang's error handling, lack of grace, and poor tooling for use cases like this. Based on my experience, _that_ is what will cause robustness issues. Please don't take my response as unreceptive or defensive. I really do appreciate the discussion and if I'm saying something wrong, or if you want to explain further, please do. For alternatives, you linked to https://github.com/Mr-Anyone/abi https://github.com/Mr-Anyone/abi which is 3 months old and has 0 stars (and so I assume 0 users and 0 years of battle testing). You also linked to https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-library/84554 https://discourse.llvm.org/t/llvm-introduce-an-abi-lowering-... which I agree would be great, _if/when it becomes available_. So, out of all of the options, I'll ask clearly and sincerely: is there really a _better_ option which exists today? CppInterOp is an implementation detail of jank. If we can replace C++ string generation with more IR generation and a portable ABI mechanism, _and_ if Clang can provide the sufficient libraries to make it so that I don't need to rely on C++ strings to be certain that my template specializations get the correct instantiation, I am definitely open to replacing CppInterOp. From all I've seen, we're not there yet.

almostgotcaught 1y ago
> which is 3 months old and has 0 stars (and so I assume 0 users and 0 years of battle testing) ah my bad i meant to link to this one https://github.com/scrossuk/llvm-abi https://github.com/scrossuk/llvm-abi which inspired the gsoc. > is there really a _better_ option which exists today? today the "best in class" approach is swift's which fully (well tries to) model cpp AST and do what i suggested (emitting code directly): https://github.com/swiftlang/swift/blob/c09135b8f30c0cec8f5f1fee7e09e55184164ef2/lib/IRGen/GenCall.cpp#L2335-L2359 https://github.com/swiftlang/swift/blob/c09135b8f30c0cec8f5f...