5 ms·

ICPP – Run C++ anywhere like a script

83 points by davikr 2y ago

celrod 2y ago
How feasible would it be for something like gdb to be able to use a C++ interpreter (whether icpp, or even a souped up `constexpr` interpreter from the compiler) to help with "optimized out" functions? gdb also doesn't handle overloaded functions well, e.g. `x[i]`.
- mananaysiempre 2y ago
  It does though? Just compiled a small program that creates a vector, and GDB is perfectly happy accessing it using this syntax. It will even print std::string’s correctly if you cast them to const char* by hand. (Linux x86-64, GDB 14.2.)
  - scintill76 2y ago
    > It will even print std::string’s correctly if you cast them to const char* by hand What does that mean? I think `print str.c_str()` has worked for me in GDB before, but sounds like you did something different.
    - deleted 2y ago
      [deleted]
    - mananaysiempre 2y ago
      I was observing that `p (const char *)str` also worked in my experiment, but I’m far from a C++ expert and upon double-checking this seems to have been more of an accident than intended behaviour, because there is no operator const_pointer in basic_string that I can find. Definitely use `p str.c_str()`.
      - twoodfin 2y ago
        If your std::string was using a short string optimization, that would explain the “accident”. Some implementations even put char[0] at the first byte in the optimized form.
        scintill76 2y ago
        That explanation doesn't work IMO, unless `str` is a std::string pointer, which is contrary to the syntax GP suggested with `str.c_str()`. It doesn't seem possible in actual C++ that the cast from non-pointer to pointer would work at all (even if a small string happens to be inlined at offset 0.) Like GP, I looked for a conversion operator, and I don't think it's there. Maybe it is a feature of the gdb parser.
        twoodfin 2y ago
        Good point, but if it’s a long string, 2/3 of the most common implementations would make the first word the c_str()-equivalent pointer: https://devblogs.microsoft.com/oldnewthing/20240510-00/?p=109742 https://devblogs.microsoft.com/oldnewthing/20240510-00/?p=10...
        fluoridation 2y ago
        So it's actually printing *(const char **)&s?
        twoodfin 2y ago
        The first pointer-sized chunk of the string structure is a pointer to the C-string representation. So the cast works as written.
        fluoridation 2y ago
        Well, no, because (const char *)str is nonsense, if str is an std::string.
  - celrod 2y ago
    I've defined a few pretty printers, but `operator[]` doesn't work for my user-defined types. Knowing it works for vectors, I'll try and experiment to see if there's something that'll make it work. (gdb) p unrolls_[0] Could not find operator[]. (gdb) p unrolls_[(long)0] Could not find operator[]. (gdb) p unrolls_.data_.mem[0] $2 = { `unrolls_[i]` works within C++. This `operator[]` method isn't even templated (although the container type is); the index is hard-coded to be of type `ptrdiff_t`, which is `long` on my platform. I'm on Linux, gdb 15.1.
    - mananaysiempre 2y ago
      > This `operator[]` method isn't even templated (although the container type is) That might be it. If that operator isn’t actually ever emitted out of line, then GDB will (naturally) have nothing to call. If it helps, with the following program template<typename T> struct Foo { int operator[](long i) { return i * 3; } }; Foo<bool> bar; template int Foo<bool>::operator[](long); // [*] int main(void) { Foo<int> foo; __asm__("int3"); return foo[19]; } compiled at -g -O0 I can both `p foo[19]` and `p bar[19]`, but if I comment out the explicit instantiation marked [*], the latter no longer works. At -g -O2, the former does not work because `foo` no longer actually exists, but the latter does, provided the instantiation is left in.
      - celrod 2y ago
        Can confirm, this works for me in my actual examples, thanks!
- tester756 2y ago
  this "optimized out" thing is bullshit as hell
- Conscat 2y ago
  GDB does have hooks for interpreters to be executed within it, but I haven't managed to make this work. https://sourceware.org/gdb/current/onlinedocs/gdb.html/JIT-Interface.html#:~:text=This%20chapter%20documents%20GDB's%20just,performance%20while%20maintaining%20platform%20independence https://sourceware.org/gdb/current/onlinedocs/gdb.html/JIT-I....
Klasiaster 2y ago
Related is cargo script for Rust: https://doc.rust-lang.org/cargo/reference/unstable.html#script https://doc.rust-lang.org/cargo/reference/unstable.html#scri... (nightly only)
- ComputerGuru 2y ago
  You can do it out of the box with rust, no need for any tools, because you can strategically mix shell and rust in the same code: https://neosmart.net/blog/self-compiling-rust-code/ https://neosmart.net/blog/self-compiling-rust-code/
bsenftner 2y ago
Why interpret at all? Back in the mid to early 90's I started embedding C++ compilers into the game engines I wrote, where the "game scripting language" was just #define macros hiding the C++ syntax so the game level developers, who worked in this script, could be basically anyone that could code. Their "script" would compile to a DLL that was hot loaded. What they were doing in their scripts would compile in under 5 seconds, and they were good to go. If they ran into problems, one of the game engine developers would just run their "script" in the IDE debugger. Borrowed this idea from Nothing Real, the developers of Shake, the video/film compositing system.
- jokoon 2y ago
  I don't suppose modern os would let you do that today, sounds like a security nightmare
  - shortrounddev2 2y ago
    This is just JIT'ing. It's used in python and lua commonly
    - pjmlp 2y ago
      I wish, JIT and Python still isn't something to brag about.
      - greenavocado 2y ago
        Python 3.13 has an experimental JIT compiler https://peps.python.org/pep-0744/ https://peps.python.org/pep-0744/
        pjmlp 2y ago
        As I said, not something to brag about.
    - HelloNurse 2y ago
      It's just loading a dynamic library from an arbitrary file, as traditionally done for third party software addons (e.g. Photoshop plugins since the early 1990s and VST instruments and effects since the late 1990s).
bobajeff 2y ago
I wonder if I can use this to learn a large c++ codebase like Chromium. One of the issues I had trying to learn chromium was that in order to play and experiment with their classes/functions I needed to spend several minutes to link my little test code with their static libraries just to be able to see if my understanding of them was correct. Which is just too long of time for such experiments so I gave up.
- hackit2 2y ago
  Last time I checked out Chromium code base it was about 300-400 Megs of uncompressed cpp files. Lets not also forget the fact you also needed to run some code generator script that generated another 200 Megs of DOM files, or interface files. At that point in time I gave up and went to sleep and never touched it again.
  - lioeters 2y ago
    I really hope Ladybird is able to stay relatively small and approachable, it would be wonderful to have as a truly customizable open-source browser that's not a massive codebase that takes forever to compile and develop.
    - hackit2 2y ago
      I do think there are some geniuses working on the Chromium code base, and I would imagine there are really good reasons for doing it their way. I would imagine also Ladybird over time will face the same problems, and come up with similar solutions as Chromium team. All I know is most of the large scale C/C++ code bases eventually become these monolithic giant code bases that require some really specialised software tools to compile and link.
      - pjmlp 2y ago
        Not only C and C++, I have seen this scale with most Fortune 500 projects I have been involved.
- 2y ago
ranger_danger 2y ago
This is really cool... https://github.com/vpand/icpp-qt https://github.com/vpand/icpp-qt Ok now things are getting interesting. I think this could be used to add easily shareable/hackable plugins to existing C++ projects.
- fluoridation 2y ago
  I've done it by embedding libclang into an executable. You still have to be really careful to keep ABI compatibility between the host and the JITed plugin, if you want to send and receive complex C++ objects. Most likely you'll need to set up a simple C ABI and reconstruct the objects on either side of the interface. The last thing you want is to send std::string across a DLL boundary.
Jeaye 2y ago
Along the lines of scripting is interactive programming. I'm working on a native Clojure dialect on LLVM with C++ interop, qalled jank. It can JIT compile C++ code, can be embedded into any C++-compatible application, and is a full Clojure dialect which doesn't hid any of its C++ runtinme. So you can do inline C++, compile C++ sources alongside your jank. and require them like a normal Clojure namespace. Worth a look if you're using C++ but you're craving something more interactive. https://jank-lang.org/ https://jank-lang.org/
mindblah 2y ago
Folks who like this kind of thing should definitely check out CERN's Root framework. I've been using its C++ interpreter in a Jupyter notebook environment to learn C++. It's probably also quite a bit more mature than this project. https://root.cern/ https://root.cern/
- rramadass 2y ago
  This is Great! Thanks for the pointer. Any other great open source tools like this that you can share?
  - mindblah 2y ago
    Well, one thing you can use alongside this project is a small library called cpp-dump, which lets you pretty print variables. https://github.com/philip82148/cpp-dump https://github.com/philip82148/cpp-dump It's just a normal library you can use with any compiled project, but it works nicely with Root C++ for built-in and std types. Great if (say) you're working through implementing vector or linear algebra and want a nice way to display your multi-dimensional arrays and vectors. Just copy the project folder somewhere (most conveniently where you invoke the interpreter from), do `#include "cpp-dump/dump.hpp` and then `cpp_dump(myVariable)` to print your variables. You can see how it looks in this example where I was mucking about with permutations of vectors: https://i.imgur.com/yRpY5Bj.png https://i.imgur.com/yRpY5Bj.png

83 points by davikr 2y ago

How feasible would it be for something like gdb to be able to use a C++ interpreter (whether icpp, or even a souped up `constexpr` interpreter from the compiler) to help with "optimized out" functions? gdb also doesn't handle overloaded functions well, e.g. `x[i]`.

mananaysiempre 2y ago
It does though? Just compiled a small program that creates a vector, and GDB is perfectly happy accessing it using this syntax. It will even print std::string’s correctly if you cast them to const char* by hand. (Linux x86-64, GDB 14.2.)
- scintill76 2y ago
  > It will even print std::string’s correctly if you cast them to const char* by hand What does that mean? I think `print str.c_str()` has worked for me in GDB before, but sounds like you did something different.
  - deleted 2y ago
    [deleted]
  - mananaysiempre 2y ago
    I was observing that `p (const char *)str` also worked in my experiment, but I’m far from a C++ expert and upon double-checking this seems to have been more of an accident than intended behaviour, because there is no operator const_pointer in basic_string that I can find. Definitely use `p str.c_str()`.
    - twoodfin 2y ago
      If your std::string was using a short string optimization, that would explain the “accident”. Some implementations even put char[0] at the first byte in the optimized form.
      - scintill76 2y ago
        That explanation doesn't work IMO, unless `str` is a std::string pointer, which is contrary to the syntax GP suggested with `str.c_str()`. It doesn't seem possible in actual C++ that the cast from non-pointer to pointer would work at all (even if a small string happens to be inlined at offset 0.) Like GP, I looked for a conversion operator, and I don't think it's there. Maybe it is a feature of the gdb parser.
        twoodfin 2y ago
        Good point, but if it’s a long string, 2/3 of the most common implementations would make the first word the c_str()-equivalent pointer: https://devblogs.microsoft.com/oldnewthing/20240510-00/?p=109742 https://devblogs.microsoft.com/oldnewthing/20240510-00/?p=10...
        fluoridation 2y ago
        So it's actually printing *(const char **)&s?
        twoodfin 2y ago
        The first pointer-sized chunk of the string structure is a pointer to the C-string representation. So the cast works as written.
        fluoridation 2y ago
        Well, no, because (const char *)str is nonsense, if str is an std::string.
- celrod 2y ago
  I've defined a few pretty printers, but `operator[]` doesn't work for my user-defined types. Knowing it works for vectors, I'll try and experiment to see if there's something that'll make it work. (gdb) p unrolls_[0] Could not find operator[]. (gdb) p unrolls_[(long)0] Could not find operator[]. (gdb) p unrolls_.data_.mem[0] $2 = { `unrolls_[i]` works within C++. This `operator[]` method isn't even templated (although the container type is); the index is hard-coded to be of type `ptrdiff_t`, which is `long` on my platform. I'm on Linux, gdb 15.1.
  - mananaysiempre 2y ago
    > This `operator[]` method isn't even templated (although the container type is) That might be it. If that operator isn’t actually ever emitted out of line, then GDB will (naturally) have nothing to call. If it helps, with the following program template<typename T> struct Foo { int operator[](long i) { return i * 3; } }; Foo<bool> bar; template int Foo<bool>::operator[](long); // [*] int main(void) { Foo<int> foo; __asm__("int3"); return foo[19]; } compiled at -g -O0 I can both `p foo[19]` and `p bar[19]`, but if I comment out the explicit instantiation marked [*], the latter no longer works. At -g -O2, the former does not work because `foo` no longer actually exists, but the latter does, provided the instantiation is left in.
    - celrod 2y ago
      Can confirm, this works for me in my actual examples, thanks!
tester756 2y ago
this "optimized out" thing is bullshit as hell
Conscat 2y ago
GDB does have hooks for interpreters to be executed within it, but I haven't managed to make this work. https://sourceware.org/gdb/current/onlinedocs/gdb.html/JIT-Interface.html#:~:text=This%20chapter%20documents%20GDB's%20just,performance%20while%20maintaining%20platform%20independence https://sourceware.org/gdb/current/onlinedocs/gdb.html/JIT-I....

Klasiaster 2y ago

Related is cargo script for Rust: https://doc.rust-lang.org/cargo/reference/unstable.html#script https://doc.rust-lang.org/cargo/reference/unstable.html#scri... (nightly only)

ComputerGuru 2y ago
You can do it out of the box with rust, no need for any tools, because you can strategically mix shell and rust in the same code: https://neosmart.net/blog/self-compiling-rust-code/ https://neosmart.net/blog/self-compiling-rust-code/

bsenftner 2y ago

Why interpret at all? Back in the mid to early 90's I started embedding C++ compilers into the game engines I wrote, where the "game scripting language" was just #define macros hiding the C++ syntax so the game level developers, who worked in this script, could be basically anyone that could code. Their "script" would compile to a DLL that was hot loaded. What they were doing in their scripts would compile in under 5 seconds, and they were good to go. If they ran into problems, one of the game engine developers would just run their "script" in the IDE debugger. Borrowed this idea from Nothing Real, the developers of Shake, the video/film compositing system.

jokoon 2y ago
I don't suppose modern os would let you do that today, sounds like a security nightmare
- shortrounddev2 2y ago
  This is just JIT'ing. It's used in python and lua commonly
  - pjmlp 2y ago
    I wish, JIT and Python still isn't something to brag about.
    - greenavocado 2y ago
      Python 3.13 has an experimental JIT compiler https://peps.python.org/pep-0744/ https://peps.python.org/pep-0744/
      - pjmlp 2y ago
        As I said, not something to brag about.
  - HelloNurse 2y ago
    It's just loading a dynamic library from an arbitrary file, as traditionally done for third party software addons (e.g. Photoshop plugins since the early 1990s and VST instruments and effects since the late 1990s).

bobajeff 2y ago

I wonder if I can use this to learn a large c++ codebase like Chromium. One of the issues I had trying to learn chromium was that in order to play and experiment with their classes/functions I needed to spend several minutes to link my little test code with their static libraries just to be able to see if my understanding of them was correct. Which is just too long of time for such experiments so I gave up.

hackit2 2y ago
Last time I checked out Chromium code base it was about 300-400 Megs of uncompressed cpp files. Lets not also forget the fact you also needed to run some code generator script that generated another 200 Megs of DOM files, or interface files. At that point in time I gave up and went to sleep and never touched it again.
- lioeters 2y ago
  I really hope Ladybird is able to stay relatively small and approachable, it would be wonderful to have as a truly customizable open-source browser that's not a massive codebase that takes forever to compile and develop.
  - hackit2 2y ago
    I do think there are some geniuses working on the Chromium code base, and I would imagine there are really good reasons for doing it their way. I would imagine also Ladybird over time will face the same problems, and come up with similar solutions as Chromium team. All I know is most of the large scale C/C++ code bases eventually become these monolithic giant code bases that require some really specialised software tools to compile and link.
    - pjmlp 2y ago
      Not only C and C++, I have seen this scale with most Fortune 500 projects I have been involved.
2y ago

ranger_danger 2y ago

This is really cool... https://github.com/vpand/icpp-qt https://github.com/vpand/icpp-qt Ok now things are getting interesting. I think this could be used to add easily shareable/hackable plugins to existing C++ projects.

fluoridation 2y ago
I've done it by embedding libclang into an executable. You still have to be really careful to keep ABI compatibility between the host and the JITed plugin, if you want to send and receive complex C++ objects. Most likely you'll need to set up a simple C ABI and reconstruct the objects on either side of the interface. The last thing you want is to send std::string across a DLL boundary.

Jeaye 2y ago

Along the lines of scripting is interactive programming. I'm working on a native Clojure dialect on LLVM with C++ interop, qalled jank. It can JIT compile C++ code, can be embedded into any C++-compatible application, and is a full Clojure dialect which doesn't hid any of its C++ runtinme. So you can do inline C++, compile C++ sources alongside your jank. and require them like a normal Clojure namespace. Worth a look if you're using C++ but you're craving something more interactive. https://jank-lang.org/ https://jank-lang.org/

mindblah 2y ago

Folks who like this kind of thing should definitely check out CERN's Root framework. I've been using its C++ interpreter in a Jupyter notebook environment to learn C++. It's probably also quite a bit more mature than this project. https://root.cern/ https://root.cern/

rramadass 2y ago
This is Great! Thanks for the pointer. Any other great open source tools like this that you can share?
- mindblah 2y ago
  Well, one thing you can use alongside this project is a small library called cpp-dump, which lets you pretty print variables. https://github.com/philip82148/cpp-dump https://github.com/philip82148/cpp-dump It's just a normal library you can use with any compiled project, but it works nicely with Root C++ for built-in and std types. Great if (say) you're working through implementing vector or linear algebra and want a nice way to display your multi-dimensional arrays and vectors. Just copy the project folder somewhere (most conveniently where you invoke the interpreter from), do `#include "cpp-dump/dump.hpp` and then `cpp_dump(myVariable)` to print your variables. You can see how it looks in this example where I was mucking about with permutations of vectors: https://i.imgur.com/yRpY5Bj.png https://i.imgur.com/yRpY5Bj.png