3 ms·

This metaprogramming reminds me of the post where they serialized data to ruby code to get around a slow XML parser: https://news.ycombinator.com/item?id=363119

This metaprogramming reminds me of the post where they serialized data to ruby code to get around a slow XML parser: https://news.ycombinator.com/item?id=36311924 https://news.ycombinator.com/item?id=36311924 I’m curious if anyone has found a serious application for metaprogramming large amounts of data in a high level language like this? So far the only application I have thought of was meta programming a series of api calls before executing them so the entire transaction can be replayed as a script.

SCUSKU 3y ago
Somewhat pedantic here, but would the authors use of python to generate C code be considered metaprogramming as opposed to code generation? I thought metaprogramming typically involved writing code that can alter/augment itself, like new syntax. Personally, I have found a lot of use in code generation, for example using RTKQuery's codegen tool that accepts an OpenAPI schema and outputs a JS client.
- teo_zero 3y ago
  I would argue that code generation is a form of metaprogramming, no opposition here.
Retr0id 3y ago
A while back I was reverse engineering a mobile game that used a completely custom serialization format for game data, most of it in one giant "config" blob several megabytes in size. I'd written a parser and serializer for it in Python, using custom classes to represent the various nested object types, but I didn't have a convenient way to explore and edit the data. So, I wrote some code to parse the giant blob and emit a python source file, consisting of a whole bunch of nested class constructors (with named parameters etc. making it quite readable), and a final function call to serialize the whole lot back into a blob. Now I could edit that source file and run it, and have my source-level edits reflected in the output blob (which I could subsequently feed back into the game engine to see the results of my changes).
- baq 3y ago
  Note: this is essentially what the AWS CDK does. Yet another example that configures is code is configuration is code is…
rented_mule 3y ago
I don't know if I'd go as far as calling it serious, but... I have a string of 150 individually addressable RGB LEDs on a fake tree in my living room. It's controlled by an ESP32 (running ESPHome). I started generating little animations for different times of year (in the form of ESPHome light effects). I ran into a few problems: - The edit/compile/flash/debug cycle was painfully slow. - Expressing the animations in C++ sapped inspiration. - Some parts of the animations are more computationally intensive than others, so getting consistent "frame rates" was difficult. That "frame rates" bit made me realize these are just little videos with 150x1 pixel resolution. Now I write the animations in Python on my laptop and that code has two backends. One renders to the screen for quick iteration on the animations. The other plays one cycle through the loop of each animation and emits the results as C++ arrays and some variables (e.g., name, desired frame rate, etc.). Then the only other C++ code needed is the logic for passing the array data to the LED library. There is no compression or anything else fancy. I can fit about a dozen animations with 100 to 200 frames each on a $4 microcontroller with Wi-Fi. They start almost instantly when applying power, and they play at a very consistent frame rate. The highest I have tried is 60 fps and that is no problem.
- tda 3y ago
  I quit iterating on my animations (led strip based bedlight with ESPHome/ESP32) due to the exact same frustrations (and that writing c++ in a yaml file is not really great). Care to share the code for your approach?
Someone 3y ago
https://en.wikipedia.org/wiki/X_BitMap https://en.wikipedia.org/wiki/X_BitMap and https://en.wikipedia.org/wiki/X_PixMap https://en.wikipedia.org/wiki/X_PixMap come to mind, but they usually were written by an image editor, not by a program specifically written by the programmer who needed such code in their program.
vitiral 3y ago
Protocall buffers are effectively code generation and are central to Google's tech stack. Does that count?

SCUSKU 3y ago

Somewhat pedantic here, but would the authors use of python to generate C code be considered metaprogramming as opposed to code generation? I thought metaprogramming typically involved writing code that can alter/augment itself, like new syntax. Personally, I have found a lot of use in code generation, for example using RTKQuery's codegen tool that accepts an OpenAPI schema and outputs a JS client.

teo_zero 3y ago
I would argue that code generation is a form of metaprogramming, no opposition here.

Retr0id 3y ago

A while back I was reverse engineering a mobile game that used a completely custom serialization format for game data, most of it in one giant "config" blob several megabytes in size. I'd written a parser and serializer for it in Python, using custom classes to represent the various nested object types, but I didn't have a convenient way to explore and edit the data. So, I wrote some code to parse the giant blob and emit a python source file, consisting of a whole bunch of nested class constructors (with named parameters etc. making it quite readable), and a final function call to serialize the whole lot back into a blob. Now I could edit that source file and run it, and have my source-level edits reflected in the output blob (which I could subsequently feed back into the game engine to see the results of my changes).

baq 3y ago
Note: this is essentially what the AWS CDK does. Yet another example that configures is code is configuration is code is…

rented_mule 3y ago

I don't know if I'd go as far as calling it serious, but... I have a string of 150 individually addressable RGB LEDs on a fake tree in my living room. It's controlled by an ESP32 (running ESPHome). I started generating little animations for different times of year (in the form of ESPHome light effects). I ran into a few problems: - The edit/compile/flash/debug cycle was painfully slow. - Expressing the animations in C++ sapped inspiration. - Some parts of the animations are more computationally intensive than others, so getting consistent "frame rates" was difficult. That "frame rates" bit made me realize these are just little videos with 150x1 pixel resolution. Now I write the animations in Python on my laptop and that code has two backends. One renders to the screen for quick iteration on the animations. The other plays one cycle through the loop of each animation and emits the results as C++ arrays and some variables (e.g., name, desired frame rate, etc.). Then the only other C++ code needed is the logic for passing the array data to the LED library. There is no compression or anything else fancy. I can fit about a dozen animations with 100 to 200 frames each on a $4 microcontroller with Wi-Fi. They start almost instantly when applying power, and they play at a very consistent frame rate. The highest I have tried is 60 fps and that is no problem.

tda 3y ago
I quit iterating on my animations (led strip based bedlight with ESPHome/ESP32) due to the exact same frustrations (and that writing c++ in a yaml file is not really great). Care to share the code for your approach?

Someone 3y ago

https://en.wikipedia.org/wiki/X_BitMap https://en.wikipedia.org/wiki/X_BitMap and https://en.wikipedia.org/wiki/X_PixMap https://en.wikipedia.org/wiki/X_PixMap come to mind, but they usually were written by an image editor, not by a program specifically written by the programmer who needed such code in their program.

vitiral 3y ago

Protocall buffers are effectively code generation and are central to Google's tech stack. Does that count?