3 ms·
The linked-to example uses a query-dependent expression which resists easy pre-compilation. The author says the overhead of building a Python expression as a st
by eesmith 2y ago
The linked-to example uses a query-dependent expression which resists easy pre-compilation. The author says the overhead of building a Python expression as a string then byte compiling it is too high, which would also hold for a ctypes/cffi implementation.
In my case I had a lot of config dictionaries like:
default_output_args = {
"tsv": {"sep": "tab", "quoting": "always"},
"csv": {"sep": "comma", "quoting": "always"},
"json": {"indent": None, "allow_nan": True},
...
}
and a generic API like:
def open_writer(destination, format="csv", output_args=None):
...
which would let 'format' specify the output format, with the corresponding default arguments, and with the output_args able to override the defaults.
I found the generic API too annoying when I knew I wanted a given format, so I added format-specific APIs with kwargs set to the default values, like:
def open_csv_writer(destination, sep="tab", quoting="always"):
...
def open_json_writer(destination, indent=None, allow_nan=True):
...
And I wanted to keep the generic API and format-specific APIs synchronized, because the generic API is useful too.
My original version processed default_output_args to generate the byte code for the different writer functions, setting up the function definitions correctly.
This is not something which can be done with ctypes.
- 082349872349872 2y ago> resists easy pre-compilation Right, I was (in pre W^X days; compare Windows' BitBlt) compiling machine code onto the stack and executing it.
- eesmith 2y agoI still know people doing on-the-fly query compilation. They control the hardware it runs on, so they don't have to worry about that.