13 ms·
For me, this is the critical bit here: > One of the problems of patching code using ast package of Python is that it loses all the formatting and comments of t
by heinrichhartman 5y ago
For me, this is the critical bit here:
> One of the problems of patching code using ast package of Python is that it loses all the formatting and comments of the original source code. This can be solved by making the patch script a little smarter. Instead of having it unparse the entire patched AST and write that to disk, we can make it unparse only the nodes it modified and insert the modified code at the corresponding line number in the file. The ast nodes have lineno attribute that can be used to retrieve the line number of the file to be injected the patched code with.
Parsing & Patching the AST is relatively easy, with the provided tooling.
I always struggled to write the modifications out in a minimal way.
Looks like line-number information does get you over this hump.
EDIT: Looks like the python community has already a package for this: https://github.com/PyCQA/redbaron https://github.com/PyCQA/redbaron
- shpx 5y agoAutoformatting the code with Black before and after unparsing is how I avoided this problem.
- trankas 5y agoNever used it but it appears that Facebook/Instagram have a format preserving CST library for Python: https://github.com/Instagram/LibCST https://github.com/Instagram/LibCST
- BiteCode_dev 5y agoFor preserving comments, use redbaron: https://github.com/PyCQA/redbaron https://github.com/PyCQA/redbaron It's lossless, besides, the API is much nicer.
- megous 5y agoI wrote a tool for refactoring JS a few years ago, and the way I solved this was to not modify the AST and regenerate the code from it, but only to use position information from it, and patch source code directly based on that positional information.