4 ms·

This looks impressive! I should spend some more time reading through what you have, but can you answer: what parts of this should I be looking at if I just wa

This looks impressive! I should spend some more time reading through what you have, but can you answer: what parts of this should I be looking at if I just want something to generate an AST for C99 (no transpiling needed)? There’s some source analysis I’d like to do (on student C code) and right now I’m considering the (python-based) C parser in CFFI, but your’s might be more complete?

thradams 4y ago
The part that is the "transpiler" is visit.c and visit.h. You can use it as reference to create different "visits". For instance, I am implementing a code format at "visitformat.c". I have implemented a "naming convention checker" inside the parser, but it also can be a "visit". Static analysis etc..can be a visit. Inside the visit context you will find struct ast ast; That is the AST. I would say the syntactic analysis parser is 100% complete C23. Semantic analysis that is not 100%.
G4E 4y ago
You can also use pycparser[0]. It is fully compatible C99, but be careful it doesn't support gnu extensions (like attributes, #indent, asm() ...). You can however work around most of them by -D defining them to empty macro in the argument. [0] https://github.com/eliben/pycparser https://github.com/eliben/pycparser
- an1sotropy 4y ago
  Right, pycparser is what CFFI uses. I’ve seen some really cryptic error messages when it tries to process some of my C header files (since worked around), and I’m curious what else is out there. The ability to preserve info about formatting that the OP noted is especially interesting. As long as we’re on this tangent, here’s the challenge I’m facing with automated analysis of student code: from foo.c make bar.c which is identical to foo.c except that comments have been turned into spaces. I think this is annoyingly non-trivial.
  - thradams 4y ago
    To remove comments from source all you need is a tokenizer. You don't need all the tokens, just the "preprocessor tokens". For instance literal strings, ppnumbers ... Then /comments/ can be replaced with 1 space and //comments by \n
    - an1sotropy 4y ago
      Multi-line comments would need to be turned into multiple blank lines, but yes, thank you for pointing out that I've been over-thinking this. I will look into what is the path of least resistance for this tokenizer-based transformation.

thradams 4y ago

The part that is the "transpiler" is visit.c and visit.h. You can use it as reference to create different "visits". For instance, I am implementing a code format at "visitformat.c". I have implemented a "naming convention checker" inside the parser, but it also can be a "visit". Static analysis etc..can be a visit. Inside the visit context you will find struct ast ast; That is the AST. I would say the syntactic analysis parser is 100% complete C23. Semantic analysis that is not 100%.

G4E 4y ago

You can also use pycparser[0]. It is fully compatible C99, but be careful it doesn't support gnu extensions (like attributes, #indent, asm() ...). You can however work around most of them by -D defining them to empty macro in the argument. [0] https://github.com/eliben/pycparser https://github.com/eliben/pycparser

an1sotropy 4y ago
Right, pycparser is what CFFI uses. I’ve seen some really cryptic error messages when it tries to process some of my C header files (since worked around), and I’m curious what else is out there. The ability to preserve info about formatting that the OP noted is especially interesting. As long as we’re on this tangent, here’s the challenge I’m facing with automated analysis of student code: from foo.c make bar.c which is identical to foo.c except that comments have been turned into spaces. I think this is annoyingly non-trivial.
- thradams 4y ago
  To remove comments from source all you need is a tokenizer. You don't need all the tokens, just the "preprocessor tokens". For instance literal strings, ppnumbers ... Then /comments/ can be replaced with 1 space and //comments by \n
  - an1sotropy 4y ago
    Multi-line comments would need to be turned into multiple blank lines, but yes, thank you for pointing out that I've been over-thinking this. I will look into what is the path of least resistance for this tokenizer-based transformation.