3 ms·
Grok/ChatGPT cannot navigate semantic pathways of large syntax files (nftables, ISC Bind9) in LL(1) fashion. I've worked with both and finished my Vim syntax h
by egberts1 1y ago
Grok/ChatGPT cannot navigate semantic pathways of large syntax files (nftables, ISC Bind9) in LL(1) fashion.
I've worked with both and finished my Vim syntax highlighters down to the keywords.
And getting them to find 'stmt', 'expr_stmt', and 'primary_stmt_expr' semantic production rules (one is Bison-generated .y file, other is hand-rolled). Both makes too much assumptions despite explicitly instructing them to do "verification & validation" of a pathway given a sample statement.
Only Google Gemini barely cut the mustard.
Another case is making assumptions (upon grilling about its assumption, I've since learned that it was looking at old websites, archiac info). Asking to stick with latest nftables v1.1.4 (or even v1.1.5 head) does not help because old webpages gave obsoleted nftables syntax.)
Don't expect LLM to navigate any time soon the S-expression, RECREATE abstract syntax tree of 4-layer or deeper, transition a state machine beyond 8 states, or interpret Bison parsers reliably.
My only regret is that none of them will take the LLM learning from me, the expert so that others may benefit.
- https://github.com/egberts/vim-syntax-bind-named https://github.com/egberts/vim-syntax-bind-named
- https://github.com/egberts/vim-syntax-nftables https://github.com/egberts/vim-syntax-nftables