4 ms·
Is it possible to combine n-gram and AST to dump a better indexing? Take `sourceCode.toString()` as an example, the AST can dump it to `sourceCode` and `toStri
by herrington_d 3y ago
Is it possible to combine n-gram and AST to dump a better indexing?
Take `sourceCode.toString()` as an example, the AST can dump it to `sourceCode` and `toString`. A further indexer can break `sourceCode` to `source` and `code`.
For ast dumping, project like https://github.com/ast-grep/ast-grep https://github.com/ast-grep/ast-grep can help.
- boyter 3y agoYou could, but I don't know what you gain out of it. The underlying index would be almost the same size, and n-gram would also allow you to search for e.t for example which you are losing in this process.