7 ms·

ClangQL: A tool to run SQL-like query on C/C++ Code

134 points by bubblehack3r 2y ago

giancarlostoro 2y ago
As cool as it looks, I have to ask: why?
- Hendrikto 2y ago
  One obvious example would be refactoring. Many patterns are not (easily) expressible as regexes.
  - eddd-ddde 2y ago
    There is also ast grep [0] 0: https://ast-grep.github.io/guide/introduction.html https://ast-grep.github.io/guide/introduction.html
  - giancarlostoro 2y ago
    Ah so this is useful for IDE tooling and I guess someone grokking to see how much impact a change has, yeah makes sense.
  - yodsanklai 2y ago
    Wouldn't you use tools like clang-tidy for that use case? wouldn't it be more flexible and general? Also does this project let you rewrite code or only run queries?
- deleted 2y ago
  [deleted]
- bastawhiz 2y ago
  How else are you supposed to fluff up your performance self review with real numbers when you can't use lines of code?
frabert 2y ago
I made a similar tool with the same name a couple of years ago :D https://github.com/frabert/ClangQL https://github.com/frabert/ClangQL
- mgaunard 2y ago
  Yours looks much better to be honest.
- wizzledonker 2y ago
  Yours actually looks useful for my use case (remote debugging wrappers) The schema of the linked post doesn't look useful to me at all haha
cozzyd 2y ago
Wonder how it deals with templates
morgante 2y ago
Cool to see another query language for source code! Yours is definitely closer to SQL than GritQL is.[0] I particularly like the count semantics. [0] https://github.com/getgrit/gritql https://github.com/getgrit/gritql
ranger_danger 2y ago
Do you have any binaries? Stock ubuntu 22.04 does not have new enough versions of things to build this.
wruza 2y ago
Why no line:col info?
mgaunard 2y ago
I'm not too familiar with Rust, but took a look at the Cargo.lock file of this project. It depends on half the universe for some reason, sometimes multiple versions of the same library. And that doesn't even include the main dependency which is libclang. Is the Rust ecosystem just dependency hell?
- lpribis 2y ago
  > Is the Rust ecosystem just dependency hell? Not quite to the extent of the js ecosystem, but yes. Especially for a purported systems language there's a lefpad-esque problem of people making tiny and somewhat useless libraries to learn or pad their resume which then get depended on by the world.
- plasticeagle 2y ago
  I think it's an inevitable outcome of a build system that makes it so easy to pull in packages - and so easy to create and publish them. This is why C++'s weakness - difficulty of consuming third party libraries - is actually a strength. If you have to work hard to get third party code in there, you tend to make much better choices and keep your dependencies to a minimum. In Rust, like JS and to a lesser extent Python also, there is no pressure to reduce dependencies. So you end up with this kind of a problem. Good luck upgrading one of those packages when its found to contain a bug.
  - demi56 2y ago
    [dead]
  - mgaunard 2y ago
    It's not difficult to consume third party C++ libraries -- what's difficult is to ensure third party code is maintained up to your standards and compatible with all the other choices you've made about your environment.
- tialaramex 2y ago
  My understanding is that a "dependency hell" first has to be a hell.
rodrigobellusci 2y ago
Maaaan I've wanted this for a while for Java and Dart (for flutter apps). Nice job!
owlstuffing 2y ago
Bit of a tangent, but if we commonly stored source in a standard, fully attributed AST instead of caveman text, caching/indexing and deterministic search would be a breeze. So would a million other useful applications such as every other IDE feature and source control. As the decades go by it confounds me that we are still messing with plain text, and with virtually no resistance to it. Maybe it's to keep the tabs vs. spaces flame war alive. _shrug_
- zokier 2y ago
  tbh the on-disk storage format is pretty irrelevant, its all just bits and bytes on disk anyways. for many cases you can consider source text a serialization of ast; no matter how you store it you still need to parse the data somehow. Sure some formats are easier to parse than others (hello sexprs), but that is just minor perf aspect, conceptually its all the same. this closely relates to the missing type discussion, there is no universal "tree" type, and as such it is also difficult to construct any universal "ast" type; different uses need trees structured in different ways: https://news.ycombinator.com/item?id=39592444 https://news.ycombinator.com/item?id=39592444
  - owlstuffing 2y ago
    > tbh the on-disk storage format is pretty irrelevant, it’s all just bits… Nonsense. A completely attributed AST is the result of a full parse. The information in the AST is vastly richer than source. > there is no universal tree type Obviously. It should be understood that the idea is to have a standard AST API/format _per language_. As such it would be standard practice to provide this information as part of the parser/compiler library.
- flohofwoe 2y ago
  This is such an obvious and old idea that it would have killed text as source code by now if it would be an actually *good* idea ;) (IIRC one attempt was an IDE from IBM in the 90s).
  - owlstuffing 2y ago
boywitharupee 2y ago
i wonder if we can train a foundational model on this data which will eventually allow to semantically search the codebase?

134 points by bubblehack3r 2y ago

giancarlostoro 2y ago

As cool as it looks, I have to ask: why?

Hendrikto 2y ago
One obvious example would be refactoring. Many patterns are not (easily) expressible as regexes.
- eddd-ddde 2y ago
  There is also ast grep [0] 0: https://ast-grep.github.io/guide/introduction.html https://ast-grep.github.io/guide/introduction.html
- giancarlostoro 2y ago
  Ah so this is useful for IDE tooling and I guess someone grokking to see how much impact a change has, yeah makes sense.
- yodsanklai 2y ago
  Wouldn't you use tools like clang-tidy for that use case? wouldn't it be more flexible and general? Also does this project let you rewrite code or only run queries?
deleted 2y ago
[deleted]
bastawhiz 2y ago
How else are you supposed to fluff up your performance self review with real numbers when you can't use lines of code?

frabert 2y ago

I made a similar tool with the same name a couple of years ago :D https://github.com/frabert/ClangQL https://github.com/frabert/ClangQL

mgaunard 2y ago
Yours looks much better to be honest.
wizzledonker 2y ago
Yours actually looks useful for my use case (remote debugging wrappers) The schema of the linked post doesn't look useful to me at all haha

cozzyd 2y ago

Wonder how it deals with templates

morgante 2y ago

Cool to see another query language for source code! Yours is definitely closer to SQL than GritQL is.[0] I particularly like the count semantics. [0] https://github.com/getgrit/gritql https://github.com/getgrit/gritql

ranger_danger 2y ago

Do you have any binaries? Stock ubuntu 22.04 does not have new enough versions of things to build this.

wruza 2y ago

Why no line:col info?

mgaunard 2y ago

I'm not too familiar with Rust, but took a look at the Cargo.lock file of this project. It depends on half the universe for some reason, sometimes multiple versions of the same library. And that doesn't even include the main dependency which is libclang. Is the Rust ecosystem just dependency hell?

lpribis 2y ago
> Is the Rust ecosystem just dependency hell? Not quite to the extent of the js ecosystem, but yes. Especially for a purported systems language there's a lefpad-esque problem of people making tiny and somewhat useless libraries to learn or pad their resume which then get depended on by the world.
plasticeagle 2y ago
I think it's an inevitable outcome of a build system that makes it so easy to pull in packages - and so easy to create and publish them. This is why C++'s weakness - difficulty of consuming third party libraries - is actually a strength. If you have to work hard to get third party code in there, you tend to make much better choices and keep your dependencies to a minimum. In Rust, like JS and to a lesser extent Python also, there is no pressure to reduce dependencies. So you end up with this kind of a problem. Good luck upgrading one of those packages when its found to contain a bug.
- demi56 2y ago
  [dead]
- mgaunard 2y ago
  It's not difficult to consume third party C++ libraries -- what's difficult is to ensure third party code is maintained up to your standards and compatible with all the other choices you've made about your environment.
tialaramex 2y ago
My understanding is that a "dependency hell" first has to be a hell.

rodrigobellusci 2y ago

Maaaan I've wanted this for a while for Java and Dart (for flutter apps). Nice job!

owlstuffing 2y ago

Bit of a tangent, but if we commonly stored source in a standard, fully attributed AST instead of caveman text, caching/indexing and deterministic search would be a breeze. So would a million other useful applications such as every other IDE feature and source control. As the decades go by it confounds me that we are still messing with plain text, and with virtually no resistance to it. Maybe it's to keep the tabs vs. spaces flame war alive. _shrug_

zokier 2y ago
tbh the on-disk storage format is pretty irrelevant, its all just bits and bytes on disk anyways. for many cases you can consider source text a serialization of ast; no matter how you store it you still need to parse the data somehow. Sure some formats are easier to parse than others (hello sexprs), but that is just minor perf aspect, conceptually its all the same. this closely relates to the missing type discussion, there is no universal "tree" type, and as such it is also difficult to construct any universal "ast" type; different uses need trees structured in different ways: https://news.ycombinator.com/item?id=39592444 https://news.ycombinator.com/item?id=39592444
- owlstuffing 2y ago
  > tbh the on-disk storage format is pretty irrelevant, it’s all just bits… Nonsense. A completely attributed AST is the result of a full parse. The information in the AST is vastly richer than source. > there is no universal tree type Obviously. It should be understood that the idea is to have a standard AST API/format _per language_. As such it would be standard practice to provide this information as part of the parser/compiler library.
flohofwoe 2y ago
This is such an obvious and old idea that it would have killed text as source code by now if it would be an actually *good* idea ;) (IIRC one attempt was an IDE from IBM in the 90s).
- owlstuffing 2y ago

boywitharupee 2y ago

i wonder if we can train a foundational model on this data which will eventually allow to semantically search the codebase?