4 ms·

https://github.com/shish/pgosquery/blob/master/pgosquery/init.py https://github.com/shish/pgosquery/blob/master/pgosquery/__i... Wow. That does look easy.

https://github.com/shish/pgosquery/blob/master/pgosquery/__init__.py https://github.com/shish/pgosquery/blob/master/pgosquery/__i... Wow. That does look easy.

Shish2k 12y ago
Indeed it is; TBH, having seen how easy this was to make, I wonder if there would be any merit to making a stripped-down postgres package which is just daemon + SQL parser + FDW interface (so no built-in storage engine) -- the FDW interface is amazing for building SQL interfaces for arbitrary data sets, but the whole postgres bundle seems a little heavy-weight and complex to install if that's the only thing you want it for... Or I guess you could start from the other direction, and make a FDW/Multicorn-compatible API for SQLite? (Facebook's OSQuery is SQLite-based, but it looks like the virtual table interface over there is way more complex)
- mamcx 12y ago
  And the opposite? Have the sqlite storage but no the sql parser?
  - rakoo 12y ago
    What would be the use case ? Data already exists on the OS, no need to copy it on another storage and risk having stale values.
- crxgames 12y ago
  That would remove some of my hesitancy in letting this run as root for full process information. At least it limits what code could be exploited in the daemon process to some extent.
- johnteslade 12y ago
  I tried this with sqlite and the apsw python library. A quick example is at https://github.com/johnteslade/pysqliteosquery https://github.com/johnteslade/pysqliteosquery The interface is not quite as nice as Multicorn's but certainly nicer than the C/C++ SQLlite interface. But you could probably create a wrapper around apsw so it looked similar to Multicorn.
  - Erwin 12y ago
    FYI, you can use ":memory:" as the sqlite data source, instead of having to conditionally drop the tables (unless you actually do want to store some persistent data on disk).
    - johnteslade 12y ago
      Good spot. The in memory table also runs 3 times quicker.
- batbomb 12y ago
  Don't forget about Presto
hippowithgas 12y ago
I can see we are going to get one of these DB / OS Adapters each day now. Looking forward to BerkDBOSQuery - because, well, the Berkeley DB needs some love too.
orf 12y ago
Yeah, heres a link to the awesome psutil library it uses: https://github.com/giampaolo/psutil https://github.com/giampaolo/psutil It could be made a bit more efficient by filtering the processes[1], but the code would not look nearly as lovely. 1. http://multicorn.org/implementing-an-fdw/#idoptimizations http://multicorn.org/implementing-an-fdw/#idoptimizations

https://github.com/shish/pgosquery/blob/master/pgosquery/__init__.py https://github.com/shish/pgosquery/blob/master/pgosquery/__i... Wow. That does look easy.

Shish2k 12y ago

Indeed it is; TBH, having seen how easy this was to make, I wonder if there would be any merit to making a stripped-down postgres package which is just daemon + SQL parser + FDW interface (so no built-in storage engine) -- the FDW interface is amazing for building SQL interfaces for arbitrary data sets, but the whole postgres bundle seems a little heavy-weight and complex to install if that's the only thing you want it for... Or I guess you could start from the other direction, and make a FDW/Multicorn-compatible API for SQLite? (Facebook's OSQuery is SQLite-based, but it looks like the virtual table interface over there is way more complex)

mamcx 12y ago
And the opposite? Have the sqlite storage but no the sql parser?
- rakoo 12y ago
  What would be the use case ? Data already exists on the OS, no need to copy it on another storage and risk having stale values.
crxgames 12y ago
That would remove some of my hesitancy in letting this run as root for full process information. At least it limits what code could be exploited in the daemon process to some extent.
johnteslade 12y ago
I tried this with sqlite and the apsw python library. A quick example is at https://github.com/johnteslade/pysqliteosquery https://github.com/johnteslade/pysqliteosquery The interface is not quite as nice as Multicorn's but certainly nicer than the C/C++ SQLlite interface. But you could probably create a wrapper around apsw so it looked similar to Multicorn.
- Erwin 12y ago
  FYI, you can use ":memory:" as the sqlite data source, instead of having to conditionally drop the tables (unless you actually do want to store some persistent data on disk).
  - johnteslade 12y ago
    Good spot. The in memory table also runs 3 times quicker.
batbomb 12y ago
Don't forget about Presto

hippowithgas 12y ago

I can see we are going to get one of these DB / OS Adapters each day now. Looking forward to BerkDBOSQuery - because, well, the Berkeley DB needs some love too.

orf 12y ago

Yeah, heres a link to the awesome psutil library it uses: https://github.com/giampaolo/psutil https://github.com/giampaolo/psutil It could be made a bit more efficient by filtering the processes[1], but the code would not look nearly as lovely. 1. http://multicorn.org/implementing-an-fdw/#idoptimizations http://multicorn.org/implementing-an-fdw/#idoptimizations