nathanwallace

7 ms·

31.
by nathanwallace 3y ago
High priority tickets are naturally noisy and fixed quickly for impact reasons. Age reports create a good balance by gradually increasing the "noise" of all tickets regardless of priority.
32.
by nathanwallace 3y ago
Glad you are enjoying Steampipe and thanks for the kind words :-). If you have any suggestions for where you'd like to see the platform go please let us know!
33.
by nathanwallace 3y ago
100% - automation assists us to stay on top of it. But, human connection is the key! (In hindsight, our subhead text "automation with a human touch" got it the wrong way around.)
34.
How we manage 200 open-source repos(turbot.com)
169 points by nathanwallace 3y ago|32 comments
35.
select * from shell; Use SQL to exec commands across hosts(steampipe.io)
14 points by nathanwallace 3y ago|1 comments
36.
by nathanwallace 3y ago
Use SQL to run shell commands against any host - local or remote, Linux or Windows, through proxies & bastions. Aggregates results from multiple hosts. Steampipe - https://steampipe.io exec plugin - https://hub.st
37.
by nathanwallace 4y ago
In a similar vein, Steampipe [1] has a Google Sheets plugin [2]. It uses Postgres Foreign Data Wrappers instead of SQLite as the virtual table engine. (Disclaimer: I'm a lead on this open source project.) 1 - https://github.
38.
by nathanwallace 4y ago
Steampipe runs an embedded Postgres database [1]. One approach would be to connect that Postgres database with other databases using the usual (advanced) techniques like Foreign Data Wrappers [2]? With that connection in place, joins would
39.
by nathanwallace 4y ago
Correct - it's just Postgres, so all your favorite clients / tools work as usual. We have docs for a number of example integrations [1] that work with Steampipe CLI (or cloud). 1 - https://steampipe.io/docs/cl
40.
by nathanwallace 4y ago
Readers who like SQL may also enjoy Steampipe [1], an open source tool to live query 99+ services with SQL (e.g. AWS, GitHub, CSV, Kubernetes, etc). It uses Postgres Foreign Data Wrappers under the hood and supports joins etc across the ser
41.
by nathanwallace 4y ago
Graph DB == connect the dots. Relational DB == join the dots. :-) Steampipe [1] is an open source project [2] to live query all your cloud resources with SQL (via Postgres FDWs). It includes a dashboards as code layer written in HCL + SQL.
42.
by nathanwallace 4y ago
Excited to see Steampipe shared here - thanks zJayv! I'm a lead on the project, so sharing some quick info below and happy to answer any questions. Steampipe [1] is open source and uses Postgres foreign data wrappers under the hood [2]
43.
by nathanwallace 4y ago
Good idea - thanks, love the feedback :-)
44.
by nathanwallace 4y ago
Thanks atonse for the shout out! Steampipe [1] is an open source CLI to query your cloud resources (e.g. GitHub, AWS, Splunk, etc) with SQL. The GitHub plugin has 44 tables to query [2]. The "GitHub Sherlock" mod includes 34 autom
45.
by nathanwallace 4y ago
Sounds interesting - let us know how you go!
46.
by nathanwallace 4y ago
Please open GitHub issues (or PRs) for any tables you'd like to added! If there is a cost API then we'd definitely like to have a table to query it. In the meantime, you may find the "Thrifty" mods useful - they have man
47.
by nathanwallace 4y ago
hmmm ... I just tested in Incognito mode and it's working for me. Could you please try again? We are hiring for multiple roles!
48.
by nathanwallace 4y ago
100% - works great with Grafana, Metabase, Tableau, etc because it’s just Postgres :-) If you are into the idea of "dashboards as code", you may also enjoy our HCL+SQL approach in mods [1] - runs in your cli, open source, composab
49.
by nathanwallace 4y ago
Awesome - thanks for the shout out in golangweekly!
50.
by nathanwallace 4y ago
We have work in progress to support go-getter [1] allowing use of files from many different sources. There is a WIP branch [2] building that into the Steampipe Plugin SDK, so it can be immediately enabled for all our file based plugins (CSV
51.
by nathanwallace 4y ago
Postgres FDWs are extensions installed in Postgres. Multicorn provides an easy way to implement Postgres FDW extensions in Python, leading to many FDW implementations. Steampipe runs as a Postgres FDW extension with a pluggable interface to
52.
by nathanwallace 4y ago
Steampipe computes the SQL part in a single Postgres instance (not distributed) - we've not found this to be a significant limit (so far). A difference in this case is that the query is really combining and organizing results that may
53.
by nathanwallace 4y ago
I feel your pain - but Steampipe won't solve this (yet). We have AWS Thrifty [1] to scan your env for cost savings and other tables to query your spend [2], but haven't tackled the 1.8GB of pricing JSON data into a table yet. 1 -
54.
by nathanwallace 4y ago
The premise of virtual tables (Postgres FDW) is to not store the data but instead query it from the original source. So, the primary performance challenge is API throttling. Steampipe optimizes these API calls with smart caching, massive pa
55.
by nathanwallace 4y ago
While it's extensible, the primary use case for Steampipe so far is to query DevOps data (e.g. AWS, GitHub, Kubernetes, Terraform files, etc) - so often they are working within the Steampipe DB itself and combining with CSV data etc. B
56.
by nathanwallace 4y ago
Sorry you didn't have a smooth start there :-(. Not sure how the behavior you are describing is happening. We'd be happy to help get things going if you want to open an issue [1] or discuss on Slack [2]. 1 - https://git
57.
by nathanwallace 4y ago
osquery is awesome. It was definitely part of our inspiration!
58.
by nathanwallace 4y ago
Python and boto3 are very powerful in many cases for sure - glad they work well for you. No doubt that multi-region for 1k+ accounts is a large surface area, so we'd love to learn from your experience! Performance is an ongoing focus a
59.
by nathanwallace 4y ago
Yes - joins work across tables and schemas! This (toy) example connects IAM user records with Slack user data: select u.name, s.id, s.display_name from aws_iam_user as u, slack_user as s where u.name = s.email
60.
by nathanwallace 4y ago
Because it's just Postgres, connecting Jupyter notebooks to Steampipe via SQL works really well :-)

More ›