4 ms·
To add somewhat of a counterpoint to the other response, I've tried the Steampipe CSV plugin and got 50x slower performance vs OctoSQL[0], which is itself 5x sl
by cube2222 4y ago
To add somewhat of a counterpoint to the other response, I've tried the Steampipe CSV plugin and got 50x slower performance vs OctoSQL[0], which is itself 5x slower than something like DataFusion[1]. The CSV plugin doesn't contact any external API's so it should be a good benchmark of the plugin architecture, though it might just not be optimized yet.
That said, I don't imagine this ever being a bottleneck for the main use case of Steampipe - in that case I think the APIs themselves will always be the limiting part. But it does - potentially - speak to what you can expect if you'd like to extend your usage of Steampipe to more than just DevOps data.
I've used the benchmark available in the OctoSQL README.
[0]: https://github.com/cube2222/octosql https://github.com/cube2222/octosql
[1]: https://github.com/apache/arrow-datafusion https://github.com/apache/arrow-datafusion
Disclaimer: author of OctoSQL
- davesque 4y agoYep, I'm aware of DataFusion and I'd expect nothing less than top notch performance from it. Interesting comparison.