3 ms·
Using “Git for data” to find and analyze test regressions
- zachmu 7y agoBlog author here. This post discusses how we use our git for data product, Dolt, to store test results, and how this lets us use SQL queries to analyze test regressions on our 6M+ test query dataset. Example: dolt_sql_performance> select nightly.query_string, nightly.error_message -> from nightly_dolt_results nightly join releases_dolt_results release -> on nightly.test_file = release.test_file and nightly.line_num = releases.line_num -> where nightly.result = 'not ok' and release.result='ok' limit 1; +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+ | query_string | error_message | +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+ | SELECT * FROM tab0 WHERE - + 63 IN ( col3, - col3 * col1 + + ( + + col4 ), 67 * col3 + - + CAST( - col1 AS DECIMAL ) + col0 ) | Unexpected error invalid type: string | +-------------------------------------------------------------------------------------------------------------------------------+---------------------------------------+
- timsehn 7y agoCEO of company that built this here. You need a versioned SQL database in your toolbox.
- detaro 7y agohm. pushing from the test runners to a central place would make sense for that, and could be quite interesting to keep track of some things, but it doesn't seem like there's a viable way of doing that internally (=not relying on some external service) yet?