5 ms·
Doesn't that mean you have to copy all your data into duckdb? I'd imagine with even a modest data warehouse, loading the entire thing to duckdb would be unfeasi
by mritchie712 4y ago
Doesn't that mean you have to copy all your data into duckdb? I'd imagine with even a modest data warehouse, loading the entire thing to duckdb would be unfeasible.
- RobinL 4y agoThe tests are being run against small test datasets of 10s of rows rather than the real data that test whether the behaviour of transforms/joins etc. is as expected. You're right, this approach wouldn't be sensible if the tests have to use large production tables
- mritchie712 4y agoHow do you get the small datasets? Selecting a random sample from the source?
- wnolens 4y agoThe test itself would provide a test dataset. That way you can cook up the interesting cases and ensure your queries work on them.