3 ms·
Hey congarts on the launch ! One question comes to mind is about the T in ETL, ingestion tools like Fivetran and Stitch allows you to do light transformations o
by mardo5 4y ago
Hey congarts on the launch !
One question comes to mind is about the T in ETL, ingestion tools like Fivetran and Stitch allows you to do light transformations on the incoming data. This process is usually done by the ingestion team that hands the data to different value teams.
- ctc24 4y agoThanks! In the case of traditional ETL tools, a lot of that transformation is usually schema cleaning, which is necessary because the data is extracted from an API and so needs a little love before it’s in analysis-ready shape (renaming columns, joining a couple tables, and so on). When the vendor is in charge of defining the data model, they know what format is most helpful to surface the data in and so can transform it to that format before the load (TEL?!). In this model, it’s also still possible for the data team on the recipient side to further transform data before handing it to value teams. They can run dbt or whichever transform tool they use once it lands in their warehouse (so more like ELT, which is broadly what the industry is moving to).