3 ms·
99% of our pipelineBuilders are purely based on the available building blocks [that cover most of PySpark functions: https://www.palantir.com/docs/foundry/pipel
by lolive 1y ago
99% of our pipelineBuilders are purely based on the available building blocks [that cover most of PySpark functions: https://www.palantir.com/docs/foundry/pipeline-builder/functions-index/ https://www.palantir.com/docs/foundry/pipeline-builder/funct...]. Most notably we do join/filters/explode/group, Dozens of them, realigning data from different parts of the company.
Having most of those PySpark functions exposed visually as building blocks with debug points available at each step simply outperforms a codeWorkbook in term of maintainability and readability.