3 ms·
I am not sure about the design choices of the pandas library or of R, but generally, many programming languages limit the number of processes they spawn during
by pama 3y ago
I am not sure about the design choices of the pandas library or of R, but generally, many programming languages limit the number of processes they spawn during evaluation. Moreover, designing a pipeline that produces a complete output before it is processed by the next function tends to be simpler in these languages. The shell environment is explicit about process creation and offers interactivity, leading to an expectation that a pipeline of processes might not complete before being interrupted. As a result, true pipelining is highly effective in such contexts. Some languages, like Haskell, or certain libraries, employ lazy evaluation to achieve similar outcomes. For instance, by using the polars library instead of pandas, one can achieve true row-by-row pipelining for specific operations in Python.