4 ms·

"Pure functions" is usually a bad idea with large numeric data structures if they are non-lazy - because you'll end up copying the data. One of the common pure

"Pure functions" is usually a bad idea with large numeric data structures if they are non-lazy - because you'll end up copying the data. One of the common pure alternatives is to make them lazy - i.e., all those pure methods don't do anything but rather just collect the information on what to do with your data. But then you need to write an execution engine for arbitrary computation graph, which ain't easy.

nevi-me 7y ago
Yes, that's the approach that I'm taking with the library, but with a simple execution flow. One of the things that used to bite me a lot with Pandas was having to manually free memory from intermediate dataframes that I would create. I'm expecting Rust to work better in this case, because when executing a 3-step pipeline, step 2 consumes the output from step 1, and the dataframe is freed from memory. I'll write an experiment on this in the coming weeks/months, to see if my assumption would work.
nightcracker 7y ago
> "Pure functions" is usually a bad idea with large numeric data structures if they are non-lazy - because you'll end up copying the data. This isn't necessarily true due to Rust's strong ownership. Methods can take `self` (without reference) as an argument which means the method takes ownership of itself. There is no copy for 'double' in the below example, yet it is pure: #[derive(Clone)] struct A { v: Vec<i32>, } impl A { fn double(mut self) -> Self { for x in &mut self.v { *x *= 2; } self } } fn main() { let a = A { v: vec![2, 3, 4] }; let b = a.double(); dbg!(b.v[1]); } If you now tried to access `a` the compiler would error out, saying you're trying to access a moved-from variable. If you still wanted to keep the original `a` around you simply write `let b = a.clone().double()`.

nevi-me 7y ago

Yes, that's the approach that I'm taking with the library, but with a simple execution flow. One of the things that used to bite me a lot with Pandas was having to manually free memory from intermediate dataframes that I would create. I'm expecting Rust to work better in this case, because when executing a 3-step pipeline, step 2 consumes the output from step 1, and the dataframe is freed from memory. I'll write an experiment on this in the coming weeks/months, to see if my assumption would work.

nightcracker 7y ago

> "Pure functions" is usually a bad idea with large numeric data structures if they are non-lazy - because you'll end up copying the data. This isn't necessarily true due to Rust's strong ownership. Methods can take `self` (without reference) as an argument which means the method takes ownership of itself. There is no copy for 'double' in the below example, yet it is pure: #[derive(Clone)] struct A { v: Vec<i32>, } impl A { fn double(mut self) -> Self { for x in &mut self.v { *x *= 2; } self } } fn main() { let a = A { v: vec![2, 3, 4] }; let b = a.double(); dbg!(b.v[1]); } If you now tried to access `a` the compiler would error out, saying you're trying to access a moved-from variable. If you still wanted to keep the original `a` around you simply write `let b = a.clone().double()`.