3 ms·
Assuming the data isn't changed, thanks to CoW forking wouldn't cause any extra memory usage. If only a subset of data is changed, same thing - only the changed
by roblabla 7y ago
Assuming the data isn't changed, thanks to CoW forking wouldn't cause any extra memory usage. If only a subset of data is changed, same thing - only the changed cells will take extra space. The problem only occurs when the whole variable changes - in which case yeah, you're SOL. I wonder what the usage patterns are for such datasets?
- ivalm 7y agoPersonal experience: when first looking at the data I often do lots of map /reduce style operations which might transform large portions of the dataframe. Question, if you use CoW then presumably your variable blocks are no longer contiguous, wouldn't this really slow down vector operations?
- roblabla 7y ago> Question, if you use CoW then presumably your variable blocks are no longer contiguous, wouldn't this really slow down vector operations? I don't think so. Vector operations require the data to be aligned to whatever the vector size is, no? E.G. 16-byte vector ops require the data to be aligned to 16-byte, etc... At least that's my understanding.