3 ms·
> Chunked and windowed iteration, in iterutils I need this. Makes so many functional expressions so simple.
by candiodari 8y ago
> Chunked and windowed iteration, in iterutils
I need this. Makes so many functional expressions so simple.
- pmoriarty 8y agoCould you give some examples?
- btgeekboy 8y agoI wrote some similar code one time that generated SQL which was manually reviewed. It generated a massive WHERE IN clause. Oracle only accepts 1000 values. So I fed it a list of thousands of values, and it chunked them into rows of N objects, and into larger chunks of 1000. Having some code to do that automatically would be quite nice.
- Bootvis 8y agoWhy not insert the values in WHERE IN to a temporary helper table and join on that? It seems somewhat easier and might perform better.
- orf 8y agoOracle has a limit on the number of parameters in a query, not just in the `WHERE IN` clause. So you'd need to insert them into the helper table in batches of 1000.
- Bootvis 8y agoGood point, but isn’t that done automatically by whatever tool your using? And if the table your selecting from is big enough I would expect that this setup would be faster but it’s hard to say. It would also more easily guarantee that your results will be consistent.
- gugagore 8y agoTwo that I run into all the time: 1. pairwise iteration over a (cylic) list of points representing the ordered vertices of a polygon will give you a sequence of edges 2. np.diff, takes differences between consecutive pairs. Sometimes you have a different kind of "difference" operation, though. The pairwise iteration helps you express that.
- bbayles 8y agoSee also more-itertools for lots of other things in this vein.