4 ms·
I'm just happy for itertools.batched for chunking iterables: https://docs.python.org/3.12/library/itertools.html#itertools.batched https://docs.python.org/3.12/
by georgehotelling 3y ago
I'm just happy for itertools.batched for chunking iterables: https://docs.python.org/3.12/library/itertools.html#itertools.batched https://docs.python.org/3.12/library/itertools.html#itertool...
- kastden 3y agoThis is the greatest addition since f-strings!
- philshem 3y agoYes. I’ve written explicit code that needed this 100s of times.
- intalentive 3y agoYeah I've memorized it by now: for i in range(len(lst) // batch_size + 1): batch = lst[i * batch_size : (i + 1) * batch_size]
- hddqsb 3y agoYou have a minor bug -- when len(lst) is a multiple of batch_size, this will have an extra iteration at the end with an empty batch. The fixed version is `range((len(lst) + batch_size - 1) // batch_size)`, which emulates `ceil(len(lst) / batch_size)`. Yet more proof that this should be part of stdlib :) Personally I think I'd actually write it like this: for i in range(0, len(lst), batch_size): batch = lst[i:i+batch_size] The docs give another pretty nice implementation using iter() and islice() in a loop (but it uses the walrus operator `:=` so it requires Python 3.8+ as written).
- rossant 3y agoBlatant reason why a native solution was long overdue.
- deleted 3y ago[deleted]
- kzrdude 3y agoGreat call-out! (Mistakes elided..)
- LegionMammal978 3y agoWhat do you mean by "empty sequence in"? The function doesn't raise if the input iterable is empty: it only raises if the chunk size n is 0. While that does have a natural interpretation of returning an infinite sequence of empty tuples, such a behavior would be qualitatively different than for other chunk sizes. The caller would never be able to retrieve any elements from the input iterable, and the output would be infinite even if the input is finite. In that light, it makes some sense (IMO) to avoid letting applications hit such an edge case unintentionally.
- kzrdude 3y agoI skimmed that too quickly and was mistaken. Thanks.
- deleted 3y ago[deleted]
- zem 3y agoyeah! that's been in the ruby stdlib practically from day one, no idea why python was so resistant to it.
- ehsankia 3y ago99% of my more_itertools imports are exactly for this. there's 1-2 other stuff from more_itertools that I think should make it to itertools. I'd actually like to see statistics from huge monorepos/opensource about usage stats of various more_itertools functions.
- miiiiiike 3y agoSame but for the ‘batch’, ‘ibatch’, and ‘abatch’ functions I started writing back in 2008.
- abyesilyurt 3y agoCheckout more-itertools for more variants: https://pypi.org/project/more-itertools/ https://pypi.org/project/more-itertools/