Lazy Chunking: Generators on endless data — Python Exercise

Overview

Process data in batches without loading it all into memory — even data that never ends.

Generators that keep state between yields
itertools.count() and islice()
Yielding the leftover batch at the end

Short Course

A generator only produces the next value when asked, so it can work on inputs that are huge or even infinite. itertools has helpers to take just what you need.

from itertools import count, islice

def evens():
    for n in count():          # 0, 1, 2, ... forever
        if n % 2 == 0:
            yield n

list(islice(evens(), 4))       # [0, 2, 4, 6] — stops after 4

Task

Write a generator chunked(iterable, size) that yields lists of size items, plus a shorter last list if items are left over. It must work on endless iterators, so don't convert the input to a list.

Test Suites Pending

Stdout