Py
Python/Iterators & Generators/Iterators & the Iterator Protocol

Iterators & the Iterator Protocol

An iterator produces values one at a time via __next__, raising StopIteration when it runs out — the single protocol every for loop, comprehension and unpacking in Python relies on.

Advanced35mInterview Production

Learning objectives

  • › Distinguish an iterable (has __iter__) from an iterator (has __next__)
  • › Implement a custom iterator with __iter__/__next__ and end it with StopIteration
  • › Explain how a for loop desugars to iter(), next() and a StopIteration catch
  • › Recognise that iterators are single-pass and avoid re-iterating an exhausted one

Prerequisites

Functionsfor loopsClasses & dunder methods

Version notes

  • · In Python 3 the method is `__next__`; Python 2 spelled it `next`. The built-in `next(it, default)` wraps it and can return a default instead of raising StopIteration.
  • · The two-argument form `iter(callable, sentinel)` has existed since Python 2.2 (PEP 234), which introduced the iterator protocol.
  • · Per PEP 479 (default in 3.7), a StopIteration that escapes a generator becomes a RuntimeError — relevant once you build iterators with `yield`.

Simple explanation

What it is. Two roles sit behind every loop. An *iterable* is anything you can loop over — it implements __iter__, which returns a fresh *iterator*. The iterator does the real work: its __next__ hands back the next value each time it is called and raises StopIteration once there is nothing left. `iter(obj)` gets the iterator; `next(it)` pulls the next value.

Why it exists. Lists, tuples, strings, dicts, sets, files, ranges and database cursors are wildly different inside, yet one for loop works on all of them. That is because they all agree on the same tiny contract — the iterator protocol. Code that consumes an iterable never has to know what it is looping over.

The problem it solves. It lets you loop uniformly over anything and, crucially, produce values lazily. An iterator can yield items from a 100 GB file, a network socket or an infinite sequence one at a time, without ever holding them all in memory.

How it works. `for x in obj:` desugars to roughly: get an iterator with `it = iter(obj)`, then loop calling `x = next(it)` and running the body, until `next` raises StopIteration, which the loop catches to stop. A custom iterator just implements __iter__ (returning self) and __next__ (returning the next value or raising StopIteration).

Analogy: An iterable is a book; an iterator is a bookmark. The book has no position — you can place several bookmarks in it, each remembering its own page. A bookmark moves forward one page at a time and, once it reaches the end, it is spent; to read again you get a new bookmark (a new iterator), not a new book.

Common misconceptions

  • "An iterable and an iterator are the same thing." — A list is iterable but is *not* its own iterator; `iter(mylist)` returns a separate list_iterator each time. An iterator, by contrast, returns *itself* from __iter__.
  • "You can loop over an iterator as many times as you want." — An iterator is single-pass. Once __next__ has raised StopIteration it stays exhausted, and a second loop sees nothing.
  • "StopIteration is an error you must catch." — It is the normal, expected end-of-data signal. The for loop catches it for you; you only meet it when you call next() by hand.
  • "next() returns None when there's nothing left." — It raises StopIteration, unless you pass a default: `next(it, default)`.

Common alternatives

  • Generators (`yield`) — the concise way to build an iterator without writing a class
  • itertools building blocks — count(), chain(), islice() compose iterators lazily in C
  • Building a full list up front when the data is small and you need indexing or repeated passes
  • iter(callable, sentinel) to turn a plain function into an iterator that stops at a marker value

Interactive visualization

Step 1 / 6
1nums = [10, 20]
2it = iter(nums)     # __iter__ -> a fresh iterator
3print(next(it))     # __next__ -> 10
4print(next(it))     # __next__ -> 20
5print(next(it))     # exhausted -> StopIteration

Heap / objects

listlist

[10, 20]

A list is iterable, not an iterator

nums is iterable: it can produce an iterator on demand, but holds no cursor itself.

Code examples

BeginnerIterable vs iterator
python
Loading editor…

Note: The list is the iterable; iter() produces a separate list_iterator that tracks position and, once drained, raises StopIteration.

Behind the scenes

  • ▸`iter(obj)` calls `obj.__iter__()` and `next(it)` calls `it.__next__()` — the built-ins are thin wrappers over the dunder methods.
  • ▸An iterator must return `self` from __iter__. That is exactly why you can pass an iterator straight to a for loop, and also why iterating the same iterator twice yields nothing the second time.
  • ▸StopIteration is ordinary control flow, not an error: CPython's `FOR_ITER` bytecode calls __next__ and, when it raises StopIteration, jumps past the loop body.
  • ▸`iter(callable, sentinel)` builds an iterator that calls `callable()` repeatedly until it returns the sentinel — e.g. `iter(lambda: f.read(1024), b'')` reads a file in chunks until EOF.
  • ▸itertools ships lazy, C-level iterators: `count(10)` counts forever, `chain(a, b)` concatenates without copying, and `islice(it, 5)` takes a slice of any iterator without materialising it.

Interview insights

Key definitions

  • · Iterable: an object implementing __iter__, which returns an iterator (lists, dicts, strings, files, ranges).
  • · Iterator: an object implementing __next__ (returns the next value or raises StopIteration) and __iter__ returning itself.
  • · StopIteration: the exception an iterator raises to signal there are no more values; for loops catch it to stop.
What is the difference between an iterable and an iterator?

Concise answer: An iterable can produce an iterator via __iter__ (a list, string, dict). An iterator is the thing that actually yields values one by one via __next__ and tracks its position — and it is single-use.

Detailed answer: An iterable implements __iter__ and returns a *fresh* iterator each time it is called — that is why you can loop over a list repeatedly. An iterator implements __next__ (returning the next value or raising StopIteration) and returns itself from __iter__, so it carries the current position and is exhausted after one pass. Every iterator is iterable, but not every iterable is an iterator.

Common mistake: Assuming a list is its own iterator that can be 'reset' — it isn't; iter(list) gives a new iterator each call.

Follow-ups:

  • · What does iter() return for a list?
  • · How do you make a class iterable?
How does a `for` loop work under the hood?

Concise answer: for calls iter() on the iterable to get an iterator, then repeatedly calls next() on it, running the body each time, until next() raises StopIteration.

Detailed answer: `for x in obj:` is sugar for: it = iter(obj); while True: try x = next(it) except StopIteration: break; <body>. At the bytecode level CPython uses GET_ITER once and FOR_ITER each iteration; FOR_ITER calls __next__ and, on StopIteration, jumps past the loop. This is why anything implementing the iterator protocol — files, generators, custom classes — works with for.

Common mistake: Thinking for loops use indexing/len() — they use the iterator protocol, so they work on things with no length (files, generators).

Follow-ups:

  • · What bytecode does a for loop compile to?
  • · Does a for loop call __iter__ once or every iteration?
Why can looping over an iterator twice give you nothing the second time, and how do you fix it?

Concise answer: Iterators are single-pass: once exhausted they stay exhausted. Iterate the underlying iterable again (which hands out a fresh iterator), or materialise the values into a list if you need repeated passes.

Detailed answer: An iterator carries its position and returns itself from __iter__, so a second for loop resumes at the already-exhausted end and immediately gets StopIteration. Functions like sum(), max() and list() each fully consume it. The fix is to keep the *iterable* around and call iter() again (a fresh iterator per pass), design your class so __iter__ returns a new iterator, or store the results in a list when the data is small enough to reuse.

Common mistake: Storing an iterator (a map/zip/generator object) and expecting to loop over it more than once.

Follow-ups:

  • · Is a generator single-use too?
  • · How would you make a re-iterable class?

Knowledge check

What's the output?Question 1 of 3

What does this print?

python
it = iter([1, 2, 3])
print(sum(it))
print(sum(it))
Multiple choiceQuestion 2 of 3

What must an iterator's __iter__ method return?

Multiple choiceQuestion 3 of 3

How does a for loop know when to stop?

Score: 0/3