playwright-roll

---
name: playwright-roll
description: Roll Playwright Python to a new driver version. Walks the upstream `docs/src/api/` commit range, ports each public-API change, suppresses the rest in `expected_api_mismatch.txt`, regenerates the typed surface, and adds tests.
license: Apache-2.0
---

# Rolling Playwright Python

The goal of a roll is to move the driver pin in `DRIVER_VERSION` to a new release, port every public API change introduced upstream during that interval, and suppress the rest, so that `./scripts/update_api.sh` runs clean and the test suite still passes.

The previous human-facing summary lives in `../../../ROLLING.md`. This skill is the operational playbook — read it end to end before starting.

## Mental model

The Python port is hand-written code in `playwright/_impl/`, plus a generator (`scripts/generate_*.py`, `scripts/documentation_provider.py`) that:

1. introspects the Python `_impl` classes via `inspect`,
2. emits typed wrapper classes into `playwright/{async,sync}_api/_generated.py`, and
3. diffs the introspected surface against the Playwright `api.json` (generated from the upstream docs — see step 2).

Anything in `api.json` that is missing or differently typed in `_impl/` causes generation to fail. Three resolutions:

- **PORT** — the new API is intended for Python (no `langs.only` filter, or `langs.only` includes `"python"`). Implement it in `_impl/`.
- **MISMATCH** — the API genuinely exists for Python but is shaped differently (a callback signature uses unions, a kwarg uses a legacy name, etc.) and there's a justified reason to keep the divergence. Add a precise line to `scripts/expected_api_mismatch.txt` with a comment explaining *why*.
- **N/A** — the commit only touches docs, has `* langs: js` (or any other filter that excludes Python), is server-side, Electron-only, or was reverted later in the same release. No action.

The upstream documentation source of truth is `docs/src/api/*.md` in the playwright repo. Every `## method:` / `## property:` / `## event:` / `### option:` / `### param:` block has an optional `* langs: js` (or `js, python`, etc.) filter. The Python doclint resolves these into `langs` fields on each member of `api.json`. **An empty `langs: {}` means "all languages including Python" — *implement it*, don't suppress it.**

> **The mistake the 1.59 roll made twice over:** classifying things as "internal tooling, N/A for Python" based on the *name* of the API (Screencast, Debugger, pickLocator, clearConsoleMessages, artifactsDir, …). Almost all of those had empty `langs: {}` in `api.json` and were real Python APIs. Sounding tooling-y is not a `langs` filter. **The `langs` field on the member in `api.json` is the only authoritative signal.** When in doubt, dump it (see "Verifying classifications" below).

## Process

### 1. Set up the env

`CONTRIBUTING.md` covers this. Notes from past rolls:

- The repo requires Python 3.10+. If `python3.10` isn't available, use `python3` (3.12 is fine).
- If `python3-venv` is missing system-wide, use `uv venv env` instead, then `uv pip install --python env/bin/python --upgrade pip`. Don't try to `apt install` — sudo is denied in the harness.
- Always activate the venv before any `pip`, `pytest`, `mypy`, or `pre-commit` invocation.

### 2. Bump the driver pin, download it, and generate api.json

You need a nearby `microsoft/playwright` checkout for the docs walk and for
`api.json` generation. Point `PW_SRC_DIR` at it and check out the new tag there:

```sh
export PW_SRC_DIR=../playwright
git -C "$PW_SRC_DIR" fetch --tags origin
git -C "$PW_SRC_DIR" checkout v<new>      # e.g. v1.62.0
```

Then bump the pins and assemble the driver:

```sh
# Edit DRIVER_VERSION (repo root): the playwright-core npm version for the new
# release, no "v" prefix, e.g.  1.62.0
python scripts/update_node_version.py     # refresh NODE_VERSION to the current LTS

source env/bin/activate
python -m build --wheel       # downloads playwright-core @ DRIVER_VERSION + Node.js, assembles the driver
playwright install chromium   # NOT --with-deps; sudo is denied

# api.json isn't in the bundle, and the walk below inspects `langs` from it.
# Generate a copy to inspect (update_api.sh generates its own temp copy in step 6):
API_JSON_MODE=1 node "$PW_SRC_DIR/utils/doclint/generateApiJson.js" > /tmp/api.json
```

The wheel build just downloads the `playwright-core` npm package at
`DRIVER_VERSION` and the matching Node.js binary (no source build, no Node/npm/git
toolchain), and unpacks the driver under `playwright/driver/`. `api.json` is the
one piece not shipped in the bundle — it's generated from `$PW_SRC_DIR` on demand
(here to `/tmp/api.json` for the walk, and into a temp file passed via
`PW_API_JSON` by `./scripts/update_api.sh` during codegen).

### 3. Identify the commit range

Use the nearby `microsoft/playwright` checkout at `$PW_SRC_DIR` (from step 2).
Bring it up to date and ensure release branches/tags are present before walking
the range:

```sh
git -C "$PW_SRC_DIR" fetch --tags
git -C "$PW_SRC_DIR" fetch origin 'release-*:release-*'
```

There is sometimes no `vX.Y.0` tag for the latest release (the bots cut release branches first and tag later). Anchor on commits, not tags.

The diff range is "every commit on the new release branch since the previous release was cut". Anchor commits:

- **Previous release end**: the `chore: bump version to vX.Y.0-next` commit on `main`. That commit is the first commit *after* the previous release (X.Y-1) was cut. Use its parent (`<sha>~1`) as the lower bound.
  ```sh
  git -C "$PW_SRC_DIR" log --all --grep="bump version to v" --oneline | head
  ```
- **New release end**: the tip of `release-<new>` (or the matching tag if it exists).

Save the commit list, oldest first, scoped to `docs/src/api/`:

```sh
git -C "$PW_SRC_DIR" log <prev-anchor>~1..release-<new> --oneline --reverse -- docs/src/api > /tmp/roll-<new>-commits.md
```

A normal roll yields 50–100 commits. If you see 0 or thousands, the range is wrong.

Format the file as a markdown checklist and add the standard preamble (status legend, where to look up `api.json` etc.) — see the file from the 1.58→1.59 roll for the template.

### 4. Walk the commit list

For each commit, in chronological order:

```sh
git -C "$PW_SRC_DIR" show <sha> -- docs/src/api/
```

Look for:
- `## (async )?method:` / `## property:` / `## event:` additions or removals;
- `* langs: ...` lines on those blocks;
- `### param:` / `### option:` additions or removals;
- new `class-X.md` files (whole new classes — usually `langs: js`);
- type changes in `- returns:` lines.

Classify and act.

#### Verifying classifications (do this before suppressing anything)

Before tagging anything as MISMATCH or N/A based on appearance, dump the actual `langs` from `api.json`:

```python
import json
data = json.load(open("/tmp/api.json"))
classes = {c["name"]: c for c in data}
for cls_name in ["Page", "BrowserContext", "Screencast", "Debugger"]:
    cls = classes.get(cls_name)
    if not cls:
        continue
    print(f"\n{cls_name}: cls_langs={cls.get('langs', {})}")
    for m in cls["members"]:
        print(f"  {m['name']} kind={m.get('kind')} langs={m.get('langs', {})}")
```

For options/params nested inside an `Object`-typed arg, walk one level deeper:

```python
for a in member.get("args", []):
    if a["name"] == "options":
        for prop in a.get("type", {}).get("properties", []):
            print(prop["name"], prop.get("langs", {}))
```

A few rules of thumb that catch most "actually a PORT" cases:
- If the *containing class* has empty `langs: {}` and the *member* has empty `langs: {}`, it's for Python — implement it.
- If the member is empty but a single *option* has `langs: js`, the method is for Python and you only skip that option (e.g. `Screencast.start.size` is `langs: js` while `Screencast.start` itself isn't).
- If you're about to add three or more `Method not implemented:` entries for the same class, stop — you almost certainly need to implement the class.

#### PORT

Implement the change in `playwright/_impl/<module>.py`. Use the upstream JS implementation as a reference: `$PW_SRC_DIR/packages/playwright-core/src/client/<module>.ts`. Translate idioms:

| Upstream JS | Python |
|---|---|
| `async foo(): Promise<X>` | `async def foo(self) -> X:` |
| `foo(): X` (sync getter, no args, no body) | `@property def foo(self) -> X:` (the doc generator treats argument-less sync getters as properties — see `documentation_provider.py:133`. If you make it a method instead, you'll get a "Method vs property mismatch" error.) |
| `await this._channel.foo({ a, b })` | `await self._channel.send("foo", None, locals_to_params(locals()))` |
| `(await this._channel.foo()).value` | `await self._channel.send("foo", None)` (Python's `send()` auto-unwraps single-key responses; only call `send_return_as_dict` when the protocol returns multiple keys.) |
| `(await this._channel.foo()).artifact` (multi-key, may be empty) | `result = await self._channel.send_return_as_dict("foo", None); (result or {}).get("artifact")` — `send_return_as_dict` returns **`None`** (not `{}`) when the protocol response carries no fields. |
| `try { ... } catch (e) { if (isTargetClosedError(e)) return; throw e; }` | `try: ...; except Exception as e: if is_target_closed_error(e): return; raise` (import from `playwright._impl._errors`) |
| Inline `[Object]` return like `{endpoint: string}` | A `TypedDict` in `playwright/_impl/_api_structures.py` — *not* `Dict[str, str]`. The doc generator serializes TypedDicts as `{field: type, ...}` via `get_type_hints` and that matches the inline-object form exactly. See `RemoteAddr`, `BrowserBindResult`, `DebuggerPausedDetails`. |
| `binary` event/return field | The Python channel layer hands you a base64 string. Decode with `base64.b64decode(value)` before exposing as `bytes`. See `Screencast._dispatch_frame`. |

When implementing a new ChannelOwner subclass (one constructed by the protocol with `(parent, type, guid, initializer)`):
1. Register it in `playwright/_impl/_object_factory.py:create_remote_object` — otherwise the guid resolves to `DummyObject` and downstream code breaks mysteriously.
2. Import it and add it to `generated_types` in `scripts/generate_api.py`, plus add a `XxxImpl` import in the `header` string.

When implementing a non-ChannelOwner wrapper class (a plain class that holds a Page/Context reference, like `Screencast`, `Clock`):
- Set `self._loop = parent._loop` and `self._dispatcher_fiber = parent._dispatcher_fiber` in `__init__`. The generated `AsyncBase`/`SyncBase` wrappers read these; missing them gives `AttributeError: 'X' object has no attribute '_loop'` at first use.

When adding a new TypedDict in `_api_structures.py`:
- Add it to the `from playwright._impl._api_structures import …` line in `scripts/generate_api.py` so the generator can resolve it as a forward reference in type hints.
- Re-export it from both `playwright/async_api/__init__.py` and `playwright/sync_api/__init__.py`: assignment line plus an entry in `__all__`. Same pattern as `ViewportSize`, `RemoteAddr`.

If the new API was previously suppressed in `expected_api_mismatch.txt`, **remove that line** when implementing it.

If a doc rename involves a *positional* parameter (no default, before any `*`), users almost certainly call it positionally — you can rename freely. The 1.59 `BrowserType.connect.wsEndpoint` → `endpoint` is the canonical example. Don't suppress this kind of rename; just rename in `_impl/`. **Important corollary:** when docs rename a param, the wire-protocol field usually also changed in `packages/protocol/src/protocol.yml` and the server-side dispatcher in `packages/playwright-core/src/server/dispatchers/*Dispatcher.ts`. If so, you must also update the channel-send dict key (e.g. `{"wsEndpoint": …}` → `{"endpoint": …}`). A "Parameter not documented" suppression for a renamed param is a code smell hiding a wire-protocol bug.

#### MISMATCH

A MISMATCH is a *justified, durable* divergence between the docs and the Python surface. Use it sparingly — most apparent mismatches turn out to be PORTs you skipped. Legitimate examples in the current `expected_api_mismatch.txt`:

- Hidden internal kwargs (`Browser.new_context(default_browser_type=)`).
- Callback signatures where Python explicitly unions one-arg and two-arg variants but the docs document only the canonical form (`Page.route(handler=)`, `WebSocketRoute.on_close(handler=)`).

Add a precise line to `scripts/expected_api_mismatch.txt` with a `# comment` group header explaining *why* the divergence is intentional. The exact wording comes from the generator's error message. Examples:

```
# One vs two arguments in the callback, Python explicitly unions.
Parameter type mismatch in Page.route(handler=): documented as Callable[[Route, Request], Union[Any, Any]], code has Union[Callable[[Route, Request], Any], Callable[[Route], Any]]
```

The generator removes lines from `expected_api_mismatch.txt` that no longer match an error. If you see "No longer there: …" in the script's stderr, delete that line.

**Do not suppress** these — they're PORTs in disguise:

- "Internal tooling" classes/methods whose `langs` field is empty (`Screencast.*`, `Debugger.*`, `Page.pick_locator`, `BrowserContext.debugger`, `Browser.bind/unbind`, `Page.{clear,console}_*`). The 1.59 roll suppressed all of these initially, then had to walk every one back. Verify `langs` first.
- A renamed positional parameter (`Parameter not documented: X.y(old_name=)` + `Parameter not implemented: X.y(new_name=)`). Just rename in `_impl/` and update the channel-send dict key.
- A `Parameter type mismatch in X.y(return=): documented as {field: T}, code has Dict[str, T]`. Use a TypedDict.

#### N/A

Common N/A flavors:
- Whole new class with `langs: js` (Disposable, Inspector/Screencast, Debugger, Overlay).
- Members with `langs: js` (most "tooling" / MCP / agentic features).
- Doc-only edits (typo fixes, "Improve `not` property sections", etc.).
- Reverts that cancel an earlier add in the same range (always check the rest of the range before porting something that gets reverted).
- Java/C# `langs:` blocks.
- Electron-only changes (`docs/src/api/class-electron.md`).

Tick the box in `/tmp/roll-<new>-commits.md` with one line: `[x] <sha> <subject> — <classification>: <one-liner>`.

### 5. Regenerate

```sh
PW_SRC_DIR=../playwright ./scripts/update_api.sh   # PW_SRC_DIR already exported in step 2
```

The script does, in order:
1. generates `api.json` from `$PW_SRC_DIR` into a temp file and exports `PW_API_JSON`,
2. `git checkout HEAD -- playwright/{async,sync}_api/_generated.py` (resets to last committed),
3. runs `scripts/generate_{sync,async}_api.py` (they read `api.json` via `PW_API_JSON`), dumping to `.x` then renaming into place,
4. invokes `pre-commit run --files` on the generated files.

**CI no longer verifies that `_generated.py` is in sync** (the Lint job dropped the
"Verify generated API is up to date" step so it needn't check out upstream). So
regenerating here and committing the result is on you — don't skip it.

Failure modes and fixes:

| Symptom | Cause | Fix |
|---|---|---|
| `Method not implemented: X.y` | `api.json` documents `X.y`, no Python impl exists. | PORT it, or add a MISMATCH line. |
| `Parameter not implemented: X.y(z=)` | New parameter on existing method. | Add the kwarg in `_impl/`, or MISMATCH. |
| `Method vs property mismatch: X.y` | You implemented as a method but the doc treats it as a property (sync, no args, has return type). | Add `@property` in `_impl/`. |
| `Method not documented: X.y` | Python has it but `api.json` doesn't. | The upstream removed the API; remove from `_impl/` and from `_generated.py` callers. |
| `Parameter type mismatch in X.y(z=): documented as ..., code has ...` | Type signature doesn't line up. | Match the type in `_impl/`, or MISMATCH it for known historical divergences. |
| `pyright … reportInconsistentOverload` (single-event class) | A class gained its first event in `api.json`, so the generator emits one `Literal[…]` overload + impl, which pyright wants ≥2 of. | The generator already handles this — `documentation_provider.print_events` emits a second `@typing.overload` with `event: str` plus a generic impl. If you regress this, see how 1.59 handled `CDPSession` getting a `close` event. |
| `pre-commit` keeps reformatting `_generated.py` on each run | First run after regen always reformats once; rerun until idle. | `pre-commit run --all-files` to settle. |
| `Parameter not documented: X.y(z=)` | Python has a kwarg the docs don't mention (e.g. legacy name from a doc rename). | If the param is positional with no default, just rename it in `_impl/`. Check `protocol.yml` and the server dispatcher — if the wire field renamed too, also update the channel-send dict key. Only MISMATCH for genuinely hidden internal kwargs (`default_browser_type`). |
| `KeyError: 'templates'` deep in `inner_serialize_doc_type` | A `Promise<X>\|X` union upstream collapsed to a bare `Promise` with no templates in `api.json`. | `documentation_provider.inner_serialize_doc_type` should treat that as `Any` (`if "templates" not in type: return "Any"`). |
| `"void" is not defined (reportUndefinedVariable)` in generated event handlers | `api.json` has a `void`-typed event payload that the serializer left as the literal string `"void"`. | `documentation_provider.inner_serialize_doc_type` should map `"void"` to `"None"` alongside `"null"`. |
| `AttributeError: 'X' object has no attribute '_loop'` (or `_dispatcher_fiber`) at first use of a new wrapper class | The non-ChannelOwner wrapper isn't initializing the fields the generated `AsyncBase`/`SyncBase` reads. | In the wrapper's `__init__`, set `self._loop = parent._loop` and `self._dispatcher_fiber = parent._dispatcher_fiber`. |
| `'NoneType' object has no attribute 'get'` after `send_return_as_dict` | Method's protocol response carries no fields and `send_return_as_dict` returned `None`. | `(result or {}).get(...)`. |
| Frame/buffer payload arrives as a `str` instead of `bytes` | Protocol `binary` fields cross the wire as base64. | `base64.b64decode(value)` in the impl before exposing. |

After the script settles, run `pre-commit run --all-files` once more to confirm everything is idle.

### 6. Add tests

For each PORT, add one async test and a matching sync test. Conventions:

- Tests go in the existing topic file (`test_page_network_request.py`, `test_browsercontext.py`, `test_dialog.py`, …) — don't create new files unless there's no obvious home.
- Use `from playwright.async_api import …`, **not** `from playwright._impl._page import Page` (the impl class doesn't have the public wrappers like `expect_console_message`).
- For event-info objects, `await message_info.value` (it's an `async` property).
- Don't write tests that hang the page (e.g. `page.evaluate(... fetch slow ...)` followed by `page.close()` from a fixture) — the request task gets a `TargetClosedError`. Use `page.on("event", handler)` to capture state at event time instead.
- `playwright install chromium` (no `--with-deps`) is sufficient for the test suite under sandbox.

### 7. Update existing high-touch artifacts

- `DRIVER_VERSION` and `NODE_VERSION`: already done in step 2.
- `README.md`: gets the chromium/firefox/webkit version table updated automatically by `scripts/update_versions.py` (called from `update_api.sh`). Don't edit by hand.
- The "Backport changes" tracking issue on GitHub (filed by `microsoft-playwright-automation`) is the *intent* tracker, but it's frequently out of sync with what's actually been ported. Treat it as a starting point, not the source of truth — the `docs/src/api/` commit walk is authoritative.

### 8. Final verification

```sh
pre-commit run --all-files
mypy playwright                 # 2 pre-existing errors in _json_pipe.py and _artifact.py are unrelated
pytest --browser chromium tests/async/<files-you-touched> tests/sync/<files-you-touched>
```

Then a smoke regression on a few neighboring suites (`tests/async/test_browser*.py`, `test_cdp_session.py`, `test_tracing.py`, `test_dialog.py`, `test_page_*.py`) to make sure nothing inherent to the port shifted.

## Reference: `expected_api_mismatch.txt` line forms

Exact strings the generator emits (and that this file must contain to suppress):

```
Method not implemented: <Class>.<snake_method>
Parameter not implemented: <Class>.<snake_method>(<snake_param>=)
Parameter not documented: <Class>.<snake_method>(<snake_param>=)
Method vs property mismatch: <Class>.<snake_method>
Method not documented: <Class>.<snake_method>
Parameter type mismatch in <Class>.<snake_method>(<arg>=): documented as <T1>, code has <T2>
```

Class names use the upstream PascalCase (`BrowserContext`, `BrowserType`); method/param names are converted to `snake_case` matching the Python surface.

## Tips & gotchas

- **`langs.only` is your filter — and the only filter.** Don't classify by name (`Screencast`, `Debugger`, `pickLocator`) or by intuition ("looks like internal tooling"). Always check `langs` in `api.json`. The 1.59 roll cost two extra audit passes by trusting names over `langs`.
- **Audit your own classifications a second time.** After the first walk through the commit range, before opening the PR, re-read every line in `expected_api_mismatch.txt` you added during this roll and ask "is this divergence justified, or did I skip a port?" Run the `langs`-dump snippet on each suspicious entry. The 1.59 roll's first PR had ~20 wrong suppressions; the second pass cut them to 0.
- **A cluster of suppressions on the same class is a smell.** If you're about to add five `Method not implemented: Foo.*` lines, you're almost certainly looking at a class that needs to be implemented. Implement the whole thing once and the suppressions disappear.
- **Watch for revert pairs in the same range.** 1.59 added and reverted `Browser.isRemote` (#39613 / #39620) inside the same release. Walking chronologically lets you skip the add when you see the revert later.
- **Watch for rename-revert pairs.** 1.59 had `Locator.normalize` → `Locator.toCode` (#39648) → `Locator.normalize` (#39754). Final state wins; only port the last.
- **Doc renames almost always include a wire-protocol rename.** Whenever you see `### param: X.y.oldName` → `### param: X.y.newName` in a doc commit, also `git -C "$PW_SRC_DIR" show <sha> -- packages/protocol/src/protocol.yml` and the corresponding `*Dispatcher.ts` file. If the wire field changed too, the channel-send dict key in `_impl/` must change. Suppressing the doc-side mismatch is hiding a real bug — the previous Python code is silently sending an unknown field that the new server ignores.
- **TypedDicts beat `Dict[str, X]` for any structured return.** When the docs describe a return as `[Object]` with named fields (or even `[Object=Foo]`), define a `TypedDict` in `_api_structures.py`, re-export from both public `__init__.py` files, and use it. Zero runtime cost (it's still a `dict`), and the doc generator's type comparator matches by structure via `get_type_hints`.
- **Positional renames are free.** A param with no default before any `*` separator is positional-or-keyword in Python, but realistic call sites pass it positionally. Renaming such a param doesn't break callers.
- **The "Backport changes" GitHub issue can be misleading.** In the 1.59 roll its checkboxes were all marked `[x]` with annotations like "✅ IMPLEMENTED", but several of those features had not actually been merged into the Python port. Trust the `docs/src/api/` walk over the issue.
- **`api.json` may carry doclint quirks.** 1.59 hit two: `Promise<X>|X` collapsed to a bare `Promise` with no `templates`, and `void`-typed events serialized as the literal string `"void"`. Both are upstream artifacts; patch `inner_serialize_doc_type` to handle them rather than fighting the api.json side.
- **Don't edit `_generated.py` to fix lint or typing.** Fix `_impl/`, `documentation_provider.py`, or `expected_api_mismatch.txt` instead. Hand-editing the generated file is reverted on the next regen.
- **`/tmp/roll-<new>-commits.md` is a working artifact, not a deliverable.** Don't commit it. The commit message and PR description are where the audit summary belongs.