testing

---
name: testing
description: Run all tests or individual tests for Usd Optimize (repo.sh on Linux, repo.bat on Windows). Use for unit, binding, and coverage runs.
license: Apache-2.0
---

<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

# Testing Usd Optimize

## What this skill covers

Search this doc for keywords like `pytest`, `doctest`, `python`, `coverage`, `gcov`, `-k`, `-s`, `--config` to jump.

- **Run All Tests** — `./repo.sh test`.
- **Test Suites** — `cpp`, `python` and how to scope a run.
- **Build Config** — running tests in `release` vs `debug`.
- **Running Individual Tests** — Doctest filters and Python suite limitations.
- **Test Locations** — where each suite's tests live in the tree.
- **Code Coverage (Linux only)** — gcov collection workflow.

Companion skills: `build` (must run before tests), `new-validator` (authoring validator rules and their `test_validators_*` tests) and `run-validators` (running the performance validators against a USD asset).

> The repo ships two equivalent entry scripts: `./repo.sh` for Linux/bash-compatible shells and `repo.bat` for Windows `cmd.exe`/PowerShell. They accept the same arguments — every example below shows both. Pick whichever matches the active shell.

## Run All Tests

```bash
./repo.sh test       # Linux / bash
repo.bat test        # Windows cmd.exe / PowerShell
```

This runs the default test suites: **cpp** (C++ unit tests) and **python** (Python binding tests). The defaults come from `repo.toml` → `[repo_test].default_suite = ["cpp", "python"]`.

> **Heads-up on the Run Summary:** the final `repo_test Run Summary` block only counts `glob_and_exec`-kind suites, so it reports `All 1 tests processes returned 0` even when cpp also ran. Confirm cpp ran by looking earlier in the output for `[doctest] Status: SUCCESS!` lines — don't trust the trailing summary alone.

## Test Suites

Available suites: `cpp`, `python`, `all`

```bash
# Run only C++ unit tests
./repo.sh test -s cpp
repo.bat test -s cpp

# Run only Python binding tests
./repo.sh test -s python
repo.bat test -s python

# Run all suites
./repo.sh test -s all
repo.bat test -s all
```

> **Note:** `test.cuda.utils/` under `source/tests/` is a helper *shared library* (`TestCudaUtils.so` / `.dll`) used by the cpp suite for `isCudaAvailable()` threading tests — it is **not** itself a separately runnable test suite.

## Build Config

Tests default to the **release** build. To test against debug:

```bash
./repo.sh test -c debug
repo.bat test -c debug
```

## Running Individual Tests

### C++ Unit Tests (Doctest)

The C++ tests use the Doctest framework. The test executable is at `_build/<platform>/<config>/test.cpp` (Linux) or `_build/windows-x86_64/<config>/test.cpp.exe` (Windows).

Use `-e=` to pass extra arguments through to the doctest executable (the `=` is required so that `--` prefixed values aren't parsed as repo flags):

```bash
# Run a specific test case by name
./repo.sh test -s cpp -e="--test-case=_deletePrims"
repo.bat test -s cpp -e="--test-case=_deletePrims"

# Run test cases matching a pattern
./repo.sh test -s cpp -e="--test-case=*Plugins*"
repo.bat test -s cpp -e="--test-case=*Plugins*"

# List all available test cases without running them
./repo.sh test -s cpp -e="--list-test-cases"
repo.bat test -s cpp -e="--list-test-cases"
```

You can also run the wrapper script directly for full doctest CLI control (use the bundled wrapper — `.sh` on Linux, `.bat` on Windows — so `LD_LIBRARY_PATH` / `PATH` are set correctly):

```bash
# Linux
./_build/linux-x86_64/release/test.cpp.sh --test-case="*Plugins*"
./_build/linux-x86_64/release/test.cpp.sh --test-case="_deletePrims" --subcase="DeleteOption::ePrimOnly"

# Windows (cmd.exe / PowerShell)
.\_build\windows-x86_64\release\test.cpp.bat --test-case="*Plugins*"
.\_build\windows-x86_64\release\test.cpp.bat --test-case="_deletePrims" --subcase="DeleteOption::ePrimOnly"
```

### Python Binding Tests

Run Python binding tests through the repo wrapper so the bundled Python, USD,
and Usd Optimize paths are set correctly:

```bash
./repo.sh test -s python                    # run everything
repo.bat test -s python
```

To run an individual test, pass a `module`, `module.Class`, or
`module.Class.method` spec. `-e=` forwards the arg through to the Python test
runner (`run_discover.py`):

```bash
# Run every test in test_operation_pivot.py
./repo.sh test -s python -e=test_operation_pivot
repo.bat test -s python -e=test_operation_pivot

# Run all tests in a class
./repo.sh test -s python -e=test_operation_pivot.Test_Operation_Pivot
repo.bat test -s python -e=test_operation_pivot.Test_Operation_Pivot

# Run a single test method
./repo.sh test -s python -e=test_operation_pivot.Test_Operation_Pivot.test_pivot_xforms_center
repo.bat test -s python -e=test_operation_pivot.Test_Operation_Pivot.test_pivot_xforms_center

# Substring pattern across module.Class.method names (pytest-style -k)
./repo.sh test -s python -e=-k -e=pivot_xforms
repo.bat test -s python -e=-k -e=pivot_xforms
```

A trailing `.py` on the module name is accepted and stripped, so
`-e=test_operation_pivot.py` works the same as `-e=test_operation_pivot`.

You can also invoke the generated wrapper directly for the same effect (the
wrapper sets `LD_LIBRARY_PATH` / `PATH` for you):

```bash
# Linux
./_build/linux-x86_64/release/test.python.sh test_operation_pivot
./_build/linux-x86_64/release/test.python.sh -k pivot_xforms

# Windows (cmd.exe / PowerShell)
.\_build\windows-x86_64\release\test.python.bat test_operation_pivot
.\_build\windows-x86_64\release\test.python.bat -k pivot_xforms
```

Do **not** use `repo.sh test -s python -f "test_operation_*.py"` for filtering:
`-f` filters the generated wrapper executable list (`test.python.sh` /
`.bat`), not the files under `source/tests/test.python/`. Use `-e=` instead.

## Test Locations

```
source/tests/
├── test.cpp/usd_optimize.core/   # C++ unit tests (Doctest)
│   ├── TestPlugins.cpp
│   ├── TestMerge.cpp
│   ├── TestDeletePrims.cpp
│   ├── TestOptimizeMaterials.cpp
│   ├── TestSpatialAnalysis.cpp
│   └── TestMisc.cpp
├── test.python/                          # Python binding tests
│   ├── test_operation_merge.py
│   ├── test_operation_decimate_meshes.py
│   ├── test_operation_shrinkwrap.py
│   └── ... (one file per operation)
└── test.cuda.utils/                      # Helper shared lib for cpp suite (not a runnable suite)
```

## Code Coverage (Linux only)

The gcov-based coverage flow only runs on Linux because it depends on a gcc-built binary; `repo.bat` does not expose this command.

```bash
./repo.sh build --rebuild --config release --enable-gcov
./repo.sh test
./repo.sh cxx_coverage --collect --generate-html --remove --report
# Output: _build/coverage/
./repo.sh cxx_coverage --zero-coverage   # Reset counters
```

## Purpose

Drive the repo's test runner across the C++ unit suite (Doctest) and the
Python bindings suite. Cover both "run everything" and supported targeted
invocations (C++ test case filters, debug config, coverage run). Always go through `./repo.sh test`
or `repo.bat test` — never invoke the test executables raw, because
the wrappers set the library and Python paths the build expects.

## Prerequisites

- A built repo for the config you're testing — `./repo.sh build`
  (or `--config debug` for a debug-config run). The `build` skill
  covers this.
- For coverage: a `--enable-gcov` rebuild on Linux. Coverage is not
  available on Windows.
- For Python binding tests: nothing extra — the bundled Python is
  resolved by the test wrapper.

## Limitations

- This skill runs tests; it does **not** author them. New tests for a
  new operation are produced by the `new-operation` skill (Step 3).
- The Run Summary block at the end of `repo.sh test` only counts
  `glob_and_exec`-kind suites — the C++ suite is reported earlier in
  the output. Don't trust the trailing summary line alone.
- Code coverage (`cxx_coverage`) is **Linux only**.
- This skill does not run validator-driver tests against external
  assets — those use `run-validators` against the asset directly.

## Troubleshooting

Tests are themselves a form of validation; when they fail, treat the
failure as the signal to look at — don't paper over it.

| Symptom | Likely cause | Fix |
|---|---|---|
| All tests fail with `library not found` / `DLL load failed` | Built one config but ran tests against another (e.g. built `debug`, ran default `release`). | Match `--config` between `build` and `test` (or pass `-c debug` to both). |
| `[doctest] Status: SUCCESS!` but the trailing summary says `1 tests processes returned 0` | Quirk of the runner — the C++ suite isn't `glob_and_exec`, so the summary undercounts. | Scroll up to confirm the `[doctest]` line. The exit code is authoritative. |
| Python binding test errors with `ImportError: cannot import name UsdOptimizeCore` | Build didn't produce the bindings, or `PYTHONPATH` is shadowed. | Check `_build/<platform>/<config>/python/` exists. Run via `./repo.sh test -s python` (don't invoke pytest directly). |
| `repo.sh test -s python -f "test_operation_foo.py"` reports `All 0 tests processes returned 0` | `-f` filtered out the `test.python` wrapper instead of selecting a Python test file. | Use `-e=test_operation_foo` (not `-f`) to forward the spec through to `run_discover.py`. |
| Coverage HTML is empty after `cxx_coverage --collect` | Counters weren't zeroed, or no test was actually executed under the gcov build. | Run `./repo.sh cxx_coverage --zero-coverage` first, then `./repo.sh test`, then `--collect --generate-html`. |
| A specific Doctest case won't match `--test-case` filter | Pattern needs quoting — shells can eat `*`. | Use `-e="--test-case=*Plugins*"` (the `=` is required so the runner doesn't parse the value as its own flag). |