property-based-testing

---
name: property-based-testing
description: >
license: MIT
---

# Property-Based Testing


Canonical FP bar: [`docs/fcis-engineering-rules.md`](../../docs/fcis-engineering-rules.md) — **Functional Core, Imperative Shell**: pure domain modules; side effects at edges. Prefer testing pure core without Repo; use DataCase only for true persistence boundaries.

## RULES — Follow these with no exceptions

**1.** **Define properties, not examples** — state what should always be true, not specific inputs/outputs
**2.** **Test invariants, not specific values** — "output length equals input length" not "output is [1, 2, 3]"
**3.** **Leverage shrinking** — let StreamData find minimal failing cases; add constraints like `min_length: 1` or `integer(1..100)` when shrunk cases expose invalid generator inputs


## Setup

```elixir
# mix.exs
defp deps do
  [
    {:stream_data, "~> 1.0", only: :test}
  ]
end
```


## Basic Property Test

```elixir
defmodule MyApp.StringUtilsTest do
  use ExUnit.Case, async: true
  use ExUnitProperties

  property "reversing a string twice returns the original" do
    check all string <- string(:ascii) do
      assert string |> String.reverse() |> String.reverse() == string
    end
  end

  property "length of reversed string equals original length" do
    check all string <- string(:ascii) do
      assert String.length(String.reverse(string)) == String.length(string)
    end
  end
end
```


## Generators

### Less Obvious Built-ins

```elixir
# Combine multiple generators with one_of
one_of([constant(:a), constant(:b), constant(:c)])
one_of([integer(), string(:ascii), boolean()])

# Compose generators with gen all
gen all x <- integer(0..100),
        y <- integer(0..100),
        x != y do
  {x, y}
end

# Non-empty collections
list_of(integer(), min_length: 1)
map_of(string(:alphanumeric), integer())
```

### Custom Generators

```elixir
# Generate a valid email
def email_generator do
  gen all name <- string(:alphanumeric, min_length: 3),
          domain <- string(:alphanumeric, min_length: 3) do
    "#{name}@#{domain}.com"
  end
end

# Generate a user struct
def user_generator do
  gen all email <- email_generator(),
          age <- integer(18..120) do
    %User{email: email, age: age}
  end
end

# Use in tests
property "users have valid emails" do
  check all user <- user_generator() do
    assert user.email =~ ~r/@.*\.com$/
    assert user.age >= 18 and user.age <= 120
  end
end
```


## Testing Invariants

Key invariants to test for collections: ordering, element preservation, and idempotence.

```elixir
defmodule MyApp.SortingTest do
  use ExUnit.Case, async: true
  use ExUnitProperties

  # Invariant: sorted list is ordered
  property "sorted list is ordered" do
    check all list <- list_of(integer()) do
      sorted = Enum.sort(list)

      sorted
      |> Enum.chunk_every(2, 1, :discard)
      |> Enum.each(fn [a, b] -> assert a <= b end)
    end
  end

  # Invariant: sorting preserves all elements
  property "sorted list contains same elements" do
    check all list <- list_of(integer()) do
      assert Enum.sort(list) |> Enum.frequencies() == Enum.frequencies(list)
    end
  end
end
```


## Shrinking

StreamData automatically shrinks failing inputs to the smallest example that still fails. The
property below *intentionally* fails to illustrate shrinking output:

```elixir
property "no list contains its own length as element" do
  check all list <- list_of(integer()) do
    refute list |> Enum.member?(length(list))
  end
end

# StreamData will find and report:
# Failed with generated values: [0]
# (shrunk from something like [1, 5, 0, 3, 2])
```


## Workflow: Write → Run → Interpret → Refine

1. **Write property** — define an invariant using `check all` with appropriate generators
2. **Run test** — `mix test test/my_test.exs`
3. **Interpret shrinking output** — on failure, StreamData reports both the original failing input and the shrunk minimal example:
   ```
   ** (ExUnit.AssertionError)
   Failed with generated values (after 3 successful runs):

       * Clause:    list <- list_of(integer())
         Generated: [0]
         (shrunk from [42, -7, 0, 15])
   ```
4. **Refine generator or property** — if the shrunk case reveals a generator producing invalid inputs, add constraints (e.g. `min_length: 1`, `integer(1..100)`); if it reveals a real bug, fix the implementation
5. **Re-run** — confirm the fix holds across new generated cases


## Common Pitfalls

| ❌ Don't | ✅ Do |
|----------|-------|
| Assert exact outputs (`== [1, 2, 3]`) | Assert invariants that always hold (ordering, length, membership) |
| Generate unconstrained inputs that break the code under test | Constrain generators (`min_length: 1`, `integer(1..100)`) to valid domains |
| Ignore the shrunk minimal case in failure output | Read the shrunk values first — they point straight at the bug |
| Use `check all` without `use ExUnitProperties` | Add `use ExUnitProperties` alongside `use ExUnit.Case` |
| Write one giant property covering many behaviours | Split into focused properties, one invariant each |
| Leave generators unbounded, causing slow/huge cases | Bound ranges and collection sizes to keep runs fast |

---

## Integration

| Predecessor | This Skill | Successor |
|-------------|------------|-----------|
| testing-essentials | property-based-testing | None (standalone) |