Design a CI workflow that fails fast, isolates flaky tests, and doesn't punish contributors for a broken shared runner

Act as a senior software engineer who owns CI reliability for this team. I need a CI workflow designed for a specific repo and its specific pain points, not a generic starter template. Work through this in phases.

PHASE 1 — UNDERSTAND CURRENT STATE
Repo and stack:
Python 3.12 monorepo, pytest with ~4,200 tests across 6 services, roughly 40 PRs merged per week

Current CI provider and constraints:
GitHub Actions, self-hosted runner pool of 8 machines, no budget for GitHub-hosted large runners

Known pain points:
Three integration tests fail intermittently about 15% of the time depending on run order, and PRs regularly get re-run 2-3 times before merging

Ask me up to three clarifying questions only if something above is ambiguous enough to change the design meaningfully (for example, whether tests can run in parallel across multiple containers, or whether secrets are already in a vault). Do not ask questions you could reasonably infer.

PHASE 2 — PIPELINE DESIGN
Propose the stage structure (e.g., lint → unit → integration → build → deploy-gate) with what runs in parallel versus what must be sequential, and why. For each stage, specify what causes it to fail fast versus what's allowed to be non-blocking (informational only). Design a caching strategy for dependencies and build artifacts specific to the stack given, not a generic 'cache node_modules' answer — name the actual cache keys and invalidation triggers.

PHASE 3 — FLAKY TEST HANDLING
Given the known pain points, propose a concrete quarantine mechanism: how a flaky test gets flagged, where it's tracked, how it's prevented from blocking merges while still being visible so it doesn't get forgotten forever, and who is accountable for un-quarantining it. This must be a real, checkable process, not just 'add retries to flaky tests' — retries alone hide a flaky test's existence instead of fixing it.

PHASE 4 — FAILURE MODES
Name the two most likely ways this pipeline design breaks in practice (e.g., cache poisoning across branches, a shared runner pool getting starved during a release week) and what monitoring or alert would catch each one.

OUTPUT FORMAT
1. Any clarifying questions (skip if none needed).
2. The pipeline stage diagram described in prose or an ordered list.
3. The actual CI config file for the provider given, in a code block.
4. The flaky-test quarantine process as a numbered procedure.
5. The two failure modes and their corresponding alerts.