cloud-serverless

---
name: cloud-serverless
description: >-
license: Apache-2.0
---

# Redpanda Cloud API: Serverless Clusters

Redpanda Cloud Serverless clusters are fully managed Kafka-compatible clusters
provisioned through the public Control Plane API at `https://api.redpanda.com`.
You authenticate once with OAuth2 client credentials, then create a
`ResourceGroup`, pick a `ServerlessRegion`, and call `POST /v1/serverless/clusters`.
The API returns an async `Operation`; when the Operation reaches
`STATE_COMPLETED`, the cluster has reached `STATE_READY` and its
`dataplane_api.url` is live for Kafka workloads and the Data Plane REST API.
(`STATE_COMPLETED` is the Operation state; `STATE_READY` is the cluster state —
they are distinct state machines.)

This skill does **not** cover BYOC (Bring Your Own Cloud) clusters — see the
`cloud-byoc` skill for those. For the CLI equivalent, see the `rpk-cloud` skill.

## Quickstart

Copy-paste these commands. Replace `CLIENT_ID` and `CLIENT_SECRET` with the
credentials from your Redpanda Cloud service account.

```bash
# ── 1. Get a bearer token ─────────────────────────────────────────────────────
TOKEN=$(curl -s -X POST https://auth.prd.cloud.redpanda.com/oauth/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id=${CLIENT_ID}" \
  -d "client_secret=${CLIENT_SECRET}" \
  -d "audience=cloudv2-production.redpanda.cloud" \
  | jq -r .access_token)

# ── 2. List available serverless regions ──────────────────────────────────────
curl -s -X GET "https://api.redpanda.com/v1/serverless/regions?cloud_provider=CLOUD_PROVIDER_AWS" \
  -H "Authorization: Bearer ${TOKEN}" | jq .

# ── 3. Create a resource group (billing/org container) ───────────────────────
RG_ID=$(curl -s -X POST https://api.redpanda.com/v1/resource-groups \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"resource_group": {"name": "my-dev-group"}}' \
  | jq -r .resource_group.id)

# ── 4. Create a serverless cluster ────────────────────────────────────────────
OP_ID=$(curl -s -X POST https://api.redpanda.com/v1/serverless/clusters \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d "{
    \"serverless_cluster\": {
      \"name\": \"my-cluster\",
      \"resource_group_id\": \"${RG_ID}\",
      \"serverless_region\": \"us-east-1\"
    }
  }" | jq -r .operation.id)

# ── 5. Poll the operation until STATE_COMPLETED ───────────────────────────────
watch -n 5 "curl -s https://api.redpanda.com/v1/operations/${OP_ID} \
  -H 'Authorization: Bearer ${TOKEN}' | jq '{state: .operation.state}'"

# ── 6. Get the cluster and its data-plane URL ─────────────────────────────────
# Read the cluster ID directly from the completed operation (resource_id or
# metadata.serverless_cluster_id) — more reliable than listing by name.
OP_RESP=$(curl -s "https://api.redpanda.com/v1/operations/${OP_ID}" \
  -H "Authorization: Bearer ${TOKEN}")
CLUSTER_ID=$(echo "${OP_RESP}" | jq -r '.operation.resource_id')

CLUSTER=$(curl -s "https://api.redpanda.com/v1/serverless/clusters/${CLUSTER_ID}" \
  -H "Authorization: Bearer ${TOKEN}" | jq '.serverless_cluster')
echo "${CLUSTER}" | jq '{id: .id, state: .state, dataplane_url: .dataplane_api.url}'

DP_URL=$(echo "${CLUSTER}" | jq -r '.dataplane_api.url')

# ── 7. Create a topic via the data-plane URL ──────────────────────────────────
curl -s -X POST "${DP_URL}/v1/topics" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{"topic": {"name": "events", "partition_count": 3, "replication_factor": 3}}'
```

## Authentication

The Control Plane API and Data Plane API both use the same OAuth2 bearer token.
See [Authentication Reference](references/auth.md) for full details.

| Field | Value |
|---|---|
| Token endpoint | `https://auth.prd.cloud.redpanda.com/oauth/token` |
| Audience | `cloudv2-production.redpanda.cloud` |
| Grant type | `client_credentials` |
| Control Plane base URL | `https://api.redpanda.com` |

Credentials come from a **ServiceAccount** created in the Redpanda Cloud console
or via `POST /v1/service-accounts`. The response includes a `client_id` and
`client_secret` (the secret is only shown once on creation; save it securely).

## Control Plane Resources

### ResourceGroup

A ResourceGroup is the billing and organizational container for clusters. It
must exist before you can create a ServerlessCluster.

```
POST   /v1/resource-groups
GET    /v1/resource-groups/{id}
GET    /v1/resource-groups          # list with optional filter.name_contains
PATCH  /v1/resource-groups/{resource_group.id}
DELETE /v1/resource-groups/{id}
```

Name constraints: 3–253 alphanumeric characters and hyphens (`^[a-zA-Z0-9-]+$`).

### ServerlessRegion

Regions represent the cloud-provider geographic areas where Serverless clusters
can be placed. List them before creating a cluster to confirm `placement.enabled`.

```
GET /v1/serverless/regions?cloud_provider=CLOUD_PROVIDER_AWS
GET /v1/serverless/regions?cloud_provider=CLOUD_PROVIDER_GCP
GET /v1/serverless/region?cloud_provider=CLOUD_PROVIDER_AWS&name=us-east-1
```

`cloud_provider` accepts `CLOUD_PROVIDER_AWS`, `CLOUD_PROVIDER_GCP`, or
`CLOUD_PROVIDER_AZURE`. Use `filter.placement_enabled_only=true` to skip
regions that cannot currently accept new clusters.

### ServerlessCluster Lifecycle

Full CRUD plus the Operation pattern:

```
POST   /v1/serverless/clusters           → 202 CreateServerlessClusterOperation
GET    /v1/serverless/clusters/{id}      → ServerlessCluster
GET    /v1/serverless/clusters           → ListServerlessClustersResponse (paginated)
PATCH  /v1/serverless/clusters/{id}      → 202 UpdateServerlessClusterOperation
DELETE /v1/serverless/clusters/{id}      → 202 DeleteServerlessClusterOperation
GET    /v1/serverless/clusters/{id}/prometheus/credentials
```

**State machine** (from the proto):

| State | Meaning |
|---|---|
| `STATE_PLACING` | Finding a cell with sufficient resources |
| `STATE_CREATING` | Creating control-plane state |
| `STATE_READY` | Running and accepting requests |
| `STATE_SUSPENDED` | Running but blocking external requests |
| `STATE_DELETING` | Removal in progress |
| `STATE_FAILED` | Could not reach READY from PLACING or CREATING |

**Create request fields** (grounded in `serverless.proto`):

| Field | Required | Notes |
|---|---|---|
| `name` | yes | 3–128 chars, `^[A-Za-z0-9-_:]+$` |
| `resource_group_id` | yes | UUID of an existing ResourceGroup |
| `serverless_region` | yes | Region name string, e.g. `"us-east-1"` |
| `tags` | no | `map<string,string>`, max 50 pairs |
| `networking_config` | no | Both fields default to `STATE_UNSPECIFIED` (0); resolved as public enabled, private disabled |

**Response fields on GET** (output-only):

| Field | Notes |
|---|---|
| `id` | 20-char opaque ID |
| `state` | One of the states above |
| `kafka_api.seed_brokers[]` | Public bootstrap servers |
| `kafka_api.private_seed_brokers[]` | Private bootstrap (when private networking enabled) |
| `dataplane_api.url` | Base URL for the per-cluster Data Plane REST API |
| `schema_registry.url` | Schema Registry public endpoint |
| `console_url` | Redpanda Console web UI URL |
| `prometheus.url` | Prometheus scrape endpoint |

### Operations

Create, Update, and Delete each return an `Operation` whose `type` is
`TYPE_CREATE_SERVERLESS_CLUSTER`, `TYPE_UPDATE_SERVERLESS_CLUSTER`, or
`TYPE_DELETE_SERVERLESS_CLUSTER` respectively, so you can filter operations by
any of the three. Poll at `GET /v1/operations/{id}`.

```
GET /v1/operations/{id}
GET /v1/operations               # list with filter.type_in / filter.state / filter.resource_id
```

Operation states: `STATE_IN_PROGRESS`, `STATE_COMPLETED`, `STATE_FAILED`.
When `state == STATE_COMPLETED`, the `response` field contains the final
resource. When `state == STATE_FAILED`, the `error` field contains a
`google.rpc.Status`.

### ServerlessPrivateLink (AWS PrivateLink)

The `private_link_id` field on a cluster refers to a **ServerlessPrivateLink**
resource, managed by its own control-plane service (**AWS only**). Create the
private link first, then pass its 20-char `id` as `private_link_id` when you
create a cluster with private networking enabled. Serverless on AWS went GA in
Feb 2026 with PrivateLink support.

```
POST   /v1/serverless/private-links       → 202 Operation (TYPE_CREATE_SERVERLESS_PRIVATE_LINK)
GET    /v1/serverless/private-links/{id}
GET    /v1/serverless/private-links        # list (paginated, filterable)
PATCH  /v1/serverless/private-links/{id}   → 202 Operation
DELETE /v1/serverless/private-links/{id}   → 202 Operation
```

Required create fields: `name`, `resource_group_id`, `cloudprovider`
(`CLOUD_PROVIDER_AWS` only — CEL-enforced with `aws_config`),
`aws_config.allowed_principals` (min 1 AWS principal ARN), and
`serverless_region`. See
[Control Plane: Serverless](references/control-plane-serverless.md#serverlessprivatelink)
for the full field-level reference and the private-networking workflow.

## Data Plane API

Once your cluster is `STATE_READY`, use `dataplane_api.url` as the base URL.
The same bearer token is valid. Base path is `/v1`. Always read the URL from
the API response — do not construct it manually. The real DNS pattern (from
`openapi.controlplane.yaml` examples) is
`https://<cluster-id>.any.<region>.mpx.prd.cloud.redpanda.com`.

Available services (grounded in `dataplane.go` and `openapi.dataplane.yaml`):

| Service | Endpoint prefix |
|---|---|
| Topic | `/v1/topics` |
| ACL | `/v1/acls` |
| User | `/v1/users` |
| Secret | `/v1/secrets` |
| Pipeline (Redpanda Connect) | `/v1/redpanda-connect/pipelines` |
| Security (RBAC roles) | `/v1/roles` |
| Quota | `/v1/quotas` |
| KafkaConnect | `/v1/kafka-connect/clusters/{cluster_name}/connectors` |

See [Data Plane Reference](references/data-plane.md) for examples and
availability notes on Serverless clusters.

## Enterprise Features on Serverless

Redpanda Cloud is a managed deployment of **Redpanda Enterprise Edition** — the
license is included, so there is no `rpk cluster license` step for tenants. The
enterprise differentiators you configure on Serverless are exposed as **topic
configuration keys** (via `${DP_URL}/v1/topics`) and **RBAC roles** (via
`${DP_URL}/v1/roles`), not cluster/node config:

| Feature (Enterprise) | How to set on Serverless | Key(s) |
|---|---|---|
| Iceberg Topics | Topic config | `redpanda.iceberg.mode` (`disabled`/`key_value`/`value_schema_id_prefix`/`value_schema_latest`), `redpanda.iceberg.target.lag.ms`, `redpanda.iceberg.partition.spec`, `redpanda.iceberg.invalid.record.action` (`drop`/`dlq_table`), `redpanda.iceberg.delete` |
| Server-Side Schema ID Validation | Topic config | `redpanda.key.schema.id.validation`, `redpanda.value.schema.id.validation`, `redpanda.key.subject.name.strategy`, `redpanda.value.subject.name.strategy` (`TopicNameStrategy`/`RecordNameStrategy`/`TopicRecordNameStrategy`) |
| Leadership Pinning | Topic config | `redpanda.leaders.preference` (inherits cluster `default_leaders_preference`) |
| RBAC | Data Plane SecurityService | `${DP_URL}/v1/roles` |

Cluster/node-level enterprise features (Tiered Storage internals, FIPS,
Continuous Data Balancing thresholds, Audit Logging, Remote Read Replicas,
Cloud Topics, Shadow Linking / cross-cluster DR, Whole Cluster Restore) are
**managed by Redpanda** on Serverless and are not tenant-configurable — use a
BYOC/Dedicated or self-managed cluster for those. See
[Enterprise Features Reference](references/enterprise-features.md) for the full
key tables, accepted values, defaults, mode semantics, and self-managed mappings.

## Serverless vs BYOC

| | Serverless | BYOC |
|---|---|---|
| Infra ownership | Redpanda manages everything | Customer's cloud account |
| Network resources | Not required | Requires creating a `Network` resource |
| Cluster type | Serverless (separate `ServerlessCluster` resource; no `Cluster.Type`) | BYOC (`Cluster.Type` = `TYPE_BYOC`) |
| Provisioning agent | None | `rpk cloud byoc apply` runs Terraform |
| Billing | Redpanda bills per-use | Customer pays cloud provider directly |

## Reference Directory

- [Authentication](references/auth.md): OAuth2 client-credentials flow, token endpoint, audience, Authorization header, service-account creation, and token refresh.
- [Control Plane: Serverless](references/control-plane-serverless.md): ResourceGroup, ServerlessRegion, and ServerlessCluster field-level reference, the async Operation pattern, state machine, and list filtering.
- [Data Plane](references/data-plane.md): Using the per-cluster Data Plane API URL, Topic/ACL/User/Secret/Pipeline endpoints with curl examples.
- [Enterprise Features](references/enterprise-features.md): Enterprise differentiators configurable on Serverless — Iceberg Topics, Server-Side Schema ID Validation, Leadership Pinning, and RBAC — with every topic config key, accepted values, defaults, Iceberg mode semantics, license-requirement notes, and which cluster-level features are managed by Redpanda (not tenant-configurable).