deepgram-python-audio-intelligence

---
name: deepgram-python-audio-intelligence
description: Use when writing or reviewing Python code in this repo that calls Deepgram audio analytics overlays on `/v1/listen` - summarize, topics, intents, sentiment, diarize, redact, detect_language, entity detection. Same endpoint as plain STT but with analytics params. Covers both REST (`client.listen.v1.media.transcribe_url`/`transcribe_file`) and the WSS-supported subset (`client.listen.v1.connect`). Use `deepgram-python-speech-to-text` for plain transcription, `deepgram-python-text-intelligence` for analytics on already-transcribed text. Triggers include "diarize", "summarize audio", "sentiment from audio", "redact PII", "topic detection audio", "audio intelligence", "detect language audio".
license: MIT
---

# Using Deepgram Audio Intelligence (Python SDK)

Analytics overlays applied to `/v1/listen` transcription: summarize, topics, intents, sentiment, language detection, diarization, redaction, entities. Same endpoint / same client methods as STT — enable features via params.

## When to use this product

- You have **audio** (file, URL, or live stream) and want analytics alongside the transcript.
- REST is the primary path — most analytics are REST-only.

**Use a different skill when:**
- You want a pure transcript with no analytics → `deepgram-python-speech-to-text`.
- Your input is already transcribed text → `deepgram-python-text-intelligence` (`/v1/read`).
- You need conversational turn-taking → `deepgram-python-conversational-stt`.
- You need a full interactive agent → `deepgram-python-voice-agent`.

## Feature availability: REST vs WSS

| Feature | REST | WSS |
|---|---|---|
| `diarize` | yes | yes |
| `redact` | yes | yes |
| `punctuate`, `smart_format` | yes | yes |
| Entity detection | yes | yes |
| `summarize` | yes | **no** |
| `topics` | yes | **no** |
| `intents` | yes | **no** |
| `sentiment` | yes | **no** |
| `detect_language` | yes | **no** |
| `custom_topic` / `custom_intent` | yes | **no** |

For the WSS-only subset, same code path as `deepgram-python-speech-to-text`.

## Authentication

```python
from dotenv import load_dotenv
load_dotenv()

from deepgram import DeepgramClient
client = DeepgramClient()
```

Header: `Authorization: Token <api_key>`.

## Quick start — REST with full analytics

```python
response = client.listen.v1.media.transcribe_url(
    url="https://dpgr.am/spacewalk.wav",
    model="nova-3",
    smart_format=True,
    punctuate=True,
    diarize=True,              # speaker separation
    summarize="v2",            # "v2" for the current model; True also accepted on /v1/listen
    topics=True,
    intents=True,
    sentiment=True,
    detect_language=True,
    redact=["pci", "pii"],     # or Sequence[str]
    language="en-US",
)

r = response.results
print("transcript:", r.channels[0].alternatives[0].transcript)
print("summary:",    r.summary)
print("topics:",     r.topics)
print("intents:",    r.intents)
print("sentiments:", r.sentiments)
print("detected_language:", r.channels[0].detected_language)

# Speaker diarization
for word in r.channels[0].alternatives[0].words or []:
    speaker = getattr(word, "speaker", None)
    if speaker is not None:
        print(f"Speaker {speaker}: {word.word}")
```

## Quick start — REST file

```python
with open("call.wav", "rb") as f:
    audio = f.read()

response = client.listen.v1.media.transcribe_file(
    request=audio,
    model="nova-3",
    diarize=True,
    redact=["pii"],
    summarize="v2",
    topics=True,
)
```

## Quick start — diarization with word-level timings

Enable speaker separation and word-level timestamps in a single request, then iterate the per-word objects to build a speaker-labelled transcript with timing.

```python
response = client.listen.v1.media.transcribe_url(
    url="https://dpgr.am/spacewalk.wav",
    model="nova-3",
    diarize=True,        # tag each word with a speaker id
    smart_format=True,   # punctuated_word for cleaner output
    punctuate=True,
)

words = response.results.channels[0].alternatives[0].words or []

# Per-word: speaker, timestamps, confidence
for w in words:
    speaker = getattr(w, "speaker", None)
    text = w.punctuated_word or w.word
    print(f"[speaker {speaker}] {text}  ({w.start:.2f}s–{w.end:.2f}s, conf={w.confidence:.2f})")

# Group consecutive words by speaker into utterances
from itertools import groupby
for speaker, group in groupby(words, key=lambda w: getattr(w, "speaker", None)):
    text = " ".join((w.punctuated_word or w.word) for w in group)
    print(f"Speaker {speaker}: {text}")
```

Per-word fields available on each entry:

| Field | Type | Description |
|---|---|---|
| `word` | `str` | Lowercase token |
| `punctuated_word` | `str \| None` | Token with smart-formatted casing/punctuation (when `smart_format=True`) |
| `start`, `end` | `float` | Audio timestamps in seconds |
| `confidence` | `float` | 0.0–1.0 confidence |
| `speaker` | `int \| None` | Speaker id (when `diarize=True`); `None` if diarization disabled |
| `speaker_confidence` | `float \| None` | Speaker-id confidence |

For a higher-level breakdown, set `utterances=True` to get pre-grouped speaker turns at `response.results.utterances`. Set `paragraphs=True` for a `paragraphs` view organised by speaker turn boundaries.

## Quick start — WSS subset (diarize / redact / entities only)

```python
import threading
from deepgram.core.events import EventType

with client.listen.v1.connect(model="nova-3", diarize=True, redact=["pii"]) as conn:
    conn.on(EventType.MESSAGE, lambda m: print(m))
    threading.Thread(target=conn.start_listening, daemon=True).start()
    for chunk in audio_chunks:
        conn.send_media(chunk)
    conn.send_finalize()
```

## Key parameters

`summarize`, `topics`, `intents`, `sentiment`, `detect_language`, `diarize`, `redact`, `custom_topic`, `custom_topic_mode`, `custom_intent`, `custom_intent_mode`, `detect_entities`, plus all the standard STT params (`model`, `language`, `encoding`, `sample_rate`, ...).

`redact` is typed as `Optional[str]` in the current generated SDK (`src/deepgram/listen/v1/media/client.py`). Pass a single redaction mode such as `"pci"`, `"pii"`, `"numbers"`, or `"phi"`. Multi-mode redaction at the transport level is supported by sending `redact` as a repeated query parameter — check `src/deepgram/types/listen_v1redact.py` for the current type and fall back to raw query-param construction (or multiple calls) if you need several modes. The earlier `Union[str, Sequence[str]]` override is no longer carried in `.fernignore`.

## API reference (layered)

1. **In-repo reference**: `reference.md` — "Listen V1 Media" (REST params include all analytics flags), "Listen V1 Connect" (WSS-supported subset).
2. **OpenAPI (REST)**: https://developers.deepgram.com/openapi.yaml
3. **AsyncAPI (WSS)**: https://developers.deepgram.com/asyncapi.yaml
4. **Context7**: library ID `/llmstxt/developers_deepgram_llms_txt`.
5. **Product docs**:
   - https://developers.deepgram.com/docs/stt-intelligence-feature-overview
   - https://developers.deepgram.com/docs/summarization
   - https://developers.deepgram.com/docs/topic-detection
   - https://developers.deepgram.com/docs/intent-recognition
   - https://developers.deepgram.com/docs/sentiment-analysis
   - https://developers.deepgram.com/docs/language-detection
   - https://developers.deepgram.com/docs/redaction
   - https://developers.deepgram.com/docs/diarization

## Gotchas

1. **`summarize` on `/v1/listen` accepts a boolean OR the string `"v2"`.** Use `"v2"` to pin the current summarization model; `True` also works (maps to the default model). `/v1/read` is the reverse — it accepts boolean only. If you need summarization on already-transcribed text, see `deepgram-python-text-intelligence`.
2. **Sentiment / topics / intents / summarize / detect_language are REST-only.** Don't pass them on WSS — they'll be ignored or rejected.
3. **English-only** for sentiment / topics / intents / summarize.
4. **Not all models support all overlays.** Flux / Base models have restrictions. Stick to `nova-3` unless you have a reason.
5. **Redaction values** are `pci`, `pii`, `phi`, `numbers`, etc. — not arbitrary strings.
6. **`custom_topic` / `custom_intent` need a mode** (`"extended"` or `"strict"`).
7. **Diarization is noisy on short / low-quality audio.** Expect speaker churn on <30s clips.

## Example files in this repo

- `examples/15-transcription-advanced-options.py` — smart_format, punctuate, diarize
- `tests/wire/test_listen_v1_media.py` — wire test covering intelligence params

## Related skills

- `deepgram-python-speech-to-text` — same endpoint, plain transcription
- `deepgram-python-text-intelligence` — same analytics, text input
- `deepgram-python-conversational-stt` — Flux for turn-taking
- `deepgram-python-voice-agent` — interactive assistants

## Central product skills

For cross-language Deepgram product knowledge — the consolidated API reference, documentation finder, focused runnable recipes, third-party integration examples, and MCP setup — install the central skills:

```bash
npx skills add deepgram/skills
```

This SDK ships language-idiomatic code skills; `deepgram/skills` ships cross-language product knowledge (see `api`, `docs`, `recipes`, `examples`, `starters`, `setup-mcp`).