RAG over podcasts: a clean transcript primitive for LLM pipelines

The typical RAG-over-podcasts pipeline

Most podcast RAG cookbooks (Haystack, LangChain, LlamaIndex) follow the same shape:

1. Locate and download audio
2. Transcribe with Whisper or AssemblyAI
3. Run diarization (Pyannote or built-in) and align with transcription
4. Identify speakers from context (LLM pass)
5. Chunk and embed
6. Store in a vector DB
7. Retrieve + generate at query time

Steps 1–4 are infrastructure work that has nothing to do with retrieval quality, and you have to ship them before you can tune the part that does.

The Spoken version

# Step 1–4 collapse into one call
import requests

resp = requests.get(
    f"https://spoken.md/transcripts/{episode_id}",
    headers={"x-api-key": "pt_demo"},
)
markdown = resp.text  # Real speaker names, timestamps, paragraph breaks

# Step 5: chunk
from langchain_text_splitters import MarkdownTextSplitter
chunks = MarkdownTextSplitter(chunk_size=1000, chunk_overlap=150).split_text(markdown)

# Steps 6–7: embed, store, retrieve as normal

Speaker names come through as bold Markdown (**Andrew Huberman** (0:00)), which means chunks naturally carry attribution. When the retriever returns a passage, you know who said it without an extra metadata layer.

Why this matters for retrieval quality

• Speaker context survives chunking. "Speaker 0" doesn't help an LLM reason about authority or perspective. "Andrew Huberman" does.
• Markdown structure beats raw text. Standard Markdown splitters respect paragraph boundaries, which align with natural speaker turns.
• Timestamps as metadata. Each turn includes a timestamp you can attach to chunks for citation-style answers ("Andrew Huberman, around 0:45, says…").
• Idempotent fetches. Repeat fetches of the same episode don't charge again, so you can re-chunk during pipeline development without burning credits.

Reference pipeline

Below is the shape of a working podcast RAG pipeline using Spoken. Adapt to your stack of choice.

from langchain_text_splitters import MarkdownTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
import requests, re

EPISODE_IDS = ["...", "..."]  # IDs from /search

def fetch(eid):
    r = requests.get(f"https://spoken.md/transcripts/{eid}",
                     headers={"x-api-key": "YOUR_KEY"})
    r.raise_for_status()
    return r.text

splitter = MarkdownTextSplitter(chunk_size=1200, chunk_overlap=180)
docs = []
for eid in EPISODE_IDS:
    md = fetch(eid)
    for chunk in splitter.split_text(md):
        # Extract leading speaker for metadata
        speaker_match = re.search(r"\*\*([^*]+)\*\*", chunk)
        speaker = speaker_match.group(1) if speaker_match else None
        docs.append({"text": chunk, "metadata": {"episode": eid, "speaker": speaker}})

store = Chroma.from_texts(
    [d["text"] for d in docs],
    OpenAIEmbeddings(),
    metadatas=[d["metadata"] for d in docs],
)

Cost at podcast-RAG scale

A back-catalogue of 500 episodes at one hour each:

• Whisper API + Pyannote + LLM-naming pipeline: ~$0.36/hr Whisper × 500 = $180, plus diarization compute and naming-LLM tokens, typically $250–$400 once infrastructure is included.
• AssemblyAI Universal + LLM-naming: ~$0.17/hr × 500 = $85, plus the naming pass, typically $120–$180 total.
• Spoken at the 2,000-pack rate: $0.08 × 500 = $40. No naming pass needed.

Re-fetches are free, so you can keep changing your chunker without paying again.

FAQ

Thousands of transcripts fetched by developers building summarizers, RAG pipelines, and podcast tools