RAG over podcasts: a clean transcript primitive for LLM pipelines

Building a retrieval-augmented generation pipeline that reasons over podcast content. How to skip the transcription step and start with Markdown that's already chunk-ready.

By Robert Tomko · Last updated May 2026

The typical RAG-over-podcasts pipeline

Most podcast RAG cookbooks (Haystack, LangChain, LlamaIndex) follow the same shape:

1. Locate and download audio
2. Transcribe with Whisper or AssemblyAI
3. Run diarization (Pyannote or built-in) and align with transcription
4. Identify speakers from context (LLM pass)
5. Chunk and embed
6. Store in a vector DB
7. Retrieve + generate at query time

Steps 1–4 are infrastructure work that has nothing to do with retrieval quality. They're table stakes you have to ship before you can start tuning the part that actually matters.

The Spoken version

# Step 1–4 collapse into one call
import requests

resp = requests.get(
    f"https://spoken.md/transcripts/{episode_id}",
    headers={"x-api-key": "pt_demo"},
)
markdown = resp.text  # Real speaker names, timestamps, paragraph breaks

# Step 5: chunk
from langchain_text_splitters import MarkdownTextSplitter
chunks = MarkdownTextSplitter(chunk_size=1000, chunk_overlap=150).split_text(markdown)

# Steps 6–7: embed, store, retrieve as normal

Speaker names come through as bold Markdown (**Andrew Huberman** (0:00)), which means chunks naturally carry attribution. When the retriever returns a passage, you know who said it without an extra metadata layer.

Why this matters for retrieval quality

• Speaker context survives chunking. "Speaker 0" doesn't help an LLM reason about authority or perspective. "Andrew Huberman" does.
• Markdown structure beats raw text. Standard Markdown splitters respect paragraph boundaries, which align with natural speaker turns.
• Timestamps as metadata. Each turn includes a timestamp you can attach to chunks for citation-style answers ("Andrew Huberman, around 0:45, says…").
• Idempotent fetches. Repeat fetches of the same episode don't charge again, so you can re-chunk during pipeline development without burning credits.

Reference pipeline

Below is the shape of a working podcast RAG pipeline using Spoken. Adapt to your stack of choice.

from langchain_text_splitters import MarkdownTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
import requests, re

EPISODE_IDS = ["...", "..."]  # IDs from /search

def fetch(eid):
    r = requests.get(f"https://spoken.md/transcripts/{eid}",
                     headers={"x-api-key": "YOUR_KEY"})
    r.raise_for_status()
    return r.text

splitter = MarkdownTextSplitter(chunk_size=1200, chunk_overlap=180)
docs = []
for eid in EPISODE_IDS:
    md = fetch(eid)
    for chunk in splitter.split_text(md):
        # Extract leading speaker for metadata
        speaker_match = re.search(r"\*\*([^*]+)\*\*", chunk)
        speaker = speaker_match.group(1) if speaker_match else None
        docs.append({"text": chunk, "metadata": {"episode": eid, "speaker": speaker}})

store = Chroma.from_texts(
    [d["text"] for d in docs],
    OpenAIEmbeddings(),
    metadatas=[d["metadata"] for d in docs],
)

Cost at podcast-RAG scale

A back-catalogue of 500 episodes at one hour each:

• Whisper API + Pyannote + LLM-naming pipeline: ~$0.36/hr Whisper × 500 = $180 plus diarization compute plus naming-LLM tokens — typically $250–$400 once infrastructure is included.
• AssemblyAI Universal + LLM-naming: ~$0.17/hr × 500 = $85 plus the naming pass — typically $120–$180 total.
• Spoken at the 2,000-pack rate: $0.08 × 500 = $40. No naming pass needed.

And re-fetches during pipeline iteration are free, so you can keep tweaking your chunker without re-paying for transcription.

FAQ