Workflow

Durable RAG Answer Workflow with Inngest

Wrap a retrieval-augmented answer pipeline in an Inngest durable function so each step — query the vector DB, fetch context, generate with the LLM, save, notify — is checkpointed and automatically retried on failure, with built-in throttling for LLM rate limits.

What This Builds

A support RAG answer is several fallible network calls in a row: query the vector DB, pull the source text, call the LLM, persist the result, notify the client. If any one fails, you do not want to redo the whole thing or lose the run. This recipe wraps that pipeline in an Inngest durable function, where each step.run is checkpointed independently and retried automatically on failure — so a transient LLM timeout only retries the LLM step, not the retrieval that already succeeded.

This is the event-driven, durable-execution backbone for a support chatbot: an event like ai/answer.requested triggers the function, and Inngest guarantees the steps run reliably to completion.

Architecture

Event: answer.requested

Inngest durable function

step: query vector DB

step: read source content

step.invoke: LLM completion (throttled)

step: save answer to DB

step: notify client (websocket/push)

The Stack

  • Inngest for event-driven durable execution: steps, automatic retries, and flow control (throttling, concurrency, debounce) without managing queues or state.
  • An LLM provider such as OpenAI for the generation step.
  • A vector store such as Qdrant for the retrieval step.

Step-by-Step Outline

  1. Define the function and trigger. Create an Inngest function bound to an event (e.g. ai/summarize.content or ai/answer.requested).
  2. Query the vector DB. In a step.run('query-vectordb', ...), embed the question and fetch the top matches from your vector store. The step result is checkpointed.
  3. Fetch source content. In another step, retrieve the underlying transcript/article text for the matched chunks (e.g. from object storage or your DB).
  4. Generate. Use step.invoke to call a shared, throttled LLM-completion function, passing the question plus retrieved context. Routing LLM calls through one throttled function caps concurrent requests and respects provider rate limits.
  5. Persist and notify. Save the generated answer in a step.run, then push it to the client (websocket/notification) in a final step. Because each step is independently retried, the pipeline survives transient failures without duplicating work.

Why This Shape Works

Inngest’s own RAG example is exactly this: query a vector DB, retrieve a transcript, combine and summarize via the LLM, then save and notify — with steps guaranteeing automatic retries on failure. The durable-step model is the right fit for support answers because the failure modes (LLM timeouts, rate limits, flaky network) are precisely what step-level checkpointing and throttling are built to absorb. The same pattern applies on Trigger.dev if you prefer its TypeScript task model.

Source