Knowledge Base

Live-Docs Migration Assistant with Firecrawl and Qdrant

Scrape a framework's current docs with Firecrawl, index them in Qdrant, and run a LangGraph ReAct agent that answers migration questions with up-to-date, cited syntax instead of stale LLM memory.

What This Builds

This recipe builds an assistant that answers “how do I do X in the new version?” using a framework’s live documentation rather than the model’s training data. Firecrawl scrapes the docs site into clean markdown, the pages are embedded into Qdrant, and a LangGraph ReAct agent decides when to retrieve, then answers with current, cited syntax.

This directly targets the migration pain point: LLMs have a training cutoff, frameworks don’t. As the Firecrawl LangGraph tutorial puts it, when you need current syntax for a library’s latest version, an outdated model won’t help — so you give the agent live docs and it cites its sources.

Product Shape

This is a knowledge base feeding an agent, plus a one-time (or scheduled) ingestion workflow. The retrieval is agentic: a ReAct loop decides whether a question needs a docs lookup at all, which keeps simple questions cheap and grounds hard ones in real pages. Every answer cites the page it came from, so a developer can trust the snippet before pasting it into a migration.

The Stack

  • Firecrawl — crawls and scrapes the documentation site into clean, LLM-ready markdown.
  • Qdrant Cloud Free Tier — stores the doc embeddings for fast semantic retrieval.
  • LangChain / LangGraph — runs the ReAct agent that decides when to retrieve and synthesizes cited answers.
  • Jina AI Search Foundation — embeddings (and an optional reranker) for the doc chunks.
  • A reasoning model such as the DeepSeek Platform for answer generation.

Step-by-Step Outline

  1. Use Firecrawl to crawl the target docs site (the new framework version) and return clean markdown per page.
  2. Chunk the markdown, embed it with Jina embeddings, and upsert into a Qdrant collection with page-URL metadata.
  3. Build a LangGraph ReAct agent with a single search_docs tool that queries Qdrant.
  4. The agent decides per question whether to retrieve; on retrieval it pulls the top chunks and their source URLs.
  5. Synthesize an answer that includes the current syntax and cites the doc pages used.
  6. Schedule a periodic re-crawl so the index tracks docs updates; expose the agent as a chat endpoint or a migration helper CLI.

Why This Shape Works

Grounding answers in freshly scraped docs sidesteps the model’s training cutoff — the single biggest reason LLMs give wrong migration syntax. The ReAct pattern avoids retrieving on every turn, and citing source URLs makes each suggestion verifiable rather than a guess.

Source