PickSkill
← Back

pminervini/deep-research-mcp

README.md
Rendered from GitHub raw
View raw ↗

Deep Research MCP

CI License: MIT Python MCP Claude Code OpenAI Gemini

A Python-based agent that integrates research providers with Claude Code through the Model Context Protocol (MCP). It supports OpenAI (Responses API with web search and code interpreter, or Chat Completions API for broad provider compatibility), Gemini Deep Research via the Interactions API, and the open-source Open Deep Research stack (based on smolagents).

Prerequisites

  • Python 3.11+
  • uv installed
  • One of:
    • OpenAI API access (Responses API models, e.g., o4-mini-deep-research-2025-06-26)
    • Gemini API access with the Interactions API / Deep Research agent enabled
    • Open Deep Research dependencies (installed via uv sync --extra open-deep-research)
  • Claude Code, or any other assistant supporting MCP integration

Installation

Recommended setup (resolves the latest compatible versions):

# Install runtime dependencies + project in editable mode
uv sync --upgrade
 
# Development tooling (pytest, black, pylint, mypy, pre-commit)
uv sync --upgrade --extra dev
 
# Enable the pre-commit hook so black runs automatically before each commit
uv run pre-commit install
 
# Optional docs tooling
uv sync --upgrade --extra docs
 
# Optional Open Deep Research provider dependencies
uv sync --upgrade --extra open-deep-research

Compatibility setup (pip-based):

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .

requirements.txt is intentionally unpinned (uses >=) and can be edited directly. uv.lock is not tracked in this repository.

Code Layout

  • src/deep_research_mcp/agent.py: orchestration layer; owns clarification, instruction building, callbacks, and delegates provider work to backends
  • src/deep_research_mcp/backends/: provider-specific implementations for OpenAI, Gemini, and Open Deep Research
  • src/deep_research_mcp/mcp_server.py: FastMCP server and tool entrypoints
  • src/deep_research_mcp/clarification.py: clarification agents, sessions, and enrichment flow
  • src/deep_research_mcp/prompts/: YAML prompt templates used by clarification and instruction building
  • tests/: pytest suite covering configuration, MCP integration, prompts, results, and clarification flows

Configuration

Configuration File

Create a ~/.deep_research file in your home directory using TOML format.

Library note: ResearchConfig.load() explicitly reads this file and applies environment variable overrides. ResearchConfig.from_env() reads environment variables only.

Common settings:

[research]                                  # Core Deep Research functionality
provider = "openai"                         # Available options: "openai", "gemini", "open-deep-research" -- defaults to "openai"
api_style = "responses"                     # Only applies to provider="openai"; use "chat_completions" for Perplexity, Groq, Ollama, etc.
model = "o4-mini-deep-research-2025-06-26"  # OpenAI: model identifier; Gemini: agent id; ODR: LiteLLM model identifier, e.g., openai/qwen/qwen3-coder-30b
api_key = "your-api-key"                    # API key, optional
base_url = "https://api.openai.com/v1"      # OpenAI: OpenAI-compatible endpoint; Gemini: https://generativelanguage.googleapis.com; ODR: LiteLLM-compatible endpoint
 
# Task behavior
timeout = 1800
poll_interval = 30
 
# Largely based on https://cookbook.openai.com/examples/deep_research_api/introduction_to_deep_research_api_agents
[clarification]                                       # Optional query clarification component
enable = true
triage_model = "gpt-5-mini"
clarifier_model = "gpt-5-mini"
instruction_builder_model = "gpt-5-mini"
api_key = "sk-your-api-key"             # Optional, overrides api_key
base_url = "https://api.openai.com/v1"  # Optional, overrides base_url
 
[logging]
level = "INFO"

OpenAI provider example:

[research]
provider = "openai"
model = "o4-mini-deep-research-2025-06-26"  # OpenAI model
api_key = "sk-..."                          # Defaults to OPENAI_API_KEY
base_url = "https://api.openai.com/v1"      # OpenAI-compatible endpoint
timeout = 1800
poll_interval = 30

Gemini Deep Research provider example:

[research]
provider = "gemini"
model = "deep-research-pro-preview-12-2025"     # Gemini Deep Research agent id
api_key = "AIza..."                             # Defaults to GEMINI_API_KEY or GOOGLE_API_KEY
base_url = "https://generativelanguage.googleapis.com"
timeout = 1800
poll_interval = 30

Perplexity (via Sonar Deep Research and Perplexity's OpenAI-compatible endpoint) provider example:

[research]
provider = "openai"
api_style = "chat_completions"              # Required for Perplexity (no Responses API)
model = "sonar-deep-research"               # Perplexity's Sonar Deep Research
api_key = "ppl-..."                         # Defaults to OPENAI_API_KEY
base_url = "https://api.perplexity.ai"      # Perplexity's OpenAI-compatible endpoint
timeout = 1800

Open Deep Research provider example:

[research]
provider = "open-deep-research"
model = "openai/qwen/qwen3-coder-30b"  # LiteLLM-compatible model id
base_url = "http://localhost:1234/v1"  # LiteLLM-compatible endpoint (local or remote)
api_key = ""                           # Optional if endpoint requires it
timeout = 1800

Ollama (local) provider example:

[research]
provider = "openai"
api_style = "chat_completions"
model = "llama3.1"                     # Any model available in your Ollama instance
base_url = "http://localhost:11434/v1" # Ollama's OpenAI-compatible endpoint
api_key = ""                           # Not required for local Ollama
timeout = 600

Generic OpenAI-compatible Chat Completions provider (Groq, Together AI, vLLM, etc.):

[research]
provider = "openai"
api_style = "chat_completions"
model = "your-model-name"
api_key = "your-api-key"
base_url = "https://api.your-provider.com/v1"
timeout = 600

Optional env variables for Open Deep Research tools:

  • SERPAPI_API_KEY or SERPER_API_KEY: enable Google-style search
  • HF_TOKEN: optional, logs into Hugging Face Hub for gated models

Claude Code Integration

  1. Configure MCP Server

Choose one of the transports below.

Option A: stdio (recommended when Claude Code should spawn the server itself)

If your provider credentials are already stored in ~/.deep_research, the minimal setup is:

claude mcp add deep-research -- uv run --directory /path/to/deep-research-mcp deep-research-mcp

If you want Claude Code to pass OPENAI_API_KEY through to the spawned MCP process explicitly, use:

claude mcp add -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  deep-research -- \
  uv run --directory /path/to/deep-research-mcp deep-research-mcp

Option B: HTTP (recommended when you want to run the server separately)

Start the server in one terminal:

OPENAI_API_KEY="$OPENAI_API_KEY" \
uv run --directory /path/to/deep-research-mcp \
  deep-research-mcp --transport http --host 127.0.0.1 --port 8080

Then add the HTTP MCP server in Claude Code:

claude mcp add --transport http deep-research-http http://127.0.0.1:8080/mcp

Replace /path/to/deep-research-mcp/ with the actual path to your cloned repository. The verified Streamable HTTP endpoint is http://127.0.0.1:8080/mcp.

For multi-hour research, raise Claude Code's tool timeout before launching the CLI and rely on incremental status polls:

export MCP_TOOL_TIMEOUT=14400000  # 4 hours
claude --mcp-config ./.mcp.json

Kick off work with deep_research or research_with_context, note the returned job ID, and call research_status to stream progress without letting any single tool call stagnate.

  1. Use in Claude Code:
    • The research tools will appear in Claude Code's tool palette
    • Simply ask Claude to "research [your topic]" and it will use the Deep Research agent
    • For clarified research, ask Claude to "research [topic] with clarification" to get follow-up questions

OpenAI Codex Integration

  1. Configure MCP Server

Choose one of the transports below.

Option A: stdio (recommended when Codex should spawn the server itself)

Add the MCP server configuration to your ~/.codex/config.toml file:

[mcp_servers.deep-research]
command = "uv"
args = ["run", "--directory", "/path/to/deep-research-mcp", "deep-research-mcp"]
# If your provider credentials live in shell env vars rather than ~/.deep_research,
# pass them through to the MCP subprocess explicitly:
env_vars = ["OPENAI_API_KEY"]
startup_timeout_ms = 30000  # 30 seconds for server startup
request_timeout_ms = 7200000  # 2 hours for long-running research tasks
# Alternatively, set tool_timeout_sec when using newer Codex clients
# tool_timeout_sec = 14400.0     # 4 hours for deep research runs

Replace /path/to/deep-research-mcp/ with the actual path to your cloned repository.

If your credentials are already configured in ~/.deep_research, env_vars is optional. It is required when you expect the spawned MCP server to inherit OPENAI_API_KEY from the parent shell.

Option B: HTTP (recommended when you want to run the server separately)

Start the server in one terminal:

OPENAI_API_KEY="$OPENAI_API_KEY" \
uv run --directory /path/to/deep-research-mcp \
  deep-research-mcp --transport http --host 127.0.0.1 --port 8080

Then add this to ~/.codex/config.toml:

[mcp_servers.deep-research-http]
url = "http://127.0.0.1:8080/mcp"
tool_timeout_sec = 14400.0

The verified Streamable HTTP endpoint is http://127.0.0.1:8080/mcp.

Important timeout configuration:

  • startup_timeout_ms: Time allowed for the MCP server to start (default: 30000ms / 30 seconds)
  • request_timeout_ms: Maximum time for research queries to complete (recommended: 7200000ms / 2 hours for comprehensive research)
  • tool_timeout_sec: Preferred for newer Codex clients; set this to a large value (e.g., 14400.0 for 4 hours) when you expect long-running research.
  • Kick off research once to capture the job ID, then poll research_status so each tool call remains short and avoids hitting client timeouts.

Without proper timeout configuration, long-running research queries may fail with "request timed out" errors.

  1. Use in OpenAI Codex:
    • The research tools will be available automatically when you start Codex
    • Ask Codex to "research [your topic]" and it will use the Deep Research MCP server
    • For clarified research, ask for "research [topic] with clarification"

Gemini CLI Integration

  1. Configure MCP Server

Add the MCP server using Gemini CLI's built-in command:

gemini mcp add deep-research -- uv run --directory /path/to/deep-research-mcp deep-research-mcp

Or manually add to your ~/.gemini/settings.json file:

{
  "mcpServers": {
    "deep-research": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/deep-research-mcp", "deep-research-mcp"],
      "env": {
        "RESEARCH_PROVIDER": "gemini",
        "GEMINI_API_KEY": "$GEMINI_API_KEY"
      }
    }
  }
}

Replace /path/to/deep-research-mcp/ with the actual path to your cloned repository.

  1. Use in Gemini CLI:
    • Start Gemini CLI with gemini
    • The research tools will be available automatically
    • Ask Gemini to "research [your topic]" and it will use the Deep Research MCP server
    • Use /mcp command to view server status and available tools

HTTP transport: If your Gemini environment supports MCP-over-HTTP, you may run the server with --transport http and configure Gemini with the server URL.

Usage

As a Standalone Python Module

import asyncio
from deep_research_mcp.agent import DeepResearchAgent
from deep_research_mcp.config import ResearchConfig
 
async def main():
    # Initialize configuration
    config = ResearchConfig.load()
    
    # Create agent
    agent = DeepResearchAgent(config)
    
    # Perform research
    result = await agent.research(
        query="What are the latest advances in quantum computing?",
        system_prompt="Focus on practical applications and recent breakthroughs"
    )
    
    # Print results
    print(f"Report: {result.final_report}")
    print(f"Citations: {result.citations}")
    print(f"Research steps: {result.reasoning_steps}")
    print(f"Execution time: {result.execution_time:.2f}s")
 
# Run the research
asyncio.run(main())

As an MCP Server

Two transports are supported: stdio (default) and HTTP streaming.

# 1) stdio (default) — for editors/CLIs that spawn a local process
uv run deep-research-mcp
 
# 2) HTTP streaming — start a local HTTP MCP server
uv run deep-research-mcp --transport http --host 127.0.0.1 --port 8080

Notes:

  • HTTP mode uses streaming responses provided by FastMCP. The tools in this server return their full results when a research task completes; streaming is still beneficial for compatible clients and for future incremental outputs.
  • The verified Streamable HTTP endpoint is /mcp, so the default local URL is http://127.0.0.1:8080/mcp.
  • If you start the server outside the client and rely on environment variables for credentials, export them before launching the server process. If you use stdio and let the client spawn the server, make sure the client passes the required env vars through.

Example Queries

# Basic research query
result = await agent.research("Explain the transformer architecture in AI")
 
# Research with code analysis
result = await agent.research(
    query="Analyze global temperature trends over the last 50 years",
    include_code_interpreter=True
)
 
# Custom system instructions
result = await agent.research(
    query="Review the safety considerations for AGI development",
    system_prompt="""
    Provide a balanced analysis including:
    - Technical challenges
    - Current safety research
    - Regulatory approaches
    - Industry perspectives
    Include specific examples and data where available.
    """
)
 
# With clarification (requires ENABLE_CLARIFICATION=true)
clarification_result = agent.start_clarification("quantum computing applications")
if clarification_result.get("needs_clarification"):
    # Answer questions programmatically or present to user
    answers = ["Hardware applications", "Last 5 years", "Commercial products"]
    agent.add_clarification_answers(clarification_result["session_id"], answers)
    enriched_query = agent.get_enriched_query(clarification_result["session_id"])
    result = await agent.research(enriched_query)

Clarification Features

The agent includes an optional clarification system to improve research quality through follow-up questions.

Configuration

Enable clarification in your ~/.deep_research file:

[clarification]
enable_clarification = true
triage_model = "gpt-5-mini"                                    # Optional, defaults to gpt-5-mini
clarifier_model = "gpt-5-mini"                                 # Optional, defaults to gpt-5-mini
instruction_builder_model = "gpt-5-mini"                       # Optional, defaults to gpt-5-mini
clarification_api_key = "sk-your-clarification-api-key-here"   # Optional custom API key for clarification models
clarification_base_url = "https://custom-api.example.com/v1"   # Optional custom endpoint for clarification models

Clarification and instruction-building remain OpenAI-compatible chat flows. If your main research provider is gemini or open-deep-research, set clarification_api_key / clarification_base_url explicitly, or provide OPENAI_API_KEY / OPENAI_BASE_URL in the environment for those helper models.

Usage Flow

  1. Start Clarification:

    result = agent.start_clarification("your research query")
  2. Check if Questions are Needed:

    if result.get("needs_clarification"):
        questions = result["questions"]
        session_id = result["session_id"]
  3. Provide Answers:

    answers = ["answer1", "answer2", "answer3"]
    agent.add_clarification_answers(session_id, answers)
  4. Get Enriched Query:

    enriched_query = agent.get_enriched_query(session_id)
    final_result = await agent.research(enriched_query)

Integration with AI Assistants

When using with AI Assistants via MCP tools:

  1. Request Clarification: Use deep_research() with request_clarification=True
  2. Answer Questions: The AI Assistant will present questions to you
  3. Deep Research: The AI Asssitant will automatically use research_with_context() with your answers

API Reference

DeepResearchAgent

The main class for performing research operations.

Methods

  • research(query, system_prompt=None, include_code_interpreter=True, callback_url=None)

    • Performs deep research on a query
    • callback_url: optional webhook notified when research completes
    • Returns: Dictionary with final report, citations, and metadata
  • get_task_status(task_id)

    • Check the status of a research task
    • Returns: Task status information
  • start_clarification(query)

    • Analyze query and generate clarifying questions if needed
    • Returns: Dictionary with questions and session ID
  • add_clarification_answers(session_id, answers)

    • Add answers to clarification questions
    • Returns: Session status information
  • get_enriched_query(session_id)

    • Generate enriched query from clarification session
    • Returns: Enhanced query string

ResearchConfig

Configuration class for the research agent.

Parameters

  • provider: Research provider (openai, gemini, or open-deep-research; default: openai)
  • api_style: API style for the openai provider (responses or chat_completions; default: responses). Ignored for gemini and open-deep-research.
  • model: Model identifier
    • OpenAI: Responses model (e.g., gpt-5-mini)
    • Gemini: Deep Research agent id (for example deep-research-pro-preview-12-2025)
    • Open Deep Research: LiteLLM model id (e.g., openai/qwen/qwen3-coder-30b)
  • api_key: API key for the configured endpoint (optional). Defaults to env OPENAI_API_KEY for openai, GEMINI_API_KEY / GOOGLE_API_KEY for gemini.
  • base_url: Provider API base URL (optional). Defaults to https://api.openai.com/v1 for openai, https://generativelanguage.googleapis.com for gemini, and http://localhost:1234/v1 for open-deep-research.
  • timeout: Maximum time for research in seconds (default: 1800)
  • poll_interval: Polling interval in seconds (default: 30)
  • enable_clarification: Enable clarifying questions (default: False)
  • triage_model: Model for query analysis (default: gpt-5-mini)
  • clarifier_model: Model for query enrichment (default: gpt-5-mini)
  • clarification_api_key: Custom API key for clarification models (optional; defaults to the main OpenAI credentials when provider=openai, otherwise falls back to env OPENAI_API_KEY if present)
  • clarification_base_url: Custom OpenAI-compatible endpoint for clarification models (optional; defaults to the main OpenAI endpoint when provider=openai, otherwise falls back to env OPENAI_BASE_URL if present)

Development

Running Tests

# Install dev dependencies
uv sync --extra dev
 
# Run all tests
uv run pytest -v
 
# Run with coverage
uv run pytest --cov=deep_research_mcp tests/
 
# Run specific test file
uv run pytest tests/test_agents.py

Lint, Format, Type Check

uv run black .
uv run pylint src/deep_research_mcp tests
uv run mypy src/deep_research_mcp