{"owner":"assafelovic","repo":"gpt-researcher","hasSkills":true,"totalSkillsCount":19,"totalTokensCount":24513,"categories":["anthropic-skill","plugin-manifest"],"hasMcp":false,"mcpConfig":null,"found":[".claude/SKILL.md",".claude/references/adding-features.md",".claude/references/advanced-patterns.md",".claude/references/api-reference.md",".claude/references/architecture.md",".claude/references/components.md",".claude/references/config-reference.md",".claude/references/deep-research.md",".claude/references/flows.md",".claude/references/mcp.md",".claude/references/multi-agents.md",".claude/references/prompts.md",".claude/references/retrievers.md",".codex-plugin/plugin.json","evals/README.md","evals/context_filter/README.md","evals/quality_eval/README.md","evals/simple_evals/logs/README.md","skills/gpt-researcher/SKILL.md"],"skills":{".claude/SKILL.md":"---\nname: gpt-researcher\ndescription: GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines.\n---\n\n# GPT Researcher Development Skill\n\nGPT Researcher is an LLM-based autonomous agent using a planner-executor-publisher pattern with parallelized agent work for speed and reliability.\n\n## Quick Start\n\n### Basic Python Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\", # or detailed_report, deep, outline_report\n report_source=\"web\", # or local, hybrid\n )\n await researcher.conduct_research()\n report = await researcher.write_report()\n print(report)\n\nasyncio.run(main())\n```\n\n### Run Servers\n\n```bash\n# Backend\npython -m uvicorn backend.server.server:app --reload --port 8000\n\n# Frontend\ncd frontend/nextjs && npm install && npm run dev\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes |\n|------|--------------|-------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| Search engines | `gpt_researcher/retrievers/` | Various retrievers |\n\n---\n\n## Architecture Overview\n\n```\nUser Query → GPTResearcher.__init__()\n │\n ▼\n choose_agent() → (agent_type, role_prompt)\n │\n ▼\n ResearchConductor.conduct_research()\n ├── plan_research() → sub_queries\n ├── For each sub_query:\n │ └── _process_sub_query() → context\n └── Aggregate contexts\n │\n ▼\n [Optional] ImageGenerator.plan_and_generate_images()\n │\n ▼\n ReportGenerator.write_report() → Markdown report\n```\n\n**For detailed architecture diagrams**: See [references/architecture.md](references/architecture.md)\n\n---\n\n## Core Patterns\n\n### Adding a New Feature (8-Step Pattern)\n\n1. **Config** → Add to `gpt_researcher/config/variables/default.py`\n2. **Provider** → Create in `gpt_researcher/llm_provider/my_feature/`\n3. **Skill** → Create in `gpt_researcher/skills/my_feature.py`\n4. **Agent** → Integrate in `gpt_researcher/agent.py`\n5. **Prompts** → Update `gpt_researcher/prompts.py`\n6. **WebSocket** → Events via `stream_output()`\n7. **Frontend** → Handle events in `useWebSocket.ts`\n8. **Docs** → Create `docs/docs/gpt-researcher/gptr/my_feature.md`\n\n**For complete feature addition guide with Image Generation case study**: See [references/adding-features.md](references/adding-features.md)\n\n### Adding a New Retriever\n\n```python\n# 1. Create: gpt_researcher/retrievers/my_retriever/my_retriever.py\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n \n async def search(self, max_results: int = 10) -> list[dict]:\n # Return: [{\"title\": str, \"href\": str, \"body\": str}]\n pass\n\n# 2. Register in gpt_researcher/actions/retriever.py\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n\n# 3. Export in gpt_researcher/retrievers/__init__.py\n```\n\n**For complete retriever documentation**: See [references/retrievers.md](references/retrievers.md)\n\n---\n\n## Configuration\n\nConfig keys are **lowercased** when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\nPriority: Environment Variables → JSON Config File → Default Values\n\n**For complete configuration reference**: See [references/config-reference.md](references/config-reference.md)\n\n---\n\n## Common Integration Points\n\n### WebSocket Streaming\n\n```python\nclass WebSocketHandler:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=WebSocketHandler())\n```\n\n### MCP Data Sources\n\n```python\nresearcher = GPTResearcher(\n query=\"Open source AI projects\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\", # or \"fast\", \"disabled\"\n)\n```\n\n**For MCP integration details**: See [references/mcp.md](references/mcp.md)\n\n### Deep Research Mode\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers recursive tree-like exploration\n)\n```\n\n**For deep research configuration**: See [references/deep-research.md](references/deep-research.md)\n\n---\n\n## Error Handling\n\nAlways use graceful degradation in skills:\n\n```python\nasync def execute(self, ...):\n if not self.is_enabled():\n return [] # Don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n await stream_output(\"logs\", \"error\", f\"⚠️ {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n---\n\n## Critical Gotchas\n\n| ❌ Mistake | ✅ Correct |\n|-----------|-----------|\n| `config.MY_VAR` | `config.my_var` (lowercased) |\n| Editing pip-installed package | `pip install -e .` |\n| Forgetting async/await | All research methods are async |\n| `websocket.send_json()` on None | Check `if websocket:` first |\n| Not registering retriever | Add to `retriever.py` match statement |\n\n---\n\n## Reference Documentation\n\n| Topic | File |\n|-------|------|\n| System architecture & diagrams | [references/architecture.md](references/architecture.md) |\n| Core components & signatures | [references/components.md](references/components.md) |\n| Research flow & data flow | [references/flows.md](references/flows.md) |\n| Prompt system | [references/prompts.md](references/prompts.md) |\n| Retriever system | [references/retrievers.md](references/retrievers.md) |\n| MCP integration | [references/mcp.md](references/mcp.md) |\n| Deep research mode | [references/deep-research.md](references/deep-research.md) |\n| Multi-agent system | [references/multi-agents.md](references/multi-agents.md) |\n| Adding features guide | [references/adding-features.md](references/adding-features.md) |\n| Advanced patterns | [references/advanced-patterns.md](references/advanced-patterns.md) |\n| REST & WebSocket API | [references/api-reference.md](references/api-reference.md) |\n| Configuration variables | [references/config-reference.md](references/config-reference.md) |\n",".claude/references/adding-features.md":"# Adding Features Guide\n\n## Table of Contents\n- [The 8-Step Pattern](#the-8-step-pattern)\n- [Image Generation Case Study](#image-generation-case-study)\n- [Testing New Features](#testing-new-features)\n\n---\n\n## The 8-Step Pattern\n\n```\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│1.CONFIG│ → │2.PROVIDER│ → │3.SKILL │ → │4.AGENT │\n└────────┘ └────────┘ └────────┘ └────────┘\n ↓ ↓ ↓ ↓\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│5.PROMPTS│ → │6.WEBSOCKET│→ │7.FRONTEND│→ │8.DOCS │\n└────────┘ └────────┘ └────────┘ └────────┘\n```\n\n### Step 1: Add Configuration\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n \"MY_FEATURE_ENABLED\": False,\n \"MY_FEATURE_MODEL\": \"model-name\",\n \"MY_FEATURE_MAX_ITEMS\": 3,\n}\n```\n\n**File:** `gpt_researcher/config/variables/base.py`\n\n```python\nclass BaseConfig(TypedDict):\n \"MY_FEATURE_ENABLED\": bool\n \"MY_FEATURE_MODEL\": Union[str, None]\n \"MY_FEATURE_MAX_ITEMS\": int\n```\n\n### Step 2: Create Provider\n\n**File:** `gpt_researcher/llm_provider/my_feature/my_provider.py`\n\n```python\nclass MyFeatureProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"MY_API_KEY\")\n self.model = model\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n async def execute(self, input_data: str) -> Dict[str, Any]:\n # API implementation\n pass\n```\n\nExport in `gpt_researcher/llm_provider/__init__.py`.\n\n### Step 3: Create Skill\n\n**File:** `gpt_researcher/skills/my_feature.py`\n\n```python\nclass MyFeatureSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.provider = MyFeatureProvider(...)\n \n def is_enabled(self) -> bool:\n return getattr(self.config, 'my_feature_enabled', False) and self.provider.is_enabled()\n \n async def execute(self, context: str, query: str) -> List[Dict]:\n if not self.is_enabled():\n return []\n \n await stream_output(\"logs\", \"my_feature_start\", \"🚀 Starting...\", self.researcher.websocket)\n results = await self.provider.execute(context)\n await stream_output(\"logs\", \"my_feature_complete\", \"✅ Done\", self.researcher.websocket)\n \n return results\n```\n\nExport in `gpt_researcher/skills/__init__.py`.\n\n### Step 4: Integrate into Agent\n\n**File:** `gpt_researcher/agent.py`\n\n```python\ndef __init__(self, ...):\n if self.cfg.my_feature_enabled:\n from gpt_researcher.skills import MyFeatureSkill\n self.my_feature = MyFeatureSkill(self)\n else:\n self.my_feature = None\n self.my_feature_results = []\n\nasync def conduct_research(self, ...):\n # ... existing ...\n if self.my_feature and self.my_feature.is_enabled():\n self.my_feature_results = await self.my_feature.execute(self.context, self.query)\n```\n\n### Step 5: Update Prompts\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_my_feature_prompt(context: str, query: str) -> str:\n return f\"\"\"...\"\"\"\n```\n\n### Step 6: WebSocket Events\n\nAlready handled via `stream_output()` in skill.\n\n### Step 7: Frontend (if needed)\n\n**File:** `frontend/nextjs/hooks/useWebSocket.ts`\n\n```typescript\nif (data.content === 'my_feature_start') {\n setStatus('processing');\n}\n```\n\n### Step 8: Documentation\n\nCreate `docs/docs/gpt-researcher/gptr/my_feature.md`.\n\n---\n\n## Image Generation Case Study\n\nThis section shows the **actual implementation** of the Image Generation feature as a reference.\n\n### 1. Configuration Added\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n # ... existing ...\n \"IMAGE_GENERATION_MODEL\": \"models/gemini-2.5-flash-image\",\n \"IMAGE_GENERATION_MAX_IMAGES\": 3,\n \"IMAGE_GENERATION_ENABLED\": False,\n \"IMAGE_GENERATION_STYLE\": \"dark\", # dark, light, auto\n}\n```\n\n### 2. Provider Created\n\n**File:** `gpt_researcher/llm_provider/image/image_generator.py`\n\n```python\nclass ImageGeneratorProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"GOOGLE_API_KEY\")\n self.model = model or \"models/gemini-2.5-flash-image\"\n self._client = None\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n def _build_enhanced_prompt(self, prompt: str, context: str = \"\", style: str = \"dark\") -> str:\n \"\"\"Add styling instructions to prompt.\"\"\"\n if style == \"dark\":\n style_instructions = \"\"\"\n Style: Dark mode professional infographic\n - Background: Dark (#0d1117)\n - Accents: Teal/cyan (#14b8a6)\n - Clean, modern, minimalist\n \"\"\"\n # ... handle light, auto\n return f\"{style_instructions}\\n\\nCreate: {prompt}\\n\\nContext: {context}\"\n \n async def generate_image(\n self,\n prompt: str,\n context: str = \"\",\n research_id: str = \"\",\n style: str = \"dark\",\n ) -> List[Dict[str, Any]]:\n \"\"\"Generate image using Gemini.\"\"\"\n full_prompt = self._build_enhanced_prompt(prompt, context, style)\n \n # Call Gemini API\n response = await self._generate_with_gemini(full_prompt, output_path, ...)\n \n return [{\"url\": f\"/outputs/images/{research_id}/img_{hash}.png\", ...}]\n```\n\n### 3. Skill Created\n\n**File:** `gpt_researcher/skills/image_generator.py`\n\n```python\nclass ImageGenerator:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.image_provider = ImageGeneratorProvider(\n api_key=os.getenv(\"GOOGLE_API_KEY\"),\n model=getattr(self.config, 'image_generation_model', None),\n )\n self.max_images = getattr(self.config, 'image_generation_max_images', 3)\n self.style = getattr(self.config, 'image_generation_style', 'dark')\n \n def is_enabled(self) -> bool:\n enabled = getattr(self.config, 'image_generation_enabled', False)\n return enabled and self.image_provider.is_enabled()\n \n async def plan_and_generate_images(\n self,\n research_context: str,\n research_query: str,\n research_id: str,\n websocket: Any,\n ) -> List[Dict[str, Any]]:\n \"\"\"\n 1. Use LLM to identify visual concepts from context\n 2. Generate images in parallel\n 3. Return list of image metadata\n \"\"\"\n # Stream progress\n await stream_output(\"logs\", \"image_planning\", \"🎨 Planning images...\", websocket)\n \n # LLM identifies concepts\n concepts = await self._plan_image_concepts(research_context, research_query)\n \n # Generate images in parallel\n generated_images = []\n for i, concept in enumerate(concepts[:self.max_images]):\n await stream_output(\"logs\", \"image_generating\", \n f\"🖼️ Generating image {i+1}/{len(concepts)}...\", websocket)\n \n images = await self.image_provider.generate_image(\n prompt=concept[\"prompt\"],\n context=concept.get(\"context\", \"\"),\n research_id=research_id,\n style=self.style,\n )\n generated_images.extend(images)\n \n await stream_output(\"logs\", \"images_ready\", \n f\"✅ Generated {len(generated_images)} images\", websocket)\n \n return generated_images\n```\n\n### 4. Agent Integration\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nclass GPTResearcher:\n def __init__(self, ...):\n # ... existing init ...\n \n # Initialize image generator if enabled\n if self.cfg.image_generation_enabled:\n from gpt_researcher.skills import ImageGenerator\n self.image_generator = ImageGenerator(self)\n else:\n self.image_generator = None\n \n self.available_images: List[Dict[str, Any]] = []\n self.research_id = self._generate_research_id(query)\n \n async def conduct_research(self, on_progress=None):\n # ... existing research ...\n \n self.context = await self.research_conductor.conduct_research()\n \n # Pre-generate images after research, before report writing\n if self.cfg.image_generation_enabled and self.image_generator and self.image_generator.is_enabled():\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n \n async def write_report(self, ...):\n report = await self.report_generator.write_report(\n # ... existing params ...\n available_images=self.available_images, # Pass to report writer\n )\n return report\n```\n\n### 5. Prompt Updated\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_report_prompt(..., available_images: List[Dict[str, Any]] = []):\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title', 'Untitled')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES - Embed where relevant using ![Title](URL):\n{image_list}\n\"\"\"\n \n return f\"\"\"...(existing prompt)...\n{image_instruction}\n\"\"\"\n```\n\n---\n\n## Testing New Features\n\n```python\n# tests/test_my_feature.py\nimport pytest\nfrom gpt_researcher import GPTResearcher\n\n@pytest.mark.asyncio\nasync def test_my_feature_disabled():\n \"\"\"Test that feature is skipped when disabled.\"\"\"\n researcher = GPTResearcher(query=\"test\")\n # MY_FEATURE_ENABLED defaults to False\n assert researcher.my_feature is None\n\n@pytest.mark.asyncio\nasync def test_my_feature_enabled(monkeypatch):\n \"\"\"Test feature execution when enabled.\"\"\"\n monkeypatch.setenv(\"MY_FEATURE_ENABLED\", \"true\")\n monkeypatch.setenv(\"MY_API_KEY\", \"test-key\")\n \n researcher = GPTResearcher(query=\"test\")\n assert researcher.my_feature is not None\n assert researcher.my_feature.is_enabled()\n```\n\n### Running Tests\n\n```bash\n# All tests\npython -m pytest tests/\n\n# Specific test\npython -m pytest tests/test_my_feature.py -v\n\n# With coverage\npython -m pytest tests/ --cov=gpt_researcher\n```\n",".claude/references/advanced-patterns.md":"# Advanced Patterns Reference\n\n## Table of Contents\n- [Custom Callbacks](#custom-callbacks)\n- [Custom WebSocket Handler](#custom-websocket-handler)\n- [LangChain Integration](#langchain-integration)\n- [Search Restrictions](#search-restrictions)\n- [Error Handling Patterns](#error-handling-patterns)\n\n---\n\n## Custom Callbacks\n\n```python\ndef cost_callback(cost: float):\n print(f\"API call cost: ${cost}\")\n\nresearcher = GPTResearcher(query=\"...\")\nresearcher.add_costs = cost_callback # Override cost tracking\n```\n\n---\n\n## Custom WebSocket Handler\n\n```python\nclass CustomWebSocket:\n def __init__(self):\n self.messages = []\n \n async def send_json(self, data):\n self.messages.append(data)\n if data['type'] == 'logs':\n print(f\"Progress: {data['output']}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=CustomWebSocket())\n```\n\n---\n\n## LangChain Integration\n\n### Using with LangChain Documents\n\n```python\nfrom langchain.document_loaders import DirectoryLoader\n\nloader = DirectoryLoader('./docs', glob=\"**/*.md\")\ndocuments = loader.load()\n\nresearcher = GPTResearcher(\n query=\"Summarize the documentation\",\n report_source=\"langchain_documents\",\n documents=documents,\n)\n```\n\n### Using with Vector Store\n\n```python\nfrom langchain.vectorstores import Chroma\n\nvectorstore = Chroma.from_documents(documents, embeddings)\n\nresearcher = GPTResearcher(\n query=\"Find relevant information\",\n report_source=\"langchain_vectorstore\",\n vector_store=vectorstore,\n vector_store_filter={\"source\": \"docs\"},\n)\n```\n\n---\n\n## Search Restrictions\n\n### Restricting Search Domains\n\n```python\nresearcher = GPTResearcher(\n query=\"Company news\",\n query_domains=[\"reuters.com\", \"bloomberg.com\", \"wsj.com\"],\n)\n```\n\n### Using Specific Source URLs\n\n```python\nresearcher = GPTResearcher(\n query=\"Analyze these articles\",\n source_urls=[\n \"https://example.com/article1\",\n \"https://example.com/article2\",\n ],\n complement_source_urls=True, # Also do web search\n)\n```\n\n---\n\n## Error Handling Patterns\n\n### Graceful Degradation\n\n```python\n# In skills, always check is_enabled()\nasync def execute(self, ...):\n if not self.is_enabled():\n logger.warning(\"Feature not enabled, skipping\")\n return [] # Return empty, don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n logger.error(f\"Feature error: {e}\")\n await stream_output(\"logs\", \"feature_error\", f\"⚠️ Error: {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n### API Rate Limiting\n\n```python\n# Providers should handle rate limits\nasync def execute(self, ...):\n try:\n return await self._call_api(...)\n except RateLimitError as e:\n logger.warning(f\"Rate limited, waiting...\")\n await asyncio.sleep(60)\n return await self._call_api(...) # Retry\n```\n\n### WebSocket None Check\n\n```python\n# Always check websocket before sending\nif self.researcher.websocket:\n await stream_output(\"logs\", \"event\", \"message\", self.researcher.websocket)\n```\n",".claude/references/api-reference.md":"# API Reference\n\n## Table of Contents\n- [REST API](#rest-api)\n- [WebSocket API](#websocket-api)\n- [Python Client](#python-client)\n- [Output Files](#output-files)\n\n---\n\n## REST API\n\nBase URL: `http://localhost:8000`\n\n### Generate Report\n\n**POST `/report/`**\n\n```json\n{\n \"task\": \"What are the latest AI developments?\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"query_domains\": [],\n \"generate_in_background\": false\n}\n```\n\n**Response:**\n\n```json\n{\n \"research_id\": \"task_1234567890_query\",\n \"research_information\": {\n \"source_urls\": [\"...\"],\n \"research_costs\": 0.05,\n \"visited_urls\": [\"...\"],\n \"research_images\": [\"...\"]\n },\n \"report\": \"# Research Report\\n\\n...\",\n \"docx_path\": \"outputs/task_123.docx\",\n \"pdf_path\": \"outputs/task_123.pdf\"\n}\n```\n\n### Chat with Report\n\n**POST `/api/chat`**\n\n```json\n{\n \"report\": \"The full report text...\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"What are the key findings?\"}\n ]\n}\n```\n\n### Report Management\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/api/reports` | List all reports |\n| GET | `/api/reports/{id}` | Get single report |\n| POST | `/api/reports` | Create/update report |\n| PUT | `/api/reports/{id}` | Update report |\n| DELETE | `/api/reports/{id}` | Delete report |\n\n### File Operations\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/files/` | List uploaded documents |\n| POST | `/upload/` | Upload document |\n| DELETE | `/files/{filename}` | Delete file |\n| GET | `/outputs/{filename}` | Get output file |\n\n---\n\n## WebSocket API\n\n**Endpoint:** `ws://localhost:8000/ws`\n\n### Send Research Request\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"mcp_enabled\": false,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": []\n}\n```\n\n### Message Types (Server → Client)\n\n| Type | Content | Description |\n|------|---------|-------------|\n| `logs` | `starting_research` | Research initiated |\n| `logs` | `planning_research` | Generating sub-queries |\n| `logs` | `running_subquery_research` | Researching sub-query |\n| `logs` | `research_step_finalized` | Research complete |\n| `logs` | `agent_generated` | Agent role selected |\n| `logs` | `scraping_urls` | Scraping web pages |\n| `logs` | `mcp_optimization` | MCP processing |\n| `logs` | `image_planning` | Planning images |\n| `logs` | `images_ready` | Images generated |\n| `report` | - | Streaming report chunks |\n| `report_complete` | - | Final complete report |\n| `path` | `pdf`, `docx`, `md` | Output file paths |\n| `error` | - | Error messages |\n| `human_feedback` | `request` | Request user input |\n\n### Message Format\n\n```json\n{\n \"type\": \"logs\",\n \"content\": \"starting_research\",\n \"output\": \"🔍 Starting the research task...\",\n \"metadata\": null\n}\n```\n\n### Frontend Handler Example\n\n```typescript\nws.onmessage = (event) => {\n const data = JSON.parse(event.data);\n \n switch (data.type) {\n case 'logs':\n setLogs(prev => [...prev, data]);\n break;\n case 'report':\n setAnswer(prev => prev + data.output);\n break;\n case 'report_complete':\n setAnswer(data.output);\n break;\n case 'path':\n setPaths(prev => ({...prev, [data.content]: data.output}));\n break;\n case 'error':\n setError(data.output);\n break;\n }\n};\n```\n\n---\n\n## Python Client\n\n### Basic Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\",\n )\n \n await researcher.conduct_research()\n report = await researcher.write_report()\n \n print(f\"Report: {report}\")\n print(f\"Costs: ${researcher.get_costs()}\")\n\nasyncio.run(main())\n```\n\n### With MCP\n\n```python\nresearcher = GPTResearcher(\n query=\"Research topic\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\",\n)\n```\n\n### With WebSocket Streaming\n\n```python\nclass MockWebSocket:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(\n query=\"Research topic\",\n websocket=MockWebSocket(),\n)\n```\n\n### GPTResearcher Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `query` | str | required | Research question |\n| `report_type` | str | `research_report` | Type of report |\n| `report_source` | str | `web` | Data source |\n| `tone` | Tone | `Objective` | Writing tone |\n| `source_urls` | list | `[]` | Specific URLs to research |\n| `document_urls` | list | `[]` | Document URLs |\n| `query_domains` | list | `[]` | Restrict to domains |\n| `config_path` | str | None | Path to JSON config |\n| `websocket` | WebSocket | None | For streaming |\n| `mcp_configs` | list | `[]` | MCP server configs |\n| `mcp_strategy` | str | `fast` | MCP strategy |\n| `verbose` | bool | `True` | Verbose output |\n\n---\n\n## Output Files\n\n```\noutputs/\n├── task_{timestamp}_{query}.md\n├── task_{timestamp}_{query}.pdf\n├── task_{timestamp}_{query}.docx\n└── images/\n └── {research_id}/\n └── img_{hash}_{index}.png\n```\n\n---\n\n## Error Codes\n\n| Code | Description |\n|------|-------------|\n| 400 | Bad Request - Invalid parameters |\n| 404 | Not Found - Report not found |\n| 429 | Rate Limited - API quota exceeded |\n| 500 | Internal Server Error |\n",".claude/references/architecture.md":"# Architecture Reference\n\n## Table of Contents\n- [System Layers](#system-layers)\n- [Key File Locations](#key-file-locations)\n\n---\n\n## System Layers\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ USER REQUEST │\n│ (query, report_type, report_source, tone, mcp_configs) │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ BACKEND API LAYER │\n│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │\n│ │ FastAPI Server │ │ WebSocket Manager│ │ Report Store │ │\n│ │ backend/server/ │ │ Real-time events │ │ JSON persistence│ │\n│ │ app.py │ │ websocket_mgr.py │ │ report_store.py │ │\n│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ GPTResearcher (gpt_researcher/agent.py) │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ SKILLS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ ResearchConductor│ │ ReportGenerator │ │ ContextManager │ │ │\n│ │ │ Plan & gather │ │ Write reports │ │ Similarity search│ │ │\n│ │ │ researcher.py │ │ writer.py │ │ context_manager │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ BrowserManager │ │ SourceCurator │ │ ImageGenerator │ │ │\n│ │ │ Web scraping │ │ Rank sources │ │ Gemini images │ │ │\n│ │ │ browser.py │ │ curator.py │ │ image_generator │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ DeepResearchSkill│ │ │\n│ │ │ Recursive depth │ │ │\n│ │ │ deep_research.py│ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ ACTIONS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ report_generation│ │ query_processing│ │ web_scraping │ │ │\n│ │ │ LLM report write│ │ Sub-query plan │ │ URL scraping │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ retriever.py │ │ agent_creator │ │ markdown_process│ │ │\n│ │ │ Get retrievers │ │ Choose agent │ │ Parse markdown │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ PROVIDERS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ LLM Provider │ │ Retrievers │ │ Scrapers │ │ │\n│ │ │ OpenAI,Anthropic│ │ Tavily,Google │ │ BS4,Playwright │ │ │\n│ │ │ Google,Groq... │ │ Bing,MCP... │ │ PDF,DOCX... │ │ │\n│ │ │ llm_provider/ │ │ retrievers/ │ │ scraper/ │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ ImageGenerator │ │ │\n│ │ │ Gemini/Imagen │ │ │\n│ │ │ llm_provider/ │ │ │\n│ │ │ image/ │ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ CONFIGURATION LAYER │\n│ gpt_researcher/config/ │\n│ │\n│ Environment Variables → JSON Config File → Default Values │\n│ (highest) (medium) (lowest) │\n│ │\n│ config.py loads and merges all sources │\n│ variables/default.py contains all defaults │\n│ variables/base.py defines TypedDict for type safety │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes/Functions |\n|------|--------------|----------------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| Context/embeddings | `gpt_researcher/skills/context_manager.py` | `ContextManager` |\n| Source ranking | `gpt_researcher/skills/curator.py` | `SourceCurator` |\n| Deep research | `gpt_researcher/skills/deep_research.py` | `DeepResearchSkill` |\n| Image generation | `gpt_researcher/skills/image_generator.py` | `ImageGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| Config types | `gpt_researcher/config/variables/base.py` | `BaseConfig` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| WebSocket mgmt | `backend/server/websocket_manager.py` | `WebSocketManager`, `run_agent` |\n| Report types | `backend/report_type/` | `BasicReport`, `DetailedReport` |\n| Search engines | `gpt_researcher/retrievers/` | `TavilySearch`, `GoogleSearch`, etc. |\n| Web scraping | `gpt_researcher/scraper/` | Various scrapers |\n| Enums | `gpt_researcher/utils/enum.py` | `ReportType`, `ReportSource`, `Tone` |\n",".claude/references/components.md":"# Core Components & Method Signatures\n\n## Table of Contents\n- [GPTResearcher](#gptresearcher)\n- [ResearchConductor](#researchconductor)\n- [ReportGenerator](#reportgenerator)\n\n---\n\n## GPTResearcher\n\n**File:** `gpt_researcher/agent.py`\n\nThe main orchestrator class. Full initialization signature:\n\n```python\nclass GPTResearcher:\n def __init__(\n self,\n query: str, # Research question (required)\n report_type: str = \"research_report\", # research_report, detailed_report, deep, outline_report, resource_report\n report_format: str = \"markdown\", # Output format\n report_source: str = \"web\", # web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n tone: Tone = Tone.Objective, # Writing tone (see Tone enum)\n source_urls: list[str] | None = None, # Specific URLs to research\n document_urls: list[str] | None = None, # Document URLs to include\n complement_source_urls: bool = False, # Add web search to source_urls\n query_domains: list[str] | None = None, # Restrict search to domains\n documents=None, # LangChain document objects\n vector_store=None, # LangChain vector store\n vector_store_filter=None, # Filter for vector store\n config_path=None, # Path to JSON config file\n websocket=None, # WebSocket for streaming\n agent=None, # Pre-defined agent type\n role=None, # Pre-defined agent role\n parent_query: str = \"\", # Parent query for subtopics\n subtopics: list | None = None, # Subtopics to research\n visited_urls: set | None = None, # Already visited URLs\n verbose: bool = True, # Verbose logging\n context=None, # Pre-loaded context\n headers: dict | None = None, # HTTP headers\n max_subtopics: int = 5, # Max subtopics for detailed\n log_handler=None, # Custom log handler\n prompt_family: str | None = None, # Custom prompt family\n mcp_configs: list[dict] | None = None, # MCP server configurations\n mcp_max_iterations: int | None = None, # Deprecated, use mcp_strategy\n mcp_strategy: str | None = None, # fast, deep, disabled\n **kwargs\n ):\n```\n\n### Key Methods\n\n```python\nasync def conduct_research(self, on_progress=None) -> str:\n \"\"\"\n Main research orchestration.\n \n 1. Selects agent role via LLM (choose_agent)\n 2. Delegates to ResearchConductor\n 3. Optionally generates images if enabled\n \n Returns: Accumulated research context as string\n \"\"\"\n\nasync def write_report(\n self, \n existing_headers: list = [], # Headers to avoid duplication\n relevant_written_contents: list = [], # Previous content for context\n ext_context=None, # External context override\n custom_prompt=\"\" # Custom prompt override\n) -> str:\n \"\"\"\n Generate final report from context.\n \n Returns: Markdown report string\n \"\"\"\n\ndef get_costs(self) -> float:\n \"\"\"Returns total accumulated API costs.\"\"\"\n\ndef add_costs(self, cost: float) -> None:\n \"\"\"Add to running cost total (used as callback).\"\"\"\n```\n\n---\n\n## ResearchConductor\n\n**File:** `gpt_researcher/skills/researcher.py`\n\nManages the research process:\n\n```python\nclass ResearchConductor:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.logger = logging.getLogger(__name__)\n\n async def plan_research(self, query: str, query_domains=None) -> list:\n \"\"\"\n Generate sub-queries from main query using LLM.\n \n 1. Gets initial search results\n 2. Calls plan_research_outline() to generate sub-queries\n \n Returns: List of sub-query strings\n \"\"\"\n\n async def conduct_research(self) -> str:\n \"\"\"\n Main research execution based on report_source.\n \n Handles: web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n \n For each source type:\n 1. Load/search data\n 2. Process sub-queries\n 3. Combine context\n 4. Optionally curate sources\n \n Returns: Combined research context string\n \"\"\"\n\n async def _process_sub_query(\n self, \n sub_query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"\n Process a single sub-query.\n \n 1. Get MCP context (if configured, based on strategy)\n 2. Scrape URLs from search results\n 3. Get similar content via embeddings\n 4. Combine MCP + web context\n \n Returns: Combined context for this sub-query\n \"\"\"\n\n async def _get_context_by_web_search(\n self, \n query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"Web-based research with sub-query planning.\"\"\"\n\n async def _scrape_data_by_urls(\n self, \n sub_query: str, \n query_domains: list = []\n ) -> list:\n \"\"\"Search and scrape URLs for a sub-query.\"\"\"\n```\n\n---\n\n## ReportGenerator\n\n**File:** `gpt_researcher/skills/writer.py`\n\n```python\nclass ReportGenerator:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.research_params = {\n \"query\": researcher.query,\n \"agent_role_prompt\": researcher.cfg.agent_role or researcher.role,\n \"report_type\": researcher.report_type,\n \"report_source\": researcher.report_source,\n \"tone\": researcher.tone,\n \"websocket\": researcher.websocket,\n \"cfg\": researcher.cfg,\n \"headers\": researcher.headers,\n }\n\n async def write_report(\n self,\n existing_headers: list = [],\n relevant_written_contents: list = [],\n ext_context=None,\n custom_prompt=\"\",\n available_images: list = [], # Pre-generated images to embed\n ) -> str:\n \"\"\"\n Generate report using LLM.\n \n Calls generate_report() action with context and images.\n \n Returns: Markdown report\n \"\"\"\n\n async def write_introduction(self, ...) -> str:\n \"\"\"Write report introduction section.\"\"\"\n\n async def write_conclusion(self, ...) -> str:\n \"\"\"Write report conclusion with references.\"\"\"\n```\n",".claude/references/config-reference.md":"# Configuration Reference\n\n## Table of Contents\n- [Required Variables](#required-variables)\n- [LLM Configuration](#llm-configuration)\n- [Provider API Keys](#provider-api-keys)\n- [Retriever Configuration](#retriever-configuration)\n- [Report Configuration](#report-configuration)\n- [Feature Toggles](#feature-toggles)\n- [Configuration Priority](#configuration-priority)\n- [Example .env](#example-env)\n\n---\n\n## Required Variables\n\n```bash\nOPENAI_API_KEY=sk-... # Or another LLM provider key\nTAVILY_API_KEY=tvly-... # Or another retriever key\n```\n\n---\n\n## LLM Configuration\n\n```bash\nLLM_PROVIDER=openai # openai, anthropic, google, groq, together, etc.\nFAST_LLM=gpt-4o-mini # Quick tasks (summarization)\nSMART_LLM=gpt-4o # Complex reasoning (report writing)\nSTRATEGIC_LLM=o3-mini # Planning (agent selection)\nTEMPERATURE=0.4 # 0.0-1.0\nMAX_TOKENS=4000\nREASONING_EFFORT=medium # For o-series: low, medium, high\n```\n\n---\n\n## Provider API Keys\n\n```bash\n# OpenAI\nOPENAI_API_KEY=sk-...\nOPENAI_BASE_URL=https://api.openai.com/v1\n\n# Anthropic\nANTHROPIC_API_KEY=sk-ant-...\n\n# Google\nGOOGLE_API_KEY=AIza...\n\n# Groq\nGROQ_API_KEY=gsk_...\n\n# Azure OpenAI\nAZURE_OPENAI_API_KEY=...\nAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\n```\n\n---\n\n## Retriever Configuration\n\n```bash\nRETRIEVER=tavily # Single or comma-separated: tavily,google,mcp\nMAX_SEARCH_RESULTS_PER_QUERY=5\nMAX_URLS_TO_SCRAPE=10\nSIMILARITY_THRESHOLD=0.42\n```\n\n### Retriever API Keys\n\n```bash\nTAVILY_API_KEY=tvly-...\nGOOGLE_API_KEY=AIza...\nGOOGLE_CX_KEY=...\nBING_API_KEY=...\nSERPER_API_KEY=...\nSERPAPI_API_KEY=...\nEXA_API_KEY=...\n```\n\n---\n\n## Report Configuration\n\n```bash\nREPORT_FORMAT=apa # apa, mla, chicago, harvard, ieee\nTOTAL_WORDS=1000\nLANGUAGE=english\nCURATE_SOURCES=true\n```\n\n---\n\n## Feature Toggles\n\n### Image Generation\n\n```bash\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza...\nIMAGE_GENERATION_MODEL=models/gemini-2.5-flash-image\nIMAGE_GENERATION_MAX_IMAGES=3\nIMAGE_GENERATION_STYLE=dark # dark, light, auto\n```\n\n### Deep Research\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n### MCP\n\n```bash\nMCP_STRATEGY=fast # fast, deep, disabled\n```\n\n### Local Documents\n\n```bash\nDOC_PATH=./my-docs\n# Supports: PDF, DOCX, TXT, CSV, XLSX, PPTX, MD\n```\n\n### Server\n\n```bash\nHOST=0.0.0.0\nPORT=8000\nVERBOSE=true\n```\n\n---\n\n## Configuration Priority\n\n```\nEnvironment Variables (highest)\n ↓\nJSON Config File (if provided)\n ↓\nDefault Values (lowest)\n```\n\n**Important:** Config keys are lowercased when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\n---\n\n## Example .env\n\n```bash\n# Required\nOPENAI_API_KEY=sk-your-key\nTAVILY_API_KEY=tvly-your-key\n\n# LLM\nFAST_LLM=gpt-4o-mini\nSMART_LLM=gpt-4o\n\n# Report\nTOTAL_WORDS=1000\nLANGUAGE=english\n\n# Optional: Images\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza-your-key\nIMAGE_GENERATION_STYLE=dark\n```\n",".claude/references/deep-research.md":"# Deep Research Mode Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [DeepResearchSkill](#deepresearchskill)\n- [Usage](#usage)\n\n---\n\n## Overview\n\nDeep Research uses recursive tree-like exploration with configurable depth and breadth.\n\n---\n\n## Configuration\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n---\n\n## DeepResearchSkill\n\n**File:** `gpt_researcher/skills/deep_research.py`\n\n```python\nclass DeepResearchSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.breadth = getattr(researcher.cfg, 'deep_research_breadth', 4)\n self.depth = getattr(researcher.cfg, 'deep_research_depth', 2)\n self.concurrency_limit = getattr(researcher.cfg, 'deep_research_concurrency', 2)\n self.learnings = []\n self.research_sources = []\n self.context = []\n\n async def deep_research(self, query: str, on_progress=None) -> str:\n \"\"\"\n Recursive research with depth and breadth.\n \n 1. Research main topic\n 2. Generate subtopics (breadth)\n 3. For each subtopic, recursively research (depth)\n 4. Aggregate all findings\n 5. Generate comprehensive report\n \"\"\"\n```\n\n---\n\n## Usage\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers deep research\n)\nawait researcher.conduct_research()\nreport = await researcher.write_report()\n```\n\n### Research Tree Structure\n\n```\nQuery: \"Quantum Computing\"\n├── Subtopic 1: Hardware (depth 1)\n│ ├── Subtopic 1.1: Superconducting qubits (depth 2)\n│ └── Subtopic 1.2: Ion traps (depth 2)\n├── Subtopic 2: Algorithms (depth 1)\n│ ├── Subtopic 2.1: Shor's algorithm (depth 2)\n│ └── Subtopic 2.2: Grover's algorithm (depth 2)\n├── Subtopic 3: Applications (depth 1)\n│ └── ...\n└── Subtopic 4: Challenges (depth 1)\n └── ...\n```\n\nWith `DEEP_RESEARCH_BREADTH=4` and `DEEP_RESEARCH_DEPTH=2`, this explores 4 subtopics at each level, going 2 levels deep.\n",".claude/references/flows.md":"# Research Flow & Data Flow\n\n## Table of Contents\n- [End-to-End Research Flow](#end-to-end-research-flow)\n- [Data Flow Between Components](#data-flow-between-components)\n\n---\n\n## End-to-End Research Flow\n\n### 1. Request Entry\n\n**File:** `backend/server/app.py`\n\n```python\n# REST API endpoint\n@app.post(\"/report/\")\nasync def generate_report(research_request: ResearchRequest, background_tasks: BackgroundTasks):\n research_id = sanitize_filename(f\"task_{int(time.time())}_{research_request.task}\")\n # Calls write_report() which uses run_agent()\n\n# WebSocket endpoint\n@app.websocket(\"/ws\")\nasync def websocket_endpoint(websocket: WebSocket):\n await manager.connect(websocket)\n await handle_websocket_communication(websocket, manager)\n```\n\n### 2. Agent Runner\n\n**File:** `backend/server/websocket_manager.py`\n\n```python\nasync def run_agent(task, report_type, report_source, source_urls, ...):\n \"\"\"Main entry point for research execution.\"\"\"\n # Create logs handler\n logs_handler = CustomLogsHandler(websocket, task)\n \n # Configure MCP if enabled\n if mcp_enabled and mcp_configs:\n os.environ[\"RETRIEVER\"] = f\"{current_retriever},mcp\"\n os.environ[\"MCP_STRATEGY\"] = mcp_strategy\n \n # Route based on report type\n if report_type == \"multi_agents\":\n report = await run_research_task(query=task, websocket=logs_handler, ...)\n elif report_type == ReportType.DetailedReport.value:\n researcher = DetailedReport(query=task, ...)\n report = await researcher.run()\n else:\n researcher = BasicReport(query=task, ...)\n report = await researcher.run()\n \n return report\n```\n\n### 3. Research Phase\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nasync def conduct_research(self, on_progress=None):\n # Handle deep research separately\n if self.report_type == ReportType.DeepResearch.value and self.deep_researcher:\n return await self._handle_deep_research(on_progress)\n \n # Choose agent role via LLM\n if not (self.agent and self.role):\n self.agent, self.role = await choose_agent(\n query=self.query,\n cfg=self.cfg,\n parent_query=self.parent_query,\n cost_callback=self.add_costs,\n headers=self.headers,\n prompt_family=self.prompt_family,\n )\n \n # Conduct research\n self.context = await self.research_conductor.conduct_research()\n \n # Generate images if enabled (pre-generation for seamless UX)\n if self.cfg.image_generation_enabled and self.image_generator:\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n```\n\n### 4. Sub-Query Processing\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\nasync def _process_sub_query(self, sub_query: str, scraped_data: list = [], query_domains: list = []):\n # MCP Strategy handling\n mcp_retrievers = [r for r in self.researcher.retrievers if \"mcpretriever\" in r.__name__.lower()]\n mcp_strategy = self._get_mcp_strategy()\n \n if mcp_retrievers:\n if mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n # Reuse cached MCP results\n mcp_context = self._mcp_results_cache.copy()\n elif mcp_strategy == \"deep\":\n # Run MCP for every sub-query\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n \n # Get web search context\n if not scraped_data:\n scraped_data = await self._scrape_data_by_urls(sub_query, query_domains)\n \n # Get similar content via embeddings\n if scraped_data:\n web_context = await self.researcher.context_manager.get_similar_content_by_query(\n sub_query, scraped_data\n )\n \n # Combine MCP + web context\n combined_context = self._combine_mcp_and_web_context(mcp_context, web_context, sub_query)\n return combined_context\n```\n\n### 5. Report Generation\n\n**File:** `gpt_researcher/actions/report_generation.py`\n\n```python\nasync def generate_report(\n query: str,\n context: str,\n agent_role_prompt: str,\n report_type: str,\n websocket=None,\n cfg=None,\n tone=None,\n headers=None,\n cost_callback=None,\n prompt_family=None,\n available_images: list = [],\n **kwargs\n) -> str:\n \"\"\"Generate report using LLM.\"\"\"\n # Get prompt generator\n generate_prompt = prompt_family.get_prompt_by_report_type(report_type)\n \n # Build prompt with context and available images\n content = generate_prompt(\n query, context, report_source,\n report_format=cfg.report_format,\n tone=tone,\n total_words=cfg.total_words,\n language=cfg.language,\n available_images=available_images,\n )\n \n # Call LLM\n report = await create_chat_completion(\n model=cfg.smart_llm,\n messages=[{\"role\": \"user\", \"content\": content}],\n temperature=cfg.temperature,\n llm_provider=cfg.smart_llm_provider,\n max_tokens=cfg.smart_token_limit,\n llm_kwargs=cfg.llm_kwargs,\n cost_callback=cost_callback,\n )\n \n return report\n```\n\n---\n\n## Data Flow Between Components\n\n```\nUser Query\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ GPTResearcher.__init__() │\n│ • Loads Config (env → json → defaults) │\n│ • Initializes skills: ResearchConductor, ReportGenerator, etc │\n│ • Initializes retrievers based on RETRIEVER env var │\n│ • Initializes ImageGenerator if IMAGE_GENERATION_ENABLED │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.conduct_research()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ choose_agent() │\n│ Input: query, config │\n│ Output: (agent_type: str, role_prompt: str) │\n│ • LLM selects best agent role for the query │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ResearchConductor.conduct_research() │\n│ Input: self.researcher (has query, config, retrievers) │\n│ Output: context: str │\n│ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ plan_research() │ │\n│ │ Input: query │ │\n│ │ Output: sub_queries: list[str] │ │\n│ │ • Calls LLM to generate 3-5 sub-queries │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ For each sub_query: │ │\n│ │ _process_sub_query() │ │\n│ │ Input: sub_query │ │\n│ │ Output: sub_context: str │ │\n│ │ │ │\n│ │ 1. MCP retrieval (if configured) │ │\n│ │ → mcp_context: list[dict] │ │\n│ │ │ │\n│ │ 2. Web search via retrievers │ │\n│ │ → search_results: list[dict] │ │\n│ │ │ │\n│ │ 3. Scrape URLs │ │\n│ │ → scraped_content: list[dict] │ │\n│ │ │ │\n│ │ 4. Similarity search via embeddings │ │\n│ │ → relevant_context: str │ │\n│ │ │ │\n│ │ 5. Combine MCP + web context │ │\n│ │ → combined_context: str │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ Aggregate all sub_contexts → final context: str │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ If IMAGE_GENERATION_ENABLED:\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ImageGenerator.plan_and_generate_images() │\n│ Input: context, query, research_id │\n│ Output: available_images: list[dict] │\n│ [{\"url\": \"/outputs/images/.../img.png\", │\n│ \"title\": \"...\", \"description\": \"...\"}] │\n│ │\n│ 1. LLM analyzes context for visual concepts │\n│ 2. Generates 2-3 images in parallel via Gemini │\n│ 3. Saves to outputs/images/{research_id}/ │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.write_report()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ReportGenerator.write_report() │\n│ Input: context, available_images │\n│ Output: report: str (markdown) │\n│ │\n│ → generate_report() action │\n│ • Builds prompt with context + image list │\n│ • LLM generates report with embedded images │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ Output │\n│ • Streamed via WebSocket (type: \"report\") │\n│ • Final via WebSocket (type: \"report_complete\") │\n│ • Exported to PDF, DOCX, Markdown │\n│ • Saved to outputs/ directory │\n└─────────────────────────────────────────────────────────────────┘\n```\n",".claude/references/mcp.md":"# MCP Integration Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [Strategy Options](#strategy-options)\n- [Processing Logic](#processing-logic)\n\n---\n\n## Overview\n\nMCP (Model Context Protocol) enables research from specialized data sources (GitHub, databases, APIs) alongside web search.\n\n---\n\n## Configuration\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n mcp_configs=[\n {\n \"name\": \"github\", # Server name\n \"command\": \"npx\", # Command to start\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}, # Environment vars\n },\n {\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@anthropic/mcp-server-filesystem\", \"/docs\"],\n },\n {\n \"name\": \"remote\",\n \"connection_url\": \"ws://server:8080\", # WebSocket connection\n \"connection_type\": \"websocket\",\n \"connection_token\": \"auth_token\",\n }\n ],\n mcp_strategy=\"fast\", # fast, deep, disabled\n)\n```\n\n---\n\n## Strategy Options\n\n| Strategy | Behavior | Use Case |\n|----------|----------|----------|\n| `fast` (default) | Run MCP once with original query, cache results | Performance-focused |\n| `deep` | Run MCP for every sub-query | Maximum thoroughness |\n| `disabled` | Skip MCP entirely | Web-only research |\n\n---\n\n## Processing Logic\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\n# At start of research (for 'fast' strategy)\nif mcp_strategy == \"fast\":\n mcp_context = await self._execute_mcp_research_for_queries([query], mcp_retrievers)\n self._mcp_results_cache = mcp_context # Cache for reuse\n\n# During sub-query processing\nif mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n mcp_context = self._mcp_results_cache.copy() # Reuse cache\nelif mcp_strategy == \"deep\":\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n```\n\n### WebSocket Request Example\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"mcp_enabled\": true,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": [\n {\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}\n }\n ]\n}\n```\n",".claude/references/multi-agents.md":"# Multi-Agent System Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Agent Roles](#agent-roles)\n- [Workflow](#workflow)\n- [Usage](#usage)\n\n---\n\n## Overview\n\n**Directory:** `multi_agents/`\n\nLangGraph-based multi-agent system. Generates 5-6 page reports with multiple agents collaborating.\n\n---\n\n## Agent Roles\n\n| Agent | File | Role |\n|-------|------|------|\n| Human | - | Oversees and provides feedback |\n| Chief Editor | `agents/orchestrator.py` | Master coordinator via LangGraph |\n| Researcher | Uses GPTResearcher | Deep research on topics |\n| Editor | `agents/editor.py` | Plans outline and structure |\n| Reviewer | `agents/reviewer.py` | Validates research correctness |\n| Reviser | `agents/reviser.py` | Revises based on feedback |\n| Writer | `agents/writer.py` | Compiles final report |\n| Publisher | `agents/publisher.py` | Exports to PDF, DOCX, Markdown |\n\n---\n\n## Workflow\n\n```\n1. Browser (GPTResearcher) → Initial research\n2. Editor → Plans report outline\n3. For each outline topic (parallel):\n a. Researcher → In-depth subtopic research\n b. Reviewer → Validates draft\n c. Reviser → Revises until satisfactory\n4. Writer → Compiles final report\n5. Publisher → Exports to multiple formats\n```\n\n---\n\n## Usage\n\n### Via API\n\n```python\nreport_type = \"multi_agents\"\n```\n\n### Via WebSocket\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"multi_agents\",\n \"tone\": \"Analytical\"\n}\n```\n\n### Directly in Python\n\n```python\nfrom multi_agents.main import run_research_task\n\nreport = await run_research_task(\n query=\"Comprehensive analysis of market trends\",\n websocket=handler,\n tone=Tone.Analytical,\n)\n```\n\n### Configuration File\n\n**File:** `multi_agents/task.json`\n\nConfigure the multi-agent research task parameters and agent behaviors.\n",".claude/references/prompts.md":"# Prompt System Reference\n\n## Table of Contents\n- [PromptFamily Class](#promptfamily-class)\n- [Key Prompt Examples](#key-prompt-examples)\n\n---\n\n## PromptFamily Class\n\n**File:** `gpt_researcher/prompts.py`\n\nAll prompts are centralized in the `PromptFamily` class. This allows for model-specific prompt variations.\n\n```python\nclass PromptFamily:\n \"\"\"\n General purpose class for prompt formatting.\n Can be overwritten with model-specific derived classes.\n \"\"\"\n\n def __init__(self, config: Config):\n self.cfg = config\n\n @staticmethod\n def get_prompt_by_report_type(report_type: str):\n \"\"\"Returns the appropriate prompt generator for the report type.\"\"\"\n match report_type:\n case ReportType.ResearchReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.DetailedReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.OutlineReport.value:\n return PromptFamily.generate_outline_report_prompt\n # ... etc\n```\n\n---\n\n## Key Prompt Examples\n\n### Agent Selection Prompt\n\n```python\n@staticmethod\ndef generate_agent_role_prompt(query: str, parent_query: str = \"\") -> str:\n return f\"\"\"Analyze the research query and select the most appropriate agent role.\n\nQuery: \"{query}\"\n{f'Parent Query: \"{parent_query}\"' if parent_query else ''}\n\nBased on the query, determine:\n1. The domain expertise needed\n2. The research approach required\n3. The appropriate agent persona\n\nReturn a JSON object with:\n- \"agent\": The agent type (e.g., \"Research Analyst\", \"Technical Writer\")\n- \"role\": A detailed role description for how the agent should approach this research\n\"\"\"\n```\n\n### Research Planning Prompt\n\n```python\n@staticmethod\ndef generate_search_queries_prompt(\n query: str,\n parent_query: str = \"\",\n report_type: str = \"\",\n max_iterations: int = 3,\n context: str = \"\",\n) -> str:\n return f\"\"\"Generate {max_iterations} focused search queries to research: \"{query}\"\n\nContext from initial search:\n{context}\n\nRequirements:\n- Each query should explore a different aspect\n- Queries should be specific and searchable\n- Consider the report type: {report_type}\n\nReturn a JSON array of query strings.\n\"\"\"\n```\n\n### Report Generation Prompt (with images)\n\n```python\n@staticmethod\ndef generate_report_prompt(\n question: str,\n context: str,\n report_source: str,\n report_format=\"apa\",\n total_words=1000,\n tone=None,\n language=\"english\",\n available_images: list = [],\n) -> str:\n # Build image embedding instruction if images available\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES (embed where relevant):\n{image_list}\n\nUse markdown format: ![Title](URL)\n\"\"\"\n\n return f\"\"\"Information: \"{context}\"\n---\nUsing the above information, answer: \"{question}\" in a detailed report.\n\n- Format: {report_format}\n- Length: ~{total_words} words\n- Tone: {tone.value if tone else \"Objective\"}\n- Language: {language}\n- Include citations for all factual claims\n{image_instruction}\n\"\"\"\n```\n\n### MCP Tool Selection Prompt\n\n```python\n@staticmethod\ndef generate_mcp_tool_selection_prompt(query: str, tools_info: list, max_tools: int = 3) -> str:\n return f\"\"\"Select the most relevant tools for researching: \"{query}\"\n\nAVAILABLE TOOLS:\n{json.dumps(tools_info, indent=2)}\n\nSelect exactly {max_tools} tools ranked by relevance.\n\nReturn JSON:\n{{\n \"selected_tools\": [\n {{\"index\": 0, \"name\": \"tool_name\", \"relevance_score\": 9, \"reason\": \"...\"}}\n ]\n}}\n\"\"\"\n```\n",".claude/references/retrievers.md":"# Retriever System Reference\n\n## Table of Contents\n- [Available Retrievers](#available-retrievers)\n- [Retriever Selection](#retriever-selection)\n- [Adding a New Retriever](#adding-a-new-retriever)\n\n---\n\n## Available Retrievers\n\n**Directory:** `gpt_researcher/retrievers/`\n\n| Retriever | Class | API Key Env Var |\n|-----------|-------|-----------------|\n| Tavily | `TavilySearch` | `TAVILY_API_KEY` |\n| Google | `GoogleSearch` | `GOOGLE_API_KEY`, `GOOGLE_CX_KEY` |\n| Bing | `BingSearch` | `BING_API_KEY` |\n| Brave | `BraveSearch` | `BRAVE_API_KEY` |\n| DuckDuckGo | `Duckduckgo` | None |\n| SearX | `SearxSearch` | `SEARX_URL` |\n| Serper | `SerperSearch` | `SERPER_API_KEY` |\n| SerpAPI | `SerpApiSearch` | `SERPAPI_API_KEY` |\n| SearchAPI | `SearchApiSearch` | `SEARCHAPI_API_KEY` |\n| Exa | `ExaSearch` | `EXA_API_KEY` |\n| GroundRoute | `GroundRouteSearch` | `GROUNDROUTE_API_KEY` |\n| fastCRW | `CRWRetriever` | `CRW_API_KEY` |\n| BoCha | `BoChaSearch` | `BOCHA_API_KEY` |\n| Xquik | `XquikSearch` | `XQUIK_API_KEY` |\n| GetXAPI (X/Twitter) | `GetXAPISearch` | `GETXAPI_API_KEY` |\n| arXiv | `ArxivSearch` | None |\n| Semantic Scholar | `SemanticScholarSearch` | None |\n| PubMed Central | `PubMedCentralSearch` | None |\n| OpenAlex | `OpenAlexSearch` | `OPENALEX_API_KEY`, `OPENALEX_EMAIL` |\n| MCP | `MCPRetriever` | Per-server |\n| Custom | `CustomRetriever` | User-defined |\n\n> The authoritative list is the `match` statement in `gpt_researcher/actions/retriever.py` (`get_retriever`).\n\n---\n\n## Retriever Selection\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ndef get_retriever(retriever: str):\n \"\"\"Get a retriever class by name.\"\"\"\n match retriever:\n case \"tavily\":\n from gpt_researcher.retrievers import TavilySearch\n return TavilySearch\n case \"google\":\n from gpt_researcher.retrievers import GoogleSearch\n return GoogleSearch\n case \"mcp\":\n from gpt_researcher.retrievers import MCPRetriever\n return MCPRetriever\n # ... etc\n\ndef get_retrievers(headers: dict[str, str], cfg) -> list:\n \"\"\"\n Resolve which retriever(s) to use, in priority order:\n headers[\"retrievers\"] / headers[\"retriever\"] → cfg.retrievers / cfg.retriever\n → default (TavilySearch). Accepts a comma-separated string or a list.\n\n Usage: RETRIEVER=tavily,google,mcp\n \"\"\"\n # ... resolve names from headers/cfg (see actions/retriever.py) ...\n # Invalid names fall back to get_default_retriever():\n return [get_retriever(r) or get_default_retriever() for r in retriever_names]\n```\n\n---\n\n## Adding a New Retriever\n\n### Step 1: Create Retriever File\n\n**File:** `gpt_researcher/retrievers/my_retriever/my_retriever.py`\n\n```python\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n self.headers = headers\n \n async def search(self, max_results: int = 10) -> list[dict]:\n \"\"\"\n Returns list of:\n {\n \"title\": str,\n \"href\": str,\n \"body\": str\n }\n \"\"\"\n # Implementation\n pass\n```\n\n### Step 2: Register in retriever.py\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n```\n\n### Step 3: Export in __init__.py\n\n**File:** `gpt_researcher/retrievers/__init__.py`\n\n```python\nfrom .my_retriever import MyRetriever\n__all__ = [..., \"MyRetriever\"]\n```\n\n### Step 4: Usage\n\n```bash\nRETRIEVER=tavily,my_retriever\n```\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n # Will use both Tavily and your custom retriever\n)\n```\n",".codex-plugin/plugin.json":"{\n \"name\": \"gpt-researcher\",\n \"version\": \"0.1.0\",\n \"description\": \"Autonomous deep research from Codex via MCP\",\n \"author\": {\n \"name\": \"assafelovic\",\n \"url\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"homepage\": \"https://github.com/assafelovic/gpt-researcher\",\n \"repository\": \"https://github.com/assafelovic/gpt-researcher\",\n \"keywords\": [\n \"mcp\",\n \"codex\"\n ],\n \"mcpServers\": \"./.mcp.json\",\n \"interface\": {\n \"displayName\": \"GPT Researcher\",\n \"shortDescription\": \"Autonomous deep research from Codex via MCP\",\n \"longDescription\": \"An autonomous agent that conducts deep research on any data using any LLM providers.\",\n \"category\": \"Research\",\n \"websiteURL\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"skills\": \"./skills/\"\n}","evals/README.md":"# GPT-Researcher Evaluations\n\nThis directory contains evaluation tools and frameworks for assessing the performance of GPT-Researcher across different research tasks.\n\n## Evaluation Overview\n\nThree complementary evaluation systems, each answering a different question:\n\n| System | Question | Requires ground truth? |\n| --- | --- | --- |\n| `simple_evals/` | Is the answer factually correct? | Yes (OpenAI SimpleQA dataset) |\n| `quality_eval/` | Is the report well-sourced and complete? | No — works on any query |\n| `hallucination_eval/` | Does the report fabricate content? | No — compares against scraped sources |\n\n**Quick start:**\n\n```bash\n# Factual accuracy (needs ground-truth answers)\npython -m evals.simple_evals.run_eval --num_examples 10\n\n# Report quality (works on any query, zero-cost metrics only)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Hallucination detection\npython -m evals.hallucination_eval.run_eval -n 5\n```\n\n## Simple Evaluations (`simple_evals/`)\n\nThe `simple_evals` directory contains a straightforward evaluation framework adapted from [OpenAI's simple-evals system](https://github.com/openai/simple-evals), specifically designed to measure short-form factuality in large language models. Our implementation is based on OpenAI's [SimpleQA evaluation methodology](https://github.com/openai/simple-evals/blob/main/simpleqa_eval.py), following their zero-shot, chain-of-thought approach while adapting it for GPT-Researcher's specific use case.\n\n### Components\n\n- `simpleqa_eval.py`: Core evaluation logic for grading research responses\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `requirements.txt`: Dependencies required for running evaluations\n\n### Test Dataset\n\nThe `problems/` directory contains the evaluation dataset:\n\n- `Simple QA Test Set.csv`: A comprehensive collection of factual questions and their correct answers, mirrored from OpenAI's original test set. This dataset serves as the ground truth for evaluating GPT-Researcher's ability to find and report accurate information. The file is maintained locally to ensure consistent evaluation benchmarks and prevent any potential upstream changes from affecting our testing methodology.\n\n### Evaluation Logs\n\nThe `logs/` directory contains evaluation run histories preserved in version control.\n\nTwo formats are kept side-by-side:\n\n| Format | Example filename | Purpose |\n| --- | --- | --- |\n| Plain text (legacy) | `SimpleQA Eval 100 Problems 2-22-25.txt` | Human-readable terminal output |\n| Structured JSON | `eval_2025-02-22_10-32-11_n100.jsonl` | Machine-readable, suitable for trend analysis |\n\nEach JSON log file contains one record with three sections: `run_metadata`, `aggregate_metrics`, and per-query `results`. See [`example_output.json`](simple_evals/example_output.json) for the full schema.\n\n**Note:** Unlike typical log directories, this folder and its contents are intentionally tracked in git to maintain a historical record of evaluation runs.\n\n### Features\n\n- Measures factual accuracy of research responses\n- Uses GPT-4 as a grading model (configurable)\n ```python\n # In run_eval.py, you can customize the grader model:\n grader_model = ChatOpenAI(\n temperature=0, # Lower temperature for more consistent grading\n model_name=\"gpt-4-turbo\", # Can be changed to other OpenAI models\n openai_api_key=os.getenv(\"OPENAI_API_KEY\")\n )\n ```\n- Grades responses on a three-point scale:\n - `CORRECT`: Answer fully contains important information without contradictions\n - `INCORRECT`: Answer contains factual contradictions\n - `NOT_ATTEMPTED`: Answer neither confirms nor contradicts the target\n\n**Note on Grader Configuration:** While the default grader uses GPT-4-turbo, you can modify the model and its parameters to use different OpenAI models or adjust the temperature for different grading behaviors. This is independent of the researcher's configuration, allowing you to optimize for cost or performance as needed.\n\n### Metrics Tracked\n\n- Accuracy rate and F1 score\n- Cost — total and per-query average\n- Latency — average, p50 (median), and p95 per query\n- Success/failure counts and answer attempt rates\n- Source coverage per query\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/simple_evals\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\nLANGCHAIN_API_KEY=your_langchain_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py --num_examples \n```\n\nThe `num_examples` parameter determines how many random test queries to evaluate (default: 1).\n\n#### Customizing Researcher Behavior\n\nThe evaluation uses GPTResearcher with default settings, but you can modify `run_eval.py` to customize the researcher's behavior:\n\n```python\nresearcher = GPTResearcher(\n query=query,\n report_type=ReportType.ResearchReport.value, # Type of report to generate\n report_format=\"markdown\", # Output format\n report_source=ReportSource.Web.value, # Source of research\n tone=Tone.Objective, # Writing tone\n verbose=True # Enable detailed logging\n)\n```\n\nThese parameters can be adjusted to evaluate different research configurations or output formats. For a complete list of configuration options, see the [configuration documentation](https://docs.gptr.dev/docs/gpt-researcher/gptr/config).\n\n**Note on Configuration Independence:** The evaluation system is designed to be independent of the researcher's configuration. This means you can use different LLMs and settings for evaluation versus research. For example:\n- Evaluation could use GPT-4-turbo for grading while the researcher uses Claude 3.5 Sonnet for research\n- Different retrievers, embeddings, or report formats can be used\n- Token limits and other parameters can be customized separately\n\nThis separation allows for unbiased evaluation across different researcher configurations. However, please note that this feature is currently experimental and needs further testing.\n\n### Output\n\nEach run produces two outputs:\n\n#### Console summary\n\n```text\n=== Evaluation Summary ===\nTotal queries tested: 100\nSuccessful queries: 100\nFailed queries: 0\n\n=== AGGREGATE METRICS ===\nDebug counts:\nTotal successful: 100\nCORRECT: 92\nINCORRECT: 7\nNOT_ATTEMPTED: 1\n{\n \"correct_rate\": 0.92,\n \"incorrect_rate\": 0.07,\n \"not_attempted_rate\": 0.01,\n \"answer_rate\": 0.99,\n \"accuracy\": 0.9292929292929293,\n \"f1\": 0.9246231155778895\n}\n========================\nAccuracy: 0.929\nF1 Score: 0.925\n\nTotal cost: $9.6000\nAverage cost per query: $0.0960\nLatency — avg: 48.3s p50: 45.1s p95: 89.2s\n\nStructured log saved → evals/simple_evals/logs/eval_2025-02-22_10-32-11_n100.jsonl\n```\n\n**Structured JSON log** (`logs/eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`)\n\nSee [`example_output.json`](simple_evals/example_output.json) for the full schema. Key fields:\n\n```\nrun_metadata → timestamp, git_commit, grader_model, researcher_model\naggregate_metrics → accuracy, f1, cost, avg/p50/p95 latency, success/fail counts\nresults → per-query list with grade, score, latency, cost, sources\n```\n\n## Quality Evaluation (`quality_eval/`)\n\nMeasures the structural and content quality of research reports without requiring ground-truth answers. All metrics run on any open-ended query.\n\nAll metric logic lives in `metrics.py`. The runner is `run_eval.py`.\n\n### Metrics\n\n| Metric | Description | LLM calls | Cost |\n| --- | --- | --- | --- |\n| **Citation Faithfulness** | Are sources listed in the report's `## References` actually cited in the body? (catches decorative refs) | 0 | $0 |\n| **Source Diversity** | Domain variety via ratio + Shannon entropy | 0 | $0 |\n| **Source Authority** | Heuristic score per domain (.gov=1.0 → .com=0.5); unknown domains optionally LLM-scored | 0–1 | $0–$0.01 |\n| **Subtopic Coverage** | LLM-judged coverage of expected subtopics (adaptive count) | 2 | ~$0.02/query |\n| **Hallucination** | Binary faithfulness check — report vs. scraped sources | 1 | ~$0.01/query |\n| **Unsupported Claim Rate** | Claim-level credibility: supported (1.0) / inferred (0.3–0.9) / unsupported (0.0) | 2 | ~$0.02/query |\n\n**Citation Faithfulness** measures whether sources the report *lists* in its `## References` are *actually cited* in the body (a low score flags decorative references). It deliberately does not reward raw coverage — reading many sources and citing only the useful ones is good practice, like a real researcher.\n\n**Unsupported Claim Rate** distinguishes between:\n\n- `supported` — claim directly found in source text (score 1.0)\n- `inferred` — reasonable synthesis from multiple sources; never applied to data claims (score 0.3–0.9)\n- `unsupported` — no traceable basis in sources (score 0.0)\n\n### Metric Grouping & Tooling\n\nThe metrics group into three families: **Source Quality** (diversity, authority), **Writing Behavior** (citation faithfulness, subtopic coverage), and **Hallucination** (hallucination check, unsupported claim) — i.e. *is the input good*, *how was it written*, *is it correct*.\n\nTwo tools complement `run_eval.py` (shared metric logic lives in `metrics.py`):\n\n- **`benchmark.py`** — runs multiple researcher configurations (single- vs. multi-agent × model) over a fixed topic set and prints a side-by-side table. `--compare LOG_A LOG_B` diffs two past runs; `--summary-only` re-prints from saved logs.\n- **`perturbation.py`** — a behavioral test that deliberately degrades a report (remove inline citations, swap sources to low-authority, corrupt claims) and checks each metric responds monotonically. This validates metric *reliability* (directional sensitivity), which is separate from whether a metric captures *quality*.\n\n### Running\n\n```bash\n# Run all 6 metrics (most expensive, ~$0.06/query)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost only (citation ratio, diversity, authority)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Skip individual LLM metrics selectively\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic\npython -m evals.quality_eval.run_eval --num_examples 10 --no-hallucination\npython -m evals.quality_eval.run_eval --num_examples 10 --no-unsupported-claim\n```\n\nLogs are saved to `evals/quality_eval/logs/quality_eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`.\n\n### Example Output\n\n```text\nQuality eval: 5 queries subtopic=off hallucination=off unsupported_claim=on\n\n>> Query: What legal frameworks govern cross-border data transfers?\n citation_faithful: 0.95 (1 listed-only)\n diversity_ratio: 1.00 entropy=4.32\n authority_score: 0.62\n avg_claim_score: 0.79 (unsupported=0.11 inferred=0.22 n=9)\n latency=37.2s cost=$0.13\n\n=== QUALITY EVAL SUMMARY ===\n avg_citation_faithfulness: 0.94\n avg_diversity_ratio: 0.796\n avg_domain_entropy: 3.525\n avg_authority_score: 0.597\n avg_claim_score: 0.428\n avg_unsupported_claim_rate: 0.535\n avg_inferred_claim_rate: 0.148\n avg_latency_seconds: 33.6\n avg_cost: 0.116\n```\n\n### Dependencies\n\n```bash\npip install langchain-openai python-dotenv\n\n# Hallucination metric only:\npip install judges\n```\n\nEnvironment variables required: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n\n---\n\n## Hallucination Evaluation (`hallucination_eval/`)\n\nThe `hallucination_eval` directory contains tools for evaluating GPT-Researcher's outputs for hallucination. This evaluation system compares the generated research reports against their source materials to detect non-factual or hallucinated content, ensuring the reliability and accuracy of the research outputs.\n\n### Components\n\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `evaluate.py`: Core evaluation logic for detecting hallucinations\n- `inputs/`: Directory containing test queries\n - `search_queries.jsonl`: Collection of research queries for evaluation\n- `results/`: Directory containing evaluation results\n - `evaluation_records.jsonl`: Detailed per-query evaluation records\n - `aggregate_results.json`: Summary metrics across all evaluations\n\n### Features\n\n- Evaluates research reports against source materials\n- Provides detailed reasoning for hallucination detection\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/hallucination_eval\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py -n \n```\n\nThe `-n` parameter determines how many queries to evaluate from the test set (default: 1).\n\n### Sample Result\n\n```json\n{\n \"total_queries\": 1,\n \"successful_queries\": 1,\n \"total_responses\": 1,\n \"total_evaluated\": 1,\n \"total_hallucinated\": 0,\n \"hallucination_rate\": 0.0,\n \"results\": [\n {\n \"input\": \"What are the latest developments in quantum computing?\",\n \"output\": \"Research report content...\",\n \"source\": \"Source material content...\",\n \"is_hallucination\": false,\n \"confidence_score\": 0.95,\n \"reasoning\": \"The summary accurately reflects the source material with proper citations...\"\n }\n ]\n}\n```\n","evals/context_filter/README.md":"# Context filter benchmark: embeddings vs Jev vs no filter\n\n![Context filter benchmark results](results.png)\n\nEvery scraped page reaches a GPT Researcher report through one step: for each\nsub-query, the scraped pages are cut into chunks and a filter decides which\nchunks become the writer's context. Until now that filter was embedding\nsimilarity. This benchmark asks two questions:\n\n1. Is there a better filter? We test [TypeSafe's Jev](https://docs.typesafe.ai/),\n a model that returns calibrated, typed decisions, scoring each chunk on how\n useful it is for answering the question.\n2. With today's long context windows, do we need a filter at all?\n\n## Method\n\n**Fixed inputs.** `collect.py` runs 28 real research tasks once and records\nthe exact `(sub_query, pages)` pairs that reached the filter: 20 SimpleQA\nquestions with known answers (fixed seed, from `evals/simple_evals`) and 8\nopen-ended research questions. `replay.py` rebuilds the context from those\nsame pages with each strategy and writes the report with the same model\n(`gpt-5.4`). Search and scraping, the noisiest parts of a run, are held\nconstant, so the filter is the only thing that differs.\n\n**Strategies.**\n\n| strategy | what reaches the writer |\n|---|---|\n| `embeddings` | today: 1000-char chunks, cosine similarity > 0.42, top 10 per sub-query |\n| `jev` | same chunks, scored by Jev on a 4-level usefulness rubric, top 10 scoring ≥ 1.5 |\n| `jev_top10` | Jev's 10 best chunks, no threshold (same budget as embeddings) |\n| `jev_large` | Jev on 3000-char chunks, top 4 (≈ same characters, a third of the calls) |\n| `jev_wide` | Jev threshold 1.5, up to 30 chunks |\n| `jev_broad` | Jev on 3000-char chunks, threshold 1.0, up to 12 (open-ended only) |\n| `bm25` | keyword (BM25) ranking of the same chunks, top 10; runs locally, no API or model |\n| `bm25_rel` | BM25, keeping chunks scoring ≥ 50% of the best chunk, top 10 |\n| `bm25_wide` | BM25 with the same relative threshold, up to 25 chunks (shipped as `keyword`) |\n| `none_capped` | no ranking: pages in search order, capped at 48k characters per sub-query |\n| `none` | no filter: every page in full (capped at 50k chars per page, as today) |\n| `truncate` | control: 10 chunks round-robin across pages, no relevance ranking |\n\n**Quality signals** (`judge.py`):\n\n- **SimpleQA accuracy**: each report graded with the repo's SimpleQA grader.\n- **Context precision**: the share of context chunks an independent judge\n (`gpt-5.4-mini`) rates as relevant to the question. This measures the filter\n itself. It isn't defined for `none`.\n- **Pairwise preference**: a blind head-to-head judged by `gpt-5.4`, run in both\n orders. A win counts only if the strategy wins in *both* orders; any\n disagreement is a tie.\n\n## Results\n\n### Cost and speed (28 questions)\n\n| strategy | filter time p50 / p90 | filter $ | context tokens p50 (max) | write $ | total $ / report |\n|---|---|---|---|---|---|\n| embeddings | 1.0s / 1.3s | 0.0028 | 7.6k (10.7k) | 0.114 | **0.117** |\n| jev | 2.4s / 6.0s ¹ | 0.0054 | 4.5k (10.6k) | 0.110 | **0.115** |\n| jev_top10 | 2.7s / 6.2s | 0.0054 | 8.6k (10.6k) | 0.119 | 0.124 |\n| jev_large | 1.2s / 5.1s | 0.0031 | 6.5k | 0.111 | 0.114 |\n| truncate | 0.0s | — | 8.0k (11.0k) | 0.117 | 0.117 |\n| none | 0.0s | — | **33.4k (107.7k)** | **0.192** | **0.192** |\n\n¹ Measured with 32 concurrent Jev calls. At 64, the new default, the same\nfiltering takes **1.7s p50 / 3.6s p90**; 128 was no faster.\n\nReport writing takes the same ~45s for every strategy: generating the output\ndominates, and input size barely matters. Removing the filter makes nothing\nfaster; it makes each report 65% more expensive.\n\n### Quality\n\n| strategy | SimpleQA correct | context precision | vs embeddings (W/T/L) | vs none (W/T/L) |\n|---|---|---|---|---|\n| embeddings | 18/20 | 0.46 | — | 3 / 12 / 13 |\n| **jev** | 19/20 | **0.73** | **15 / 10 / 3** | 6 / 12 / 10 |\n| jev_top10 | 19/20 | 0.50 | 13 / 10 / 5 | — |\n| jev_large | 19/20 | 0.72 | 11 / 10 / 7 | 8 / 7 / 13 |\n| jev_wide | 20/20 | **0.74** | 13 / 8 / 7 | 7 / 11 / 10 |\n| truncate | 19/20 | 0.33 | 7 / 8 / 13 | — |\n| none | 20/20 | — | 13 / 11 / 4 | — |\n\n### Open-ended questions only (8)\n\n| strategy | context p50 | total $ / report | vs embeddings | vs none |\n|---|---|---|---|---|\n| embeddings | 8.8k | 0.155 | — | 0 / 1 / 7 |\n| jev | 9.1k | 0.160 | 5 / 2 / 1 | 0 / 2 / 6 |\n| jev_broad | 26.9k | 0.191 | **7 / 1 / 0** | 0 / 3 / 5 |\n| none | 61.3k | **0.284** | 8 / 0 / 0 | — |\n\nThe `none` reports on open-ended questions were *shorter* than the embeddings\nreports (2,099 vs 2,416 median words) and still preferred, so the judge is not\nrewarding length.\n\n### Filters that need nothing: keyword (BM25)\n\nWithout a Jev key, GPT Researcher needs a filter with no API, model or\nembeddings. The candidates, on the same 28 runs:\n\n| strategy | SimpleQA | context precision | vs embeddings (W/T/L) | vs none (W/T/L) | filter p50 | context p50 | total $ |\n|---|---|---|---|---|---|---|---|\n| bm25 | 18/20 | 0.40 | 7 / 9 / 12 | 3 / 12 / 13 | 0.02s | 8.1k | 0.118 |\n| bm25_rel | 19/20 | 0.53 | 7 / 13 / 8 | 5 / 10 / 13 | 0.02s | 4.7k | **0.107** |\n| **bm25_wide** | 19/20 | 0.51 | **14 / 8 / 6** | 6 / 9 / 13 | 0.02s | 6.4k | 0.116 |\n| none_capped | 20/20 | 0.54 | 11 / 13 / 4 | 6 / 10 / 12 | 0.00s | 23.8k | 0.152 |\n| *embeddings* | *18/20* | *0.46* | — | *3 / 12 / 13* | *1.04s* | *7.6k* | *0.117* |\n\n`bm25_wide` ships as the `keyword` fallback: at least as good as embeddings on\nevery measure, at the same cost, about 50× faster to filter, and with nothing\nto set up. Its 14–6 head-to-head is directional (sign test p ≈ 0.12), not as\nstrong as Jev's 15–3. As with Jev, the relative threshold is what does the\nwork: plain top-10 BM25 loses to embeddings 7–12.\n\nGPT Researcher removes duplicate URLs across sub-queries before scraping\n(0 of 300 pages recurred), so \"sources that came back most often\" isn't a\nsignal available at this step.\n\n## What this shows\n\n1. **Jev is a better filter than embeddings at the same cost.** 73% of the\n chunks Jev selects are relevant, against 46% for embeddings; the unranked\n control gets 33%. Jev reports beat embeddings reports 15–3 head-to-head\n (sign test on the 18 non-ties, p ≈ 0.008). Every Jev variant beat\n embeddings. Total cost per report is the same; both filters cost well\n under a cent.\n2. **Most of the gain comes from the threshold.** `jev_top10`, forced to fill\n 10 slots, drops to 0.50 precision. Cosine similarity cannot say \"nothing\n else here is worth including\"; a calibrated usefulness score can.\n3. **Skipping the filter gives the best open-ended reports, at 65–83% higher\n cost.** More source material makes for broader reports. But on a *standard*\n research report the context already reaches 108k tokens; detailed and deep\n research run many more sub-queries and would overflow context windows and\n budgets. For fact-finding questions it adds nothing: SimpleQA is at\n ceiling for every strategy, and Jev vs none is 6–10–4.\n4. **No integration is needed for a good filter.** Keyword ranking with a\n relative threshold matches or beats embeddings at the same cost (above),\n so the default without a Jev key is `keyword`, and nothing requires an\n embeddings provider.\n5. **SimpleQA is saturated.** 18–20 of 20 for every strategy: when search\n finds the fact, any reasonable filter keeps it. Differences of one question\n are noise.\n\n## Caveats\n\n- 28 questions, judged by one model family (`gpt-5.4`), which also writes the\n reports. Self-preference would apply to every strategy equally, but the\n pairwise numbers are directional evidence, not a leaderboard.\n- One writer model and one report type (`research_report`). Detailed and deep\n research were not replayed.\n- Jev latency depends on chunk count and concurrency. `jev_large` matches\n embeddings at the median; tails come from large pages.\n\n## Reproduce\n\n```bash\nexport OPENAI_API_KEY=... TAVILY_API_KEY=... TYPESAFE_API_KEY=...\npython -m evals.context_filter.collect --out runs/ # ~$0.40, ~20 min\npython -m evals.context_filter.replay --runs runs/ --out results/\npython -m evals.context_filter.judge --results results/\npython -m evals.context_filter.judge --results results/ --baseline none --only jev embeddings\npython -m evals.context_filter.replay --runs runs/ --out results/ --strategies bm25 bm25_rel bm25_wide none_capped\n```\n\n`results/` in this folder holds the scores and per-run metrics from the run\nabove (`runs.json`, `scored_*.json`, `summary_*.json`). Scraped pages and\ngenerated reports are not committed.\n\nTotal spend: about $40 in tracked writing and filtering across all strategies\n(of which $0.59 was Jev; keyword and none cost nothing to filter), plus an\nestimated $8–15 for judging.\n","evals/quality_eval/README.md":"# quality_eval — Report Quality Evaluation for Autonomous Research Agents\n\nEvaluate the quality of a GPT-Researcher report on **any open-ended query, with no\nground-truth answer required**. Where `simple_evals` needs a labelled dataset and\n`hallucination_eval` gives a binary check, `quality_eval` scores a report across\nseveral interpretable dimensions of *how well it was researched and written*.\n\nIt follows the metric abstraction of established frameworks (DeepEval / RAGAS),\nand **extends it for autonomous research** — evaluating not just the answer, but\nthe quality of the sources the agent found on its own.\n\n## What it measures\n\n| Group | Aligns with | Metric | Question it answers |\n|---|---|---|---|\n| **Faithfulness** | RAGAS Faithfulness | `unsupported_claim` | Are the report's claims grounded in the sources? |\n| **Answer Relevancy** | RAGAS answer relevancy | `subtopic_coverage` | Does it cover the subtopics a thorough report should? |\n| **Source Quality** | *extension (autonomous agent)* | `source_authority`, `source_diversity` | Are the sources the agent found authoritative and varied? |\n| **Citation** | ALCE citation precision | `citation_faithfulness` | Are the listed references actually cited in the body? |\n\nZero-cost metrics (citation, diversity, authority) run by default; the two LLM\nmetrics are opt-out via flags, so you can trade cost for depth.\n\n## Concept alignment (why this is more than ad-hoc metrics)\n\nThe suite maps onto standard evaluation ontologies where they apply, and marks\nwhere autonomous research needs more:\n\n- **Faithfulness / Answer Relevancy** ← align with **RAGAS**. `unsupported_claim`\n even uses the field's standard three-way support scale (supported / inferred /\n unsupported), matching ALCE-style *full / partial / no support*.\n- **Citation** ← aligns with **ALCE citation precision** (of the references it\n lists, how many are actually used). Current implementation is *structural*; a\n *semantic* version (does the citation truly support its claim, NLI-style) is on\n the roadmap.\n- **Source Quality** ← a deliberate **extension**. RAG-oriented frameworks assume\n the context is *given*, so they have no notion of \"source quality\". A research\n agent finds its own sources, so authority and diversity of those sources become\n first-class metrics — this is the dimension standard RAG eval can't express.\n\n## Architecture\n\nA thin, dependency-free framework layer (mirrors DeepEval's `BaseMetric` +\ntest-case design):\n\n```text\nbase.py BaseMetric / EvalSample / MetricResult — abstractions (+ `sources`, the RAG-framework gap)\nsuite.py 5 metric wrappers + evaluate() entry — standardized I/O + concept-alignment metadata\nmetrics.py raw metric functions — pure logic (unit-tested, perturbation-validated)\nperturbation.py behavioral reliability test — graded degradation, asserts monotonic response\nrun_eval.py / benchmark.py — runners (single query / multi-config comparison)\n```\n\nEvery metric declares its `group` and the standard concept it `aligned_with`, so\nthe ontology above is encoded in code, not just docs. `MetricResult.score` is\nalways *higher = better* (0–1) so results aggregate and compare uniformly.\n\n## Reliability (perturbation-validated)\n\nThe metrics aren't just implemented — they're checked for *reliability*.\n`perturbation.py` deliberately degrades a report (remove citations, swap to\nlow-authority sources, corrupt claims) in graded steps and asserts each metric\nresponds **monotonically**. This surfaced and fixed real defects that mocked unit\ntests and final outputs couldn't reveal — e.g. batch LLM scoring was letting one\nclaim contaminate another's score, fixed by scoring each claim independently.\n\n## Usage\n\n```bash\n# Full evaluation (all metrics)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost metrics only\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-unsupported-claim\n\n# Reliability check\npython -m evals.quality_eval.perturbation\n\n# Compare configs / runs\npython -m evals.quality_eval.benchmark --configs single-gpt4o single-gpt4o-mini\npython -m evals.quality_eval.benchmark --compare LOG_A LOG_B\n```\n\nProgrammatic use via the standardized entry point:\n\n```python\nfrom evals.quality_eval.base import EvalSample\nfrom evals.quality_eval.suite import evaluate, default_metrics\n\nsample = EvalSample(query=q, report=report, sources=urls, context=ctx)\nresults = await evaluate(sample, default_metrics(), grader_model)\nfor r in results:\n print(r.group, r.name, r.score) # typed, tagged, higher = better\n```\n\n## Roadmap\n\n- **Semantic citation precision** (does a citation truly support its claim,\n NLI/LLM-judge) → promote the `Citation` group to a full `Precision` family.\n- **True answer relevancy** (on-topic-ness), complementing `subtopic_coverage`'s\n completeness view.\n- **Component-level evaluation** of the multi-agent pipeline (planner / researcher\n / writer / reviewer) to locate bottlenecks — the natural next phase.\n\nDependencies: `langchain-openai`, `python-dotenv` (+ `judges` for the optional\nhallucination check). Env: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n","evals/simple_evals/logs/README.md":"# Evaluation Results\n\nThis directory contains historical evaluation results for GPT-Researcher using the SimpleQA methodology.\n\n## Latest Results\n\n### [SimpleQA Eval 100 Problems 2-22-25](./SimpleQA%20Eval%20100%20Problems%202-22-25.txt)\n\nEvaluation run by [Kelly Abbott (kga245)](https://github.com/kga245)\n\n**Summary:**\n- Date: February 22, 2025\n- Sample Size: 100 queries\n- Success Rate: 100% (100/100 queries completed)\n\n**Performance Metrics:**\n- Accuracy: 92.9%\n- F1 Score: 92.5%\n- Answer Rate: 99%\n\n**Response Distribution:**\n- Correct: 92%\n- Incorrect: 7%\n- Not Attempted: 1%\n\n**Cost Efficiency:**\n- Total Cost: $9.60\n- Average Cost per Query: $0.096\n\nThis evaluation demonstrates strong performance in factual accuracy while maintaining reasonable cost efficiency. The high answer rate (99%) and accuracy (92.9%) suggest that GPT-Researcher is effective at finding and reporting accurate information.\n\n## Historical Context\n\nThese logs are maintained in version control to:\n1. Track performance improvements over time\n2. Provide benchmarks for future enhancements\n3. Enable analysis of different configurations\n4. Ensure transparency in our evaluation process\n\nEach log file contains detailed information about:\n- Individual query results\n- Source citations\n- Cost breakdowns\n- Error analysis\n- Aggregate metrics\n\n## Running New Evaluations\n\nTo generate new evaluation logs, see the [main evaluation documentation](../README.md) for instructions on running evaluations with different configurations or sample sizes. ","skills/gpt-researcher/SKILL.md":"---\nname: gpt-researcher\ndescription: Autonomous deep research from Codex via MCP\n---\n\n# GPT Researcher for Codex\n\nUse GPT Researcher from Codex via MCP.\n\n## When to use\n- When you need gpt-researcher capabilities in your Codex workflow\n- See https://github.com/assafelovic/gpt-researcher for full setup instructions\n"},"files":{".claude/SKILL.md":"---\nname: gpt-researcher\ndescription: GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines.\n---\n\n# GPT Researcher Development Skill\n\nGPT Researcher is an LLM-based autonomous agent using a planner-executor-publisher pattern with parallelized agent work for speed and reliability.\n\n## Quick Start\n\n### Basic Python Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\", # or detailed_report, deep, outline_report\n report_source=\"web\", # or local, hybrid\n )\n await researcher.conduct_research()\n report = await researcher.write_report()\n print(report)\n\nasyncio.run(main())\n```\n\n### Run Servers\n\n```bash\n# Backend\npython -m uvicorn backend.server.server:app --reload --port 8000\n\n# Frontend\ncd frontend/nextjs && npm install && npm run dev\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes |\n|------|--------------|-------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| Search engines | `gpt_researcher/retrievers/` | Various retrievers |\n\n---\n\n## Architecture Overview\n\n```\nUser Query → GPTResearcher.__init__()\n │\n ▼\n choose_agent() → (agent_type, role_prompt)\n │\n ▼\n ResearchConductor.conduct_research()\n ├── plan_research() → sub_queries\n ├── For each sub_query:\n │ └── _process_sub_query() → context\n └── Aggregate contexts\n │\n ▼\n [Optional] ImageGenerator.plan_and_generate_images()\n │\n ▼\n ReportGenerator.write_report() → Markdown report\n```\n\n**For detailed architecture diagrams**: See [references/architecture.md](references/architecture.md)\n\n---\n\n## Core Patterns\n\n### Adding a New Feature (8-Step Pattern)\n\n1. **Config** → Add to `gpt_researcher/config/variables/default.py`\n2. **Provider** → Create in `gpt_researcher/llm_provider/my_feature/`\n3. **Skill** → Create in `gpt_researcher/skills/my_feature.py`\n4. **Agent** → Integrate in `gpt_researcher/agent.py`\n5. **Prompts** → Update `gpt_researcher/prompts.py`\n6. **WebSocket** → Events via `stream_output()`\n7. **Frontend** → Handle events in `useWebSocket.ts`\n8. **Docs** → Create `docs/docs/gpt-researcher/gptr/my_feature.md`\n\n**For complete feature addition guide with Image Generation case study**: See [references/adding-features.md](references/adding-features.md)\n\n### Adding a New Retriever\n\n```python\n# 1. Create: gpt_researcher/retrievers/my_retriever/my_retriever.py\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n \n async def search(self, max_results: int = 10) -> list[dict]:\n # Return: [{\"title\": str, \"href\": str, \"body\": str}]\n pass\n\n# 2. Register in gpt_researcher/actions/retriever.py\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n\n# 3. Export in gpt_researcher/retrievers/__init__.py\n```\n\n**For complete retriever documentation**: See [references/retrievers.md](references/retrievers.md)\n\n---\n\n## Configuration\n\nConfig keys are **lowercased** when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\nPriority: Environment Variables → JSON Config File → Default Values\n\n**For complete configuration reference**: See [references/config-reference.md](references/config-reference.md)\n\n---\n\n## Common Integration Points\n\n### WebSocket Streaming\n\n```python\nclass WebSocketHandler:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=WebSocketHandler())\n```\n\n### MCP Data Sources\n\n```python\nresearcher = GPTResearcher(\n query=\"Open source AI projects\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\", # or \"fast\", \"disabled\"\n)\n```\n\n**For MCP integration details**: See [references/mcp.md](references/mcp.md)\n\n### Deep Research Mode\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers recursive tree-like exploration\n)\n```\n\n**For deep research configuration**: See [references/deep-research.md](references/deep-research.md)\n\n---\n\n## Error Handling\n\nAlways use graceful degradation in skills:\n\n```python\nasync def execute(self, ...):\n if not self.is_enabled():\n return [] # Don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n await stream_output(\"logs\", \"error\", f\"⚠️ {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n---\n\n## Critical Gotchas\n\n| ❌ Mistake | ✅ Correct |\n|-----------|-----------|\n| `config.MY_VAR` | `config.my_var` (lowercased) |\n| Editing pip-installed package | `pip install -e .` |\n| Forgetting async/await | All research methods are async |\n| `websocket.send_json()` on None | Check `if websocket:` first |\n| Not registering retriever | Add to `retriever.py` match statement |\n\n---\n\n## Reference Documentation\n\n| Topic | File |\n|-------|------|\n| System architecture & diagrams | [references/architecture.md](references/architecture.md) |\n| Core components & signatures | [references/components.md](references/components.md) |\n| Research flow & data flow | [references/flows.md](references/flows.md) |\n| Prompt system | [references/prompts.md](references/prompts.md) |\n| Retriever system | [references/retrievers.md](references/retrievers.md) |\n| MCP integration | [references/mcp.md](references/mcp.md) |\n| Deep research mode | [references/deep-research.md](references/deep-research.md) |\n| Multi-agent system | [references/multi-agents.md](references/multi-agents.md) |\n| Adding features guide | [references/adding-features.md](references/adding-features.md) |\n| Advanced patterns | [references/advanced-patterns.md](references/advanced-patterns.md) |\n| REST & WebSocket API | [references/api-reference.md](references/api-reference.md) |\n| Configuration variables | [references/config-reference.md](references/config-reference.md) |\n",".claude/references/adding-features.md":"# Adding Features Guide\n\n## Table of Contents\n- [The 8-Step Pattern](#the-8-step-pattern)\n- [Image Generation Case Study](#image-generation-case-study)\n- [Testing New Features](#testing-new-features)\n\n---\n\n## The 8-Step Pattern\n\n```\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│1.CONFIG│ → │2.PROVIDER│ → │3.SKILL │ → │4.AGENT │\n└────────┘ └────────┘ └────────┘ └────────┘\n ↓ ↓ ↓ ↓\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│5.PROMPTS│ → │6.WEBSOCKET│→ │7.FRONTEND│→ │8.DOCS │\n└────────┘ └────────┘ └────────┘ └────────┘\n```\n\n### Step 1: Add Configuration\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n \"MY_FEATURE_ENABLED\": False,\n \"MY_FEATURE_MODEL\": \"model-name\",\n \"MY_FEATURE_MAX_ITEMS\": 3,\n}\n```\n\n**File:** `gpt_researcher/config/variables/base.py`\n\n```python\nclass BaseConfig(TypedDict):\n \"MY_FEATURE_ENABLED\": bool\n \"MY_FEATURE_MODEL\": Union[str, None]\n \"MY_FEATURE_MAX_ITEMS\": int\n```\n\n### Step 2: Create Provider\n\n**File:** `gpt_researcher/llm_provider/my_feature/my_provider.py`\n\n```python\nclass MyFeatureProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"MY_API_KEY\")\n self.model = model\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n async def execute(self, input_data: str) -> Dict[str, Any]:\n # API implementation\n pass\n```\n\nExport in `gpt_researcher/llm_provider/__init__.py`.\n\n### Step 3: Create Skill\n\n**File:** `gpt_researcher/skills/my_feature.py`\n\n```python\nclass MyFeatureSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.provider = MyFeatureProvider(...)\n \n def is_enabled(self) -> bool:\n return getattr(self.config, 'my_feature_enabled', False) and self.provider.is_enabled()\n \n async def execute(self, context: str, query: str) -> List[Dict]:\n if not self.is_enabled():\n return []\n \n await stream_output(\"logs\", \"my_feature_start\", \"🚀 Starting...\", self.researcher.websocket)\n results = await self.provider.execute(context)\n await stream_output(\"logs\", \"my_feature_complete\", \"✅ Done\", self.researcher.websocket)\n \n return results\n```\n\nExport in `gpt_researcher/skills/__init__.py`.\n\n### Step 4: Integrate into Agent\n\n**File:** `gpt_researcher/agent.py`\n\n```python\ndef __init__(self, ...):\n if self.cfg.my_feature_enabled:\n from gpt_researcher.skills import MyFeatureSkill\n self.my_feature = MyFeatureSkill(self)\n else:\n self.my_feature = None\n self.my_feature_results = []\n\nasync def conduct_research(self, ...):\n # ... existing ...\n if self.my_feature and self.my_feature.is_enabled():\n self.my_feature_results = await self.my_feature.execute(self.context, self.query)\n```\n\n### Step 5: Update Prompts\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_my_feature_prompt(context: str, query: str) -> str:\n return f\"\"\"...\"\"\"\n```\n\n### Step 6: WebSocket Events\n\nAlready handled via `stream_output()` in skill.\n\n### Step 7: Frontend (if needed)\n\n**File:** `frontend/nextjs/hooks/useWebSocket.ts`\n\n```typescript\nif (data.content === 'my_feature_start') {\n setStatus('processing');\n}\n```\n\n### Step 8: Documentation\n\nCreate `docs/docs/gpt-researcher/gptr/my_feature.md`.\n\n---\n\n## Image Generation Case Study\n\nThis section shows the **actual implementation** of the Image Generation feature as a reference.\n\n### 1. Configuration Added\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n # ... existing ...\n \"IMAGE_GENERATION_MODEL\": \"models/gemini-2.5-flash-image\",\n \"IMAGE_GENERATION_MAX_IMAGES\": 3,\n \"IMAGE_GENERATION_ENABLED\": False,\n \"IMAGE_GENERATION_STYLE\": \"dark\", # dark, light, auto\n}\n```\n\n### 2. Provider Created\n\n**File:** `gpt_researcher/llm_provider/image/image_generator.py`\n\n```python\nclass ImageGeneratorProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"GOOGLE_API_KEY\")\n self.model = model or \"models/gemini-2.5-flash-image\"\n self._client = None\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n def _build_enhanced_prompt(self, prompt: str, context: str = \"\", style: str = \"dark\") -> str:\n \"\"\"Add styling instructions to prompt.\"\"\"\n if style == \"dark\":\n style_instructions = \"\"\"\n Style: Dark mode professional infographic\n - Background: Dark (#0d1117)\n - Accents: Teal/cyan (#14b8a6)\n - Clean, modern, minimalist\n \"\"\"\n # ... handle light, auto\n return f\"{style_instructions}\\n\\nCreate: {prompt}\\n\\nContext: {context}\"\n \n async def generate_image(\n self,\n prompt: str,\n context: str = \"\",\n research_id: str = \"\",\n style: str = \"dark\",\n ) -> List[Dict[str, Any]]:\n \"\"\"Generate image using Gemini.\"\"\"\n full_prompt = self._build_enhanced_prompt(prompt, context, style)\n \n # Call Gemini API\n response = await self._generate_with_gemini(full_prompt, output_path, ...)\n \n return [{\"url\": f\"/outputs/images/{research_id}/img_{hash}.png\", ...}]\n```\n\n### 3. Skill Created\n\n**File:** `gpt_researcher/skills/image_generator.py`\n\n```python\nclass ImageGenerator:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.image_provider = ImageGeneratorProvider(\n api_key=os.getenv(\"GOOGLE_API_KEY\"),\n model=getattr(self.config, 'image_generation_model', None),\n )\n self.max_images = getattr(self.config, 'image_generation_max_images', 3)\n self.style = getattr(self.config, 'image_generation_style', 'dark')\n \n def is_enabled(self) -> bool:\n enabled = getattr(self.config, 'image_generation_enabled', False)\n return enabled and self.image_provider.is_enabled()\n \n async def plan_and_generate_images(\n self,\n research_context: str,\n research_query: str,\n research_id: str,\n websocket: Any,\n ) -> List[Dict[str, Any]]:\n \"\"\"\n 1. Use LLM to identify visual concepts from context\n 2. Generate images in parallel\n 3. Return list of image metadata\n \"\"\"\n # Stream progress\n await stream_output(\"logs\", \"image_planning\", \"🎨 Planning images...\", websocket)\n \n # LLM identifies concepts\n concepts = await self._plan_image_concepts(research_context, research_query)\n \n # Generate images in parallel\n generated_images = []\n for i, concept in enumerate(concepts[:self.max_images]):\n await stream_output(\"logs\", \"image_generating\", \n f\"🖼️ Generating image {i+1}/{len(concepts)}...\", websocket)\n \n images = await self.image_provider.generate_image(\n prompt=concept[\"prompt\"],\n context=concept.get(\"context\", \"\"),\n research_id=research_id,\n style=self.style,\n )\n generated_images.extend(images)\n \n await stream_output(\"logs\", \"images_ready\", \n f\"✅ Generated {len(generated_images)} images\", websocket)\n \n return generated_images\n```\n\n### 4. Agent Integration\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nclass GPTResearcher:\n def __init__(self, ...):\n # ... existing init ...\n \n # Initialize image generator if enabled\n if self.cfg.image_generation_enabled:\n from gpt_researcher.skills import ImageGenerator\n self.image_generator = ImageGenerator(self)\n else:\n self.image_generator = None\n \n self.available_images: List[Dict[str, Any]] = []\n self.research_id = self._generate_research_id(query)\n \n async def conduct_research(self, on_progress=None):\n # ... existing research ...\n \n self.context = await self.research_conductor.conduct_research()\n \n # Pre-generate images after research, before report writing\n if self.cfg.image_generation_enabled and self.image_generator and self.image_generator.is_enabled():\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n \n async def write_report(self, ...):\n report = await self.report_generator.write_report(\n # ... existing params ...\n available_images=self.available_images, # Pass to report writer\n )\n return report\n```\n\n### 5. Prompt Updated\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_report_prompt(..., available_images: List[Dict[str, Any]] = []):\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title', 'Untitled')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES - Embed where relevant using ![Title](URL):\n{image_list}\n\"\"\"\n \n return f\"\"\"...(existing prompt)...\n{image_instruction}\n\"\"\"\n```\n\n---\n\n## Testing New Features\n\n```python\n# tests/test_my_feature.py\nimport pytest\nfrom gpt_researcher import GPTResearcher\n\n@pytest.mark.asyncio\nasync def test_my_feature_disabled():\n \"\"\"Test that feature is skipped when disabled.\"\"\"\n researcher = GPTResearcher(query=\"test\")\n # MY_FEATURE_ENABLED defaults to False\n assert researcher.my_feature is None\n\n@pytest.mark.asyncio\nasync def test_my_feature_enabled(monkeypatch):\n \"\"\"Test feature execution when enabled.\"\"\"\n monkeypatch.setenv(\"MY_FEATURE_ENABLED\", \"true\")\n monkeypatch.setenv(\"MY_API_KEY\", \"test-key\")\n \n researcher = GPTResearcher(query=\"test\")\n assert researcher.my_feature is not None\n assert researcher.my_feature.is_enabled()\n```\n\n### Running Tests\n\n```bash\n# All tests\npython -m pytest tests/\n\n# Specific test\npython -m pytest tests/test_my_feature.py -v\n\n# With coverage\npython -m pytest tests/ --cov=gpt_researcher\n```\n",".claude/references/advanced-patterns.md":"# Advanced Patterns Reference\n\n## Table of Contents\n- [Custom Callbacks](#custom-callbacks)\n- [Custom WebSocket Handler](#custom-websocket-handler)\n- [LangChain Integration](#langchain-integration)\n- [Search Restrictions](#search-restrictions)\n- [Error Handling Patterns](#error-handling-patterns)\n\n---\n\n## Custom Callbacks\n\n```python\ndef cost_callback(cost: float):\n print(f\"API call cost: ${cost}\")\n\nresearcher = GPTResearcher(query=\"...\")\nresearcher.add_costs = cost_callback # Override cost tracking\n```\n\n---\n\n## Custom WebSocket Handler\n\n```python\nclass CustomWebSocket:\n def __init__(self):\n self.messages = []\n \n async def send_json(self, data):\n self.messages.append(data)\n if data['type'] == 'logs':\n print(f\"Progress: {data['output']}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=CustomWebSocket())\n```\n\n---\n\n## LangChain Integration\n\n### Using with LangChain Documents\n\n```python\nfrom langchain.document_loaders import DirectoryLoader\n\nloader = DirectoryLoader('./docs', glob=\"**/*.md\")\ndocuments = loader.load()\n\nresearcher = GPTResearcher(\n query=\"Summarize the documentation\",\n report_source=\"langchain_documents\",\n documents=documents,\n)\n```\n\n### Using with Vector Store\n\n```python\nfrom langchain.vectorstores import Chroma\n\nvectorstore = Chroma.from_documents(documents, embeddings)\n\nresearcher = GPTResearcher(\n query=\"Find relevant information\",\n report_source=\"langchain_vectorstore\",\n vector_store=vectorstore,\n vector_store_filter={\"source\": \"docs\"},\n)\n```\n\n---\n\n## Search Restrictions\n\n### Restricting Search Domains\n\n```python\nresearcher = GPTResearcher(\n query=\"Company news\",\n query_domains=[\"reuters.com\", \"bloomberg.com\", \"wsj.com\"],\n)\n```\n\n### Using Specific Source URLs\n\n```python\nresearcher = GPTResearcher(\n query=\"Analyze these articles\",\n source_urls=[\n \"https://example.com/article1\",\n \"https://example.com/article2\",\n ],\n complement_source_urls=True, # Also do web search\n)\n```\n\n---\n\n## Error Handling Patterns\n\n### Graceful Degradation\n\n```python\n# In skills, always check is_enabled()\nasync def execute(self, ...):\n if not self.is_enabled():\n logger.warning(\"Feature not enabled, skipping\")\n return [] # Return empty, don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n logger.error(f\"Feature error: {e}\")\n await stream_output(\"logs\", \"feature_error\", f\"⚠️ Error: {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n### API Rate Limiting\n\n```python\n# Providers should handle rate limits\nasync def execute(self, ...):\n try:\n return await self._call_api(...)\n except RateLimitError as e:\n logger.warning(f\"Rate limited, waiting...\")\n await asyncio.sleep(60)\n return await self._call_api(...) # Retry\n```\n\n### WebSocket None Check\n\n```python\n# Always check websocket before sending\nif self.researcher.websocket:\n await stream_output(\"logs\", \"event\", \"message\", self.researcher.websocket)\n```\n",".claude/references/api-reference.md":"# API Reference\n\n## Table of Contents\n- [REST API](#rest-api)\n- [WebSocket API](#websocket-api)\n- [Python Client](#python-client)\n- [Output Files](#output-files)\n\n---\n\n## REST API\n\nBase URL: `http://localhost:8000`\n\n### Generate Report\n\n**POST `/report/`**\n\n```json\n{\n \"task\": \"What are the latest AI developments?\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"query_domains\": [],\n \"generate_in_background\": false\n}\n```\n\n**Response:**\n\n```json\n{\n \"research_id\": \"task_1234567890_query\",\n \"research_information\": {\n \"source_urls\": [\"...\"],\n \"research_costs\": 0.05,\n \"visited_urls\": [\"...\"],\n \"research_images\": [\"...\"]\n },\n \"report\": \"# Research Report\\n\\n...\",\n \"docx_path\": \"outputs/task_123.docx\",\n \"pdf_path\": \"outputs/task_123.pdf\"\n}\n```\n\n### Chat with Report\n\n**POST `/api/chat`**\n\n```json\n{\n \"report\": \"The full report text...\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"What are the key findings?\"}\n ]\n}\n```\n\n### Report Management\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/api/reports` | List all reports |\n| GET | `/api/reports/{id}` | Get single report |\n| POST | `/api/reports` | Create/update report |\n| PUT | `/api/reports/{id}` | Update report |\n| DELETE | `/api/reports/{id}` | Delete report |\n\n### File Operations\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/files/` | List uploaded documents |\n| POST | `/upload/` | Upload document |\n| DELETE | `/files/{filename}` | Delete file |\n| GET | `/outputs/{filename}` | Get output file |\n\n---\n\n## WebSocket API\n\n**Endpoint:** `ws://localhost:8000/ws`\n\n### Send Research Request\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"mcp_enabled\": false,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": []\n}\n```\n\n### Message Types (Server → Client)\n\n| Type | Content | Description |\n|------|---------|-------------|\n| `logs` | `starting_research` | Research initiated |\n| `logs` | `planning_research` | Generating sub-queries |\n| `logs` | `running_subquery_research` | Researching sub-query |\n| `logs` | `research_step_finalized` | Research complete |\n| `logs` | `agent_generated` | Agent role selected |\n| `logs` | `scraping_urls` | Scraping web pages |\n| `logs` | `mcp_optimization` | MCP processing |\n| `logs` | `image_planning` | Planning images |\n| `logs` | `images_ready` | Images generated |\n| `report` | - | Streaming report chunks |\n| `report_complete` | - | Final complete report |\n| `path` | `pdf`, `docx`, `md` | Output file paths |\n| `error` | - | Error messages |\n| `human_feedback` | `request` | Request user input |\n\n### Message Format\n\n```json\n{\n \"type\": \"logs\",\n \"content\": \"starting_research\",\n \"output\": \"🔍 Starting the research task...\",\n \"metadata\": null\n}\n```\n\n### Frontend Handler Example\n\n```typescript\nws.onmessage = (event) => {\n const data = JSON.parse(event.data);\n \n switch (data.type) {\n case 'logs':\n setLogs(prev => [...prev, data]);\n break;\n case 'report':\n setAnswer(prev => prev + data.output);\n break;\n case 'report_complete':\n setAnswer(data.output);\n break;\n case 'path':\n setPaths(prev => ({...prev, [data.content]: data.output}));\n break;\n case 'error':\n setError(data.output);\n break;\n }\n};\n```\n\n---\n\n## Python Client\n\n### Basic Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\",\n )\n \n await researcher.conduct_research()\n report = await researcher.write_report()\n \n print(f\"Report: {report}\")\n print(f\"Costs: ${researcher.get_costs()}\")\n\nasyncio.run(main())\n```\n\n### With MCP\n\n```python\nresearcher = GPTResearcher(\n query=\"Research topic\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\",\n)\n```\n\n### With WebSocket Streaming\n\n```python\nclass MockWebSocket:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(\n query=\"Research topic\",\n websocket=MockWebSocket(),\n)\n```\n\n### GPTResearcher Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `query` | str | required | Research question |\n| `report_type` | str | `research_report` | Type of report |\n| `report_source` | str | `web` | Data source |\n| `tone` | Tone | `Objective` | Writing tone |\n| `source_urls` | list | `[]` | Specific URLs to research |\n| `document_urls` | list | `[]` | Document URLs |\n| `query_domains` | list | `[]` | Restrict to domains |\n| `config_path` | str | None | Path to JSON config |\n| `websocket` | WebSocket | None | For streaming |\n| `mcp_configs` | list | `[]` | MCP server configs |\n| `mcp_strategy` | str | `fast` | MCP strategy |\n| `verbose` | bool | `True` | Verbose output |\n\n---\n\n## Output Files\n\n```\noutputs/\n├── task_{timestamp}_{query}.md\n├── task_{timestamp}_{query}.pdf\n├── task_{timestamp}_{query}.docx\n└── images/\n └── {research_id}/\n └── img_{hash}_{index}.png\n```\n\n---\n\n## Error Codes\n\n| Code | Description |\n|------|-------------|\n| 400 | Bad Request - Invalid parameters |\n| 404 | Not Found - Report not found |\n| 429 | Rate Limited - API quota exceeded |\n| 500 | Internal Server Error |\n",".claude/references/architecture.md":"# Architecture Reference\n\n## Table of Contents\n- [System Layers](#system-layers)\n- [Key File Locations](#key-file-locations)\n\n---\n\n## System Layers\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ USER REQUEST │\n│ (query, report_type, report_source, tone, mcp_configs) │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ BACKEND API LAYER │\n│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │\n│ │ FastAPI Server │ │ WebSocket Manager│ │ Report Store │ │\n│ │ backend/server/ │ │ Real-time events │ │ JSON persistence│ │\n│ │ app.py │ │ websocket_mgr.py │ │ report_store.py │ │\n│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ GPTResearcher (gpt_researcher/agent.py) │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ SKILLS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ ResearchConductor│ │ ReportGenerator │ │ ContextManager │ │ │\n│ │ │ Plan & gather │ │ Write reports │ │ Similarity search│ │ │\n│ │ │ researcher.py │ │ writer.py │ │ context_manager │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ BrowserManager │ │ SourceCurator │ │ ImageGenerator │ │ │\n│ │ │ Web scraping │ │ Rank sources │ │ Gemini images │ │ │\n│ │ │ browser.py │ │ curator.py │ │ image_generator │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ DeepResearchSkill│ │ │\n│ │ │ Recursive depth │ │ │\n│ │ │ deep_research.py│ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ ACTIONS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ report_generation│ │ query_processing│ │ web_scraping │ │ │\n│ │ │ LLM report write│ │ Sub-query plan │ │ URL scraping │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ retriever.py │ │ agent_creator │ │ markdown_process│ │ │\n│ │ │ Get retrievers │ │ Choose agent │ │ Parse markdown │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ PROVIDERS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ LLM Provider │ │ Retrievers │ │ Scrapers │ │ │\n│ │ │ OpenAI,Anthropic│ │ Tavily,Google │ │ BS4,Playwright │ │ │\n│ │ │ Google,Groq... │ │ Bing,MCP... │ │ PDF,DOCX... │ │ │\n│ │ │ llm_provider/ │ │ retrievers/ │ │ scraper/ │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ ImageGenerator │ │ │\n│ │ │ Gemini/Imagen │ │ │\n│ │ │ llm_provider/ │ │ │\n│ │ │ image/ │ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ CONFIGURATION LAYER │\n│ gpt_researcher/config/ │\n│ │\n│ Environment Variables → JSON Config File → Default Values │\n│ (highest) (medium) (lowest) │\n│ │\n│ config.py loads and merges all sources │\n│ variables/default.py contains all defaults │\n│ variables/base.py defines TypedDict for type safety │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes/Functions |\n|------|--------------|----------------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| Context/embeddings | `gpt_researcher/skills/context_manager.py` | `ContextManager` |\n| Source ranking | `gpt_researcher/skills/curator.py` | `SourceCurator` |\n| Deep research | `gpt_researcher/skills/deep_research.py` | `DeepResearchSkill` |\n| Image generation | `gpt_researcher/skills/image_generator.py` | `ImageGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| Config types | `gpt_researcher/config/variables/base.py` | `BaseConfig` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| WebSocket mgmt | `backend/server/websocket_manager.py` | `WebSocketManager`, `run_agent` |\n| Report types | `backend/report_type/` | `BasicReport`, `DetailedReport` |\n| Search engines | `gpt_researcher/retrievers/` | `TavilySearch`, `GoogleSearch`, etc. |\n| Web scraping | `gpt_researcher/scraper/` | Various scrapers |\n| Enums | `gpt_researcher/utils/enum.py` | `ReportType`, `ReportSource`, `Tone` |\n",".claude/references/components.md":"# Core Components & Method Signatures\n\n## Table of Contents\n- [GPTResearcher](#gptresearcher)\n- [ResearchConductor](#researchconductor)\n- [ReportGenerator](#reportgenerator)\n\n---\n\n## GPTResearcher\n\n**File:** `gpt_researcher/agent.py`\n\nThe main orchestrator class. Full initialization signature:\n\n```python\nclass GPTResearcher:\n def __init__(\n self,\n query: str, # Research question (required)\n report_type: str = \"research_report\", # research_report, detailed_report, deep, outline_report, resource_report\n report_format: str = \"markdown\", # Output format\n report_source: str = \"web\", # web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n tone: Tone = Tone.Objective, # Writing tone (see Tone enum)\n source_urls: list[str] | None = None, # Specific URLs to research\n document_urls: list[str] | None = None, # Document URLs to include\n complement_source_urls: bool = False, # Add web search to source_urls\n query_domains: list[str] | None = None, # Restrict search to domains\n documents=None, # LangChain document objects\n vector_store=None, # LangChain vector store\n vector_store_filter=None, # Filter for vector store\n config_path=None, # Path to JSON config file\n websocket=None, # WebSocket for streaming\n agent=None, # Pre-defined agent type\n role=None, # Pre-defined agent role\n parent_query: str = \"\", # Parent query for subtopics\n subtopics: list | None = None, # Subtopics to research\n visited_urls: set | None = None, # Already visited URLs\n verbose: bool = True, # Verbose logging\n context=None, # Pre-loaded context\n headers: dict | None = None, # HTTP headers\n max_subtopics: int = 5, # Max subtopics for detailed\n log_handler=None, # Custom log handler\n prompt_family: str | None = None, # Custom prompt family\n mcp_configs: list[dict] | None = None, # MCP server configurations\n mcp_max_iterations: int | None = None, # Deprecated, use mcp_strategy\n mcp_strategy: str | None = None, # fast, deep, disabled\n **kwargs\n ):\n```\n\n### Key Methods\n\n```python\nasync def conduct_research(self, on_progress=None) -> str:\n \"\"\"\n Main research orchestration.\n \n 1. Selects agent role via LLM (choose_agent)\n 2. Delegates to ResearchConductor\n 3. Optionally generates images if enabled\n \n Returns: Accumulated research context as string\n \"\"\"\n\nasync def write_report(\n self, \n existing_headers: list = [], # Headers to avoid duplication\n relevant_written_contents: list = [], # Previous content for context\n ext_context=None, # External context override\n custom_prompt=\"\" # Custom prompt override\n) -> str:\n \"\"\"\n Generate final report from context.\n \n Returns: Markdown report string\n \"\"\"\n\ndef get_costs(self) -> float:\n \"\"\"Returns total accumulated API costs.\"\"\"\n\ndef add_costs(self, cost: float) -> None:\n \"\"\"Add to running cost total (used as callback).\"\"\"\n```\n\n---\n\n## ResearchConductor\n\n**File:** `gpt_researcher/skills/researcher.py`\n\nManages the research process:\n\n```python\nclass ResearchConductor:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.logger = logging.getLogger(__name__)\n\n async def plan_research(self, query: str, query_domains=None) -> list:\n \"\"\"\n Generate sub-queries from main query using LLM.\n \n 1. Gets initial search results\n 2. Calls plan_research_outline() to generate sub-queries\n \n Returns: List of sub-query strings\n \"\"\"\n\n async def conduct_research(self) -> str:\n \"\"\"\n Main research execution based on report_source.\n \n Handles: web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n \n For each source type:\n 1. Load/search data\n 2. Process sub-queries\n 3. Combine context\n 4. Optionally curate sources\n \n Returns: Combined research context string\n \"\"\"\n\n async def _process_sub_query(\n self, \n sub_query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"\n Process a single sub-query.\n \n 1. Get MCP context (if configured, based on strategy)\n 2. Scrape URLs from search results\n 3. Get similar content via embeddings\n 4. Combine MCP + web context\n \n Returns: Combined context for this sub-query\n \"\"\"\n\n async def _get_context_by_web_search(\n self, \n query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"Web-based research with sub-query planning.\"\"\"\n\n async def _scrape_data_by_urls(\n self, \n sub_query: str, \n query_domains: list = []\n ) -> list:\n \"\"\"Search and scrape URLs for a sub-query.\"\"\"\n```\n\n---\n\n## ReportGenerator\n\n**File:** `gpt_researcher/skills/writer.py`\n\n```python\nclass ReportGenerator:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.research_params = {\n \"query\": researcher.query,\n \"agent_role_prompt\": researcher.cfg.agent_role or researcher.role,\n \"report_type\": researcher.report_type,\n \"report_source\": researcher.report_source,\n \"tone\": researcher.tone,\n \"websocket\": researcher.websocket,\n \"cfg\": researcher.cfg,\n \"headers\": researcher.headers,\n }\n\n async def write_report(\n self,\n existing_headers: list = [],\n relevant_written_contents: list = [],\n ext_context=None,\n custom_prompt=\"\",\n available_images: list = [], # Pre-generated images to embed\n ) -> str:\n \"\"\"\n Generate report using LLM.\n \n Calls generate_report() action with context and images.\n \n Returns: Markdown report\n \"\"\"\n\n async def write_introduction(self, ...) -> str:\n \"\"\"Write report introduction section.\"\"\"\n\n async def write_conclusion(self, ...) -> str:\n \"\"\"Write report conclusion with references.\"\"\"\n```\n",".claude/references/config-reference.md":"# Configuration Reference\n\n## Table of Contents\n- [Required Variables](#required-variables)\n- [LLM Configuration](#llm-configuration)\n- [Provider API Keys](#provider-api-keys)\n- [Retriever Configuration](#retriever-configuration)\n- [Report Configuration](#report-configuration)\n- [Feature Toggles](#feature-toggles)\n- [Configuration Priority](#configuration-priority)\n- [Example .env](#example-env)\n\n---\n\n## Required Variables\n\n```bash\nOPENAI_API_KEY=sk-... # Or another LLM provider key\nTAVILY_API_KEY=tvly-... # Or another retriever key\n```\n\n---\n\n## LLM Configuration\n\n```bash\nLLM_PROVIDER=openai # openai, anthropic, google, groq, together, etc.\nFAST_LLM=gpt-4o-mini # Quick tasks (summarization)\nSMART_LLM=gpt-4o # Complex reasoning (report writing)\nSTRATEGIC_LLM=o3-mini # Planning (agent selection)\nTEMPERATURE=0.4 # 0.0-1.0\nMAX_TOKENS=4000\nREASONING_EFFORT=medium # For o-series: low, medium, high\n```\n\n---\n\n## Provider API Keys\n\n```bash\n# OpenAI\nOPENAI_API_KEY=sk-...\nOPENAI_BASE_URL=https://api.openai.com/v1\n\n# Anthropic\nANTHROPIC_API_KEY=sk-ant-...\n\n# Google\nGOOGLE_API_KEY=AIza...\n\n# Groq\nGROQ_API_KEY=gsk_...\n\n# Azure OpenAI\nAZURE_OPENAI_API_KEY=...\nAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\n```\n\n---\n\n## Retriever Configuration\n\n```bash\nRETRIEVER=tavily # Single or comma-separated: tavily,google,mcp\nMAX_SEARCH_RESULTS_PER_QUERY=5\nMAX_URLS_TO_SCRAPE=10\nSIMILARITY_THRESHOLD=0.42\n```\n\n### Retriever API Keys\n\n```bash\nTAVILY_API_KEY=tvly-...\nGOOGLE_API_KEY=AIza...\nGOOGLE_CX_KEY=...\nBING_API_KEY=...\nSERPER_API_KEY=...\nSERPAPI_API_KEY=...\nEXA_API_KEY=...\n```\n\n---\n\n## Report Configuration\n\n```bash\nREPORT_FORMAT=apa # apa, mla, chicago, harvard, ieee\nTOTAL_WORDS=1000\nLANGUAGE=english\nCURATE_SOURCES=true\n```\n\n---\n\n## Feature Toggles\n\n### Image Generation\n\n```bash\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza...\nIMAGE_GENERATION_MODEL=models/gemini-2.5-flash-image\nIMAGE_GENERATION_MAX_IMAGES=3\nIMAGE_GENERATION_STYLE=dark # dark, light, auto\n```\n\n### Deep Research\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n### MCP\n\n```bash\nMCP_STRATEGY=fast # fast, deep, disabled\n```\n\n### Local Documents\n\n```bash\nDOC_PATH=./my-docs\n# Supports: PDF, DOCX, TXT, CSV, XLSX, PPTX, MD\n```\n\n### Server\n\n```bash\nHOST=0.0.0.0\nPORT=8000\nVERBOSE=true\n```\n\n---\n\n## Configuration Priority\n\n```\nEnvironment Variables (highest)\n ↓\nJSON Config File (if provided)\n ↓\nDefault Values (lowest)\n```\n\n**Important:** Config keys are lowercased when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\n---\n\n## Example .env\n\n```bash\n# Required\nOPENAI_API_KEY=sk-your-key\nTAVILY_API_KEY=tvly-your-key\n\n# LLM\nFAST_LLM=gpt-4o-mini\nSMART_LLM=gpt-4o\n\n# Report\nTOTAL_WORDS=1000\nLANGUAGE=english\n\n# Optional: Images\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza-your-key\nIMAGE_GENERATION_STYLE=dark\n```\n",".claude/references/deep-research.md":"# Deep Research Mode Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [DeepResearchSkill](#deepresearchskill)\n- [Usage](#usage)\n\n---\n\n## Overview\n\nDeep Research uses recursive tree-like exploration with configurable depth and breadth.\n\n---\n\n## Configuration\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n---\n\n## DeepResearchSkill\n\n**File:** `gpt_researcher/skills/deep_research.py`\n\n```python\nclass DeepResearchSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.breadth = getattr(researcher.cfg, 'deep_research_breadth', 4)\n self.depth = getattr(researcher.cfg, 'deep_research_depth', 2)\n self.concurrency_limit = getattr(researcher.cfg, 'deep_research_concurrency', 2)\n self.learnings = []\n self.research_sources = []\n self.context = []\n\n async def deep_research(self, query: str, on_progress=None) -> str:\n \"\"\"\n Recursive research with depth and breadth.\n \n 1. Research main topic\n 2. Generate subtopics (breadth)\n 3. For each subtopic, recursively research (depth)\n 4. Aggregate all findings\n 5. Generate comprehensive report\n \"\"\"\n```\n\n---\n\n## Usage\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers deep research\n)\nawait researcher.conduct_research()\nreport = await researcher.write_report()\n```\n\n### Research Tree Structure\n\n```\nQuery: \"Quantum Computing\"\n├── Subtopic 1: Hardware (depth 1)\n│ ├── Subtopic 1.1: Superconducting qubits (depth 2)\n│ └── Subtopic 1.2: Ion traps (depth 2)\n├── Subtopic 2: Algorithms (depth 1)\n│ ├── Subtopic 2.1: Shor's algorithm (depth 2)\n│ └── Subtopic 2.2: Grover's algorithm (depth 2)\n├── Subtopic 3: Applications (depth 1)\n│ └── ...\n└── Subtopic 4: Challenges (depth 1)\n └── ...\n```\n\nWith `DEEP_RESEARCH_BREADTH=4` and `DEEP_RESEARCH_DEPTH=2`, this explores 4 subtopics at each level, going 2 levels deep.\n",".claude/references/flows.md":"# Research Flow & Data Flow\n\n## Table of Contents\n- [End-to-End Research Flow](#end-to-end-research-flow)\n- [Data Flow Between Components](#data-flow-between-components)\n\n---\n\n## End-to-End Research Flow\n\n### 1. Request Entry\n\n**File:** `backend/server/app.py`\n\n```python\n# REST API endpoint\n@app.post(\"/report/\")\nasync def generate_report(research_request: ResearchRequest, background_tasks: BackgroundTasks):\n research_id = sanitize_filename(f\"task_{int(time.time())}_{research_request.task}\")\n # Calls write_report() which uses run_agent()\n\n# WebSocket endpoint\n@app.websocket(\"/ws\")\nasync def websocket_endpoint(websocket: WebSocket):\n await manager.connect(websocket)\n await handle_websocket_communication(websocket, manager)\n```\n\n### 2. Agent Runner\n\n**File:** `backend/server/websocket_manager.py`\n\n```python\nasync def run_agent(task, report_type, report_source, source_urls, ...):\n \"\"\"Main entry point for research execution.\"\"\"\n # Create logs handler\n logs_handler = CustomLogsHandler(websocket, task)\n \n # Configure MCP if enabled\n if mcp_enabled and mcp_configs:\n os.environ[\"RETRIEVER\"] = f\"{current_retriever},mcp\"\n os.environ[\"MCP_STRATEGY\"] = mcp_strategy\n \n # Route based on report type\n if report_type == \"multi_agents\":\n report = await run_research_task(query=task, websocket=logs_handler, ...)\n elif report_type == ReportType.DetailedReport.value:\n researcher = DetailedReport(query=task, ...)\n report = await researcher.run()\n else:\n researcher = BasicReport(query=task, ...)\n report = await researcher.run()\n \n return report\n```\n\n### 3. Research Phase\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nasync def conduct_research(self, on_progress=None):\n # Handle deep research separately\n if self.report_type == ReportType.DeepResearch.value and self.deep_researcher:\n return await self._handle_deep_research(on_progress)\n \n # Choose agent role via LLM\n if not (self.agent and self.role):\n self.agent, self.role = await choose_agent(\n query=self.query,\n cfg=self.cfg,\n parent_query=self.parent_query,\n cost_callback=self.add_costs,\n headers=self.headers,\n prompt_family=self.prompt_family,\n )\n \n # Conduct research\n self.context = await self.research_conductor.conduct_research()\n \n # Generate images if enabled (pre-generation for seamless UX)\n if self.cfg.image_generation_enabled and self.image_generator:\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n```\n\n### 4. Sub-Query Processing\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\nasync def _process_sub_query(self, sub_query: str, scraped_data: list = [], query_domains: list = []):\n # MCP Strategy handling\n mcp_retrievers = [r for r in self.researcher.retrievers if \"mcpretriever\" in r.__name__.lower()]\n mcp_strategy = self._get_mcp_strategy()\n \n if mcp_retrievers:\n if mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n # Reuse cached MCP results\n mcp_context = self._mcp_results_cache.copy()\n elif mcp_strategy == \"deep\":\n # Run MCP for every sub-query\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n \n # Get web search context\n if not scraped_data:\n scraped_data = await self._scrape_data_by_urls(sub_query, query_domains)\n \n # Get similar content via embeddings\n if scraped_data:\n web_context = await self.researcher.context_manager.get_similar_content_by_query(\n sub_query, scraped_data\n )\n \n # Combine MCP + web context\n combined_context = self._combine_mcp_and_web_context(mcp_context, web_context, sub_query)\n return combined_context\n```\n\n### 5. Report Generation\n\n**File:** `gpt_researcher/actions/report_generation.py`\n\n```python\nasync def generate_report(\n query: str,\n context: str,\n agent_role_prompt: str,\n report_type: str,\n websocket=None,\n cfg=None,\n tone=None,\n headers=None,\n cost_callback=None,\n prompt_family=None,\n available_images: list = [],\n **kwargs\n) -> str:\n \"\"\"Generate report using LLM.\"\"\"\n # Get prompt generator\n generate_prompt = prompt_family.get_prompt_by_report_type(report_type)\n \n # Build prompt with context and available images\n content = generate_prompt(\n query, context, report_source,\n report_format=cfg.report_format,\n tone=tone,\n total_words=cfg.total_words,\n language=cfg.language,\n available_images=available_images,\n )\n \n # Call LLM\n report = await create_chat_completion(\n model=cfg.smart_llm,\n messages=[{\"role\": \"user\", \"content\": content}],\n temperature=cfg.temperature,\n llm_provider=cfg.smart_llm_provider,\n max_tokens=cfg.smart_token_limit,\n llm_kwargs=cfg.llm_kwargs,\n cost_callback=cost_callback,\n )\n \n return report\n```\n\n---\n\n## Data Flow Between Components\n\n```\nUser Query\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ GPTResearcher.__init__() │\n│ • Loads Config (env → json → defaults) │\n│ • Initializes skills: ResearchConductor, ReportGenerator, etc │\n│ • Initializes retrievers based on RETRIEVER env var │\n│ • Initializes ImageGenerator if IMAGE_GENERATION_ENABLED │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.conduct_research()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ choose_agent() │\n│ Input: query, config │\n│ Output: (agent_type: str, role_prompt: str) │\n│ • LLM selects best agent role for the query │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ResearchConductor.conduct_research() │\n│ Input: self.researcher (has query, config, retrievers) │\n│ Output: context: str │\n│ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ plan_research() │ │\n│ │ Input: query │ │\n│ │ Output: sub_queries: list[str] │ │\n│ │ • Calls LLM to generate 3-5 sub-queries │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ For each sub_query: │ │\n│ │ _process_sub_query() │ │\n│ │ Input: sub_query │ │\n│ │ Output: sub_context: str │ │\n│ │ │ │\n│ │ 1. MCP retrieval (if configured) │ │\n│ │ → mcp_context: list[dict] │ │\n│ │ │ │\n│ │ 2. Web search via retrievers │ │\n│ │ → search_results: list[dict] │ │\n│ │ │ │\n│ │ 3. Scrape URLs │ │\n│ │ → scraped_content: list[dict] │ │\n│ │ │ │\n│ │ 4. Similarity search via embeddings │ │\n│ │ → relevant_context: str │ │\n│ │ │ │\n│ │ 5. Combine MCP + web context │ │\n│ │ → combined_context: str │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ Aggregate all sub_contexts → final context: str │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ If IMAGE_GENERATION_ENABLED:\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ImageGenerator.plan_and_generate_images() │\n│ Input: context, query, research_id │\n│ Output: available_images: list[dict] │\n│ [{\"url\": \"/outputs/images/.../img.png\", │\n│ \"title\": \"...\", \"description\": \"...\"}] │\n│ │\n│ 1. LLM analyzes context for visual concepts │\n│ 2. Generates 2-3 images in parallel via Gemini │\n│ 3. Saves to outputs/images/{research_id}/ │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.write_report()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ReportGenerator.write_report() │\n│ Input: context, available_images │\n│ Output: report: str (markdown) │\n│ │\n│ → generate_report() action │\n│ • Builds prompt with context + image list │\n│ • LLM generates report with embedded images │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ Output │\n│ • Streamed via WebSocket (type: \"report\") │\n│ • Final via WebSocket (type: \"report_complete\") │\n│ • Exported to PDF, DOCX, Markdown │\n│ • Saved to outputs/ directory │\n└─────────────────────────────────────────────────────────────────┘\n```\n",".claude/references/mcp.md":"# MCP Integration Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [Strategy Options](#strategy-options)\n- [Processing Logic](#processing-logic)\n\n---\n\n## Overview\n\nMCP (Model Context Protocol) enables research from specialized data sources (GitHub, databases, APIs) alongside web search.\n\n---\n\n## Configuration\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n mcp_configs=[\n {\n \"name\": \"github\", # Server name\n \"command\": \"npx\", # Command to start\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}, # Environment vars\n },\n {\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@anthropic/mcp-server-filesystem\", \"/docs\"],\n },\n {\n \"name\": \"remote\",\n \"connection_url\": \"ws://server:8080\", # WebSocket connection\n \"connection_type\": \"websocket\",\n \"connection_token\": \"auth_token\",\n }\n ],\n mcp_strategy=\"fast\", # fast, deep, disabled\n)\n```\n\n---\n\n## Strategy Options\n\n| Strategy | Behavior | Use Case |\n|----------|----------|----------|\n| `fast` (default) | Run MCP once with original query, cache results | Performance-focused |\n| `deep` | Run MCP for every sub-query | Maximum thoroughness |\n| `disabled` | Skip MCP entirely | Web-only research |\n\n---\n\n## Processing Logic\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\n# At start of research (for 'fast' strategy)\nif mcp_strategy == \"fast\":\n mcp_context = await self._execute_mcp_research_for_queries([query], mcp_retrievers)\n self._mcp_results_cache = mcp_context # Cache for reuse\n\n# During sub-query processing\nif mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n mcp_context = self._mcp_results_cache.copy() # Reuse cache\nelif mcp_strategy == \"deep\":\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n```\n\n### WebSocket Request Example\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"mcp_enabled\": true,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": [\n {\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}\n }\n ]\n}\n```\n",".claude/references/multi-agents.md":"# Multi-Agent System Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Agent Roles](#agent-roles)\n- [Workflow](#workflow)\n- [Usage](#usage)\n\n---\n\n## Overview\n\n**Directory:** `multi_agents/`\n\nLangGraph-based multi-agent system. Generates 5-6 page reports with multiple agents collaborating.\n\n---\n\n## Agent Roles\n\n| Agent | File | Role |\n|-------|------|------|\n| Human | - | Oversees and provides feedback |\n| Chief Editor | `agents/orchestrator.py` | Master coordinator via LangGraph |\n| Researcher | Uses GPTResearcher | Deep research on topics |\n| Editor | `agents/editor.py` | Plans outline and structure |\n| Reviewer | `agents/reviewer.py` | Validates research correctness |\n| Reviser | `agents/reviser.py` | Revises based on feedback |\n| Writer | `agents/writer.py` | Compiles final report |\n| Publisher | `agents/publisher.py` | Exports to PDF, DOCX, Markdown |\n\n---\n\n## Workflow\n\n```\n1. Browser (GPTResearcher) → Initial research\n2. Editor → Plans report outline\n3. For each outline topic (parallel):\n a. Researcher → In-depth subtopic research\n b. Reviewer → Validates draft\n c. Reviser → Revises until satisfactory\n4. Writer → Compiles final report\n5. Publisher → Exports to multiple formats\n```\n\n---\n\n## Usage\n\n### Via API\n\n```python\nreport_type = \"multi_agents\"\n```\n\n### Via WebSocket\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"multi_agents\",\n \"tone\": \"Analytical\"\n}\n```\n\n### Directly in Python\n\n```python\nfrom multi_agents.main import run_research_task\n\nreport = await run_research_task(\n query=\"Comprehensive analysis of market trends\",\n websocket=handler,\n tone=Tone.Analytical,\n)\n```\n\n### Configuration File\n\n**File:** `multi_agents/task.json`\n\nConfigure the multi-agent research task parameters and agent behaviors.\n",".claude/references/prompts.md":"# Prompt System Reference\n\n## Table of Contents\n- [PromptFamily Class](#promptfamily-class)\n- [Key Prompt Examples](#key-prompt-examples)\n\n---\n\n## PromptFamily Class\n\n**File:** `gpt_researcher/prompts.py`\n\nAll prompts are centralized in the `PromptFamily` class. This allows for model-specific prompt variations.\n\n```python\nclass PromptFamily:\n \"\"\"\n General purpose class for prompt formatting.\n Can be overwritten with model-specific derived classes.\n \"\"\"\n\n def __init__(self, config: Config):\n self.cfg = config\n\n @staticmethod\n def get_prompt_by_report_type(report_type: str):\n \"\"\"Returns the appropriate prompt generator for the report type.\"\"\"\n match report_type:\n case ReportType.ResearchReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.DetailedReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.OutlineReport.value:\n return PromptFamily.generate_outline_report_prompt\n # ... etc\n```\n\n---\n\n## Key Prompt Examples\n\n### Agent Selection Prompt\n\n```python\n@staticmethod\ndef generate_agent_role_prompt(query: str, parent_query: str = \"\") -> str:\n return f\"\"\"Analyze the research query and select the most appropriate agent role.\n\nQuery: \"{query}\"\n{f'Parent Query: \"{parent_query}\"' if parent_query else ''}\n\nBased on the query, determine:\n1. The domain expertise needed\n2. The research approach required\n3. The appropriate agent persona\n\nReturn a JSON object with:\n- \"agent\": The agent type (e.g., \"Research Analyst\", \"Technical Writer\")\n- \"role\": A detailed role description for how the agent should approach this research\n\"\"\"\n```\n\n### Research Planning Prompt\n\n```python\n@staticmethod\ndef generate_search_queries_prompt(\n query: str,\n parent_query: str = \"\",\n report_type: str = \"\",\n max_iterations: int = 3,\n context: str = \"\",\n) -> str:\n return f\"\"\"Generate {max_iterations} focused search queries to research: \"{query}\"\n\nContext from initial search:\n{context}\n\nRequirements:\n- Each query should explore a different aspect\n- Queries should be specific and searchable\n- Consider the report type: {report_type}\n\nReturn a JSON array of query strings.\n\"\"\"\n```\n\n### Report Generation Prompt (with images)\n\n```python\n@staticmethod\ndef generate_report_prompt(\n question: str,\n context: str,\n report_source: str,\n report_format=\"apa\",\n total_words=1000,\n tone=None,\n language=\"english\",\n available_images: list = [],\n) -> str:\n # Build image embedding instruction if images available\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES (embed where relevant):\n{image_list}\n\nUse markdown format: ![Title](URL)\n\"\"\"\n\n return f\"\"\"Information: \"{context}\"\n---\nUsing the above information, answer: \"{question}\" in a detailed report.\n\n- Format: {report_format}\n- Length: ~{total_words} words\n- Tone: {tone.value if tone else \"Objective\"}\n- Language: {language}\n- Include citations for all factual claims\n{image_instruction}\n\"\"\"\n```\n\n### MCP Tool Selection Prompt\n\n```python\n@staticmethod\ndef generate_mcp_tool_selection_prompt(query: str, tools_info: list, max_tools: int = 3) -> str:\n return f\"\"\"Select the most relevant tools for researching: \"{query}\"\n\nAVAILABLE TOOLS:\n{json.dumps(tools_info, indent=2)}\n\nSelect exactly {max_tools} tools ranked by relevance.\n\nReturn JSON:\n{{\n \"selected_tools\": [\n {{\"index\": 0, \"name\": \"tool_name\", \"relevance_score\": 9, \"reason\": \"...\"}}\n ]\n}}\n\"\"\"\n```\n",".claude/references/retrievers.md":"# Retriever System Reference\n\n## Table of Contents\n- [Available Retrievers](#available-retrievers)\n- [Retriever Selection](#retriever-selection)\n- [Adding a New Retriever](#adding-a-new-retriever)\n\n---\n\n## Available Retrievers\n\n**Directory:** `gpt_researcher/retrievers/`\n\n| Retriever | Class | API Key Env Var |\n|-----------|-------|-----------------|\n| Tavily | `TavilySearch` | `TAVILY_API_KEY` |\n| Google | `GoogleSearch` | `GOOGLE_API_KEY`, `GOOGLE_CX_KEY` |\n| Bing | `BingSearch` | `BING_API_KEY` |\n| Brave | `BraveSearch` | `BRAVE_API_KEY` |\n| DuckDuckGo | `Duckduckgo` | None |\n| SearX | `SearxSearch` | `SEARX_URL` |\n| Serper | `SerperSearch` | `SERPER_API_KEY` |\n| SerpAPI | `SerpApiSearch` | `SERPAPI_API_KEY` |\n| SearchAPI | `SearchApiSearch` | `SEARCHAPI_API_KEY` |\n| Exa | `ExaSearch` | `EXA_API_KEY` |\n| GroundRoute | `GroundRouteSearch` | `GROUNDROUTE_API_KEY` |\n| fastCRW | `CRWRetriever` | `CRW_API_KEY` |\n| BoCha | `BoChaSearch` | `BOCHA_API_KEY` |\n| Xquik | `XquikSearch` | `XQUIK_API_KEY` |\n| GetXAPI (X/Twitter) | `GetXAPISearch` | `GETXAPI_API_KEY` |\n| arXiv | `ArxivSearch` | None |\n| Semantic Scholar | `SemanticScholarSearch` | None |\n| PubMed Central | `PubMedCentralSearch` | None |\n| OpenAlex | `OpenAlexSearch` | `OPENALEX_API_KEY`, `OPENALEX_EMAIL` |\n| MCP | `MCPRetriever` | Per-server |\n| Custom | `CustomRetriever` | User-defined |\n\n> The authoritative list is the `match` statement in `gpt_researcher/actions/retriever.py` (`get_retriever`).\n\n---\n\n## Retriever Selection\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ndef get_retriever(retriever: str):\n \"\"\"Get a retriever class by name.\"\"\"\n match retriever:\n case \"tavily\":\n from gpt_researcher.retrievers import TavilySearch\n return TavilySearch\n case \"google\":\n from gpt_researcher.retrievers import GoogleSearch\n return GoogleSearch\n case \"mcp\":\n from gpt_researcher.retrievers import MCPRetriever\n return MCPRetriever\n # ... etc\n\ndef get_retrievers(headers: dict[str, str], cfg) -> list:\n \"\"\"\n Resolve which retriever(s) to use, in priority order:\n headers[\"retrievers\"] / headers[\"retriever\"] → cfg.retrievers / cfg.retriever\n → default (TavilySearch). Accepts a comma-separated string or a list.\n\n Usage: RETRIEVER=tavily,google,mcp\n \"\"\"\n # ... resolve names from headers/cfg (see actions/retriever.py) ...\n # Invalid names fall back to get_default_retriever():\n return [get_retriever(r) or get_default_retriever() for r in retriever_names]\n```\n\n---\n\n## Adding a New Retriever\n\n### Step 1: Create Retriever File\n\n**File:** `gpt_researcher/retrievers/my_retriever/my_retriever.py`\n\n```python\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n self.headers = headers\n \n async def search(self, max_results: int = 10) -> list[dict]:\n \"\"\"\n Returns list of:\n {\n \"title\": str,\n \"href\": str,\n \"body\": str\n }\n \"\"\"\n # Implementation\n pass\n```\n\n### Step 2: Register in retriever.py\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n```\n\n### Step 3: Export in __init__.py\n\n**File:** `gpt_researcher/retrievers/__init__.py`\n\n```python\nfrom .my_retriever import MyRetriever\n__all__ = [..., \"MyRetriever\"]\n```\n\n### Step 4: Usage\n\n```bash\nRETRIEVER=tavily,my_retriever\n```\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n # Will use both Tavily and your custom retriever\n)\n```\n",".codex-plugin/plugin.json":"{\n \"name\": \"gpt-researcher\",\n \"version\": \"0.1.0\",\n \"description\": \"Autonomous deep research from Codex via MCP\",\n \"author\": {\n \"name\": \"assafelovic\",\n \"url\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"homepage\": \"https://github.com/assafelovic/gpt-researcher\",\n \"repository\": \"https://github.com/assafelovic/gpt-researcher\",\n \"keywords\": [\n \"mcp\",\n \"codex\"\n ],\n \"mcpServers\": \"./.mcp.json\",\n \"interface\": {\n \"displayName\": \"GPT Researcher\",\n \"shortDescription\": \"Autonomous deep research from Codex via MCP\",\n \"longDescription\": \"An autonomous agent that conducts deep research on any data using any LLM providers.\",\n \"category\": \"Research\",\n \"websiteURL\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"skills\": \"./skills/\"\n}","evals/README.md":"# GPT-Researcher Evaluations\n\nThis directory contains evaluation tools and frameworks for assessing the performance of GPT-Researcher across different research tasks.\n\n## Evaluation Overview\n\nThree complementary evaluation systems, each answering a different question:\n\n| System | Question | Requires ground truth? |\n| --- | --- | --- |\n| `simple_evals/` | Is the answer factually correct? | Yes (OpenAI SimpleQA dataset) |\n| `quality_eval/` | Is the report well-sourced and complete? | No — works on any query |\n| `hallucination_eval/` | Does the report fabricate content? | No — compares against scraped sources |\n\n**Quick start:**\n\n```bash\n# Factual accuracy (needs ground-truth answers)\npython -m evals.simple_evals.run_eval --num_examples 10\n\n# Report quality (works on any query, zero-cost metrics only)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Hallucination detection\npython -m evals.hallucination_eval.run_eval -n 5\n```\n\n## Simple Evaluations (`simple_evals/`)\n\nThe `simple_evals` directory contains a straightforward evaluation framework adapted from [OpenAI's simple-evals system](https://github.com/openai/simple-evals), specifically designed to measure short-form factuality in large language models. Our implementation is based on OpenAI's [SimpleQA evaluation methodology](https://github.com/openai/simple-evals/blob/main/simpleqa_eval.py), following their zero-shot, chain-of-thought approach while adapting it for GPT-Researcher's specific use case.\n\n### Components\n\n- `simpleqa_eval.py`: Core evaluation logic for grading research responses\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `requirements.txt`: Dependencies required for running evaluations\n\n### Test Dataset\n\nThe `problems/` directory contains the evaluation dataset:\n\n- `Simple QA Test Set.csv`: A comprehensive collection of factual questions and their correct answers, mirrored from OpenAI's original test set. This dataset serves as the ground truth for evaluating GPT-Researcher's ability to find and report accurate information. The file is maintained locally to ensure consistent evaluation benchmarks and prevent any potential upstream changes from affecting our testing methodology.\n\n### Evaluation Logs\n\nThe `logs/` directory contains evaluation run histories preserved in version control.\n\nTwo formats are kept side-by-side:\n\n| Format | Example filename | Purpose |\n| --- | --- | --- |\n| Plain text (legacy) | `SimpleQA Eval 100 Problems 2-22-25.txt` | Human-readable terminal output |\n| Structured JSON | `eval_2025-02-22_10-32-11_n100.jsonl` | Machine-readable, suitable for trend analysis |\n\nEach JSON log file contains one record with three sections: `run_metadata`, `aggregate_metrics`, and per-query `results`. See [`example_output.json`](simple_evals/example_output.json) for the full schema.\n\n**Note:** Unlike typical log directories, this folder and its contents are intentionally tracked in git to maintain a historical record of evaluation runs.\n\n### Features\n\n- Measures factual accuracy of research responses\n- Uses GPT-4 as a grading model (configurable)\n ```python\n # In run_eval.py, you can customize the grader model:\n grader_model = ChatOpenAI(\n temperature=0, # Lower temperature for more consistent grading\n model_name=\"gpt-4-turbo\", # Can be changed to other OpenAI models\n openai_api_key=os.getenv(\"OPENAI_API_KEY\")\n )\n ```\n- Grades responses on a three-point scale:\n - `CORRECT`: Answer fully contains important information without contradictions\n - `INCORRECT`: Answer contains factual contradictions\n - `NOT_ATTEMPTED`: Answer neither confirms nor contradicts the target\n\n**Note on Grader Configuration:** While the default grader uses GPT-4-turbo, you can modify the model and its parameters to use different OpenAI models or adjust the temperature for different grading behaviors. This is independent of the researcher's configuration, allowing you to optimize for cost or performance as needed.\n\n### Metrics Tracked\n\n- Accuracy rate and F1 score\n- Cost — total and per-query average\n- Latency — average, p50 (median), and p95 per query\n- Success/failure counts and answer attempt rates\n- Source coverage per query\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/simple_evals\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\nLANGCHAIN_API_KEY=your_langchain_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py --num_examples \n```\n\nThe `num_examples` parameter determines how many random test queries to evaluate (default: 1).\n\n#### Customizing Researcher Behavior\n\nThe evaluation uses GPTResearcher with default settings, but you can modify `run_eval.py` to customize the researcher's behavior:\n\n```python\nresearcher = GPTResearcher(\n query=query,\n report_type=ReportType.ResearchReport.value, # Type of report to generate\n report_format=\"markdown\", # Output format\n report_source=ReportSource.Web.value, # Source of research\n tone=Tone.Objective, # Writing tone\n verbose=True # Enable detailed logging\n)\n```\n\nThese parameters can be adjusted to evaluate different research configurations or output formats. For a complete list of configuration options, see the [configuration documentation](https://docs.gptr.dev/docs/gpt-researcher/gptr/config).\n\n**Note on Configuration Independence:** The evaluation system is designed to be independent of the researcher's configuration. This means you can use different LLMs and settings for evaluation versus research. For example:\n- Evaluation could use GPT-4-turbo for grading while the researcher uses Claude 3.5 Sonnet for research\n- Different retrievers, embeddings, or report formats can be used\n- Token limits and other parameters can be customized separately\n\nThis separation allows for unbiased evaluation across different researcher configurations. However, please note that this feature is currently experimental and needs further testing.\n\n### Output\n\nEach run produces two outputs:\n\n#### Console summary\n\n```text\n=== Evaluation Summary ===\nTotal queries tested: 100\nSuccessful queries: 100\nFailed queries: 0\n\n=== AGGREGATE METRICS ===\nDebug counts:\nTotal successful: 100\nCORRECT: 92\nINCORRECT: 7\nNOT_ATTEMPTED: 1\n{\n \"correct_rate\": 0.92,\n \"incorrect_rate\": 0.07,\n \"not_attempted_rate\": 0.01,\n \"answer_rate\": 0.99,\n \"accuracy\": 0.9292929292929293,\n \"f1\": 0.9246231155778895\n}\n========================\nAccuracy: 0.929\nF1 Score: 0.925\n\nTotal cost: $9.6000\nAverage cost per query: $0.0960\nLatency — avg: 48.3s p50: 45.1s p95: 89.2s\n\nStructured log saved → evals/simple_evals/logs/eval_2025-02-22_10-32-11_n100.jsonl\n```\n\n**Structured JSON log** (`logs/eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`)\n\nSee [`example_output.json`](simple_evals/example_output.json) for the full schema. Key fields:\n\n```\nrun_metadata → timestamp, git_commit, grader_model, researcher_model\naggregate_metrics → accuracy, f1, cost, avg/p50/p95 latency, success/fail counts\nresults → per-query list with grade, score, latency, cost, sources\n```\n\n## Quality Evaluation (`quality_eval/`)\n\nMeasures the structural and content quality of research reports without requiring ground-truth answers. All metrics run on any open-ended query.\n\nAll metric logic lives in `metrics.py`. The runner is `run_eval.py`.\n\n### Metrics\n\n| Metric | Description | LLM calls | Cost |\n| --- | --- | --- | --- |\n| **Citation Faithfulness** | Are sources listed in the report's `## References` actually cited in the body? (catches decorative refs) | 0 | $0 |\n| **Source Diversity** | Domain variety via ratio + Shannon entropy | 0 | $0 |\n| **Source Authority** | Heuristic score per domain (.gov=1.0 → .com=0.5); unknown domains optionally LLM-scored | 0–1 | $0–$0.01 |\n| **Subtopic Coverage** | LLM-judged coverage of expected subtopics (adaptive count) | 2 | ~$0.02/query |\n| **Hallucination** | Binary faithfulness check — report vs. scraped sources | 1 | ~$0.01/query |\n| **Unsupported Claim Rate** | Claim-level credibility: supported (1.0) / inferred (0.3–0.9) / unsupported (0.0) | 2 | ~$0.02/query |\n\n**Citation Faithfulness** measures whether sources the report *lists* in its `## References` are *actually cited* in the body (a low score flags decorative references). It deliberately does not reward raw coverage — reading many sources and citing only the useful ones is good practice, like a real researcher.\n\n**Unsupported Claim Rate** distinguishes between:\n\n- `supported` — claim directly found in source text (score 1.0)\n- `inferred` — reasonable synthesis from multiple sources; never applied to data claims (score 0.3–0.9)\n- `unsupported` — no traceable basis in sources (score 0.0)\n\n### Metric Grouping & Tooling\n\nThe metrics group into three families: **Source Quality** (diversity, authority), **Writing Behavior** (citation faithfulness, subtopic coverage), and **Hallucination** (hallucination check, unsupported claim) — i.e. *is the input good*, *how was it written*, *is it correct*.\n\nTwo tools complement `run_eval.py` (shared metric logic lives in `metrics.py`):\n\n- **`benchmark.py`** — runs multiple researcher configurations (single- vs. multi-agent × model) over a fixed topic set and prints a side-by-side table. `--compare LOG_A LOG_B` diffs two past runs; `--summary-only` re-prints from saved logs.\n- **`perturbation.py`** — a behavioral test that deliberately degrades a report (remove inline citations, swap sources to low-authority, corrupt claims) and checks each metric responds monotonically. This validates metric *reliability* (directional sensitivity), which is separate from whether a metric captures *quality*.\n\n### Running\n\n```bash\n# Run all 6 metrics (most expensive, ~$0.06/query)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost only (citation ratio, diversity, authority)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Skip individual LLM metrics selectively\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic\npython -m evals.quality_eval.run_eval --num_examples 10 --no-hallucination\npython -m evals.quality_eval.run_eval --num_examples 10 --no-unsupported-claim\n```\n\nLogs are saved to `evals/quality_eval/logs/quality_eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`.\n\n### Example Output\n\n```text\nQuality eval: 5 queries subtopic=off hallucination=off unsupported_claim=on\n\n>> Query: What legal frameworks govern cross-border data transfers?\n citation_faithful: 0.95 (1 listed-only)\n diversity_ratio: 1.00 entropy=4.32\n authority_score: 0.62\n avg_claim_score: 0.79 (unsupported=0.11 inferred=0.22 n=9)\n latency=37.2s cost=$0.13\n\n=== QUALITY EVAL SUMMARY ===\n avg_citation_faithfulness: 0.94\n avg_diversity_ratio: 0.796\n avg_domain_entropy: 3.525\n avg_authority_score: 0.597\n avg_claim_score: 0.428\n avg_unsupported_claim_rate: 0.535\n avg_inferred_claim_rate: 0.148\n avg_latency_seconds: 33.6\n avg_cost: 0.116\n```\n\n### Dependencies\n\n```bash\npip install langchain-openai python-dotenv\n\n# Hallucination metric only:\npip install judges\n```\n\nEnvironment variables required: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n\n---\n\n## Hallucination Evaluation (`hallucination_eval/`)\n\nThe `hallucination_eval` directory contains tools for evaluating GPT-Researcher's outputs for hallucination. This evaluation system compares the generated research reports against their source materials to detect non-factual or hallucinated content, ensuring the reliability and accuracy of the research outputs.\n\n### Components\n\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `evaluate.py`: Core evaluation logic for detecting hallucinations\n- `inputs/`: Directory containing test queries\n - `search_queries.jsonl`: Collection of research queries for evaluation\n- `results/`: Directory containing evaluation results\n - `evaluation_records.jsonl`: Detailed per-query evaluation records\n - `aggregate_results.json`: Summary metrics across all evaluations\n\n### Features\n\n- Evaluates research reports against source materials\n- Provides detailed reasoning for hallucination detection\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/hallucination_eval\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py -n \n```\n\nThe `-n` parameter determines how many queries to evaluate from the test set (default: 1).\n\n### Sample Result\n\n```json\n{\n \"total_queries\": 1,\n \"successful_queries\": 1,\n \"total_responses\": 1,\n \"total_evaluated\": 1,\n \"total_hallucinated\": 0,\n \"hallucination_rate\": 0.0,\n \"results\": [\n {\n \"input\": \"What are the latest developments in quantum computing?\",\n \"output\": \"Research report content...\",\n \"source\": \"Source material content...\",\n \"is_hallucination\": false,\n \"confidence_score\": 0.95,\n \"reasoning\": \"The summary accurately reflects the source material with proper citations...\"\n }\n ]\n}\n```\n","evals/context_filter/README.md":"# Context filter benchmark: embeddings vs Jev vs no filter\n\n![Context filter benchmark results](results.png)\n\nEvery scraped page reaches a GPT Researcher report through one step: for each\nsub-query, the scraped pages are cut into chunks and a filter decides which\nchunks become the writer's context. Until now that filter was embedding\nsimilarity. This benchmark asks two questions:\n\n1. Is there a better filter? We test [TypeSafe's Jev](https://docs.typesafe.ai/),\n a model that returns calibrated, typed decisions, scoring each chunk on how\n useful it is for answering the question.\n2. With today's long context windows, do we need a filter at all?\n\n## Method\n\n**Fixed inputs.** `collect.py` runs 28 real research tasks once and records\nthe exact `(sub_query, pages)` pairs that reached the filter: 20 SimpleQA\nquestions with known answers (fixed seed, from `evals/simple_evals`) and 8\nopen-ended research questions. `replay.py` rebuilds the context from those\nsame pages with each strategy and writes the report with the same model\n(`gpt-5.4`). Search and scraping, the noisiest parts of a run, are held\nconstant, so the filter is the only thing that differs.\n\n**Strategies.**\n\n| strategy | what reaches the writer |\n|---|---|\n| `embeddings` | today: 1000-char chunks, cosine similarity > 0.42, top 10 per sub-query |\n| `jev` | same chunks, scored by Jev on a 4-level usefulness rubric, top 10 scoring ≥ 1.5 |\n| `jev_top10` | Jev's 10 best chunks, no threshold (same budget as embeddings) |\n| `jev_large` | Jev on 3000-char chunks, top 4 (≈ same characters, a third of the calls) |\n| `jev_wide` | Jev threshold 1.5, up to 30 chunks |\n| `jev_broad` | Jev on 3000-char chunks, threshold 1.0, up to 12 (open-ended only) |\n| `bm25` | keyword (BM25) ranking of the same chunks, top 10; runs locally, no API or model |\n| `bm25_rel` | BM25, keeping chunks scoring ≥ 50% of the best chunk, top 10 |\n| `bm25_wide` | BM25 with the same relative threshold, up to 25 chunks (shipped as `keyword`) |\n| `none_capped` | no ranking: pages in search order, capped at 48k characters per sub-query |\n| `none` | no filter: every page in full (capped at 50k chars per page, as today) |\n| `truncate` | control: 10 chunks round-robin across pages, no relevance ranking |\n\n**Quality signals** (`judge.py`):\n\n- **SimpleQA accuracy**: each report graded with the repo's SimpleQA grader.\n- **Context precision**: the share of context chunks an independent judge\n (`gpt-5.4-mini`) rates as relevant to the question. This measures the filter\n itself. It isn't defined for `none`.\n- **Pairwise preference**: a blind head-to-head judged by `gpt-5.4`, run in both\n orders. A win counts only if the strategy wins in *both* orders; any\n disagreement is a tie.\n\n## Results\n\n### Cost and speed (28 questions)\n\n| strategy | filter time p50 / p90 | filter $ | context tokens p50 (max) | write $ | total $ / report |\n|---|---|---|---|---|---|\n| embeddings | 1.0s / 1.3s | 0.0028 | 7.6k (10.7k) | 0.114 | **0.117** |\n| jev | 2.4s / 6.0s ¹ | 0.0054 | 4.5k (10.6k) | 0.110 | **0.115** |\n| jev_top10 | 2.7s / 6.2s | 0.0054 | 8.6k (10.6k) | 0.119 | 0.124 |\n| jev_large | 1.2s / 5.1s | 0.0031 | 6.5k | 0.111 | 0.114 |\n| truncate | 0.0s | — | 8.0k (11.0k) | 0.117 | 0.117 |\n| none | 0.0s | — | **33.4k (107.7k)** | **0.192** | **0.192** |\n\n¹ Measured with 32 concurrent Jev calls. At 64, the new default, the same\nfiltering takes **1.7s p50 / 3.6s p90**; 128 was no faster.\n\nReport writing takes the same ~45s for every strategy: generating the output\ndominates, and input size barely matters. Removing the filter makes nothing\nfaster; it makes each report 65% more expensive.\n\n### Quality\n\n| strategy | SimpleQA correct | context precision | vs embeddings (W/T/L) | vs none (W/T/L) |\n|---|---|---|---|---|\n| embeddings | 18/20 | 0.46 | — | 3 / 12 / 13 |\n| **jev** | 19/20 | **0.73** | **15 / 10 / 3** | 6 / 12 / 10 |\n| jev_top10 | 19/20 | 0.50 | 13 / 10 / 5 | — |\n| jev_large | 19/20 | 0.72 | 11 / 10 / 7 | 8 / 7 / 13 |\n| jev_wide | 20/20 | **0.74** | 13 / 8 / 7 | 7 / 11 / 10 |\n| truncate | 19/20 | 0.33 | 7 / 8 / 13 | — |\n| none | 20/20 | — | 13 / 11 / 4 | — |\n\n### Open-ended questions only (8)\n\n| strategy | context p50 | total $ / report | vs embeddings | vs none |\n|---|---|---|---|---|\n| embeddings | 8.8k | 0.155 | — | 0 / 1 / 7 |\n| jev | 9.1k | 0.160 | 5 / 2 / 1 | 0 / 2 / 6 |\n| jev_broad | 26.9k | 0.191 | **7 / 1 / 0** | 0 / 3 / 5 |\n| none | 61.3k | **0.284** | 8 / 0 / 0 | — |\n\nThe `none` reports on open-ended questions were *shorter* than the embeddings\nreports (2,099 vs 2,416 median words) and still preferred, so the judge is not\nrewarding length.\n\n### Filters that need nothing: keyword (BM25)\n\nWithout a Jev key, GPT Researcher needs a filter with no API, model or\nembeddings. The candidates, on the same 28 runs:\n\n| strategy | SimpleQA | context precision | vs embeddings (W/T/L) | vs none (W/T/L) | filter p50 | context p50 | total $ |\n|---|---|---|---|---|---|---|---|\n| bm25 | 18/20 | 0.40 | 7 / 9 / 12 | 3 / 12 / 13 | 0.02s | 8.1k | 0.118 |\n| bm25_rel | 19/20 | 0.53 | 7 / 13 / 8 | 5 / 10 / 13 | 0.02s | 4.7k | **0.107** |\n| **bm25_wide** | 19/20 | 0.51 | **14 / 8 / 6** | 6 / 9 / 13 | 0.02s | 6.4k | 0.116 |\n| none_capped | 20/20 | 0.54 | 11 / 13 / 4 | 6 / 10 / 12 | 0.00s | 23.8k | 0.152 |\n| *embeddings* | *18/20* | *0.46* | — | *3 / 12 / 13* | *1.04s* | *7.6k* | *0.117* |\n\n`bm25_wide` ships as the `keyword` fallback: at least as good as embeddings on\nevery measure, at the same cost, about 50× faster to filter, and with nothing\nto set up. Its 14–6 head-to-head is directional (sign test p ≈ 0.12), not as\nstrong as Jev's 15–3. As with Jev, the relative threshold is what does the\nwork: plain top-10 BM25 loses to embeddings 7–12.\n\nGPT Researcher removes duplicate URLs across sub-queries before scraping\n(0 of 300 pages recurred), so \"sources that came back most often\" isn't a\nsignal available at this step.\n\n## What this shows\n\n1. **Jev is a better filter than embeddings at the same cost.** 73% of the\n chunks Jev selects are relevant, against 46% for embeddings; the unranked\n control gets 33%. Jev reports beat embeddings reports 15–3 head-to-head\n (sign test on the 18 non-ties, p ≈ 0.008). Every Jev variant beat\n embeddings. Total cost per report is the same; both filters cost well\n under a cent.\n2. **Most of the gain comes from the threshold.** `jev_top10`, forced to fill\n 10 slots, drops to 0.50 precision. Cosine similarity cannot say \"nothing\n else here is worth including\"; a calibrated usefulness score can.\n3. **Skipping the filter gives the best open-ended reports, at 65–83% higher\n cost.** More source material makes for broader reports. But on a *standard*\n research report the context already reaches 108k tokens; detailed and deep\n research run many more sub-queries and would overflow context windows and\n budgets. For fact-finding questions it adds nothing: SimpleQA is at\n ceiling for every strategy, and Jev vs none is 6–10–4.\n4. **No integration is needed for a good filter.** Keyword ranking with a\n relative threshold matches or beats embeddings at the same cost (above),\n so the default without a Jev key is `keyword`, and nothing requires an\n embeddings provider.\n5. **SimpleQA is saturated.** 18–20 of 20 for every strategy: when search\n finds the fact, any reasonable filter keeps it. Differences of one question\n are noise.\n\n## Caveats\n\n- 28 questions, judged by one model family (`gpt-5.4`), which also writes the\n reports. Self-preference would apply to every strategy equally, but the\n pairwise numbers are directional evidence, not a leaderboard.\n- One writer model and one report type (`research_report`). Detailed and deep\n research were not replayed.\n- Jev latency depends on chunk count and concurrency. `jev_large` matches\n embeddings at the median; tails come from large pages.\n\n## Reproduce\n\n```bash\nexport OPENAI_API_KEY=... TAVILY_API_KEY=... TYPESAFE_API_KEY=...\npython -m evals.context_filter.collect --out runs/ # ~$0.40, ~20 min\npython -m evals.context_filter.replay --runs runs/ --out results/\npython -m evals.context_filter.judge --results results/\npython -m evals.context_filter.judge --results results/ --baseline none --only jev embeddings\npython -m evals.context_filter.replay --runs runs/ --out results/ --strategies bm25 bm25_rel bm25_wide none_capped\n```\n\n`results/` in this folder holds the scores and per-run metrics from the run\nabove (`runs.json`, `scored_*.json`, `summary_*.json`). Scraped pages and\ngenerated reports are not committed.\n\nTotal spend: about $40 in tracked writing and filtering across all strategies\n(of which $0.59 was Jev; keyword and none cost nothing to filter), plus an\nestimated $8–15 for judging.\n","evals/quality_eval/README.md":"# quality_eval — Report Quality Evaluation for Autonomous Research Agents\n\nEvaluate the quality of a GPT-Researcher report on **any open-ended query, with no\nground-truth answer required**. Where `simple_evals` needs a labelled dataset and\n`hallucination_eval` gives a binary check, `quality_eval` scores a report across\nseveral interpretable dimensions of *how well it was researched and written*.\n\nIt follows the metric abstraction of established frameworks (DeepEval / RAGAS),\nand **extends it for autonomous research** — evaluating not just the answer, but\nthe quality of the sources the agent found on its own.\n\n## What it measures\n\n| Group | Aligns with | Metric | Question it answers |\n|---|---|---|---|\n| **Faithfulness** | RAGAS Faithfulness | `unsupported_claim` | Are the report's claims grounded in the sources? |\n| **Answer Relevancy** | RAGAS answer relevancy | `subtopic_coverage` | Does it cover the subtopics a thorough report should? |\n| **Source Quality** | *extension (autonomous agent)* | `source_authority`, `source_diversity` | Are the sources the agent found authoritative and varied? |\n| **Citation** | ALCE citation precision | `citation_faithfulness` | Are the listed references actually cited in the body? |\n\nZero-cost metrics (citation, diversity, authority) run by default; the two LLM\nmetrics are opt-out via flags, so you can trade cost for depth.\n\n## Concept alignment (why this is more than ad-hoc metrics)\n\nThe suite maps onto standard evaluation ontologies where they apply, and marks\nwhere autonomous research needs more:\n\n- **Faithfulness / Answer Relevancy** ← align with **RAGAS**. `unsupported_claim`\n even uses the field's standard three-way support scale (supported / inferred /\n unsupported), matching ALCE-style *full / partial / no support*.\n- **Citation** ← aligns with **ALCE citation precision** (of the references it\n lists, how many are actually used). Current implementation is *structural*; a\n *semantic* version (does the citation truly support its claim, NLI-style) is on\n the roadmap.\n- **Source Quality** ← a deliberate **extension**. RAG-oriented frameworks assume\n the context is *given*, so they have no notion of \"source quality\". A research\n agent finds its own sources, so authority and diversity of those sources become\n first-class metrics — this is the dimension standard RAG eval can't express.\n\n## Architecture\n\nA thin, dependency-free framework layer (mirrors DeepEval's `BaseMetric` +\ntest-case design):\n\n```text\nbase.py BaseMetric / EvalSample / MetricResult — abstractions (+ `sources`, the RAG-framework gap)\nsuite.py 5 metric wrappers + evaluate() entry — standardized I/O + concept-alignment metadata\nmetrics.py raw metric functions — pure logic (unit-tested, perturbation-validated)\nperturbation.py behavioral reliability test — graded degradation, asserts monotonic response\nrun_eval.py / benchmark.py — runners (single query / multi-config comparison)\n```\n\nEvery metric declares its `group` and the standard concept it `aligned_with`, so\nthe ontology above is encoded in code, not just docs. `MetricResult.score` is\nalways *higher = better* (0–1) so results aggregate and compare uniformly.\n\n## Reliability (perturbation-validated)\n\nThe metrics aren't just implemented — they're checked for *reliability*.\n`perturbation.py` deliberately degrades a report (remove citations, swap to\nlow-authority sources, corrupt claims) in graded steps and asserts each metric\nresponds **monotonically**. This surfaced and fixed real defects that mocked unit\ntests and final outputs couldn't reveal — e.g. batch LLM scoring was letting one\nclaim contaminate another's score, fixed by scoring each claim independently.\n\n## Usage\n\n```bash\n# Full evaluation (all metrics)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost metrics only\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-unsupported-claim\n\n# Reliability check\npython -m evals.quality_eval.perturbation\n\n# Compare configs / runs\npython -m evals.quality_eval.benchmark --configs single-gpt4o single-gpt4o-mini\npython -m evals.quality_eval.benchmark --compare LOG_A LOG_B\n```\n\nProgrammatic use via the standardized entry point:\n\n```python\nfrom evals.quality_eval.base import EvalSample\nfrom evals.quality_eval.suite import evaluate, default_metrics\n\nsample = EvalSample(query=q, report=report, sources=urls, context=ctx)\nresults = await evaluate(sample, default_metrics(), grader_model)\nfor r in results:\n print(r.group, r.name, r.score) # typed, tagged, higher = better\n```\n\n## Roadmap\n\n- **Semantic citation precision** (does a citation truly support its claim,\n NLI/LLM-judge) → promote the `Citation` group to a full `Precision` family.\n- **True answer relevancy** (on-topic-ness), complementing `subtopic_coverage`'s\n completeness view.\n- **Component-level evaluation** of the multi-agent pipeline (planner / researcher\n / writer / reviewer) to locate bottlenecks — the natural next phase.\n\nDependencies: `langchain-openai`, `python-dotenv` (+ `judges` for the optional\nhallucination check). Env: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n","evals/simple_evals/logs/README.md":"# Evaluation Results\n\nThis directory contains historical evaluation results for GPT-Researcher using the SimpleQA methodology.\n\n## Latest Results\n\n### [SimpleQA Eval 100 Problems 2-22-25](./SimpleQA%20Eval%20100%20Problems%202-22-25.txt)\n\nEvaluation run by [Kelly Abbott (kga245)](https://github.com/kga245)\n\n**Summary:**\n- Date: February 22, 2025\n- Sample Size: 100 queries\n- Success Rate: 100% (100/100 queries completed)\n\n**Performance Metrics:**\n- Accuracy: 92.9%\n- F1 Score: 92.5%\n- Answer Rate: 99%\n\n**Response Distribution:**\n- Correct: 92%\n- Incorrect: 7%\n- Not Attempted: 1%\n\n**Cost Efficiency:**\n- Total Cost: $9.60\n- Average Cost per Query: $0.096\n\nThis evaluation demonstrates strong performance in factual accuracy while maintaining reasonable cost efficiency. The high answer rate (99%) and accuracy (92.9%) suggest that GPT-Researcher is effective at finding and reporting accurate information.\n\n## Historical Context\n\nThese logs are maintained in version control to:\n1. Track performance improvements over time\n2. Provide benchmarks for future enhancements\n3. Enable analysis of different configurations\n4. Ensure transparency in our evaluation process\n\nEach log file contains detailed information about:\n- Individual query results\n- Source citations\n- Cost breakdowns\n- Error analysis\n- Aggregate metrics\n\n## Running New Evaluations\n\nTo generate new evaluation logs, see the [main evaluation documentation](../README.md) for instructions on running evaluations with different configurations or sample sizes. ","skills/gpt-researcher/SKILL.md":"---\nname: gpt-researcher\ndescription: Autonomous deep research from Codex via MCP\n---\n\n# GPT Researcher for Codex\n\nUse GPT Researcher from Codex via MCP.\n\n## When to use\n- When you need gpt-researcher capabilities in your Codex workflow\n- See https://github.com/assafelovic/gpt-researcher for full setup instructions\n"},"items":[{"name":"adding-features.md","path":".claude/references/adding-features.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/adding-features.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Adding Features Guide\n\n## Table of Contents\n- [The 8-Step Pattern](#the-8-step-pattern)\n- [Image Generation Case Study](#image-generation-case-study)\n- [Testing New Features](#testing-new-features)\n\n---\n\n## The 8-Step Pattern\n\n```\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│1.CONFIG│ → │2.PROVIDER│ → │3.SKILL │ → │4.AGENT │\n└────────┘ └────────┘ └────────┘ └────────┘\n ↓ ↓ ↓ ↓\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│5.PROMPTS│ → │6.WEBSOCKET│→ │7.FRONTEND│→ │8.DOCS │\n└────────┘ └────────┘ └────────┘ └────────┘\n```\n\n### Step 1: Add Configuration\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n \"MY_FEATURE_ENABLED\": False,\n \"MY_FEATURE_MODEL\": \"model-name\",\n \"MY_FEATURE_MAX_ITEMS\": 3,\n}\n```\n\n**File:** `gpt_researcher/config/variables/base.py`\n\n```python\nclass BaseConfig(TypedDict):\n \"MY_FEATURE_ENABLED\": bool\n \"MY_FEATURE_MODEL\": Union[str, None]\n \"MY_FEATURE_MAX_ITEMS\": int\n```\n\n### Step 2: Create Provider\n\n**File:** `gpt_researcher/llm_provider/my_feature/my_provider.py`\n\n```python\nclass MyFeatureProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"MY_API_KEY\")\n self.model = model\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n async def execute(self, input_data: str) -> Dict[str, Any]:\n # API implementation\n pass\n```\n\nExport in `gpt_researcher/llm_provider/__init__.py`.\n\n### Step 3: Create Skill\n\n**File:** `gpt_researcher/skills/my_feature.py`\n\n```python\nclass MyFeatureSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.provider = MyFeatureProvider(...)\n \n def is_enabled(self) -> bool:\n return getattr(self.config, 'my_feature_enabled', False) and self.provider.is_enabled()\n \n async def execute(self, context: str, query: str) -> List[Dict]:\n if not self.is_enabled():\n return []\n \n await stream_output(\"logs\", \"my_feature_start\", \"🚀 Starting...\", self.researcher.websocket)\n results = await self.provider.execute(context)\n await stream_output(\"logs\", \"my_feature_complete\", \"✅ Done\", self.researcher.websocket)\n \n return results\n```\n\nExport in `gpt_researcher/skills/__init__.py`.\n\n### Step 4: Integrate into Agent\n\n**File:** `gpt_researcher/agent.py`\n\n```python\ndef __init__(self, ...):\n if self.cfg.my_feature_enabled:\n from gpt_researcher.skills import MyFeatureSkill\n self.my_feature = MyFeatureSkill(self)\n else:\n self.my_feature = None\n self.my_feature_results = []\n\nasync def conduct_research(self, ...):\n # ... existing ...\n if self.my_feature and self.my_feature.is_enabled():\n self.my_feature_results = await self.my_feature.execute(self.context, self.query)\n```\n\n### Step 5: Update Prompts\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_my_feature_prompt(context: str, query: str) -> str:\n return f\"\"\"...\"\"\"\n```\n\n### Step 6: WebSocket Events\n\nAlready handled via `stream_output()` in skill.\n\n### Step 7: Frontend (if needed)\n\n**File:** `frontend/nextjs/hooks/useWebSocket.ts`\n\n```typescript\nif (data.content === 'my_feature_start') {\n setStatus('processing');\n}\n```\n\n### Step 8: Documentation\n\nCreate `docs/docs/gpt-researcher/gptr/my_feature.md`.\n\n---\n\n## Image Generation Case Study\n\nThis section shows the **actual implementation** of the Image Generation feature as a reference.\n\n### 1. Configuration Added\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n # ... existing ...\n \"IMAGE_GENERATION_MODEL\": \"models/gemini-2.5-flash-image\",\n \"IMAGE_GENERATION_MAX_IMAGES\": 3,\n \"IMAGE_GENERATION_ENABLED\": False,\n \"IMAGE_GENERATION_STYLE\": \"dark\", # dark, light, auto\n}\n```\n\n### 2. Provider Created\n\n**File:** `gpt_researcher/llm_provider/image/image_generator.py`\n\n```python\nclass ImageGeneratorProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"GOOGLE_API_KEY\")\n self.model = model or \"models/gemini-2.5-flash-image\"\n self._client = None\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n def _build_enhanced_prompt(self, prompt: str, context: str = \"\", style: str = \"dark\") -> str:\n \"\"\"Add styling instructions to prompt.\"\"\"\n if style == \"dark\":\n style_instructions = \"\"\"\n Style: Dark mode professional infographic\n - Background: Dark (#0d1117)\n - Accents: Teal/cyan (#14b8a6)\n - Clean, modern, minimalist\n \"\"\"\n # ... handle light, auto\n return f\"{style_instructions}\\n\\nCreate: {prompt}\\n\\nContext: {context}\"\n \n async def generate_image(\n self,\n prompt: str,\n context: str = \"\",\n research_id: str = \"\",\n style: str = \"dark\",\n ) -> List[Dict[str, Any]]:\n \"\"\"Generate image using Gemini.\"\"\"\n full_prompt = self._build_enhanced_prompt(prompt, context, style)\n \n # Call Gemini API\n response = await self._generate_with_gemini(full_prompt, output_path, ...)\n \n return [{\"url\": f\"/outputs/images/{research_id}/img_{hash}.png\", ...}]\n```\n\n### 3. Skill Created\n\n**File:** `gpt_researcher/skills/image_generator.py`\n\n```python\nclass ImageGenerator:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.image_provider = ImageGeneratorProvider(\n api_key=os.getenv(\"GOOGLE_API_KEY\"),\n model=getattr(self.config, 'image_generation_model', None),\n )\n self.max_images = getattr(self.config, 'image_generation_max_images', 3)\n self.style = getattr(self.config, 'image_generation_style', 'dark')\n \n def is_enabled(self) -> bool:\n enabled = getattr(self.config, 'image_generation_enabled', False)\n return enabled and self.image_provider.is_enabled()\n \n async def plan_and_generate_images(\n self,\n research_context: str,\n research_query: str,\n research_id: str,\n websocket: Any,\n ) -> List[Dict[str, Any]]:\n \"\"\"\n 1. Use LLM to identify visual concepts from context\n 2. Generate images in parallel\n 3. Return list of image metadata\n \"\"\"\n # Stream progress\n await stream_output(\"logs\", \"image_planning\", \"🎨 Planning images...\", websocket)\n \n # LLM identifies concepts\n concepts = await self._plan_image_concepts(research_context, research_query)\n \n # Generate images in parallel\n generated_images = []\n for i, concept in enumerate(concepts[:self.max_images]):\n await stream_output(\"logs\", \"image_generating\", \n f\"🖼️ Generating image {i+1}/{len(concepts)}...\", websocket)\n \n images = await self.image_provider.generate_image(\n prompt=concept[\"prompt\"],\n context=concept.get(\"context\", \"\"),\n research_id=research_id,\n style=self.style,\n )\n generated_images.extend(images)\n \n await stream_output(\"logs\", \"images_ready\", \n f\"✅ Generated {len(generated_images)} images\", websocket)\n \n return generated_images\n```\n\n### 4. Agent Integration\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nclass GPTResearcher:\n def __init__(self, ...):\n # ... existing init ...\n \n # Initialize image generator if enabled\n if self.cfg.image_generation_enabled:\n from gpt_researcher.skills import ImageGenerator\n self.image_generator = ImageGenerator(self)\n else:\n self.image_generator = None\n \n self.available_images: List[Dict[str, Any]] = []\n self.research_id = self._generate_research_id(query)\n \n async def conduct_research(self, on_progress=None):\n # ... existing research ...\n \n self.context = await self.research_conductor.conduct_research()\n \n # Pre-generate images after research, before report writing\n if self.cfg.image_generation_enabled and self.image_generator and self.image_generator.is_enabled():\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n \n async def write_report(self, ...):\n report = await self.report_generator.write_report(\n # ... existing params ...\n available_images=self.available_images, # Pass to report writer\n )\n return report\n```\n\n### 5. Prompt Updated\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_report_prompt(..., available_images: List[Dict[str, Any]] = []):\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title', 'Untitled')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES - Embed where relevant using ![Title](URL):\n{image_list}\n\"\"\"\n \n return f\"\"\"...(existing prompt)...\n{image_instruction}\n\"\"\"\n```\n\n---\n\n## Testing New Features\n\n```python\n# tests/test_my_feature.py\nimport pytest\nfrom gpt_researcher import GPTResearcher\n\n@pytest.mark.asyncio\nasync def test_my_feature_disabled():\n \"\"\"Test that feature is skipped when disabled.\"\"\"\n researcher = GPTResearcher(query=\"test\")\n # MY_FEATURE_ENABLED defaults to False\n assert researcher.my_feature is None\n\n@pytest.mark.asyncio\nasync def test_my_feature_enabled(monkeypatch):\n \"\"\"Test feature execution when enabled.\"\"\"\n monkeypatch.setenv(\"MY_FEATURE_ENABLED\", \"true\")\n monkeypatch.setenv(\"MY_API_KEY\", \"test-key\")\n \n researcher = GPTResearcher(query=\"test\")\n assert researcher.my_feature is not None\n assert researcher.my_feature.is_enabled()\n```\n\n### Running Tests\n\n```bash\n# All tests\npython -m pytest tests/\n\n# Specific test\npython -m pytest tests/test_my_feature.py -v\n\n# With coverage\npython -m pytest tests/ --cov=gpt_researcher\n```\n","isInternal":false,"tokens":2496,"sizeBytes":10941},{"name":"advanced-patterns.md","path":".claude/references/advanced-patterns.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/advanced-patterns.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Advanced Patterns Reference\n\n## Table of Contents\n- [Custom Callbacks](#custom-callbacks)\n- [Custom WebSocket Handler](#custom-websocket-handler)\n- [LangChain Integration](#langchain-integration)\n- [Search Restrictions](#search-restrictions)\n- [Error Handling Patterns](#error-handling-patterns)\n\n---\n\n## Custom Callbacks\n\n```python\ndef cost_callback(cost: float):\n print(f\"API call cost: ${cost}\")\n\nresearcher = GPTResearcher(query=\"...\")\nresearcher.add_costs = cost_callback # Override cost tracking\n```\n\n---\n\n## Custom WebSocket Handler\n\n```python\nclass CustomWebSocket:\n def __init__(self):\n self.messages = []\n \n async def send_json(self, data):\n self.messages.append(data)\n if data['type'] == 'logs':\n print(f\"Progress: {data['output']}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=CustomWebSocket())\n```\n\n---\n\n## LangChain Integration\n\n### Using with LangChain Documents\n\n```python\nfrom langchain.document_loaders import DirectoryLoader\n\nloader = DirectoryLoader('./docs', glob=\"**/*.md\")\ndocuments = loader.load()\n\nresearcher = GPTResearcher(\n query=\"Summarize the documentation\",\n report_source=\"langchain_documents\",\n documents=documents,\n)\n```\n\n### Using with Vector Store\n\n```python\nfrom langchain.vectorstores import Chroma\n\nvectorstore = Chroma.from_documents(documents, embeddings)\n\nresearcher = GPTResearcher(\n query=\"Find relevant information\",\n report_source=\"langchain_vectorstore\",\n vector_store=vectorstore,\n vector_store_filter={\"source\": \"docs\"},\n)\n```\n\n---\n\n## Search Restrictions\n\n### Restricting Search Domains\n\n```python\nresearcher = GPTResearcher(\n query=\"Company news\",\n query_domains=[\"reuters.com\", \"bloomberg.com\", \"wsj.com\"],\n)\n```\n\n### Using Specific Source URLs\n\n```python\nresearcher = GPTResearcher(\n query=\"Analyze these articles\",\n source_urls=[\n \"https://example.com/article1\",\n \"https://example.com/article2\",\n ],\n complement_source_urls=True, # Also do web search\n)\n```\n\n---\n\n## Error Handling Patterns\n\n### Graceful Degradation\n\n```python\n# In skills, always check is_enabled()\nasync def execute(self, ...):\n if not self.is_enabled():\n logger.warning(\"Feature not enabled, skipping\")\n return [] # Return empty, don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n logger.error(f\"Feature error: {e}\")\n await stream_output(\"logs\", \"feature_error\", f\"⚠️ Error: {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n### API Rate Limiting\n\n```python\n# Providers should handle rate limits\nasync def execute(self, ...):\n try:\n return await self._call_api(...)\n except RateLimitError as e:\n logger.warning(f\"Rate limited, waiting...\")\n await asyncio.sleep(60)\n return await self._call_api(...) # Retry\n```\n\n### WebSocket None Check\n\n```python\n# Always check websocket before sending\nif self.researcher.websocket:\n await stream_output(\"logs\", \"event\", \"message\", self.researcher.websocket)\n```\n","isInternal":false,"tokens":714,"sizeBytes":3098},{"name":"api-reference.md","path":".claude/references/api-reference.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/api-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# API Reference\n\n## Table of Contents\n- [REST API](#rest-api)\n- [WebSocket API](#websocket-api)\n- [Python Client](#python-client)\n- [Output Files](#output-files)\n\n---\n\n## REST API\n\nBase URL: `http://localhost:8000`\n\n### Generate Report\n\n**POST `/report/`**\n\n```json\n{\n \"task\": \"What are the latest AI developments?\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"query_domains\": [],\n \"generate_in_background\": false\n}\n```\n\n**Response:**\n\n```json\n{\n \"research_id\": \"task_1234567890_query\",\n \"research_information\": {\n \"source_urls\": [\"...\"],\n \"research_costs\": 0.05,\n \"visited_urls\": [\"...\"],\n \"research_images\": [\"...\"]\n },\n \"report\": \"# Research Report\\n\\n...\",\n \"docx_path\": \"outputs/task_123.docx\",\n \"pdf_path\": \"outputs/task_123.pdf\"\n}\n```\n\n### Chat with Report\n\n**POST `/api/chat`**\n\n```json\n{\n \"report\": \"The full report text...\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"What are the key findings?\"}\n ]\n}\n```\n\n### Report Management\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/api/reports` | List all reports |\n| GET | `/api/reports/{id}` | Get single report |\n| POST | `/api/reports` | Create/update report |\n| PUT | `/api/reports/{id}` | Update report |\n| DELETE | `/api/reports/{id}` | Delete report |\n\n### File Operations\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/files/` | List uploaded documents |\n| POST | `/upload/` | Upload document |\n| DELETE | `/files/{filename}` | Delete file |\n| GET | `/outputs/{filename}` | Get output file |\n\n---\n\n## WebSocket API\n\n**Endpoint:** `ws://localhost:8000/ws`\n\n### Send Research Request\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"mcp_enabled\": false,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": []\n}\n```\n\n### Message Types (Server → Client)\n\n| Type | Content | Description |\n|------|---------|-------------|\n| `logs` | `starting_research` | Research initiated |\n| `logs` | `planning_research` | Generating sub-queries |\n| `logs` | `running_subquery_research` | Researching sub-query |\n| `logs` | `research_step_finalized` | Research complete |\n| `logs` | `agent_generated` | Agent role selected |\n| `logs` | `scraping_urls` | Scraping web pages |\n| `logs` | `mcp_optimization` | MCP processing |\n| `logs` | `image_planning` | Planning images |\n| `logs` | `images_ready` | Images generated |\n| `report` | - | Streaming report chunks |\n| `report_complete` | - | Final complete report |\n| `path` | `pdf`, `docx`, `md` | Output file paths |\n| `error` | - | Error messages |\n| `human_feedback` | `request` | Request user input |\n\n### Message Format\n\n```json\n{\n \"type\": \"logs\",\n \"content\": \"starting_research\",\n \"output\": \"🔍 Starting the research task...\",\n \"metadata\": null\n}\n```\n\n### Frontend Handler Example\n\n```typescript\nws.onmessage = (event) => {\n const data = JSON.parse(event.data);\n \n switch (data.type) {\n case 'logs':\n setLogs(prev => [...prev, data]);\n break;\n case 'report':\n setAnswer(prev => prev + data.output);\n break;\n case 'report_complete':\n setAnswer(data.output);\n break;\n case 'path':\n setPaths(prev => ({...prev, [data.content]: data.output}));\n break;\n case 'error':\n setError(data.output);\n break;\n }\n};\n```\n\n---\n\n## Python Client\n\n### Basic Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\",\n )\n \n await researcher.conduct_research()\n report = await researcher.write_report()\n \n print(f\"Report: {report}\")\n print(f\"Costs: ${researcher.get_costs()}\")\n\nasyncio.run(main())\n```\n\n### With MCP\n\n```python\nresearcher = GPTResearcher(\n query=\"Research topic\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\",\n)\n```\n\n### With WebSocket Streaming\n\n```python\nclass MockWebSocket:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(\n query=\"Research topic\",\n websocket=MockWebSocket(),\n)\n```\n\n### GPTResearcher Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `query` | str | required | Research question |\n| `report_type` | str | `research_report` | Type of report |\n| `report_source` | str | `web` | Data source |\n| `tone` | Tone | `Objective` | Writing tone |\n| `source_urls` | list | `[]` | Specific URLs to research |\n| `document_urls` | list | `[]` | Document URLs |\n| `query_domains` | list | `[]` | Restrict to domains |\n| `config_path` | str | None | Path to JSON config |\n| `websocket` | WebSocket | None | For streaming |\n| `mcp_configs` | list | `[]` | MCP server configs |\n| `mcp_strategy` | str | `fast` | MCP strategy |\n| `verbose` | bool | `True` | Verbose output |\n\n---\n\n## Output Files\n\n```\noutputs/\n├── task_{timestamp}_{query}.md\n├── task_{timestamp}_{query}.pdf\n├── task_{timestamp}_{query}.docx\n└── images/\n └── {research_id}/\n └── img_{hash}_{index}.png\n```\n\n---\n\n## Error Codes\n\n| Code | Description |\n|------|-------------|\n| 400 | Bad Request - Invalid parameters |\n| 404 | Not Found - Report not found |\n| 429 | Rate Limited - API quota exceeded |\n| 500 | Internal Server Error |\n","isInternal":false,"tokens":1523,"sizeBytes":5835},{"name":"architecture.md","path":".claude/references/architecture.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/architecture.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Architecture Reference\n\n## Table of Contents\n- [System Layers](#system-layers)\n- [Key File Locations](#key-file-locations)\n\n---\n\n## System Layers\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ USER REQUEST │\n│ (query, report_type, report_source, tone, mcp_configs) │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ BACKEND API LAYER │\n│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │\n│ │ FastAPI Server │ │ WebSocket Manager│ │ Report Store │ │\n│ │ backend/server/ │ │ Real-time events │ │ JSON persistence│ │\n│ │ app.py │ │ websocket_mgr.py │ │ report_store.py │ │\n│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ GPTResearcher (gpt_researcher/agent.py) │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ SKILLS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ ResearchConductor│ │ ReportGenerator │ │ ContextManager │ │ │\n│ │ │ Plan & gather │ │ Write reports │ │ Similarity search│ │ │\n│ │ │ researcher.py │ │ writer.py │ │ context_manager │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ BrowserManager │ │ SourceCurator │ │ ImageGenerator │ │ │\n│ │ │ Web scraping │ │ Rank sources │ │ Gemini images │ │ │\n│ │ │ browser.py │ │ curator.py │ │ image_generator │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ DeepResearchSkill│ │ │\n│ │ │ Recursive depth │ │ │\n│ │ │ deep_research.py│ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ ACTIONS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ report_generation│ │ query_processing│ │ web_scraping │ │ │\n│ │ │ LLM report write│ │ Sub-query plan │ │ URL scraping │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ retriever.py │ │ agent_creator │ │ markdown_process│ │ │\n│ │ │ Get retrievers │ │ Choose agent │ │ Parse markdown │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ PROVIDERS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ LLM Provider │ │ Retrievers │ │ Scrapers │ │ │\n│ │ │ OpenAI,Anthropic│ │ Tavily,Google │ │ BS4,Playwright │ │ │\n│ │ │ Google,Groq... │ │ Bing,MCP... │ │ PDF,DOCX... │ │ │\n│ │ │ llm_provider/ │ │ retrievers/ │ │ scraper/ │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ ImageGenerator │ │ │\n│ │ │ Gemini/Imagen │ │ │\n│ │ │ llm_provider/ │ │ │\n│ │ │ image/ │ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ CONFIGURATION LAYER │\n│ gpt_researcher/config/ │\n│ │\n│ Environment Variables → JSON Config File → Default Values │\n│ (highest) (medium) (lowest) │\n│ │\n│ config.py loads and merges all sources │\n│ variables/default.py contains all defaults │\n│ variables/base.py defines TypedDict for type safety │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes/Functions |\n|------|--------------|----------------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| Context/embeddings | `gpt_researcher/skills/context_manager.py` | `ContextManager` |\n| Source ranking | `gpt_researcher/skills/curator.py` | `SourceCurator` |\n| Deep research | `gpt_researcher/skills/deep_research.py` | `DeepResearchSkill` |\n| Image generation | `gpt_researcher/skills/image_generator.py` | `ImageGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| Config types | `gpt_researcher/config/variables/base.py` | `BaseConfig` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| WebSocket mgmt | `backend/server/websocket_manager.py` | `WebSocketManager`, `run_agent` |\n| Report types | `backend/report_type/` | `BasicReport`, `DetailedReport` |\n| Search engines | `gpt_researcher/retrievers/` | `TavilySearch`, `GoogleSearch`, etc. |\n| Web scraping | `gpt_researcher/scraper/` | Various scrapers |\n| Enums | `gpt_researcher/utils/enum.py` | `ReportType`, `ReportSource`, `Tone` |\n","isInternal":false,"tokens":1872,"sizeBytes":11977},{"name":"components.md","path":".claude/references/components.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/components.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Core Components & Method Signatures\n\n## Table of Contents\n- [GPTResearcher](#gptresearcher)\n- [ResearchConductor](#researchconductor)\n- [ReportGenerator](#reportgenerator)\n\n---\n\n## GPTResearcher\n\n**File:** `gpt_researcher/agent.py`\n\nThe main orchestrator class. Full initialization signature:\n\n```python\nclass GPTResearcher:\n def __init__(\n self,\n query: str, # Research question (required)\n report_type: str = \"research_report\", # research_report, detailed_report, deep, outline_report, resource_report\n report_format: str = \"markdown\", # Output format\n report_source: str = \"web\", # web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n tone: Tone = Tone.Objective, # Writing tone (see Tone enum)\n source_urls: list[str] | None = None, # Specific URLs to research\n document_urls: list[str] | None = None, # Document URLs to include\n complement_source_urls: bool = False, # Add web search to source_urls\n query_domains: list[str] | None = None, # Restrict search to domains\n documents=None, # LangChain document objects\n vector_store=None, # LangChain vector store\n vector_store_filter=None, # Filter for vector store\n config_path=None, # Path to JSON config file\n websocket=None, # WebSocket for streaming\n agent=None, # Pre-defined agent type\n role=None, # Pre-defined agent role\n parent_query: str = \"\", # Parent query for subtopics\n subtopics: list | None = None, # Subtopics to research\n visited_urls: set | None = None, # Already visited URLs\n verbose: bool = True, # Verbose logging\n context=None, # Pre-loaded context\n headers: dict | None = None, # HTTP headers\n max_subtopics: int = 5, # Max subtopics for detailed\n log_handler=None, # Custom log handler\n prompt_family: str | None = None, # Custom prompt family\n mcp_configs: list[dict] | None = None, # MCP server configurations\n mcp_max_iterations: int | None = None, # Deprecated, use mcp_strategy\n mcp_strategy: str | None = None, # fast, deep, disabled\n **kwargs\n ):\n```\n\n### Key Methods\n\n```python\nasync def conduct_research(self, on_progress=None) -> str:\n \"\"\"\n Main research orchestration.\n \n 1. Selects agent role via LLM (choose_agent)\n 2. Delegates to ResearchConductor\n 3. Optionally generates images if enabled\n \n Returns: Accumulated research context as string\n \"\"\"\n\nasync def write_report(\n self, \n existing_headers: list = [], # Headers to avoid duplication\n relevant_written_contents: list = [], # Previous content for context\n ext_context=None, # External context override\n custom_prompt=\"\" # Custom prompt override\n) -> str:\n \"\"\"\n Generate final report from context.\n \n Returns: Markdown report string\n \"\"\"\n\ndef get_costs(self) -> float:\n \"\"\"Returns total accumulated API costs.\"\"\"\n\ndef add_costs(self, cost: float) -> None:\n \"\"\"Add to running cost total (used as callback).\"\"\"\n```\n\n---\n\n## ResearchConductor\n\n**File:** `gpt_researcher/skills/researcher.py`\n\nManages the research process:\n\n```python\nclass ResearchConductor:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.logger = logging.getLogger(__name__)\n\n async def plan_research(self, query: str, query_domains=None) -> list:\n \"\"\"\n Generate sub-queries from main query using LLM.\n \n 1. Gets initial search results\n 2. Calls plan_research_outline() to generate sub-queries\n \n Returns: List of sub-query strings\n \"\"\"\n\n async def conduct_research(self) -> str:\n \"\"\"\n Main research execution based on report_source.\n \n Handles: web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n \n For each source type:\n 1. Load/search data\n 2. Process sub-queries\n 3. Combine context\n 4. Optionally curate sources\n \n Returns: Combined research context string\n \"\"\"\n\n async def _process_sub_query(\n self, \n sub_query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"\n Process a single sub-query.\n \n 1. Get MCP context (if configured, based on strategy)\n 2. Scrape URLs from search results\n 3. Get similar content via embeddings\n 4. Combine MCP + web context\n \n Returns: Combined context for this sub-query\n \"\"\"\n\n async def _get_context_by_web_search(\n self, \n query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"Web-based research with sub-query planning.\"\"\"\n\n async def _scrape_data_by_urls(\n self, \n sub_query: str, \n query_domains: list = []\n ) -> list:\n \"\"\"Search and scrape URLs for a sub-query.\"\"\"\n```\n\n---\n\n## ReportGenerator\n\n**File:** `gpt_researcher/skills/writer.py`\n\n```python\nclass ReportGenerator:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.research_params = {\n \"query\": researcher.query,\n \"agent_role_prompt\": researcher.cfg.agent_role or researcher.role,\n \"report_type\": researcher.report_type,\n \"report_source\": researcher.report_source,\n \"tone\": researcher.tone,\n \"websocket\": researcher.websocket,\n \"cfg\": researcher.cfg,\n \"headers\": researcher.headers,\n }\n\n async def write_report(\n self,\n existing_headers: list = [],\n relevant_written_contents: list = [],\n ext_context=None,\n custom_prompt=\"\",\n available_images: list = [], # Pre-generated images to embed\n ) -> str:\n \"\"\"\n Generate report using LLM.\n \n Calls generate_report() action with context and images.\n \n Returns: Markdown report\n \"\"\"\n\n async def write_introduction(self, ...) -> str:\n \"\"\"Write report introduction section.\"\"\"\n\n async def write_conclusion(self, ...) -> str:\n \"\"\"Write report conclusion with references.\"\"\"\n```\n","isInternal":false,"tokens":1432,"sizeBytes":6699},{"name":"config-reference.md","path":".claude/references/config-reference.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/config-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Configuration Reference\n\n## Table of Contents\n- [Required Variables](#required-variables)\n- [LLM Configuration](#llm-configuration)\n- [Provider API Keys](#provider-api-keys)\n- [Retriever Configuration](#retriever-configuration)\n- [Report Configuration](#report-configuration)\n- [Feature Toggles](#feature-toggles)\n- [Configuration Priority](#configuration-priority)\n- [Example .env](#example-env)\n\n---\n\n## Required Variables\n\n```bash\nOPENAI_API_KEY=sk-... # Or another LLM provider key\nTAVILY_API_KEY=tvly-... # Or another retriever key\n```\n\n---\n\n## LLM Configuration\n\n```bash\nLLM_PROVIDER=openai # openai, anthropic, google, groq, together, etc.\nFAST_LLM=gpt-4o-mini # Quick tasks (summarization)\nSMART_LLM=gpt-4o # Complex reasoning (report writing)\nSTRATEGIC_LLM=o3-mini # Planning (agent selection)\nTEMPERATURE=0.4 # 0.0-1.0\nMAX_TOKENS=4000\nREASONING_EFFORT=medium # For o-series: low, medium, high\n```\n\n---\n\n## Provider API Keys\n\n```bash\n# OpenAI\nOPENAI_API_KEY=sk-...\nOPENAI_BASE_URL=https://api.openai.com/v1\n\n# Anthropic\nANTHROPIC_API_KEY=sk-ant-...\n\n# Google\nGOOGLE_API_KEY=AIza...\n\n# Groq\nGROQ_API_KEY=gsk_...\n\n# Azure OpenAI\nAZURE_OPENAI_API_KEY=...\nAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\n```\n\n---\n\n## Retriever Configuration\n\n```bash\nRETRIEVER=tavily # Single or comma-separated: tavily,google,mcp\nMAX_SEARCH_RESULTS_PER_QUERY=5\nMAX_URLS_TO_SCRAPE=10\nSIMILARITY_THRESHOLD=0.42\n```\n\n### Retriever API Keys\n\n```bash\nTAVILY_API_KEY=tvly-...\nGOOGLE_API_KEY=AIza...\nGOOGLE_CX_KEY=...\nBING_API_KEY=...\nSERPER_API_KEY=...\nSERPAPI_API_KEY=...\nEXA_API_KEY=...\n```\n\n---\n\n## Report Configuration\n\n```bash\nREPORT_FORMAT=apa # apa, mla, chicago, harvard, ieee\nTOTAL_WORDS=1000\nLANGUAGE=english\nCURATE_SOURCES=true\n```\n\n---\n\n## Feature Toggles\n\n### Image Generation\n\n```bash\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza...\nIMAGE_GENERATION_MODEL=models/gemini-2.5-flash-image\nIMAGE_GENERATION_MAX_IMAGES=3\nIMAGE_GENERATION_STYLE=dark # dark, light, auto\n```\n\n### Deep Research\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n### MCP\n\n```bash\nMCP_STRATEGY=fast # fast, deep, disabled\n```\n\n### Local Documents\n\n```bash\nDOC_PATH=./my-docs\n# Supports: PDF, DOCX, TXT, CSV, XLSX, PPTX, MD\n```\n\n### Server\n\n```bash\nHOST=0.0.0.0\nPORT=8000\nVERBOSE=true\n```\n\n---\n\n## Configuration Priority\n\n```\nEnvironment Variables (highest)\n ↓\nJSON Config File (if provided)\n ↓\nDefault Values (lowest)\n```\n\n**Important:** Config keys are lowercased when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\n---\n\n## Example .env\n\n```bash\n# Required\nOPENAI_API_KEY=sk-your-key\nTAVILY_API_KEY=tvly-your-key\n\n# LLM\nFAST_LLM=gpt-4o-mini\nSMART_LLM=gpt-4o\n\n# Report\nTOTAL_WORDS=1000\nLANGUAGE=english\n\n# Optional: Images\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza-your-key\nIMAGE_GENERATION_STYLE=dark\n```\n","isInternal":false,"tokens":922,"sizeBytes":3127},{"name":"deep-research.md","path":".claude/references/deep-research.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/deep-research.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Deep Research Mode Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [DeepResearchSkill](#deepresearchskill)\n- [Usage](#usage)\n\n---\n\n## Overview\n\nDeep Research uses recursive tree-like exploration with configurable depth and breadth.\n\n---\n\n## Configuration\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n---\n\n## DeepResearchSkill\n\n**File:** `gpt_researcher/skills/deep_research.py`\n\n```python\nclass DeepResearchSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.breadth = getattr(researcher.cfg, 'deep_research_breadth', 4)\n self.depth = getattr(researcher.cfg, 'deep_research_depth', 2)\n self.concurrency_limit = getattr(researcher.cfg, 'deep_research_concurrency', 2)\n self.learnings = []\n self.research_sources = []\n self.context = []\n\n async def deep_research(self, query: str, on_progress=None) -> str:\n \"\"\"\n Recursive research with depth and breadth.\n \n 1. Research main topic\n 2. Generate subtopics (breadth)\n 3. For each subtopic, recursively research (depth)\n 4. Aggregate all findings\n 5. Generate comprehensive report\n \"\"\"\n```\n\n---\n\n## Usage\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers deep research\n)\nawait researcher.conduct_research()\nreport = await researcher.write_report()\n```\n\n### Research Tree Structure\n\n```\nQuery: \"Quantum Computing\"\n├── Subtopic 1: Hardware (depth 1)\n│ ├── Subtopic 1.1: Superconducting qubits (depth 2)\n│ └── Subtopic 1.2: Ion traps (depth 2)\n├── Subtopic 2: Algorithms (depth 1)\n│ ├── Subtopic 2.1: Shor's algorithm (depth 2)\n│ └── Subtopic 2.2: Grover's algorithm (depth 2)\n├── Subtopic 3: Applications (depth 1)\n│ └── ...\n└── Subtopic 4: Challenges (depth 1)\n └── ...\n```\n\nWith `DEEP_RESEARCH_BREADTH=4` and `DEEP_RESEARCH_DEPTH=2`, this explores 4 subtopics at each level, going 2 levels deep.\n","isInternal":false,"tokens":580,"sizeBytes":2204},{"name":"flows.md","path":".claude/references/flows.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/flows.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Research Flow & Data Flow\n\n## Table of Contents\n- [End-to-End Research Flow](#end-to-end-research-flow)\n- [Data Flow Between Components](#data-flow-between-components)\n\n---\n\n## End-to-End Research Flow\n\n### 1. Request Entry\n\n**File:** `backend/server/app.py`\n\n```python\n# REST API endpoint\n@app.post(\"/report/\")\nasync def generate_report(research_request: ResearchRequest, background_tasks: BackgroundTasks):\n research_id = sanitize_filename(f\"task_{int(time.time())}_{research_request.task}\")\n # Calls write_report() which uses run_agent()\n\n# WebSocket endpoint\n@app.websocket(\"/ws\")\nasync def websocket_endpoint(websocket: WebSocket):\n await manager.connect(websocket)\n await handle_websocket_communication(websocket, manager)\n```\n\n### 2. Agent Runner\n\n**File:** `backend/server/websocket_manager.py`\n\n```python\nasync def run_agent(task, report_type, report_source, source_urls, ...):\n \"\"\"Main entry point for research execution.\"\"\"\n # Create logs handler\n logs_handler = CustomLogsHandler(websocket, task)\n \n # Configure MCP if enabled\n if mcp_enabled and mcp_configs:\n os.environ[\"RETRIEVER\"] = f\"{current_retriever},mcp\"\n os.environ[\"MCP_STRATEGY\"] = mcp_strategy\n \n # Route based on report type\n if report_type == \"multi_agents\":\n report = await run_research_task(query=task, websocket=logs_handler, ...)\n elif report_type == ReportType.DetailedReport.value:\n researcher = DetailedReport(query=task, ...)\n report = await researcher.run()\n else:\n researcher = BasicReport(query=task, ...)\n report = await researcher.run()\n \n return report\n```\n\n### 3. Research Phase\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nasync def conduct_research(self, on_progress=None):\n # Handle deep research separately\n if self.report_type == ReportType.DeepResearch.value and self.deep_researcher:\n return await self._handle_deep_research(on_progress)\n \n # Choose agent role via LLM\n if not (self.agent and self.role):\n self.agent, self.role = await choose_agent(\n query=self.query,\n cfg=self.cfg,\n parent_query=self.parent_query,\n cost_callback=self.add_costs,\n headers=self.headers,\n prompt_family=self.prompt_family,\n )\n \n # Conduct research\n self.context = await self.research_conductor.conduct_research()\n \n # Generate images if enabled (pre-generation for seamless UX)\n if self.cfg.image_generation_enabled and self.image_generator:\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n```\n\n### 4. Sub-Query Processing\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\nasync def _process_sub_query(self, sub_query: str, scraped_data: list = [], query_domains: list = []):\n # MCP Strategy handling\n mcp_retrievers = [r for r in self.researcher.retrievers if \"mcpretriever\" in r.__name__.lower()]\n mcp_strategy = self._get_mcp_strategy()\n \n if mcp_retrievers:\n if mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n # Reuse cached MCP results\n mcp_context = self._mcp_results_cache.copy()\n elif mcp_strategy == \"deep\":\n # Run MCP for every sub-query\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n \n # Get web search context\n if not scraped_data:\n scraped_data = await self._scrape_data_by_urls(sub_query, query_domains)\n \n # Get similar content via embeddings\n if scraped_data:\n web_context = await self.researcher.context_manager.get_similar_content_by_query(\n sub_query, scraped_data\n )\n \n # Combine MCP + web context\n combined_context = self._combine_mcp_and_web_context(mcp_context, web_context, sub_query)\n return combined_context\n```\n\n### 5. Report Generation\n\n**File:** `gpt_researcher/actions/report_generation.py`\n\n```python\nasync def generate_report(\n query: str,\n context: str,\n agent_role_prompt: str,\n report_type: str,\n websocket=None,\n cfg=None,\n tone=None,\n headers=None,\n cost_callback=None,\n prompt_family=None,\n available_images: list = [],\n **kwargs\n) -> str:\n \"\"\"Generate report using LLM.\"\"\"\n # Get prompt generator\n generate_prompt = prompt_family.get_prompt_by_report_type(report_type)\n \n # Build prompt with context and available images\n content = generate_prompt(\n query, context, report_source,\n report_format=cfg.report_format,\n tone=tone,\n total_words=cfg.total_words,\n language=cfg.language,\n available_images=available_images,\n )\n \n # Call LLM\n report = await create_chat_completion(\n model=cfg.smart_llm,\n messages=[{\"role\": \"user\", \"content\": content}],\n temperature=cfg.temperature,\n llm_provider=cfg.smart_llm_provider,\n max_tokens=cfg.smart_token_limit,\n llm_kwargs=cfg.llm_kwargs,\n cost_callback=cost_callback,\n )\n \n return report\n```\n\n---\n\n## Data Flow Between Components\n\n```\nUser Query\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ GPTResearcher.__init__() │\n│ • Loads Config (env → json → defaults) │\n│ • Initializes skills: ResearchConductor, ReportGenerator, etc │\n│ • Initializes retrievers based on RETRIEVER env var │\n│ • Initializes ImageGenerator if IMAGE_GENERATION_ENABLED │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.conduct_research()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ choose_agent() │\n│ Input: query, config │\n│ Output: (agent_type: str, role_prompt: str) │\n│ • LLM selects best agent role for the query │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ResearchConductor.conduct_research() │\n│ Input: self.researcher (has query, config, retrievers) │\n│ Output: context: str │\n│ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ plan_research() │ │\n│ │ Input: query │ │\n│ │ Output: sub_queries: list[str] │ │\n│ │ • Calls LLM to generate 3-5 sub-queries │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ For each sub_query: │ │\n│ │ _process_sub_query() │ │\n│ │ Input: sub_query │ │\n│ │ Output: sub_context: str │ │\n│ │ │ │\n│ │ 1. MCP retrieval (if configured) │ │\n│ │ → mcp_context: list[dict] │ │\n│ │ │ │\n│ │ 2. Web search via retrievers │ │\n│ │ → search_results: list[dict] │ │\n│ │ │ │\n│ │ 3. Scrape URLs │ │\n│ │ → scraped_content: list[dict] │ │\n│ │ │ │\n│ │ 4. Similarity search via embeddings │ │\n│ │ → relevant_context: str │ │\n│ │ │ │\n│ │ 5. Combine MCP + web context │ │\n│ │ → combined_context: str │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ Aggregate all sub_contexts → final context: str │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ If IMAGE_GENERATION_ENABLED:\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ImageGenerator.plan_and_generate_images() │\n│ Input: context, query, research_id │\n│ Output: available_images: list[dict] │\n│ [{\"url\": \"/outputs/images/.../img.png\", │\n│ \"title\": \"...\", \"description\": \"...\"}] │\n│ │\n│ 1. LLM analyzes context for visual concepts │\n│ 2. Generates 2-3 images in parallel via Gemini │\n│ 3. Saves to outputs/images/{research_id}/ │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.write_report()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ReportGenerator.write_report() │\n│ Input: context, available_images │\n│ Output: report: str (markdown) │\n│ │\n│ → generate_report() action │\n│ • Builds prompt with context + image list │\n│ • LLM generates report with embedded images │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ Output │\n│ • Streamed via WebSocket (type: \"report\") │\n│ • Final via WebSocket (type: \"report_complete\") │\n│ • Exported to PDF, DOCX, Markdown │\n│ • Saved to outputs/ directory │\n└─────────────────────────────────────────────────────────────────┘\n```\n","isInternal":false,"tokens":2298,"sizeBytes":13378},{"name":"mcp.md","path":".claude/references/mcp.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/mcp.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# MCP Integration Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [Strategy Options](#strategy-options)\n- [Processing Logic](#processing-logic)\n\n---\n\n## Overview\n\nMCP (Model Context Protocol) enables research from specialized data sources (GitHub, databases, APIs) alongside web search.\n\n---\n\n## Configuration\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n mcp_configs=[\n {\n \"name\": \"github\", # Server name\n \"command\": \"npx\", # Command to start\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}, # Environment vars\n },\n {\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@anthropic/mcp-server-filesystem\", \"/docs\"],\n },\n {\n \"name\": \"remote\",\n \"connection_url\": \"ws://server:8080\", # WebSocket connection\n \"connection_type\": \"websocket\",\n \"connection_token\": \"auth_token\",\n }\n ],\n mcp_strategy=\"fast\", # fast, deep, disabled\n)\n```\n\n---\n\n## Strategy Options\n\n| Strategy | Behavior | Use Case |\n|----------|----------|----------|\n| `fast` (default) | Run MCP once with original query, cache results | Performance-focused |\n| `deep` | Run MCP for every sub-query | Maximum thoroughness |\n| `disabled` | Skip MCP entirely | Web-only research |\n\n---\n\n## Processing Logic\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\n# At start of research (for 'fast' strategy)\nif mcp_strategy == \"fast\":\n mcp_context = await self._execute_mcp_research_for_queries([query], mcp_retrievers)\n self._mcp_results_cache = mcp_context # Cache for reuse\n\n# During sub-query processing\nif mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n mcp_context = self._mcp_results_cache.copy() # Reuse cache\nelif mcp_strategy == \"deep\":\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n```\n\n### WebSocket Request Example\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"mcp_enabled\": true,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": [\n {\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}\n }\n ]\n}\n```\n","isInternal":false,"tokens":597,"sizeBytes":2438},{"name":"multi-agents.md","path":".claude/references/multi-agents.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/multi-agents.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Multi-Agent System Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Agent Roles](#agent-roles)\n- [Workflow](#workflow)\n- [Usage](#usage)\n\n---\n\n## Overview\n\n**Directory:** `multi_agents/`\n\nLangGraph-based multi-agent system. Generates 5-6 page reports with multiple agents collaborating.\n\n---\n\n## Agent Roles\n\n| Agent | File | Role |\n|-------|------|------|\n| Human | - | Oversees and provides feedback |\n| Chief Editor | `agents/orchestrator.py` | Master coordinator via LangGraph |\n| Researcher | Uses GPTResearcher | Deep research on topics |\n| Editor | `agents/editor.py` | Plans outline and structure |\n| Reviewer | `agents/reviewer.py` | Validates research correctness |\n| Reviser | `agents/reviser.py` | Revises based on feedback |\n| Writer | `agents/writer.py` | Compiles final report |\n| Publisher | `agents/publisher.py` | Exports to PDF, DOCX, Markdown |\n\n---\n\n## Workflow\n\n```\n1. Browser (GPTResearcher) → Initial research\n2. Editor → Plans report outline\n3. For each outline topic (parallel):\n a. Researcher → In-depth subtopic research\n b. Reviewer → Validates draft\n c. Reviser → Revises until satisfactory\n4. Writer → Compiles final report\n5. Publisher → Exports to multiple formats\n```\n\n---\n\n## Usage\n\n### Via API\n\n```python\nreport_type = \"multi_agents\"\n```\n\n### Via WebSocket\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"multi_agents\",\n \"tone\": \"Analytical\"\n}\n```\n\n### Directly in Python\n\n```python\nfrom multi_agents.main import run_research_task\n\nreport = await run_research_task(\n query=\"Comprehensive analysis of market trends\",\n websocket=handler,\n tone=Tone.Analytical,\n)\n```\n\n### Configuration File\n\n**File:** `multi_agents/task.json`\n\nConfigure the multi-agent research task parameters and agent behaviors.\n","isInternal":false,"tokens":445,"sizeBytes":1791},{"name":"prompts.md","path":".claude/references/prompts.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/prompts.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Prompt System Reference\n\n## Table of Contents\n- [PromptFamily Class](#promptfamily-class)\n- [Key Prompt Examples](#key-prompt-examples)\n\n---\n\n## PromptFamily Class\n\n**File:** `gpt_researcher/prompts.py`\n\nAll prompts are centralized in the `PromptFamily` class. This allows for model-specific prompt variations.\n\n```python\nclass PromptFamily:\n \"\"\"\n General purpose class for prompt formatting.\n Can be overwritten with model-specific derived classes.\n \"\"\"\n\n def __init__(self, config: Config):\n self.cfg = config\n\n @staticmethod\n def get_prompt_by_report_type(report_type: str):\n \"\"\"Returns the appropriate prompt generator for the report type.\"\"\"\n match report_type:\n case ReportType.ResearchReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.DetailedReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.OutlineReport.value:\n return PromptFamily.generate_outline_report_prompt\n # ... etc\n```\n\n---\n\n## Key Prompt Examples\n\n### Agent Selection Prompt\n\n```python\n@staticmethod\ndef generate_agent_role_prompt(query: str, parent_query: str = \"\") -> str:\n return f\"\"\"Analyze the research query and select the most appropriate agent role.\n\nQuery: \"{query}\"\n{f'Parent Query: \"{parent_query}\"' if parent_query else ''}\n\nBased on the query, determine:\n1. The domain expertise needed\n2. The research approach required\n3. The appropriate agent persona\n\nReturn a JSON object with:\n- \"agent\": The agent type (e.g., \"Research Analyst\", \"Technical Writer\")\n- \"role\": A detailed role description for how the agent should approach this research\n\"\"\"\n```\n\n### Research Planning Prompt\n\n```python\n@staticmethod\ndef generate_search_queries_prompt(\n query: str,\n parent_query: str = \"\",\n report_type: str = \"\",\n max_iterations: int = 3,\n context: str = \"\",\n) -> str:\n return f\"\"\"Generate {max_iterations} focused search queries to research: \"{query}\"\n\nContext from initial search:\n{context}\n\nRequirements:\n- Each query should explore a different aspect\n- Queries should be specific and searchable\n- Consider the report type: {report_type}\n\nReturn a JSON array of query strings.\n\"\"\"\n```\n\n### Report Generation Prompt (with images)\n\n```python\n@staticmethod\ndef generate_report_prompt(\n question: str,\n context: str,\n report_source: str,\n report_format=\"apa\",\n total_words=1000,\n tone=None,\n language=\"english\",\n available_images: list = [],\n) -> str:\n # Build image embedding instruction if images available\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES (embed where relevant):\n{image_list}\n\nUse markdown format: ![Title](URL)\n\"\"\"\n\n return f\"\"\"Information: \"{context}\"\n---\nUsing the above information, answer: \"{question}\" in a detailed report.\n\n- Format: {report_format}\n- Length: ~{total_words} words\n- Tone: {tone.value if tone else \"Objective\"}\n- Language: {language}\n- Include citations for all factual claims\n{image_instruction}\n\"\"\"\n```\n\n### MCP Tool Selection Prompt\n\n```python\n@staticmethod\ndef generate_mcp_tool_selection_prompt(query: str, tools_info: list, max_tools: int = 3) -> str:\n return f\"\"\"Select the most relevant tools for researching: \"{query}\"\n\nAVAILABLE TOOLS:\n{json.dumps(tools_info, indent=2)}\n\nSelect exactly {max_tools} tools ranked by relevance.\n\nReturn JSON:\n{{\n \"selected_tools\": [\n {{\"index\": 0, \"name\": \"tool_name\", \"relevance_score\": 9, \"reason\": \"...\"}}\n ]\n}}\n\"\"\"\n```\n","isInternal":false,"tokens":834,"sizeBytes":3719},{"name":"retrievers.md","path":".claude/references/retrievers.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/references/retrievers.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# Retriever System Reference\n\n## Table of Contents\n- [Available Retrievers](#available-retrievers)\n- [Retriever Selection](#retriever-selection)\n- [Adding a New Retriever](#adding-a-new-retriever)\n\n---\n\n## Available Retrievers\n\n**Directory:** `gpt_researcher/retrievers/`\n\n| Retriever | Class | API Key Env Var |\n|-----------|-------|-----------------|\n| Tavily | `TavilySearch` | `TAVILY_API_KEY` |\n| Google | `GoogleSearch` | `GOOGLE_API_KEY`, `GOOGLE_CX_KEY` |\n| Bing | `BingSearch` | `BING_API_KEY` |\n| Brave | `BraveSearch` | `BRAVE_API_KEY` |\n| DuckDuckGo | `Duckduckgo` | None |\n| SearX | `SearxSearch` | `SEARX_URL` |\n| Serper | `SerperSearch` | `SERPER_API_KEY` |\n| SerpAPI | `SerpApiSearch` | `SERPAPI_API_KEY` |\n| SearchAPI | `SearchApiSearch` | `SEARCHAPI_API_KEY` |\n| Exa | `ExaSearch` | `EXA_API_KEY` |\n| GroundRoute | `GroundRouteSearch` | `GROUNDROUTE_API_KEY` |\n| fastCRW | `CRWRetriever` | `CRW_API_KEY` |\n| BoCha | `BoChaSearch` | `BOCHA_API_KEY` |\n| Xquik | `XquikSearch` | `XQUIK_API_KEY` |\n| GetXAPI (X/Twitter) | `GetXAPISearch` | `GETXAPI_API_KEY` |\n| arXiv | `ArxivSearch` | None |\n| Semantic Scholar | `SemanticScholarSearch` | None |\n| PubMed Central | `PubMedCentralSearch` | None |\n| OpenAlex | `OpenAlexSearch` | `OPENALEX_API_KEY`, `OPENALEX_EMAIL` |\n| MCP | `MCPRetriever` | Per-server |\n| Custom | `CustomRetriever` | User-defined |\n\n> The authoritative list is the `match` statement in `gpt_researcher/actions/retriever.py` (`get_retriever`).\n\n---\n\n## Retriever Selection\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ndef get_retriever(retriever: str):\n \"\"\"Get a retriever class by name.\"\"\"\n match retriever:\n case \"tavily\":\n from gpt_researcher.retrievers import TavilySearch\n return TavilySearch\n case \"google\":\n from gpt_researcher.retrievers import GoogleSearch\n return GoogleSearch\n case \"mcp\":\n from gpt_researcher.retrievers import MCPRetriever\n return MCPRetriever\n # ... etc\n\ndef get_retrievers(headers: dict[str, str], cfg) -> list:\n \"\"\"\n Resolve which retriever(s) to use, in priority order:\n headers[\"retrievers\"] / headers[\"retriever\"] → cfg.retrievers / cfg.retriever\n → default (TavilySearch). Accepts a comma-separated string or a list.\n\n Usage: RETRIEVER=tavily,google,mcp\n \"\"\"\n # ... resolve names from headers/cfg (see actions/retriever.py) ...\n # Invalid names fall back to get_default_retriever():\n return [get_retriever(r) or get_default_retriever() for r in retriever_names]\n```\n\n---\n\n## Adding a New Retriever\n\n### Step 1: Create Retriever File\n\n**File:** `gpt_researcher/retrievers/my_retriever/my_retriever.py`\n\n```python\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n self.headers = headers\n \n async def search(self, max_results: int = 10) -> list[dict]:\n \"\"\"\n Returns list of:\n {\n \"title\": str,\n \"href\": str,\n \"body\": str\n }\n \"\"\"\n # Implementation\n pass\n```\n\n### Step 2: Register in retriever.py\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n```\n\n### Step 3: Export in __init__.py\n\n**File:** `gpt_researcher/retrievers/__init__.py`\n\n```python\nfrom .my_retriever import MyRetriever\n__all__ = [..., \"MyRetriever\"]\n```\n\n### Step 4: Usage\n\n```bash\nRETRIEVER=tavily,my_retriever\n```\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n # Will use both Tavily and your custom retriever\n)\n```\n","isInternal":false,"tokens":1099,"sizeBytes":3678},{"name":"SKILL.md","path":".claude/SKILL.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.claude/SKILL.md","title":"gpt-researcher","category":"anthropic-skill","format":"markdown","content":"---\nname: gpt-researcher\ndescription: GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines.\n---\n\n# GPT Researcher Development Skill\n\nGPT Researcher is an LLM-based autonomous agent using a planner-executor-publisher pattern with parallelized agent work for speed and reliability.\n\n## Quick Start\n\n### Basic Python Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\", # or detailed_report, deep, outline_report\n report_source=\"web\", # or local, hybrid\n )\n await researcher.conduct_research()\n report = await researcher.write_report()\n print(report)\n\nasyncio.run(main())\n```\n\n### Run Servers\n\n```bash\n# Backend\npython -m uvicorn backend.server.server:app --reload --port 8000\n\n# Frontend\ncd frontend/nextjs && npm install && npm run dev\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes |\n|------|--------------|-------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| Search engines | `gpt_researcher/retrievers/` | Various retrievers |\n\n---\n\n## Architecture Overview\n\n```\nUser Query → GPTResearcher.__init__()\n │\n ▼\n choose_agent() → (agent_type, role_prompt)\n │\n ▼\n ResearchConductor.conduct_research()\n ├── plan_research() → sub_queries\n ├── For each sub_query:\n │ └── _process_sub_query() → context\n └── Aggregate contexts\n │\n ▼\n [Optional] ImageGenerator.plan_and_generate_images()\n │\n ▼\n ReportGenerator.write_report() → Markdown report\n```\n\n**For detailed architecture diagrams**: See [references/architecture.md](references/architecture.md)\n\n---\n\n## Core Patterns\n\n### Adding a New Feature (8-Step Pattern)\n\n1. **Config** → Add to `gpt_researcher/config/variables/default.py`\n2. **Provider** → Create in `gpt_researcher/llm_provider/my_feature/`\n3. **Skill** → Create in `gpt_researcher/skills/my_feature.py`\n4. **Agent** → Integrate in `gpt_researcher/agent.py`\n5. **Prompts** → Update `gpt_researcher/prompts.py`\n6. **WebSocket** → Events via `stream_output()`\n7. **Frontend** → Handle events in `useWebSocket.ts`\n8. **Docs** → Create `docs/docs/gpt-researcher/gptr/my_feature.md`\n\n**For complete feature addition guide with Image Generation case study**: See [references/adding-features.md](references/adding-features.md)\n\n### Adding a New Retriever\n\n```python\n# 1. Create: gpt_researcher/retrievers/my_retriever/my_retriever.py\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n \n async def search(self, max_results: int = 10) -> list[dict]:\n # Return: [{\"title\": str, \"href\": str, \"body\": str}]\n pass\n\n# 2. Register in gpt_researcher/actions/retriever.py\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n\n# 3. Export in gpt_researcher/retrievers/__init__.py\n```\n\n**For complete retriever documentation**: See [references/retrievers.md](references/retrievers.md)\n\n---\n\n## Configuration\n\nConfig keys are **lowercased** when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\nPriority: Environment Variables → JSON Config File → Default Values\n\n**For complete configuration reference**: See [references/config-reference.md](references/config-reference.md)\n\n---\n\n## Common Integration Points\n\n### WebSocket Streaming\n\n```python\nclass WebSocketHandler:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=WebSocketHandler())\n```\n\n### MCP Data Sources\n\n```python\nresearcher = GPTResearcher(\n query=\"Open source AI projects\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\", # or \"fast\", \"disabled\"\n)\n```\n\n**For MCP integration details**: See [references/mcp.md](references/mcp.md)\n\n### Deep Research Mode\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers recursive tree-like exploration\n)\n```\n\n**For deep research configuration**: See [references/deep-research.md](references/deep-research.md)\n\n---\n\n## Error Handling\n\nAlways use graceful degradation in skills:\n\n```python\nasync def execute(self, ...):\n if not self.is_enabled():\n return [] # Don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n await stream_output(\"logs\", \"error\", f\"⚠️ {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n---\n\n## Critical Gotchas\n\n| ❌ Mistake | ✅ Correct |\n|-----------|-----------|\n| `config.MY_VAR` | `config.my_var` (lowercased) |\n| Editing pip-installed package | `pip install -e .` |\n| Forgetting async/await | All research methods are async |\n| `websocket.send_json()` on None | Check `if websocket:` first |\n| Not registering retriever | Add to `retriever.py` match statement |\n\n---\n\n## Reference Documentation\n\n| Topic | File |\n|-------|------|\n| System architecture & diagrams | [references/architecture.md](references/architecture.md) |\n| Core components & signatures | [references/components.md](references/components.md) |\n| Research flow & data flow | [references/flows.md](references/flows.md) |\n| Prompt system | [references/prompts.md](references/prompts.md) |\n| Retriever system | [references/retrievers.md](references/retrievers.md) |\n| MCP integration | [references/mcp.md](references/mcp.md) |\n| Deep research mode | [references/deep-research.md](references/deep-research.md) |\n| Multi-agent system | [references/multi-agents.md](references/multi-agents.md) |\n| Adding features guide | [references/adding-features.md](references/adding-features.md) |\n| Advanced patterns | [references/advanced-patterns.md](references/advanced-patterns.md) |\n| REST & WebSocket API | [references/api-reference.md](references/api-reference.md) |\n| Configuration variables | [references/config-reference.md](references/config-reference.md) |\n","frontmatter":{"name":"gpt-researcher","description":"GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines."},"isInternal":false,"tokens":1755,"sizeBytes":7327},{"name":"README.md","path":"evals/context_filter/README.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/evals/context_filter/README.md","title":"Context_filter Skill","category":"anthropic-skill","format":"markdown","content":"# Context filter benchmark: embeddings vs Jev vs no filter\n\n![Context filter benchmark results](results.png)\n\nEvery scraped page reaches a GPT Researcher report through one step: for each\nsub-query, the scraped pages are cut into chunks and a filter decides which\nchunks become the writer's context. Until now that filter was embedding\nsimilarity. This benchmark asks two questions:\n\n1. Is there a better filter? We test [TypeSafe's Jev](https://docs.typesafe.ai/),\n a model that returns calibrated, typed decisions, scoring each chunk on how\n useful it is for answering the question.\n2. With today's long context windows, do we need a filter at all?\n\n## Method\n\n**Fixed inputs.** `collect.py` runs 28 real research tasks once and records\nthe exact `(sub_query, pages)` pairs that reached the filter: 20 SimpleQA\nquestions with known answers (fixed seed, from `evals/simple_evals`) and 8\nopen-ended research questions. `replay.py` rebuilds the context from those\nsame pages with each strategy and writes the report with the same model\n(`gpt-5.4`). Search and scraping, the noisiest parts of a run, are held\nconstant, so the filter is the only thing that differs.\n\n**Strategies.**\n\n| strategy | what reaches the writer |\n|---|---|\n| `embeddings` | today: 1000-char chunks, cosine similarity > 0.42, top 10 per sub-query |\n| `jev` | same chunks, scored by Jev on a 4-level usefulness rubric, top 10 scoring ≥ 1.5 |\n| `jev_top10` | Jev's 10 best chunks, no threshold (same budget as embeddings) |\n| `jev_large` | Jev on 3000-char chunks, top 4 (≈ same characters, a third of the calls) |\n| `jev_wide` | Jev threshold 1.5, up to 30 chunks |\n| `jev_broad` | Jev on 3000-char chunks, threshold 1.0, up to 12 (open-ended only) |\n| `bm25` | keyword (BM25) ranking of the same chunks, top 10; runs locally, no API or model |\n| `bm25_rel` | BM25, keeping chunks scoring ≥ 50% of the best chunk, top 10 |\n| `bm25_wide` | BM25 with the same relative threshold, up to 25 chunks (shipped as `keyword`) |\n| `none_capped` | no ranking: pages in search order, capped at 48k characters per sub-query |\n| `none` | no filter: every page in full (capped at 50k chars per page, as today) |\n| `truncate` | control: 10 chunks round-robin across pages, no relevance ranking |\n\n**Quality signals** (`judge.py`):\n\n- **SimpleQA accuracy**: each report graded with the repo's SimpleQA grader.\n- **Context precision**: the share of context chunks an independent judge\n (`gpt-5.4-mini`) rates as relevant to the question. This measures the filter\n itself. It isn't defined for `none`.\n- **Pairwise preference**: a blind head-to-head judged by `gpt-5.4`, run in both\n orders. A win counts only if the strategy wins in *both* orders; any\n disagreement is a tie.\n\n## Results\n\n### Cost and speed (28 questions)\n\n| strategy | filter time p50 / p90 | filter $ | context tokens p50 (max) | write $ | total $ / report |\n|---|---|---|---|---|---|\n| embeddings | 1.0s / 1.3s | 0.0028 | 7.6k (10.7k) | 0.114 | **0.117** |\n| jev | 2.4s / 6.0s ¹ | 0.0054 | 4.5k (10.6k) | 0.110 | **0.115** |\n| jev_top10 | 2.7s / 6.2s | 0.0054 | 8.6k (10.6k) | 0.119 | 0.124 |\n| jev_large | 1.2s / 5.1s | 0.0031 | 6.5k | 0.111 | 0.114 |\n| truncate | 0.0s | — | 8.0k (11.0k) | 0.117 | 0.117 |\n| none | 0.0s | — | **33.4k (107.7k)** | **0.192** | **0.192** |\n\n¹ Measured with 32 concurrent Jev calls. At 64, the new default, the same\nfiltering takes **1.7s p50 / 3.6s p90**; 128 was no faster.\n\nReport writing takes the same ~45s for every strategy: generating the output\ndominates, and input size barely matters. Removing the filter makes nothing\nfaster; it makes each report 65% more expensive.\n\n### Quality\n\n| strategy | SimpleQA correct | context precision | vs embeddings (W/T/L) | vs none (W/T/L) |\n|---|---|---|---|---|\n| embeddings | 18/20 | 0.46 | — | 3 / 12 / 13 |\n| **jev** | 19/20 | **0.73** | **15 / 10 / 3** | 6 / 12 / 10 |\n| jev_top10 | 19/20 | 0.50 | 13 / 10 / 5 | — |\n| jev_large | 19/20 | 0.72 | 11 / 10 / 7 | 8 / 7 / 13 |\n| jev_wide | 20/20 | **0.74** | 13 / 8 / 7 | 7 / 11 / 10 |\n| truncate | 19/20 | 0.33 | 7 / 8 / 13 | — |\n| none | 20/20 | — | 13 / 11 / 4 | — |\n\n### Open-ended questions only (8)\n\n| strategy | context p50 | total $ / report | vs embeddings | vs none |\n|---|---|---|---|---|\n| embeddings | 8.8k | 0.155 | — | 0 / 1 / 7 |\n| jev | 9.1k | 0.160 | 5 / 2 / 1 | 0 / 2 / 6 |\n| jev_broad | 26.9k | 0.191 | **7 / 1 / 0** | 0 / 3 / 5 |\n| none | 61.3k | **0.284** | 8 / 0 / 0 | — |\n\nThe `none` reports on open-ended questions were *shorter* than the embeddings\nreports (2,099 vs 2,416 median words) and still preferred, so the judge is not\nrewarding length.\n\n### Filters that need nothing: keyword (BM25)\n\nWithout a Jev key, GPT Researcher needs a filter with no API, model or\nembeddings. The candidates, on the same 28 runs:\n\n| strategy | SimpleQA | context precision | vs embeddings (W/T/L) | vs none (W/T/L) | filter p50 | context p50 | total $ |\n|---|---|---|---|---|---|---|---|\n| bm25 | 18/20 | 0.40 | 7 / 9 / 12 | 3 / 12 / 13 | 0.02s | 8.1k | 0.118 |\n| bm25_rel | 19/20 | 0.53 | 7 / 13 / 8 | 5 / 10 / 13 | 0.02s | 4.7k | **0.107** |\n| **bm25_wide** | 19/20 | 0.51 | **14 / 8 / 6** | 6 / 9 / 13 | 0.02s | 6.4k | 0.116 |\n| none_capped | 20/20 | 0.54 | 11 / 13 / 4 | 6 / 10 / 12 | 0.00s | 23.8k | 0.152 |\n| *embeddings* | *18/20* | *0.46* | — | *3 / 12 / 13* | *1.04s* | *7.6k* | *0.117* |\n\n`bm25_wide` ships as the `keyword` fallback: at least as good as embeddings on\nevery measure, at the same cost, about 50× faster to filter, and with nothing\nto set up. Its 14–6 head-to-head is directional (sign test p ≈ 0.12), not as\nstrong as Jev's 15–3. As with Jev, the relative threshold is what does the\nwork: plain top-10 BM25 loses to embeddings 7–12.\n\nGPT Researcher removes duplicate URLs across sub-queries before scraping\n(0 of 300 pages recurred), so \"sources that came back most often\" isn't a\nsignal available at this step.\n\n## What this shows\n\n1. **Jev is a better filter than embeddings at the same cost.** 73% of the\n chunks Jev selects are relevant, against 46% for embeddings; the unranked\n control gets 33%. Jev reports beat embeddings reports 15–3 head-to-head\n (sign test on the 18 non-ties, p ≈ 0.008). Every Jev variant beat\n embeddings. Total cost per report is the same; both filters cost well\n under a cent.\n2. **Most of the gain comes from the threshold.** `jev_top10`, forced to fill\n 10 slots, drops to 0.50 precision. Cosine similarity cannot say \"nothing\n else here is worth including\"; a calibrated usefulness score can.\n3. **Skipping the filter gives the best open-ended reports, at 65–83% higher\n cost.** More source material makes for broader reports. But on a *standard*\n research report the context already reaches 108k tokens; detailed and deep\n research run many more sub-queries and would overflow context windows and\n budgets. For fact-finding questions it adds nothing: SimpleQA is at\n ceiling for every strategy, and Jev vs none is 6–10–4.\n4. **No integration is needed for a good filter.** Keyword ranking with a\n relative threshold matches or beats embeddings at the same cost (above),\n so the default without a Jev key is `keyword`, and nothing requires an\n embeddings provider.\n5. **SimpleQA is saturated.** 18–20 of 20 for every strategy: when search\n finds the fact, any reasonable filter keeps it. Differences of one question\n are noise.\n\n## Caveats\n\n- 28 questions, judged by one model family (`gpt-5.4`), which also writes the\n reports. Self-preference would apply to every strategy equally, but the\n pairwise numbers are directional evidence, not a leaderboard.\n- One writer model and one report type (`research_report`). Detailed and deep\n research were not replayed.\n- Jev latency depends on chunk count and concurrency. `jev_large` matches\n embeddings at the median; tails come from large pages.\n\n## Reproduce\n\n```bash\nexport OPENAI_API_KEY=... TAVILY_API_KEY=... TYPESAFE_API_KEY=...\npython -m evals.context_filter.collect --out runs/ # ~$0.40, ~20 min\npython -m evals.context_filter.replay --runs runs/ --out results/\npython -m evals.context_filter.judge --results results/\npython -m evals.context_filter.judge --results results/ --baseline none --only jev embeddings\npython -m evals.context_filter.replay --runs runs/ --out results/ --strategies bm25 bm25_rel bm25_wide none_capped\n```\n\n`results/` in this folder holds the scores and per-run metrics from the run\nabove (`runs.json`, `scored_*.json`, `summary_*.json`). Scraped pages and\ngenerated reports are not committed.\n\nTotal spend: about $40 in tracked writing and filtering across all strategies\n(of which $0.59 was Jev; keyword and none cost nothing to filter), plus an\nestimated $8–15 for judging.\n","isInternal":false,"tokens":2796,"sizeBytes":8791},{"name":"README.md","path":"evals/quality_eval/README.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/evals/quality_eval/README.md","title":"Quality_eval Skill","category":"anthropic-skill","format":"markdown","content":"# quality_eval — Report Quality Evaluation for Autonomous Research Agents\n\nEvaluate the quality of a GPT-Researcher report on **any open-ended query, with no\nground-truth answer required**. Where `simple_evals` needs a labelled dataset and\n`hallucination_eval` gives a binary check, `quality_eval` scores a report across\nseveral interpretable dimensions of *how well it was researched and written*.\n\nIt follows the metric abstraction of established frameworks (DeepEval / RAGAS),\nand **extends it for autonomous research** — evaluating not just the answer, but\nthe quality of the sources the agent found on its own.\n\n## What it measures\n\n| Group | Aligns with | Metric | Question it answers |\n|---|---|---|---|\n| **Faithfulness** | RAGAS Faithfulness | `unsupported_claim` | Are the report's claims grounded in the sources? |\n| **Answer Relevancy** | RAGAS answer relevancy | `subtopic_coverage` | Does it cover the subtopics a thorough report should? |\n| **Source Quality** | *extension (autonomous agent)* | `source_authority`, `source_diversity` | Are the sources the agent found authoritative and varied? |\n| **Citation** | ALCE citation precision | `citation_faithfulness` | Are the listed references actually cited in the body? |\n\nZero-cost metrics (citation, diversity, authority) run by default; the two LLM\nmetrics are opt-out via flags, so you can trade cost for depth.\n\n## Concept alignment (why this is more than ad-hoc metrics)\n\nThe suite maps onto standard evaluation ontologies where they apply, and marks\nwhere autonomous research needs more:\n\n- **Faithfulness / Answer Relevancy** ← align with **RAGAS**. `unsupported_claim`\n even uses the field's standard three-way support scale (supported / inferred /\n unsupported), matching ALCE-style *full / partial / no support*.\n- **Citation** ← aligns with **ALCE citation precision** (of the references it\n lists, how many are actually used). Current implementation is *structural*; a\n *semantic* version (does the citation truly support its claim, NLI-style) is on\n the roadmap.\n- **Source Quality** ← a deliberate **extension**. RAG-oriented frameworks assume\n the context is *given*, so they have no notion of \"source quality\". A research\n agent finds its own sources, so authority and diversity of those sources become\n first-class metrics — this is the dimension standard RAG eval can't express.\n\n## Architecture\n\nA thin, dependency-free framework layer (mirrors DeepEval's `BaseMetric` +\ntest-case design):\n\n```text\nbase.py BaseMetric / EvalSample / MetricResult — abstractions (+ `sources`, the RAG-framework gap)\nsuite.py 5 metric wrappers + evaluate() entry — standardized I/O + concept-alignment metadata\nmetrics.py raw metric functions — pure logic (unit-tested, perturbation-validated)\nperturbation.py behavioral reliability test — graded degradation, asserts monotonic response\nrun_eval.py / benchmark.py — runners (single query / multi-config comparison)\n```\n\nEvery metric declares its `group` and the standard concept it `aligned_with`, so\nthe ontology above is encoded in code, not just docs. `MetricResult.score` is\nalways *higher = better* (0–1) so results aggregate and compare uniformly.\n\n## Reliability (perturbation-validated)\n\nThe metrics aren't just implemented — they're checked for *reliability*.\n`perturbation.py` deliberately degrades a report (remove citations, swap to\nlow-authority sources, corrupt claims) in graded steps and asserts each metric\nresponds **monotonically**. This surfaced and fixed real defects that mocked unit\ntests and final outputs couldn't reveal — e.g. batch LLM scoring was letting one\nclaim contaminate another's score, fixed by scoring each claim independently.\n\n## Usage\n\n```bash\n# Full evaluation (all metrics)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost metrics only\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-unsupported-claim\n\n# Reliability check\npython -m evals.quality_eval.perturbation\n\n# Compare configs / runs\npython -m evals.quality_eval.benchmark --configs single-gpt4o single-gpt4o-mini\npython -m evals.quality_eval.benchmark --compare LOG_A LOG_B\n```\n\nProgrammatic use via the standardized entry point:\n\n```python\nfrom evals.quality_eval.base import EvalSample\nfrom evals.quality_eval.suite import evaluate, default_metrics\n\nsample = EvalSample(query=q, report=report, sources=urls, context=ctx)\nresults = await evaluate(sample, default_metrics(), grader_model)\nfor r in results:\n print(r.group, r.name, r.score) # typed, tagged, higher = better\n```\n\n## Roadmap\n\n- **Semantic citation precision** (does a citation truly support its claim,\n NLI/LLM-judge) → promote the `Citation` group to a full `Precision` family.\n- **True answer relevancy** (on-topic-ness), complementing `subtopic_coverage`'s\n completeness view.\n- **Component-level evaluation** of the multi-agent pipeline (planner / researcher\n / writer / reviewer) to locate bottlenecks — the natural next phase.\n\nDependencies: `langchain-openai`, `python-dotenv` (+ `judges` for the optional\nhallucination check). Env: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n","isInternal":false,"tokens":1210,"sizeBytes":5239},{"name":"README.md","path":"evals/README.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/evals/README.md","title":"Evals Skill","category":"anthropic-skill","format":"markdown","content":"# GPT-Researcher Evaluations\n\nThis directory contains evaluation tools and frameworks for assessing the performance of GPT-Researcher across different research tasks.\n\n## Evaluation Overview\n\nThree complementary evaluation systems, each answering a different question:\n\n| System | Question | Requires ground truth? |\n| --- | --- | --- |\n| `simple_evals/` | Is the answer factually correct? | Yes (OpenAI SimpleQA dataset) |\n| `quality_eval/` | Is the report well-sourced and complete? | No — works on any query |\n| `hallucination_eval/` | Does the report fabricate content? | No — compares against scraped sources |\n\n**Quick start:**\n\n```bash\n# Factual accuracy (needs ground-truth answers)\npython -m evals.simple_evals.run_eval --num_examples 10\n\n# Report quality (works on any query, zero-cost metrics only)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Hallucination detection\npython -m evals.hallucination_eval.run_eval -n 5\n```\n\n## Simple Evaluations (`simple_evals/`)\n\nThe `simple_evals` directory contains a straightforward evaluation framework adapted from [OpenAI's simple-evals system](https://github.com/openai/simple-evals), specifically designed to measure short-form factuality in large language models. Our implementation is based on OpenAI's [SimpleQA evaluation methodology](https://github.com/openai/simple-evals/blob/main/simpleqa_eval.py), following their zero-shot, chain-of-thought approach while adapting it for GPT-Researcher's specific use case.\n\n### Components\n\n- `simpleqa_eval.py`: Core evaluation logic for grading research responses\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `requirements.txt`: Dependencies required for running evaluations\n\n### Test Dataset\n\nThe `problems/` directory contains the evaluation dataset:\n\n- `Simple QA Test Set.csv`: A comprehensive collection of factual questions and their correct answers, mirrored from OpenAI's original test set. This dataset serves as the ground truth for evaluating GPT-Researcher's ability to find and report accurate information. The file is maintained locally to ensure consistent evaluation benchmarks and prevent any potential upstream changes from affecting our testing methodology.\n\n### Evaluation Logs\n\nThe `logs/` directory contains evaluation run histories preserved in version control.\n\nTwo formats are kept side-by-side:\n\n| Format | Example filename | Purpose |\n| --- | --- | --- |\n| Plain text (legacy) | `SimpleQA Eval 100 Problems 2-22-25.txt` | Human-readable terminal output |\n| Structured JSON | `eval_2025-02-22_10-32-11_n100.jsonl` | Machine-readable, suitable for trend analysis |\n\nEach JSON log file contains one record with three sections: `run_metadata`, `aggregate_metrics`, and per-query `results`. See [`example_output.json`](simple_evals/example_output.json) for the full schema.\n\n**Note:** Unlike typical log directories, this folder and its contents are intentionally tracked in git to maintain a historical record of evaluation runs.\n\n### Features\n\n- Measures factual accuracy of research responses\n- Uses GPT-4 as a grading model (configurable)\n ```python\n # In run_eval.py, you can customize the grader model:\n grader_model = ChatOpenAI(\n temperature=0, # Lower temperature for more consistent grading\n model_name=\"gpt-4-turbo\", # Can be changed to other OpenAI models\n openai_api_key=os.getenv(\"OPENAI_API_KEY\")\n )\n ```\n- Grades responses on a three-point scale:\n - `CORRECT`: Answer fully contains important information without contradictions\n - `INCORRECT`: Answer contains factual contradictions\n - `NOT_ATTEMPTED`: Answer neither confirms nor contradicts the target\n\n**Note on Grader Configuration:** While the default grader uses GPT-4-turbo, you can modify the model and its parameters to use different OpenAI models or adjust the temperature for different grading behaviors. This is independent of the researcher's configuration, allowing you to optimize for cost or performance as needed.\n\n### Metrics Tracked\n\n- Accuracy rate and F1 score\n- Cost — total and per-query average\n- Latency — average, p50 (median), and p95 per query\n- Success/failure counts and answer attempt rates\n- Source coverage per query\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/simple_evals\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\nLANGCHAIN_API_KEY=your_langchain_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py --num_examples \n```\n\nThe `num_examples` parameter determines how many random test queries to evaluate (default: 1).\n\n#### Customizing Researcher Behavior\n\nThe evaluation uses GPTResearcher with default settings, but you can modify `run_eval.py` to customize the researcher's behavior:\n\n```python\nresearcher = GPTResearcher(\n query=query,\n report_type=ReportType.ResearchReport.value, # Type of report to generate\n report_format=\"markdown\", # Output format\n report_source=ReportSource.Web.value, # Source of research\n tone=Tone.Objective, # Writing tone\n verbose=True # Enable detailed logging\n)\n```\n\nThese parameters can be adjusted to evaluate different research configurations or output formats. For a complete list of configuration options, see the [configuration documentation](https://docs.gptr.dev/docs/gpt-researcher/gptr/config).\n\n**Note on Configuration Independence:** The evaluation system is designed to be independent of the researcher's configuration. This means you can use different LLMs and settings for evaluation versus research. For example:\n- Evaluation could use GPT-4-turbo for grading while the researcher uses Claude 3.5 Sonnet for research\n- Different retrievers, embeddings, or report formats can be used\n- Token limits and other parameters can be customized separately\n\nThis separation allows for unbiased evaluation across different researcher configurations. However, please note that this feature is currently experimental and needs further testing.\n\n### Output\n\nEach run produces two outputs:\n\n#### Console summary\n\n```text\n=== Evaluation Summary ===\nTotal queries tested: 100\nSuccessful queries: 100\nFailed queries: 0\n\n=== AGGREGATE METRICS ===\nDebug counts:\nTotal successful: 100\nCORRECT: 92\nINCORRECT: 7\nNOT_ATTEMPTED: 1\n{\n \"correct_rate\": 0.92,\n \"incorrect_rate\": 0.07,\n \"not_attempted_rate\": 0.01,\n \"answer_rate\": 0.99,\n \"accuracy\": 0.9292929292929293,\n \"f1\": 0.9246231155778895\n}\n========================\nAccuracy: 0.929\nF1 Score: 0.925\n\nTotal cost: $9.6000\nAverage cost per query: $0.0960\nLatency — avg: 48.3s p50: 45.1s p95: 89.2s\n\nStructured log saved → evals/simple_evals/logs/eval_2025-02-22_10-32-11_n100.jsonl\n```\n\n**Structured JSON log** (`logs/eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`)\n\nSee [`example_output.json`](simple_evals/example_output.json) for the full schema. Key fields:\n\n```\nrun_metadata → timestamp, git_commit, grader_model, researcher_model\naggregate_metrics → accuracy, f1, cost, avg/p50/p95 latency, success/fail counts\nresults → per-query list with grade, score, latency, cost, sources\n```\n\n## Quality Evaluation (`quality_eval/`)\n\nMeasures the structural and content quality of research reports without requiring ground-truth answers. All metrics run on any open-ended query.\n\nAll metric logic lives in `metrics.py`. The runner is `run_eval.py`.\n\n### Metrics\n\n| Metric | Description | LLM calls | Cost |\n| --- | --- | --- | --- |\n| **Citation Faithfulness** | Are sources listed in the report's `## References` actually cited in the body? (catches decorative refs) | 0 | $0 |\n| **Source Diversity** | Domain variety via ratio + Shannon entropy | 0 | $0 |\n| **Source Authority** | Heuristic score per domain (.gov=1.0 → .com=0.5); unknown domains optionally LLM-scored | 0–1 | $0–$0.01 |\n| **Subtopic Coverage** | LLM-judged coverage of expected subtopics (adaptive count) | 2 | ~$0.02/query |\n| **Hallucination** | Binary faithfulness check — report vs. scraped sources | 1 | ~$0.01/query |\n| **Unsupported Claim Rate** | Claim-level credibility: supported (1.0) / inferred (0.3–0.9) / unsupported (0.0) | 2 | ~$0.02/query |\n\n**Citation Faithfulness** measures whether sources the report *lists* in its `## References` are *actually cited* in the body (a low score flags decorative references). It deliberately does not reward raw coverage — reading many sources and citing only the useful ones is good practice, like a real researcher.\n\n**Unsupported Claim Rate** distinguishes between:\n\n- `supported` — claim directly found in source text (score 1.0)\n- `inferred` — reasonable synthesis from multiple sources; never applied to data claims (score 0.3–0.9)\n- `unsupported` — no traceable basis in sources (score 0.0)\n\n### Metric Grouping & Tooling\n\nThe metrics group into three families: **Source Quality** (diversity, authority), **Writing Behavior** (citation faithfulness, subtopic coverage), and **Hallucination** (hallucination check, unsupported claim) — i.e. *is the input good*, *how was it written*, *is it correct*.\n\nTwo tools complement `run_eval.py` (shared metric logic lives in `metrics.py`):\n\n- **`benchmark.py`** — runs multiple researcher configurations (single- vs. multi-agent × model) over a fixed topic set and prints a side-by-side table. `--compare LOG_A LOG_B` diffs two past runs; `--summary-only` re-prints from saved logs.\n- **`perturbation.py`** — a behavioral test that deliberately degrades a report (remove inline citations, swap sources to low-authority, corrupt claims) and checks each metric responds monotonically. This validates metric *reliability* (directional sensitivity), which is separate from whether a metric captures *quality*.\n\n### Running\n\n```bash\n# Run all 6 metrics (most expensive, ~$0.06/query)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost only (citation ratio, diversity, authority)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Skip individual LLM metrics selectively\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic\npython -m evals.quality_eval.run_eval --num_examples 10 --no-hallucination\npython -m evals.quality_eval.run_eval --num_examples 10 --no-unsupported-claim\n```\n\nLogs are saved to `evals/quality_eval/logs/quality_eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`.\n\n### Example Output\n\n```text\nQuality eval: 5 queries subtopic=off hallucination=off unsupported_claim=on\n\n>> Query: What legal frameworks govern cross-border data transfers?\n citation_faithful: 0.95 (1 listed-only)\n diversity_ratio: 1.00 entropy=4.32\n authority_score: 0.62\n avg_claim_score: 0.79 (unsupported=0.11 inferred=0.22 n=9)\n latency=37.2s cost=$0.13\n\n=== QUALITY EVAL SUMMARY ===\n avg_citation_faithfulness: 0.94\n avg_diversity_ratio: 0.796\n avg_domain_entropy: 3.525\n avg_authority_score: 0.597\n avg_claim_score: 0.428\n avg_unsupported_claim_rate: 0.535\n avg_inferred_claim_rate: 0.148\n avg_latency_seconds: 33.6\n avg_cost: 0.116\n```\n\n### Dependencies\n\n```bash\npip install langchain-openai python-dotenv\n\n# Hallucination metric only:\npip install judges\n```\n\nEnvironment variables required: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n\n---\n\n## Hallucination Evaluation (`hallucination_eval/`)\n\nThe `hallucination_eval` directory contains tools for evaluating GPT-Researcher's outputs for hallucination. This evaluation system compares the generated research reports against their source materials to detect non-factual or hallucinated content, ensuring the reliability and accuracy of the research outputs.\n\n### Components\n\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `evaluate.py`: Core evaluation logic for detecting hallucinations\n- `inputs/`: Directory containing test queries\n - `search_queries.jsonl`: Collection of research queries for evaluation\n- `results/`: Directory containing evaluation results\n - `evaluation_records.jsonl`: Detailed per-query evaluation records\n - `aggregate_results.json`: Summary metrics across all evaluations\n\n### Features\n\n- Evaluates research reports against source materials\n- Provides detailed reasoning for hallucination detection\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/hallucination_eval\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py -n \n```\n\nThe `-n` parameter determines how many queries to evaluate from the test set (default: 1).\n\n### Sample Result\n\n```json\n{\n \"total_queries\": 1,\n \"successful_queries\": 1,\n \"total_responses\": 1,\n \"total_evaluated\": 1,\n \"total_hallucinated\": 0,\n \"hallucination_rate\": 0.0,\n \"results\": [\n {\n \"input\": \"What are the latest developments in quantum computing?\",\n \"output\": \"Research report content...\",\n \"source\": \"Source material content...\",\n \"is_hallucination\": false,\n \"confidence_score\": 0.95,\n \"reasoning\": \"The summary accurately reflects the source material with proper citations...\"\n }\n ]\n}\n```\n","isInternal":false,"tokens":3267,"sizeBytes":13499},{"name":"README.md","path":"evals/simple_evals/logs/README.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/evals/simple_evals/logs/README.md","title":"Logs Skill","category":"anthropic-skill","format":"markdown","content":"# Evaluation Results\n\nThis directory contains historical evaluation results for GPT-Researcher using the SimpleQA methodology.\n\n## Latest Results\n\n### [SimpleQA Eval 100 Problems 2-22-25](./SimpleQA%20Eval%20100%20Problems%202-22-25.txt)\n\nEvaluation run by [Kelly Abbott (kga245)](https://github.com/kga245)\n\n**Summary:**\n- Date: February 22, 2025\n- Sample Size: 100 queries\n- Success Rate: 100% (100/100 queries completed)\n\n**Performance Metrics:**\n- Accuracy: 92.9%\n- F1 Score: 92.5%\n- Answer Rate: 99%\n\n**Response Distribution:**\n- Correct: 92%\n- Incorrect: 7%\n- Not Attempted: 1%\n\n**Cost Efficiency:**\n- Total Cost: $9.60\n- Average Cost per Query: $0.096\n\nThis evaluation demonstrates strong performance in factual accuracy while maintaining reasonable cost efficiency. The high answer rate (99%) and accuracy (92.9%) suggest that GPT-Researcher is effective at finding and reporting accurate information.\n\n## Historical Context\n\nThese logs are maintained in version control to:\n1. Track performance improvements over time\n2. Provide benchmarks for future enhancements\n3. Enable analysis of different configurations\n4. Ensure transparency in our evaluation process\n\nEach log file contains detailed information about:\n- Individual query results\n- Source citations\n- Cost breakdowns\n- Error analysis\n- Aggregate metrics\n\n## Running New Evaluations\n\nTo generate new evaluation logs, see the [main evaluation documentation](../README.md) for instructions on running evaluations with different configurations or sample sizes. ","isInternal":false,"tokens":355,"sizeBytes":1525},{"name":"SKILL.md","path":"skills/gpt-researcher/SKILL.md","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/skills/gpt-researcher/SKILL.md","title":"gpt-researcher","category":"anthropic-skill","format":"markdown","content":"---\nname: gpt-researcher\ndescription: Autonomous deep research from Codex via MCP\n---\n\n# GPT Researcher for Codex\n\nUse GPT Researcher from Codex via MCP.\n\n## When to use\n- When you need gpt-researcher capabilities in your Codex workflow\n- See https://github.com/assafelovic/gpt-researcher for full setup instructions\n","frontmatter":{"name":"gpt-researcher","description":"Autonomous deep research from Codex via MCP"},"isInternal":false,"tokens":83,"sizeBytes":317},{"name":"plugin.json","path":".codex-plugin/plugin.json","rawUrl":"https://raw.githubusercontent.com/assafelovic/gpt-researcher/HEAD/.codex-plugin/plugin.json","title":"gpt-researcher","category":"plugin-manifest","format":"json","content":"{\n \"name\": \"gpt-researcher\",\n \"version\": \"0.1.0\",\n \"description\": \"Autonomous deep research from Codex via MCP\",\n \"author\": {\n \"name\": \"assafelovic\",\n \"url\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"homepage\": \"https://github.com/assafelovic/gpt-researcher\",\n \"repository\": \"https://github.com/assafelovic/gpt-researcher\",\n \"keywords\": [\n \"mcp\",\n \"codex\"\n ],\n \"mcpServers\": \"./.mcp.json\",\n \"interface\": {\n \"displayName\": \"GPT Researcher\",\n \"shortDescription\": \"Autonomous deep research from Codex via MCP\",\n \"longDescription\": \"An autonomous agent that conducts deep research on any data using any LLM providers.\",\n \"category\": \"Research\",\n \"websiteURL\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"skills\": \"./skills/\"\n}","frontmatter":{"name":"gpt-researcher","version":"0.1.0","description":"Autonomous deep research from Codex via MCP","author":{"name":"assafelovic","url":"https://github.com/assafelovic/gpt-researcher"},"homepage":"https://github.com/assafelovic/gpt-researcher","repository":"https://github.com/assafelovic/gpt-researcher","keywords":["mcp","codex"],"mcpServers":"./.mcp.json","interface":{"displayName":"GPT Researcher","shortDescription":"Autonomous deep research from Codex via MCP","longDescription":"An autonomous agent that conducts deep research on any data using any LLM providers.","category":"Research","websiteURL":"https://github.com/assafelovic/gpt-researcher"},"skills":"./skills/"},"isInternal":false,"tokens":235,"sizeBytes":784}],"systemPromptSnippet":"\n\n# Adding Features Guide\n\n## Table of Contents\n- [The 8-Step Pattern](#the-8-step-pattern)\n- [Image Generation Case Study](#image-generation-case-study)\n- [Testing New Features](#testing-new-features)\n\n---\n\n## The 8-Step Pattern\n\n```\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│1.CONFIG│ → │2.PROVIDER│ → │3.SKILL │ → │4.AGENT │\n└────────┘ └────────┘ └────────┘ └────────┘\n ↓ ↓ ↓ ↓\n┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐\n│5.PROMPTS│ → │6.WEBSOCKET│→ │7.FRONTEND│→ │8.DOCS │\n└────────┘ └────────┘ └────────┘ └────────┘\n```\n\n### Step 1: Add Configuration\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n \"MY_FEATURE_ENABLED\": False,\n \"MY_FEATURE_MODEL\": \"model-name\",\n \"MY_FEATURE_MAX_ITEMS\": 3,\n}\n```\n\n**File:** `gpt_researcher/config/variables/base.py`\n\n```python\nclass BaseConfig(TypedDict):\n \"MY_FEATURE_ENABLED\": bool\n \"MY_FEATURE_MODEL\": Union[str, None]\n \"MY_FEATURE_MAX_ITEMS\": int\n```\n\n### Step 2: Create Provider\n\n**File:** `gpt_researcher/llm_provider/my_feature/my_provider.py`\n\n```python\nclass MyFeatureProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"MY_API_KEY\")\n self.model = model\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n async def execute(self, input_data: str) -> Dict[str, Any]:\n # API implementation\n pass\n```\n\nExport in `gpt_researcher/llm_provider/__init__.py`.\n\n### Step 3: Create Skill\n\n**File:** `gpt_researcher/skills/my_feature.py`\n\n```python\nclass MyFeatureSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.provider = MyFeatureProvider(...)\n \n def is_enabled(self) -> bool:\n return getattr(self.config, 'my_feature_enabled', False) and self.provider.is_enabled()\n \n async def execute(self, context: str, query: str) -> List[Dict]:\n if not self.is_enabled():\n return []\n \n await stream_output(\"logs\", \"my_feature_start\", \"🚀 Starting...\", self.researcher.websocket)\n results = await self.provider.execute(context)\n await stream_output(\"logs\", \"my_feature_complete\", \"✅ Done\", self.researcher.websocket)\n \n return results\n```\n\nExport in `gpt_researcher/skills/__init__.py`.\n\n### Step 4: Integrate into Agent\n\n**File:** `gpt_researcher/agent.py`\n\n```python\ndef __init__(self, ...):\n if self.cfg.my_feature_enabled:\n from gpt_researcher.skills import MyFeatureSkill\n self.my_feature = MyFeatureSkill(self)\n else:\n self.my_feature = None\n self.my_feature_results = []\n\nasync def conduct_research(self, ...):\n # ... existing ...\n if self.my_feature and self.my_feature.is_enabled():\n self.my_feature_results = await self.my_feature.execute(self.context, self.query)\n```\n\n### Step 5: Update Prompts\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_my_feature_prompt(context: str, query: str) -> str:\n return f\"\"\"...\"\"\"\n```\n\n### Step 6: WebSocket Events\n\nAlready handled via `stream_output()` in skill.\n\n### Step 7: Frontend (if needed)\n\n**File:** `frontend/nextjs/hooks/useWebSocket.ts`\n\n```typescript\nif (data.content === 'my_feature_start') {\n setStatus('processing');\n}\n```\n\n### Step 8: Documentation\n\nCreate `docs/docs/gpt-researcher/gptr/my_feature.md`.\n\n---\n\n## Image Generation Case Study\n\nThis section shows the **actual implementation** of the Image Generation feature as a reference.\n\n### 1. Configuration Added\n\n**File:** `gpt_researcher/config/variables/default.py`\n\n```python\nDEFAULT_CONFIG: BaseConfig = {\n # ... existing ...\n \"IMAGE_GENERATION_MODEL\": \"models/gemini-2.5-flash-image\",\n \"IMAGE_GENERATION_MAX_IMAGES\": 3,\n \"IMAGE_GENERATION_ENABLED\": False,\n \"IMAGE_GENERATION_STYLE\": \"dark\", # dark, light, auto\n}\n```\n\n### 2. Provider Created\n\n**File:** `gpt_researcher/llm_provider/image/image_generator.py`\n\n```python\nclass ImageGeneratorProvider:\n def __init__(self, api_key: str = None, model: str = None):\n self.api_key = api_key or os.getenv(\"GOOGLE_API_KEY\")\n self.model = model or \"models/gemini-2.5-flash-image\"\n self._client = None\n \n def is_enabled(self) -> bool:\n return bool(self.api_key and self.model)\n \n def _build_enhanced_prompt(self, prompt: str, context: str = \"\", style: str = \"dark\") -> str:\n \"\"\"Add styling instructions to prompt.\"\"\"\n if style == \"dark\":\n style_instructions = \"\"\"\n Style: Dark mode professional infographic\n - Background: Dark (#0d1117)\n - Accents: Teal/cyan (#14b8a6)\n - Clean, modern, minimalist\n \"\"\"\n # ... handle light, auto\n return f\"{style_instructions}\\n\\nCreate: {prompt}\\n\\nContext: {context}\"\n \n async def generate_image(\n self,\n prompt: str,\n context: str = \"\",\n research_id: str = \"\",\n style: str = \"dark\",\n ) -> List[Dict[str, Any]]:\n \"\"\"Generate image using Gemini.\"\"\"\n full_prompt = self._build_enhanced_prompt(prompt, context, style)\n \n # Call Gemini API\n response = await self._generate_with_gemini(full_prompt, output_path, ...)\n \n return [{\"url\": f\"/outputs/images/{research_id}/img_{hash}.png\", ...}]\n```\n\n### 3. Skill Created\n\n**File:** `gpt_researcher/skills/image_generator.py`\n\n```python\nclass ImageGenerator:\n def __init__(self, researcher):\n self.researcher = researcher\n self.config = researcher.cfg\n self.image_provider = ImageGeneratorProvider(\n api_key=os.getenv(\"GOOGLE_API_KEY\"),\n model=getattr(self.config, 'image_generation_model', None),\n )\n self.max_images = getattr(self.config, 'image_generation_max_images', 3)\n self.style = getattr(self.config, 'image_generation_style', 'dark')\n \n def is_enabled(self) -> bool:\n enabled = getattr(self.config, 'image_generation_enabled', False)\n return enabled and self.image_provider.is_enabled()\n \n async def plan_and_generate_images(\n self,\n research_context: str,\n research_query: str,\n research_id: str,\n websocket: Any,\n ) -> List[Dict[str, Any]]:\n \"\"\"\n 1. Use LLM to identify visual concepts from context\n 2. Generate images in parallel\n 3. Return list of image metadata\n \"\"\"\n # Stream progress\n await stream_output(\"logs\", \"image_planning\", \"🎨 Planning images...\", websocket)\n \n # LLM identifies concepts\n concepts = await self._plan_image_concepts(research_context, research_query)\n \n # Generate images in parallel\n generated_images = []\n for i, concept in enumerate(concepts[:self.max_images]):\n await stream_output(\"logs\", \"image_generating\", \n f\"🖼️ Generating image {i+1}/{len(concepts)}...\", websocket)\n \n images = await self.image_provider.generate_image(\n prompt=concept[\"prompt\"],\n context=concept.get(\"context\", \"\"),\n research_id=research_id,\n style=self.style,\n )\n generated_images.extend(images)\n \n await stream_output(\"logs\", \"images_ready\", \n f\"✅ Generated {len(generated_images)} images\", websocket)\n \n return generated_images\n```\n\n### 4. Agent Integration\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nclass GPTResearcher:\n def __init__(self, ...):\n # ... existing init ...\n \n # Initialize image generator if enabled\n if self.cfg.image_generation_enabled:\n from gpt_researcher.skills import ImageGenerator\n self.image_generator = ImageGenerator(self)\n else:\n self.image_generator = None\n \n self.available_images: List[Dict[str, Any]] = []\n self.research_id = self._generate_research_id(query)\n \n async def conduct_research(self, on_progress=None):\n # ... existing research ...\n \n self.context = await self.research_conductor.conduct_research()\n \n # Pre-generate images after research, before report writing\n if self.cfg.image_generation_enabled and self.image_generator and self.image_generator.is_enabled():\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n \n async def write_report(self, ...):\n report = await self.report_generator.write_report(\n # ... existing params ...\n available_images=self.available_images, # Pass to report writer\n )\n return report\n```\n\n### 5. Prompt Updated\n\n**File:** `gpt_researcher/prompts.py`\n\n```python\n@staticmethod\ndef generate_report_prompt(..., available_images: List[Dict[str, Any]] = []):\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title', 'Untitled')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES - Embed where relevant using ![Title](URL):\n{image_list}\n\"\"\"\n \n return f\"\"\"...(existing prompt)...\n{image_instruction}\n\"\"\"\n```\n\n---\n\n## Testing New Features\n\n```python\n# tests/test_my_feature.py\nimport pytest\nfrom gpt_researcher import GPTResearcher\n\n@pytest.mark.asyncio\nasync def test_my_feature_disabled():\n \"\"\"Test that feature is skipped when disabled.\"\"\"\n researcher = GPTResearcher(query=\"test\")\n # MY_FEATURE_ENABLED defaults to False\n assert researcher.my_feature is None\n\n@pytest.mark.asyncio\nasync def test_my_feature_enabled(monkeypatch):\n \"\"\"Test feature execution when enabled.\"\"\"\n monkeypatch.setenv(\"MY_FEATURE_ENABLED\", \"true\")\n monkeypatch.setenv(\"MY_API_KEY\", \"test-key\")\n \n researcher = GPTResearcher(query=\"test\")\n assert researcher.my_feature is not None\n assert researcher.my_feature.is_enabled()\n```\n\n### Running Tests\n\n```bash\n# All tests\npython -m pytest tests/\n\n# Specific test\npython -m pytest tests/test_my_feature.py -v\n\n# With coverage\npython -m pytest tests/ --cov=gpt_researcher\n```\n\n\n# Advanced Patterns Reference\n\n## Table of Contents\n- [Custom Callbacks](#custom-callbacks)\n- [Custom WebSocket Handler](#custom-websocket-handler)\n- [LangChain Integration](#langchain-integration)\n- [Search Restrictions](#search-restrictions)\n- [Error Handling Patterns](#error-handling-patterns)\n\n---\n\n## Custom Callbacks\n\n```python\ndef cost_callback(cost: float):\n print(f\"API call cost: ${cost}\")\n\nresearcher = GPTResearcher(query=\"...\")\nresearcher.add_costs = cost_callback # Override cost tracking\n```\n\n---\n\n## Custom WebSocket Handler\n\n```python\nclass CustomWebSocket:\n def __init__(self):\n self.messages = []\n \n async def send_json(self, data):\n self.messages.append(data)\n if data['type'] == 'logs':\n print(f\"Progress: {data['output']}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=CustomWebSocket())\n```\n\n---\n\n## LangChain Integration\n\n### Using with LangChain Documents\n\n```python\nfrom langchain.document_loaders import DirectoryLoader\n\nloader = DirectoryLoader('./docs', glob=\"**/*.md\")\ndocuments = loader.load()\n\nresearcher = GPTResearcher(\n query=\"Summarize the documentation\",\n report_source=\"langchain_documents\",\n documents=documents,\n)\n```\n\n### Using with Vector Store\n\n```python\nfrom langchain.vectorstores import Chroma\n\nvectorstore = Chroma.from_documents(documents, embeddings)\n\nresearcher = GPTResearcher(\n query=\"Find relevant information\",\n report_source=\"langchain_vectorstore\",\n vector_store=vectorstore,\n vector_store_filter={\"source\": \"docs\"},\n)\n```\n\n---\n\n## Search Restrictions\n\n### Restricting Search Domains\n\n```python\nresearcher = GPTResearcher(\n query=\"Company news\",\n query_domains=[\"reuters.com\", \"bloomberg.com\", \"wsj.com\"],\n)\n```\n\n### Using Specific Source URLs\n\n```python\nresearcher = GPTResearcher(\n query=\"Analyze these articles\",\n source_urls=[\n \"https://example.com/article1\",\n \"https://example.com/article2\",\n ],\n complement_source_urls=True, # Also do web search\n)\n```\n\n---\n\n## Error Handling Patterns\n\n### Graceful Degradation\n\n```python\n# In skills, always check is_enabled()\nasync def execute(self, ...):\n if not self.is_enabled():\n logger.warning(\"Feature not enabled, skipping\")\n return [] # Return empty, don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n logger.error(f\"Feature error: {e}\")\n await stream_output(\"logs\", \"feature_error\", f\"⚠️ Error: {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n### API Rate Limiting\n\n```python\n# Providers should handle rate limits\nasync def execute(self, ...):\n try:\n return await self._call_api(...)\n except RateLimitError as e:\n logger.warning(f\"Rate limited, waiting...\")\n await asyncio.sleep(60)\n return await self._call_api(...) # Retry\n```\n\n### WebSocket None Check\n\n```python\n# Always check websocket before sending\nif self.researcher.websocket:\n await stream_output(\"logs\", \"event\", \"message\", self.researcher.websocket)\n```\n\n\n# API Reference\n\n## Table of Contents\n- [REST API](#rest-api)\n- [WebSocket API](#websocket-api)\n- [Python Client](#python-client)\n- [Output Files](#output-files)\n\n---\n\n## REST API\n\nBase URL: `http://localhost:8000`\n\n### Generate Report\n\n**POST `/report/`**\n\n```json\n{\n \"task\": \"What are the latest AI developments?\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"query_domains\": [],\n \"generate_in_background\": false\n}\n```\n\n**Response:**\n\n```json\n{\n \"research_id\": \"task_1234567890_query\",\n \"research_information\": {\n \"source_urls\": [\"...\"],\n \"research_costs\": 0.05,\n \"visited_urls\": [\"...\"],\n \"research_images\": [\"...\"]\n },\n \"report\": \"# Research Report\\n\\n...\",\n \"docx_path\": \"outputs/task_123.docx\",\n \"pdf_path\": \"outputs/task_123.pdf\"\n}\n```\n\n### Chat with Report\n\n**POST `/api/chat`**\n\n```json\n{\n \"report\": \"The full report text...\",\n \"messages\": [\n {\"role\": \"user\", \"content\": \"What are the key findings?\"}\n ]\n}\n```\n\n### Report Management\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/api/reports` | List all reports |\n| GET | `/api/reports/{id}` | Get single report |\n| POST | `/api/reports` | Create/update report |\n| PUT | `/api/reports/{id}` | Update report |\n| DELETE | `/api/reports/{id}` | Delete report |\n\n### File Operations\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/files/` | List uploaded documents |\n| POST | `/upload/` | Upload document |\n| DELETE | `/files/{filename}` | Delete file |\n| GET | `/outputs/{filename}` | Get output file |\n\n---\n\n## WebSocket API\n\n**Endpoint:** `ws://localhost:8000/ws`\n\n### Send Research Request\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"report_source\": \"web\",\n \"tone\": \"Objective\",\n \"source_urls\": [],\n \"mcp_enabled\": false,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": []\n}\n```\n\n### Message Types (Server → Client)\n\n| Type | Content | Description |\n|------|---------|-------------|\n| `logs` | `starting_research` | Research initiated |\n| `logs` | `planning_research` | Generating sub-queries |\n| `logs` | `running_subquery_research` | Researching sub-query |\n| `logs` | `research_step_finalized` | Research complete |\n| `logs` | `agent_generated` | Agent role selected |\n| `logs` | `scraping_urls` | Scraping web pages |\n| `logs` | `mcp_optimization` | MCP processing |\n| `logs` | `image_planning` | Planning images |\n| `logs` | `images_ready` | Images generated |\n| `report` | - | Streaming report chunks |\n| `report_complete` | - | Final complete report |\n| `path` | `pdf`, `docx`, `md` | Output file paths |\n| `error` | - | Error messages |\n| `human_feedback` | `request` | Request user input |\n\n### Message Format\n\n```json\n{\n \"type\": \"logs\",\n \"content\": \"starting_research\",\n \"output\": \"🔍 Starting the research task...\",\n \"metadata\": null\n}\n```\n\n### Frontend Handler Example\n\n```typescript\nws.onmessage = (event) => {\n const data = JSON.parse(event.data);\n \n switch (data.type) {\n case 'logs':\n setLogs(prev => [...prev, data]);\n break;\n case 'report':\n setAnswer(prev => prev + data.output);\n break;\n case 'report_complete':\n setAnswer(data.output);\n break;\n case 'path':\n setPaths(prev => ({...prev, [data.content]: data.output}));\n break;\n case 'error':\n setError(data.output);\n break;\n }\n};\n```\n\n---\n\n## Python Client\n\n### Basic Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\",\n )\n \n await researcher.conduct_research()\n report = await researcher.write_report()\n \n print(f\"Report: {report}\")\n print(f\"Costs: ${researcher.get_costs()}\")\n\nasyncio.run(main())\n```\n\n### With MCP\n\n```python\nresearcher = GPTResearcher(\n query=\"Research topic\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\",\n)\n```\n\n### With WebSocket Streaming\n\n```python\nclass MockWebSocket:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(\n query=\"Research topic\",\n websocket=MockWebSocket(),\n)\n```\n\n### GPTResearcher Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `query` | str | required | Research question |\n| `report_type` | str | `research_report` | Type of report |\n| `report_source` | str | `web` | Data source |\n| `tone` | Tone | `Objective` | Writing tone |\n| `source_urls` | list | `[]` | Specific URLs to research |\n| `document_urls` | list | `[]` | Document URLs |\n| `query_domains` | list | `[]` | Restrict to domains |\n| `config_path` | str | None | Path to JSON config |\n| `websocket` | WebSocket | None | For streaming |\n| `mcp_configs` | list | `[]` | MCP server configs |\n| `mcp_strategy` | str | `fast` | MCP strategy |\n| `verbose` | bool | `True` | Verbose output |\n\n---\n\n## Output Files\n\n```\noutputs/\n├── task_{timestamp}_{query}.md\n├── task_{timestamp}_{query}.pdf\n├── task_{timestamp}_{query}.docx\n└── images/\n └── {research_id}/\n └── img_{hash}_{index}.png\n```\n\n---\n\n## Error Codes\n\n| Code | Description |\n|------|-------------|\n| 400 | Bad Request - Invalid parameters |\n| 404 | Not Found - Report not found |\n| 429 | Rate Limited - API quota exceeded |\n| 500 | Internal Server Error |\n\n\n# Architecture Reference\n\n## Table of Contents\n- [System Layers](#system-layers)\n- [Key File Locations](#key-file-locations)\n\n---\n\n## System Layers\n\n```\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ USER REQUEST │\n│ (query, report_type, report_source, tone, mcp_configs) │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ BACKEND API LAYER │\n│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │\n│ │ FastAPI Server │ │ WebSocket Manager│ │ Report Store │ │\n│ │ backend/server/ │ │ Real-time events │ │ JSON persistence│ │\n│ │ app.py │ │ websocket_mgr.py │ │ report_store.py │ │\n│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ GPTResearcher (gpt_researcher/agent.py) │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ SKILLS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ ResearchConductor│ │ ReportGenerator │ │ ContextManager │ │ │\n│ │ │ Plan & gather │ │ Write reports │ │ Similarity search│ │ │\n│ │ │ researcher.py │ │ writer.py │ │ context_manager │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ BrowserManager │ │ SourceCurator │ │ ImageGenerator │ │ │\n│ │ │ Web scraping │ │ Rank sources │ │ Gemini images │ │ │\n│ │ │ browser.py │ │ curator.py │ │ image_generator │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ DeepResearchSkill│ │ │\n│ │ │ Recursive depth │ │ │\n│ │ │ deep_research.py│ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ ACTIONS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ report_generation│ │ query_processing│ │ web_scraping │ │ │\n│ │ │ LLM report write│ │ Sub-query plan │ │ URL scraping │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ retriever.py │ │ agent_creator │ │ markdown_process│ │ │\n│ │ │ Get retrievers │ │ Choose agent │ │ Parse markdown │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ ┌───────────────────────────────────────────────────────────────────────┐ │\n│ │ PROVIDERS LAYER │ │\n│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │\n│ │ │ LLM Provider │ │ Retrievers │ │ Scrapers │ │ │\n│ │ │ OpenAI,Anthropic│ │ Tavily,Google │ │ BS4,Playwright │ │ │\n│ │ │ Google,Groq... │ │ Bing,MCP... │ │ PDF,DOCX... │ │ │\n│ │ │ llm_provider/ │ │ retrievers/ │ │ scraper/ │ │ │\n│ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │\n│ │ ┌─────────────────┐ │ │\n│ │ │ ImageGenerator │ │ │\n│ │ │ Gemini/Imagen │ │ │\n│ │ │ llm_provider/ │ │ │\n│ │ │ image/ │ │ │\n│ │ └─────────────────┘ │ │\n│ └───────────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────────────────┐\n│ CONFIGURATION LAYER │\n│ gpt_researcher/config/ │\n│ │\n│ Environment Variables → JSON Config File → Default Values │\n│ (highest) (medium) (lowest) │\n│ │\n│ config.py loads and merges all sources │\n│ variables/default.py contains all defaults │\n│ variables/base.py defines TypedDict for type safety │\n└─────────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes/Functions |\n|------|--------------|----------------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| Context/embeddings | `gpt_researcher/skills/context_manager.py` | `ContextManager` |\n| Source ranking | `gpt_researcher/skills/curator.py` | `SourceCurator` |\n| Deep research | `gpt_researcher/skills/deep_research.py` | `DeepResearchSkill` |\n| Image generation | `gpt_researcher/skills/image_generator.py` | `ImageGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| Config types | `gpt_researcher/config/variables/base.py` | `BaseConfig` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| WebSocket mgmt | `backend/server/websocket_manager.py` | `WebSocketManager`, `run_agent` |\n| Report types | `backend/report_type/` | `BasicReport`, `DetailedReport` |\n| Search engines | `gpt_researcher/retrievers/` | `TavilySearch`, `GoogleSearch`, etc. |\n| Web scraping | `gpt_researcher/scraper/` | Various scrapers |\n| Enums | `gpt_researcher/utils/enum.py` | `ReportType`, `ReportSource`, `Tone` |\n\n\n# Core Components & Method Signatures\n\n## Table of Contents\n- [GPTResearcher](#gptresearcher)\n- [ResearchConductor](#researchconductor)\n- [ReportGenerator](#reportgenerator)\n\n---\n\n## GPTResearcher\n\n**File:** `gpt_researcher/agent.py`\n\nThe main orchestrator class. Full initialization signature:\n\n```python\nclass GPTResearcher:\n def __init__(\n self,\n query: str, # Research question (required)\n report_type: str = \"research_report\", # research_report, detailed_report, deep, outline_report, resource_report\n report_format: str = \"markdown\", # Output format\n report_source: str = \"web\", # web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n tone: Tone = Tone.Objective, # Writing tone (see Tone enum)\n source_urls: list[str] | None = None, # Specific URLs to research\n document_urls: list[str] | None = None, # Document URLs to include\n complement_source_urls: bool = False, # Add web search to source_urls\n query_domains: list[str] | None = None, # Restrict search to domains\n documents=None, # LangChain document objects\n vector_store=None, # LangChain vector store\n vector_store_filter=None, # Filter for vector store\n config_path=None, # Path to JSON config file\n websocket=None, # WebSocket for streaming\n agent=None, # Pre-defined agent type\n role=None, # Pre-defined agent role\n parent_query: str = \"\", # Parent query for subtopics\n subtopics: list | None = None, # Subtopics to research\n visited_urls: set | None = None, # Already visited URLs\n verbose: bool = True, # Verbose logging\n context=None, # Pre-loaded context\n headers: dict | None = None, # HTTP headers\n max_subtopics: int = 5, # Max subtopics for detailed\n log_handler=None, # Custom log handler\n prompt_family: str | None = None, # Custom prompt family\n mcp_configs: list[dict] | None = None, # MCP server configurations\n mcp_max_iterations: int | None = None, # Deprecated, use mcp_strategy\n mcp_strategy: str | None = None, # fast, deep, disabled\n **kwargs\n ):\n```\n\n### Key Methods\n\n```python\nasync def conduct_research(self, on_progress=None) -> str:\n \"\"\"\n Main research orchestration.\n \n 1. Selects agent role via LLM (choose_agent)\n 2. Delegates to ResearchConductor\n 3. Optionally generates images if enabled\n \n Returns: Accumulated research context as string\n \"\"\"\n\nasync def write_report(\n self, \n existing_headers: list = [], # Headers to avoid duplication\n relevant_written_contents: list = [], # Previous content for context\n ext_context=None, # External context override\n custom_prompt=\"\" # Custom prompt override\n) -> str:\n \"\"\"\n Generate final report from context.\n \n Returns: Markdown report string\n \"\"\"\n\ndef get_costs(self) -> float:\n \"\"\"Returns total accumulated API costs.\"\"\"\n\ndef add_costs(self, cost: float) -> None:\n \"\"\"Add to running cost total (used as callback).\"\"\"\n```\n\n---\n\n## ResearchConductor\n\n**File:** `gpt_researcher/skills/researcher.py`\n\nManages the research process:\n\n```python\nclass ResearchConductor:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.logger = logging.getLogger(__name__)\n\n async def plan_research(self, query: str, query_domains=None) -> list:\n \"\"\"\n Generate sub-queries from main query using LLM.\n \n 1. Gets initial search results\n 2. Calls plan_research_outline() to generate sub-queries\n \n Returns: List of sub-query strings\n \"\"\"\n\n async def conduct_research(self) -> str:\n \"\"\"\n Main research execution based on report_source.\n \n Handles: web, local, hybrid, azure, langchain_documents, langchain_vectorstore\n \n For each source type:\n 1. Load/search data\n 2. Process sub-queries\n 3. Combine context\n 4. Optionally curate sources\n \n Returns: Combined research context string\n \"\"\"\n\n async def _process_sub_query(\n self, \n sub_query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"\n Process a single sub-query.\n \n 1. Get MCP context (if configured, based on strategy)\n 2. Scrape URLs from search results\n 3. Get similar content via embeddings\n 4. Combine MCP + web context\n \n Returns: Combined context for this sub-query\n \"\"\"\n\n async def _get_context_by_web_search(\n self, \n query: str, \n scraped_data: list = [], \n query_domains: list = []\n ) -> str:\n \"\"\"Web-based research with sub-query planning.\"\"\"\n\n async def _scrape_data_by_urls(\n self, \n sub_query: str, \n query_domains: list = []\n ) -> list:\n \"\"\"Search and scrape URLs for a sub-query.\"\"\"\n```\n\n---\n\n## ReportGenerator\n\n**File:** `gpt_researcher/skills/writer.py`\n\n```python\nclass ReportGenerator:\n def __init__(self, researcher: GPTResearcher):\n self.researcher = researcher\n self.research_params = {\n \"query\": researcher.query,\n \"agent_role_prompt\": researcher.cfg.agent_role or researcher.role,\n \"report_type\": researcher.report_type,\n \"report_source\": researcher.report_source,\n \"tone\": researcher.tone,\n \"websocket\": researcher.websocket,\n \"cfg\": researcher.cfg,\n \"headers\": researcher.headers,\n }\n\n async def write_report(\n self,\n existing_headers: list = [],\n relevant_written_contents: list = [],\n ext_context=None,\n custom_prompt=\"\",\n available_images: list = [], # Pre-generated images to embed\n ) -> str:\n \"\"\"\n Generate report using LLM.\n \n Calls generate_report() action with context and images.\n \n Returns: Markdown report\n \"\"\"\n\n async def write_introduction(self, ...) -> str:\n \"\"\"Write report introduction section.\"\"\"\n\n async def write_conclusion(self, ...) -> str:\n \"\"\"Write report conclusion with references.\"\"\"\n```\n\n\n# Configuration Reference\n\n## Table of Contents\n- [Required Variables](#required-variables)\n- [LLM Configuration](#llm-configuration)\n- [Provider API Keys](#provider-api-keys)\n- [Retriever Configuration](#retriever-configuration)\n- [Report Configuration](#report-configuration)\n- [Feature Toggles](#feature-toggles)\n- [Configuration Priority](#configuration-priority)\n- [Example .env](#example-env)\n\n---\n\n## Required Variables\n\n```bash\nOPENAI_API_KEY=sk-... # Or another LLM provider key\nTAVILY_API_KEY=tvly-... # Or another retriever key\n```\n\n---\n\n## LLM Configuration\n\n```bash\nLLM_PROVIDER=openai # openai, anthropic, google, groq, together, etc.\nFAST_LLM=gpt-4o-mini # Quick tasks (summarization)\nSMART_LLM=gpt-4o # Complex reasoning (report writing)\nSTRATEGIC_LLM=o3-mini # Planning (agent selection)\nTEMPERATURE=0.4 # 0.0-1.0\nMAX_TOKENS=4000\nREASONING_EFFORT=medium # For o-series: low, medium, high\n```\n\n---\n\n## Provider API Keys\n\n```bash\n# OpenAI\nOPENAI_API_KEY=sk-...\nOPENAI_BASE_URL=https://api.openai.com/v1\n\n# Anthropic\nANTHROPIC_API_KEY=sk-ant-...\n\n# Google\nGOOGLE_API_KEY=AIza...\n\n# Groq\nGROQ_API_KEY=gsk_...\n\n# Azure OpenAI\nAZURE_OPENAI_API_KEY=...\nAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\n```\n\n---\n\n## Retriever Configuration\n\n```bash\nRETRIEVER=tavily # Single or comma-separated: tavily,google,mcp\nMAX_SEARCH_RESULTS_PER_QUERY=5\nMAX_URLS_TO_SCRAPE=10\nSIMILARITY_THRESHOLD=0.42\n```\n\n### Retriever API Keys\n\n```bash\nTAVILY_API_KEY=tvly-...\nGOOGLE_API_KEY=AIza...\nGOOGLE_CX_KEY=...\nBING_API_KEY=...\nSERPER_API_KEY=...\nSERPAPI_API_KEY=...\nEXA_API_KEY=...\n```\n\n---\n\n## Report Configuration\n\n```bash\nREPORT_FORMAT=apa # apa, mla, chicago, harvard, ieee\nTOTAL_WORDS=1000\nLANGUAGE=english\nCURATE_SOURCES=true\n```\n\n---\n\n## Feature Toggles\n\n### Image Generation\n\n```bash\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza...\nIMAGE_GENERATION_MODEL=models/gemini-2.5-flash-image\nIMAGE_GENERATION_MAX_IMAGES=3\nIMAGE_GENERATION_STYLE=dark # dark, light, auto\n```\n\n### Deep Research\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n### MCP\n\n```bash\nMCP_STRATEGY=fast # fast, deep, disabled\n```\n\n### Local Documents\n\n```bash\nDOC_PATH=./my-docs\n# Supports: PDF, DOCX, TXT, CSV, XLSX, PPTX, MD\n```\n\n### Server\n\n```bash\nHOST=0.0.0.0\nPORT=8000\nVERBOSE=true\n```\n\n---\n\n## Configuration Priority\n\n```\nEnvironment Variables (highest)\n ↓\nJSON Config File (if provided)\n ↓\nDefault Values (lowest)\n```\n\n**Important:** Config keys are lowercased when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\n---\n\n## Example .env\n\n```bash\n# Required\nOPENAI_API_KEY=sk-your-key\nTAVILY_API_KEY=tvly-your-key\n\n# LLM\nFAST_LLM=gpt-4o-mini\nSMART_LLM=gpt-4o\n\n# Report\nTOTAL_WORDS=1000\nLANGUAGE=english\n\n# Optional: Images\nIMAGE_GENERATION_ENABLED=true\nGOOGLE_API_KEY=AIza-your-key\nIMAGE_GENERATION_STYLE=dark\n```\n\n\n# Deep Research Mode Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [DeepResearchSkill](#deepresearchskill)\n- [Usage](#usage)\n\n---\n\n## Overview\n\nDeep Research uses recursive tree-like exploration with configurable depth and breadth.\n\n---\n\n## Configuration\n\n```bash\nDEEP_RESEARCH_BREADTH=4 # Subtopics per level\nDEEP_RESEARCH_DEPTH=2 # Recursion levels\nDEEP_RESEARCH_CONCURRENCY=2 # Parallel tasks\n```\n\n---\n\n## DeepResearchSkill\n\n**File:** `gpt_researcher/skills/deep_research.py`\n\n```python\nclass DeepResearchSkill:\n def __init__(self, researcher):\n self.researcher = researcher\n self.breadth = getattr(researcher.cfg, 'deep_research_breadth', 4)\n self.depth = getattr(researcher.cfg, 'deep_research_depth', 2)\n self.concurrency_limit = getattr(researcher.cfg, 'deep_research_concurrency', 2)\n self.learnings = []\n self.research_sources = []\n self.context = []\n\n async def deep_research(self, query: str, on_progress=None) -> str:\n \"\"\"\n Recursive research with depth and breadth.\n \n 1. Research main topic\n 2. Generate subtopics (breadth)\n 3. For each subtopic, recursively research (depth)\n 4. Aggregate all findings\n 5. Generate comprehensive report\n \"\"\"\n```\n\n---\n\n## Usage\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers deep research\n)\nawait researcher.conduct_research()\nreport = await researcher.write_report()\n```\n\n### Research Tree Structure\n\n```\nQuery: \"Quantum Computing\"\n├── Subtopic 1: Hardware (depth 1)\n│ ├── Subtopic 1.1: Superconducting qubits (depth 2)\n│ └── Subtopic 1.2: Ion traps (depth 2)\n├── Subtopic 2: Algorithms (depth 1)\n│ ├── Subtopic 2.1: Shor's algorithm (depth 2)\n│ └── Subtopic 2.2: Grover's algorithm (depth 2)\n├── Subtopic 3: Applications (depth 1)\n│ └── ...\n└── Subtopic 4: Challenges (depth 1)\n └── ...\n```\n\nWith `DEEP_RESEARCH_BREADTH=4` and `DEEP_RESEARCH_DEPTH=2`, this explores 4 subtopics at each level, going 2 levels deep.\n\n\n# Research Flow & Data Flow\n\n## Table of Contents\n- [End-to-End Research Flow](#end-to-end-research-flow)\n- [Data Flow Between Components](#data-flow-between-components)\n\n---\n\n## End-to-End Research Flow\n\n### 1. Request Entry\n\n**File:** `backend/server/app.py`\n\n```python\n# REST API endpoint\n@app.post(\"/report/\")\nasync def generate_report(research_request: ResearchRequest, background_tasks: BackgroundTasks):\n research_id = sanitize_filename(f\"task_{int(time.time())}_{research_request.task}\")\n # Calls write_report() which uses run_agent()\n\n# WebSocket endpoint\n@app.websocket(\"/ws\")\nasync def websocket_endpoint(websocket: WebSocket):\n await manager.connect(websocket)\n await handle_websocket_communication(websocket, manager)\n```\n\n### 2. Agent Runner\n\n**File:** `backend/server/websocket_manager.py`\n\n```python\nasync def run_agent(task, report_type, report_source, source_urls, ...):\n \"\"\"Main entry point for research execution.\"\"\"\n # Create logs handler\n logs_handler = CustomLogsHandler(websocket, task)\n \n # Configure MCP if enabled\n if mcp_enabled and mcp_configs:\n os.environ[\"RETRIEVER\"] = f\"{current_retriever},mcp\"\n os.environ[\"MCP_STRATEGY\"] = mcp_strategy\n \n # Route based on report type\n if report_type == \"multi_agents\":\n report = await run_research_task(query=task, websocket=logs_handler, ...)\n elif report_type == ReportType.DetailedReport.value:\n researcher = DetailedReport(query=task, ...)\n report = await researcher.run()\n else:\n researcher = BasicReport(query=task, ...)\n report = await researcher.run()\n \n return report\n```\n\n### 3. Research Phase\n\n**File:** `gpt_researcher/agent.py`\n\n```python\nasync def conduct_research(self, on_progress=None):\n # Handle deep research separately\n if self.report_type == ReportType.DeepResearch.value and self.deep_researcher:\n return await self._handle_deep_research(on_progress)\n \n # Choose agent role via LLM\n if not (self.agent and self.role):\n self.agent, self.role = await choose_agent(\n query=self.query,\n cfg=self.cfg,\n parent_query=self.parent_query,\n cost_callback=self.add_costs,\n headers=self.headers,\n prompt_family=self.prompt_family,\n )\n \n # Conduct research\n self.context = await self.research_conductor.conduct_research()\n \n # Generate images if enabled (pre-generation for seamless UX)\n if self.cfg.image_generation_enabled and self.image_generator:\n self.available_images = await self.image_generator.plan_and_generate_images(\n research_context=self.context,\n research_query=self.query,\n research_id=self.research_id,\n websocket=self.websocket,\n )\n \n return self.context\n```\n\n### 4. Sub-Query Processing\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\nasync def _process_sub_query(self, sub_query: str, scraped_data: list = [], query_domains: list = []):\n # MCP Strategy handling\n mcp_retrievers = [r for r in self.researcher.retrievers if \"mcpretriever\" in r.__name__.lower()]\n mcp_strategy = self._get_mcp_strategy()\n \n if mcp_retrievers:\n if mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n # Reuse cached MCP results\n mcp_context = self._mcp_results_cache.copy()\n elif mcp_strategy == \"deep\":\n # Run MCP for every sub-query\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n \n # Get web search context\n if not scraped_data:\n scraped_data = await self._scrape_data_by_urls(sub_query, query_domains)\n \n # Get similar content via embeddings\n if scraped_data:\n web_context = await self.researcher.context_manager.get_similar_content_by_query(\n sub_query, scraped_data\n )\n \n # Combine MCP + web context\n combined_context = self._combine_mcp_and_web_context(mcp_context, web_context, sub_query)\n return combined_context\n```\n\n### 5. Report Generation\n\n**File:** `gpt_researcher/actions/report_generation.py`\n\n```python\nasync def generate_report(\n query: str,\n context: str,\n agent_role_prompt: str,\n report_type: str,\n websocket=None,\n cfg=None,\n tone=None,\n headers=None,\n cost_callback=None,\n prompt_family=None,\n available_images: list = [],\n **kwargs\n) -> str:\n \"\"\"Generate report using LLM.\"\"\"\n # Get prompt generator\n generate_prompt = prompt_family.get_prompt_by_report_type(report_type)\n \n # Build prompt with context and available images\n content = generate_prompt(\n query, context, report_source,\n report_format=cfg.report_format,\n tone=tone,\n total_words=cfg.total_words,\n language=cfg.language,\n available_images=available_images,\n )\n \n # Call LLM\n report = await create_chat_completion(\n model=cfg.smart_llm,\n messages=[{\"role\": \"user\", \"content\": content}],\n temperature=cfg.temperature,\n llm_provider=cfg.smart_llm_provider,\n max_tokens=cfg.smart_token_limit,\n llm_kwargs=cfg.llm_kwargs,\n cost_callback=cost_callback,\n )\n \n return report\n```\n\n---\n\n## Data Flow Between Components\n\n```\nUser Query\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ GPTResearcher.__init__() │\n│ • Loads Config (env → json → defaults) │\n│ • Initializes skills: ResearchConductor, ReportGenerator, etc │\n│ • Initializes retrievers based on RETRIEVER env var │\n│ • Initializes ImageGenerator if IMAGE_GENERATION_ENABLED │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.conduct_research()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ choose_agent() │\n│ Input: query, config │\n│ Output: (agent_type: str, role_prompt: str) │\n│ • LLM selects best agent role for the query │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ResearchConductor.conduct_research() │\n│ Input: self.researcher (has query, config, retrievers) │\n│ Output: context: str │\n│ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ plan_research() │ │\n│ │ Input: query │ │\n│ │ Output: sub_queries: list[str] │ │\n│ │ • Calls LLM to generate 3-5 sub-queries │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ For each sub_query: │ │\n│ │ _process_sub_query() │ │\n│ │ Input: sub_query │ │\n│ │ Output: sub_context: str │ │\n│ │ │ │\n│ │ 1. MCP retrieval (if configured) │ │\n│ │ → mcp_context: list[dict] │ │\n│ │ │ │\n│ │ 2. Web search via retrievers │ │\n│ │ → search_results: list[dict] │ │\n│ │ │ │\n│ │ 3. Scrape URLs │ │\n│ │ → scraped_content: list[dict] │ │\n│ │ │ │\n│ │ 4. Similarity search via embeddings │ │\n│ │ → relevant_context: str │ │\n│ │ │ │\n│ │ 5. Combine MCP + web context │ │\n│ │ → combined_context: str │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │ │\n│ ▼ │\n│ Aggregate all sub_contexts → final context: str │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ If IMAGE_GENERATION_ENABLED:\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ImageGenerator.plan_and_generate_images() │\n│ Input: context, query, research_id │\n│ Output: available_images: list[dict] │\n│ [{\"url\": \"/outputs/images/.../img.png\", │\n│ \"title\": \"...\", \"description\": \"...\"}] │\n│ │\n│ 1. LLM analyzes context for visual concepts │\n│ 2. Generates 2-3 images in parallel via Gemini │\n│ 3. Saves to outputs/images/{research_id}/ │\n└─────────────────────────────────────────────────────────────────┘\n │\n │ researcher.write_report()\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ ReportGenerator.write_report() │\n│ Input: context, available_images │\n│ Output: report: str (markdown) │\n│ │\n│ → generate_report() action │\n│ • Builds prompt with context + image list │\n│ • LLM generates report with embedded images │\n└─────────────────────────────────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ Output │\n│ • Streamed via WebSocket (type: \"report\") │\n│ • Final via WebSocket (type: \"report_complete\") │\n│ • Exported to PDF, DOCX, Markdown │\n│ • Saved to outputs/ directory │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n\n# MCP Integration Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Configuration](#configuration)\n- [Strategy Options](#strategy-options)\n- [Processing Logic](#processing-logic)\n\n---\n\n## Overview\n\nMCP (Model Context Protocol) enables research from specialized data sources (GitHub, databases, APIs) alongside web search.\n\n---\n\n## Configuration\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n mcp_configs=[\n {\n \"name\": \"github\", # Server name\n \"command\": \"npx\", # Command to start\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}, # Environment vars\n },\n {\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@anthropic/mcp-server-filesystem\", \"/docs\"],\n },\n {\n \"name\": \"remote\",\n \"connection_url\": \"ws://server:8080\", # WebSocket connection\n \"connection_type\": \"websocket\",\n \"connection_token\": \"auth_token\",\n }\n ],\n mcp_strategy=\"fast\", # fast, deep, disabled\n)\n```\n\n---\n\n## Strategy Options\n\n| Strategy | Behavior | Use Case |\n|----------|----------|----------|\n| `fast` (default) | Run MCP once with original query, cache results | Performance-focused |\n| `deep` | Run MCP for every sub-query | Maximum thoroughness |\n| `disabled` | Skip MCP entirely | Web-only research |\n\n---\n\n## Processing Logic\n\n**File:** `gpt_researcher/skills/researcher.py`\n\n```python\n# At start of research (for 'fast' strategy)\nif mcp_strategy == \"fast\":\n mcp_context = await self._execute_mcp_research_for_queries([query], mcp_retrievers)\n self._mcp_results_cache = mcp_context # Cache for reuse\n\n# During sub-query processing\nif mcp_strategy == \"fast\" and self._mcp_results_cache is not None:\n mcp_context = self._mcp_results_cache.copy() # Reuse cache\nelif mcp_strategy == \"deep\":\n mcp_context = await self._execute_mcp_research_for_queries([sub_query], mcp_retrievers)\n```\n\n### WebSocket Request Example\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"research_report\",\n \"mcp_enabled\": true,\n \"mcp_strategy\": \"fast\",\n \"mcp_configs\": [\n {\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": \"...\"}\n }\n ]\n}\n```\n\n\n# Multi-Agent System Reference\n\n## Table of Contents\n- [Overview](#overview)\n- [Agent Roles](#agent-roles)\n- [Workflow](#workflow)\n- [Usage](#usage)\n\n---\n\n## Overview\n\n**Directory:** `multi_agents/`\n\nLangGraph-based multi-agent system. Generates 5-6 page reports with multiple agents collaborating.\n\n---\n\n## Agent Roles\n\n| Agent | File | Role |\n|-------|------|------|\n| Human | - | Oversees and provides feedback |\n| Chief Editor | `agents/orchestrator.py` | Master coordinator via LangGraph |\n| Researcher | Uses GPTResearcher | Deep research on topics |\n| Editor | `agents/editor.py` | Plans outline and structure |\n| Reviewer | `agents/reviewer.py` | Validates research correctness |\n| Reviser | `agents/reviser.py` | Revises based on feedback |\n| Writer | `agents/writer.py` | Compiles final report |\n| Publisher | `agents/publisher.py` | Exports to PDF, DOCX, Markdown |\n\n---\n\n## Workflow\n\n```\n1. Browser (GPTResearcher) → Initial research\n2. Editor → Plans report outline\n3. For each outline topic (parallel):\n a. Researcher → In-depth subtopic research\n b. Reviewer → Validates draft\n c. Reviser → Revises until satisfactory\n4. Writer → Compiles final report\n5. Publisher → Exports to multiple formats\n```\n\n---\n\n## Usage\n\n### Via API\n\n```python\nreport_type = \"multi_agents\"\n```\n\n### Via WebSocket\n\n```json\n{\n \"task\": \"Research query\",\n \"report_type\": \"multi_agents\",\n \"tone\": \"Analytical\"\n}\n```\n\n### Directly in Python\n\n```python\nfrom multi_agents.main import run_research_task\n\nreport = await run_research_task(\n query=\"Comprehensive analysis of market trends\",\n websocket=handler,\n tone=Tone.Analytical,\n)\n```\n\n### Configuration File\n\n**File:** `multi_agents/task.json`\n\nConfigure the multi-agent research task parameters and agent behaviors.\n\n\n# Prompt System Reference\n\n## Table of Contents\n- [PromptFamily Class](#promptfamily-class)\n- [Key Prompt Examples](#key-prompt-examples)\n\n---\n\n## PromptFamily Class\n\n**File:** `gpt_researcher/prompts.py`\n\nAll prompts are centralized in the `PromptFamily` class. This allows for model-specific prompt variations.\n\n```python\nclass PromptFamily:\n \"\"\"\n General purpose class for prompt formatting.\n Can be overwritten with model-specific derived classes.\n \"\"\"\n\n def __init__(self, config: Config):\n self.cfg = config\n\n @staticmethod\n def get_prompt_by_report_type(report_type: str):\n \"\"\"Returns the appropriate prompt generator for the report type.\"\"\"\n match report_type:\n case ReportType.ResearchReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.DetailedReport.value:\n return PromptFamily.generate_report_prompt\n case ReportType.OutlineReport.value:\n return PromptFamily.generate_outline_report_prompt\n # ... etc\n```\n\n---\n\n## Key Prompt Examples\n\n### Agent Selection Prompt\n\n```python\n@staticmethod\ndef generate_agent_role_prompt(query: str, parent_query: str = \"\") -> str:\n return f\"\"\"Analyze the research query and select the most appropriate agent role.\n\nQuery: \"{query}\"\n{f'Parent Query: \"{parent_query}\"' if parent_query else ''}\n\nBased on the query, determine:\n1. The domain expertise needed\n2. The research approach required\n3. The appropriate agent persona\n\nReturn a JSON object with:\n- \"agent\": The agent type (e.g., \"Research Analyst\", \"Technical Writer\")\n- \"role\": A detailed role description for how the agent should approach this research\n\"\"\"\n```\n\n### Research Planning Prompt\n\n```python\n@staticmethod\ndef generate_search_queries_prompt(\n query: str,\n parent_query: str = \"\",\n report_type: str = \"\",\n max_iterations: int = 3,\n context: str = \"\",\n) -> str:\n return f\"\"\"Generate {max_iterations} focused search queries to research: \"{query}\"\n\nContext from initial search:\n{context}\n\nRequirements:\n- Each query should explore a different aspect\n- Queries should be specific and searchable\n- Consider the report type: {report_type}\n\nReturn a JSON array of query strings.\n\"\"\"\n```\n\n### Report Generation Prompt (with images)\n\n```python\n@staticmethod\ndef generate_report_prompt(\n question: str,\n context: str,\n report_source: str,\n report_format=\"apa\",\n total_words=1000,\n tone=None,\n language=\"english\",\n available_images: list = [],\n) -> str:\n # Build image embedding instruction if images available\n image_instruction = \"\"\n if available_images:\n image_list = \"\\n\".join([\n f\"- Title: {img.get('title')}\\n URL: {img['url']}\"\n for img in available_images\n ])\n image_instruction = f\"\"\"\nAVAILABLE IMAGES (embed where relevant):\n{image_list}\n\nUse markdown format: ![Title](URL)\n\"\"\"\n\n return f\"\"\"Information: \"{context}\"\n---\nUsing the above information, answer: \"{question}\" in a detailed report.\n\n- Format: {report_format}\n- Length: ~{total_words} words\n- Tone: {tone.value if tone else \"Objective\"}\n- Language: {language}\n- Include citations for all factual claims\n{image_instruction}\n\"\"\"\n```\n\n### MCP Tool Selection Prompt\n\n```python\n@staticmethod\ndef generate_mcp_tool_selection_prompt(query: str, tools_info: list, max_tools: int = 3) -> str:\n return f\"\"\"Select the most relevant tools for researching: \"{query}\"\n\nAVAILABLE TOOLS:\n{json.dumps(tools_info, indent=2)}\n\nSelect exactly {max_tools} tools ranked by relevance.\n\nReturn JSON:\n{{\n \"selected_tools\": [\n {{\"index\": 0, \"name\": \"tool_name\", \"relevance_score\": 9, \"reason\": \"...\"}}\n ]\n}}\n\"\"\"\n```\n\n\n# Retriever System Reference\n\n## Table of Contents\n- [Available Retrievers](#available-retrievers)\n- [Retriever Selection](#retriever-selection)\n- [Adding a New Retriever](#adding-a-new-retriever)\n\n---\n\n## Available Retrievers\n\n**Directory:** `gpt_researcher/retrievers/`\n\n| Retriever | Class | API Key Env Var |\n|-----------|-------|-----------------|\n| Tavily | `TavilySearch` | `TAVILY_API_KEY` |\n| Google | `GoogleSearch` | `GOOGLE_API_KEY`, `GOOGLE_CX_KEY` |\n| Bing | `BingSearch` | `BING_API_KEY` |\n| Brave | `BraveSearch` | `BRAVE_API_KEY` |\n| DuckDuckGo | `Duckduckgo` | None |\n| SearX | `SearxSearch` | `SEARX_URL` |\n| Serper | `SerperSearch` | `SERPER_API_KEY` |\n| SerpAPI | `SerpApiSearch` | `SERPAPI_API_KEY` |\n| SearchAPI | `SearchApiSearch` | `SEARCHAPI_API_KEY` |\n| Exa | `ExaSearch` | `EXA_API_KEY` |\n| GroundRoute | `GroundRouteSearch` | `GROUNDROUTE_API_KEY` |\n| fastCRW | `CRWRetriever` | `CRW_API_KEY` |\n| BoCha | `BoChaSearch` | `BOCHA_API_KEY` |\n| Xquik | `XquikSearch` | `XQUIK_API_KEY` |\n| GetXAPI (X/Twitter) | `GetXAPISearch` | `GETXAPI_API_KEY` |\n| arXiv | `ArxivSearch` | None |\n| Semantic Scholar | `SemanticScholarSearch` | None |\n| PubMed Central | `PubMedCentralSearch` | None |\n| OpenAlex | `OpenAlexSearch` | `OPENALEX_API_KEY`, `OPENALEX_EMAIL` |\n| MCP | `MCPRetriever` | Per-server |\n| Custom | `CustomRetriever` | User-defined |\n\n> The authoritative list is the `match` statement in `gpt_researcher/actions/retriever.py` (`get_retriever`).\n\n---\n\n## Retriever Selection\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ndef get_retriever(retriever: str):\n \"\"\"Get a retriever class by name.\"\"\"\n match retriever:\n case \"tavily\":\n from gpt_researcher.retrievers import TavilySearch\n return TavilySearch\n case \"google\":\n from gpt_researcher.retrievers import GoogleSearch\n return GoogleSearch\n case \"mcp\":\n from gpt_researcher.retrievers import MCPRetriever\n return MCPRetriever\n # ... etc\n\ndef get_retrievers(headers: dict[str, str], cfg) -> list:\n \"\"\"\n Resolve which retriever(s) to use, in priority order:\n headers[\"retrievers\"] / headers[\"retriever\"] → cfg.retrievers / cfg.retriever\n → default (TavilySearch). Accepts a comma-separated string or a list.\n\n Usage: RETRIEVER=tavily,google,mcp\n \"\"\"\n # ... resolve names from headers/cfg (see actions/retriever.py) ...\n # Invalid names fall back to get_default_retriever():\n return [get_retriever(r) or get_default_retriever() for r in retriever_names]\n```\n\n---\n\n## Adding a New Retriever\n\n### Step 1: Create Retriever File\n\n**File:** `gpt_researcher/retrievers/my_retriever/my_retriever.py`\n\n```python\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n self.headers = headers\n \n async def search(self, max_results: int = 10) -> list[dict]:\n \"\"\"\n Returns list of:\n {\n \"title\": str,\n \"href\": str,\n \"body\": str\n }\n \"\"\"\n # Implementation\n pass\n```\n\n### Step 2: Register in retriever.py\n\n**File:** `gpt_researcher/actions/retriever.py`\n\n```python\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n```\n\n### Step 3: Export in __init__.py\n\n**File:** `gpt_researcher/retrievers/__init__.py`\n\n```python\nfrom .my_retriever import MyRetriever\n__all__ = [..., \"MyRetriever\"]\n```\n\n### Step 4: Usage\n\n```bash\nRETRIEVER=tavily,my_retriever\n```\n\n```python\nresearcher = GPTResearcher(\n query=\"...\",\n # Will use both Tavily and your custom retriever\n)\n```\n\n\n---\nname: gpt-researcher\ndescription: GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines.\n---\n\n# GPT Researcher Development Skill\n\nGPT Researcher is an LLM-based autonomous agent using a planner-executor-publisher pattern with parallelized agent work for speed and reliability.\n\n## Quick Start\n\n### Basic Python Usage\n\n```python\nfrom gpt_researcher import GPTResearcher\nimport asyncio\n\nasync def main():\n researcher = GPTResearcher(\n query=\"What are the latest AI developments?\",\n report_type=\"research_report\", # or detailed_report, deep, outline_report\n report_source=\"web\", # or local, hybrid\n )\n await researcher.conduct_research()\n report = await researcher.write_report()\n print(report)\n\nasyncio.run(main())\n```\n\n### Run Servers\n\n```bash\n# Backend\npython -m uvicorn backend.server.server:app --reload --port 8000\n\n# Frontend\ncd frontend/nextjs && npm install && npm run dev\n```\n\n---\n\n## Key File Locations\n\n| Need | Primary File | Key Classes |\n|------|--------------|-------------|\n| Main orchestrator | `gpt_researcher/agent.py` | `GPTResearcher` |\n| Research logic | `gpt_researcher/skills/researcher.py` | `ResearchConductor` |\n| Report writing | `gpt_researcher/skills/writer.py` | `ReportGenerator` |\n| All prompts | `gpt_researcher/prompts.py` | `PromptFamily` |\n| Configuration | `gpt_researcher/config/config.py` | `Config` |\n| Config defaults | `gpt_researcher/config/variables/default.py` | `DEFAULT_CONFIG` |\n| API server | `backend/server/app.py` | FastAPI `app` |\n| Search engines | `gpt_researcher/retrievers/` | Various retrievers |\n\n---\n\n## Architecture Overview\n\n```\nUser Query → GPTResearcher.__init__()\n │\n ▼\n choose_agent() → (agent_type, role_prompt)\n │\n ▼\n ResearchConductor.conduct_research()\n ├── plan_research() → sub_queries\n ├── For each sub_query:\n │ └── _process_sub_query() → context\n └── Aggregate contexts\n │\n ▼\n [Optional] ImageGenerator.plan_and_generate_images()\n │\n ▼\n ReportGenerator.write_report() → Markdown report\n```\n\n**For detailed architecture diagrams**: See [references/architecture.md](references/architecture.md)\n\n---\n\n## Core Patterns\n\n### Adding a New Feature (8-Step Pattern)\n\n1. **Config** → Add to `gpt_researcher/config/variables/default.py`\n2. **Provider** → Create in `gpt_researcher/llm_provider/my_feature/`\n3. **Skill** → Create in `gpt_researcher/skills/my_feature.py`\n4. **Agent** → Integrate in `gpt_researcher/agent.py`\n5. **Prompts** → Update `gpt_researcher/prompts.py`\n6. **WebSocket** → Events via `stream_output()`\n7. **Frontend** → Handle events in `useWebSocket.ts`\n8. **Docs** → Create `docs/docs/gpt-researcher/gptr/my_feature.md`\n\n**For complete feature addition guide with Image Generation case study**: See [references/adding-features.md](references/adding-features.md)\n\n### Adding a New Retriever\n\n```python\n# 1. Create: gpt_researcher/retrievers/my_retriever/my_retriever.py\nclass MyRetriever:\n def __init__(self, query: str, headers: dict = None):\n self.query = query\n \n async def search(self, max_results: int = 10) -> list[dict]:\n # Return: [{\"title\": str, \"href\": str, \"body\": str}]\n pass\n\n# 2. Register in gpt_researcher/actions/retriever.py\ncase \"my_retriever\":\n from gpt_researcher.retrievers.my_retriever import MyRetriever\n return MyRetriever\n\n# 3. Export in gpt_researcher/retrievers/__init__.py\n```\n\n**For complete retriever documentation**: See [references/retrievers.md](references/retrievers.md)\n\n---\n\n## Configuration\n\nConfig keys are **lowercased** when accessed:\n\n```python\n# In default.py: \"SMART_LLM\": \"gpt-4o\"\n# Access as: self.cfg.smart_llm # lowercase!\n```\n\nPriority: Environment Variables → JSON Config File → Default Values\n\n**For complete configuration reference**: See [references/config-reference.md](references/config-reference.md)\n\n---\n\n## Common Integration Points\n\n### WebSocket Streaming\n\n```python\nclass WebSocketHandler:\n async def send_json(self, data):\n print(f\"[{data['type']}] {data.get('output', '')}\")\n\nresearcher = GPTResearcher(query=\"...\", websocket=WebSocketHandler())\n```\n\n### MCP Data Sources\n\n```python\nresearcher = GPTResearcher(\n query=\"Open source AI projects\",\n mcp_configs=[{\n \"name\": \"github\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-github\"],\n \"env\": {\"GITHUB_TOKEN\": os.getenv(\"GITHUB_TOKEN\")}\n }],\n mcp_strategy=\"deep\", # or \"fast\", \"disabled\"\n)\n```\n\n**For MCP integration details**: See [references/mcp.md](references/mcp.md)\n\n### Deep Research Mode\n\n```python\nresearcher = GPTResearcher(\n query=\"Comprehensive analysis of quantum computing\",\n report_type=\"deep\", # Triggers recursive tree-like exploration\n)\n```\n\n**For deep research configuration**: See [references/deep-research.md](references/deep-research.md)\n\n---\n\n## Error Handling\n\nAlways use graceful degradation in skills:\n\n```python\nasync def execute(self, ...):\n if not self.is_enabled():\n return [] # Don't crash\n \n try:\n result = await self.provider.execute(...)\n return result\n except Exception as e:\n await stream_output(\"logs\", \"error\", f\"⚠️ {e}\", self.websocket)\n return [] # Graceful degradation\n```\n\n---\n\n## Critical Gotchas\n\n| ❌ Mistake | ✅ Correct |\n|-----------|-----------|\n| `config.MY_VAR` | `config.my_var` (lowercased) |\n| Editing pip-installed package | `pip install -e .` |\n| Forgetting async/await | All research methods are async |\n| `websocket.send_json()` on None | Check `if websocket:` first |\n| Not registering retriever | Add to `retriever.py` match statement |\n\n---\n\n## Reference Documentation\n\n| Topic | File |\n|-------|------|\n| System architecture & diagrams | [references/architecture.md](references/architecture.md) |\n| Core components & signatures | [references/components.md](references/components.md) |\n| Research flow & data flow | [references/flows.md](references/flows.md) |\n| Prompt system | [references/prompts.md](references/prompts.md) |\n| Retriever system | [references/retrievers.md](references/retrievers.md) |\n| MCP integration | [references/mcp.md](references/mcp.md) |\n| Deep research mode | [references/deep-research.md](references/deep-research.md) |\n| Multi-agent system | [references/multi-agents.md](references/multi-agents.md) |\n| Adding features guide | [references/adding-features.md](references/adding-features.md) |\n| Advanced patterns | [references/advanced-patterns.md](references/advanced-patterns.md) |\n| REST & WebSocket API | [references/api-reference.md](references/api-reference.md) |\n| Configuration variables | [references/config-reference.md](references/config-reference.md) |\n\n\n# Context filter benchmark: embeddings vs Jev vs no filter\n\n![Context filter benchmark results](results.png)\n\nEvery scraped page reaches a GPT Researcher report through one step: for each\nsub-query, the scraped pages are cut into chunks and a filter decides which\nchunks become the writer's context. Until now that filter was embedding\nsimilarity. This benchmark asks two questions:\n\n1. Is there a better filter? We test [TypeSafe's Jev](https://docs.typesafe.ai/),\n a model that returns calibrated, typed decisions, scoring each chunk on how\n useful it is for answering the question.\n2. With today's long context windows, do we need a filter at all?\n\n## Method\n\n**Fixed inputs.** `collect.py` runs 28 real research tasks once and records\nthe exact `(sub_query, pages)` pairs that reached the filter: 20 SimpleQA\nquestions with known answers (fixed seed, from `evals/simple_evals`) and 8\nopen-ended research questions. `replay.py` rebuilds the context from those\nsame pages with each strategy and writes the report with the same model\n(`gpt-5.4`). Search and scraping, the noisiest parts of a run, are held\nconstant, so the filter is the only thing that differs.\n\n**Strategies.**\n\n| strategy | what reaches the writer |\n|---|---|\n| `embeddings` | today: 1000-char chunks, cosine similarity > 0.42, top 10 per sub-query |\n| `jev` | same chunks, scored by Jev on a 4-level usefulness rubric, top 10 scoring ≥ 1.5 |\n| `jev_top10` | Jev's 10 best chunks, no threshold (same budget as embeddings) |\n| `jev_large` | Jev on 3000-char chunks, top 4 (≈ same characters, a third of the calls) |\n| `jev_wide` | Jev threshold 1.5, up to 30 chunks |\n| `jev_broad` | Jev on 3000-char chunks, threshold 1.0, up to 12 (open-ended only) |\n| `bm25` | keyword (BM25) ranking of the same chunks, top 10; runs locally, no API or model |\n| `bm25_rel` | BM25, keeping chunks scoring ≥ 50% of the best chunk, top 10 |\n| `bm25_wide` | BM25 with the same relative threshold, up to 25 chunks (shipped as `keyword`) |\n| `none_capped` | no ranking: pages in search order, capped at 48k characters per sub-query |\n| `none` | no filter: every page in full (capped at 50k chars per page, as today) |\n| `truncate` | control: 10 chunks round-robin across pages, no relevance ranking |\n\n**Quality signals** (`judge.py`):\n\n- **SimpleQA accuracy**: each report graded with the repo's SimpleQA grader.\n- **Context precision**: the share of context chunks an independent judge\n (`gpt-5.4-mini`) rates as relevant to the question. This measures the filter\n itself. It isn't defined for `none`.\n- **Pairwise preference**: a blind head-to-head judged by `gpt-5.4`, run in both\n orders. A win counts only if the strategy wins in *both* orders; any\n disagreement is a tie.\n\n## Results\n\n### Cost and speed (28 questions)\n\n| strategy | filter time p50 / p90 | filter $ | context tokens p50 (max) | write $ | total $ / report |\n|---|---|---|---|---|---|\n| embeddings | 1.0s / 1.3s | 0.0028 | 7.6k (10.7k) | 0.114 | **0.117** |\n| jev | 2.4s / 6.0s ¹ | 0.0054 | 4.5k (10.6k) | 0.110 | **0.115** |\n| jev_top10 | 2.7s / 6.2s | 0.0054 | 8.6k (10.6k) | 0.119 | 0.124 |\n| jev_large | 1.2s / 5.1s | 0.0031 | 6.5k | 0.111 | 0.114 |\n| truncate | 0.0s | — | 8.0k (11.0k) | 0.117 | 0.117 |\n| none | 0.0s | — | **33.4k (107.7k)** | **0.192** | **0.192** |\n\n¹ Measured with 32 concurrent Jev calls. At 64, the new default, the same\nfiltering takes **1.7s p50 / 3.6s p90**; 128 was no faster.\n\nReport writing takes the same ~45s for every strategy: generating the output\ndominates, and input size barely matters. Removing the filter makes nothing\nfaster; it makes each report 65% more expensive.\n\n### Quality\n\n| strategy | SimpleQA correct | context precision | vs embeddings (W/T/L) | vs none (W/T/L) |\n|---|---|---|---|---|\n| embeddings | 18/20 | 0.46 | — | 3 / 12 / 13 |\n| **jev** | 19/20 | **0.73** | **15 / 10 / 3** | 6 / 12 / 10 |\n| jev_top10 | 19/20 | 0.50 | 13 / 10 / 5 | — |\n| jev_large | 19/20 | 0.72 | 11 / 10 / 7 | 8 / 7 / 13 |\n| jev_wide | 20/20 | **0.74** | 13 / 8 / 7 | 7 / 11 / 10 |\n| truncate | 19/20 | 0.33 | 7 / 8 / 13 | — |\n| none | 20/20 | — | 13 / 11 / 4 | — |\n\n### Open-ended questions only (8)\n\n| strategy | context p50 | total $ / report | vs embeddings | vs none |\n|---|---|---|---|---|\n| embeddings | 8.8k | 0.155 | — | 0 / 1 / 7 |\n| jev | 9.1k | 0.160 | 5 / 2 / 1 | 0 / 2 / 6 |\n| jev_broad | 26.9k | 0.191 | **7 / 1 / 0** | 0 / 3 / 5 |\n| none | 61.3k | **0.284** | 8 / 0 / 0 | — |\n\nThe `none` reports on open-ended questions were *shorter* than the embeddings\nreports (2,099 vs 2,416 median words) and still preferred, so the judge is not\nrewarding length.\n\n### Filters that need nothing: keyword (BM25)\n\nWithout a Jev key, GPT Researcher needs a filter with no API, model or\nembeddings. The candidates, on the same 28 runs:\n\n| strategy | SimpleQA | context precision | vs embeddings (W/T/L) | vs none (W/T/L) | filter p50 | context p50 | total $ |\n|---|---|---|---|---|---|---|---|\n| bm25 | 18/20 | 0.40 | 7 / 9 / 12 | 3 / 12 / 13 | 0.02s | 8.1k | 0.118 |\n| bm25_rel | 19/20 | 0.53 | 7 / 13 / 8 | 5 / 10 / 13 | 0.02s | 4.7k | **0.107** |\n| **bm25_wide** | 19/20 | 0.51 | **14 / 8 / 6** | 6 / 9 / 13 | 0.02s | 6.4k | 0.116 |\n| none_capped | 20/20 | 0.54 | 11 / 13 / 4 | 6 / 10 / 12 | 0.00s | 23.8k | 0.152 |\n| *embeddings* | *18/20* | *0.46* | — | *3 / 12 / 13* | *1.04s* | *7.6k* | *0.117* |\n\n`bm25_wide` ships as the `keyword` fallback: at least as good as embeddings on\nevery measure, at the same cost, about 50× faster to filter, and with nothing\nto set up. Its 14–6 head-to-head is directional (sign test p ≈ 0.12), not as\nstrong as Jev's 15–3. As with Jev, the relative threshold is what does the\nwork: plain top-10 BM25 loses to embeddings 7–12.\n\nGPT Researcher removes duplicate URLs across sub-queries before scraping\n(0 of 300 pages recurred), so \"sources that came back most often\" isn't a\nsignal available at this step.\n\n## What this shows\n\n1. **Jev is a better filter than embeddings at the same cost.** 73% of the\n chunks Jev selects are relevant, against 46% for embeddings; the unranked\n control gets 33%. Jev reports beat embeddings reports 15–3 head-to-head\n (sign test on the 18 non-ties, p ≈ 0.008). Every Jev variant beat\n embeddings. Total cost per report is the same; both filters cost well\n under a cent.\n2. **Most of the gain comes from the threshold.** `jev_top10`, forced to fill\n 10 slots, drops to 0.50 precision. Cosine similarity cannot say \"nothing\n else here is worth including\"; a calibrated usefulness score can.\n3. **Skipping the filter gives the best open-ended reports, at 65–83% higher\n cost.** More source material makes for broader reports. But on a *standard*\n research report the context already reaches 108k tokens; detailed and deep\n research run many more sub-queries and would overflow context windows and\n budgets. For fact-finding questions it adds nothing: SimpleQA is at\n ceiling for every strategy, and Jev vs none is 6–10–4.\n4. **No integration is needed for a good filter.** Keyword ranking with a\n relative threshold matches or beats embeddings at the same cost (above),\n so the default without a Jev key is `keyword`, and nothing requires an\n embeddings provider.\n5. **SimpleQA is saturated.** 18–20 of 20 for every strategy: when search\n finds the fact, any reasonable filter keeps it. Differences of one question\n are noise.\n\n## Caveats\n\n- 28 questions, judged by one model family (`gpt-5.4`), which also writes the\n reports. Self-preference would apply to every strategy equally, but the\n pairwise numbers are directional evidence, not a leaderboard.\n- One writer model and one report type (`research_report`). Detailed and deep\n research were not replayed.\n- Jev latency depends on chunk count and concurrency. `jev_large` matches\n embeddings at the median; tails come from large pages.\n\n## Reproduce\n\n```bash\nexport OPENAI_API_KEY=... TAVILY_API_KEY=... TYPESAFE_API_KEY=...\npython -m evals.context_filter.collect --out runs/ # ~$0.40, ~20 min\npython -m evals.context_filter.replay --runs runs/ --out results/\npython -m evals.context_filter.judge --results results/\npython -m evals.context_filter.judge --results results/ --baseline none --only jev embeddings\npython -m evals.context_filter.replay --runs runs/ --out results/ --strategies bm25 bm25_rel bm25_wide none_capped\n```\n\n`results/` in this folder holds the scores and per-run metrics from the run\nabove (`runs.json`, `scored_*.json`, `summary_*.json`). Scraped pages and\ngenerated reports are not committed.\n\nTotal spend: about $40 in tracked writing and filtering across all strategies\n(of which $0.59 was Jev; keyword and none cost nothing to filter), plus an\nestimated $8–15 for judging.\n\n\n# quality_eval — Report Quality Evaluation for Autonomous Research Agents\n\nEvaluate the quality of a GPT-Researcher report on **any open-ended query, with no\nground-truth answer required**. Where `simple_evals` needs a labelled dataset and\n`hallucination_eval` gives a binary check, `quality_eval` scores a report across\nseveral interpretable dimensions of *how well it was researched and written*.\n\nIt follows the metric abstraction of established frameworks (DeepEval / RAGAS),\nand **extends it for autonomous research** — evaluating not just the answer, but\nthe quality of the sources the agent found on its own.\n\n## What it measures\n\n| Group | Aligns with | Metric | Question it answers |\n|---|---|---|---|\n| **Faithfulness** | RAGAS Faithfulness | `unsupported_claim` | Are the report's claims grounded in the sources? |\n| **Answer Relevancy** | RAGAS answer relevancy | `subtopic_coverage` | Does it cover the subtopics a thorough report should? |\n| **Source Quality** | *extension (autonomous agent)* | `source_authority`, `source_diversity` | Are the sources the agent found authoritative and varied? |\n| **Citation** | ALCE citation precision | `citation_faithfulness` | Are the listed references actually cited in the body? |\n\nZero-cost metrics (citation, diversity, authority) run by default; the two LLM\nmetrics are opt-out via flags, so you can trade cost for depth.\n\n## Concept alignment (why this is more than ad-hoc metrics)\n\nThe suite maps onto standard evaluation ontologies where they apply, and marks\nwhere autonomous research needs more:\n\n- **Faithfulness / Answer Relevancy** ← align with **RAGAS**. `unsupported_claim`\n even uses the field's standard three-way support scale (supported / inferred /\n unsupported), matching ALCE-style *full / partial / no support*.\n- **Citation** ← aligns with **ALCE citation precision** (of the references it\n lists, how many are actually used). Current implementation is *structural*; a\n *semantic* version (does the citation truly support its claim, NLI-style) is on\n the roadmap.\n- **Source Quality** ← a deliberate **extension**. RAG-oriented frameworks assume\n the context is *given*, so they have no notion of \"source quality\". A research\n agent finds its own sources, so authority and diversity of those sources become\n first-class metrics — this is the dimension standard RAG eval can't express.\n\n## Architecture\n\nA thin, dependency-free framework layer (mirrors DeepEval's `BaseMetric` +\ntest-case design):\n\n```text\nbase.py BaseMetric / EvalSample / MetricResult — abstractions (+ `sources`, the RAG-framework gap)\nsuite.py 5 metric wrappers + evaluate() entry — standardized I/O + concept-alignment metadata\nmetrics.py raw metric functions — pure logic (unit-tested, perturbation-validated)\nperturbation.py behavioral reliability test — graded degradation, asserts monotonic response\nrun_eval.py / benchmark.py — runners (single query / multi-config comparison)\n```\n\nEvery metric declares its `group` and the standard concept it `aligned_with`, so\nthe ontology above is encoded in code, not just docs. `MetricResult.score` is\nalways *higher = better* (0–1) so results aggregate and compare uniformly.\n\n## Reliability (perturbation-validated)\n\nThe metrics aren't just implemented — they're checked for *reliability*.\n`perturbation.py` deliberately degrades a report (remove citations, swap to\nlow-authority sources, corrupt claims) in graded steps and asserts each metric\nresponds **monotonically**. This surfaced and fixed real defects that mocked unit\ntests and final outputs couldn't reveal — e.g. batch LLM scoring was letting one\nclaim contaminate another's score, fixed by scoring each claim independently.\n\n## Usage\n\n```bash\n# Full evaluation (all metrics)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost metrics only\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-unsupported-claim\n\n# Reliability check\npython -m evals.quality_eval.perturbation\n\n# Compare configs / runs\npython -m evals.quality_eval.benchmark --configs single-gpt4o single-gpt4o-mini\npython -m evals.quality_eval.benchmark --compare LOG_A LOG_B\n```\n\nProgrammatic use via the standardized entry point:\n\n```python\nfrom evals.quality_eval.base import EvalSample\nfrom evals.quality_eval.suite import evaluate, default_metrics\n\nsample = EvalSample(query=q, report=report, sources=urls, context=ctx)\nresults = await evaluate(sample, default_metrics(), grader_model)\nfor r in results:\n print(r.group, r.name, r.score) # typed, tagged, higher = better\n```\n\n## Roadmap\n\n- **Semantic citation precision** (does a citation truly support its claim,\n NLI/LLM-judge) → promote the `Citation` group to a full `Precision` family.\n- **True answer relevancy** (on-topic-ness), complementing `subtopic_coverage`'s\n completeness view.\n- **Component-level evaluation** of the multi-agent pipeline (planner / researcher\n / writer / reviewer) to locate bottlenecks — the natural next phase.\n\nDependencies: `langchain-openai`, `python-dotenv` (+ `judges` for the optional\nhallucination check). Env: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n\n\n# GPT-Researcher Evaluations\n\nThis directory contains evaluation tools and frameworks for assessing the performance of GPT-Researcher across different research tasks.\n\n## Evaluation Overview\n\nThree complementary evaluation systems, each answering a different question:\n\n| System | Question | Requires ground truth? |\n| --- | --- | --- |\n| `simple_evals/` | Is the answer factually correct? | Yes (OpenAI SimpleQA dataset) |\n| `quality_eval/` | Is the report well-sourced and complete? | No — works on any query |\n| `hallucination_eval/` | Does the report fabricate content? | No — compares against scraped sources |\n\n**Quick start:**\n\n```bash\n# Factual accuracy (needs ground-truth answers)\npython -m evals.simple_evals.run_eval --num_examples 10\n\n# Report quality (works on any query, zero-cost metrics only)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Hallucination detection\npython -m evals.hallucination_eval.run_eval -n 5\n```\n\n## Simple Evaluations (`simple_evals/`)\n\nThe `simple_evals` directory contains a straightforward evaluation framework adapted from [OpenAI's simple-evals system](https://github.com/openai/simple-evals), specifically designed to measure short-form factuality in large language models. Our implementation is based on OpenAI's [SimpleQA evaluation methodology](https://github.com/openai/simple-evals/blob/main/simpleqa_eval.py), following their zero-shot, chain-of-thought approach while adapting it for GPT-Researcher's specific use case.\n\n### Components\n\n- `simpleqa_eval.py`: Core evaluation logic for grading research responses\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `requirements.txt`: Dependencies required for running evaluations\n\n### Test Dataset\n\nThe `problems/` directory contains the evaluation dataset:\n\n- `Simple QA Test Set.csv`: A comprehensive collection of factual questions and their correct answers, mirrored from OpenAI's original test set. This dataset serves as the ground truth for evaluating GPT-Researcher's ability to find and report accurate information. The file is maintained locally to ensure consistent evaluation benchmarks and prevent any potential upstream changes from affecting our testing methodology.\n\n### Evaluation Logs\n\nThe `logs/` directory contains evaluation run histories preserved in version control.\n\nTwo formats are kept side-by-side:\n\n| Format | Example filename | Purpose |\n| --- | --- | --- |\n| Plain text (legacy) | `SimpleQA Eval 100 Problems 2-22-25.txt` | Human-readable terminal output |\n| Structured JSON | `eval_2025-02-22_10-32-11_n100.jsonl` | Machine-readable, suitable for trend analysis |\n\nEach JSON log file contains one record with three sections: `run_metadata`, `aggregate_metrics`, and per-query `results`. See [`example_output.json`](simple_evals/example_output.json) for the full schema.\n\n**Note:** Unlike typical log directories, this folder and its contents are intentionally tracked in git to maintain a historical record of evaluation runs.\n\n### Features\n\n- Measures factual accuracy of research responses\n- Uses GPT-4 as a grading model (configurable)\n ```python\n # In run_eval.py, you can customize the grader model:\n grader_model = ChatOpenAI(\n temperature=0, # Lower temperature for more consistent grading\n model_name=\"gpt-4-turbo\", # Can be changed to other OpenAI models\n openai_api_key=os.getenv(\"OPENAI_API_KEY\")\n )\n ```\n- Grades responses on a three-point scale:\n - `CORRECT`: Answer fully contains important information without contradictions\n - `INCORRECT`: Answer contains factual contradictions\n - `NOT_ATTEMPTED`: Answer neither confirms nor contradicts the target\n\n**Note on Grader Configuration:** While the default grader uses GPT-4-turbo, you can modify the model and its parameters to use different OpenAI models or adjust the temperature for different grading behaviors. This is independent of the researcher's configuration, allowing you to optimize for cost or performance as needed.\n\n### Metrics Tracked\n\n- Accuracy rate and F1 score\n- Cost — total and per-query average\n- Latency — average, p50 (median), and p95 per query\n- Success/failure counts and answer attempt rates\n- Source coverage per query\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/simple_evals\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\nLANGCHAIN_API_KEY=your_langchain_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py --num_examples \n```\n\nThe `num_examples` parameter determines how many random test queries to evaluate (default: 1).\n\n#### Customizing Researcher Behavior\n\nThe evaluation uses GPTResearcher with default settings, but you can modify `run_eval.py` to customize the researcher's behavior:\n\n```python\nresearcher = GPTResearcher(\n query=query,\n report_type=ReportType.ResearchReport.value, # Type of report to generate\n report_format=\"markdown\", # Output format\n report_source=ReportSource.Web.value, # Source of research\n tone=Tone.Objective, # Writing tone\n verbose=True # Enable detailed logging\n)\n```\n\nThese parameters can be adjusted to evaluate different research configurations or output formats. For a complete list of configuration options, see the [configuration documentation](https://docs.gptr.dev/docs/gpt-researcher/gptr/config).\n\n**Note on Configuration Independence:** The evaluation system is designed to be independent of the researcher's configuration. This means you can use different LLMs and settings for evaluation versus research. For example:\n- Evaluation could use GPT-4-turbo for grading while the researcher uses Claude 3.5 Sonnet for research\n- Different retrievers, embeddings, or report formats can be used\n- Token limits and other parameters can be customized separately\n\nThis separation allows for unbiased evaluation across different researcher configurations. However, please note that this feature is currently experimental and needs further testing.\n\n### Output\n\nEach run produces two outputs:\n\n#### Console summary\n\n```text\n=== Evaluation Summary ===\nTotal queries tested: 100\nSuccessful queries: 100\nFailed queries: 0\n\n=== AGGREGATE METRICS ===\nDebug counts:\nTotal successful: 100\nCORRECT: 92\nINCORRECT: 7\nNOT_ATTEMPTED: 1\n{\n \"correct_rate\": 0.92,\n \"incorrect_rate\": 0.07,\n \"not_attempted_rate\": 0.01,\n \"answer_rate\": 0.99,\n \"accuracy\": 0.9292929292929293,\n \"f1\": 0.9246231155778895\n}\n========================\nAccuracy: 0.929\nF1 Score: 0.925\n\nTotal cost: $9.6000\nAverage cost per query: $0.0960\nLatency — avg: 48.3s p50: 45.1s p95: 89.2s\n\nStructured log saved → evals/simple_evals/logs/eval_2025-02-22_10-32-11_n100.jsonl\n```\n\n**Structured JSON log** (`logs/eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`)\n\nSee [`example_output.json`](simple_evals/example_output.json) for the full schema. Key fields:\n\n```\nrun_metadata → timestamp, git_commit, grader_model, researcher_model\naggregate_metrics → accuracy, f1, cost, avg/p50/p95 latency, success/fail counts\nresults → per-query list with grade, score, latency, cost, sources\n```\n\n## Quality Evaluation (`quality_eval/`)\n\nMeasures the structural and content quality of research reports without requiring ground-truth answers. All metrics run on any open-ended query.\n\nAll metric logic lives in `metrics.py`. The runner is `run_eval.py`.\n\n### Metrics\n\n| Metric | Description | LLM calls | Cost |\n| --- | --- | --- | --- |\n| **Citation Faithfulness** | Are sources listed in the report's `## References` actually cited in the body? (catches decorative refs) | 0 | $0 |\n| **Source Diversity** | Domain variety via ratio + Shannon entropy | 0 | $0 |\n| **Source Authority** | Heuristic score per domain (.gov=1.0 → .com=0.5); unknown domains optionally LLM-scored | 0–1 | $0–$0.01 |\n| **Subtopic Coverage** | LLM-judged coverage of expected subtopics (adaptive count) | 2 | ~$0.02/query |\n| **Hallucination** | Binary faithfulness check — report vs. scraped sources | 1 | ~$0.01/query |\n| **Unsupported Claim Rate** | Claim-level credibility: supported (1.0) / inferred (0.3–0.9) / unsupported (0.0) | 2 | ~$0.02/query |\n\n**Citation Faithfulness** measures whether sources the report *lists* in its `## References` are *actually cited* in the body (a low score flags decorative references). It deliberately does not reward raw coverage — reading many sources and citing only the useful ones is good practice, like a real researcher.\n\n**Unsupported Claim Rate** distinguishes between:\n\n- `supported` — claim directly found in source text (score 1.0)\n- `inferred` — reasonable synthesis from multiple sources; never applied to data claims (score 0.3–0.9)\n- `unsupported` — no traceable basis in sources (score 0.0)\n\n### Metric Grouping & Tooling\n\nThe metrics group into three families: **Source Quality** (diversity, authority), **Writing Behavior** (citation faithfulness, subtopic coverage), and **Hallucination** (hallucination check, unsupported claim) — i.e. *is the input good*, *how was it written*, *is it correct*.\n\nTwo tools complement `run_eval.py` (shared metric logic lives in `metrics.py`):\n\n- **`benchmark.py`** — runs multiple researcher configurations (single- vs. multi-agent × model) over a fixed topic set and prints a side-by-side table. `--compare LOG_A LOG_B` diffs two past runs; `--summary-only` re-prints from saved logs.\n- **`perturbation.py`** — a behavioral test that deliberately degrades a report (remove inline citations, swap sources to low-authority, corrupt claims) and checks each metric responds monotonically. This validates metric *reliability* (directional sensitivity), which is separate from whether a metric captures *quality*.\n\n### Running\n\n```bash\n# Run all 6 metrics (most expensive, ~$0.06/query)\npython -m evals.quality_eval.run_eval --num_examples 5\n\n# Zero-cost only (citation ratio, diversity, authority)\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic --no-hallucination --no-unsupported-claim\n\n# Skip individual LLM metrics selectively\npython -m evals.quality_eval.run_eval --num_examples 10 --no-subtopic\npython -m evals.quality_eval.run_eval --num_examples 10 --no-hallucination\npython -m evals.quality_eval.run_eval --num_examples 10 --no-unsupported-claim\n```\n\nLogs are saved to `evals/quality_eval/logs/quality_eval_YYYY-MM-DD_HH-MM-SS_n{N}.jsonl`.\n\n### Example Output\n\n```text\nQuality eval: 5 queries subtopic=off hallucination=off unsupported_claim=on\n\n>> Query: What legal frameworks govern cross-border data transfers?\n citation_faithful: 0.95 (1 listed-only)\n diversity_ratio: 1.00 entropy=4.32\n authority_score: 0.62\n avg_claim_score: 0.79 (unsupported=0.11 inferred=0.22 n=9)\n latency=37.2s cost=$0.13\n\n=== QUALITY EVAL SUMMARY ===\n avg_citation_faithfulness: 0.94\n avg_diversity_ratio: 0.796\n avg_domain_entropy: 3.525\n avg_authority_score: 0.597\n avg_claim_score: 0.428\n avg_unsupported_claim_rate: 0.535\n avg_inferred_claim_rate: 0.148\n avg_latency_seconds: 33.6\n avg_cost: 0.116\n```\n\n### Dependencies\n\n```bash\npip install langchain-openai python-dotenv\n\n# Hallucination metric only:\npip install judges\n```\n\nEnvironment variables required: `OPENAI_API_KEY`, `TAVILY_API_KEY`.\n\n---\n\n## Hallucination Evaluation (`hallucination_eval/`)\n\nThe `hallucination_eval` directory contains tools for evaluating GPT-Researcher's outputs for hallucination. This evaluation system compares the generated research reports against their source materials to detect non-factual or hallucinated content, ensuring the reliability and accuracy of the research outputs.\n\n### Components\n\n- `run_eval.py`: Script to execute evaluations against GPT-Researcher\n- `evaluate.py`: Core evaluation logic for detecting hallucinations\n- `inputs/`: Directory containing test queries\n - `search_queries.jsonl`: Collection of research queries for evaluation\n- `results/`: Directory containing evaluation results\n - `evaluation_records.jsonl`: Detailed per-query evaluation records\n - `aggregate_results.json`: Summary metrics across all evaluations\n\n### Features\n\n- Evaluates research reports against source materials\n- Provides detailed reasoning for hallucination detection\n\n### Running Evaluations\n\n1. Install dependencies:\n```bash\ncd evals/hallucination_eval\npip install -r requirements.txt\n```\n\n2. Set up environment variables in `.env` file:\n```bash\n# Use the root .env file\nOPENAI_API_KEY=your_openai_key_here\nTAVILY_API_KEY=your_tavily_key_here\n```\n\n3. Run evaluation:\n```bash\npython run_eval.py -n \n```\n\nThe `-n` parameter determines how many queries to evaluate from the test set (default: 1).\n\n### Sample Result\n\n```json\n{\n \"total_queries\": 1,\n \"successful_queries\": 1,\n \"total_responses\": 1,\n \"total_evaluated\": 1,\n \"total_hallucinated\": 0,\n \"hallucination_rate\": 0.0,\n \"results\": [\n {\n \"input\": \"What are the latest developments in quantum computing?\",\n \"output\": \"Research report content...\",\n \"source\": \"Source material content...\",\n \"is_hallucination\": false,\n \"confidence_score\": 0.95,\n \"reasoning\": \"The summary accurately reflects the source material with proper citations...\"\n }\n ]\n}\n```\n\n\n# Evaluation Results\n\nThis directory contains historical evaluation results for GPT-Researcher using the SimpleQA methodology.\n\n## Latest Results\n\n### [SimpleQA Eval 100 Problems 2-22-25](./SimpleQA%20Eval%20100%20Problems%202-22-25.txt)\n\nEvaluation run by [Kelly Abbott (kga245)](https://github.com/kga245)\n\n**Summary:**\n- Date: February 22, 2025\n- Sample Size: 100 queries\n- Success Rate: 100% (100/100 queries completed)\n\n**Performance Metrics:**\n- Accuracy: 92.9%\n- F1 Score: 92.5%\n- Answer Rate: 99%\n\n**Response Distribution:**\n- Correct: 92%\n- Incorrect: 7%\n- Not Attempted: 1%\n\n**Cost Efficiency:**\n- Total Cost: $9.60\n- Average Cost per Query: $0.096\n\nThis evaluation demonstrates strong performance in factual accuracy while maintaining reasonable cost efficiency. The high answer rate (99%) and accuracy (92.9%) suggest that GPT-Researcher is effective at finding and reporting accurate information.\n\n## Historical Context\n\nThese logs are maintained in version control to:\n1. Track performance improvements over time\n2. Provide benchmarks for future enhancements\n3. Enable analysis of different configurations\n4. Ensure transparency in our evaluation process\n\nEach log file contains detailed information about:\n- Individual query results\n- Source citations\n- Cost breakdowns\n- Error analysis\n- Aggregate metrics\n\n## Running New Evaluations\n\nTo generate new evaluation logs, see the [main evaluation documentation](../README.md) for instructions on running evaluations with different configurations or sample sizes.\n\n\n---\nname: gpt-researcher\ndescription: Autonomous deep research from Codex via MCP\n---\n\n# GPT Researcher for Codex\n\nUse GPT Researcher from Codex via MCP.\n\n## When to use\n- When you need gpt-researcher capabilities in your Codex workflow\n- See https://github.com/assafelovic/gpt-researcher for full setup instructions\n\n\n{\n \"name\": \"gpt-researcher\",\n \"version\": \"0.1.0\",\n \"description\": \"Autonomous deep research from Codex via MCP\",\n \"author\": {\n \"name\": \"assafelovic\",\n \"url\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"homepage\": \"https://github.com/assafelovic/gpt-researcher\",\n \"repository\": \"https://github.com/assafelovic/gpt-researcher\",\n \"keywords\": [\n \"mcp\",\n \"codex\"\n ],\n \"mcpServers\": \"./.mcp.json\",\n \"interface\": {\n \"displayName\": \"GPT Researcher\",\n \"shortDescription\": \"Autonomous deep research from Codex via MCP\",\n \"longDescription\": \"An autonomous agent that conducts deep research on any data using any LLM providers.\",\n \"category\": \"Research\",\n \"websiteURL\": \"https://github.com/assafelovic/gpt-researcher\"\n },\n \"skills\": \"./skills/\"\n}\n"}