Skill_Seekers Documentation

File: README.md Copy Chapter

Skill Seekers

English | 简体中文 | 日本語 | 한국어 | Español | Français | Deutsch | Português | Türkçe | العربية | हिन्दी | Русский

🧠 The data layer for AI systems. Skill Seekers turns documentation sites, GitHub repos, PDFs, videos, notebooks, wikis, and more — 18 source types — into structured knowledge assets, ready to power AI Skills (Claude, Gemini, OpenAI), RAG pipelines (LangChain, LlamaIndex, Pinecone), and AI coding assistants (Cursor, Windsurf, Cline). Prepare once, export to 22 targets.

💛 Sponsors Copy Chapter

Launch Partner Copy Section

Atlas Cloud — A full-modal, OpenAI-compatible AI inference platform. Skill Seekers supports it as a packaging/enhancement target via --target atlas with ATLAS_API_KEY.

Silver Sponsors Copy Section

Become a sponsor · GitHub Sponsors

🚀 Quick Start Copy Chapter

# 1. Install
pip install skill-seekers

# 2. Create a skill from any source
skill-seekers create https://docs.djangoproject.com/

# 3. Package it for your AI platform
skill-seekers package output/django --target claude

You now have output/django-claude.zip, ready to use.

# Pick a different AI agent for enhancement (default: claude)
skill-seekers create https://docs.djangoproject.com/ --agent kimi
skill-seekers create https://docs.djangoproject.com/ --agent-cmd "my-custom-agent run"

🛰️ AI-driven project scan Copy Section

Point scan at a project and an AI agent reads its manifests, README, Dockerfile/CI and sampled source imports — then emits one config per detected framework, plus a -codebase.json for your own code:

skill-seekers scan ./my-react-app --out ./configs/scanned/
# → react.json, vite.json, tailwind.json, jest.json, my-react-app-codebase.json

skill-seekers create ./configs/scanned/react.json

If a detection has no existing preset, the AI generates a fresh config; on exit you can optionally publish it back to the community registry.

All 18 source types Copy Section

skill-seekers create facebook/react            # GitHub repository
skill-seekers create ./my-project              # Local codebase
skill-seekers create manual.pdf                # PDF
skill-seekers create report.docx               # Word
skill-seekers create book.epub                 # EPUB
skill-seekers create notebook.ipynb            # Jupyter
skill-seekers create openapi.yaml              # OpenAPI/Swagger
skill-seekers create presentation.pptx         # PowerPoint
skill-seekers create guide.adoc                # AsciiDoc
skill-seekers create page.html                 # Local HTML (or a whole dir)
skill-seekers create feed.rss                  # RSS/Atom
skill-seekers create curl.1                    # Man page

# Video (YouTube, Vimeo, or local — needs skill-seekers[video])
skill-seekers create --video-url https://www.youtube.com/watch?v=... --name mytutorial
skill-seekers create --setup                   # auto-install GPU-aware visual deps

skill-seekers create --space-key TEAM --name wiki               # Confluence
skill-seekers create --database-id ... --name docs              # Notion
skill-seekers create --chat-export-path ./slack-export --name team-chat  # Slack/Discord

See the Scraping Guide for every source type and its options.

📦 Installation Copy Chapter

pip install skill-seekers              # Core: scraping, GitHub, PDF, packaging
pip install skill-seekers[all-llms]    # + every LLM platform
pip install skill-seekers[mcp]         # + MCP server
pip install skill-seekers[all]         # Everything

Not sure what you need? Run the wizard: skill-seekers-setup

All installation extras

Video visual deps (GPU-aware): after installing skill-seekers[video-full], run skill-seekers create --setup to auto-detect your GPU and install the matching PyTorch variant + easyocr.

Prerequisites: Python 3.10+, Git. New here? → Bulletproof Quick Start 🎯

📚 Documentation Copy Chapter

Complete documentation index: docs/README.md

🎯 What you get Copy Chapter

Export targets (22) Copy Section

skill-seekers package output/react --target claude      # → Claude Skill (ZIP + YAML)
skill-seekers package output/react --target langchain   # → LangChain Documents
skill-seekers package output/react --target llama-index # → LlamaIndex TextNodes
skill-seekers package output/react --target ibm-bob     # → IBM Bob skill directory

LLM platforms (12): claude · gemini · openai · minimax · opencode · kimi · deepseek · qwen · openrouter · together · fireworks · markdown
RAG & vector (8): langchain · llama-index · haystack · chroma · faiss · weaviate · qdrant · pinecone
Other (2): atlas · ibm-bob

See the Feature Matrix for per-platform support details.

Why it matters Copy Section

• ⚡ 99% faster — days of manual data prep → 15–45 minutes
• 🎯 Real skill quality — 500+ line SKILL.md files with examples, patterns, and guides
• 📊 RAG-ready chunks — smart chunking preserves code blocks and context
• 🔄 Multi-source — combine docs + GitHub + PDFs + videos into one knowledge asset
• 🌐 One prep, every target — export to 22 targets without re-scraping
• ✅ Battle-tested — 3,900+ tests, 68 workflow presets, production-ready

✨ Key capabilities Copy Chapter

Documentation scraping — SPA discovery, llms.txt, smart categorization

Three-layer discovery for JavaScript SPA sites (sitemap.xml → llms.txt → headless browser rendering), automatic llms.txt detection (10× faster when present), smart topic categorization, and a lenient HTML parser fallback so broken markup still scrapes.

→ Scraping Guide · llms.txt Support

GitHub & codebase analysis (C3.x) — AST parsing, pattern detection, how-to guides

Three-stream architecture: code analysis (AST, design patterns, tests), documentation (README, docs/, wiki), and community (issues, PRs, metadata). The C3.x pipeline adds 10 GoF pattern detectors across 9 languages, usage examples extracted from tests, AI-written how-to guides, config extraction, and architecture overviews.

skill-seekers create ./my-project --preset quick          # 1–2 min, surface level
skill-seekers create ./my-project --preset standard       # balanced (default)
skill-seekers create ./my-project --preset comprehensive  # deep, exhaustive

→ Pattern Detection · How-To Guides · Test Example Extraction

AI enhancement — API or local agents, 68 workflow presets

Every AI call runs through one transport, in API mode (Anthropic, Google Gemini, OpenAI, Moonshot/Kimi, MiniMax) or LOCAL mode (Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents — no API costs). Control depth with --enhance-level 0-3 and pick an agent with --agent.

→ Enhancement Guide · Enhancement Modes · Multi-Agent Setup

Unified multi-source scraping — combine many sources into one skill

One config can pull documentation, GitHub, PDFs, videos, and more into a single knowledge asset, with conflict detection and pairwise synthesis across sources.

→ Unified Scraping

Video extraction — transcripts, frames, on-screen code

YouTube, Vimeo, and local files. Three-tier transcript fallback (subtitles → YouTube transcript API → local Whisper), plus optional visual extraction that OCRs on-screen code from sampled frames.

→ Video Guide

Quality, sync & scale

Quality scoring with a gate (skill-seekers quality output/react/ --threshold 7), provisional English readability metrics (informational — they never affect the score), doc-change detection with scheduled re-scrapes and notifications, streaming ingestion for very large doc sets, and incremental updates.

→ Large Documentation · Code Quality

🔌 MCP Integration (40 tools) Copy Chapter

Skill Seekers ships an MCP server for Claude Code, Cursor, Windsurf, VS Code + Cline, and IntelliJ IDEA.

# stdio mode (Claude Code, VS Code + Cline)
python -m skill_seekers.mcp.server_fastmcp

# HTTP mode (Cursor, Windsurf, IntelliJ)
python -m skill_seekers.mcp.server_fastmcp --transport http --port 8765

Then just ask your assistant: "Package and upload the React skill."

→ MCP Setup · MCP Reference · HTTP Transport

🤖 Installing to AI agents Copy Chapter

Skills install automatically into 19 AI coding agents:

skill-seekers install-agent output/react/ --agent cursor
skill-seekers install-agent output/react/ --agent all      # every detected agent
skill-seekers install-agent output/react/ --agent cursor --dry-run

Uploading to Claude Copy Section

export ANTHROPIC_API_KEY=sk-ant-...
skill-seekers package output/react/ --upload   # package + upload
skill-seekers upload output/react.zip          # upload an existing zip

No API key? Package it and upload output/react.zip manually at claude.ai/skills.

→ Upload Guide

⚙️ How it works Copy Chapter

graph LR
    A[Documentation Website] --> B[Skill Seekers]
    B --> C[Scraper]
    B --> D[AI Enhancement]
    B --> E[Packager]
    C --> F[Organized References]
    D --> F
    F --> E
    E --> G[AI Skill .zip]
    G --> H[Upload to AI Platform]

1. Scrape — extract every page (checking llms.txt first)
2. Categorize — organize content into topics (API, guides, tutorials, …)
3. Enhance — AI writes a comprehensive SKILL.md with examples
4. Package — bundle into a platform-ready artifact
5. Upload — ship it to your AI platform (optional)

Architecture Copy Section

8 core modules + 5 utility modules (~200 classes):

→ UML Architecture · API Reference · Skill Architecture

🆕 New in v3.9.0 Copy Chapter

• HTML parser fallback for broken markup (#96) — severely malformed pages no longer scrape as empty; well-formed pages are byte-identical.
• Transient-failure retries — the doc scraper (#97) and MCP fetch_config (#92) now retry connection blips and 5xx with backoff; 4xx still fails fast.
• Whisper transcription fallback (#420) — local videos without subtitles finally get a real transcript.
• MiniMax image OCR + registry-driven multimodal providers (#423) — providers declare their wire protocol and image capability; China-issued keys work against the right endpoint.
• Token-lean GitHub issue defaults (#169) — GitHub skills no longer bundle full closed-issue history by default.
• Env-driven CORS across all three servers (#422, #424) — no more wildcard origins with credentials.

Full history: CHANGELOG.md

📈 Performance Copy Chapter

🐛 Troubleshooting Copy Chapter

skill-seekers doctor          # diagnose installation & environment
skill-seekers sync-config     # detect config drift

Common issues and fixes: Troubleshooting Guide · TROUBLESHOOTING.md

🤝 Contributing Copy Chapter

Contributions are welcome — see CONTRIBUTING.md.

• 📋 Development Roadmap & Tasks — pick any task
• 💬 Discussions — questions and ideas
• 🐛 Issues — bugs and feature requests

📝 License Copy Chapter

MIT — see LICENSE.

🔒 Security Copy Chapter

🌐 Ecosystem Copy Chapter

Skill Seekers is a multi-repo project:

Want to contribute? The website and configs repos are great starting points for new contributors!

File: api/README.md Copy Chapter

Skill Seekers Config API

FastAPI backend for discovering and downloading Skill Seekers configuration files.

🚀 Endpoints Copy Chapter

Base URL Copy Section

• Production: https://skillseekersweb.com
• Local: http://localhost:8000

Available Endpoints Copy Section

1. **GET /** - API Information

Returns API metadata and available endpoints.

curl https://skillseekersweb.com/

Response:

{
  "name": "Skill Seekers Config API",
  "version": "1.0.0",
  "endpoints": {
    "/api/configs": "List all available configs",
    "/api/configs/{name}": "Get specific config details",
    "/api/categories": "List all categories",
    "/docs": "API documentation"
  },
  "repository": "https://github.com/yusufkaraaslan/Skill_Seekers",
  "website": "https://skillseekersweb.com"
}

2. **GET /api/configs** - List All Configs

Returns list of all available configs with metadata.

Query Parameters:

• category (optional) - Filter by category (e.g., web-frameworks)
• tag (optional) - Filter by tag (e.g., javascript)
• type (optional) - Filter by type (single-source or unified)

# Get all configs
curl https://skillseekersweb.com/api/configs

# Filter by category
curl https://skillseekersweb.com/api/configs?category=web-frameworks

# Filter by tag
curl https://skillseekersweb.com/api/configs?tag=javascript

# Filter by type
curl https://skillseekersweb.com/api/configs?type=unified

Response:

{
  "version": "1.0.0",
  "total": 24,
  "filters": null,
  "configs": [
    {
      "name": "react",
      "description": "React framework for building user interfaces...",
      "type": "single-source",
      "category": "web-frameworks",
      "tags": ["javascript", "frontend", "documentation"],
      "primary_source": "https://react.dev/",
      "max_pages": 300,
      "file_size": 1055,
      "last_updated": "2025-11-30T09:26:07+00:00",
      "download_url": "https://skillseekersweb.com/api/download/react.json",
      "config_file": "react.json"
    }
  ]
}

3. **GET /api/configs/{name}** - Get Specific Config

Returns detailed information about a specific config.

curl https://skillseekersweb.com/api/configs/react

Response:

{
  "name": "react",
  "description": "React framework for building user interfaces...",
  "type": "single-source",
  "category": "web-frameworks",
  "tags": ["javascript", "frontend", "documentation"],
  "primary_source": "https://react.dev/",
  "max_pages": 300,
  "file_size": 1055,
  "last_updated": "2025-11-30T09:26:07+00:00",
  "download_url": "https://skillseekersweb.com/api/download/react.json",
  "config_file": "react.json"
}

4. **GET /api/categories** - List Categories

Returns all available categories with config counts.

curl https://skillseekersweb.com/api/categories

Response:

{
  "total_categories": 5,
  "categories": {
    "web-frameworks": 7,
    "game-engines": 2,
    "devops": 2,
    "css-frameworks": 1,
    "uncategorized": 12
  }
}

5. **GET /api/download/{config_name}** - Download Config File

Downloads the actual config JSON file.

# Download react config
curl -O https://skillseekersweb.com/api/download/react.json

# Download with just name (auto-adds .json)
curl -O https://skillseekersweb.com/api/download/react

6. **GET /health** - Health Check

Health check endpoint for monitoring.

curl https://skillseekersweb.com/health

Response:

{
  "status": "healthy",
  "service": "skill-seekers-api"
}

7. **GET /docs** - API Documentation

Interactive OpenAPI documentation (Swagger UI).

Visit: https://skillseekersweb.com/docs

📦 Metadata Fields Copy Chapter

Each config includes the following metadata:

🏗️ Categories Copy Chapter

Configs are auto-categorized into:

• web-frameworks - Web development frameworks (React, Django, FastAPI, etc.)
• game-engines - Game development engines (Godot, Unity, etc.)
• devops - DevOps tools (Kubernetes, Ansible, etc.)
• css-frameworks - CSS frameworks (Tailwind, etc.)
• development-tools - Dev tools (Claude Code, etc.)
• gaming - Gaming platforms (Steam, etc.)
• uncategorized - Other configs

🏷️ Tags Copy Chapter

Common tags include:

• Language: javascript, python, php
• Domain: frontend, backend, devops, game-development
• Type: documentation, github, pdf, multi-source
• Tech: css, testing, api

🚀 Local Development Copy Chapter

Setup Copy Section

# Install dependencies
cd api
pip install -r requirements.txt

# Run server
python main.py

API will be available at http://localhost:8000

Testing Copy Section

# Test health check
curl http://localhost:8000/health

# List all configs
curl http://localhost:8000/api/configs

# Get specific config
curl http://localhost:8000/api/configs/react

# Download config
curl -O http://localhost:8000/api/download/react.json

📝 Deployment Copy Chapter

Render Copy Section

This API is configured for Render deployment via render.yaml.

1. Push to GitHub
2. Connect repository to Render
3. Render auto-deploys from render.yaml
4. Configure custom domain: skillseekersweb.com

🔗 Links Copy Chapter

• API Documentation: https://skillseekersweb.com/docs
• GitHub Repository: https://github.com/yusufkaraaslan/Skill_Seekers
• Main Project: https://github.com/yusufkaraaslan/Skill_Seekers#readme

File: distribution/smithery/README.md Copy Chapter

Skill Seekers — Smithery MCP Registry

Publishing guide for the Skill Seekers MCP server on Smithery.

Status Copy Chapter

• Namespace created: yusufkaraaslan
• Server created: yusufkaraaslan/skill-seekers
• Server page: https://smithery.ai/servers/yusufkaraaslan/skill-seekers
• Release status: Needs re-publish (initial release failed — Smithery couldn't scan GitHub URL as MCP endpoint)

Publishing Copy Chapter

Smithery requires a live, scannable MCP HTTP endpoint for URL-based publishing. Two options:

Option A: Publish via Web UI (Recommended) Copy Section

1. Go to https://smithery.ai/servers/yusufkaraaslan/skill-seekers/releases
2. The server already exists — create a new release
3. For the "Local" tab: follow the prompts to publish as a stdio server
4. For the "URL" tab: provide a hosted HTTP endpoint URL

Option B: Deploy HTTP endpoint first, then publish via CLI Copy Section

1. Deploy the MCP server on Render/Railway/Fly.io:
   bash Copy

   # Using existing Dockerfile.mcp
   docker build -f Dockerfile.mcp -t skill-seekers-mcp .
   # Deploy to your hosting provider

2. Publish the live URL:
   bash Copy

   npx @smithery/cli@latest auth login
   npx @smithery/cli@latest mcp publish "https://your-deployed-url/mcp" \
     -n yusufkaraaslan/skill-seekers

CLI Authentication (already done) Copy Section

# Install via npx (no global install needed)
npx @smithery/cli@latest auth login
npx @smithery/cli@latest namespace show   # Should show: yusufkaraaslan

After Publishing Copy Section

Update the server page with metadata:

Display name: Skill Seekers — AI Skill & RAG Toolkit

Description:

Transform 18 source types into AI-ready skills and RAG knowledge. Ingest documentation sites, GitHub repos, PDFs, Jupyter notebooks, videos, Confluence, Notion, Slack/Discord exports, and more. Package for 21+ LLM platforms including Claude, GPT, Gemini, LangChain, LlamaIndex, and vector databases.

Tags: ai, rag, documentation, skills, preprocessing, mcp, knowledge-base, vector-database

User Installation Copy Chapter

Once published, users can add the server to their MCP client:

# Via Smithery CLI (adds to Claude Desktop, Cursor, etc.)
smithery mcp add yusufkaraaslan/skill-seekers --client claude

# Or configure manually — users need skill-seekers installed:
pip install skill-seekers[mcp]

Manual MCP Configuration Copy Section

For clients that use JSON config (Claude Desktop, Claude Code, Cursor):

{
  "mcpServers": {
    "skill-seekers": {
      "command": "python",
      "args": ["-m", "skill_seekers.mcp.server_fastmcp"]
    }
  }
}

Available Tools (40) Copy Chapter

Maintenance Copy Chapter

• Update description/tags on major releases
• No code changes needed — users always get the latest via pip install

Notes Copy Chapter

• Smithery CLI v4.7.0 removed the --transport stdio flag from the docs
• The CLI publish command only supports URL-based (external) publishing
• For local/stdio servers, use the web UI at smithery.ai/servers/new
• The namespace and server entity are already created; only the release needs to succeed

File: distribution/github-action/README.md Copy Chapter

Skill Seekers GitHub Action

Transform documentation, GitHub repos, PDFs, videos, and 13 other source types into AI-ready skills and RAG knowledge — directly in your CI/CD pipeline.

Quick Start Copy Chapter

- uses: yusufkaraaslan/skill-seekers-action@v3
  with:
    source: 'https://react.dev'

Inputs Copy Chapter

Outputs Copy Chapter

Examples Copy Chapter

Auto-update documentation skill weekly Copy Section

name: Update AI Skills
on:
  schedule:
    - cron: '0 6 * * 1'  # Every Monday 6am UTC
  workflow_dispatch:

jobs:
  update-skills:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: yusufkaraaslan/skill-seekers-action@v3
        with:
          source: 'https://react.dev'
          target: 'langchain'

      - uses: actions/upload-artifact@v4
        with:
          name: react-skill
          path: output/

Generate skill from GitHub repo Copy Section

- uses: yusufkaraaslan/skill-seekers-action@v3
  with:
    source: 'pallets/flask'
    command: 'create'
    target: 'claude'

Process PDF documentation Copy Section

- uses: actions/checkout@v4

- uses: yusufkaraaslan/skill-seekers-action@v3
  with:
    source: 'docs/api-reference.pdf'
    command: 'create'

Unified multi-source build with config Copy Section

- uses: actions/checkout@v4

- uses: yusufkaraaslan/skill-seekers-action@v3
  with:
    config: 'configs/my-project.json'
    command: 'create'
    target: 'openai'

Commit generated skill back to repo Copy Section

- uses: actions/checkout@v4

- uses: yusufkaraaslan/skill-seekers-action@v3
  id: generate
  with:
    source: 'https://fastapi.tiangolo.com'

- name: Commit skill
  run: |
    git config user.name "github-actions[bot]"
    git config user.email "github-actions[bot]@users.noreply.github.com"
    git add output/
    git diff --staged --quiet || git commit -m "Update AI skill: ${{ steps.generate.outputs.skill-name }}"
    git push

Environment Variables Copy Chapter

Pass API keys as environment variables for AI-enhanced skills:

env:
  ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Supported Source Types Copy Chapter

License Copy Chapter

MIT

File: distribution/claude-plugin/README.md Copy Chapter

Skill Seekers — Claude Code Plugin

Transform 18 source types into AI-ready skills and RAG knowledge, directly from Claude Code.

Installation Copy Chapter

From the Official Plugin Directory Copy Section

/plugin install skill-seekers@claude-plugin-directory

Or browse for it in /plugin > Discover.

Local Installation (for development) Copy Section

claude --plugin-dir ./path/to/skill-seekers-plugin

Prerequisites Copy Section

The plugin requires skill-seekers to be installed:

pip install skill-seekers[mcp]

What's Included Copy Chapter

MCP Server (40 tools) Copy Section

The plugin bundles the Skill Seekers MCP server providing tools for:

• Scraping documentation, GitHub repos, PDFs, videos, and 13 other source types
• Packaging skills for 21+ LLM platforms
• Exporting to vector databases (Weaviate, Chroma, FAISS, Qdrant)
• Managing configs, workflows, and sources

Slash Commands Copy Section

Agent Skill Copy Section

The skill-builder skill is automatically available to Claude. It detects source types and uses the appropriate MCP tools to build skills autonomously.

Usage Examples Copy Chapter

# Create a skill from a documentation site
/skill-seekers:create-skill https://react.dev

# Create from a GitHub repo, targeting LangChain
/skill-seekers:create-skill pallets/flask --target langchain

# Full install workflow with AI enhancement
/skill-seekers:install-skill https://fastapi.tiangolo.com --enhance

# Sync an existing config
/skill-seekers:sync-config react

Or just ask Claude naturally:

"Create an AI skill from the React documentation"
"Scrape the Flask GitHub repo and package it for OpenAI"
"Export my skill to a Chroma vector database"

The skill-builder agent skill will automatically detect the intent and use the right tools.

Remote MCP Alternative Copy Chapter

By default, the plugin runs the MCP server locally via python -m skill_seekers.mcp.server_fastmcp. To use a remote server instead, edit .mcp.json:

{
  "skill-seekers": {
    "type": "http",
    "url": "https://your-hosted-server.com/mcp"
  }
}

Supported Source Types Copy Chapter

Documentation (web), GitHub repos, PDFs, Word docs, EPUBs, videos, local codebases, Jupyter notebooks, HTML files, OpenAPI specs, AsciiDoc, PowerPoint, RSS/Atom feeds, man pages, Confluence, Notion, Slack/Discord exports.

License Copy Chapter

MIT — https://github.com/yusufkaraaslan/Skill_Seekers

File: docs/agents/research/2026-03-14-epub-input-support-affected-files.md Copy Chapter

date: 2026-03-14T12:54:24.700367+00:00 git_commit: 7c90a4b9c9bccac8341b0769550d77aae3b4e524 branch: development topic: "What files would be affected to add .epub support for input" tags: [research, codebase, epub, input-format, scraper] status: complete Copy Chapter

🗄️ Historical — completed — This research informed the EPUB implementation that is now live in Skill Seekers v3.6.0.

Research: What files would be affected to add .epub support for input

Research Question Copy Chapter

What files would be affected to add .epub support for input.

Summary Copy Chapter

Adding .epub input support follows an established pattern already used for PDF and Word (.docx) formats. The codebase has a consistent multi-layer architecture for document input formats: source detection, argument definitions, parser registration, create command routing, standalone scraper module, and tests. Based on analysis of the existing PDF and Word implementations, 16 existing files would need modification and 4 new files would need to be created.

Detailed Findings Copy Chapter

New Files to Create (4 files) Copy Section

Existing Files to Modify (16 files) Copy Section

1. Source Detection Layer

src/skill_seekers/cli/source_detector.py (3 locations)

• SourceDetector.detect() (line ~60): Add .epub extension check, following the .docx pattern at line 63-64:

  python Copy

  if source.endswith(".epub"):
      return cls._detect_epub(source)

• New method _detect_epub(): Add detection method (following _detect_word() at lines 124-129):

  python Copy

  @classmethod
  def _detect_epub(cls, source: str) -> SourceInfo:
      name = os.path.splitext(os.path.basename(source))[0]
      return SourceInfo(
          type="epub", parsed={"file_path": source}, suggested_name=name, raw_input=source
      )

• validate_source() (line ~250): Add epub validation block (following the word block at lines 273-278)

• Error message (line ~94): Add EPUB example to the ValueError help text

2. CLI Dispatcher

src/skill_seekers/cli/main.py (2 locations)

• COMMAND_MODULES dict (line ~46): Add epub entry:

  python Copy

  "epub": "skill_seekers.cli.epub_scraper",

• Module docstring (line ~1): Add epub to the commands list

3. Create Command Routing

src/skill_seekers/cli/create_command.py (3 locations)

• _route_to_scraper() (line ~121): Add elif self.source_info.type == "epub": routing case

• New _route_epub() method: Following the _route_word() pattern at lines 331-352:

  python Copy

  def _route_epub(self) -> int:
      from skill_seekers.cli import epub_scraper
      argv = ["epub_scraper"]
      file_path = self.source_info.parsed["file_path"]
      argv.extend(["--epub", file_path])
      self._add_common_args(argv)
      # epub-specific args here
      ...

• main() epilog (line ~537): Add EPUB example and source auto-detection entry

• Progressive help (line ~590): Add --help-epub flag and handler block

4. Argument Definitions

src/skill_seekers/cli/arguments/create.py (4 locations)

• New EPUB_ARGUMENTS dict (~line 401): Define epub-specific arguments (e.g., --epub file path flag), following the WORD_ARGUMENTS pattern at lines 402-411

• get_source_specific_arguments() (line 595): Add "epub": EPUB_ARGUMENTS to the source_args dict

• add_create_arguments() (line 676): Add epub mode block:

  python Copy

  if mode in ["epub", "all"]:
      for arg_name, arg_def in EPUB_ARGUMENTS.items():
          parser.add_argument(*arg_def["flags"], **arg_def["kwargs"])

5. Parser Registration

src/skill_seekers/cli/parsers/__init__.py (2 locations)

• Import (line ~15): Add from .epub_parser import EpubParser

• PARSERS list (line ~46): Add EpubParser() entry (near WordParser() and PDFParser())

6. Package Configuration

pyproject.toml (3 locations)

• [project.optional-dependencies] (line ~111): Add epub optional dependency group:

  toml Copy

  epub = [
      "ebooklib>=0.18",
  ]

• all optional dependency group (line ~178): Add epub dependency to the combined all group

• [project.scripts] (line ~224): Add standalone entry point:

  toml Copy

  skill-seekers-epub = "skill_seekers.cli.epub_scraper:main"

7. Argument Commons

src/skill_seekers/cli/arguments/common.py

• No changes strictly required, but add_all_standard_arguments() is called by the new arguments/epub.py (no modification needed — it's used as-is)

8. Documentation / Configuration

CLAUDE.md (2 locations)

• Commands section: Add epub to the list of subcommands
• Key source files table: Add epub_scraper.py entry

CONTRIBUTING.md — Potentially update with epub format mention

CHANGELOG.md — New feature entry

Files NOT Affected Copy Section

These files do not need changes:

• unified_scraper.py — Multi-source configs could add epub support later but it's not required for basic input support
• Platform adaptors (adaptors/*.py) — Adaptors work on the output side (packaging), not input
• Enhancement system (enhance_skill.py, enhance_skill_local.py) — Works generically on SKILL.md
• MCP server (mcp/server_fastmcp.py) — Operates on completed skills
• pdf_extractor_poc.py — PDF-specific extraction; epub needs its own extractor

Code References Copy Chapter

Pattern to Follow (Word .docx implementation) Copy Section

• src/skill_seekers/cli/word_scraper.py:1-750 — Full scraper with WordToSkillConverter class
• src/skill_seekers/cli/arguments/word.py:1-75 — Argument definitions with add_word_arguments()
• src/skill_seekers/cli/parsers/word_parser.py:1-33 — Parser class extending SubcommandParser
• tests/test_word_scraper.py:1-750 — Comprehensive test suite with 130+ tests

Key Integration Points Copy Section

• src/skill_seekers/cli/source_detector.py:57-65 — File extension detection order
• src/skill_seekers/cli/source_detector.py:124-129 — _detect_word() method (template for _detect_epub())
• src/skill_seekers/cli/create_command.py:121-143 — _route_to_scraper() dispatch
• src/skill_seekers/cli/create_command.py:331-352 — _route_word() (template for _route_epub())
• src/skill_seekers/cli/arguments/create.py:401-411 — WORD_ARGUMENTS dict (template)
• src/skill_seekers/cli/arguments/create.py:595-604 — get_source_specific_arguments() mapping
• src/skill_seekers/cli/arguments/create.py:676-678 — add_create_arguments() mode handling
• src/skill_seekers/cli/parsers/__init__.py:35-59 — PARSERS registry list
• src/skill_seekers/cli/main.py:46-70 — COMMAND_MODULES dict
• pyproject.toml:111-115 — Optional dependency group pattern (docx)
• pyproject.toml:213-246 — Script entry points

Data Flow Architecture Copy Section

The epub scraper would follow the same three-step pipeline as Word/PDF:

1. Extract — Parse .epub file → sections with text, headings, code, images → save to output/{name}_extracted.json
2. Categorize — Group sections by chapters/keywords
3. Build — Generate SKILL.md, references/*.md, references/index.md, assets/

The intermediate JSON format uses the same structure as Word/PDF:

{
    "source_file": str,
    "metadata": {"title", "author", "created", ...},
    "total_sections": int,
    "total_code_blocks": int,
    "total_images": int,
    "languages_detected": {str: int},
    "pages": [  # sections
        {
            "section_number": int,
            "heading": str,
            "text": str,
            "code_samples": [...],
            "images": [...],
            "headings": [...]
        }
    ]
}

Architecture Documentation Copy Chapter

Document Input Format Pattern Copy Section

Each input format follows a consistent architecture:

[source_detector.py] → detect type by extension
        ↓
[create_command.py] → route to scraper
        ↓
[{format}_scraper.py] → extract → categorize → build skill
        ↓
[output/{name}/] → SKILL.md + references/ + assets/

Supporting files per format:

• arguments/{format}.py — CLI argument definitions
• parsers/{format}_parser.py — Subcommand parser class
• tests/test_{format}_scraper.py — Test suite

Dependency Guard Pattern Copy Section

The Word scraper uses an optional dependency guard that epub should replicate:

try:
    import ebooklib
    from ebooklib import epub
    EPUB_AVAILABLE = True
except ImportError:
    EPUB_AVAILABLE = False

def _check_epub_deps():
    if not EPUB_AVAILABLE:
        raise RuntimeError(
            "ebooklib is required for EPUB support.\n"
            'Install with: pip install "skill-seekers[epub]"\n'
            "Or: pip install ebooklib"
        )

Summary Table Copy Chapter

Open Questions Copy Chapter

• Should epub support reuse any of the existing HTML parsing from word_scraper.py (which uses mammoth to convert to HTML then parses with BeautifulSoup)? EPUB internally contains XHTML files, so BeautifulSoup parsing would be directly applicable.
• Should the epub scraper support DRM-protected files, or only DRM-free epub files?
• Should epub-specific arguments include options like --chapter-range (similar to PDF's --pages)?

File: docs/agents/plans/2026-03-14-epub-input-support.md Copy Chapter

date: 2026-03-14T19:30:35.172407+00:00 git_commit: 7c90a4b9c9bccac8341b0769550d77aae3b4e524 branch: development topic: "Add EPUB Input Support" tags: [plan, epub, scraper, input-format] status: complete Copy Chapter

🗄️ Historical — completed — This plan was fully executed. EPUB input support is live in Skill Seekers v3.6.0 (epub_scraper.py, arguments/epub.py, parsers/epub_parser.py, 107+ tests).

Add EPUB Input Support — Implementation Plan

Overview Copy Chapter

Add .epub as an input format for Skill Seekers, enabling skill-seekers create book.epub and skill-seekers epub --epub book.epub. Follows the established Word/PDF scraper pattern: source detection → routing → extraction → categorize → build skill.

Authoritative reference: W3C EPUB 3.3 Specification (also covers EPUB 2 backward compatibility).

Current State Analysis Copy Chapter

The codebase has a consistent multi-layer architecture for document input formats. PDF and Word (.docx) serve as direct analogs. The Word scraper (word_scraper.py) is the closest pattern match since both Word and EPUB produce HTML/XHTML that is parsed with BeautifulSoup.

Key Discoveries: Copy Section

• Word scraper converts .docx → HTML (via mammoth) → BeautifulSoup parse → intermediate JSON → SKILL.md (word_scraper.py:96-235)
• EPUB files contain XHTML natively (per W3C spec §5), so the mammoth conversion step is unnecessary — BeautifulSoup can parse EPUB XHTML content directly
• Source detection uses file extension matching (source_detector.py:57-65)
• Optional dependencies use a guard pattern with try/except ImportError and a _check_*_deps() function (word_scraper.py:21-40)
• The ebooklib library (v0.18+) provides epub.read_epub() returning an EpubBook with spine iteration, metadata access via get_metadata('DC', key), and item content via get_content()/get_body_content()
• ebooklib has a known bug: EPUB 3 files read TOC from NCX instead of NAV (issue #200); workaround: options={"ignore_ncx": True}
• ebooklib loads entire EPUB into memory — acceptable for typical books but relevant for edge cases

Desired End State Copy Chapter

Running skill-seekers create book.epub produces:

output/book/
├── SKILL.md              # Main skill file with metadata, concepts, code examples
├── references/
│   ├── index.md          # Category index with statistics
│   └── book.md           # Chapter content (or multiple files if categorized)
├── scripts/
└── assets/
    └── *.png|*.jpg       # Extracted images

CLI Output Mockup Copy Section

$ skill-seekers create programming-rust.epub

ℹ️  Detected source type: epub
ℹ️  Routing to epub scraper...

🔍 Extracting from EPUB: programming-rust.epub
   Title: Programming Rust, 2nd Edition
   Author: Jim Blandy, Jason Orendorff
   Language: en
   Chapters: 23 (spine items)

📄 Processing chapters...
   Chapter 1/23: Why Rust? (2 sections, 1 code block)
   Chapter 2/23: A Tour of Rust (5 sections, 12 code blocks)
   ...
   Chapter 23/23: Macros (4 sections, 8 code blocks)

📊 Extraction complete:
   Sections: 142
   Code blocks: 287 (Rust: 245, Shell: 28, TOML: 14)
   Images: 34
   Tables: 12

💾 Saved extracted data to: output/programming-rust_extracted.json

📋 Categorizing content...
✅ Created 1 category (single EPUB source)
   - programming-rust: 142 sections

📝 Generating reference files...
   Generated: output/programming-rust/references/programming-rust.md
   Generated: output/programming-rust/references/index.md

✅ Skill built successfully: output/programming-rust/

📦 Next step: Package with: skill-seekers package output/programming-rust/

Verification: Copy Section

• skill-seekers create book.epub produces valid output directory
• skill-seekers create book.epub --name mybook works standalone
• skill-seekers create book.epub --dry-run shows config without processing
• All ~2,540+ existing tests still pass (982 passed, 1 pre-existing failure)
• New test suite has 100+ tests covering happy path, errors, and edge cases (107 tests, 14 classes)

What We're NOT Doing Copy Chapter

• DRM decryption (detect and error gracefully with clear message)
• EPUB writing/creation (read-only)
• Media overlay / audio / video extraction (ignore gracefully)
• Fixed-layout OCR (detect and warn; extract whatever text exists in XHTML)
• --chapter-range flag (can be added later)
• Unified scraper (unified_scraper.py) EPUB support (separate future task)
• MCP tool for EPUB (separate future task)

Implementation Approach Copy Chapter

Follow the Word scraper pattern exactly, with EPUB-specific extraction logic:

1. Phase 1: Core epub_scraper.py — the EpubToSkillConverter class
2. Phase 2: CLI integration — source detection, arguments, parser, routing, entry points
3. Phase 3: Comprehensive test suite — 100+ tests across 11 test classes
4. Phase 4: Documentation updates

Phase 1: Core EPUB Scraper Copy Chapter

Overview Copy Section

Create epub_scraper.py with EpubToSkillConverter class following the Word scraper pattern. This is the bulk of new code.

Changes Required: Copy Section

[x] 1. Optional dependency in pyproject.toml

File: pyproject.toml
Changes: Add epub optional dependency group and include in all group

# After the docx group (~line 115)
# EPUB (.epub) support
epub = [
    "ebooklib>=0.18",
]

Add "ebooklib>=0.18", to the all group (~line 178).

[x] 2. Create `src/skill_seekers/cli/epub_scraper.py`

File: src/skill_seekers/cli/epub_scraper.py (new)
Changes: Full EPUB scraper module

Structure (following word_scraper.py pattern):

"""
EPUB Documentation to Skill Converter

Converts EPUB e-books into skills.
Uses ebooklib for EPUB parsing, BeautifulSoup for XHTML content extraction.

Usage:
    skill-seekers create book.epub --name myskill
    skill-seekers create book_extracted.json
"""

import argparse
import json
import logging
import os
import re
import sys
from pathlib import Path

# Optional dependency guard
try:
    import ebooklib
    from ebooklib import epub
    EPUB_AVAILABLE = True
except ImportError:
    EPUB_AVAILABLE = False

# BeautifulSoup is a core dependency (always available)
from bs4 import BeautifulSoup, Comment

logger = logging.getLogger(__name__)


def _check_epub_deps():
    """Raise RuntimeError if ebooklib is not installed."""
    if not EPUB_AVAILABLE:
        raise RuntimeError(
            "ebooklib is required for EPUB support.\n"
            'Install with: pip install "skill-seekers[epub]"\n'
            "Or: pip install ebooklib"
        )


def infer_description_from_epub(metadata: dict | None = None, name: str = "") -> str:
    """Infer skill description from EPUB metadata."""
    if metadata:
        if metadata.get("description") and len(metadata["description"]) > 20:
            desc = metadata["description"].strip()
            if len(desc) > 150:
                desc = desc[:147] + "..."
            return f"Use when {desc.lower()}"
        if metadata.get("title") and len(metadata["title"]) > 10:
            return f"Use when working with {metadata['title'].lower()}"
    return (
        f"Use when referencing {name} documentation"
        if name
        else "Use when referencing this documentation"
    )

EpubToSkillConverter class methods:

class EpubToSkillConverter:
    def __init__(self, config: dict):
        self.config = config
        self.name = config["name"]
        self.epub_path = config.get("epub_path", "")
        self.description = config.get(
            "description", f"Use when referencing {self.name} documentation"
        )
        self.skill_dir = f"output/{self.name}"
        self.data_file = f"output/{self.name}_extracted.json"
        self.categories = config.get("categories", {})
        self.extracted_data = None

    def extract_epub(self) -> bool:
        """Extract content from EPUB file.

        Workflow:
        1. Check dependencies (ebooklib)
        2. Detect DRM via META-INF/encryption.xml (fail fast)
        3. Read EPUB via ebooklib with ignore_ncx=True (EPUB 3 TOC bug workaround)
        4. Extract Dublin Core metadata (title, creator, language, publisher, date, description, subject)
        5. Iterate spine items in reading order
        6. For each ITEM_DOCUMENT: parse XHTML with BeautifulSoup
        7. Split by h1/h2 heading boundaries into sections
        8. Extract code blocks from /<code> elements
        9. Extract images from EpubImage items
        10. Detect code languages via LanguageDetector
        11. Save intermediate JSON to {name}_extracted.json

        Returns True on success.
        Raises RuntimeError for DRM-protected files.
        Raises FileNotFoundError for missing files.
        Raises ValueError for invalid EPUB files.
        """

DRM detection (per W3C spec §4.2.6.3.2):

def _detect_drm(self, book) -> bool:
    """Detect DRM by checking for encryption.xml with non-font-obfuscation entries.

    Per W3C EPUB 3.3 spec: encryption.xml is present when resources are encrypted.
    Font obfuscation (IDPF algorithm http://www.idpf.org/2008/embedding or
    Adobe algorithm http://ns.adobe.com/pdf/enc#RC) is NOT DRM — only font mangling.

    Actual DRM uses algorithms like:
    - Adobe ADEPT: http://ns.adobe.com/adept namespace
    - Apple FairPlay: http://itunes.apple.com/dataenc
    - Readium LCP: http://readium.org/2014/01/lcp
    """

Metadata extraction (per W3C spec §5.2, Dublin Core):

def _extract_metadata(self, book) -> dict:
    """Extract Dublin Core metadata from EPUB.

    Per W3C EPUB 3.3 spec: required elements are dc:identifier, dc:title, dc:language.
    Optional: dc:creator, dc:contributor, dc:date, dc:description, dc:publisher,
    dc:subject, dc:rights, dc:type, dc:coverage, dc:source, dc:relation, dc:format.

    ebooklib API: book.get_metadata('DC', key) returns list of (value, attrs) tuples.
    """
    def _get_one(key):
        data = book.get_metadata('DC', key)
        return data[0][0] if data else None

    def _get_list(key):
        data = book.get_metadata('DC', key)
        return [x[0] for x in data] if data else []

    return {
        "title": _get_one('title') or "Untitled",
        "author": ", ".join(_get_list('creator')) or None,
        "language": _get_one('language') or "en",
        "publisher": _get_one('publisher'),
        "date": _get_one('date'),
        "description": _get_one('description'),
        "subject": ", ".join(_get_list('subject')) or None,
        "rights": _get_one('rights'),
        "identifier": _get_one('identifier'),
    }

Content extraction (per W3C spec §5 — XHTML Content Documents use XML serialization of HTML5):

def _extract_spine_content(self, book) -> list[dict]:
    """Extract content from spine items in reading order.

    Per W3C EPUB 3.3 spec §3.4.8: spine defines ordered list of content documents.
    Linear="yes" (default) items form the primary reading order.
    Linear="no" items are auxiliary (footnotes, glossary).

    Per spec §5: XHTML content documents use XML syntax of HTML5.
    Parse with BeautifulSoup, split by h1/h2 heading boundaries.
    """
    sections = []
    section_number = 0

    for item_id, linear in book.spine:
        item = book.get_item_with_id(item_id)
        if not item or item.get_type() != ebooklib.ITEM_DOCUMENT:
            continue

        soup = BeautifulSoup(item.get_content(), 'html.parser')

        # Remove scripts, styles, comments (not useful for text extraction)
        for tag in soup(['script', 'style']):
            tag.decompose()
        for comment in soup.find_all(string=lambda t: isinstance(t, Comment)):
            comment.extract()

        body = soup.find('body')
        if not body:
            continue

        # Split by h1/h2 heading boundaries (same as word_scraper)
        # Each heading starts a new section
        ...

Image extraction (per W3C spec §3.3 — core media types include JPEG, PNG, GIF, WebP, SVG):

def _extract_images(self, book) -> list[dict]:
    """Extract images from EPUB manifest.

    Per W3C EPUB 3.3 spec §3.3: core image media types are
    image/gif, image/jpeg, image/png, image/svg+xml, image/webp.

    ebooklib API: book.get_items_of_type(ebooklib.ITEM_IMAGE)
    returns EpubImage items with get_content() (bytes) and media_type.

    SVG images (ITEM_VECTOR) handled separately.
    """

The remaining methods (categorize_content, build_skill, _generate_reference_file, _generate_index, _generate_skill_md, _format_key_concepts, _format_patterns_from_content, _sanitize_filename) follow the Word scraper pattern exactly — they operate on the same intermediate JSON structure.

main() function (following word_scraper.py:923-1059):

/* Detailed source-code truncated for AI context efficiency. */

Success Criteria: Copy Section

Automated Verification:

• ruff check src/skill_seekers/cli/epub_scraper.py passes
• ruff format --check src/skill_seekers/cli/epub_scraper.py passes
• mypy src/skill_seekers/cli/epub_scraper.py passes (continue-on-error)
• pip install -e ".[epub]" installs successfully

Manual Verification:

• Verify import ebooklib works after install
• Review epub_scraper.py structure matches word_scraper.py pattern

Implementation Note: After completing this phase and all automated verification passes, pause here for manual confirmation from the human before proceeding to the next phase.

Phase 2: CLI Integration Copy Chapter

Overview Copy Section

Wire the EPUB scraper into the CLI: source detection, argument definitions, parser registration, create command routing, and entry points.

Changes Required: Copy Section

[x] 1. Source detection

File: src/skill_seekers/cli/source_detector.py
Changes: Add .epub extension detection, _detect_epub() method, validation, and error message update

Add after the .docx check (line 64):

if source.endswith(".epub"):
    return cls._detect_epub(source)

Add _detect_epub() method (following _detect_word() at line 124):

@classmethod
def _detect_epub(cls, source: str) -> SourceInfo:
    """Detect EPUB file source."""
    name = os.path.splitext(os.path.basename(source))[0]
    return SourceInfo(
        type="epub", parsed={"file_path": source}, suggested_name=name, raw_input=source
    )

Add epub validation in validate_source() (after word block at line 278):

elif source_info.type == "epub":
    file_path = source_info.parsed["file_path"]
    if not os.path.exists(file_path):
        raise ValueError(f"EPUB file does not exist: {file_path}")
    if not os.path.isfile(file_path):
        raise ValueError(f"Path is not a file: {file_path}")

Add EPUB example to the ValueError message (line 94):

"  EPUB:  skill-seekers create ebook.epub\n"

[x] 2. Argument definitions

File: src/skill_seekers/cli/arguments/epub.py (new)
Changes: EPUB-specific argument definitions

"""EPUB-specific CLI arguments."""

import argparse
from typing import Any

from .common import add_all_standard_arguments

EPUB_ARGUMENTS: dict[str, dict[str, Any]] = {
    "epub": {
        "flags": ("--epub",),
        "kwargs": {
            "type": str,
            "help": "Direct EPUB file path",
            "metavar": "PATH",
        },
    },
    "from_json": {
        "flags": ("--from-json",),
        "kwargs": {
            "type": str,
            "help": "Build skill from extracted JSON",
            "metavar": "FILE",
        },
    },
}


def add_epub_arguments(parser: argparse.ArgumentParser) -> None:
    """Add EPUB-specific arguments to parser."""
    add_all_standard_arguments(parser)

    # Override enhance-level default to 0 for EPUB
    for action in parser._actions:
        if hasattr(action, "dest") and action.dest == "enhance_level":
            action.default = 0
            action.help = (
                "AI enhancement level (auto-detects API vs LOCAL mode): "
                "0=disabled (default for EPUB), 1=SKILL.md only, "
                "2=+architecture/config, 3=full enhancement. "
                "Mode selection: uses API if ANTHROPIC_API_KEY is set, "
                "otherwise LOCAL (Claude Code)"
            )

    for arg_name, arg_def in EPUB_ARGUMENTS.items():
        parser.add_argument(*arg_def["flags"], **arg_def["kwargs"])

[x] 3. Create command argument integration

File: src/skill_seekers/cli/arguments/create.py
Changes: Add EPUB_ARGUMENTS dict, register in helper functions, add mode handling

Add after WORD_ARGUMENTS (~line 411):

# EPUB specific (from epub.py)
EPUB_ARGUMENTS: dict[str, dict[str, Any]] = {
    "epub": {
        "flags": ("--epub",),
        "kwargs": {
            "type": str,
            "help": "EPUB file path",
            "metavar": "PATH",
        },
    },
}

Add to get_source_specific_arguments() (line 595):

"epub": EPUB_ARGUMENTS,

Add to add_create_arguments() (after word block at line 678):

if mode in ["epub", "all"]:
    for arg_name, arg_def in EPUB_ARGUMENTS.items():
        parser.add_argument(*arg_def["flags"], **arg_def["kwargs"])

[x] 4. Parser class

File: src/skill_seekers/cli/parsers/epub_parser.py (new)
Changes: Subcommand parser for standalone skill-seekers epub command

"""Parser for epub subcommand."""

from .base import SubcommandParser
from skill_seekers.cli.arguments.epub import add_epub_arguments


class EpubParser(SubcommandParser):
    """Parser for EPUB extraction command."""

    @property
    def name(self) -> str:
        return "epub"

    @property
    def help(self) -> str:
        return "Extract from EPUB e-book (.epub)"

    @property
    def description(self) -> str:
        return "Extract content from EPUB e-book (.epub) and generate skill"

    def add_arguments(self, parser):
        add_epub_arguments(parser)

[x] 5. Parser registration

File: src/skill_seekers/cli/parsers/__init__.py
Changes: Import and register EpubParser

Add import (after WordParser import, line 15):

from .epub_parser import EpubParser

Add to PARSERS list (after WordParser(), line 46):

EpubParser(),

[x] 6. CLI dispatcher

File: src/skill_seekers/cli/main.py
Changes: Add epub to COMMAND_MODULES dict and module docstring

Add to COMMAND_MODULES (after "word" entry, line 52):

"epub": "skill_seekers.cli.epub_scraper",

Add to module docstring (after "word" line, line 15):

#    epub                 Extract from EPUB e-book (.epub)

[x] 7. Create command routing

File: src/skill_seekers/cli/create_command.py
Changes: Add _route_epub() method, routing case, help flag, and epilog example

Add to _route_to_scraper() (after word case, line 136):

elif self.source_info.type == "epub":
    return self._route_epub()

Add _route_epub() method (after _route_word(), line 352):

def _route_epub(self) -> int:
    """Route to EPUB scraper (epub_scraper.py)."""
    from skill_seekers.cli import epub_scraper

    argv = ["epub_scraper"]
    file_path = self.source_info.parsed["file_path"]
    argv.extend(["--epub", file_path])
    self._add_common_args(argv)

    logger.debug(f"Calling epub_scraper with argv: {argv}")
    original_argv = sys.argv
    try:
        sys.argv = argv
        return epub_scraper.main()
    finally:
        sys.argv = original_argv

Add to epilog (line 543, after DOCX example):

EPUB:     skill-seekers create ebook.epub

Add to Source Auto-Detection section:

• file.epub → EPUB extraction

Add --help-epub flag and handler (after --help-word at line 592):

parser.add_argument(
    "--help-epub", action="store_true", help=argparse.SUPPRESS, dest="_help_epub"
)

Add handler block (after _help_word block at line 654):

elif args._help_epub:
    parser_epub = argparse.ArgumentParser(
        prog="skill-seekers create",
        description="Create skill from EPUB e-book (.epub)",
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    add_create_arguments(parser_epub, mode="epub")
    parser_epub.print_help()
    return 0

[x] 8. Entry point

File: pyproject.toml
Changes: Add standalone entry point

Add after skill-seekers-word (line 224):

skill-seekers-epub = "skill_seekers.cli.epub_scraper:main"

[x] 9. Positional argument handling in main.py

File: src/skill_seekers/cli/main.py
Changes: Add "input_file" is already in the positional list at line 153, so no change needed. Verify _reconstruct_argv handles epub correctly through the standard delegation path.

Success Criteria: Copy Section

Automated Verification:

• ruff check src/skill_seekers/cli/source_detector.py src/skill_seekers/cli/arguments/epub.py src/skill_seekers/cli/parsers/epub_parser.py src/skill_seekers/cli/create_command.py passes
• ruff format --check src/skill_seekers/cli/ passes
• pip install -e ".[epub]" installs with all entry points
• skill-seekers create --help shows EPUB-specific help
• skill-seekers create --help-epub shows EPUB arguments (via standalone entry point skill-seekers-create)
• skill-seekers create nonexistent.epub gives clear error about missing file
• Existing tests still pass: pytest tests/ -v -x -m "not slow and not integration" (875 passed, 1 pre-existing unrelated failure in test_git_sources_e2e)

Manual Verification:

• skill-seekers --help lists epub command
• skill-seekers create book.epub --dry-run shows dry run output

Implementation Note: After completing this phase and all automated verification passes, pause here for manual confirmation from the human before proceeding to the next phase.

Phase 3: Comprehensive Test Suite Copy Chapter

Overview Copy Section

Create tests/test_epub_scraper.py with 100+ tests across 11 test classes, covering happy path, negative cases, edge cases, and CLI integration.

Changes Required: Copy Section

[x] 1. Create test file

File: tests/test_epub_scraper.py (new)
Changes: Comprehensive test suite following test_word_scraper.py patterns

"""
Tests for EPUB scraper (epub_scraper.py).

Covers: initialization, extraction, categorization, skill building,
code blocks, tables, images, error handling, JSON workflow, CLI arguments,
helper functions, source detection, DRM detection, and edge cases.

Tests use mock data and do not require actual EPUB files or ebooklib installed.
"""

import json
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest.mock import MagicMock, patch, PropertyMock


# Conditional import (same pattern as test_word_scraper.py)
try:
    from skill_seekers.cli.epub_scraper import (
        EpubToSkillConverter,
        infer_description_from_epub,
        _score_code_quality,
        _check_epub_deps,
        EPUB_AVAILABLE,
    )
    IMPORT_OK = True
except ImportError:
    IMPORT_OK = False

Helper factory function:

/* Detailed source-code truncated for AI context efficiency. */

Test Classes and Methods: Copy Section

[x] Class 1: `TestEpubToSkillConverterInit` (8 tests)

Happy path:

• test_init_with_name_and_epub_path — basic config with name + epub_path
• test_init_with_full_config — config with all fields (name, epub_path, description, categories)
• test_default_description_uses_name — description defaults to "Use when referencing {name} documentation"
• test_skill_dir_uses_name — skill_dir is output/{name}
• test_data_file_uses_name — data_file is output/{name}_extracted.json

Negative:

• test_init_requires_name — missing "name" key raises KeyError
• test_init_empty_name — empty string name still works (no crash)

Edge case:

• test_init_with_special_characters_in_name — name with spaces/dashes sanitized for paths

[x] Class 2: `TestEpubExtraction` (12 tests)

Happy path:

• test_extract_basic_epub — mock ebooklib, verify sections extracted in spine order
• test_extract_metadata — verify Dublin Core metadata extraction (title, creator, language, etc.)
• test_extract_multiple_chapters — multiple spine items produce multiple sections
• test_extract_code_blocks — <code> elements extracted with language detection
• test_extract_images — ITEM_IMAGE items extracted with correct content
• test_heading_boundary_splitting — h1/h2 boundaries create new sections

Negative:

• test_extract_missing_file_raises_error — FileNotFoundError for nonexistent path
• test_extract_invalid_epub_raises_error — ValueError for corrupted/non-EPUB file
• test_extract_deps_not_installed — RuntimeError with install instructions when ebooklib missing

Edge cases:

• test_extract_empty_spine — EPUB with no spine items produces empty sections list
• test_extract_spine_item_no_body — XHTML without `` tag skipped gracefully
• test_extract_non_linear_spine_items — linear="no" items still extracted (included but flagged)

[x] Class 3: `TestEpubDrmDetection` (6 tests)

Happy path:

• test_no_drm_detected — normal EPUB without encryption.xml returns False

Negative:

• test_drm_detected_adobe_adept — encryption.xml with Adobe namespace raises RuntimeError
• test_drm_detected_apple_fairplay — encryption.xml with Apple namespace raises RuntimeError
• test_drm_detected_readium_lcp — encryption.xml with Readium namespace raises RuntimeError

Edge cases:

• test_font_obfuscation_not_drm — encryption.xml with only IDPF font obfuscation algorithm (http://www.idpf.org/2008/embedding) is NOT DRM, extraction proceeds
• test_drm_error_message_is_clear — error message mentions DRM and suggests removing protection

[x] Class 4: `TestEpubCategorization` (8 tests)

Happy path:

• test_single_source_creates_one_category — single EPUB creates category named after file
• test_keyword_categorization — sections matched to categories by keyword scoring
• test_no_categories_uses_default — no category config creates single "content" category

Negative:

• test_categorize_empty_sections — empty sections list produces empty categories
• test_categorize_no_keyword_matches — unmatched sections go to "other" category

Edge cases:

• test_categorize_single_section — one section creates one category
• test_categorize_many_sections — 50+ sections categorized correctly
• test_categorize_preserves_section_order — sections maintain original order within categories

[x] Class 5: `TestEpubSkillBuilding` (10 tests)

Happy path:

• test_build_creates_directory_structure — output/{name}/, references/, scripts/, assets/ created
• test_build_generates_skill_md — SKILL.md created with YAML frontmatter
• test_build_generates_reference_files — reference markdown files created per category
• test_build_generates_index — references/index.md created with category links
• test_skill_md_contains_metadata — SKILL.md includes title, author, language from metadata
• test_skill_md_yaml_frontmatter — frontmatter has name and description fields

Negative:

• test_build_without_extracted_data_fails — calling build_skill() before extraction raises error

Edge cases:

• test_build_overwrites_existing_output — re-running build overwrites existing files
• test_build_with_long_name — name > 64 chars truncated in YAML frontmatter
• test_build_with_unicode_content — Unicode text (CJK, Arabic, emoji) preserved correctly

[x] Class 6: `TestEpubCodeBlocks` (8 tests)

Happy path:

• test_code_blocks_included_in_reference_files — code samples appear in reference markdown
• test_code_blocks_in_skill_md_top_15 — SKILL.md shows top 15 code examples by quality
• test_code_language_grouped — code examples grouped by language in SKILL.md

Edge cases:

• test_empty_code_block — ```
  /* Detailed source-code truncated for AI context efficiency. */

skill-seekers create book.epub --name myskill

Add to "Unified create" examples:

skill-seekers create book.epub

Add to Key source files table:

| Core scraping | `cli/epub_scraper.py` |

Add to "Adding things → New create command flags" section:

- Source-specific → `EPUB_ARGUMENTS`

[x] 2. Update CHANGELOG.md

File: CHANGELOG.md
Changes: Add entry for EPUB support under next version

### Added
- EPUB (.epub) input support via `skill-seekers create book.epub` or `skill-seekers epub --epub book.epub`
- Extracts chapters, metadata, code blocks, images, and tables from EPUB 2 and EPUB 3 files
- DRM detection with clear error messages
- Optional dependency: `pip install "skill-seekers[epub]"`

Success Criteria: Copy Section

Automated Verification:

• ruff check passes on any modified files
• pytest tests/ -v -x -m "not slow and not integration" — all tests still pass (982 passed, 1 pre-existing failure)

Manual Verification:

• CLAUDE.md accurately reflects new commands
• CHANGELOG.md entry is clear and complete

Implementation Note: After completing this phase and all automated verification passes, pause here for manual confirmation from the human before proceeding.

Testing Strategy Copy Chapter

Unit Tests (Phase 3 — ~108 tests): Copy Section

By category:

Integration Tests: Copy Section

• Full extract → categorize → build workflow with mock ebooklib
• JSON round-trip (extract → save → load → build)

Manual Testing Steps: Copy Section

1. pip install -e ".[epub]" — verify install
2. skill-seekers create book.epub with a real EPUB file — verify output directory structure
3. skill-seekers create book.epub --dry-run — verify dry run output
4. skill-seekers create drm-book.epub — verify DRM error message
5. skill-seekers create nonexistent.epub — verify file-not-found error
6. Open generated SKILL.md — verify content quality and structure

Performance Considerations Copy Chapter

• ebooklib loads entire EPUB into memory. For typical books (<50MB), this is fine
• For very large EPUBs (100MB+), memory usage may spike. No mitigation needed for v1 — document as known limitation
• BeautifulSoup parsing of XHTML is fast. No performance concerns expected

Migration Notes Copy Chapter

• No migration needed — this is a new feature with no existing data to migrate
• Optional dependency (ebooklib) means existing installs are unaffected
• No breaking changes to any existing commands or APIs

References Copy Chapter

• W3C EPUB 3.3 Specification — authoritative source of truth
• W3C EPUB Reading Systems 3.3 — reading system requirements
• ebooklib GitHub — Python EPUB library
• ebooklib PyPI — v0.20, Python 3.9-3.13
• Research document — affected files analysis
• Similar implementation: src/skill_seekers/cli/word_scraper.py — closest analog
• Similar tests: tests/test_word_scraper.py — test pattern template

File: docs/advanced/custom-workflows.md Copy Chapter

Custom Workflows Guide

Skill Seekers v3.9.0
Create custom AI enhancement workflows

What are Custom Workflows? Copy Chapter

Workflows are YAML-defined, multi-stage AI enhancement pipelines:

my-workflow.yaml
├── name
├── description
├── variables (optional)
└── stages (1-10)
    ├── name
    ├── type (builtin/custom)
    ├── target (skill_md/references/)
    ├── prompt
    └── uses_history (optional)

Basic Workflow Structure Copy Chapter

name: my-custom
stages:
  - name: stage-one
    type: builtin
    target: skill_md
    prompt: |
      Improve the SKILL.md by adding...
      
  - name: stage-two
    type: custom
    target: references
    prompt: |
      Enhance the references by...

Workflow Fields Copy Chapter

Top Level Copy Section

Stage Fields Copy Section

Creating Your First Workflow Copy Chapter

Example: Performance Analysis Copy Section

# performance.yaml
name: performance-focus
variables:
  target_latency: "100ms"
  target_throughput: "1000 req/s"

stages:
  - name: performance-overview
    type: builtin
    target: skill_md
    prompt: |
      Add a "Performance" section to SKILL.md covering:
      - Benchmark results
      - Performance characteristics
      - Resource requirements
      
  - name: optimization-guide
    type: custom
    target: references
    uses_history: true
    prompt: |
      Create an optimization guide with:
      - Target latency: {target_latency}
      - Target throughput: {target_throughput}
      - Common bottlenecks
      - Optimization techniques

Install and Use Copy Section

# Add workflow
skill-seekers workflows add performance.yaml

# Use it
skill-seekers create  --enhance-workflow performance-focus

# With custom variables
skill-seekers create  \
  --enhance-workflow performance-focus \
  --var target_latency=50ms \
  --var target_throughput=5000req/s

Stage Types Copy Chapter

builtin Copy Section

Uses built-in enhancement logic:

stages:
  - name: structure-improvement
    type: builtin
    target: skill_md
    prompt: "Improve document structure"

custom Copy Section

Full custom prompt control:

stages:
  - name: custom-analysis
    type: custom
    target: skill_md
    prompt: |
      Your detailed custom prompt here...
      Can use {variables} and {history}

Targets Copy Chapter

skill_md Copy Section

Enhances the main SKILL.md file:

stages:
  - name: improve-skill
    target: skill_md
    prompt: "Add comprehensive overview section"

references Copy Section

Enhances reference files:

stages:
  - name: improve-refs
    target: references
    prompt: "Add cross-references between files"

Variables Copy Chapter

Defining Variables Copy Section

variables:
  audience: "beginners"
  focus_area: "security"
  include_examples: true

Using Variables Copy Section

stages:
  - name: customize
    prompt: |
      Tailor content for {audience}.
      Focus on {focus_area}.
      Include examples: {include_examples}

Overriding at Runtime Copy Section

skill-seekers create  \
  --enhance-workflow my-workflow \
  --var audience=experts \
  --var focus_area=performance

History Passing Copy Chapter

Access results from previous stages:

stages:
  - name: analyze
    type: custom
    target: skill_md
    prompt: "Analyze security features"
    
  - name: document
    type: custom
    target: skill_md
    uses_history: true
    prompt: |
      Based on previous analysis:
      {previous_results}
      
      Create documentation...

Advanced Example: Security Review Copy Chapter

name: comprehensive-security
variables:
  compliance_framework: "OWASP Top 10"
  risk_level: "high"

stages:
  - name: asset-inventory
    type: builtin
    target: skill_md
    prompt: |
      Document all security-sensitive components:
      - Authentication mechanisms
      - Authorization checks
      - Data validation
      - Encryption usage
      
  - name: threat-analysis
    type: custom
    target: skill_md
    uses_history: true
    prompt: |
      Based on assets: {all_history}
      
      Analyze threats for {compliance_framework}:
      - Threat vectors
      - Attack scenarios
      - Risk ratings ({risk_level} focus)
      
  - name: mitigation-guide
    type: custom
    target: references
    uses_history: true
    prompt: |
      Create mitigation guide:
      - Countermeasures
      - Best practices
      - Code examples
      - Testing strategies

Validation Copy Chapter

Validate Before Installing Copy Section

skill-seekers workflows validate ./my-workflow.yaml

Common Errors Copy Section

Best Practices Copy Chapter

1. Start Simple Copy Section

# Start with 1-2 stages
name: simple
stages:
  - name: improve
    type: builtin
    target: skill_md
    prompt: "Improve SKILL.md"

2. Use Clear Stage Names Copy Section

# Good
stages:
  - name: security-overview
  - name: vulnerability-analysis
  
# Bad
stages:
  - name: stage1
  - name: step2

3. Document Variables Copy Section

variables:
  # Target audience level: beginner, intermediate, expert
  audience: "intermediate"
  
  # Security focus area: owasp, pci, hipaa
  compliance: "owasp"

4. Test Incrementally Copy Section

# Test with dry run
skill-seekers create  \
  --enhance-workflow my-workflow \
  --workflow-dry-run

# Then actually run
skill-seekers create  \
  --enhance-workflow my-workflow

5. Chain for Complex Analysis Copy Section

# Use multiple workflows
skill-seekers create  \
  --enhance-workflow security-focus \
  --enhance-workflow performance-focus

Sharing Workflows Copy Chapter

Export Workflow Copy Section

# Get workflow content
skill-seekers workflows show my-workflow > my-workflow.yaml

Share with Team Copy Section

# Add to version control
git add my-workflow.yaml
git commit -m "Add custom security workflow"

# Team members install
skill-seekers workflows add my-workflow.yaml

Publish Copy Section

Submit to Skill Seekers community:

• GitHub Discussions
• Skill Seekers website
• Documentation contributions

See Also Copy Chapter

• Workflows Guide - Using workflows
• MCP Reference - Workflows via MCP
• Enhancement Guide - Enhancement fundamentals

File: docs/advanced/mcp-server.md Copy Chapter

MCP Server Setup Guide

Skill Seekers v3.9.0
Integrate with AI agents via Model Context Protocol

What is MCP? Copy Chapter

MCP (Model Context Protocol) lets AI agents like Claude Code control Skill Seekers through natural language:

You: "Scrape the React documentation"
Claude: ▶️ scrape_docs({"url": "https://react.dev/"})
        ✅ Done! Created output/react/

Installation Copy Chapter

# Install with MCP support
pip install skill-seekers[mcp]

# Verify
skill-seekers-mcp --version

Transport Modes Copy Chapter

stdio Mode (Default) Copy Section

For Claude Code, VS Code + Cline:

skill-seekers-mcp

Use when:

• Running in Claude Code
• Direct integration with terminal-based agents
• Simple local setup

HTTP Mode Copy Section

For Cursor, Windsurf, HTTP clients:

# Start HTTP server
skill-seekers-mcp --transport http --port 8765

# Custom host
skill-seekers-mcp --transport http --host 0.0.0.0 --port 8765

Use when:

• IDE integration (Cursor, Windsurf)
• Remote access needed
• Multiple clients

Claude Code Integration Copy Chapter

Automatic Setup Copy Section

# In Claude Code, run:
/claude add-mcp-server skill-seekers

Or manually add to ~/.claude/mcp.json:

{
  "mcpServers": {
    "skill-seekers": {
      "command": "skill-seekers-mcp",
      "env": {
        "ANTHROPIC_API_KEY": "sk-ant-...",
        "GITHUB_TOKEN": "ghp_..."
      }
    }
  }
}

Usage Copy Section

Once connected, ask Claude:

"List available configs"
"Scrape the Django documentation"
"Package output/react for Gemini"
"Enhance output/my-skill with security-focus workflow"

Cursor IDE Integration Copy Chapter

Setup Copy Section

1. Start MCP server:

skill-seekers-mcp --transport http --port 8765

2. In Cursor Settings → MCP:
   • Name: skill-seekers
   • URL: http://localhost:8765

Usage Copy Section

In Cursor chat:

"Create a skill from the current project"
"Analyze this codebase and generate a cursorrules file"

Windsurf Integration Copy Chapter

Setup Copy Section

1. Start MCP server:

skill-seekers-mcp --transport http --port 8765

2. In Windsurf Settings:
   • Add MCP server endpoint: http://localhost:8765

Available Tools Copy Chapter

40 tools organized by category:

Core Tools (9) Copy Section

• list_configs - List presets
• generate_config - Create config from URL
• validate_config - Check config
• estimate_pages - Page estimation
• scrape_docs - Scrape documentation
• package_skill - Package skill
• upload_skill - Upload to platform
• enhance_skill - AI enhancement
• install_skill - Complete workflow

Extended Tools (10) Copy Section

• scrape_github - GitHub repo
• scrape_pdf - PDF extraction
• scrape_generic - Generic scraper for 10 new source types (see below)
• scrape_codebase - Local code
• unified_scrape - Multi-source
• detect_patterns - Pattern detection
• extract_test_examples - Test examples
• build_how_to_guides - How-to guides
• extract_config_patterns - Config patterns
• detect_conflicts - Doc/code conflicts

Config Sources (5) Copy Section

• add_config_source - Register git source
• list_config_sources - List sources
• remove_config_source - Remove source
• fetch_config - Fetch configs
• submit_config - Submit configs

Vector DB (4) Copy Section

• export_to_weaviate
• export_to_chroma
• export_to_faiss
• export_to_qdrant

scrape_generic Tool Copy Section

The scrape_generic tool is the generic entry point for 10 new source types added in v3.2.0. It delegates to the appropriate CLI scraper module.

Supported source types: jupyter, html, openapi, asciidoc, pptx, rss, manpage, confluence, notion, chat

Parameters:

Usage examples:

"Scrape the Jupyter notebook analysis.ipynb"
→ scrape_generic(source_type="jupyter", name="analysis", path="analysis.ipynb")

"Extract content from the API spec"
→ scrape_generic(source_type="openapi", name="my-api", path="api-spec.yaml")

"Process the PowerPoint slides"
→ scrape_generic(source_type="pptx", name="slides", path="presentation.pptx")

"Scrape the Confluence wiki"
→ scrape_generic(source_type="confluence", name="wiki", url="https://wiki.example.com")

See MCP Reference for full details.

Common Workflows Copy Chapter

Workflow 1: Documentation Skill Copy Section

User: "Create a skill from React docs"
Claude: ▶️ scrape_docs({"url": "https://react.dev/"})
        ⏳ Scraping...
        ✅ Created output/react/
        
        ▶️ package_skill({"skill_directory": "output/react/", "target": "claude"})
        ✅ Created output/react-claude.zip
        
        Skill ready! Upload to Claude?

Workflow 2: GitHub Analysis Copy Section

User: "Analyze the facebook/react repo"
Claude: ▶️ scrape_github({"repo": "facebook/react"})
        ⏳ Analyzing...
        ✅ Created output/react/
        
        ▶️ enhance_skill({"skill_directory": "output/react/", "workflow": "architecture-comprehensive"})
        ✅ Enhanced with architecture analysis

Workflow 3: Multi-Platform Export Copy Section

User: "Create Django skill for all platforms"
Claude: ▶️ scrape_docs({"config": "django"})
        ✅ Created output/django/
        
        ▶️ package_skill({"skill_directory": "output/django/", "target": "claude"})
        ▶️ package_skill({"skill_directory": "output/django/", "target": "gemini"})
        ▶️ package_skill({"skill_directory": "output/django/", "target": "openai"})
        ✅ Created packages for all platforms

Configuration Copy Chapter

Environment Variables Copy Section

Set in ~/.claude/mcp.json or before starting server:

export ANTHROPIC_API_KEY=sk-ant-...
export GOOGLE_API_KEY=AIza...
export OPENAI_API_KEY=sk-...
export GITHUB_TOKEN=ghp_...

Server Options Copy Section

# Debug mode
skill-seekers-mcp --verbose

# Custom port
skill-seekers-mcp --port 8080

# Allow all origins (CORS)
skill-seekers-mcp --cors

Security Copy Chapter

Local Only (stdio) Copy Section

# Only accessible by local Claude Code
skill-seekers-mcp

HTTP with Auth Copy Section

# Use reverse proxy with auth
# nginx, traefik, etc.

API Key Protection Copy Section

# Don't hardcode keys
# Use environment variables
# Or secret management

Troubleshooting Copy Chapter

"Server not found" Copy Section

# Check if running
curl http://localhost:8765/health

# Restart
skill-seekers-mcp --transport http --port 8765

"Tool not available" Copy Section

# Check version
skill-seekers-mcp --version

# Update
pip install --upgrade skill-seekers[mcp]

"Connection refused" Copy Section

# Check port
lsof -i :8765

# Use different port
skill-seekers-mcp --port 8766

See Also Copy Chapter

• MCP Reference - Complete tool reference
• MCP Tools Deep Dive - Advanced usage
• MCP Protocol - Official MCP docs

File: docs/advanced/multi-source.md Copy Chapter

Multi-Source Scraping Guide

Skill Seekers v3.9.0
Combine 18 source types into one unified skill

What is Multi-Source Scraping? Copy Chapter

Combine multiple sources into a single, comprehensive skill. Skill Seekers supports 18 source types that can be freely mixed and matched:

┌──────────────┐
│ Documentation│──┐
│ (Web docs)   │  │
├──────────────┤  │
│ GitHub Repo  │  │
│ (Source code) │  │
├──────────────┤  │     ┌──────────────────┐
│ PDF / Word / │  │     │  Unified Skill   │
│ EPUB / PPTX  │──┼────▶│  (Single source  │
├──────────────┤  │     │   of truth)      │
│ Video /      │  │     └──────────────────┘
│ Jupyter / HTML│  │
├──────────────┤  │
│ OpenAPI /    │  │
│ AsciiDoc /   │  │
│ RSS / Man    │  │
├──────────────┤  │
│ Confluence / │──┘
│ Notion / Chat│
└──────────────┘

When to Use Multi-Source Copy Chapter

Use Cases Copy Section

Benefits Copy Section

• Single source of truth - One skill with all context
• Conflict detection - Find doc/code discrepancies
• Cross-references - Link between sources
• Comprehensive - No gaps in knowledge

Creating Unified Configs Copy Chapter

Basic Structure Copy Section

{
  "name": "my-framework-complete",
  "description": "Complete documentation and code",
  "merge_mode": "claude-enhanced",
  
  "sources": [
    {
      "type": "docs",
      "name": "documentation",
      "base_url": "https://docs.example.com/"
    },
    {
      "type": "github",
      "name": "source-code",
      "repo": "owner/repo"
    }
  ]
}

Source Types (17 Supported) Copy Chapter

1. Documentation (Web) Copy Section

{
  "type": "docs",
  "name": "official-docs",
  "base_url": "https://docs.framework.com/",
  "max_pages": 500,
  "categories": {
    "getting_started": ["intro", "quickstart"],
    "api": ["reference", "api"]
  }
}

2. GitHub Repository Copy Section

{
  "type": "github",
  "name": "source-code",
  "repo": "facebook/react",
  "fetch_issues": true,
  "max_issues": 100,
  "enable_codebase_analysis": true
}

3. PDF Document Copy Section

{
  "type": "pdf",
  "name": "legacy-manual",
  "pdf_path": "docs/legacy-manual.pdf",
  "enable_ocr": false
}

4. Local Codebase Copy Section

{
  "type": "local",
  "name": "internal-tools",
  "directory": "./internal-lib",
  "languages": ["Python", "JavaScript"]
}

5. Word Document (.docx) Copy Section

{
  "type": "word",
  "name": "product-spec",
  "path": "docs/specification.docx"
}

6. Video (YouTube/Vimeo/Local) Copy Section

{
  "type": "video",
  "name": "tutorial-video",
  "url": "https://www.youtube.com/watch?v=example",
  "language": "en"
}

7. EPUB Copy Section

{
  "type": "epub",
  "name": "programming-book",
  "path": "books/python-guide.epub"
}

8. Jupyter Notebook Copy Section

{
  "type": "jupyter",
  "name": "analysis-notebooks",
  "path": "notebooks/data-analysis.ipynb"
}

9. Local HTML Copy Section

{
  "type": "html",
  "name": "exported-docs",
  "path": "exports/documentation.html"
}

10. OpenAPI/Swagger Copy Section

{
  "type": "openapi",
  "name": "api-spec",
  "path": "specs/openapi.yaml"
}

11. AsciiDoc Copy Section

{
  "type": "asciidoc",
  "name": "technical-docs",
  "path": "docs/manual.adoc"
}

12. PowerPoint (.pptx) Copy Section

{
  "type": "pptx",
  "name": "architecture-deck",
  "path": "presentations/architecture.pptx"
}

13. RSS/Atom Feed Copy Section

{
  "type": "rss",
  "name": "release-feed",
  "url": "https://blog.example.com/releases.xml"
}

14. Man Pages Copy Section

{
  "type": "manpage",
  "name": "cli-reference",
  "path": "man/mytool.1"
}

15. Confluence Copy Section

{
  "type": "confluence",
  "name": "team-wiki",
  "base_url": "https://company.atlassian.net/wiki",
  "space_key": "ENGINEERING"
}

16. Notion Copy Section

{
  "type": "notion",
  "name": "project-docs",
  "workspace": "my-workspace",
  "root_page_id": "abc123def456"
}

17. Slack/Discord (Chat) Copy Section

{
  "type": "chat",
  "name": "team-discussions",
  "path": "exports/slack-export/"
}

Complete Example Copy Chapter

React Complete Skill Copy Section

{
  "name": "react-complete",
  "description": "React - docs, source, and guides",
  "merge_mode": "claude-enhanced",
  
  "sources": [
    {
      "type": "docs",
      "name": "react-docs",
      "base_url": "https://react.dev/",
      "max_pages": 300,
      "categories": {
        "getting_started": ["learn", "tutorial"],
        "api": ["reference", "hooks"],
        "advanced": ["concurrent", "suspense"]
      }
    },
    {
      "type": "github",
      "name": "react-source",
      "repo": "facebook/react",
      "fetch_issues": true,
      "max_issues": 50,
      "enable_codebase_analysis": true,
      "code_analysis_depth": "deep"
    },
    {
      "type": "pdf",
      "name": "react-patterns",
      "pdf_path": "downloads/react-patterns.pdf"
    }
  ],
  
  "conflict_detection": {
    "enabled": true,
    "rules": [
      {
        "field": "api_signature",
        "action": "flag_mismatch"
      },
      {
        "field": "version",
        "action": "warn_outdated"
      }
    ]
  },
  
  "output_structure": {
    "group_by_source": false,
    "cross_reference": true
  }
}

Running Unified Scraping Copy Chapter

Basic Command Copy Section

skill-seekers create --config react-complete.json

With Options Copy Section

# Fresh start (ignore cache)
skill-seekers create --config react-complete.json --fresh

# Dry run
skill-seekers create --config react-complete.json --dry-run

# Rule-based merging
skill-seekers create --config react-complete.json --merge-mode rule-based

Merge Modes Copy Chapter

claude-enhanced (Default) Copy Section

Uses AI to intelligently merge sources:

• Detects relationships between content
• Resolves conflicts intelligently
• Creates cross-references
• Best quality, slower

skill-seekers create --config my-config.json --merge-mode claude-enhanced

rule-based Copy Section

Uses defined rules for merging:

• Faster
• Deterministic
• Less sophisticated

skill-seekers create --config my-config.json --merge-mode rule-based

Generic Merge System Copy Section

When combining source types beyond the standard docs+github+pdf trio, the generic merge system (_generic_merge() in unified_skill_builder.py) handles any combination automatically. It uses pairwise synthesis for known combos (docs+github, docs+pdf, github+pdf) and falls back to a generic merging strategy for all other source type combinations.

AI-Powered Multi-Source Merging Copy Section

For complex multi-source projects, use the complex-merge.yaml workflow preset to apply AI-powered merging:

skill-seekers create --config my-config.json \
  --enhance-workflow complex-merge

This workflow uses Claude to intelligently reconcile content from disparate source types, resolving conflicts and creating coherent cross-references between sources that would otherwise be difficult to merge deterministically.

Conflict Detection Copy Chapter

Automatic Detection Copy Section

Finds discrepancies between sources:

{
  "conflict_detection": {
    "enabled": true,
    "rules": [
      {
        "field": "api_signature",
        "action": "flag_mismatch"
      },
      {
        "field": "version",
        "action": "warn_outdated"
      },
      {
        "field": "deprecation",
        "action": "highlight"
      }
    ]
  }
}

Conflict Report Copy Section

After scraping, check for conflicts:

# Conflicts are reported in output
ls output/react-complete/conflicts.json

# Or use MCP tool
detect_conflicts({
  "docs_source": "output/react-docs",
  "code_source": "output/react-source"
})

Output Structure Copy Chapter

Merged Output Copy Section

output/react-complete/
├── SKILL.md                    # Combined skill
├── references/
│   ├── index.md               # Master index
│   ├── getting_started.md     # From docs
│   ├── api_reference.md       # From docs
│   ├── source_overview.md     # From GitHub
│   ├── code_examples.md       # From GitHub
│   └── patterns.md            # From PDF
├── .skill-seekers/
│   ├── manifest.json          # Metadata
│   ├── sources.json           # Source list
│   └── conflicts.json         # Detected conflicts
└── cross-references.json      # Links between sources

Best Practices Copy Chapter

1. Name Sources Clearly Copy Section

{
  "sources": [
    {"type": "docs", "name": "official-docs"},
    {"type": "github", "name": "source-code"},
    {"type": "pdf", "name": "legacy-reference"},
    {"type": "openapi", "name": "api-spec"},
    {"type": "confluence", "name": "team-wiki"}
  ]
}

2. Limit Source Scope Copy Section

{
  "type": "github",
  "name": "core-source",
  "repo": "owner/repo",
  "file_patterns": ["src/**/*.py"],  // Only core files
  "exclude_patterns": ["tests/**", "docs/**"]
}

3. Enable Conflict Detection Copy Section

{
  "conflict_detection": {
    "enabled": true
  }
}

4. Use Appropriate Merge Mode Copy Section

• claude-enhanced - Best quality, for important skills
• rule-based - Faster, for testing or large datasets

5. Test Incrementally Copy Section

# Test with one source first
skill-seekers create 

# Then add sources
skill-seekers create --config my-config.json --dry-run

Troubleshooting Copy Chapter

"Source not found" Copy Section

# Check all sources exist
curl -I https://docs.example.com/
ls downloads/manual.pdf

"Merge conflicts" Copy Section

# Check conflicts report
cat output/my-skill/conflicts.json

# Adjust merge_mode
skill-seekers create --config my-config.json --merge-mode rule-based

"Out of memory" Copy Section

# Process sources separately
# Then merge manually

Examples Copy Chapter

Framework + Examples Copy Section

{
  "name": "django-complete",
  "sources": [
    {"type": "docs", "base_url": "https://docs.djangoproject.com/"},
    {"type": "github", "repo": "django/django", "fetch_issues": false}
  ]
}

Docs + OpenAPI Spec Copy Section

{
  "name": "stripe-complete",
  "sources": [
    {"type": "docs", "base_url": "https://stripe.com/docs"},
    {"type": "openapi", "path": "specs/stripe-openapi.yaml"}
  ]
}

Code + Jupyter Notebooks Copy Section

{
  "name": "ml-project",
  "sources": [
    {"type": "github", "repo": "org/ml-pipeline"},
    {"type": "jupyter", "path": "notebooks/training.ipynb"},
    {"type": "jupyter", "path": "notebooks/evaluation.ipynb"}
  ]
}

Confluence + GitHub Copy Section

{
  "name": "internal-platform",
  "sources": [
    {"type": "confluence", "base_url": "https://company.atlassian.net/wiki", "space_key": "PLATFORM"},
    {"type": "github", "repo": "company/platform-core"},
    {"type": "openapi", "path": "specs/platform-api.yaml"}
  ]
}

Legacy + Current Copy Section

{
  "name": "product-docs",
  "sources": [
    {"type": "docs", "base_url": "https://docs.example.com/v2/"},
    {"type": "pdf", "pdf_path": "v1-legacy-manual.pdf"}
  ]
}

CLI Tool (Man Pages + GitHub + AsciiDoc) Copy Section

{
  "name": "mytool-complete",
  "sources": [
    {"type": "manpage", "path": "man/mytool.1"},
    {"type": "github", "repo": "org/mytool"},
    {"type": "asciidoc", "path": "docs/user-guide.adoc"}
  ]
}

Team Knowledge (Notion + Chat + Video) Copy Section

{
  "name": "onboarding-knowledge",
  "sources": [
    {"type": "notion", "workspace": "engineering", "root_page_id": "abc123"},
    {"type": "chat", "path": "exports/slack-engineering/"},
    {"type": "video", "url": "https://www.youtube.com/playlist?list=PLonboarding"}
  ]
}

See Also Copy Chapter

• Config Format - Full JSON specification
• Scraping Guide - Individual source options
• MCP Reference - scrape_docs tool (handles unified multi-source configs)