### README
# repowise documentation
Index your codebase once. Your agent stops greping, your team stops guessing
which PR is dangerous, and both get their answers from the same place.
New here? **[Quickstart](start/QUICKSTART.md)** gets you indexed and connected to
your agent in under five minutes, with no API key.
---
## Pick your path
| You are | Start here | Then |
|---|---|---|
| **A developer wiring up an AI agent** | [Quickstart](start/QUICKSTART.md) | [Supported agents](agent/INTEGRATIONS.md) · [MCP tools](agent/MCP_TOOLS.md) · [Hooks](agent/HOOKS.md) · [Distill](agent/DISTILL.md) |
| **Living in an editor** | [VS Code extension](agent/VSCODE.md) | [Codex](agent/CODEX.md) · [opencode](agent/OPENCODE.md) |
| **A team lead watching what ships** | [Change risk](layers/CHANGE_RISK.md) | [Code health](layers/CODE_HEALTH.md) · [Bug history](layers/BUG_HISTORY.md) |
| **Running many repos** | [Workspaces](scale/WORKSPACES.md) | [Worktrees](scale/WORKTREES.md) · [Auto-sync](scale/AUTO_SYNC.md) |
| **Evaluating repowise to buy it** | [Commercial](business/COMMERCIAL.md) | [Security & compliance](business/SECURITY_COMPLIANCE.md) · [Benchmarks](BENCHMARKS.md) |
| **Contributing** | [Architecture](architecture/README.md) | [CONTRIBUTING](../.github/CONTRIBUTING.md) |
---
## Get started
| Doc | What it covers |
|-----|----------------|
| [start/QUICKSTART.md](start/QUICKSTART.md) | Install, index your repo, and connect your agent in under 5 minutes |
| [start/USER_GUIDE.md](start/USER_GUIDE.md) | The everyday guide: how the pieces fit and the workflows they support |
| [start/DASHBOARD.md](start/DASHBOARD.md) | Every view in the local web dashboard, and what each one answers |
## Connect your AI agent
| Doc | What it covers |
|-----|----------------|
| [agent/INTEGRATIONS.md](agent/INTEGRATIONS.md) | Which agents are supported at what depth, generated from the code, plus the recipe for adding one |
| [agent/MCP_TOOLS.md](agent/MCP_TOOLS.md) | The ten task-shaped tools, what each answers, and worked multi-tool examples |
| [agent/HOOKS.md](agent/HOOKS.md) | Proactive delivery: context and warnings that arrive without the agent asking |
| [agent/DISTILL.md](agent/DISTILL.md) | `repowise distill`: compress noisy command output before your agent reads it |
| [agent/VSCODE.md](agent/VSCODE.md) | The VS Code extension: health in the gutter, risk before you push, dashboards in the editor |
| [agent/CODEX.md](agent/CODEX.md) | Wiring repowise into the Codex CLI |
| [agent/OPENCODE.md](agent/OPENCODE.md) | Wiring repowise into opencode |
## The intelligence layers
| Doc | What it covers |
|-----|----------------|
| [layers/INTELLIGENCE_LAYERS.md](layers/INTELLIGENCE_LAYERS.md) | Overview of the five layers: graph, git, docs, decisions, code health |
| [layers/CODE_HEALTH.md](layers/CODE_HEALTH.md) | Defect risk, maintainability, and performance from 49 deterministic detectors |
| [layers/REFACTORING.md](layers/REFACTORING.md) | Concrete, graph-aware refactoring plans (Extract Class, Move Method, Break Cycle) |
| [layers/CHANGE_RISK.md](layers/CHANGE_RISK.md) | Score any commit or `base..HEAD` range 0-10 for defect risk |
| [layers/BUG_HISTORY.md](layers/BUG_HISTORY.md) | Which files and symbols actually get bug-fixed, and how recently |
| [layers/TEST_INTELLIGENCE.md](layers/TEST_INTELLIGENCE.md) | Coverage ingestion, untested hotspots, and running only the tests a diff touches |
| [layers/DECISIONS.md](layers/DECISIONS.md) | Architectural decisions mined from your repo and from your own agent sessions |
| [layers/DEAD_CODE.md](layers/DEAD_CODE.md) | Unreachable files, unused exports, and zombie packages by confidence tier |
| [layers/LANGUAGE_SUPPORT.md](layers/LANGUAGE_SUPPORT.md) | What works per language, across 18 parsed languages and 13 at the Full tier |
| [layers/WIKI.md](layers/WIKI.md) | The generated wiki: page types, what `update` re-renders, styles, output language |
## Scale it
| Doc | What it covers |
|-----|----------------|
| [scale/WORKSPACES.md](scale/WORKSPACES.md) | Multi-repo intelligence: cross-repo contracts, co-changes, and federated MCP |
| [scale/WORKTREES.md](scale/WORKTREES.md) | Linked git worktrees seed their index from the base checkout, with no flags |
| [scale/AUTO_SYNC.md](scale/AUTO_SYNC.md) | Keep the index fresh automatically on every commit |
| [../docker/README.md](../docker/README.md) | Running repowise in Docker |
| [../examples/](../examples/) | Copy-paste walkthroughs (Codex setup and more) |
## Reference
| Doc | What it covers |
|-----|----------------|
| [reference/CLI_REFERENCE.md](reference/CLI_REFERENCE.md) | Every command and flag |
| [reference/CONFIG.md](reference/CONFIG.md) | `.repowise/config.yaml`, `health-rules.json`, and environment variables |
| [reference/COMPUTED_GLOSSARY.md](reference/COMPUTED_GLOSSARY.md) | Definitions for every computed metric and term repowise reports |
| [reference/TELEMETRY.md](reference/TELEMETRY.md) | What anonymous telemetry collects, and how to turn it off |
| [reference/UPGRADING.md](reference/UPGRADING.md) | Notes for upgrading between versions |
| [CHANGELOG.md](CHANGELOG.md) | Release history |
## Evidence
| Doc | What it covers |
|-----|----------------|
| [BENCHMARKS.md](BENCHMARKS.md) | Agent-efficiency, distillation, and defect-prediction results, each with what it does not show |
## Teams & business
| Doc | What it covers |
|-----|----------------|
| [business/COMMERCIAL.md](business/COMMERCIAL.md) | Hosted tier, enterprise (on-prem, SSO/SCIM), and commercial licensing |
| [business/SECURITY_COMPLIANCE.md](business/SECURITY_COMPLIANCE.md) | What leaves your machine, what gets stored, and the answers your security team wants |
## Architecture & internals
How repowise is built, for contributors and the curious.
| Doc | What it covers |
|-----|----------------|
| [architecture/ARCHITECTURE.md](architecture/ARCHITECTURE.md) | The system: package layout, pipelines, MCP server |
| [architecture/code-health.md](architecture/code-health.md) | Health internals: marker computation and calibrated weights |
| [architecture/graph-algorithms.md](architecture/graph-algorithms.md) | Every graph algorithm, the intuition plus the math |
| [architecture/language-support.md](architecture/language-support.md) | The language pipeline and how the tiers work |
| [architecture/chat.md](architecture/chat.md) | Codebase chat: agent loop, streaming, artifact panel |
| [architecture/structurizr-export.md](architecture/structurizr-export.md) | Export the architecture as Structurizr DSL and render it anywhere |
| [architecture/editor-files.md](architecture/editor-files.md) | How `CLAUDE.md` and `AGENTS.md` get generated |
| [architecture/deep-dives.md](architecture/deep-dives.md) | Systems not covered elsewhere |
| [architecture/pluggable-storage.md](architecture/pluggable-storage.md) | The capability seams: storage, graph, vector, CLI, MCP |
| [design/theme-tokens.md](design/theme-tokens.md) | Resolved design tokens and the WCAG contrast matrix |
---
### Architecture/ARCHITECTURE
# repowise Architecture
repowise is an open-source, self-hostable codebase documentation engine. It generates
a structured, hierarchical wiki for any codebase, keeps it accurate as code changes,
and exposes everything through an MCP server so AI coding assistants can query it
in real time.
This document covers how the system is built, why each piece exists, and how
they fit together. Read this before contributing.
### Package READMEs
For per-package detail (installation, full API reference, all CLI flags, file maps):
| Package | README | What it covers |
|---------|--------|----------------|
| `packages/core` | [`packages/core/README.md`](../../packages/core/README.md) | Ingestion, generation, persistence, providers — all key classes with code examples |
| `packages/cli` | [`packages/cli/README.md`](../../packages/cli/README.md) | CLI entrypoints and flags; full surface in [`CLI_REFERENCE.md`](../reference/CLI_REFERENCE.md) |
| `packages/server` | [`packages/server/README.md`](../../packages/server/README.md) | All REST API endpoints, 11 MCP tools, webhook setup, scheduler jobs |
| `packages/web` | [`packages/web/README.md`](../../packages/web/README.md) | Every frontend file with purpose — API client, hooks, components, pages |
---
## Table of Contents
1. [System Overview](#1-system-overview)
2. [Repository Structure](#2-repository-structure)
3. [The Three Stores](#3-the-three-stores)
4. [Provider Abstraction Layer](#4-provider-abstraction-layer)
5. [Init Path — First-Time Documentation](#5-init-path--first-time-documentation)
6. [Maintenance Path — Keeping Docs in Sync](#6-maintenance-path--keeping-docs-in-sync)
7. [Git Intelligence](#7-git-intelligence)
8. [Dead Code Detection](#8-dead-code-detection)
9. [Decision Intelligence](#9-decision-intelligence)
10. [MCP Server](#10-mcp-server)
11. [REST API and Web UI](#11-rest-api-and-web-ui)
12. [Codebase Chat](#12-codebase-chat)
13. [Data Flow Diagrams](#13-data-flow-diagrams)
14. [Key Design Decisions](#14-key-design-decisions)
15. [Editor File Generation](#15-editor-file-generation)
16. [Adding a New Language](#16-adding-a-new-language)
17. [Adding a New LLM Provider](#17-adding-a-new-llm-provider)
---
## 1. System Overview
```
/* Detailed source-code truncated for AI context efficiency. */
```
repowise has two operational modes that share the same core engine but follow
different strategies:
- **Init** — first-time documentation of an existing codebase. May take minutes
to hours on large repos. Resumable. Supports batch API for cheaper generation.
- **Maintenance** — incremental updates triggered by git commits. Runs in seconds
to minutes. Uses change propagation through the dependency graph to only regenerate
what actually changed.
---
## 2. Repository Structure
```
/* Detailed source-code truncated for AI context efficiency. */
```
---
## 3. The Three Stores
repowise uses three separate storage systems. They are not redundant — each answers
a fundamentally different kind of question that the other two cannot answer efficiently.
### 3.1 SQL Store (SQLAlchemy + SQLite / PostgreSQL)
**Answers: what exists, what changed, when.**
The source of truth for all structured data. SQLite in development and single-server
deployments; PostgreSQL for multi-worker production deployments. The schema is identical
for both — SQLAlchemy abstracts the difference.
Key tables:
| Table | Purpose |
|-------|---------|
| `repos` | Registered repositories, sync state, provider config |
| `wiki_pages` | All generated wiki pages with content, metadata, confidence score, and a short LLM-extracted `summary` (1–3 sentences) used by `get_context` to keep responses bounded |
| `page_versions` | Full version history of every page (for diff view) |
| `symbols` | Symbol index: every function, class, method across all files |
| `answer_cache` | Memoised `get_answer` responses keyed by `(repository_id, question_hash)` plus the provider/model used. Repeated questions return at zero LLM cost; cache entries are invalidated by repository re-indexing. |
| `generation_jobs` | Job state machine with checkpoint fields for resumability |
| `webhook_events` | Every received webhook event (deduplication, audit, retry) |
| `symbol_rename_history` | Detected renames for auditing and targeted text patching |
| `graph_nodes` / `graph_edges` | SQLite-backed graph for repos exceeding 30K nodes |
| `git_metadata` | Per-file git history: commit counts, ownership, co-change partners, hotspot/stable flags |
| `dead_code_findings` | Dead code findings: unreachable files, unused exports, zombie packages |
If you delete the SQL store, you lose everything and must re-run `repowise init`.
### 3.2 Vector Store (LanceDB embedded / pgvector)
**Answers: what is semantically similar to this query.**
repowise uses a `VectorStore` abstraction with two backends, selected automatically
based on the configured SQL backend:
**LanceDB (default — SQLite mode)**
LanceDB runs embedded as a library — no separate server process. Data is stored in
`.repowise/lancedb/` using the Lance columnar format. This makes self-hosting trivial
and keeps the Docker setup simple. LanceDB is significantly faster than ChromaDB on
both write throughput (batch embedding) and ANN query latency, and it requires no
C++ build tools to install.
**pgvector (PostgreSQL mode)**
When repowise is configured with a PostgreSQL database, the `wiki_pages` table gains
an `embedding vector(N)` column via the `pgvector` PostgreSQL extension. Embeddings
are stored directly in the same SQL database — no second storage system required.
Vector similarity search uses `<=>` (cosine distance) with an HNSW index. This is
the preferred backend for multi-worker production deployments.
Every generated wiki page is embedded and stored immediately after generation.
The vector store is used in two distinct ways:
**During generation (RAG context):** When generating a wiki page for file A, the
`ContextAssembler` queries the vector store with A's exported symbol names to find
pages for files that A imports from. These are included as context in the generation
prompt. Each generated page is aware of what its dependencies actually *do* —
not just their names. See [Section 5.4](#54-context-assembly) for details.
**During search and MCP queries:** The `search_codebase` MCP tool and the web UI
search page use the vector store to find the most semantically relevant pages for a
natural language query. This is better than full-text search for questions like
"how does authentication work?" or "where is rate limiting handled?".
If you delete the vector store (LanceDB directory or pgvector embeddings), search
quality degrades and generation context becomes shallower — rebuild it by running
`repowise reindex` which re-embeds all existing SQL pages into LanceDB using
the configured embedder (Gemini or OpenAI). No LLM calls — only embedding API calls.
### 3.3 Graph Store (NetworkX / SQLite-backed)
**Answers: how are things connected, and how important are they.**
The dependency graph is a two-tier directed graph where **file nodes** represent source
files and **symbol nodes** represent individual functions, classes, and methods. Edges
include `imports`, `DEFINES`, `HAS_METHOD`, `CALLS` (with confidence 0.0–1.0),
`inherits`, `implements`, and `co_changes`. `CALLS` edges are built by `CallResolver`
using 3-tier resolution; all others are built by `GraphBuilder` during AST ingestion.
It is built by the `ASTParser` + `GraphBuilder` + `CallResolver` during ingestion and
persisted to `.repowise/graph.json` (for repos ≤ 30K nodes) or the `graph_nodes`/`graph_edges`
SQL tables (for larger repos, using the `networkit` library as a drop-in). The `graph_nodes`
table includes a `kind` column (file, symbol, package, external) and `confidence` column;
`graph_edges` includes a `confidence` column for `CALLS` edges (Alembic migration `0015`).
The graph is used for:
- **Generation ordering** — topological sort determines what to generate first
(files with no dependents get generated before files that import them, so the
richer context is available via RAG when the importing file is generated)
- **Change propagation** — when a file changes, walk the graph to find all
pages that reference its symbols and mark them as stale
- **PageRank** — runs on `file_subgraph()` (file + package nodes only) to identify
the most central files; these get "spotlight" wiki pages and richer generation prompts
- **SCC detection** — circular dependency clusters require a special generation
strategy (see [Section 5.3](#53-circular-dependencies))
- **Co-change edges** — temporal coupling from git history. Files that frequently
change together (but may have no import relationship) get `co_changes` edges.
These participate in change propagation and are shown in the graph visualization
as dashed purple lines. They do NOT affect PageRank.
- **Dead code detection** — files with `in_degree == 0` (no importers) are
candidates for unreachable file detection
- **MCP `get_dependency_path` tool** *(opt-in)* — answers "how is module A connected to module B?"
- **D3 graph visualization** in the web UI
If you delete the graph, repowise loses change propagation and generation ordering.
It can be rebuilt from scratch by re-parsing the source files.
---
## 4. Provider Abstraction Layer
Every LLM call in the entire system goes through `LLMProvider`. No provider SDK
is ever imported from business logic packages.
```
┌─────────────────┐
│ LLMProvider │ (abstract base class)
│ │
│ generate() │
│ generate_stream│
│ embed() │
│ generate_batch │ (optional, default = sequential)
│ estimate_cost │ (optional, returns None if unknown)
└────────┬────────┘
│
┌──────────────────┼──────────────────┬──────────────────┐
▼ ▼ ▼ ▼
AnthropicProvider OpenAIProvider OllamaProvider LiteLLMProvider
(claude-*) (gpt-*, any (any local model, (100+ providers,
batch API + prompt OpenAI-compat fully offline, optional dep)
caching support) endpoint) no API key)
```
### 4.1 Rate Limiter
Each provider instance wraps a `RateLimiter` using a token-bucket algorithm with
two independent buckets: requests-per-minute (RPM) and tokens-per-minute (TPM).
Before every API call, the limiter acquires from both buckets. On a 429 response,
it calls `on_rate_limit_error()` which applies exponential backoff and temporarily
reduces the refill rate. This is transparent to all callers.
Default limits are configured per provider in `.repowise/config.yaml` and can be
adjusted for users with higher API tiers.
### 4.2 Prompt Caching
For Anthropic: the `ContextAssembler` marks the system prompt + shared repository
context with cache-control breakpoints. Across the hundreds of file-page generation
calls during a large init, this shared prefix is only billed once. Cost reduction
on large repos is typically 60–90%.
> **Note:** An older Anthropic Message Batches / `--no-batch` init path is no
> longer part of the product. Init uses concurrent streaming requests under the
> rate limiter; there is no `batch_mode` config key and no `--no-batch` flag.
### 4.3 Adding a Provider
Implement `LLMProvider`, add an entry to `LANGUAGE_CONFIGS` in `providers/registry.py`,
and add a section to `.repowise/config.yaml`. See [Section 12](#12-adding-a-new-llm-provider).
---
## 5. Init Path — First-Time Documentation
`repowise init` runs when documenting a codebase for the first time. It is the
expensive, one-time operation that builds the full wiki from scratch.
### 5.1 File Traversal
`FileTraverser` walks the repository tree and produces a `FileInfo` for every
file that should be documented. It respects six layers of exclusion, applied in
priority order:
1. `.gitignore` (parsed with `pathspec`, not simple glob matching)
2. Root `.repowiseIgnore` (same syntax, user-defined at repo root)
3. **Per-directory `.repowiseIgnore`** — loaded from each directory visited during
the `os.walk`. Patterns are relative to the directory containing the file (like
git's per-directory `.gitignore`). A spec per directory is loaded once and cached
for the traversal. Example: `generated/` in `src/.repowiseIgnore` skips
`src/generated/` without affecting other directories with the same name.
4. **`extra_exclude_patterns`** (constructor param) — additional gitignore-style
patterns passed at runtime from `--exclude/-x` CLI flags or
`repo.settings["exclude_patterns"]` (set via Web UI or REST API PATCH). Applied
to both directory pruning (entire subtree skipped) and individual file filtering.
5. Hardcoded blocklist (`node_modules`, `.git`, `__pycache__`, `dist`, `build`,
`*.lock`, `*.min.js`, generated protobuf files, etc.)
6. Auto-detection of generated files (files with `// Code generated` headers)
7. Binary files (detected by null bytes in first 8KB)
8. Files over `max_file_size_kb` (default: 500KB)
**Constructor signature:**
```python
FileTraverser(
repo_root: Path,
*,
max_file_size_kb: int = 500,
extra_ignore_filename: str = ".repowiseIgnore",
extra_exclude_patterns: list[str] | None = None,
)
```
**Where `extra_exclude_patterns` comes from:**
| Source | How patterns reach traverser |
|--------|------------------------------|
| `repowise init -x vendor/ -x 'src/gen/**'` | Merged with `config.yaml exclude_patterns`, passed directly |
| `repowise update` | Read from `.repowise/config.yaml exclude_patterns` |
| Web UI Excluded Paths section | Saved to `repo.settings["exclude_patterns"]` via REST API |
| Server sync job | Read from `repo.settings.get("exclude_patterns", [])` |
The traverser also detects monorepo structure by looking for multiple
`pyproject.toml`, `package.json`, `go.mod`, or `Cargo.toml` files at depth 1–2.
When detected, each package is documented as a semi-independent unit with
cross-package edges tracked in the graph.
Each `FileInfo` is tagged with: `language`, `is_test`, `is_config`, `is_api_contract`,
`is_entry_point`, `git_hash`. These tags influence generation priority and prompt choice.
**Test files are first-class wiki targets.** The page generator includes any file
tagged `is_test=True` that has at least one extracted symbol, even if the file's
PageRank is near zero (which is typical: nothing imports test files back, so
graph-centrality metrics never select them on their own). Test files answer
questions of the form *"what test exercises X"* / *"where is Y verified"*, and
the doc layer is the right place to surface those. Users who want to exclude
tests from the wiki entirely can pass `--skip-tests` to `repowise init`.
### 5.2 AST Parsing
`ASTParser` is a single class that handles all supported languages. There are no
per-language subclasses.
**How it works:**
1. Calls `get_parser(language)` from `tree-sitter-languages` to get the right grammar
2. Parses the source into a tree-sitter parse tree
3. Loads the corresponding `queries/.scm` file (cached after first load)
4. Runs tree-sitter queries to extract symbols, imports, and exports
5. Maps results through `LANGUAGE_CONFIGS[language]` for language-specific rules
(visibility keywords, entry point patterns, etc.)
6. Returns a `ParsedFile` with a consistent shape regardless of language
**The `.scm` query files** use standard tree-sitter S-expression syntax with
consistent capture name conventions across all languages:
- `@symbol.def` — the full symbol node
- `@symbol.name` — the name identifier
- `@symbol.params` — parameter list
- `@symbol.return_type` — return type annotation
- `@symbol.docstring` — existing docstring (expanded, not repeated, by the LLM)
- `@import.statement` — full import node
- `@import.module` — the module being imported
**Adding a new language** = write one `.scm` file + add one entry to `LANGUAGE_CONFIGS`.
No changes to `ASTParser` itself. See [Section 11](#11-adding-a-new-language).
**Special handlers** live in `core/special_handlers/` and are separate from `ASTParser`.
They handle file types that are not programming languages (OpenAPI specs, Dockerfiles,
GitHub Actions YAML, Makefiles, Protobuf, GraphQL schemas). They produce the same
`ParsedFile` shape but use purpose-built parsers (PyYAML, `graphql-core`, etc.)
instead of tree-sitter.
### 5.3 Dependency Graph Construction
`GraphBuilder` takes all `ParsedFile` outputs and builds a `networkx.DiGraph`.
**Node types:**
- `file` — every source file
- `symbol` — every function, class, method, interface, etc. (added via `add_file()`)
- `package` — every package/module directory
- `external` — third-party packages (lightweight node, not fully documented)
**Edge types:**
- `imports` — file A imports from file B
- `DEFINES` — file A defines symbol B
- `HAS_METHOD` — class A has method B
- `CALLS` — symbol A calls symbol B (with confidence score 0.0–1.0)
- `inherits` — class A extends class B
- `implements` — class A implements interface B
- `instantiates` — code in A creates an instance of B
- `references` — looser reference (type annotation, generic, etc.)
- `re-exports` — A re-exports symbols from B (barrel files)
- `inter_package` — edge crossing package boundary (monorepos)
- `co_changes` — A and B frequently change in the same commit (from git history,
added by `GitIndexer` after graph construction). Weight = co-change count.
Filtered out of PageRank but included in change propagation and visualization.
**Call resolution** is handled by the `CallResolver` module (`ingestion/call_resolver.py`),
which runs after the static import graph is built. It operates in three tiers:
1. **Same-file resolution** (confidence 0.95) — call target defined in the same file
2. **Import-scoped resolution** (confidence 0.85–0.93) — target matched via named bindings
from the file's import list
3. **Global unique match** (confidence 0.50) — target is unique across the whole repo
Call sites are extracted by tree-sitter for all 14 supported languages (Python, TypeScript,
JavaScript, Go, Rust, Java, C++, C, Kotlin, Ruby, C#, Swift, Scala, PHP) using per-language `.scm` query files. Results are stored
as `CallSite` dataclasses and become `CALLS` edges in the graph.
**Named binding resolution** (`NamedBinding` dataclass in `ingestion/models.py`) ensures
that aliased imports, barrel re-exports, and namespace imports resolve to the correct
definition site. The parser's `_extract_import_bindings()` produces bindings for each
import statement, and `GraphBuilder.build()` populates `Import.resolved_file` from them.
Barrel files (`__init__.py`, `index.ts`) are followed one hop to resolve re-exports.
**Two-tier graph isolation:**
Symbol nodes and their `DEFINES`/`HAS_METHOD`/`CALLS` edges are stored in the same
`DiGraph` as file nodes, but `file_subgraph()` returns a view containing only `file`
and `package` nodes. All file-level metrics (PageRank, betweenness, SCCs, Louvain)
run on this subgraph so that the large number of symbol nodes does not distort centrality
scores.
After graph construction, the builder computes:
- **PageRank** — nodes with high PageRank are central and well-connected.
Used to decide generation priority and which symbols get spotlight pages.
- **Strongly Connected Components (SCCs)** — groups of files with circular imports.
Logged as warnings. Require special generation handling (see below).
- **Betweenness centrality** — identifies "bridge" symbols whose removal would
disconnect the graph. These are the most critical to document well.
- **Community detection (Louvain)** — discovers logical modules even when the
directory structure doesn't reflect them. These communities become module pages.
**Circular dependency handling:**
Files in an SCC cannot be generated in strict bottom-up order because each one
depends on the others. Strategy:
1. Generate each SCC member with reduced context (own source + signatures of
cycle partners only, no full docs)
2. Generate a dedicated `SCC` wiki page explaining the cycle and how to navigate it
3. In a second pass, upgrade each member's page with cross-references to the others
**Graph scalability:**
For repos with fewer than 30K nodes, the graph lives in memory as a NetworkX
`DiGraph` and is serialized to `.repowise/graph.json`.
For repos exceeding 30K nodes (configurable via `graph_backend: sqlite`), repowise
switches to `networkit` with SQLite backing via the `graph_nodes` and `graph_edges`
tables. Only the subgraph needed for each operation is loaded into memory. The
API is identical — this is transparent to all callers.
### 5.4 Context Assembly
This is the most important quality driver in the system. The `ContextAssembler`
builds the prompt context for each generation call — it does not just dump the
raw source file.
For a given file page, it assembles context in this priority order (dropping
lower-priority items if the token budget is exceeded):
1. **Source code** — full source (or chunked if file is large)
2. **Symbol signatures** — all symbols extracted from this file with their
signatures and existing docstrings
3. **Graph context** — PageRank score, cluster membership, which module this
belongs to, entry point status
4. **Git context** — ownership, significant commit messages explaining *why* code
evolved, hotspot/stable classification, co-change partners. This transforms
documentation from "here is what this code does" into "here is what it does
and why it was written this way." Git context is the last thing dropped when
over budget — it is more valuable than import summaries.
5. **Import summaries** — for each file this file imports from: the summary of
that file's already-generated wiki page (if available), or just its public
API signatures (if not yet generated)
6. **RAG context** — vector store similarity search (LanceDB or pgvector) using this file's top exported
symbols as the query. Returns the top 3 most relevant already-generated pages.
This propagates understanding upward: `AuthService`'s page will know what
`UserRepository` actually does, not just that it imports from it.
7. **Co-change context** — wiki pages for co-change partners (files that change
together without an import relationship). Reveals hidden coupling.
8. **Dead code findings** — symbols in this file flagged as unused (if any).
Listed in the generation prompt so the LLM notes them as cleanup candidates.
9. **Reverse import context** — which files import *this* file, and what they
use from it. Helps the LLM understand how this file is used in practice.
Token budget: the total assembled context targets 12K tokens, leaving room for
the generation output. Items are dropped in reverse priority order when over budget.
The source code is never dropped — it is chunked instead if too large.
**Large file chunking:**
Files over `large_file_threshold_kb` (default: 100KB) are handled differently.
Rather than passing the full source, repowise:
1. Extracts public symbol signatures only for the top-level context
2. Generates one sub-page per major class or function group
3. Synthesizes a file page from the sub-pages
Pages generated from chunks are tagged `chunked: true` in metadata.
### 5.5 Hierarchical Generation Order
Generation must follow a strict dependency-aware order. This is not optional —
generating a module page before its file pages means the module page has no
content to draw from.
```
Level 0: External API contracts (OpenAPI, proto, GraphQL)
→ self-contained, no dependencies on other pages
Level 1: Symbol spotlight pages (top 10% by PageRank)
→ short, fast, parallelizable
→ high-PageRank symbols get richer documentation
Level 2: File pages
→ uses: source + symbol docs + import summaries (from RAG)
→ parallelizable within this level
→ SCC members generate with stubs, upgraded in second pass
Level 3: SCC pages
→ must wait for all member files to complete Level 2
Level 4: Module/package pages
→ uses: all file pages within the module
Level 5: Cross-package relationship pages (monorepos only)
→ uses: all package pages
Level 6: Repository overview + architecture diagram
→ uses: all module pages + graph metrics
Level 7: Config/infra pages (Dockerfile, CI YAML, Makefile, etc.)
→ references code pages, so comes after all code docs
Level 8: Index pages (symbol index, search index)
→ built from all completed pages
```
Within each level, up to `concurrent_jobs` tasks run in parallel using
`asyncio.Semaphore`. The generation engine never exceeds this limit regardless
of how large the repo is.
Each generated page gets a `confidence_score` of `1.0` when first created.
### 5.6 Resumable Jobs
Init on a 50K-file repo can take 30–90 minutes. A crash or network interruption
should not require starting over.
The `JobSystem` persists checkpoint state after every completed page:
- `checkpoint_level` — which generation level is currently active
- `checkpoint_file_index` — position within the current level
- `completed_page_ids` — list of already-generated page IDs
- `failed_page_ids` — pages that failed (retried on resume)
On `repowise init --resume`, the set of already-written pages is read from the
vector store rather than from the checkpoint, because the store is the one
record that survives a killed process. This needs a job system and a store; with
neither, a resumed run falls back to regenerating everything. Those page ids are
skipped before their
coroutine is built, so a resumed run spends nothing on them, and the match is on
page id alone: a run resumed under a different provider still keeps what the
previous one wrote. Persistence is told which ids were skipped so the stale-page
sweep does not mistake "deliberately kept" for "no longer produced".
`repowise init` is fully idempotent. Running it twice produces the same result.
Running it after a partial previous run completes only the remaining pages.
### 5.7 Shared Persistence (`pipeline/persist.py`)
The persistence logic for storing a `PipelineResult` into the database (graph nodes,
edges, symbols, pages, git metadata, dead code findings, decision records) was
extracted from the CLI's `init_cmd.py` into `core/pipeline/persist.py`. Both the
CLI and the server's background job executor call `persist_pipeline_result()` — zero
duplication.
FTS indexing is intentionally excluded from this function. Callers must run it
separately after the session closes to avoid SQLite write-lock conflicts.
### 5.8 Background Job Executor (`server/job_executor.py`)
The server can now run full pipeline jobs in the background, triggered by the
`POST /api/repos/{id}/sync` and `POST /api/repos/{id}/full-resync` endpoints.
`execute_job()` is the single entry point, launched via `asyncio.create_task()`:
1. Marks the job as `running`
2. Resolves the LLM provider from server config
3. Runs `run_pipeline()` with a `JobProgressCallback` that writes progress to the
`GenerationJob` table (the SSE stream endpoint polls this table)
4. Persists results via `persist_pipeline_result()`
5. Marks the job as `completed` (or `failed` on error)
Progress updates are batched (every 5 items) to avoid per-item DB overhead. Before
writing the final job status, all in-flight progress tasks are drained to prevent a
late `running` update from overwriting `completed`.
Concurrent pipeline runs on the same repository are prevented at the endpoint level
(returns HTTP 409 if a pending/running job already exists).
### 5.9 Async Pipeline Improvements
The pipeline orchestrator now keeps the event loop responsive during CPU-bound work:
- File I/O uses `asyncio.wrap_future()` instead of blocking `as_completed()`
- Graph building runs in a thread via `asyncio.to_thread()`
- The parse loop yields control every 50 files with `asyncio.sleep(0)`
- Thread pool shutdown is non-blocking via `asyncio.to_thread()`
---
## 6. Maintenance Path — Keeping Docs in Sync
`repowise update` runs after a git push (triggered by webhook, GitHub Action, or
polling fallback). It is fast, targeted, and avoids regenerating pages that
haven't actually changed in a meaningful way.
### 6.1 Change Detection
`ChangeDetector` uses GitPython to compute the diff between `last_sync_commit`
and `HEAD`:
```python
changed_files = differ.get_changed_files(repo_path, since_commit="a1b2c3d")
# → ChangedFiles(added=[...], modified=[...], deleted=[...], renamed=[...])
```
Renamed files get special treatment: repowise updates all `source_files` references
in existing wiki pages and regenerates the file page (since the path context has
changed), but does *not* regenerate pages that only reference the file's symbols
(symbol names don't include paths).
### 6.2 Symbol Rename Detection
`ChangeDetector.detect_symbol_renames()` uses a heuristic to identify when a
symbol was renamed rather than deleted-and-recreated:
- Same `kind` (function → function)
- Similar signature (Levenshtein distance under threshold)
- Git blame on the new file confirms line provenance
When a rename is detected, repowise applies a targeted text patch to all pages
that only *mention* the old name (cheaper than full regeneration), and fully
regenerates pages that *document* the old symbol.
All detected renames are stored in `symbol_rename_history` for audit purposes.
### 6.3 Affected Page Computation
For each changed file, `ChangeDetector.get_affected_pages()` walks the dependency
graph to find all pages that need updating:
```
Changed file → find its symbols
→ find all wiki pages that document those symbols (direct)
→ find all wiki pages that reference those symbols (1-hop: inherits/calls)
→ find all wiki pages that reference referencing pages (2-hop: looser reference)
→ apply cascade budget
```
**Cascade budget:**
A change to a central utility module (e.g., `utils.py` imported by 200 files) would
naively require regenerating 200+ pages on every push. This is too expensive.
The `cascade_budget` config option (default: 30 pages per maintenance run) caps the
number of pages fully regenerated per push. Pages beyond the budget:
- Have their `confidence_score` decayed (see below)
- Are added to the background staleness queue
- Are regenerated by the nightly background job
### 6.4 Confidence Score System
Every wiki page has a `confidence_score` (float 0.0–1.0) representing how fresh
it is relative to the source code:
| Score | Status | Badge | Action |
|-------|--------|-------|--------|
| ≥ 0.80 | fresh | green | none |
| 0.60–0.79 | stale | yellow | queued for background regen |
| 0.30–0.59 | outdated | red | auto-queued, warning shown |
| < 0.30 | unusable | red (prominent) | force-regenerated immediately |
Decay rules applied on each maintenance run:
- Source file directly changed: `score *= 0.85`
- Referenced symbol changed (1-hop: calls/inherits): `score *= 0.95`
- Referenced symbol changed (2-hop: looser reference): `score *= 0.98`
- Co-change partner changed (no import relationship): `score *= 0.97`
- Page beyond cascade budget (time-based, 1 week since change): `score *= 0.90`
**Git-informed decay modifiers** (applied multiplicatively on top of base decay):
- Hotspot file (`is_hotspot=True`) → decays faster: direct `*= 0.94`, 1-hop `*= 0.95`
- Stable file (`is_stable=True`) → decays slower: direct `*= 1.03`
- Large change in commit message ("rewrite", "refactor", "migrate") → hard decay:
direct `*= 0.71`, 1-hop `*= 0.84`
- Cosmetic change in commit message ("typo", "lint", "format") → soft decay:
direct `*= 1.12`
Pages start at `1.0` and decay over time. Regeneration resets to `1.0`.
### 6.5 Webhook Reliability
Webhooks can be missed when the repowise server is down during a push. repowise
uses a two-layer sync strategy:
**Layer 1 (real-time):** GitHub/GitLab webhook → `POST /api/webhooks/github` →
signature verification → store in `webhook_events` → enqueue `GenerationJob`.
Response is always `200 OK` immediately — processing is async.
**Layer 2 (polling fallback):** APScheduler job runs every 15 minutes (configurable).
Compares `repos.last_sync_commit` against actual `HEAD` via GitHub API or GitPython.
If they differ, triggers an incremental update for the missing commits.
The combination ensures: even if a webhook is missed entirely, docs are at most
`polling_interval` minutes stale for the default branch.
### 6.6 Background Staleness Resolution
APScheduler runs a nightly job (configurable via cron expression) to:
1. Query all pages where `confidence_score < staleness_regen_threshold` (default: 0.60)
2. Sort by `confidence_score ASC` (most stale first)
3. Regenerate up to `background_regen_budget` pages (default: 100)
4. Log token usage and cost
This ensures no page stays stale indefinitely regardless of cascade budget constraints.
### 6.7 PR Documentation Preview
On `pull_request` webhook events, repowise runs `ChangeDetector` in dry-run mode
and posts a GitHub PR comment listing:
- Pages that will be regenerated (with links to current versions)
- Pages that will have confidence decay
- New pages that will be created (new files)
- Pages that will be deleted (deleted files)
- Estimated token cost for this PR's docs update
On merge, the actual incremental update runs.
---
## 7. Git Intelligence
repowise mines git history to make documentation significantly richer and more useful.
The `GitIndexer` runs once during `repowise init` (after graph construction, before
generation) and incrementally during `repowise update`. All git features degrade
gracefully when git metadata is unavailable — they simply skip git-enriched context.
### 7.1 GitIndexer (`packages/core/ingestion/git_indexer.py`)
The `GitIndexer` class mines git history into the `git_metadata` SQL table. For each
tracked file, it computes:
- **Commit volume** — total, last 90 days, last 30 days (churn signals)
- **Timeline** — first and last commit dates (file age)
- **Ownership** — primary owner from `git blame` (who wrote the most lines),
top 3 contributors by commit count
- **Significant commits** — up to 50 meaningful commit messages (filtered: no merges,
no dependency bumps, no chore/ci, messages > 20 chars). These explain *why*
the code evolved this way and are included in generation prompts.
- **Co-change partners** — files that changed in the same commit >= 3 times, even
without an import relationship. Reveals hidden structural coupling.
- **Derived signals** — `is_hotspot` (top-quartile decayed churn AND absolute
activity floors: >= 3 commits in 90d with real line movement), `is_stable`
(>10 commits, 0 in 90 days), `churn_percentile` (0.0–1.0)
**Performance targets:**
- 3,000 files, 10K commits → < 3 minutes
- 50,000 files, 100K commits → < 20 minutes (uses shallow history for large repos)
- Uses `asyncio.Semaphore(20)` to parallelize git log calls across files
### 7.2 How Git Intelligence Enhances Each Component
**Generation prompts:** The `file_page.j2` template includes optional git context
blocks (gated on `{% if git_metadata %}`). These add ownership attribution, evolution
context from significant commits, hotspot/stable warnings, and co-change partner
documentation. The LLM uses commit messages to explain *why* code is structured as
it is, not just *what* it does.
**Generation ordering:** Within each level, files are sorted by priority: entry
points first, then hotspots, then high PageRank, then high commit count. This
ensures the most important files generate first, making them available as RAG
context for less important files.
**Generation depth:** Files can be auto-upgraded to "thorough" depth if they are
hotspots, have > 100 commits with > 10 in 90 days, have >= 8 significant commits,
or have co-change partners. Conversely, stable + low PageRank + low commit count
files can be auto-downgraded to "minimal". Controlled by `git.depth_auto_upgrade`.
**Maintenance prompts:** When regenerating during `repowise update`, the specific
commit that triggered the update (SHA, author, message, diff) is included in the
prompt. This produces targeted updates rather than full rewrites.
**Confidence decay:** Git signals modify the base confidence decay multiplicatively.
Hotspot files decay faster (bugs are more likely). Stable files decay slower. Large
changes ("rewrite", "refactor", "migrate" in commit message) trigger aggressive
decay. Cosmetic changes ("typo", "lint", "format") trigger softer decay.
**Co-change edges:** After `GitIndexer` runs, co-change edges (`edge_type="co_changes"`)
are added to the dependency graph. They participate in change propagation (when file
A changes, its co-change partners get mild confidence decay at factor 0.97) and are
visible in the graph visualization as dashed purple lines.
**CLAUDE.md generation:** `repowise generate-claude-md` includes sections for
hotspots, stable core files, ownership map, and hidden coupling pairs.
### 7.3 Module and Repo-Level Git Summaries
Module pages include a team ownership summary: who maintains the most files, who
was most active recently. Repo overview pages include codebase health signals:
hotspot count, stable file count, top churn files, oldest file.
---
## 8. Dead Code Detection
repowise detects unreachable files, unused exports, and zombie packages using
graph traversal and SQL queries. No LLM calls — the analysis completes in < 10
seconds for any repo size.
### 8.1 DeadCodeAnalyzer (`packages/core/analysis/dead_code.py`)
The analyzer runs after `GitIndexer` during init (Step 3.6) and optionally during
`repowise update`. It produces findings with confidence scores:
**Finding types:**
- `unreachable_file` — file with `in_degree == 0`, not an entry point, test, or config
- `unused_export` — public symbol with no incoming edges (but file IS imported)
- `unused_internal` — private symbol with no `calls` edges from same file
- `zombie_package` — monorepo package with no incoming `inter_package` edges
**Confidence scoring (conservative — when in doubt, do NOT flag):**
- Unreachable + no commits in 90d + last commit > 1 year → 1.0
- Unreachable + no commits in 90d + last commit > 6 months → 0.9
- Unreachable + no commits in 90d + last commit > 90 days → 0.8
- Unreachable + no commits in 90d + file under 30 days old → 0.55
- Unreachable + no commits in 90d, no commit date to age it by → 0.7
- Unreachable but recently touched → 0.4
- `safe_to_delete` only set at confidence >= `SAFE_CONFIDENCE_THRESHOLD` (0.7),
and NOT for files matching dynamic patterns (`*Plugin*`, `*Handler*`,
`*Adapter*`, `*Middleware*`), and NOT for paths carrying a runtime-load risk
factor. The rungs above are an evidence scale, not the tier boundaries; the
boundaries live in `core/analysis/dead_code/risk_factors.py`
**Never flagged as dead:**
- `__init__.py` public re-exports
- `@pytest.fixture`, `@pytest.mark.*` symbols
- Files matching `*migrations*`, `*schema*`, `*seed*`
- TypeScript `.d.ts` files
- Files where `is_api_contract == True`
- Files named in the `whitelist` key of the analyzer's config argument (an API
parameter; no CLI flag or config file populates it today)
- Symbols matching `config.dead_code.dynamic_patterns`
### 8.2 Dead Code in Generation Prompts
The `file_page.j2` template includes an optional block for dead code findings.
When present, the LLM notes unused symbols as cleanup candidates in the
documentation, including confidence percentages and safety assessment.
### 8.3 CLI: `repowise dead-code`
```
repowise dead-code [PATH]
--min-confidence FLOAT (default: 0.4)
--safe-only only show safe_to_delete=True findings
--kind TYPE filter by kind
--package PKG filter by monorepo package
--format table|json|md
--output FILE
repowise dead-code resolve [FINDING_ID]
--status acknowledged|resolved|false_positive
--note "reason"
```
### 8.4 Integration with Other Components
Dead code findings are stored in the `dead_code_findings` SQL table with a status
field (`open` / `acknowledged` / `resolved` / `false_positive`). The web UI shows
a dedicated dead code page with sortable tables, confidence sliders, and resolve
buttons. The graph visualization supports a "show dead code" filter that adds red
dashed borders to dead code nodes.
---
## 9. Decision Intelligence
The Decision Intelligence layer captures **architectural decisions** — the *why* behind how
the system is built, what alternatives were rejected, and what constraints exist. While documentation
describes *what*, decisions capture *why*.
### Capture Sources
Decisions are extracted from four sources, each with a different confidence level:
| Source | Confidence | Status | Trigger |
|--------|-----------|--------|---------|
| **Inline markers** (`# WHY:`, `# DECISION:`, `# TRADEOFF:`, `# ADR:`, `# RATIONALE:`, `# REJECTED:`) | 0.95 | `active` | File scanning during init/update |
| **Git archaeology** (commit messages with migration/refactor signals) | 0.70–0.85 | `proposed` | Init only, reuses git layer data |
| **README/docs mining** (implicit decisions in prose) | 0.60 | `proposed` | Init only, LLM extraction |
| **CLI capture** (`repowise decision add`) | 1.00 | `active` | Manual entry |
### Data Model
Stored in the `decision_records` SQL table. JSON arrays for alternatives, consequences,
affected files, modules, tags, and evidence commits. Deduplication key:
`(repository_id, title, source, evidence_file)`.
### Staleness Tracking
Decisions have a `staleness_score` (0.0 = fresh, 1.0 = very stale) computed during
`repowise init` and recomputed on every `repowise update`. Staleness rises when affected
files receive commits after the decision was recorded. Decisions with `staleness_score > 0.5`
are flagged as stale.
### MCP Tools
- `get_why(query?)` — three modes: natural language search over decisions, path-based lookup for decisions governing a file, or no-arg for decision health dashboard (stale decisions, ungoverned hotspots, proposed decisions needing review)
- `get_context(targets, include?)` — includes decisions governing each target in its response
### CLI Commands
```
repowise decision add # interactive capture
repowise decision list # tabular list with filters
repowise decision show # full detail
repowise decision confirm # proposed → active
repowise decision dismiss # dismiss proposed (sticky tombstone)
repowise decision deprecate # active → deprecated
repowise decision health # health summary
```
### Key Files
| File | Purpose |
|------|---------|
| `core/analysis/decision_extractor.py` | All 4 capture sources + staleness computation |
| `core/persistence/models.py` | `DecisionRecord` ORM model |
| `core/persistence/crud.py` | 8 decision CRUD functions |
| `server/mcp_server/tool_why.py` | MCP tool `get_why` (3-mode: search, path, health dashboard) |
| `server/routers/decisions.py` | REST API endpoints |
| `cli/commands/decision_cmd.py` | CLI command group (7 subcommands) |
---
## 10. MCP Server
The MCP server is repowise's most valuable integration feature. It exposes the
entire wiki as a set of queryable tools that any MCP-compatible AI assistant
can call in real time.
Instead of an AI assistant reading 40 source files to understand a codebase,
it calls `get_overview()` and gets a structured, always-current architecture summary.
Instead of calling 5 tools one at a time, it calls `get_context(["src/auth/service.py", "AuthService"])` and gets docs, ownership, decisions, and freshness for all targets in one call.
The server is implemented using the [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
and supports two transports:
- **stdio** — for Claude Code, Cursor, Cline (add to their MCP config)
- **SSE** — for web-based MCP clients (served on port 7338)
### Tools (11 default in single-repo mode)
Canonical reference: [`docs/agent/MCP_TOOLS.md`](../agent/MCP_TOOLS.md).
A single-repo server advertises **11** tools by default (ten flagship +
`list_repos`). Workspace mode adds `get_architecture` and `get_blast_radius`.
Four more are registered but opt-in (`get_dependency_path`,
`get_execution_flows`, `generate_refactoring_code`, `get_conformance`).
| Tool | What it answers | When to call |
|------|----------------|-------------|
| `get_overview` | Architecture summary, module map, entry points. | First call when exploring an unfamiliar codebase. |
| `get_answer` | One-call RAG: confidence-gated synthesis with cited answers. | First call on any code question. |
| `get_context(targets, include?)` | Docs, ownership, history, decisions, freshness for files/modules/symbols. | Before reading or modifying specific code. |
| `get_symbol` | Resolve a qualified symbol id to source body, signature, and docstring. | When the question names a specific class, function, or method. |
| `search_codebase(query)` | Hybrid symbol / path / wiki search. | When you don't know where something lives. |
| `get_risk(targets)` | Hotspot score, dependents, co-change partners, risk summary per target. | Before modifying indexed files. |
| `get_change_risk(revspec?)` | Live commit / range defect score from the diff itself. | Before merging a commit or PR range. |
| `get_why(query?)` | Architectural decisions and git archaeology. | Before making architectural changes. |
| `get_dead_code` | Dead/unused code findings sorted by confidence. | Before cleanup tasks. |
| `get_health` | Code-health marker scores (defect / maintainability / performance). | Self-check before a PR or refactor. |
| `list_repos` | Repo aliases this server is serving. | Discover `repo=` targets (especially in a workspace). |
### Auto-generated Config
`repowise init` registers the MCP server directly where it can: Claude Code
(`~/.claude/settings.json`), Claude Desktop (if installed), VS Code
(`.vscode/mcp.json`), and the repo-shared `.mcp.json`. It also writes
`.repowise/mcp.json` for clients configured by hand. The completion panel says
which client was wired up and how Cursor or Codex connect (`repowise mcp .`).
---
## 11. REST API and Web UI
### REST API (FastAPI)
Served on port `7337` alongside the web UI. All endpoints are prefixed with `/api/`.
Key routers:
- `/api/repos` — register repos, trigger sync, full-resync (now launches background pipeline jobs with concurrent-run prevention)
- `/api/pages` — read pages, version history, force-regenerate single page
- `/api/search` — semantic (LanceDB or pgvector) and full-text (SQLite FTS5 / PostgreSQL tsvector) search
- `/api/jobs` — job status, SSE stream for live progress updates
- `/api/symbols` — symbol lookup, dependency path queries
- `/api/graph` — graph export in D3-compatible JSON format
- `/api/webhooks/github` — GitHub webhook handler with HMAC verification
- `/api/webhooks/gitlab` — GitLab webhook handler
- `/api/repos/{id}/git-metadata` — per-file git metadata
- `/api/repos/{id}/hotspots` — high-churn + high-complexity files
- `/api/repos/{id}/ownership` — ownership breakdown (file/module/package granularity)
- `/api/repos/{id}/co-changes` — co-change partners for a file
- `/api/repos/{id}/git-summary` — aggregate git health signals for dashboard
- `/api/repos/{id}/dead-code` — dead code findings (GET list, POST trigger analysis)
- `/api/repos/{id}/dead-code/summary` — aggregate dead code stats
- `/api/dead-code/{finding_id}` — PATCH to resolve/acknowledge findings
- `/api/repos/{id}/claude-md` — GET preview of generated CLAUDE.md section (JSON, no disk write)
- `/api/repos/{id}/claude-md/generate` — POST to regenerate and write CLAUDE.md to disk
- `/health` — liveness + readiness (checks DB + provider)
- `/metrics` — Prometheus-compatible metrics (job counts, token totals, stale count)
**Server lifecycle:**
- On startup, any jobs left in `running` state from a previous server instance are
automatically reset to `failed` (crash recovery).
- Background pipeline tasks are tracked in `app.state.background_tasks` to prevent
garbage collection of `asyncio.Task` references.
Authentication is optional. Set `REPOWISE_API_KEY` to require bearer token auth on
all non-`/health` endpoints. Default (no key set): fully open, suitable for local use.
### Web UI (Next.js 15)
Served from the same port as the API. All routes under `/`:
| Route | Content |
|-------|---------|
| `/` | Dashboard: all repos, recent jobs, stale page counts, token usage |
| `/repos/[id]` | Repo layout with file tree sidebar |
| `/repos/[id]/overview` | **Overview dashboard** — health score ring, attention panel, language donut, ownership treemap, hotspots mini, decisions timeline, module minimap, quick actions, active job banner |
| `/repos/[id]/wiki/[...slug]` | Individual wiki page with MDX rendering |
| `/repos/[id]/search` | Semantic search results |
| `/repos/[id]/graph` | D3 force-directed dependency graph |
| `/repos/[id]/symbols` | Full symbol index, sortable by PageRank |
| `/repos/[id]/coverage` | Documentation coverage metrics |
| `/repos/[id]/ownership` | Ownership treemap — files colored by primary owner, sized by LOC |
| `/repos/[id]/hotspots` | Hotspot list — top 20 files with churn + complexity bars |
| `/repos/[id]/dead-code` | Dead code report — three tabs: Files, Exports, Internals |
| `/repos/[id]/decisions` | Architectural decision records |
| `/repos/[id]/chat` | Codebase chat with streaming LLM responses |
| `/settings` | Provider config, polling interval, cascade budget |
**Key rendering behavior:**
Every backtick identifier in a wiki page that matches a known symbol becomes a
hover card showing the symbol's signature, file, and confidence score, plus a
link to the symbol's wiki page. This is built by post-processing the MDX content
client-side after initial render.
Mermaid diagrams are rendered lazily (only when scrolled into viewport) using
`mermaid.js` initialized with the repowise design theme.
Code blocks use Shiki for syntax highlighting. Each block has a "View in source"
button that deep-links to the relevant line in GitHub/GitLab.
The `` component connects to `/api/jobs/{id}/stream` via SSE
and shows a live progress display during generation (pages done/total, current
file, tokens used, estimated cost, estimated time remaining).
---
## 12. Codebase Chat
repowise includes an interactive chat interface that lets users ask questions about
their codebase and receive answers grounded in the wiki, dependency graph, git
history, and architectural decisions. The chat agent uses whichever LLM provider
the user has configured and has access to **7 tools** from the MCP surface
(see [`chat.md`](chat.md) — not the full 11-tool MCP default).
See [`docs/architecture/chat.md`](chat.md) for the full technical reference covering the
backend agentic loop, SSE streaming protocol, provider abstraction extensions,
database schema, frontend component architecture, and artifact rendering system.
**Key design points:**
- **Provider-agnostic** — the chat agent goes through the same provider abstraction
as documentation generation. A `ChatProvider` protocol extends `BaseProvider` with
`stream_chat()` for streaming + tool use without breaking existing callers.
- **Tool reuse** — the 7 chat tools are called directly as Python functions (no
subprocess round-trip). Tool schemas are defined once in `chat_tools.py` and
fed to both the LLM and the executor.
- **SSE streaming** — `POST /api/repos/{repo_id}/chat/messages` runs the agentic
loop and streams back Server-Sent Events (`text_delta`, `tool_start`,
`tool_result`, `done`, `error`).
- **Conversation persistence** — chat history is stored in `conversations` and
`chat_messages` tables, allowing replay and continuation across page refreshes.
- **Artifact panel** — tool results with rich content (wiki pages, Mermaid diagrams,
search results, risk reports) open in a slide-in artifact panel that reuses
existing frontend components.
---
## 13. Data Flow Diagrams
### Init flow
```
/* Detailed source-code truncated for AI context efficiency. */
```
### Maintenance flow (on git push)
```
git push
│
├── GitHub webhook → POST /api/webhooks/github
│ │ (verified, stored in webhook_events)
│ ▼
│ GenerationJob created (type: incremental)
│
└── (fallback) APScheduler polls every 15min
│ (if HEAD != last_sync_commit)
▼
GenerationJob created (type: incremental)
GenerationJob runs:
│
▼
ChangeDetector.get_changed_files(since=last_sync_commit)
│
├── Renamed files → update source_files refs in SQL
▼
GitIndexer.index_changed_files(changed_file_paths)
│ re-indexes only changed files + their co-change partners
│ updates co-change edges in graph
▼
ChangeDetector.detect_symbol_renames()
│
▼
ChangeDetector.get_affected_pages(cascade_budget=30)
│ includes co-change partners in staleness propagation
│
├── regenerate: list[page_id] → full regeneration (up to cascade_budget)
├── rename_patch: list[page_id] → targeted text find-and-replace
└── decay_only: list[page_id] → confidence score decay, add to staleness queue
│
▼
For each page in `regenerate`:
ContextAssembler → LLMProvider → update SQL + VectorStore + Graph
│
▼
For each page in `rename_patch`:
scan content_md for old symbol name → replace → update SQL
│
▼
For each page in `decay_only`:
confidence_score *= decay_factor
regen_queued = True
│
▼
repos.last_sync_commit = HEAD
state.json updated
```
### Server-triggered pipeline flow
```
Web UI "Sync" / "Full Re-index" button
│
▼
POST /api/repos/{id}/sync (or /full-resync)
│
├── Check: no pending/running job for this repo (else → 409)
▼
Create GenerationJob (status=pending, commit)
│
▼
asyncio.create_task(execute_job(job_id, app_state))
│ (strong ref in app.state.background_tasks)
▼
execute_job():
├── Mark job "running"
├── Resolve LLM provider from server config
├── run_pipeline() with JobProgressCallback
│ └── writes progress to GenerationJob table every 5 items
├── persist_pipeline_result() (shared with CLI)
├── FTS index new pages
├── drain_and_stop() progress tasks
└── Mark job "completed" (or "failed" on error)
```
### MCP query flow
```
Claude Code: "how does the auth module work?"
│
▼
MCP client calls repowise tool: search_codebase(query="auth module")
│
▼
VectorStore similarity search (LanceDB or pgvector) → top 5 page IDs
│
▼
SQL: fetch page content for those IDs
│
▼
Return: list[{title, content_md snippet, relevance_score, confidence_score}]
│
▼
Claude Code has structured, current documentation
instead of reading 40 files
```
---
## 14. Key Design Decisions
### Three-layer folder exclusion, not one monolithic list
repowise offers three complementary ways to skip paths during traversal, each solving
a different problem:
1. **Root `.repowiseIgnore`** — project-wide exclusions committed to the repo (like
`.gitignore`). Shared across all users, version-controlled.
2. **Per-directory `.repowiseIgnore`** — exclusions that only apply within a subtree.
Lets a monorepo package owner exclude its own generated output without polluting
the root ignore file. Patterns are relative to the directory, matching git semantics.
Specs are loaded once per directory during `os.walk` and cached by absolute path;
the root spec is pre-seeded in the cache to avoid reading it twice.
3. **`extra_exclude_patterns` (runtime)** — patterns injected without touching any
file on disk: from `--exclude/-x` CLI flags (dev workflow), from
`config.yaml exclude_patterns` (persisted per repo), or from
`repo.settings["exclude_patterns"]` (Web UI / REST API). This lets teams configure
exclusions through the UI without git access to the target repo.
All three layers use `pathspec` with `gitwildmatch` semantics — the same library used
for `.gitignore` parsing — so the full gitignore syntax works everywhere.
### `save_config()` round-trips YAML, not overwrites
The original `save_config()` wrote a fixed three-key file (`provider`, `model`,
`embedder`). This meant any other keys — such as `exclude_patterns` set via the Web UI
— would be silently dropped the next time `repowise init` ran. The updated function
loads the existing config, merges in the new values, then writes the result back.
This ensures all config sources (CLI, Web UI, REST API, manual edits) coexist safely.
### One `ASTParser` class, not one per language
Per-language differences live in `.scm` query files and `LANGUAGE_CONFIGS` dict entries.
This means adding a new language requires no changes to Python business logic — just
a new `.scm` file and a config entry. The `ASTParser` class itself never has
`if lang == "python"` branches.
### LanceDB (embedded) + pgvector, not ChromaDB
ChromaDB was the original choice but has notable drawbacks: slow write throughput on
large repos, heavy C++ build dependencies (`chroma-hnswlib`), and a bloated dependency
tree. LanceDB replaces it as the default embedded vector store:
- **No build step** — pure Python wheel, no C++ compiler required on Windows
- **Faster writes** — Lance columnar format is optimised for batch appends; embedding
50K pages during `repowise init` is measurably faster
- **Faster queries** — IVF-PQ and HNSW index support with sub-millisecond ANN search
- **Simpler data model** — tables are Arrow-native; filtering by `repo_id` or `page_type`
uses SQL-style predicates alongside the vector search, no separate metadata store needed
- **Zero server process** — same embedded story as ChromaDB: data lives in
`.repowise/lancedb/`, no container needed
When PostgreSQL is already in use (multi-worker prod), **pgvector** is preferred because
it eliminates the second storage system entirely: embeddings live as a `vector` column
in the existing `wiki_pages` table, queries are plain SQL with `<=>` cosine distance,
and backup/restore is a single `pg_dump`. The HNSW index (`CREATE INDEX ... USING hnsw`)
gives query latency on par with LanceDB at typical repowise dataset sizes.
The `VectorStore` abstraction in `packages/core/src/repowise/core/persistence/vector.py`
selects the backend at startup based on `DATABASE_URL`: SQLite → LanceDB, PostgreSQL → pgvector.
### NetworkX + SQLite fallback, not Neo4j
NetworkX is a Python library that runs in-process. Neo4j is a separate server
(Java process, GBs of RAM, authentication setup). For a graph that is derived
from source code (rebuildable at any time), the operational overhead of Neo4j
is unjustified. NetworkX handles tens of thousands of nodes comfortably.
The SQLite-backed fallback using `networkit` handles repos that genuinely exceed
in-memory limits without adding a server dependency.
### Confidence scores, not binary "fresh/stale"
A binary fresh/stale flag would require either: (a) marking everything stale on
every push (triggering full regen, expensive), or (b) only marking directly changed
pages (missing transitive effects). The confidence score model captures the gradient:
directly changed pages decay sharply, indirectly referenced pages decay gently.
Users can see at a glance how much to trust each page.
### Jinja2 prompts, not hardcoded strings
All LLM prompts are Jinja2 templates in `packages/core/queries/prompts/`. Users can
override any prompt by placing a file with the same name in `.repowise/prompts/`.
This lets power users tune generation quality without forking the project.
### Cascade budget
Without a budget, changing a central utility (imported by 200 files) would
regenerate 200+ pages on every push. With a budget (default: 30), repowise
regenerates the highest-PageRank affected pages immediately and defers the rest
to the nightly background job. No page stays stale indefinitely; the nightly job
catches everything the cascade budget missed.
### Git metadata in generation prompts, not just for display
The highest-value use of git metadata is enriching the LLM's generation context —
not just showing ownership in a sidebar. By including significant commit messages
in the prompt, the LLM can explain *why* code is structured a certain way (e.g.,
"this was refactored in March 2024 to separate auth concerns from the request
pipeline"). This context is unavailable from static analysis alone.
### Co-change edges as a separate graph layer
Co-change relationships are temporal coupling — they cannot be detected by AST
parsing. repowise adds them as `co_changes` edges after `GitIndexer` runs, but
deliberately filters them out of PageRank computation (they would skew it
artificially toward files that are often edited together for process reasons, not
architectural ones). They participate in change propagation and visualization only.
### Conservative dead code detection
repowise's dead code detector errs heavily toward false negatives. `safe_to_delete`
is set to `True` only at confidence >= 0.7 and after excluding dynamically-loaded
patterns. Dead code analysis is pure graph traversal + SQL (no LLM calls), so it
completes in seconds and can be re-run cheaply. repowise surfaces candidates —
humans decide before deleting anything.
### Async-first throughout
All database operations use async SQLAlchemy with `aiosqlite`. The event loop
is never blocked. This matters during generation: the LLM call, the DB write,
and the vector store embed (LanceDB or pgvector) can overlap with the next file's context assembly.
---
## 15. Editor File Generation
See [`architecture/editor-files.md`](editor-files.md) for the complete reference covering
architecture, all data sources, how the marker-merge system works, and how to add
support for a new editor file (cursor.md, copilot-instructions.md, etc.).
**Quick summary:** repowise can generate and maintain AI-editor configuration files
(CLAUDE.md, cursor.md, etc.) from the already-indexed codebase data — no LLM calls.
The system uses HTML comment markers to split the file into a user-owned section and
a Repowise-managed section. The user section is never touched.
The feature runs automatically after `repowise init` and `repowise update`, and can
also be run standalone:
```
repowise generate-claude-md [PATH]
```
Config opt-out:
```yaml
# .repowise/config.yaml
editor_files:
claude_md: false
```
Key files:
| File | Purpose |
|------|---------|
| `core/generation/editor_files/base.py` | `BaseEditorFileGenerator` — marker-merge logic shared by all editor-file generators |
| `core/generation/editor_files/data.py` | `EditorFileData` frozen dataclass — the data contract between fetcher and template |
| `core/generation/editor_files/fetcher.py` | `EditorFileDataFetcher` — queries DB for architecture summary, modules, hotspots, decisions |
| `core/generation/editor_files/tech_stack.py` | Filesystem scan for languages, frameworks, build commands |
| `core/generation/editor_files/claude_md.py` | `ClaudeMdGenerator` — 30-line subclass that binds filename + template |
| `core/generation/templates/claude_md.j2` | Jinja2 template for the Repowise-managed section |
| `cli/commands/claude_md_cmd.py` | `repowise generate-claude-md` CLI command |
| `server/routers/claude_md.py` | `GET/POST /api/repos/{id}/claude-md` REST endpoints |
---
## 16. Adding a New Language
1. **Write `packages/core/queries/.scm`**
Use tree-sitter S-expression syntax. Follow the capture name conventions:
`@symbol.def`, `@symbol.name`, `@symbol.params`, `@symbol.return_type`,
`@symbol.docstring`, `@import.statement`, `@import.module`, `@import.names`.
Check the tree-sitter playground for your language's node type names:
`https://tree-sitter.github.io/tree-sitter/playground`
2. **Add a `LanguageConfig` entry to `LANGUAGE_CONFIGS` in `parser.py`**
```python
"mylang": LanguageConfig(
symbol_node_types={
"function_definition": "function",
"class_definition": "class",
},
import_node_types=["import_statement"],
export_node_types=[],
visibility_fn=lambda name, mods: "private" if name.startswith("_") else "public",
entry_point_patterns=["main.ml", "app.ml"],
),
```
3. **Add a `LanguageSpec` to `LanguageRegistry`** in `ingestion/languages/registry.py`
This registers the language's identity data (extensions, entry points, manifest files,
builtin calls, heritage node types, etc.) centrally.
4. **Add the grammar dependency to `pyproject.toml`**
```toml
"tree-sitter-mylang>=0.23,<1",
```
5. **Add test files to `tests/fixtures/sample_repo/`**
At minimum: one file with a function, one with a class, one with imports.
6. **(Optional) Add per-language extractors** for bindings, heritage, visibility, docstrings,
and a dedicated import resolver in `resolvers/mylang.py`.
7. **Run `pytest tests/unit/test_parser.py -k mylang`** to verify extraction.
8. **Open a PR.** That's it — no other changes needed.
---
## 17. Adding a New LLM Provider
1. **Create `packages/core/providers/.py`**
Subclass `LLMProvider` and implement:
- `generate(request: GenerationRequest) -> GenerationResponse`
- `generate_stream(request: GenerationRequest) -> AsyncIterator[str]`
- `embed(request: EmbedRequest) -> EmbedResponse`
- `name` property
Optionally override:
- `supports_batch` → `True` if the provider has a batch API
- `generate_batch(requests) -> list[GenerationResponse]`
- `estimate_cost(input_tokens, output_tokens) -> float`
2. **Register in `providers/registry.py`**
```python
case "myprovider": return MyProvider(config)
```
3. **Add config section to `.repowise/config.yaml` docs**
```yaml
myprovider:
api_key: ${MYPROVIDER_API_KEY}
base_url: https://api.myprovider.com/v1
```
4. **Add default rate limits** to `PROVIDER_DEFAULT_LIMITS` in `rate_limiter.py`
5. **Add a `MockProvider` fixture** for tests if the provider has unique response formats
6. **Update `CONTRIBUTING.md`** with the new provider's environment variables
---
## Appendix: Configuration Reference
Full configuration with defaults (`.repowise/config.yaml`):
```
/* Detailed source-code truncated for AI context efficiency. */
```
---
### Architecture/Chat
# Codebase Chat — Technical Reference
The codebase chat feature lets users have an interactive conversation with their
codebase. The agent uses whichever LLM provider the user has configured, has
access to **7 tools** from the MCP surface (a curated chat subset — not the full
11-tool MCP default), and streams responses back to the browser in real time
showing tool calls as they happen and rendering results in an artifact panel.
---
## Table of Contents
1. [Architecture Overview](#1-architecture-overview)
2. [Database Schema](#2-database-schema)
3. [ChatProvider Protocol](#3-chatprovider-protocol)
4. [Tool Registry](#4-tool-registry)
5. [Provider Configuration](#5-provider-configuration)
6. [SSE Streaming Protocol](#6-sse-streaming-protocol)
7. [Agentic Loop](#7-agentic-loop)
8. [REST API Endpoints](#8-rest-api-endpoints)
9. [Frontend Architecture](#9-frontend-architecture)
10. [Provider-Specific Notes](#10-provider-specific-notes)
---
## 1. Architecture Overview
```
User types question
|
v
POST /api/repos/{repo_id}/chat/messages
|
v
+------ Chat Router (SSE stream) ------+
| |
| 1. Create/load conversation |
| 2. Save user message to DB |
| 3. Build LLM message history |
| 4. Call provider.stream_chat() <----+---- tool_executor callback
| | |
| v |
| 5. Stream text_delta events -------> SSE to browser
| 6. On tool_start: |
| - Execute tool (or provider |
| executes internally) |
| - Emit tool_result event -------> SSE to browser
| 7. If tool calls found: |
| - Append to history, loop to 4 |
| 8. If no tool calls: |
| - Save assistant message to DB |
| - Emit done event |
+---------------------------------------+
```
The agentic loop is in the chat router for most providers (OpenAI, Anthropic,
Ollama, LiteLLM). For Gemini, the loop runs inside `stream_chat()` using native
Content objects to preserve thought signatures. The router passes a
`tool_executor` callback that Gemini calls internally.
---
## 2. Database Schema
Two tables added in migration `0005_chat_conversations.py`:
### `conversations`
| Column | Type | Notes |
|--------|------|-------|
| `id` | `String(32)` PK | UUID hex |
| `repository_id` | `String(32)` FK | CASCADE delete |
| `title` | `Text` | Auto-generated from first 6 words |
| `created_at` | `DateTime(tz)` | |
| `updated_at` | `DateTime(tz)` | Auto-updated on new messages |
**Index:** `ix_conversations_repo_updated` on `(repository_id, updated_at)`
### `chat_messages`
| Column | Type | Notes |
|--------|------|-------|
| `id` | `String(32)` PK | UUID hex |
| `conversation_id` | `String(32)` FK | CASCADE delete |
| `role` | `String(32)` | `user` or `assistant` |
| `content_json` | `Text` | JSON blob (see below) |
| `created_at` | `DateTime(tz)` | |
**Index:** `ix_chat_messages_conv_created` on `(conversation_id, created_at)`
### Message content format
**User messages:**
```json
{"text": "What does the auth module do?"}
```
**Assistant messages:**
```json
{
"text": "The auth module handles...",
"tool_calls": [
{
"id": "call_abc123",
"name": "get_context",
"arguments": {"targets": ["src/auth"]},
"result": { ... }
}
]
}
```
---
## 3. ChatProvider Protocol
Defined in `packages/core/src/repowise/core/providers/base.py`.
The existing `BaseProvider.generate()` is untouched. A new `ChatProvider`
protocol class (using `typing.Protocol` + `@runtime_checkable`) adds streaming
chat with tool use as an opt-in capability.
```python
@runtime_checkable
class ChatProvider(Protocol):
def stream_chat(
self,
messages: list[dict], # OpenAI-format message list
tools: list[dict], # OpenAI-format tool definitions
system_prompt: str,
max_tokens: int = 8192,
temperature: float = 0.7,
request_id: str | None = None,
tool_executor: Any | None = None, # async callable(name, args) -> dict
) -> AsyncIterator[ChatStreamEvent]: ...
```
**Supporting dataclasses:**
- `ChatToolCall(id, name, arguments)` — a tool call the LLM wants to make
- `ChatStreamEvent(type, text?, tool_call?, tool_result_data?, stop_reason?, input_tokens, output_tokens)` — a single event in the stream
**Event types:**
| `type` | Populated fields | Meaning |
|--------|-----------------|---------|
| `text_delta` | `text` | Incremental text token(s) |
| `tool_start` | `tool_call` | LLM wants to call a tool |
| `tool_result` | `tool_call`, `tool_result_data` | Tool executed (by provider internally) |
| `usage` | `input_tokens`, `output_tokens` | Token usage update |
| `stop` | `stop_reason` | Generation ended (`end_turn`, `tool_use`, `max_tokens`) |
**Implementations:** Anthropic, OpenAI, Gemini, Ollama, LiteLLM. All accept the
`tool_executor` parameter; only Gemini uses it (for thought signature handling).
---
## 4. Tool Registry
Defined in `packages/server/src/repowise/server/chat_tools.py`.
Single source of truth for chat tool schemas and execution. Imports **7** MCP
tool functions from `repowise.server.mcp_server` (the chat agent does not
advertise the full MCP default surface — no `get_answer`, `get_symbol`,
`get_health`, or `list_repos` here).
```python
TOOL_REGISTRY: dict[str, ToolDef] # name -> ToolDef(name, description, parameters, function, artifact_type)
```
**Key functions:**
| Function | Purpose |
|----------|---------|
| `get_tool_schemas_for_llm()` | Returns OpenAI-format tool definitions for the LLM |
| `execute_tool(name, args)` | Runs a tool and ensures JSON-serializable output |
| `get_artifact_type(name)` | Maps tool name to frontend artifact type |
| `init_tool_state(...)` | Bridges FastAPI app state to MCP module globals |
**Tool to artifact type mapping (7 tools):**
| Tool | Artifact Type |
|------|--------------|
| `get_overview` | `overview` |
| `get_context` | `wiki_page` |
| `get_risk` | `risk_report` |
| `get_change_risk` | `risk_report` |
| `get_why` | `decisions` |
| `search_codebase` | `search_results` |
| `get_dead_code` | `dead_code` |
---
## 5. Provider Configuration
Defined in `packages/server/src/repowise/server/provider_config.py`.
API keys and active provider/model selection are stored in a server-side
`provider_config.json` file. Environment variables take precedence over stored
keys.
**Resolution order for API keys:**
1. Environment variable (e.g. `GEMINI_API_KEY`, `ANTHROPIC_API_KEY`)
2. Stored key in `provider_config.json`
**Active provider resolution:**
1. Explicitly set via `PATCH /api/providers/active`
2. Auto-detect from first configured provider
**Provider catalog:** Gemini, Anthropic, OpenAI, Ollama (local, no key), LiteLLM.
---
## 6. SSE Streaming Protocol
The chat endpoint returns `Content-Type: text/event-stream`. Each event is:
```
event: data
data: {"type": "...", ...}
```
**Event shapes:**
```jsonc
// Incremental text from the LLM
{"type": "text_delta", "text": "The auth module..."}
// LLM wants to call a tool
{"type": "tool_start", "tool_id": "call_123", "tool_name": "get_context", "input": {"targets": ["src/auth"]}}
// Tool execution completed
{"type": "tool_result", "tool_id": "call_123", "tool_name": "get_context", "summary": "Context for 1 target(s)", "artifact": {"type": "wiki_page", "data": {...}}}
// Stream complete
{"type": "done", "conversation_id": "abc123", "message_id": "def456"}
// Error
{"type": "error", "message": "Provider error: ..."}
```
**Headers:** `Cache-Control: no-cache`, `X-Accel-Buffering: no`, `Connection: keep-alive`
**Retry:** `retry: 3000` sent at stream start.
**Terminal event:** every stream ends with `done` or `error` on the `data`
channel. `useChat` switches on `type` and nothing else, so an event sent on a
different channel, or without a `type`, is dropped: the client then sees a
stream that simply stopped mid-answer. The client settles its own state when
the reader ends without a terminal event, but the server still owes it one.
---
## 7. Agentic Loop
The loop runs up to 10 iterations per request.
```
for each iteration:
1. Call provider.stream_chat(messages, tools, system_prompt, tool_executor)
2. Collect text_delta events -> stream to client
3. Collect tool_start events -> stream to client
4. Collect tool_result events (from internal execution) -> stream to client
5. If there are pending tool calls (not internally executed):
a. Execute each tool
b. Emit tool_result to client
c. Append assistant + tool results to message history
d. Continue loop
6. If no tool calls: break
```
After the loop, the assistant message (text + all tool calls with results) is
saved to the database and a `done` event is emitted.
---
## 8. REST API Endpoints
### Chat
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/repos/{repo_id}/chat/messages` | SSE stream — send a message and get a streaming response |
| `GET` | `/api/repos/{repo_id}/chat/conversations` | List conversations for a repo |
| `GET` | `/api/repos/{repo_id}/chat/conversations/{id}` | Get conversation with all messages |
| `DELETE` | `/api/repos/{repo_id}/chat/conversations/{id}` | Delete a conversation |
**POST body:**
```json
{
"message": "What does the auth module do?",
"conversation_id": null,
"provider": null,
"model": null
}
```
`conversation_id` — omit or `null` to start a new conversation.
`provider` / `model` — optional per-request overrides.
### Providers
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/providers` | List all providers with status and active selection |
| `PATCH` | `/api/providers/active` | Set active provider and model |
| `POST` | `/api/providers/{id}/key` | Store an API key |
| `DELETE` | `/api/providers/{id}/key` | Remove an API key |
---
## 9. Frontend Architecture
### API Layer (`src/lib/api/`)
- `chat.ts` — `listConversations`, `getConversation`, `deleteConversation`, `postChatMessage` (returns raw Response for SSE reading)
- `providers.ts` — `getProviders`, `setActiveProvider`, `addProviderKey`, `removeProviderKey`
### Hooks (`src/lib/hooks/`)
- `useChat(repoId)` — full chat state machine. Uses `fetch` + `ReadableStream` (not `EventSource`, which is GET-only). Manages messages, streaming state, conversation ID, error handling, and abort control. Exposes `sendMessage`, `loadConversation`, `reset`.
- `useProviders()` — SWR wrapper for provider management. Exposes `providers`, `activeProvider`, `activeModel`, `activate`, `saveKey`, `removeKey`.
### Components (`src/components/chat/`)
| Component | Purpose |
|-----------|---------|
| `ChatInterface` | Main container — empty state (greeting + suggestions + model selector) and active state (message list + input) |
| `ChatMessage` | Renders user bubble or assistant message (tool blocks + markdown) |
| `ChatMarkdown` | Client-side markdown renderer using `react-markdown` + `remark-gfm` with design token styling |
| `ToolCallBlock` | Inline tool call visualization — running (spinner), done collapsed (checkmark + summary), done expanded (input/output JSON) |
| `ArtifactPanel` | Right slide-in panel with tabs for multiple artifacts. Renders by type: markdown, Mermaid diagrams, search results, raw JSON |
| `ModelSelector` | Compact popover for switching provider/model and adding API keys inline |
| `ConversationHistory` | Dropdown listing past conversations with delete and new-conversation actions |
### Page Structure
The repo landing page (`/repos/[id]`) is the chat interface:
- Compact header (repo name + commit badge + branch badge)
- `ChatInterface` filling remaining viewport height
- Sidebar nav item updated from "Overview" to "Chat"
All other repo sub-pages (graph, wiki, coverage, etc.) are unchanged.
---
## 10. Provider-Specific Notes
### Anthropic
Uses `client.messages.stream()` with native Anthropic message format. Converts
OpenAI-format messages to Anthropic format (tool results as `user` role with
`tool_result` content blocks, tool calls as `tool_use` content blocks). The
agentic loop runs in the chat router.
### OpenAI
Uses `client.chat.completions.create(stream=True)`. Native OpenAI format —
minimal conversion needed. Tool call fragments are accumulated across stream
chunks and emitted as complete `tool_start` events. The agentic loop runs in
the chat router.
### Gemini
Uses `client.models.generate_content()` (non-streaming, in a thread pool).
**Runs the agentic loop internally** via the `tool_executor` callback to
preserve `thought_signature` on function call parts. Gemini's API requires
these signatures when replaying function calls in conversation history; the
OpenAI-format round-trip through the router would lose them. Native `Content`
objects are used throughout the internal loop.
### Ollama
Uses the OpenAI-compatible endpoint (`localhost:11434/v1`) via `AsyncOpenAI`.
Same streaming pattern as OpenAI. The agentic loop runs in the chat router.
### LiteLLM
Uses `litellm.acompletion(stream=True)`. OpenAI-compatible streaming. The
agentic loop runs in the chat router.
---
### Architecture/Code Health
# Code Health: Architecture & Internals
Companion to the user-facing [`docs/layers/CODE_HEALTH.md`](../layers/CODE_HEALTH.md). This
document is for contributors: where every piece lives, how data flows from
parsed source to the dashboard, and the extension points for adding
markers, languages, coverage formats, or alerts.
> **TL;DR.** Health analysis is a deterministic, zero-LLM Python pipeline:
> tree-sitter walks every file once -> markers vote -> scores aggregate per
> category -> results land in four SQLAlchemy tables. The MCP server, CLI,
> and Next.js dashboard all read from those tables: no JSON cache, no
> intermediate files, no LLM in the loop.
---
## 1. Layer overview
Code Health is the **fifth intelligence layer** in Repowise, alongside Graph,
Git, Docs, and Decisions. It reads from Graph and Git but never modifies
them. Its only writes are to its own four tables.
```
/* Detailed source-code truncated for AI context efficiency. */
```
Three architectural rules govern the whole layer:
1. **Zero LLM.** Every marker is AST, git, or coverage math.
2. **No JSON caches.** SQLite is the single source of truth; everything
else reads from it.
3. **No new runtime dependencies.** Pure Python over tree-sitter (already
in tree). No lizard, no jscpd, no Node.
---
## 2. Where things live
### Python: `packages/core/src/repowise/core/`
```
/* Detailed source-code truncated for AI context efficiency. */
```
### Persistence
```
core/persistence/
├── models.py # HealthFinding, HealthFileMetric, HealthSnapshot, CoverageFile
└── crud.py # save_/upsert_/get_ health functions
core/alembic/versions/
└── 000X_health_tables.py # migration that created the four tables
```
### Pipeline wiring
```
core/pipeline/
├── orchestrator.py # _run_health_analysis(): builds module_map, runs analyzer
└── persist.py # persist_pipeline_result(): writes findings/metrics/snapshot
```
### CLI
```
cli/src/repowise/cli/commands/
├── health_cmd.py # repowise health [--trend|--refactoring-targets|--module]
├── status_cmd.py # `Health: 7.4 (avg) · 6.2 (hotspots) · 2.1 (worst: ...)`
└── update_cmd.py # incremental path: HealthAnalyzer.analyze(changed_files=...)
```
### Server: MCP + REST
```
server/src/repowise/server/
├── mcp_server/
│ ├── tool_health.py # @mcp.tool get_health(targets, include, repo, limit)
│ ├── tool_risk.py # enriched: health_score, top_biomarkers, coverage_pct
│ ├── tool_context.py # include=["health"]: score, top 2 biomarkers, suggestion
│ └── tool_overview.py # code_health block with KPIs
└── routers/
└── code_health.py # /api/repos/{id}/health/{overview,files,coverage,
# refactoring-targets,modules,findings}
```
### Web dashboard
```
packages/ui/src/health/ # shared React components (used by web + future hosted frontend)
├── kpi-cards.tsx
├── file-table.tsx
├── biomarker-list.tsx
├── coverage-bar.tsx
├── module-coverage-list.tsx
├── untested-hotspot-warning.tsx
├── refactoring-card.tsx
├── refactoring-target-list.tsx
├── health-badge.tsx # score pill, colored by the 3 health bands
├── health-distribution-bar.tsx # NLOC-weighted Alert/Warning/Healthy split
├── trend-chart.tsx # repo KPI history (3 series)
├── file-trend-chart.tsx # single file's score-over-time + delta + declining flag
├── sparkline.tsx # compact inline series (drawer trend)
└── module-rollup-list.tsx
packages/web/src/app/repos/[id]/health/
├── page.tsx # KPIs + lowest-scoring files + per-module rollup
├── coverage/page.tsx # /health/coverage view
└── refactoring-targets/page.tsx # /health/refactoring-targets view
packages/web/src/components/health/
└── health-risks-panel.tsx # sidecar panel on Hotspots/Ownership/Graph pages
```
### Tests
```
tests/unit/health/ # 99+ tests
├── test_complexity_walker.py # per-language CCN/nesting assertions
├── test_biomarkers.py
├── test_structural_biomarkers.py # bumpy_road, large_method, primitive_obsession
├── test_coverage_biomarkers.py # untested_hotspot, coverage_gap
├── test_organizational_biomarkers.py
├── test_dry_violation.py
├── test_duplication.py # tokenizer, hash, detector
├── test_coverage_parsers.py # LCOV/Cobertura/Clover/JSON
├── test_scoring.py # category caps, clamping
├── test_scoring_snapshot.py # stability snapshot: locks caps + deductions
├── test_health_config.py # .repowise/health-rules.json
├── test_trends.py # diff_snapshots, declining/predicted alerts
├── test_signals.py # file_signals join + no-signal/normalization
├── test_churn_complexity.py # churn × complexity point shaping + sort + filtering
└── test_suggestions.py
tests/integration/
├── test_health_coverage_integration.py
└── test_health_perf_benchmark.py # 30 s budget on 3,000-file synthetic repo (slow)
```
---
## 3. The pipeline (init path)
`repowise init` runs `run_pipeline()` in `core/pipeline/orchestrator.py`.
`_run_health_analysis()` is a phase in that orchestrator, called between
`_run_dead_code_analysis()` and `_run_decision_extraction()`. It does four
things:
1. Builds a `{file_path: community label}` map from the graph's community
detection so `HealthFileMetric.module` is populated (module rollups are
never NULL).
2. Loads per-file override rules from `.repowise/health-rules.json` via
`HealthConfig.load(repo_path)` (a no-op when the file is absent) and
resolves them to per-file disabled sets with `to_analyzer_config()`.
3. Constructs the `HealthAnalyzer` with everything it needs: the NetworkX
graph (for dependents), `git_meta_map` (hotspot bit, owners, co-change, bus
factor), the `parsed_files` from the AST phase, and the module map.
4. Picks the sync or parallel path by repo size. tree-sitter releases the GIL
during parsing, so on repos with `>= 500` parsed files `analyze_async()`
(asyncio gather over worker threads) gives a real wall-clock speedup;
smaller repos run `analyze()` on a single thread.
The returned report rides on `PipelineResult.health_report`. Then
`core/pipeline/persist.py` writes it in one session: `save_health_metrics`,
`save_health_findings` (only when there are findings), and a
`save_health_snapshot` carrying the three KPIs plus a `{path: score}` map for
trend tracking (rolling 50-row window per repo), and a second
`{path: total_deduction}` map covering only the files whose score is held at
the floor. Both maps come from `trends.snapshot_file_maps`, which the other two
snapshot writers (`repowise health` and `repowise upgrade`) also call — a repo
whose writers disagreed would get a history whose depth changed depending on
which command last wrote it.
---
## 4. Inside `HealthAnalyzer.analyze()`
A single pass over the parsed file list. For each file the analyzer:
1. **Walks the AST** (`_walk`): `walk_file(language, source)` returns a
`FileComplexity` of functions and classes. Each `FunctionComplexity` carries
name, line range, nloc, ccn, max nesting, cognitive complexity, bumps, and
param count; each `ClassComplexity` carries method count, total nloc, the
method list, LCOM4, max method ccn, and field count.
2. **Populates symbol complexity** (`_populate_symbol_complexity`): writes
`max(ccn)` into `Symbol.complexity_estimate` as a side effect, so the
`ContextAssembler` symbol ranker benefits even when a caller never touches
the health tables.
3. **Evaluates the file** (`_evaluate_file`): builds a `FileContext` (nloc,
`has_test_file`, module, per-function and per-class metrics, the per-file
`git_meta`, graph in-degree as `dependents_count`, the repo-wide p80 of
in-degree used as the `brain_method` floor, coverage fields when ingested,
and the file's clone slice), runs `detect_all()`, scores with `score_file()`,
and attaches per-finding impacts.
After the loop, `compute_kpis()` runs over the metrics and the set of hotspot
paths (`git_meta_map[path]["is_hotspot"]`), and the analyzer returns a
`HealthReport(findings, metrics, kpis)`.
Duplication runs **once up-front** (it is cross-file by nature); each
`FileContext` gets a slice of the global clone report. The `dry_violation`
marker reads `ctx.clones` and ranks pairs by co-change frequency from
`git_meta_map[path]["co_change_partners_json"]`, so active clones rank higher
than dormant ones.
---
## 5. The markers and their categories
Each marker is a stateless class implementing the `Biomarker` Protocol from
`biomarkers/base.py`: a `name` (`"brain_method"`, `"nested_complexity"`, ...), a
`category` (see `scoring.CATEGORY_CAPS`), and a `detect(ctx: FileContext)` method
returning a list of `BiomarkerResult`s.
### The full roster
`biomarkers/registry.py` registers **49 detectors**; counting the three
governance findings written by the additive pass (`governance.py`) there
are **52 marker ids**. They divide by what each is permitted to affect:
| Group | Count | Scores into |
|---|---:|---|
| Defect-scoring | 26 | `defect` (8 of them also `maintainability`) |
| Performance | 20 | `performance` only |
| SQL | 3 | `maintainability` only |
| Governance | 3 | nothing — the finding surfaces, the score is untouched |
The authority is `scoring._BIOMARKER_DIMENSIONS`. Any biomarker **not** listed
there defaults into `defect`, which is why every `sql_*` and every performance
name must be listed explicitly: an omission would silently break the defect
golden guarantee (§6).
### Defect categories and caps
| Category | Cap | Markers |
|------------------------|------|------------|
| Organizational | −3.5 | developer_congestion, knowledge_loss, hidden_coupling, function_hotspot, code_age_volatility, ownership_risk, churn_risk, change_entropy, co_change_scatter, prior_defect, ungoverned_hotspot†, stale_governance†, contradictory_decision† |
| Structural complexity | −2.5 | brain_method, low_cohesion, god_class, nested_complexity, bumpy_road, complex_conditional |
| Test coverage | −2.0 | untested_hotspot, coverage_gap |
| Test coverage gradient | −2.0 | coverage_gradient |
| Size & complexity | −1.5 | complex_method, large_method, primitive_obsession |
| Duplication | −1.0 | dry_violation |
| Test quality | −0.5 | large_assertion_block, duplicated_assertion_block |
| Error handling | −0.5 | error_handling |
† The three governance markers carry a category and a weight, but the pass that
writes them runs *after* scoring completes and never touches
`HealthFileMetric.score` — so in practice they never deduct. They are counted
in the table above because `scoring.py` maps them, not because they move a
number.
The maintainability dimension has its own independent tables
(`_MAINTAINABILITY_CATEGORY`, caps: structural_complexity 4.0,
size_and_complexity 2.0, duplication 2.0, error_handling 2.0, sql 2.0), and the
performance dimension a single `performance` category capped at 2.0. See §6.
`large_assertion_block` and `duplicated_assertion_block` are the two
**test-quality** smells (see §5.3). They fire only on test files and sit in
a deliberately small category so a noisy test can't dominate its own score.
`large_method` is now gated on a minimal CCN floor (≥ 2) so a long-but-flat
body (a big data literal) reads as layout, not a complexity smell: a small
step toward decoupling the score from raw file size.
`ownership_risk` (long-run minor-contributor dispersion, Bird et al.) and
`churn_risk` (size-normalized relative churn, Nagappan-Ball) are git-only
process signals computed from `top_authors_json` / `lines_added_90d` /
`churn_percentile`: fields the git indexer already produces. `change_entropy`
(Hassan's History Complexity Metric) and `co_change_scatter` (breadth of
co-change coupling, D'Ambros) are likewise git-only and read the
`change_entropy` / `change_entropy_pct` fields (see §5.1) and
`co_change_partners_json`. `knowledge_loss` is activity-gated so
abandoned-but-stable files (the survivor effect) no longer fire.
`prior_defect` (recent bug-fix history, Ostrand-Weyuker / Kim's "bug cache")
is the other git-only process signal: the count of bug-fix commits touching a
file in the trailing ~6-month window, read from `prior_defect_count`. The
git indexer classifies a commit as a fix with the **same keyword rule the
defect benchmark labels fixes with** (`_constants.is_fix_commit`), counts only
non-merge commits inside the window, and anchors the window to the index's
`as_of` reference (`REPOWISE_GIT_WINDOW_ANCHOR`): so scoring a historical T0
checkout measures the fixes *before* T0, never leaking the post-T0 fixes that
form the benchmark's labels. It carries a **neutral (1.0) weight by design**:
on the calibration corpus prior-defect history is largely redundant with the
existing process signals (correlation ≈ +0.59 with `change_entropy`, +0.38 with
churn; calibrated coefficient ≈ +0.02, effort-aware Popt gain within bootstrap
noise), so it is not boosted as a predictor. It ships for its **explanatory**
value, not for a measured accuracy lift: "this file was bug-fixed N times
recently" is immediately actionable, and it uniquely flags a few files the
other signals miss.
`coverage_gradient` makes the test-coverage signal **continuous**. The two
binary coverage gates (`untested_hotspot`, `coverage_gap`) only fire below hard
thresholds (≈40–60% line coverage), so on a well-tested codebase (where most
files sit at 85–99%) the score is effectively blind to coverage even though the
uncovered fraction still carries defect signal. `coverage_gradient` deducts
health in direct proportion to that fraction: `4.0 × (1 − line_coverage_pct/100)`
health points, clamped by its category cap (binding at ≥50% uncovered). It uses
the `deduction` override on `BiomarkerResult` (a continuous magnitude that
replaces the discrete severity-to-deduction table for that finding) so it stays
**linear and per-finding attributable** (the `health_impact` contract holds). It
is **silent when no coverage report was ingested** (`line_coverage_pct is None`):
absent coverage is never imputed as uncovered. It lives in its own capped
category (`test_coverage_gradient`, −2.0) so the additive continuous signal
neither squeezes nor is squeezed by the binary gates, and it skips test files.
Calibrated offline against the defect corpus, it recovers **+0.043 corpus AUC
[95% CI +0.023, +0.061]** on the covered subset (≈65% of the continuous-feature
ceiling), Popt-neutral, and is exactly zero on repos without ingested coverage:
a purely additive improvement.
`low_cohesion` (LCOM4) and `god_class` are the two **class-level**
structural smells. They read `ctx.class_metrics`, the per-class aggregates
the walker now emits alongside per-function metrics (see §5.2).
`brain_method`'s centrality gate is **language-agnostic**: instead of a
fixed `dependents ≥ 8`, it fires when a file is in the repo's top quintile
of connected files (`repo_dependents_p80`, computed once per analyze) or
clears the absolute hub bar of 8: so it no longer goes silent on
sparse-graph languages (TS barrels, Rust) whose in-degrees are lower than
Python's.
### 5.1 Change-entropy git-layer fields
`change_entropy` is computed during the **single** FULL-tier co-change walk
(`ingestion/git_indexer/co_change.py::compute_co_changes_and_entropy`): no
extra `git log` subprocess. For each commit touching a set of tracked files
`F` (with `2 ≤ |F| ≤ 30`; wider commits are dropped as noise, Hassan's filter),
the commit's entropy is `log2(|F|)`, distributed uniformly (`1/|F|` per file)
and decayed with the same τ=180d half-life as co-change. The decay-weighted sum
per file is `git_meta["change_entropy"]`. `enrich.compute_percentiles` then
derives `change_entropy_pct` by ranking **only files with positive entropy**
(zero-entropy files, the ESSENTIAL tier or files only ever changed alone,
keep pct 0.0 so the marker stays silent). Both fields are persisted on
`git_metadata` (migration `0025`) and the additive-reconcile path back-fills
them on legacy DBs.
### 5.2 Class-level walker metrics (LCOM4)
The complexity walker emits a `ClassComplexity` per class-like node for
languages that opt in (`LanguageNodeMap.class_kinds` non-empty: Python,
TS/JS, Java, Kotlin, Rust `impl`, C++, C#; Go has no grouping node). LCOM4 is the number of
connected components in the graph whose nodes are the class's methods and
whose edges link methods that share an instance field or call one another.
Member references are detected per-language via `self`/`this`/`$this`
member-access nodes. **Safety valve:** a class with no detected member
references (a static utility, or an unmapped language) reports `lcom4 = 1`
("no signal") rather than `len(methods)`, so adding a language can only
turn signal on, never produce a false-positive flood. See
`complexity/README.md` for the full heuristic and its limits.
`error_handling` is the **advisory maintainability** marker: swallowed
catches (an empty/comment-only `catch` / `except: pass` body), Python
catch-all `except:` / `except Exception:`, Rust `.unwrap()` / `.expect()` /
panic-family macros, and Go's empty `if err != nil {}` or blank-identifier
discard of a call's error. The walker collects each occurrence (with its
line) in a whole-tree pass (module-level code included) reusing the
`LanguageNodeMap` catch kinds for the seven catch-shaped languages and
dedicated recognizers for Rust/Go; an unsupported language or parse failure
yields no hits ("no signal", never a guess). The marker emits one LOW
finding per occurrence (0.15 after its floored 0.5 weight) in its own
`error_handling` category capped at −0.5, mirroring `test_quality`'s
advisory framing. It is deliberately **excluded from the defect-weight
calibration**: on the 21-repo / 9-language T0 benchmark it is AUC-neutral
(OOF delta ≈ 0, CI crosses zero) but size-orthogonal and the least redundant
signal tested, and it ships because users expect `except: pass` flagged:
bounded so it can never move a file by more than half a point.
### 5.3 Assertion-block walker metrics (test-quality)
The same single walker pass records `assertion_blocks` on each
`FunctionComplexity`: runs of ≥ 2 consecutive assertion statements, each
`(start_line, end_line, count)`. A statement counts as an assertion when it
is a bare `assert` (`LanguageNodeMap.assert_kinds`) or its expression is a
call (`assert_call_kinds`) whose callee name starts with `assert` or
`expect`: covering `assertEqual` / `assert_eq!` / `expect(...).toBe(...)`
across xUnit and BDD styles. Opt-in per language, with all nine full-tier
languages (Python, TS/JS, Java, Kotlin, Go, Rust, C++, C#) mapped;
a language that maps neither field simply emits no blocks.
`large_assertion_block` flags a single run ≥ 15; `duplicated_assertion_block`
intersects the clone report with assertion spans. Both gate on
`coverage.is_test_file(path)` so production code is never touched.
`biomarkers/registry.py` is an **explicit list**, not auto-discovery:
keeps the registration order deterministic and lets tests inject extras
via `registered_biomarkers(extra=...)`.
---
## 6. Scoring (`scoring.py`)
Every file starts at **10.0**. Each finding contributes a per-severity
deduction (`low=0.3, medium=0.7, high=1.2, critical=2.0`), scaled by the
marker's calibrated weight multiplier (§6.1). `score_file()` then groups the
weighted findings by category, sums the raw deductions per category, and either
accepts the sum or, when it exceeds the cap, scales every per-finding deduction
in that category proportionally so the total equals the cap. This keeps the
UI's "this finding cost you X points" honest after capping. The final score is
clamped to `[1.0, 10.0]`. So even ten critical structural findings can drive
structural complexity down by at most 3.5 points, not 20.
The per-finding scaled deduction lands on `HealthFinding.health_impact`
via `attach_impacts()`: that's what the dashboard's "−2.0" badge shows.
Snapshot tests in `tests/unit/health/test_scoring_snapshot.py` lock the
category caps, severity deductions, marker-to-category mapping, and two
known-fixture scores. A retune intentionally requires updating the
snapshot in the same PR.
### 6.1 Calibrated weight multipliers
`scoring._BIOMARKER_WEIGHT_MULTIPLIER` lets the strongest empirical predictors
deduct more than the uniform severity table alone allows. The multipliers are
**calibrated offline against a defect corpus, not hand-tuned**: each file is
scored at the pre-window commit (T0, no leakage) and an L2-logistic regression,
with NLOC as an explicit control, fits each marker's defect lift beyond file
size. The runtime stays deterministic; only the learned constants ship. The
full calibration, with confidence intervals, is published in the
[benchmark report](https://github.com/repowise-dev/repowise-bench/blob/master/health-defect/BENCHMARK_REPORT.md)
and reproduced by `local-stash/calibrate_health_weights.py`.
| Weight | Markers | Rationale |
|---|---|---|
| 1.8 | `co_change_scatter` | Strongest calibrated predictor. |
| 1.51 | `change_entropy` | History Complexity Metric; second strongest. |
| 1.38 | `ownership_risk` | Long-run minor-contributor dispersion. |
| 1.34 | `nested_complexity` | Strongest structural predictor. |
| 1.1–1.33 | remaining structural complexity / size markers | Moderate calibrated lift. |
| 1.3 / 1.2 / 1.1 | `untested_hotspot` / `churn_risk` / `code_age_volatility` | Coverage-dependent and rarely-firing; keep prior weights the corpus could not fairly measure. |
| 1.0 | `prior_defect` | Neutral by design: largely redundant with the other process signals, kept for its explanatory value. |
| 0.5 (floored) | `developer_congestion`, `dry_violation`, `low_cohesion`, `brain_method`, `primitive_obsession`, `bumpy_road` | Fire widely but proved weak under leakage-free scoring; kept as maintainability and parity signals, not disabled. |
| 0.4 (de-rated) | `knowledge_loss` | Weakest of the floored set. |
The same marker stream feeds the **maintainability** signal under an
independent, expert-set weight table (the floored smells deduct at full weight
there), and the **performance** signal under its own bounded `performance`
category. The three signals share one scoring kernel against separate
weight/category/cap tables and never feed back into each other; see the
[user guide](../layers/CODE_HEALTH.md#three-health-signals-defect-risk-maintainability-and-performance)
for what each signal surfaces and why the overall score stays the defect score.
---
## 7. KPIs
Three repo-level numbers, computed in `compute_kpis()`:
- **Hotspot Health**: NLOC-weighted average over files where
`git_meta_map[path]["is_hotspot"]` is true.
- **Average Health**: NLOC-weighted average over all files.
- **Worst Performer**: lowest-scoring file + its score.
These flow into `HealthSnapshot` rows (rolling 50 per repo) and feed the
CLI status one-liner, the `get_overview()` MCP block, and the dashboard
KPI cards.
---
## 8. Trends (`trends.py`)
State-free: callers pass an oldest-first list of snapshot rows. Two
alerts:
- **Declining Health**: current is ≥ `DECLINE_THRESHOLD` (default 0.5)
below the snapshot `DECLINE_LOOKBACK` (5) positions back. Fires on the
6th+ snapshot.
- **Predicted Decline**: the three most recent snapshots are each
strictly below the one before. Magnitude is not required; direction is
the signal.
`recent_kpis(history, limit=10)` returns a newest-first serialised view
for the CLI table and MCP `get_health(include=["trend"])` response.
### Per-file trajectory
Snapshots also store a compact `{path: score}` map (`per_file_scores_json`),
so the same window yields a single file's score-over-time series:
- `file_score_series(history, path)`: oldest-first `FileTrendPoint`s,
skipping snapshots that don't carry the file. Returns `[]` below two
points (silent on thin history). This is the exact function the PR bot
reuses for its in-comment sparkline.
- `file_trend(history, path)`: wraps the series with `current` / `previous`
/ `delta` and a `declining` flag (the per-file mirror of the alerts above:
≥ `DECLINE_THRESHOLD` below the lookback point, or
`PREDICTED_DECLINE_CONSECUTIVE` consecutive drops). `snapshot_count` is the
full window size so a young repo is distinguishable from a file missing in
older snapshots.
Both are state-free; the server serialises `FileTrend` via
`_file_trend_to_dict` and embeds it in the file-detail health block, the
health-breakdown response, and the standalone trend route (§13).
#### Below the floor
The stored score is clamped to `[SCORE_FLOOR, SCORE_MAX]`, so files 12.9 and
9.1 points deep both persist as `1.0` and their series is flat however much of
the work gets done. The second snapshot map (`per_file_deductions_json`) keeps
the pre-clamp deduction for exactly those files — for every other file the
deduction is `SCORE_MAX - score`, so there is nothing to store.
Each point therefore carries `unclamped_score` alongside `score`:
`SCORE_MAX - deduction` where the snapshot recorded one, and `score` otherwise.
It is the series `_file_declining` runs on, so `declining` describes the line
that can actually move, and a file getting worse *below* the floor now trips it.
Snapshots written before this existed have no deduction map. Their floored
files stay flat, which is correct — the depth was never measured, and inventing
one would be worse than a flat line.
### Per-file signals (`signals.py`)
The same state-free pattern, applied to the git-layer + topology fields we
already persist but only buried inside marker detail cards (or omitted
entirely). `file_signals(git_meta, degrees)` joins one `GitMetadata` row with
the file's graph degree into a `FileSignals` grouped as **Process**
(`prior_defect_count`, `change_entropy_pct` normalized 0-100, 90-day line
churn, `age_days`), **People** (recent vs all-time owner + commit share), and
**Topology** (`in_degree` / `out_degree`). No recompute: pure surfacing.
The honesty rule mirrors the trend: a field is `None` only when its *source
row* is absent (no git history means process/people silent; not a graph node
means topology silent), never imputed; a genuine `prior_defect_count` of `0` is kept
as a real signal. The server serialises it via `_file_signals_to_dict` and
embeds it in the file-detail health block and the breakdown response (§13); MCP
attaches a null-dropped copy to the `get_context` health block (§12). Mirrored
as `FileSignals` in `@repowise-dev/types/health`; rendered by the shared
`file-signals-panel.tsx` in both the drawer and the file-page Health tab.
---
## 9. Incremental analysis: the `repowise update` path
Full re-analysis would be wasteful on commit-sized diffs, so
`HealthAnalyzer.analyze()` accepts a `changed_files` set. When it is present:
duplication still runs full-repo (a changed file's clone partner may be
unchanged); the per-file loop skips files not in `changed_files`; and the KPIs
are **not** recomputed, since the subset would bias them. The dashboard
recomputes KPIs from the merged DB rows instead.
`update_cmd.py` builds the changed-files set from
`change_detector.get_changed_files()`, runs the analyzer, and persists through a
helper that uses the **upsert** variants (`upsert_health_metrics`,
`upsert_health_findings`, scoped to the changed paths) so unchanged files keep
their existing rows. The full-init writers (`save_health_findings`,
`save_health_metrics`) still use delete-then-insert: simpler, and the cost is
amortised across the whole `repowise init`.
---
## 10. Persistence schema
Four tables, all in the repo's `.repowise/wiki.db`. Foreign-keyed to
`repositories.id` with `ON DELETE CASCADE`.
### `health_findings`
One row per marker hit. Lifecycle: `open → acknowledged | resolved |
false_positive` (matches Dead Code). Bulk-deleted-and-rewritten on full
init; selectively upserted on `repowise update`.
| Column | Notes |
|---|---|
| `id` | UUID PK |
| `repository_id` | FK |
| `file_path` | indexed |
| `biomarker_type` | `brain_method`, `nested_complexity`, ... |
| `severity` | `low` / `medium` / `high` / `critical` |
| `function_name` | nullable for file-level findings |
| `line_start`, `line_end` | nullable |
| `details_json` | per-marker evidence (CCN values, clone span, etc.) |
| `health_impact` | per-finding scaled deduction |
| `reason` | one-line summary string |
| `status` | lifecycle |
| `created_at`, `updated_at` | datetime |
### `health_file_metrics`
One row per file (unique on `(repository_id, file_path)`). Read directly
by the dashboard's file table.
| Column | Notes |
|---|---|
| `score` | 1.0–10.0 final |
| `max_ccn`, `max_nesting`, `nloc` | aggregate function metrics |
| `duplication_pct` | percent of NLOC covered by clones; nullable |
| `has_test_file` | paired or heuristic |
| `line_coverage_pct`, `branch_coverage_pct` | nullable |
| `module` | community label from graph; falls back to top-level dir |
| `updated_at` | datetime |
### `health_snapshots`
KPI + per-file score history. Rolling delete on insert keeps the latest
50 per repo (`HEALTH_SNAPSHOT_RETENTION` in `crud.py`).
### `coverage_files`
Per-file coverage, overwritten on every `coverage add` run. Carries the
explicit `covered_lines_json` array so the `coverage_gap` marker can
flag the exact uncovered surface, not just the percent.
---
## 11. CLI surface
`packages/cli/src/repowise/cli/commands/health_cmd.py`. Mirrors the
dead-code command's Click structure.
```bash
repowise health # KPIs + lowest-scoring files + findings
repowise health --file path/to/x.py # deep-dive one file
repowise health --module packages/server # restrict to a directory prefix
repowise health --refactoring-targets # ranked by impact / effort
repowise health --trend # last 10 snapshots + active alerts
repowise coverage add coverage.lcov # ingest coverage; can repeat
repowise coverage add coverage.xml --format cobertura
repowise health --format json | jq ...
```
`repowise status` queries the same tables for a one-line summary:
```
Health: 7.4 (avg) · 6.2 (hotspots) · 2.1 (worst: packages/server/.../app.py)
```
`repowise update` is unchanged from the user's perspective: health is
silently re-scored for changed files only.
---
## 12. MCP surface
### `get_health(targets?, include?, repo?, limit?)`
Defined in `tool_health.py`. Modes:
- **Dashboard mode** (`targets=None`): returns repo-level KPIs (with the
repo `band`) + the NLOC-weighted `distribution` across the 3 bands +
`worst_files` (top N lowest-scoring) + `top_findings` + a per-module
`modules` rollup.
- **Targeted mode** (`targets=[...]`): returns full `metrics` +
`findings` for the listed paths, plus a per-file `trends` block (compact
score series + `current` + `delta` + `declining`) for any target with at
least two snapshots of history. Targets prefixed `module:foo` expand to
the file set in that module.
`include` flags layer richer data:
| Flag | Adds |
|---|---|
| `"biomarkers"` | full findings list (already present in target mode) |
| `"coverage"` | per-file coverage rows + summary |
| `"refactoring"` | deterministic `suggestion` text on every finding |
| `"trend"` | snapshot diff + alerts + last 10 KPI rows |
### Enrichments on existing tools
- `get_risk(targets)`: each per-target row carries `health_score`,
`top_biomarkers`, `coverage_pct`, `branch_coverage_pct`.
- `get_context(targets, include=["health"])`: per-file `score`,
`max_ccn`, `max_nesting`, `nloc`, `module`, `duplication_pct`, top
2 markers (each with a `suggestion` string), a coverage block, and a
null-dropped `signals` block (process/people/topology, see §8).
- `get_overview()`: adds a `code_health` block: avg, repo `band`, hotspot,
worst performer, open finding count, and the NLOC-weighted `distribution`.
Every response carries the standard `_meta` envelope via `build_meta()`.
---
## 13. REST surface
`packages/server/src/repowise/server/routers/code_health.py`. All under
`/api/repos/{repo_id}/health/`:
| Route | Returns |
|---|---|
| `GET /overview` | summary (with repo `band`) + `distribution` + lowest-scoring files + top findings + module rollup |
| `GET /badge.svg` | self-rendered flat SVG health badge (color + `N.N/10`, no letter) |
| `GET /badge.json` | Shields.io endpoint-badge payload (`schemaVersion`/`label`/`message`/`color`/`band`) |
| `GET /files` | per-file metrics |
| `GET /files/breakdown` | one file's metric + score breakdown + findings + suggestions + per-file `trend` + `signals` |
| `GET /files/trend` | one file's score-over-time series + current delta + `declining` flag (`?file_path=`) |
| `GET /trend` | repo KPI history + alerts + last-two-snapshot per-file deltas |
| `GET /findings` | findings list (filterable by biomarker_type, severity, file_path) |
| `GET /coverage` | coverage summary + per-file rows |
| `POST /coverage` | ingest a coverage report (used by some CI integrations) |
| `GET /refactoring-targets` | ranked by `total_impact / effort_bucket` |
| `GET /churn-complexity` | churn × complexity scatter points (one per churned file: `commit_count_90d`, `max_ccn`, `nloc`, `score`, `churn_percentile`) |
| `GET /modules` | NLOC-weighted module rollup table |
Auth is the standard `verify_api_key` dependency from
`server/deps.py`.
---
## 14. Web dashboard
Three routes under `/repos/[id]/health/`:
| Route | What it shows |
|---|---|
| `/health` | KPI cards, lowest-scoring file table, top findings, **per-module rollup** (added in Phase 4) |
| `/health/coverage` | Coverage summary, untested-hotspot warnings, module-level bars, per-file drill-down |
| `/health/refactoring-targets` | Cards sorted by impact-per-effort, each with severity, marker, score, NLOC, effort bucket, **deterministic suggestion** |
Plus a sidecar `HealthRisksPanel` on the Hotspots, Ownership, and Graph
pages: surfaces the lowest-scoring files inline without touching the
shared table/graph components. The **Hotspots & churn** tab carries the
`ChurnComplexityQuadrant` (fed by `GET /churn-complexity`), toggleable in
place with the existing churn × bus-factor scatter; the file Health tab
carries the per-function "Functions by churn" blame table.
All visual primitives live in `packages/ui/src/health/` so the hosted
`frontend/` repo (separate git checkout) can reuse them: port is mostly
data fetching + auth.
---
## 15. CLAUDE.md integration
The auto-generated `CLAUDE.md` includes a `## Code health` section when
the health tables are populated. The block is intentionally short;
filter rules in `core/generation/editor_files/data.py`:
- Score ≤ 5.0 **and** file is a hotspot
- Any Brain Method in a file with > 10 dependents
- Any Untested Hotspot
- DRY violations > 70 % similarity
- Declining trend (> 1.0 drop in last 5 snapshots)
Everything else is filtered out so the CLAUDE.md doesn't drown a fresh
agent in noise. The Jinja stanza lives in
`core/generation/templates/claude_md.j2`.
---
## 16. Configuration: `.repowise/health-rules.json`
`.repowise/health-rules.json` is user-authored (the **only** JSON file in the
layer) and is loaded by `HealthConfig.load(repo_path)`. It carries repo-wide and
per-path `disabled_biomarkers` and `severity_overrides`, keyed by an fnmatch
glob over the repo-relative POSIX path (`path`, with `path_glob` and `glob` as
accepted aliases). `to_analyzer_config(file_paths)` resolves the globs to
per-file disabled sets, which the engine honors in `_evaluate_file()`. The
schema and examples are in the
[user guide](../layers/CODE_HEALTH.md#configuration).
---
## 17. Performance
Plan §4 P4.6 targets **< 30 s on a 3,000-file synthetic repo**. The
parallel path in `HealthAnalyzer.analyze_async()` parallelises tree-sitter
parsing across worker threads (`asyncio.gather` + `asyncio.to_thread`).
tree-sitter releases the GIL on parse, so this scales on single-process
CPython.
The orchestrator chooses the parallel path automatically when
`len(parsed_files) >= 500`. The benchmark lives at
`tests/integration/test_health_perf_benchmark.py` and is marked `slow`
(opt-in via `pytest -m slow` or `make health-bench`).
Other perf notes:
- **Duplication is O(total_tokens).** Bucket walk is near-linear on
repos with low duplication.
- **Walker re-parses files** because `ParsedFile` doesn't retain a
tree-sitter `Tree` across the ingestion boundary. Acceptable (~1 ms
per file); switching to a shared parse cache is a Phase 5 stretch.
- **No N² loops in scoring.** Category aggregation is O(findings).
---
## 18. Testing
| Suite | What it locks |
|---|---|
| `tests/unit/health/test_complexity_walker.py` | Per-language CCN, nesting, cognitive assertions on handcrafted fixtures |
| `tests/unit/health/test_.py` | Each marker: positive in two languages + one negative |
| `tests/unit/health/test_duplication.py` | Tokenizer normalization, rolling-hash determinism, co-change weighting |
| `tests/unit/health/test_coverage_parsers.py` | LCOV / Cobertura / Clover / repowise-JSON happy paths + edge cases |
| `tests/unit/health/test_scoring.py` | Deduction caps, clamping, KPI math |
| `tests/unit/health/test_scoring_snapshot.py` | **Stability guard**: caps, severity table, marker-to-category mapping, two known fixture scores |
| `tests/unit/health/test_trends.py` | Declining + predicted alerts, ordering, per-file series + `file_trend` |
| `tests/unit/health/test_signals.py` | `file_signals` join: no-signal vs real-zero, entropy 0-1 to 0-100, owner handoff |
| `tests/unit/health/test_churn_complexity.py` | `churn_complexity_points`: no-churn omission, complexity never filters, danger-product sort, percentile scaling |
| `tests/unit/health/test_suggestions.py` | Suggestion strings keyed correctly |
| `tests/unit/health/test_health_config.py` | `.repowise/health-rules.json` parsing + glob matching |
| `tests/integration/test_health_coverage_integration.py` | End-to-end LCOV -> analyzer -> coverage_gap fires |
| `tests/integration/test_health_perf_benchmark.py` | 30 s budget on 3,000 synthetic files (`-m slow`) |
99 unit tests + 2 integration tests at time of writing. Run with
`make health-check`.
---
## 19. Extension points
### Add a marker
1. New file under `biomarkers/` implementing the `Biomarker` Protocol.
2. Append to `_DETECTOR_FACTORIES` in `biomarkers/registry.py`.
3. Add the marker-to-category mapping in
`scoring._BIOMARKER_CATEGORY`.
4. Add a suggestion template in `suggestions._TEMPLATES`.
5. Add at least three test cases (two positive in different languages,
one negative).
6. Update `biomarkers/README.md`'s "Registered v1 detectors" list.
### Add a language to the complexity walker
Add one `LanguageNodeMap` entry to `complexity/languages.py` mapping the
language's tree-sitter control-flow node-type names to abstract `BRANCH`
/ `LOOP` / `TRY` / `BOOLEAN_OP` categories. Add a fixture under
`tests/fixtures/lang_samples//`. **No `.scm` files needed**: those
are owned by the ingestion parser.
### Add a coverage format
Drop a parser under `coverage/` returning a `CoverageReport`. Route to it
from `coverage/detector.parse`. Stdlib-only (no extra XML libraries).
### Add a per-file override
Users (not contributors) author `.repowise/health-rules.json`. To add
a new override key (beyond `disabled_biomarkers`), extend
`HealthConfig` and thread it through `to_analyzer_config()` ->
`engine._evaluate_file()`.
---
## 20. Where the layer deliberately stops
A short list of things the v1 layer **does not** do, by design. Future
phases may revisit; the constraints kept v1 shippable.
- **No LLM-generated suggestions.** `suggestions.py` is static
templates. An optional LLM mode is Phase 5, gated behind an explicit
flag.
- **No symbol-level scoring.** Score lives at the file granularity to
match how engineers think about refactor units. Symbol-level CCN
still feeds the file score via `function_metrics`.
- **No `complexity_estimate` propagation backfill.** The walker writes
the field as a side effect during the current run; old indexes don't
get touched until a re-index.
- **No predictive ML on trends.** `Predicted Decline` is a 3-snapshot
direction check, not a model. (Commit-level change risk is a separate,
shipped surface: the `analysis/change_risk/` package behind
`repowise risk` scores a commit or base..head range with a calibrated
logistic model.)
- **No letter grade.** The 1–10 score is the single number. The only
categorical layer is the 3 defect-backed bands (Healthy/Warning/Alert,
`grading.py`); a letter on top would be a third overlapping scale with
arbitrary cliffs. The legacy 4-step `scoreBand` in `ui/health/tokens.ts`
is retained only as a finer color ramp for file-table pills, not a
labeling scheme: surfaced band labels and the distribution use the 3
bands.
---
## 21. Quick lookup: where do I edit X?
| I want to... | Edit... |
|---|---|
| Tweak a category cap | `scoring.CATEGORY_CAPS` (snapshot test will fail; update it) |
| Tweak a severity deduction | `scoring._SEVERITY_DEDUCTION` (ditto) |
| Add a new marker | `biomarkers/*.py`, `registry.py`, `scoring.py`, `suggestions.py` |
| Change the suggestion text for a marker | `suggestions._TEMPLATES` |
| Adjust the trend-alert threshold | `trends.DECLINE_THRESHOLD` / `DECLINE_LOOKBACK` |
| Change snapshot retention | `crud.HEALTH_SNAPSHOT_RETENTION` |
| Add a new MCP `include` flag | `tool_health.py`: append handling near the existing `"coverage"` / `"refactoring"` branches |
| Add a new REST route | `routers/code_health.py`: auth is wired at the router level |
| Add a new dashboard view | new file under `packages/web/src/app/repos/[id]/health/`, primitives under `packages/ui/src/health/` |
| Add a CLI flag | `packages/cli/src/repowise/cli/commands/health_cmd.py` |
| Wire the analyzer into a new entry point | call `HealthAnalyzer.analyze()` directly; persist via the upsert variants if your caller is incremental |
---
## See also
- [`docs/layers/CODE_HEALTH.md`](../layers/CODE_HEALTH.md): user-facing guide.
- [`packages/core/src/repowise/core/analysis/health/README.md`](../../packages/core/src/repowise/core/analysis/health/README.md): developer overview at the layer root.
- Sub-package READMEs under `complexity/`, `coverage/`, `duplication/`, `biomarkers/`.
- [`docs/architecture/graph-algorithms.md`](graph-algorithms.md): the graph layer health depends on.
---
### Architecture/Deep Dives
# Repowise Deep Dives — Complete Guide
This document covers systems that are referenced but not fully explained in `architecture-guide.md` and `graph-algorithms-guide.md`. Each section is self-contained with full intuition, implementation details, and the math behind the algorithms.
---
## Table of Contents
1. [Dead Code Detection](#1-dead-code-detection)
2. [Decision Records (ADR) System](#2-decision-records-adr-system)
3. [Search and Vector Store Internals](#3-search-and-vector-store-internals)
4. [Incremental Updates and Webhooks](#4-incremental-updates-and-webhooks)
5. [Change Cascade Algorithm](#5-change-cascade-algorithm)
---
## 1. Dead Code Detection
### What problem does it solve?
Every codebase accumulates files and functions that nothing uses anymore. A refactor removes the last caller of `old_parser.py` but nobody deletes the file. Over months, these dead files pile up — increasing maintenance burden, confusing new developers, and inflating CI times.
Repowise's dead code analyzer finds these automatically using **pure graph traversal + git metadata**. No LLM calls. Runs in under 10 seconds.
### The four detection strategies
#### Strategy 1: Unreachable Files
**Question:** "Is anything importing this file?"
**Algorithm:**
```
For each node in the dependency graph:
Skip if: external package, non-code language, test file, fixture directory
if not is_file_reachable(node):
Nothing can get to this file → candidate for dead code
```
`is_file_reachable` is the single predicate for "can anything get to this file",
shared with the repo-overview assembler so the two cannot disagree. It rescues
entry points, API contracts, never-flag paths (`__init__.py`, config files,
migrations, shell scripts), bundler-alias shims and the package-granular
languages, and otherwise asks whether any *dependency* edge points at the file.
That last part is not a raw `in_degree`: a co-change edge ("these two files were
committed together"), a file's own symbols and a self-import are all excluded.
**Why an import edge alone isn't enough:**
Consider `plugin_auth.py`. Nothing imports it directly because the plugin framework loads it dynamically at runtime via `importlib.import_module()`. The graph doesn't capture dynamic imports as edges because they don't appear in the AST as static import statements.
Repowise handles this with multiple layers of filtering, including **dynamic import detection** — files in the same package as `importlib.import_module()` or `__import__()` calls automatically get reduced confidence scores:
**Layer 1 — Structural exclusions** (never flagged):
| Exclusion | Why |
|-----------|-----|
| Entry points (`main.py`, `index.ts`, `app.py`) | They're where execution starts, nothing imports them |
| Test files | Tests import production code, not the other way around |
| `__init__.py` | Package initializers, loaded by Python automatically |
| Config files (`setup.py`, `next.config.js`, `vite.config.ts`) | Loaded by frameworks |
| Migrations (`*migrations*`) | Run by migration tools, not imported |
| Schema/seed files | Data definitions, not code |
| Fixture directories (`fixtures/`, `testdata/`, `sample_repo/`) | Test data, not production code |
| Non-code languages (JSON, YAML, Markdown, SQL, Terraform) | No import semantics |
| API contracts (proto, graphql marked `is_api_contract`) | Consumed by code generators |
**Layer 2 — Dynamic pattern matching:**
```python
_DEFAULT_DYNAMIC_PATTERNS = (
"*Plugin", # Plugin discovery systems
"*Handler", # Event handler registration
"*Adapter", # Adapter patterns
"*Middleware", # Middleware chains
"*Mixin", # Mixin classes
"*Command", # CLI/management commands
"register_*", # Registration functions
"on_*", # Event callbacks
"*_view", # Django/Flask views
"*_endpoint", # API endpoints
"*_route", # Route handlers
"*_callback", # Callback functions
"*_signal", # Signal handlers
"*_task", # Background tasks
)
```
Files matching these patterns aren't marked `safe_to_delete` even if confidence is high, because they're likely loaded dynamically.
**Layer 2b — Framework decorator awareness:**
Functions decorated with framework-specific route/endpoint decorators are never flagged as dead code:
- Flask: `@app.route`, `@blueprint.route`, `@app.before_request`, `@app.errorhandler`, etc.
- FastAPI: `@app.get`, `@app.post`, `@router.get`, `@app.on_event`, `@app.middleware`, etc.
- Django: `@admin.register`, `@receiver`, `@login_required`, etc.
**Layer 2c — Dynamic import detection:**
Files in the same package as `importlib.import_module()` or `__import__()` calls automatically receive a reduced confidence score (capped at 0.4), since they may be loaded dynamically at runtime.
**Layer 3 — Confidence scoring with git metadata:**
This is where it gets interesting. A file with zero importers might be dead, or it might be actively used via dynamic loading. Git history helps distinguish:
```
if no_commits_in_90_days AND last_commit_over_364_days_ago:
confidence = 1.0 # Almost certainly dead — untouched for a year+
elif no_commits_in_90_days AND last_commit_over_179_days_ago:
confidence = 0.9
elif no_commits_in_90_days AND last_commit_over_89_days_ago:
confidence = 0.8
elif no_commits_in_90_days AND file_under_30_days_old:
confidence = 0.55 # Recently created — may be work in progress
elif no_commits_in_90_days:
confidence = 0.7 # No recent activity, and no commit date to age it by
else: # has recent commits
confidence = 0.4 # Suspicious but uncertain
reasoning: Nobody imports it BUT someone is actively changing it
(maybe dynamically loaded, maybe a script run manually)
```
The rungs are an evidence scale, not tier boundaries. The high/medium tier cuts
are `SAFE_CONFIDENCE_THRESHOLD` and `RISK_CAP_CONFIDENCE` in
`core/analysis/dead_code/risk_factors.py`, which every comparison reads by name.
**Intuition:** If a file is truly dead, it stops receiving commits. Active files — even dynamically loaded ones — still get bug fixes and updates. The combination of in_degree=0 (structural signal) and no-recent-commits (behavioral signal) gives high confidence.
**safe_to_delete computation:**
```
safe_to_delete = (confidence >= SAFE_CONFIDENCE_THRESHOLD) # 0.7
AND (not matches_dynamic_patterns)
AND (no runtime-load risk factors on the path)
```
A file is only marked safe to delete if we're confident it's dead, it doesn't look like a plugin/handler/adapter that might be dynamically loaded, and its path doesn't look like config / bootstrap / database / environment / script / runtime-asset code. That last condition is re-derived at read time by `effective_safe_to_delete`, so a finding persisted before the risk factors existed is still evaluated against them.
#### Strategy 2: Unused Exports
**Question:** "Is this public function/class imported by anything?"
This is more granular than unreachable files. A file might be imported, but specific exports within it might be unused.
**Algorithm:**
```
For each file in the graph:
Skip if: external, non-code, test, fixture, never-flag pattern
For each PUBLIC symbol in the file:
Skip if: has framework decorator (pytest.fixture, pytest.mark)
Skip if: matches dynamic pattern (*Handler, register_*, etc.)
has_importers = False
For each file that imports this file (predecessors):
Check edge's imported_names list
if symbol_name in imported_names OR "*" in imported_names:
has_importers = True
break
if not has_importers:
→ This export is unused
```
**Edge data is key here.** Each edge in the graph stores `imported_names` — the specific names imported across that edge. For example:
```python
# In service.py:
from auth import login, validate_token
# Edge: service.py → auth.py, imported_names = ["login", "validate_token"]
```
If `auth.py` also exports `reset_password` but no edge's `imported_names` includes it, then `reset_password` is an unused export.
**Confidence scoring for unused exports:**
```
if symbol_name ends with _DEPRECATED, _LEGACY, or _COMPAT:
confidence = 0.3 # Already marked as legacy by developer
elif file_has_other_importers: # file is used, but this symbol isn't
confidence = 1.0 # Very suspicious — file is active but this export isn't
else: # file itself has no importers
confidence = 0.7 # File and symbol both unused
```
**Why the file-imported distinction matters:**
If `auth.py` is imported by 10 files but none of them import `reset_password`, that's a strong signal — developers actively use this file but skip this function. Confidence = 1.0.
If `auth.py` itself has zero importers, the unused export is less interesting — the whole file is dead, and the unused export is just a consequence. Confidence = 0.7.
**safe_to_delete for exports:**
```
safe = (confidence >= 0.7) AND (complexity_estimate < 5)
```
Low-complexity symbols (simple functions, constants) are safer to remove than complex ones that might have non-obvious side effects.
#### Strategy 3: Unused Internals
**Question:** "Is this private function called within its own file?"
**Status: Not implemented** (returns empty list). The comment says "Higher false positive rate — off by default."
**Why it's hard:** Private functions might be called via string dispatch, decorators, metaclasses, or closures that the AST parser doesn't trace. False positives here are more disruptive than for public symbols because developers expect private functions to be internal and are less likely to question the analyzer.
#### Strategy 4: Zombie Packages
**Question:** "Does any other package in this monorepo actually use this package?"
**Algorithm:**
```
# Group files by top-level directory (= package)
packages = group_by(all_files, first_path_segment)
# Only applies to monorepos (2+ packages)
if len(packages) < 2: return []
For each package:
has_external_importers = False
For each file in this package:
For each predecessor (file importing this one):
if predecessor is from a DIFFERENT package:
has_external_importers = True
break
if not has_external_importers:
→ This package is a zombie (nothing outside it uses it)
```
**Example:**
```
packages/
├── auth/ # imported by api/ and cli/
│ ├── login.py
│ └── jwt.py
├── api/ # imported by cli/
│ └── routes.py
├── cli/ # entry point, imports auth/ and api/
│ └── main.py
└── legacy-reports/ # NOTHING outside this package imports it
├── generator.py
└── formatter.py
```
`legacy-reports/` is a zombie package. Its files might import each other internally, but no other package depends on it.
**Confidence:** Always 0.5 (medium). Packages might be used as standalone entry points, scripts, or tooling that the dependency graph doesn't capture.
**safe_to_delete:** Always `False`. Deleting an entire package is too risky for automatic recommendation.
### How findings flow to the user
```
DeadCodeAnalyzer.analyze()
│
├── _detect_unreachable_files() → findings with confidence + safe_to_delete
├── _detect_unused_exports() → findings with confidence + safe_to_delete
├── _detect_zombie_packages() → findings at 0.5 confidence, never safe
│
▼ apply min_confidence filter (default 0.4)
│
▼ persist to dead_code_findings table
│
├── REST API: /api/dead-code (filter by kind, confidence, status)
├── MCP Tool: get_dead_code (grouped into tiers: high/medium/low)
└── Web UI: tabbed view with bulk resolve/acknowledge/false-positive
```
**The MCP tool groups findings into three action tiers:**
| Tier | Confidence | Action |
|------|-----------|--------|
| **High** (>= 0.8) | Almost certainly dead | Start here. Safe quick wins. |
| **Medium** (0.5 - 0.8) | Probably dead | Review with team before deleting. |
| **Low** (< 0.5) | Suspicious | Investigate — might be dynamic loading. |
### Incremental dead code analysis
`repowise update` runs the same repo-wide `analyze()` a full index runs, and persists the whole result.
Scoping the detectors to the changed files is not available here, because dead code is a cross-file property: removing the last import of a module makes *that module* dead, and the module is not in the change set. An earlier `analyze_partial()` filtered the repo-wide report down to the changed files before persisting it, and the effect was that any file a change had made dead — or brought back to life — kept its previous verdict until someone re-indexed from scratch.
What the update path does have to be careful about is confidence, which is scored per file from git metadata. An update re-indexes git metadata for the changed files only, and a file with no metadata is indistinguishable from a file with no commits, so it would score 0.7 with `safe_to_delete=True` however actively it is committed to. The analyzer is therefore also handed the persisted per-file git fields, and the report carries `authoritative_paths`: the set of files it was actually able to score. Persistence replaces findings for those files and leaves every other file's stored verdict alone, so a partial or failed metadata read narrows what gets written rather than overwriting the index with guesses.
---
## 2. Decision Records (ADR) System
### What problem does it solve?
Code tells you **what** the system does. Comments sometimes tell you **how**. But almost nothing tells you **why** — why was this approach chosen over alternatives? What constraints forced this design? What was the tradeoff?
When those decisions live only in someone's head or a forgotten Slack thread, every new developer has to reverse-engineer the reasoning. Worse, they might unknowingly undo a deliberate tradeoff, reintroducing a problem that was already solved.
Repowise's decision system **automatically discovers architectural decisions** from four sources, stores them as structured records, tracks their staleness as code evolves, and surfaces them when developers need context.
### How decisions are discovered
#### Source 1: Inline Markers (confidence 0.95)
Developers sometimes leave breadcrumbs in code:
```python
# WHY: We use bcrypt instead of argon2 because our deployment target
# doesn't have the argon2 C bindings available.
password_hash = bcrypt.hashpw(password, bcrypt.gensalt())
# DECISION: Rate limiting is done at the application layer, not the
# load balancer, because we need per-user limits, not per-IP.
```
The extractor scans source files for regex markers:
```
# WHY: ...
# DECISION: ...
# TRADEOFF: ...
# ADR: ...
# RATIONALE: ...
# REJECTED: ...
```
When found, it captures a ±20-line context window around the marker and sends it to the LLM for structuring into a decision record (title, context, decision, rationale, alternatives, consequences).
**Why 0.95 confidence?** The developer explicitly wrote a decision marker. The intent is unambiguous. 0.95 instead of 1.0 because the LLM structuring might misinterpret the context.
#### Source 2: Git Archaeology (confidence 0.70-0.85)
Most decisions aren't marked in code. They're implicit in commit messages:
```
commit abc123: "Migrate from REST to GraphQL for the admin API — reduces
round trips from 12 to 3 for the dashboard view"
commit def456: "Switch from moment.js to date-fns — moment is 300KB,
date-fns is 15KB with tree-shaking"
```
The extractor scores commits by **decision signal keywords**:
```
"migrate", "switch to", "replace", "refactor to", "deprecate",
"remove", "adopt", "introduce", "upgrade", "rewrite", "extract",
"split", "convert", "transition", "revert"
```
Commits with these keywords are batched (groups of 5) and sent to the LLM to identify which ones represent actual architectural decisions vs routine changes.
**Why variable confidence (0.70-0.85)?** Git commits are noisier than inline markers. A commit saying "migrate database" could be a major architectural decision or just a routine migration script. The LLM assesses this and assigns confidence.
#### Source 3: README Mining (confidence 0.60)
Documentation files often contain architectural rationale:
```markdown
## Architecture
We use a message queue between the API and worker services because...
### Why SQLite?
For single-tenant deployments, PostgreSQL is overkill. SQLite gives us...
```
The extractor processes README.md, ARCHITECTURE.md, CONTRIBUTING.md, DESIGN.md, DECISIONS.md, and `docs/*.md` (up to 10 files, 50KB each).
**Why 0.60 confidence?** README content is often aspirational or outdated. It describes what the code *should* be, not necessarily what it *is* today. Lower confidence reflects this uncertainty.
#### Source 4: CLI Capture (confidence 1.0)
```bash
repowise decision add
```
Interactive prompt for manual entry. The developer directly states the decision — no extraction uncertainty.
**All four sources run in parallel** via `asyncio.gather()`. If one source fails (e.g., LLM timeout during git archaeology), the others still complete.
### Decision data model
Each decision record stores:
```
DecisionRecord
├── title: "Migrate admin API from REST to GraphQL"
├── status: "active" | "proposed" | "deprecated" | "superseded"
├── context: "Dashboard required 12 API calls to render..."
├── decision: "Use GraphQL for the admin API"
├── rationale: "Reduces round trips from 12 to 3..."
├── alternatives: ["Keep REST with batching", "Use gRPC"]
├── consequences: ["Need GraphQL schema maintenance", "Client complexity increases"]
├── affected_files: ["src/admin/schema.py", "src/admin/resolvers.py"]
├── affected_modules: ["admin"]
├── tags: ["api", "performance"]
├── source: "git_archaeology"
├── evidence_file: "src/admin/schema.py"
├── evidence_commits: ["abc123"]
├── confidence: 0.80
├── staleness_score: 0.15
└── superseded_by: null
```
**Deduplication key:** `(repository_id, title, source, evidence_file)`. The same decision discovered from two sources (e.g., inline marker + readme mention) creates separate records intentionally — this preserves provenance and lets you see where each piece of evidence came from.
### Staleness computation
Decisions go stale when the code they govern changes but the decision itself doesn't get updated. The staleness algorithm detects this drift.
**Per-file score:**
For each file in the decision's `affected_files`:
```
if file has no git metadata:
file_score = 1.0 (can't verify → assume stale)
elif file's last commit is BEFORE the decision was created:
file_score = 0.0 (file hasn't changed since decision was made → still fresh)
else: # file changed AFTER the decision
base = min(1.0, (commit_count_90d / 15) × 0.7
+ (age_days / 365) × 0.3)
conflict_boost = 0.0
For each significant commit AFTER the decision:
if commit message contains conflict keywords:
("replace", "remove", "deprecate", "migrate away",
"drop", "revert", "undo", "disable", "eliminate")
AND shares 2+ meaningful words with decision text:
conflict_boost = 0.3
file_score = min(1.0, base + conflict_boost)
```
**Breaking down the base score:**
- **70% weight on recent activity:** `commit_count_90d / 15`. If 15+ commits in 90 days, this maxes out at 1.0. Files with heavy churn since the decision was made are likely to have drifted from the original intent.
- **30% weight on age:** `age_days / 365`. Decisions older than a year get a staleness penalty simply because codebases evolve. Even without heavy churn, a year-old decision might not reflect current reality.
**The conflict boost:**
The most interesting part. If a commit message after the decision contains words like "replace", "remove", "deprecate" AND shares meaningful words with the decision text itself, that's a strong signal that someone is actively working against the decision.
**Example:**
```
Decision: "Use bcrypt for password hashing" (created 2025-06-01)
Commit (2026-01-15): "Replace bcrypt with argon2 for password hashing"
The commit contains "replace" (conflict keyword) and shares "bcrypt",
"password", "hashing" with the decision text.
→ conflict_boost = 0.3
→ This decision is likely stale (someone replaced what it decided)
```
**Aggregate score:** Average across all affected files, rounded to 3 decimals.
**Interpretation:**
- **0.0 - 0.3:** Fresh. Code hasn't materially changed since the decision.
- **0.3 - 0.5:** Moderate. Some drift, worth a review.
- **0.5 - 1.0:** Stale. High churn and/or explicit contradictory commits.
### Ungoverned hotspot detection
**Question:** "Which files change a lot but have no documented decisions explaining why?"
```
hotspot_files = files where churn_percentile >= 0.75
AND commit_count_90d >= 3
AND (temporal_hotspot_score >= 0.5 OR commit_count_90d >= 8)
governed_files = union of all affected_files across active decisions
ungoverned_hotspots = hotspot_files - governed_files
```
These are the most dangerous files in the codebase: they change frequently (risky) and nobody has documented why they're designed the way they are (opaque). New developers are most likely to introduce bugs here.
### Alignment scoring
When you query `get_why("src/auth/login.py")`, the system computes an **alignment score** — how well-governed is this file?
**Algorithm:**
```
1. Find all decisions governing this file
(file in affected_files OR module in affected_modules)
2. If no decisions → score = "none"
"This file is ungoverned — no documented rationale."
3. Count statuses:
active_count = decisions with status "active"
deprecated_count = decisions with status "deprecated" or "superseded"
stale_count = decisions with staleness_score > 0.5
proposed_count = decisions with status "proposed"
4. Compute sibling coverage:
sibling_files = other files in the same directory
sibling_decisions = decisions governing siblings
coverage = |shared_decisions| / |sibling_decisions|
5. Score decision tree:
┌─ All deprecated, no active → "low" (technical debt)
├─ >= 50% stale → "low" (rationale may be invalid)
├─ Has active + sibling_coverage >= 0.5 → "high" (well-governed)
├─ Has active + sibling_coverage < 0.5 → "medium" (unique pattern)
├─ Has active, no siblings → "high"
├─ Only proposed → "medium" (unreviewed)
└─ Mixed → "medium"
```
**Why sibling coverage matters:**
If `auth/login.py` is governed by a decision about "Use JWT for authentication" and its sibling `auth/jwt.py` is also governed by the same decision, that's a well-structured module where files share consistent architectural direction. Coverage >= 50% → "high" alignment.
If `auth/login.py` has a unique decision that no sibling shares, it might be an outlier — the decision applies narrowly, or the file doesn't fit the module's pattern. Coverage < 50% → "medium."
### Origin story
When you look up a file's decisions, the system also builds an **origin story** — a narrative reconstruction of how this file came to be:
```
Origin Story for src/auth/login.py:
Created: 2024-03-15 (732 days ago)
Created by: Alice Chen (47% of commits)
Last change: 2026-03-20 by Bob Kim
Commits: 89 total, 12 in last 90 days
Key commits:
- abc123 (2024-03-15): "Initial auth module with JWT" → [Alice]
- def456 (2024-08-22): "Migrate from session cookies to JWT" → [Alice]
- ghi789 (2025-11-03): "Add MFA support to login flow" → [Bob]
Linked decisions:
- "Use JWT for authentication" (active, confidence 0.85)
Evidence commits: abc123, def456 (messages share "JWT" keyword)
- "Add multi-factor authentication" (active, confidence 0.75)
Evidence commits: ghi789 (message shares "MFA" keyword)
```
**Commit-decision linkage** works by keyword overlap: if a commit message shares 2+ meaningful words (after removing stop words) with a decision's text, they're linked as evidence.
### Decision lifecycle
```
proposed → active → deprecated
↓ ↓
superseded ← superseded_by link
CLI commands:
repowise decision add → creates with status "active"
repowise decision confirm → proposed → active
repowise decision deprecate → sets status "deprecated"
repowise decision dismiss → dismisses a proposal (sticky; never re-proposed)
repowise decision health → shows stale, ungoverned, proposed
```
---
## 3. Search and Vector Store Internals
### The problem with keyword search
If you search for "authentication" with keyword matching, you find pages containing the word "authentication." But you miss pages about "login flow", "credential validation", or "session management" — concepts that are semantically identical but use different words.
Vector search solves this by comparing **meaning**, not characters.
### How vector search works — from text to numbers
**Step 1: Embedding.** Convert text to a vector (list of numbers):
```
"authentication module" → [0.12, -0.45, 0.78, ..., 0.03] (1536 numbers)
"login credential check" → [0.11, -0.43, 0.80, ..., 0.05] (1536 numbers)
"database connection" → [0.67, 0.22, -0.15, ..., 0.91] (1536 numbers)
```
The embedding model (OpenAI, Gemini, or mock) maps semantically similar text to nearby vectors. "Authentication" and "login" end up close together. "Database connection" ends up far away.
**Step 2: Normalize.** All vectors are L2-normalized to unit length:
```
normalized = vector / ||vector||
where ||vector|| = sqrt(v[0]² + v[1]² + ... + v[n]²)
```
After normalization, every vector has length 1.0. This is crucial because it makes cosine similarity equal to the dot product, which is cheaper to compute:
```
cosine_similarity(a, b) = (a · b) / (||a|| × ||b||)
If ||a|| = 1 and ||b|| = 1, then:
cosine_similarity(a, b) = a · b = Σ(a[i] × b[i])
```
**Step 3: Store.** Save each vector alongside its page_id and metadata.
**Step 4: Search.** Embed the query, compute similarity against all stored vectors, return top-k.
### The three vector store implementations
#### InMemoryVectorStore
Simplest implementation. Stores vectors in a Python dict:
```python
_store: dict[page_id] → (vector, metadata)
```
Search computes cosine similarity against every vector:
```python
def _cosine(a, b):
dot = sum(x * y for x, y in zip(a, b))
norm_a = sqrt(sum(x * x for x in a))
norm_b = sqrt(sum(x * x for x in b))
return dot / (norm_a * norm_b) if (norm_a * norm_b) > 0 else 0.0
```
**Time complexity:** O(N × D) per search, where N = number of pages, D = vector dimensions.
For 55 pages with 1536 dimensions, this is ~84,000 multiplications per search — trivial. For 100,000 pages, you'd want something smarter. That's where LanceDB comes in.
#### LanceDBVectorStore
Embedded vector database stored as local files in `.repowise/lancedb/`. Uses Apache Arrow columnar format with an IVF-PQ (Inverted File Index with Product Quantization) index for fast approximate nearest neighbor search.
**Schema:**
```
page_id: string
vector: list[dim]
title: string
page_type: string
target_path: string
content_snippet: string (first 200 chars)
```
**Upsert strategy:**
```python
# LanceDB 0.12+: atomic merge_insert
table.merge_insert("page_id")
.when_matched_update_all() # existing page → update vector + metadata
.when_not_matched_insert_all() # new page → insert
.execute([row])
# Fallback for older LanceDB: two operations
table.delete(f"page_id = '{safe_id}'") # delete old
table.add([row]) # insert new
```
**Why merge_insert matters:** Without it, there's a window between delete and add where the page doesn't exist. If a search runs during that window, it misses the page. merge_insert is atomic — the old and new versions are swapped in one operation.
#### PgVectorStore
Uses PostgreSQL's pgvector extension. Stores embeddings directly in the `wiki_pages` table:
```sql
-- Upsert
UPDATE wiki_pages SET embedding = CAST('[0.12,-0.45,...]' AS vector) WHERE id = 'page_id';
-- Search (cosine distance operator <=>)
SELECT id, title, content, page_type, target_path,
1 - (embedding <=> CAST('[0.11,-0.43,...]' AS vector)) AS score
FROM wiki_pages
WHERE embedding IS NOT NULL
ORDER BY embedding <=> CAST('[0.11,-0.43,...]' AS vector)
LIMIT 10;
```
**The `<=>` operator** computes cosine distance (0 = identical, 2 = opposite). Subtracting from 1 converts to similarity.
**Why raw SQL instead of ORM?** The `embedding` column is a pgvector type, which isn't declared in the SQLAlchemy ORM model. This keeps the models dialect-neutral (they work with both SQLite and PostgreSQL). The pgvector column is added by an Alembic migration that only runs on PostgreSQL.
### Full-text search (FTS)
Vector search is powerful but slow (needs embeddings, API calls). Full-text search is fast and works with exact keywords.
#### SQLite FTS5
**Index creation:**
```sql
CREATE VIRTUAL TABLE page_fts USING fts5(
page_id UNINDEXED, -- stored but not searchable
title, -- searchable
content -- searchable
);
```
**Query construction:**
The user's query is transformed into an FTS5 MATCH expression:
```
Input: "Python decorator pattern"
Step 1: Tokenize → ["python", "decorator", "pattern"]
Step 2: Remove stop words → ["python", "decorator", "pattern"] (none removed)
Step 3: Add prefix matching → "python"* OR "decorator"* OR "pattern"*
```
The `*` suffix enables prefix matching: `"auth"*` matches "authentication", "authorization", "authoring".
OR between terms gives **broad recall** — a page matching any term is returned. FTS5's built-in BM25 ranking naturally boosts pages matching **more** terms.
**Another example:**
```
Input: "the async await system"
Step 1: Tokenize → ["the", "async", "await", "system"]
Step 2: Remove stop words → ["async", "await", "system"] (removes "the")
Step 3: → "async"* OR "await"* OR "system"*
```
127 stop words are removed (a, an, the, is, are, was, were, be, been, have, has, had, do, does, did, will, would, etc.).
**Edge case:** If all words are stop words (e.g., "a the is"), the query falls back to exact phrase matching: `"a the is"`.
#### PostgreSQL
Uses `to_tsvector('english', ...)` for document representation and `plainto_tsquery('english', ...)` for queries. PostgreSQL handles stemming (running → run), stop word removal, and ranking via `ts_rank()`.
A GIN index makes searches fast without scanning every row.
### Search ranking in the MCP tool
The MCP `search_codebase` tool applies additional ranking on top of raw search scores:
**Step 1: Try semantic search** (vector store, 8-second timeout)
**Step 2: Fallback to FTS** if semantic fails or returns empty
**Step 3: Freshness boost** — recently modified files rank higher:
```
if file has commits in last 30 days: recency = 1.0
elif file has commits in last 90 days: recency = 0.5
else: recency = 0.0
boosted_score = raw_score × (1 + 0.2 × recency)
```
A file modified yesterday gets a 20% boost. A file untouched for a year gets no boost.
**Why boost freshness?** If you're searching for "authentication" and two pages match equally, the one that was recently updated is more likely to be accurate and relevant to current development.
**Step 4: Normalize confidence** relative to the best result:
```
confidence = relevance_score / max_relevance_score
```
The top result gets confidence ≈ 1.0. Other results are proportionally lower. This gives the user a relative quality signal without needing to interpret raw cosine similarity values.
---
## 4. Incremental Updates and Webhooks
### The problem
Initial documentation generation is expensive — potentially hundreds of LLM calls. After that, you want to keep docs fresh as code changes, but regenerating everything on every commit is wasteful and slow.
Repowise's incremental update system regenerates **only what changed** and its dependencies.
### Three triggers
#### Trigger 1: CLI update command
```bash
repowise update
```
Reads `.repowise/state.json` to find the last synced commit, diffs against current HEAD, and regenerates affected pages.
#### Trigger 2: Filesystem watcher
```bash
repowise watch --debounce 2000
```
Uses `watchdog` library to monitor the filesystem. When files change:
1. Add changed paths to a set
2. Start a debounce timer (default 2 seconds)
3. If more changes arrive, reset the timer
4. When the timer fires (filesystem quiet for 2 seconds), run `repowise update`
**Why debounce?** Saving a file in an editor often triggers multiple filesystem events (write, metadata change, backup file creation). Without debouncing, each save would trigger 3-4 redundant update runs. The 2-second quiet period waits until all save-related events settle.
#### Trigger 3: Webhooks
GitHub and GitLab can send POST requests when code is pushed:
**GitHub verification:**
```
Webhook arrives with:
Body: {...}
Header: X-Hub-Signature-256: sha256=abc123...
Server computes:
expected = HMAC-SHA256(secret, body)
Verification:
hmac.compare_digest(expected, received) # constant-time comparison
```
`compare_digest` is critical for security — it takes the same time regardless of where the strings differ, preventing timing attacks that could leak the secret byte by byte.
**GitLab verification:**
```
Header: X-Gitlab-Token: my-secret-token
Compare: hmac.compare_digest(expected_token, received_token)
```
Simpler — just a shared token, no HMAC computation.
**After verification:**
```
1. Store raw webhook event in database (audit trail)
2. Find matching repository by URL
3. Create GenerationJob:
- status: "pending"
- mode: "incremental"
- config: {before: "old_commit_sha", after: "new_commit_sha"}
4. Link webhook event to job
```
### The update pipeline
```
1. LOAD STATE
Read last_sync_commit from .repowise/state.json
│
2. DIFF
ChangeDetector.get_changed_files(last_sync_commit, HEAD)
→ List of FileDiff objects (added, deleted, modified, renamed)
│
3. RE-INGEST
Re-run FileTraverser + ASTParser + GraphBuilder on entire repo
(Need fresh graph to compute cascades correctly)
│
4. RE-INDEX GIT
GitIndexer.index_changed_files() — only for changed files
Update churn percentiles, hotspot flags
│
5. DETECT DECISIONS
Scan changed files for inline decision markers (WHY:, DECISION:, etc.)
Update decision staleness scores
│
6. CASCADE ANALYSIS
ChangeDetector.get_affected_pages(file_diffs, graph, cascade_budget)
→ regenerate: pages to fully regenerate (budget-limited)
→ rename_patch: pages with symbol renames (text replacement)
→ decay_only: pages to mark stale without regeneration
│
7. GENERATE
PageGenerator.generate_all() with only affected files
│
8. PERSIST
Upsert pages, git metadata, decisions
Update FTS index, vector store
Update CLAUDE.md if enabled
│
9. SAVE STATE
Write current HEAD to .repowise/state.json
```
### SSE progress streaming
Long-running jobs report progress via Server-Sent Events:
```
Client: GET /api/jobs/{job_id}/stream
Accept: text/event-stream
Server: (every 1 second)
event: progress
data: {"job_id": "abc", "status": "running", "completed_pages": 12, "total_pages": 55}
event: progress
data: {"job_id": "abc", "status": "running", "completed_pages": 25, "total_pages": 55}
...
event: done
data: {"job_id": "abc", "status": "completed", "completed_pages": 55, "total_pages": 55}
```
The server checks for client disconnection each iteration and stops streaming when the client goes away. Headers include `Cache-Control: no-cache` and `X-Accel-Buffering: no` (prevents nginx from buffering the stream).
### Scheduler polling fallback
Webhooks can fail (network issues, misconfiguration, GitHub outages). The scheduler runs two background jobs every 15 minutes:
**Job 1: Staleness checker** — finds stale/expired pages across all repos and logs them.
**Job 2: Polling fallback** — for each local repo, compares the stored HEAD commit against actual `git rev-parse HEAD`. If they differ, a webhook was missed. (Currently logs only; full auto-sync is future work.)
---
## 5. Change Cascade Algorithm
### The problem
Changing `utils.py` doesn't just affect `utils.py`'s documentation. Every file that imports `utils.py` might now have incorrect documentation too — it might reference old function names, outdated behavior, or changed signatures.
But propagating changes through the entire dependency graph could trigger hundreds of regenerations. You need a smart cascade with a budget.
### The algorithm
```
Input:
file_diffs: list of changed files with their old/new parsed versions
graph: dependency graph (directed, import edges)
cascade_budget: max pages to fully regenerate (adaptive, hard cap 50)
Output:
regenerate: set of page IDs for full LLM regeneration
rename_patch: set of page IDs needing text replacement (for renames)
decay_only: set of page IDs to mark stale without regeneration
```
**Step 1: Direct changes**
```
directly_changed = {file.path for file in file_diffs}
```
These always get regenerated (they changed, their docs are definitely wrong).
**Step 2: 1-hop cascade (reverse dependencies)**
```
one_hop = set()
for file in directly_changed:
for predecessor in graph.predecessors(file):
# predecessor imports file → predecessor's docs may reference file
one_hop.add(predecessor)
one_hop -= directly_changed # don't double-count
```
**Example:**
```
auth.py changed
graph.predecessors("auth.py") = ["main.py", "middleware.py", "api.py"]
one_hop = {"main.py", "middleware.py", "api.py"}
```
These files import `auth.py`. If `auth.py` renamed a function, their documentation might reference the old name.
**Step 3: Symbol rename detection**
For each changed file, compare old and new parsed versions to detect renames:
```
Old file symbols: [calculate, Config, validate]
New file symbols: [compute, Config, validate]
"calculate" removed, "compute" added, both are functions
Name similarity = SequenceMatcher("calculate", "compute").ratio() = 0.63
Line proximity = start lines within ±5 → bonus = 0.2
Combined = 0.63 + 0.2 = 0.83 (above threshold 0.65)
→ Detected rename: calculate → compute
```
Files referencing the old name need a text patch (string replacement in the existing doc) rather than a full regeneration.
**Step 4: Co-change decay**
```
co_change = set()
for file in directly_changed:
for partner in graph edges with edge_type="co_changes":
if partner is file's co-change partner:
co_change.add(partner)
co_change -= directly_changed
co_change -= one_hop
```
Files that frequently change alongside the modified files are marked for decay — their docs might be stale but the evidence is weaker (correlation, not causation).
**Step 5: 2-hop weak cascade (for renames only)**
```
two_hop = set()
for file in one_hop:
if file has symbol renames:
for predecessor in graph.predecessors(file):
two_hop.add(predecessor)
two_hop -= directly_changed
two_hop -= one_hop
```
If a rename propagated to a 1-hop file, files importing that 1-hop file might also need updating. But this is speculative — marked for decay only.
**Step 6: Budget application**
```
candidates = directly_changed ∪ one_hop
sorted_by_pagerank = sort(candidates, key=pagerank, descending)
regenerate = sorted_by_pagerank[:cascade_budget]
decay_only = sorted_by_pagerank[cascade_budget:] ∪ two_hop ∪ co_change
```
**Why sort by PageRank?** If the budget is 50 and there are 80 candidates, you want to regenerate the 50 most important files first. A high-PageRank file is imported by many others — if its documentation is wrong, the error propagates further. Low-PageRank leaf files can wait.
**Adaptive budget:** The budget is no longer fixed. `compute_adaptive_budget()` scales it based on change magnitude:
| Files changed | Budget |
|---------------|--------|
| 0 | 0 |
| 1 | 10 |
| 2-5 | 30 |
| 6+ | min(n × 3, 50) |
Hard cap at 50. Users can override with `--cascade-budget N`.
### Worked example
```
Codebase: 200 files, adaptive cascade_budget = 30 (3 files changed)
Developer changes: config.py
graph.predecessors("config.py") = [
"auth.py", "api.py", "db.py", "cache.py", "logger.py",
"middleware.py", "scheduler.py", "worker.py", "mailer.py",
"validator.py", "serializer.py", "router.py"
] # 12 files
directly_changed = {"config.py"} # 1 file
one_hop = {12 files above} # 12 files
candidates = 1 + 12 = 13 files (under budget of 30)
config.py renamed: LOG_LEVEL → LOGGING_LEVEL
→ rename_patch candidates: files importing LOG_LEVEL
co_change partners of config.py: ["docker-compose.yml", ".env.example"]
→ decay_only
Result:
regenerate: 13 pages (all fit within budget)
rename_patch: pages referencing LOG_LEVEL
decay_only: docker-compose.yml, .env.example docs
```
Now imagine `config.py` is imported by 80 files:
```
candidates = 1 + 80 = 81 files (over budget of 30)
Sort by PageRank:
Top 30: main.py, auth.py, api.py, ... (highest PageRank)
Remaining 51: leaf files, utilities, tests
Result:
regenerate: 30 pages (budget-limited, most important first)
decay_only: 51 pages (confidence decayed, regenerated on next run)
```
The 51 files that didn't make the cut have their confidence score reduced. They'll show as "stale" in the UI and be prioritized for regeneration on the next update.
### Confidence decay
Pages in `decay_only` don't get regenerated but their freshness score changes:
```
confidence decays linearly:
1.0 at generation time → 0.0 after expiry_threshold_days (default 30)
freshness_status:
"fresh" if hash matches AND age < 7 days
"stale" if hash changed OR age >= 7 days
"expired" if age >= 30 days (forces regeneration on next run)
```
This creates a natural queue: pages that got bumped from the cascade budget eventually hit "expired" status and get regenerated in a future update cycle, ensuring nothing stays stale forever.
---
### Architecture/Editor Files
# Editor File Generation
repowise can generate and maintain AI-editor configuration files — CLAUDE.md,
cursor.md, and similar — from the already-indexed codebase data. No LLM calls
are made. All content is derived from the repowise SQL database and filesystem
manifests.
---
## Table of Contents
1. [Why this exists](#1-why-this-exists)
2. [How it works](#2-how-it-works)
3. [The two-section file structure](#3-the-two-section-file-structure)
4. [What goes into the Repowise section](#4-what-goes-into-the-repowise-section)
5. [Data sources](#5-data-sources)
6. [File reference](#6-file-reference)
7. [CLI usage](#7-cli-usage)
8. [REST API](#8-rest-api)
9. [Configuration](#9-configuration)
10. [Adding a new editor file](#10-adding-a-new-editor-file)
---
## 1. Why this exists
Claude Code reads `CLAUDE.md` on every session start but does not proactively
call MCP tools — even when an MCP server is configured. Users have to explicitly
tell Claude Code to "use MCP" every time.
By embedding codebase intelligence and MCP workflow guidance directly into
`CLAUDE.md`, Claude Code treats it as project context and naturally reaches for
Repowise tools without being prompted.
The same principle applies to Cursor's `.cursor/rules` / `cursor.md`, GitHub
Copilot's `.github/copilot-instructions.md`, and any other file an AI editor
auto-loads at session start.
---
## 2. How it works
```
repowise init (or update)
│
▼
All wiki pages and git metadata already persisted in DB
│
▼
EditorFileDataFetcher
Queries DB for: architecture summary, top modules by PageRank,
entry points, hotspot files, active decision records
Scans filesystem for: tech stack, build commands
Returns: EditorFileData (frozen dataclass, no DB types)
│
▼
ClaudeMdGenerator.write(repo_path, data)
Renders claude_md.j2 template with EditorFileData
Reads existing CLAUDE.md (if any)
Merges: user content preserved, Repowise section replaced
Writes atomically (temp file + rename)
│
▼
CLAUDE.md written to repo root
```
**Key properties:**
- Runs after `_persist()` completes, so all data is current
- Best-effort in `repowise update` — never fails the command
- Instant — no LLM calls, typically < 200ms
- Idempotent — re-running produces identical output if data hasn't changed
- Deterministic — lists sorted by stable keys, with path asc as the final
tiebreaker. Most lists lead on PageRank desc; entry points deliberately do
not, and rank on execution-start evidence instead.
---
## 3. The two-section file structure
```markdown
# CLAUDE.md
[user's own content — untouched by repowise]
## Codebase Intelligence — myrepo (Repowise)
[auto-generated content]
```
**Merge rules (applied in `BaseEditorFileGenerator.write()`):**
| Situation | Action |
|-----------|--------|
| No CLAUDE.md exists | Create with user placeholder + Repowise section |
| File exists, no markers | Append Repowise section at bottom; leave existing content untouched |
| File exists with markers | Replace only the content between `REPOWISE:START` and `REPOWISE:END` |
The regex used for marker replacement:
```python
pattern = re.escape(MARKER_START) + r".*?" + re.escape(MARKER_END)
re.sub(pattern, new_wrapped_content, existing, flags=re.DOTALL)
```
This is the entire preservation strategy. The only way to lose user content is
to manually insert text *inside* the `REPOWISE:START` / `REPOWISE:END` block,
which the placeholder comment warns against.
---
## 4. What goes into the Repowise section
The section has three parts. Total target length: **150–250 lines**. Conciseness
is intentional — research shows AI assistants ignore bloated configuration files.
### Part 1: Codebase Intelligence
Auto-generated from indexed data. Updates on every `repowise update`.
| Sub-section | Source | Cap |
|-------------|--------|-----|
| Architecture summary | First 4 sentences from `repo_overview` wiki page | 4 sentences |
| Key Modules | `module_page` pages sorted by PageRank desc, joined with git_metadata for owner | Top 10 |
| Entry Points | The curated `kg_project_meta` list; otherwise `graph_nodes` where `is_entry_point=True`, ranked on execution-start evidence | Top 10 |
| Tech Stack | Filesystem scan (package.json, pyproject.toml, Cargo.toml, go.mod, etc.) | All detected |
| Hotspots | `git_metadata` where `is_hotspot=True`, sorted by `churn_percentile` desc | Top 5 |
### Part 2: MCP Tools Workflow Guide
**Static** — the same text for every repo. Hardcoded in `claude_md.j2`. Teaches
Claude Code *when* to call each MCP tool using natural workflow framing rather than
imperatives.
This is the most important part. The phrasing matters: "Starting a new task?
Call `get_overview()` first" is more effective than "ALWAYS call `get_overview()`
before doing anything."
### Part 3: Codebase Conventions
Auto-generated from indexed data.
| Sub-section | Source | Cap |
|-------------|--------|-----|
| Architectural Decisions | `decision_records` where `status='active'`, sorted by `staleness_score` asc | Top 8 |
| Commands | Filesystem scan: `package.json` scripts, `Makefile` targets, `pyproject.toml` pytest/ruff | All detected |
---
## 5. Data sources
### Database queries (in `EditorFileDataFetcher`)
**Architecture summary** (`_get_architecture_summary`)
```python
crud.list_pages(session, repo_id, page_type="repo_overview", limit=1)
# → extracts first 4 sentences, strips markdown headers/code fences
```
**Key modules** (`_get_key_modules`)
```python
SELECT page, GraphNode.pagerank, GraphNode.symbol_count
FROM wiki_pages
JOIN graph_nodes ON graph_nodes.node_id = wiki_pages.target_path
WHERE wiki_pages.page_type = 'module_page'
AND wiki_pages.repository_id = :repo_id
ORDER BY graph_nodes.pagerank DESC NULLS LAST
LIMIT 10
# → owner resolved via separate git_metadata lookup
```
**Entry points** (`_get_entry_points`)
The curated `kg_project_meta.entry_points_json` list wins when the curation
pass has run. Otherwise the raw flag is read unbounded and ranked in Python,
because the ordering cannot be expressed as an `ORDER BY`:
```python
SELECT node_id, pagerank, betweenness FROM graph_nodes
WHERE repository_id = :repo_id AND is_entry_point = TRUE
# then rank_entry_points(...): conventional entry name, then shallower path,
# then centrality as a tiebreak; sliced to 10 after ranking.
```
PageRank deliberately does **not** lead here. Centrality rewards fan-in, so it
floats a widely-imported barrel above the real front door — see
`generation/entry_points.py`.
**Hotspots** (`_get_hotspots`)
```python
SELECT file_path, churn_percentile, commit_count_90d, primary_owner_name
FROM git_metadata
WHERE repository_id = :repo_id AND is_hotspot = TRUE
ORDER BY churn_percentile DESC, file_path ASC -- deterministic tie-break
LIMIT 5
```
**Active decisions** (`_get_decisions`)
```python
SELECT * FROM decision_records
WHERE repository_id = :repo_id AND status = 'active'
ORDER BY staleness_score ASC
LIMIT 8
# → uses first 100 chars of rationale field
```
**Average confidence** (`_get_avg_confidence`)
```python
SELECT AVG(confidence) FROM wiki_pages WHERE repository_id = :repo_id
```
### Filesystem scan (in `tech_stack.py`)
`detect_tech_stack(repo_path)` scans the repo root for:
| File | Detects |
|------|---------|
| `package.json` | Node.js, TypeScript, React, Next.js, Vue, Express, Prisma, Tailwind, … |
| `pyproject.toml` / `setup.py` | Python, FastAPI, Django, Flask, SQLAlchemy, Celery, … |
| `Cargo.toml` | Rust |
| `go.mod` | Go (extracts version from `go X.Y` directive) |
| `pom.xml` / `build.gradle` | Java / Kotlin + Maven / Gradle |
| `Gemfile` | Ruby |
| `composer.json` | PHP |
| `Dockerfile` | Docker |
| `docker-compose.yml` | Docker Compose |
`detect_build_commands(repo_path)` returns a dict with keys from:
`build`, `test`, `lint`, `dev`, `format`, `typecheck`.
Priority: `package.json` scripts → `pyproject.toml` → `Makefile`. Each key is
only set once — the first source wins.
---
## 6. File reference
```
packages/core/src/repowise/core/generation/editor_files/
├── __init__.py Exports: ClaudeMdGenerator, EditorFileData, EditorFileDataFetcher
├── base.py BaseEditorFileGenerator — marker-merge logic, Jinja2 setup, atomic write
├── data.py Frozen dataclasses: EditorFileData, TechStackItem, KeyModule,
│ HotspotFile, DecisionSummary
├── fetcher.py EditorFileDataFetcher — all DB queries + filesystem calls
├── tech_stack.py detect_tech_stack(), detect_build_commands()
└── claude_md.py ClaudeMdGenerator — filename, marker_tag, template_name, user_placeholder
packages/core/src/repowise/core/generation/templates/
└── claude_md.j2 Jinja2 template for the Repowise-managed section
packages/cli/src/repowise/cli/commands/
├── claude_md_cmd.py `repowise generate-claude-md` command
└── init_cmd.py _maybe_generate_claude_md(), _write_claude_md_async() helpers
packages/server/src/repowise/server/routers/
└── claude_md.py GET/POST /api/repos/{repo_id}/claude-md
tests/unit/generation/
├── test_editor_file_base.py Marker-merge logic, idempotency, file structure
├── test_editor_file_fetcher.py DB query correctness with in-memory SQLite
└── test_tech_stack.py Filesystem detection with tmp_path fixtures
```
### Class hierarchy
```
BaseEditorFileGenerator (base.py)
│ filename: str — abstract property
│ marker_tag: str — abstract property
│ template_name: str — abstract property
│ user_placeholder: str — abstract property
│ render(data) → str
│ write(repo_path, data) → Path
│ render_full(repo_path, data) → str
│
└── ClaudeMdGenerator (claude_md.py)
filename = "CLAUDE.md"
marker_tag = "REPOWISE"
template_name = "claude_md.j2"
user_placeholder = "# CLAUDE.md\n\n\n"
```
### Data flow
```
EditorFileDataFetcher.fetch()
│ AsyncSession + repo_id + repo_path
│
├── crud.get_repository() → repo.name
├── _get_architecture_summary() → str (2-4 sentences)
├── _get_key_modules() → list[KeyModule]
├── _get_entry_points() → list[str]
├── detect_tech_stack() → list[TechStackItem]
├── _get_hotspots() → list[HotspotFile]
├── _get_decisions() → list[DecisionSummary]
├── detect_build_commands() → dict[str, str]
└── _get_avg_confidence() → float
│
▼
EditorFileData (frozen dataclass)
│
▼
BaseEditorFileGenerator.render(data)
│ Jinja2 template rendered with data
▼
str (managed section content, without markers)
│
▼
BaseEditorFileGenerator.write(repo_path, data)
│ Wraps with markers, merges with existing file
▼
Path (written file)
```
---
## 7. CLI usage
```
repowise generate-claude-md [PATH]
PATH Repo root to generate for (default: current directory)
--output FILE Write to a custom path instead of CLAUDE.md in repo root
--stdout Print generated content to stdout (does not write a file)
```
Examples:
```bash
# Generate CLAUDE.md for the current directory
repowise generate-claude-md .
# Preview what would be written without touching the file
repowise generate-claude-md . --stdout
# Write to a custom path
repowise generate-claude-md /path/to/repo --output /tmp/preview.md
```
**Auto-generation during `init` and `update`:**
`repowise init` generates CLAUDE.md after the persistence phase completes.
`repowise update` regenerates it (best-effort) after each incremental sync.
Both can be disabled:
```bash
# Skip CLAUDE.md on this init run and persist the preference to config
repowise init --no-claude-md .
```
**Project-local files vs. global registration:**
`init` writes in two different places, and the flags split along that line.
*Project-local*, all inside the repo, versionable, one set per repo:
`.repowise/mcp.json`, `.mcp.json`, `.claude/CLAUDE.md`, `AGENTS.md`,
`.vscode/mcp.json`, `.vscode/extensions.json`, `.codex/`. Three have an opt-out
flag (`--no-claude-md`, `--agents`, `--codex`) and the VS Code pair is
prompt-gated in an interactive run. Only `.repowise/mcp.json` and the root
`.mcp.json` are written unconditionally.
*Machine-wide*, outside the repo, one shared copy for every repo you index, all
written by `register_editor_clients()` in `editor_setup.py`:
- the `repowise` MCP entry in `~/.claude/settings.json` and in Claude Desktop's
config
- the Claude Code PostToolUse and SessionStart hooks
- `env.ENABLE_TOOL_SEARCH` in `~/.claude/settings.json` (skipped for repos on
the lean MCP tool profile, and never overwritten if you already set it)
Only the Claude integration implements `register_client`; Codex and VS Code
read project-local config, so theirs are no-ops.
The distill command-rewrite hook is machine-wide too, but it is offered
separately (`offer_distill_rewrite_hook`) because it is strictly opt-in.
`--no-editor-setup` turns off **both** groups: the machine-wide registrations
above, including the rewrite hook offer, *and* the project-local files
(`.mcp.json`, `.claude/CLAUDE.md`, `.vscode/mcp.json`,
`.vscode/extensions.json`). Only `.repowise/` is written.
```bash
# Index the repo, write nothing into it and nothing outside it
repowise init --no-editor-setup --yes .
```
It used to cover the machine-wide half only, and said so — which meant there
was no combination of flags that indexed a repo without writing four files into
the working tree, since VS Code had no opt-out flag of its own. Issue #1499 is
the report of exactly that. One switch, one meaning.
`.repowise/mcp.json` is the one deliberate exception and is written either way.
No editor reads it unless pointed at it, and it is what `repowise mcp .` prints
— so skipping it would mean opting out of editor setup also opted out of ever
opting back in.
Reach for it whenever the checkout or the binary running `init` is temporary:
a scratch clone, a release smoke test from a throwaway venv, a git worktree, a
benchmark loop over many repos. Each config holds a **single** `repowise` MCP
key, so a second `init` replaces the entry rather than adding one beside it,
and the breakage only shows up later, when the path it now points at is gone
and the MCP server quietly stops loading. `init` prints a notice when it is
about to repoint an existing entry, but the flag is how you avoid it.
Two exceptions, so the flags do not cancel each other out. `--no-editor-setup
--no-distill-hook` still records the `distill.commands.enabled: false` opt-out in
this repo's `config.yaml`. That record is repo-local, and it is the only thing
that gates an already-installed global rewrite hook off here.
The same reasoning covers the instruction files: `--no-editor-setup
--no-claude-md` still records `editor_files.claude_md: false`, and likewise for
`--no-agents-md`. Those flags mean "never generate this file", not "skip it this
once", and the generator declining on its way past used to be the only thing
that wrote the preference down — so suppressing the writes would also have
suppressed the memory of the refusal, and the next `repowise update` would have
generated the file anyway. A preference is not a write.
`REPOWISE_SKIP_EDITOR_SETUP=1` is the same switch as an env var, which is the
better fit for CI and sandboxes where no one is passing flags by hand. Either
source disables setup; the flag never re-enables what the env var turned off.
Neither is persisted to `config.yaml`: this is a per-run decision about your
machine, not a property of the repo, so a later `init` without the flag
registers normally.
---
## 8. REST API
### `GET /api/repos/{repo_id}/claude-md`
Returns the generated Repowise section as JSON. Does not write to disk.
Useful for web UI preview.
```json
{
"content": "## Codebase Intelligence — myrepo (Repowise)\n...",
"generated_at": "2026-03-28",
"repo_name": "myrepo",
"sections": ["Architecture", "Key Modules", "Entry Points", "Tech Stack",
"Hotspots (High Churn)", "Repowise MCP Tools", "Codebase Conventions"]
}
```
### `POST /api/repos/{repo_id}/claude-md/generate`
Regenerates CLAUDE.md and writes it to the repository's `local_path` on disk.
Returns 422 if `local_path` is not accessible from the server.
```json
{
"status": "generated",
"path": "/home/user/myrepo/CLAUDE.md",
"generated_at": "2026-03-28"
}
```
---
## 9. Configuration
```yaml
# .repowise/config.yaml
editor_files:
claude_md: true # default: true. Set false to disable entirely.
```
The `--no-claude-md` CLI flag sets `editor_files.claude_md: false` in
`config.yaml` and all future `repowise update` runs will skip it.
---
## 10. Adding a new editor file
Example: adding `cursor.md` support.
### Step 1 — Create the subclass
**`packages/core/src/repowise/core/generation/editor_files/cursor_md.py`**
```python
from .base import BaseEditorFileGenerator
class CursorMdGenerator(BaseEditorFileGenerator):
filename = "cursor.md"
marker_tag = "REPOWISE"
template_name = "cursor_md.j2"
user_placeholder = (
"# cursor.md\n\n"
"\n"
)
```
That's the entire subclass. All merge logic, atomic write, Jinja2 setup, and
`render()` / `write()` / `render_full()` are inherited from `BaseEditorFileGenerator`.
### Step 2 — Create the template
**`packages/core/src/repowise/core/generation/templates/cursor_md.j2`**
The template receives `data` (an `EditorFileData` instance). All fields are
the same as `claude_md.j2` — the fetcher is shared. Write cursor-specific
framing around the same data.
Minimal starting point:
```jinja2
## Project Context (Repowise)
Last indexed: {{ data.indexed_at }}.
{% if data.architecture_summary %}
### Architecture
{{ data.architecture_summary }}
{% endif %}
### Key Files
{% for ep in data.entry_points %}
- `{{ ep }}`
{% endfor %}
```
### Step 3 — Add config key
```yaml
# .repowise/config.yaml
editor_files:
claude_md: true
cursor_md: true # NEW
```
### Step 4 — Export from the subpackage
**`packages/core/src/repowise/core/generation/editor_files/__init__.py`**
```python
from .cursor_md import CursorMdGenerator # add this line
```
### Step 5 — Hook into init and update
In `_maybe_generate_claude_md()` (or extract a more generic
`_maybe_generate_editor_files()` helper):
```python
# After existing CLAUDE.md generation
if cfg.get("editor_files", {}).get("cursor_md", False): # default off
from repowise.core.generation.editor_files import CursorMdGenerator
CursorMdGenerator().write(repo_path, data)
```
Since `data` is already fetched by this point (reuse from CLAUDE.md generation),
the cursor.md write costs only the template render — no extra DB queries.
### Step 6 — Add a REST endpoint (optional)
Follow `packages/server/src/repowise/server/routers/claude_md.py` exactly.
Swap `ClaudeMdGenerator` for `CursorMdGenerator`. Register the router in `app.py`.
### Step 7 — Add tests
Follow `tests/unit/generation/test_editor_file_base.py`. The `_TestGenerator`
fixture already tests `BaseEditorFileGenerator` behavior — add a test that
instantiates `CursorMdGenerator` directly to verify the filename/marker_tag/
template_name properties and that the template renders without error.
---
### What you do NOT need to do
- Write any DB queries — `EditorFileDataFetcher` and `EditorFileData` are shared.
- Write any file I/O or merge logic — `BaseEditorFileGenerator` handles everything.
- Register a new CLI command — use `generate-claude-md` as a reference if you
want a standalone command, but it is not required.
- Update the ORM schema — no new tables needed.
The only required artifacts for a new editor file are:
1. A 10–30 line subclass (`*.py`)
2. A Jinja2 template (`*.j2`)
---