Repository: lfnovo/open-notebook
Stars: 22310
CLAUDE.md
Open Notebook - Root CLAUDE.md
This file provides architectural guidance for contributors working on Open Notebook at the project level.
Project Overview
Open Notebook is an open-source, privacy-focused alternative to Google's Notebook LM. It's an AI-powered research assistant enabling users to upload multi-modal content (PDFs, audio, video, web pages), generate intelligent notes, search semantically, chat with AI models, and produce professional podcastsβall with complete control over data and choice of AI providers.
Key Values: Privacy-first, multi-provider AI support, fully self-hosted option, open-source transparency.
---
Three-Tier Architecture
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Frontend (React/Next.js) β
β frontend/ @ port 3000 β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β - Notebooks, sources, notes, chat, podcasts, search UI β
β - Zustand state management, TanStack Query (React Query)β
β - Shadcn/ui component library with Tailwind CSS β
ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ
β HTTP REST
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ
β API (FastAPI) β
β api/ @ port 5055 β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β - REST endpoints for notebooks, sources, notes, chat β
β - LangGraph workflow orchestration β
β - Job queue for async operations (podcasts) β
β - Multi-provider AI provisioning via Esperanto β
ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ
β SurrealQL
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ
β Database (SurrealDB) β
β Graph database @ port 8000 β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β - Records: Notebook, Source, Note, ChatSession, Credentialβ
β - Relationships: source-to-notebook, note-to-source β
β - Vector embeddings for semantic search β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ---
Useful sources
User documentation is at @docs/
Tech Stack
Frontend (frontend/)
- Framework: Next.js 16 (React 19)
- Language: TypeScript
- State Management: Zustand
- Data Fetching: TanStack Query (React Query)
- Styling: Tailwind CSS + Shadcn/ui
- Build Tool: Webpack (via Next.js)
- i18n compatible: All front-end changes must also consider the translation keys
API Backend (api/ + open_notebook/)
- Framework: FastAPI 0.104+
- Language: Python 3.11+
- Workflows: LangGraph state machines
- Database: SurrealDB async driver
- AI Providers: Esperanto library (8+ providers: OpenAI, Anthropic, Google, Groq, Ollama, Mistral, DeepSeek, xAI)
- Job Queue: Surreal-Commands for async jobs (podcasts)
- Logging: Loguru
- Validation: Pydantic v2
- Testing: Pytest
Database
- SurrealDB: Graph database with built-in embedding storage and vector search
- Schema Migrations: Automatic on API startup via AsyncMigrationManager
Additional Services
- Content Processing: content-core library (file/URL extraction)
- Prompts: AI-Prompter with Jinja2 templating
- Podcast Generation: podcast-creator library
- Embeddings: Multi-provider via Esperanto
---
Architecture Highlights
1. Async-First Design
- All database queries, graph invocations, and API calls are async (await)
- SurrealDB async driver with connection pooling
- FastAPI handles concurrent requests efficiently
2. LangGraph Workflows
- source.py: Content ingestion (extract β embed β save)
- chat.py: Conversational agent with message history
- ask.py: Search + synthesis (retrieve relevant sources β LLM)
- transformation.py: Custom transformations on sources
- All use
provision_langchain_model() for smart model selection3. Multi-Provider AI
- Esperanto library: Unified interface to 8+ AI providers
- Credential system: Individual encrypted credential records per provider; models link to credentials for direct config
- ModelManager: Factory pattern with fallback logic; uses credential config when available, env vars as fallback
- Smart selection: Detects large contexts, prefers long-context models
- Override support: Per-request model configuration
4. Database Schema
- Automatic migrations: AsyncMigrationManager runs on API startup
- SurrealDB graph model: Records with relationships and embeddings
- Vector search: Built-in semantic search across all content
- Transactions: Repo functions handle ACID operations
5. Authentication
- Current: Simple password middleware (insecure, dev-only)
- Production: Replace with OAuth/JWT (see CONFIGURATION.md)
---
Important Quirks & Gotchas
API Startup
- Migrations run automatically on startup; check logs for errors
- Must start API before UI: UI depends on API for all data
- SurrealDB must be running: API fails without database connection
Frontend-Backend Communication
- Base API URL: Configured in
.env.local (default: http://localhost:5055)- CORS enabled: Configured in
api/main.py (allow all origins in dev)- Rate limiting: Not built-in; add at proxy layer for production
LangGraph Workflows
- Blocking operations: Chat/podcast workflows may take minutes; no timeout
- State persistence: Uses SQLite checkpoint storage in
/data/sqlite-db/- Model fallback: If primary model fails, falls back to cheaper/smaller model
Podcast Generation
- Async job queue:
podcast_service.py submits jobs but doesn't wait- Track status: Use
/commands/{command_id} endpoint to poll status- TTS failures: Fall back to silent audio if speech synthesis fails
Content Processing
- File extraction: Uses content-core library; supports 50+ file types
- URL handling: Extracts text + metadata from web pages
- Large files: Content processing is sync; may block API briefly
---
Component References
See dedicated CLAUDE.md files for detailed guidance:
- frontend/CLAUDE.md: React/Next.js architecture, state management, API integration
- api/CLAUDE.md: FastAPI structure, service pattern, endpoint development
- open_notebook/CLAUDE.md: Backend core, domain models, LangGraph workflows, AI provisioning
- open_notebook/domain/CLAUDE.md: Data models, repository pattern, search functions
- open_notebook/ai/CLAUDE.md: ModelManager, AI provider integration, Esperanto usage
- open_notebook/graphs/CLAUDE.md: LangGraph workflow design, state machines
- open_notebook/database/CLAUDE.md: SurrealDB operations, migrations, async patterns
---
Documentation Map
- README.md: Project overview, features, quick start
- docs/index.md: Complete user & deployment documentation
- CONFIGURATION.md: Environment variables, model configuration
- CONTRIBUTING.md: Contribution guidelines
- MAINTAINER_GUIDE.md: Release & maintenance procedures
---
Testing Strategy
- Unit tests: tests/test_domain.py, test_models_api.py
- Graph tests: tests/test_graphs.py (workflow integration)
- Utils tests: tests/test_utils.py, tests/test_chunking.py, tests/test_embedding.py
- Run all: uv run pytest tests/
- Coverage: Check with pytest --cov
---
Common Tasks
Add a New API Endpoint
1. Create router in
api/routers/feature.py2. Create service in
api/feature_service.py3. Define schemas in
api/models.py4. Register router in
api/main.py5. Test via http://localhost:5055/docs
Add a New LangGraph Workflow
1. Create
open_notebook/graphs/workflow_name.py2. Define StateDict and node functions
3. Build graph with
.add_node() / .add_edge()4. Invoke in service:
graph.ainvoke({"input": ...}, config={"..."})5. Test with sample data in
tests/Add Database Migration
1. Create
migrations/XXX_description.surql2. Write SurrealQL schema changes
3. Create
migrations/XXX_description_down.surql (optional rollback)4. API auto-detects on startup; migration runs if newer than recorded version
Deploy to Production
1. Review CONFIGURATION.md for security settings
2. Use
make docker-release for multi-platform image3. Push to Docker Hub / GitHub Container Registry
4. Deploy
docker compose --profile multi up5. Verify migrations via API logs
---
Support & Community
- Documentation: https://open-notebook.ai
- Discord: https://discord.gg/37XJPXfz2w
- Issues: https://github.com/lfnovo/open-notebook/issues
- License: MIT (see LICENSE)
README.md
<a id="readme-top"></a>
<!-- [![Contributors][contributors-shield]][contributors-url] -->
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![MIT License][license-shield]][license-url]
<!-- [![LinkedIn][linkedin-shield]][linkedin-url] -->
<!-- PROJECT LOGO -->
<br />
<div align="center">
<a href="https://github.com/lfnovo/open-notebook">
<img src="docs/assets/hero.svg" alt="Logo">
</a>
<h3 align="center">Open Notebook</h3>
<p align="center">
An open source, privacy-focused alternative to Google's Notebook LM!
<br /><strong>Join our <a href="https://discord.gg/37XJPXfz2w">Discord server</a> for help, to share workflow ideas, and suggest features!</strong>
<br />
<a href="https://www.open-notebook.ai"><strong>Checkout our website Β»</strong></a>
<br />
<br />
<a href="docs/0-START-HERE/index.md">π Get Started</a>
Β·
<a href="docs/3-USER-GUIDE/index.md">π User Guide</a>
Β·
<a href="docs/2-CORE-CONCEPTS/index.md">β¨ Features</a>
Β·
<a href="docs/1-INSTALLATION/index.md">π Deploy</a>
</p>
</div>
<p align="center">
<a href="https://trendshift.io/repositories/14536" target="_blank"><img src="https://trendshift.io/api/badge/repositories/14536" alt="lfnovo%2Fopen-notebook | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</p>
<div align="center">
<!-- Keep these links. Translations will automatically update with the README. -->
<a href="https://zdoc.app/de/lfnovo/open-notebook">Deutsch</a> |
<a href="https://zdoc.app/es/lfnovo/open-notebook">EspaΓ±ol</a> |
<a href="https://zdoc.app/fr/lfnovo/open-notebook">franΓ§ais</a> |
<a href="https://zdoc.app/ja/lfnovo/open-notebook">ζ₯ζ¬θͺ</a> |
<a href="https://zdoc.app/ko/lfnovo/open-notebook">νκ΅μ΄</a> |
<a href="https://zdoc.app/pt/lfnovo/open-notebook">PortuguΓͺs</a> |
<a href="https://zdoc.app/ru/lfnovo/open-notebook">Π ΡΡΡΠΊΠΈΠΉ</a> |
<a href="https://zdoc.app/zh/lfnovo/open-notebook">δΈζ</a>
</div>
A private, multi-model, 100% local, full-featured alternative to Notebook LM
In a world dominated by Artificial Intelligence, having the ability to think π§ and acquire new knowledge π‘, is a skill that should not be a privilege for a few, nor restricted to a single provider.
Open Notebook empowers you to:
- π Control your data - Keep your research private and secure
- π€ Choose your AI models - Support for 18+ providers including OpenAI, Anthropic, Ollama, LM Studio, and more
- π Organize multi-modal content - PDFs, videos, audio, web pages, and more
- ποΈ Generate professional podcasts - Advanced multi-speaker podcast generation
- π Search intelligently - Full-text and vector search across all your content
- π¬ Chat with context - AI conversations powered by your research
- π Multi-language UI - English, Portuguese, Chinese (Simplified & Traditional), Japanese, Russian, and Bengali support
Learn more about our project at https://www.open-notebook.ai
---
π Open Notebook vs Google Notebook LM
| Feature | Open Notebook | Google Notebook LM | Advantage |
|---------|---------------|--------------------|-----------|
| Privacy & Control | Self-hosted, your data | Google cloud only | Complete data sovereignty |
| AI Provider Choice | 18+ providers (OpenAI, Anthropic, Ollama, LM Studio, etc.) | Google models only | Flexibility and cost optimization |
| Podcast Speakers | 1-4 speakers with custom profiles | 2 speakers only | Extreme flexibility |
| Content Transformations | Custom and built-in | Limited options | Unlimited processing power |
| API Access | Full REST API | No API | Complete automation |
| Deployment | Docker, cloud, or local | Google hosted only | Deploy anywhere |
| Citations | Basic references (will improve) | Comprehensive with sources | Research integrity |
| Customization | Open source, fully customizable | Closed system | Unlimited extensibility |
| Cost | Pay only for AI usage | Free tier + Monthly subscription | Transparent and controllable |
Why Choose Open Notebook?
- π Privacy First: Your sensitive research stays completely private
- π° Cost Control: Choose cheaper AI providers or run locally with Ollama
- ποΈ Better Podcasts: Full script control and multi-speaker flexibility vs limited 2-speaker deep-dive format
- π§ Unlimited Customization: Modify, extend, and integrate as needed
- π No Vendor Lock-in: Switch providers, deploy anywhere, own your data
Built With
[![Python][Python]][Python-url] [![Next.js][Next.js]][Next-url] [![React][React]][React-url] [![SurrealDB][SurrealDB]][SurrealDB-url] [![LangChain][LangChain]][LangChain-url]
π Quick Start (2 Minutes)
Prerequisites
- Docker Desktop installed
- That's it! (API keys configured later in the UI)
Step 1: Get docker-compose.yml
Option A: Download directly
curl -o docker-compose.yml https://raw.githubusercontent.com/lfnovo/open-notebook/main/docker-compose.ymlOption B: Create the file manually
Copy this into a new file called docker-compose.yml:
services:
surrealdb:
image: surrealdb/surrealdb:v2
command: start --log info --user root --pass root rocksdb:/mydata/mydatabase.db
user: root
ports:
- "8000:8000"
volumes:
- ./surreal_data:/mydata
restart: always open_notebook:
image: lfnovo/open_notebook:v1-latest
ports:
- "8502:8502"
- "5055:5055"
environment:
- OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-string
- SURREAL_URL=ws://surrealdb:8000/rpc
- SURREAL_USER=root
- SURREAL_PASSWORD=root
- SURREAL_NAMESPACE=open_notebook
- SURREAL_DATABASE=open_notebook
volumes:
- ./notebook_data:/app/data
depends_on:
- surrealdb
restart: always
Step 2: Set Your Encryption Key
Edit
docker-compose.yml and change this line:- OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-stringto any secret value (e.g.,
my-super-secret-key-123)Step 3: Start Services
docker compose up -dWait 15-20 seconds, then open: http://localhost:8502
Step 4: Configure AI Provider
1. Go to Settings β API Keys
2. Click Add Credential
3. Choose your provider (OpenAI, Anthropic, Google, etc.)
4. Paste your API key and click Save
5. Click Test Connection β Discover Models β Register Models
Done! You're ready to create your first notebook.
Need an API key? Get one from:
OpenAI Β· Anthropic Β· Google Β· Groq (free tier)
Want free local AI? See examples/docker-compose-ollama.yml for Ollama setup
---
π More Installation Options
- With Ollama (Free Local AI) - Run models locally without API costs
- From Source (Developers) - For development and contributions
- Complete Installation Guide - All deployment scenarios
---
π Need Help?
- π€ AI Installation Assistant: CustomGPT to help you install
- π Troubleshooting: 5-minute troubleshooting guide
- π¬ Community Support: Discord Server
- π Report Issues: GitHub Issues
---
Star History

Provider Support Matrix
Thanks to the Esperanto library, we support this providers out of the box!
| Provider | LLM Support | Embedding Support | Speech-to-Text | Text-to-Speech |
|--------------|-------------|------------------|----------------|----------------|
| OpenAI | β
| β
| β
| β
|
| Anthropic | β
| β | β | β |
| Groq | β
| β | β
| β |
| Google (GenAI) | β
| β
| β | β
|
| Vertex AI | β
| β
| β | β
|
| Ollama | β
| β
| β | β |
| Perplexity | β
| β | β | β |
| ElevenLabs | β | β | β
| β
|
| Azure OpenAI | β
| β
| β | β |
| Mistral | β
| β
| β | β |
| DeepSeek | β
| β | β | β |
| Voyage | β | β
| β | β |
| xAI | β
| β | β | β |
| OpenRouter | β
| β | β | β |
| DashScope (Qwen) | β
| β | β | β |
| MiniMax | β
| β | β | β |
| OpenAI Compatible* | β
| β | β | β |
*Supports LM Studio and any OpenAI-compatible endpoint
β¨ Key Features
Core Capabilities
- π Privacy-First: Your data stays under your control - no cloud dependencies
- π― Multi-Notebook Organization: Manage multiple research projects seamlessly
- π Universal Content Support: PDFs, videos, audio, web pages, Office docs, and more
- π€ Multi-Model AI Support: 18+ providers including OpenAI, Anthropic, Ollama, Google, LM Studio, and more
- ποΈ Professional Podcast Generation: Advanced multi-speaker podcasts with Episode Profiles
- π Intelligent Search: Full-text and vector search across all your content
- π¬ Context-Aware Chat: AI conversations powered by your research materials
- π AI-Assisted Notes: Generate insights or write notes manually
Advanced Features
- β‘ Reasoning Model Support: Full support for thinking models like DeepSeek-R1 and Qwen3
- π§ Content Transformations: Powerful customizable actions to summarize and extract insights
- π Comprehensive REST API: Full programmatic access for custom integrations 
- π Optional Password Protection: Secure public deployments with authentication
- π Fine-Grained Context Control: Choose exactly what to share with AI models
- π Citations: Get answers with proper source citations
Podcast Feature

π Documentation
Getting Started
- π Introduction - Learn what Open Notebook offers
- β‘ Quick Start - Get up and running in 5 minutes
- π§ Installation - Comprehensive setup guide
- π― Your First Notebook - Step-by-step tutorial
User Guide
- π± Interface Overview - Understanding the layout
- π Notebooks - Organizing your research
- π Sources - Managing content types
- π Notes - Creating and managing notes
- π¬ Chat - AI conversations
- π Search - Finding information
Advanced Topics
- ποΈ Podcast Generation - Create professional podcasts
- π§ Content Transformations - Customize content processing
- π€ AI Models - AI model configuration
- π MCP Integration - Connect with Claude Desktop, VS Code and other MCP clients
- π§ REST API Reference - Complete API documentation
- π Security - Password protection and privacy
- π Deployment - Complete deployment guides for all scenarios
<p align="right">(<a href="#readme-top">back to top</a>)</p>
πΊοΈ Roadmap
Upcoming Features
- Live Front-End Updates: Real-time UI updates for smoother experience
- Async Processing: Faster UI through asynchronous content processing
- Cross-Notebook Sources: Reuse research materials across projects
- Bookmark Integration: Connect with your favorite bookmarking apps
Recently Completed β
- Next.js Frontend: Modern React-based frontend with improved performance
- Comprehensive REST API: Full programmatic access to all functionality
- Multi-Model Support: 18+ AI providers including OpenAI, Anthropic, Ollama, LM Studio
- Advanced Podcast Generator: Professional multi-speaker podcasts with Episode Profiles
- Content Transformations: Powerful customizable actions for content processing
- Enhanced Citations: Improved layout and finer control for source citations
- Multiple Chat Sessions: Manage different conversations within notebooks
See the open issues for a full list of proposed features and known issues.
<p align="right">(<a href="#readme-top">back to top</a>)</p>
π Need Help?
- π€ AI Installation Assistant: We have a CustomGPT built to help you install Open Notebook - it will guide you through each step!
- New to Open Notebook? Start with our Getting Started Guide
- Need installation help? Check our Installation Guide
- Want to see it in action? Try our Quick Start Tutorial
π€ Community & Contributing
Join the Community
- π¬ Discord Server - Get help, share ideas, and connect with other users
- π GitHub Issues - Report bugs and request features
- β Star this repo - Show your support and help others discover Open Notebook
Contributing
We welcome contributions! We're especially looking for help with:
- Frontend Development: Help improve our modern Next.js/React UI
- Testing & Bug Fixes: Make Open Notebook more robust
- Feature Development: Build the coolest research tool together
- Documentation: Improve guides and tutorials
Current Tech Stack: Python, FastAPI, Next.js, React, SurrealDB
Future Roadmap: Real-time updates, enhanced async processing
See our Contributing Guide for detailed information on how to get started.
<p align="right">(<a href="#readme-top">back to top</a>)</p>
π License
Open Notebook is MIT licensed. See the LICENSE file for details.
Community Support:
- π¬ Discord Server - Get help, share ideas, and connect with users
- π GitHub Issues - Report bugs and request features
- π Website - Learn more about the project
<p align="right">(<a href="#readme-top">back to top</a>)</p>
<!-- MARKDOWN LINKS & IMAGES -->
<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->
[contributors-shield]: https://img.shields.io/github/contributors/lfnovo/open-notebook.svg?style=for-the-badge
[contributors-url]: https://github.com/lfnovo/open-notebook/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/lfnovo/open-notebook.svg?style=for-the-badge
[forks-url]: https://github.com/lfnovo/open-notebook/network/members
[stars-shield]: https://img.shields.io/github/stars/lfnovo/open-notebook.svg?style=for-the-badge
[stars-url]: https://github.com/lfnovo/open-notebook/stargazers
[issues-shield]: https://img.shields.io/github/issues/lfnovo/open-notebook.svg?style=for-the-badge
[issues-url]: https://github.com/lfnovo/open-notebook/issues
[license-shield]: https://img.shields.io/github/license/lfnovo/open-notebook.svg?style=for-the-badge
[license-url]: https://github.com/lfnovo/open-notebook/blob/master/LICENSE.txt
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
[linkedin-url]: https://linkedin.com/in/lfnovo
[product-screenshot]: images/screenshot.png
[Next.js]: https://img.shields.io/badge/Next.js-000000?style=for-the-badge&logo=next.js&logoColor=white
[Next-url]: https://nextjs.org/
[React]: https://img.shields.io/badge/React-61DAFB?style=for-the-badge&logo=react&logoColor=black
[React-url]: https://reactjs.org/
[Python]: https://img.shields.io/badge/Python-3776AB?style=for-the-badge&logo=python&logoColor=white
[Python-url]: https://www.python.org/
[LangChain]: https://img.shields.io/badge/LangChain-3A3A3A?style=for-the-badge&logo=chainlink&logoColor=white
[LangChain-url]: https://www.langchain.com/
[SurrealDB]: https://img.shields.io/badge/SurrealDB-FF5E00?style=for-the-badge&logo=databricks&logoColor=white
[SurrealDB-url]: https://surrealdb.com/