Technical Documentation: pubkey/rxdb
โน๏ธ Provenance: Multi-Tier Ingestion ยท CodeWiki Reference
Overview (`README.md`)
What is RxDB?
RxDB (short for Reactive Database) is a local-first, NoSQL-database for JavaScript Applications.
Reactive means that you can not only query the current state, but subscribe to all state changes like the result of a query or even a single field of a document.
This is great for UI-based realtime applications in a way that makes it easy to develop and also has great performance benefits.
Use the quickstart, read the documentation or explore the example projects.
Used by many
RxDB is a proven technology used by thousands of developers worldwide. With its flexibility, RxDB is used in a diverse range of apps and services.
Multiplayer realtime applications
Replicate with your existing infrastructure
RxDB provides an easy to implement, battle-tested Sync Engine for realtime replication with your existing infrastructure.
You do not have to use a specific cloud or backend database. The protocol works by implementing three simple HTTP endpoints.
There are also production-ready plugins to easily replicate with GraphQL, CouchDB, Websocket, WebRTC (P2P), Supabase, Firestore, NATS or Google Drive.
Flexible storage layer
RxDB is based on a storage interface that enables you to swap out the underlying storage engine. This increases code reuse because the same database code can be used in different JavaScript environments by just switching out the storage settings.
You can use RxDB on top of LocalStorage, IndexedDB, OPFS, LokiJS, Dexie.js, in-memory, SQLite, in a WebWorker thread and even on top of FoundationDB and DenoKV.
No matter what kind of runtime you have, as long as it runs JavaScript, it can run RxDB:
Flutter
or as an
Electron Database
All the features that you need
Since its beginning in 2018, RxDB has gained a huge set of features and plugins which makes it a flexible full solution regardless of which type of application you are building. Every feature that you need now or might need in the future is already there.
| Logging
Attachments
ORM
Conflict Handling
Middleware
Signals
TanStack DB | State
Backup
Replication
Server
Storages
Local Documents | Schema Validation
Compression
Migration
Encryption
CRDT
Population |
| --- | --- | --- |
Quick start
Install
npm install rxdb rxjs --saveStore data
import {
createRxDatabase
} from 'rxdb/plugins/core';
/**
* For browsers, we use the localstorage based storage.
* In other JavaScript runtimes, we can use different storages:
* @link https://rxdb.info/rx-storage.html
*/
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
// create a database
const db = await createRxDatabase({
name: 'heroesdb', // the name of the database
storage: getRxStorageLocalstorage()
});
// add collections with a schema
await db.addCollections({
heroes: {
schema: {
version: 0,
primaryKey: 'name',
type: 'object',
properties: {
name: {
type: 'string',
maxLength: 100
},
healthpoints: {
type: 'number'
}
},
required: ['name', 'healthpoints']
}
}
});
// insert a document
await db.heroes.insert({
name: 'Bob',
healthpoints: 100
});Query data once
const aliveHeroes = await db.heroes.find({
selector: {
healthpoints: {
$gt: 0
}
}
}).exec(); // the exec() returns the result onceObserve a Query
await db.heroes.find({
selector: {
healthpoints: {
$gt: 0
}
}
})
.$ // the $ returns an observable that emits each time the result set of the query changes
.subscribe(aliveHeroes => console.dir(aliveHeroes));Get started
Get started now by reading the docs or exploring the example-projects.
Support and Contribute
- Leave a Star โ๏ธ
- Check out how you can contribute to this project.
- Read this when you have found a bug
- Buy access to the premium plugins
- Join us at discord to get help
- Follow us at LinkedIn
More content
Angular Database, Frontend Database, localStorage, React Database, Browser Database, React Native Database, PWA Database, In-memory NoSQL database, JSON database, Angular IndexedDB, React IndexedDB, Optimistic UI, local database, React Native Encryption, Vue Database, jQuery Database, Vue IndexedDB, Firestore Alternative, Firebase Realtime Database Alternative, Ionic Storage, Electron SQLite, TanStack DB Offline, DDIL Database
View llms.txt
Overview (`docs-src/README.md`)
Website
This website is built using Docusaurus 2, a modern static website generator.
Installation
$ yarnLocal Development
$ yarn startThis command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
Build
$ yarn buildThis command generates static content into the build directory and can be served using any static contents hosting service.
Deployment
Using SSH:
$ USE_SSH=true yarn deployNot using SSH:
$ GIT_USER=<Your GitHub username> yarn deployIf you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.
Overview (`examples/README.md`)
Example-projects with RxDB
In this folder you can find some example projects with RxDB.
| Technology | Status |
|---|---|
| nodeJS | done |
| react-native | done |
| ionic + capacitor | done |
| electron | done |
| angular | done |
| react | done |
| svelte | done |
| vuejs | done |
| supabase | done |
NOTICE: The maintainer is not able to maintain any additional example projects. If you want to add a new one, create your own repository and add a link to it below.
Third Party Examples:
| Technology | Third Party Link |
|---|---|
| NativeScript | Link |
| GraphQL + Svelte + Pothos | Link |
Overview (`.agent/skills/README.md`)
Agent Skills
This directory contains skills that extend the agent's capabilities.
Each skill should be a subdirectory containing a SKILL.md file with instructions.
For example:.agent/skills/my-skill/SKILL.md
Overview (`examples/angular/README.md`)
RxDB Angular example
This is an example usage of RxDB with Angular.
It implements a simple heroes-list which can be filled by the user.
Also it uses angular-universal to enable server side rendering.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/angular - run
npm run preinstall && npm install - run
npm start - Open http://127.0.0.1:4200/ IMPORTANT: do not use localhost
Important parts when using RxDB with angular:
- Make sure you have the
windowpolyfills added that are needed for some plugins.
// in polyfills.ts
(window as any).global = window;
(window as any).process = {
env: { DEBUG: undefined },
nextTick: (fn, ...args) => setTimeout(() => fn(...args)),
};- Make sure you have used the rxjs-zone.js patch otherwise the change detection will not work properly. For that you should put the following code into your
app.component.ts:
//> app.component.ts
/**
* IMPORTANT: RxDB creates rxjs observables outside of angulars zone
* So you have to import the rxjs patch to ensure changedetection works correctly.
* @link https://www.bennadel.com/blog/3448-binding-rxjs-observable-sources-outside-of-the-ngzone-in-angular-6-0-2.htm
*/
import 'zone.js/plugins/zone-patch-rxjs';Screenshot
Read more about using RxDB as Angular Database
Overview (`examples/electron/README.md`)
RxDB Electron example
This is an example usage of RxDB with the Electron RxStorage. It implements a simple heroes-list which can be filled by the user.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/electron - run
npm run preinstall && npm install --verbose(downloading electron can take a while, so use verbose to ensure its running) - run
npm start
Related
Overview (`examples/flutter/README.md`)
RxDB Flutter example
This is an example of how to use RxDB as a database inside of a Flutter application. It is able to insert documents and run and observe queries to create a reactive application that always renders the current database state.
IMPORTANT: This is highly experimental, it works but many parts are missing. If you need any missing functionality, you are expected to make a pull request.
How it works technically
RxDB is written in TypeScript and compiled to JavaScript. To run it in a Flutter application, the flutter_qjs library is used to spawn a QuickJS JavaScript runtime. RxDB itself runs in that runtime and communicates with the flutter dart runtime. To store data persistent, the LokiJS RxStorage is used together with a custom storage adapter that persists the database inside of the shared_preferences data.
In JavaScript
To use RxDB, you have to create a compatible JavaScript file that creates your RxDatabase and starts some connectors which are used by Flutter to communicate with the JavaScript RxDB database via setFlutterRxDatabaseConnector().
Use the index.js as a reference.
import {
createRxDatabase
} from 'rxdb';
import {
getRxStorageLoki
} from 'rxdb/plugins/storage-lokijs';
import {
setFlutterRxDatabaseConnector,
getLokijsAdapterFlutter
} from 'rxdb/plugins/flutter';
// do all database creation stuff in this method.
async function createDB(databaseName) {
// create the RxDatabase
const db = await createRxDatabase({
// the database.name is variable so we can change it on the flutter side
name: databaseName,
storage: getRxStorageLoki({
adapter: getLokijsAdapterFlutter()
}),
multiInstance: false
});
await db.addCollections({
heroes: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string',
maxLength: 100
},
color: {
type: 'string',
maxLength: 30
}
},
indexes: ['name'],
required: ['id', 'name', 'color']
}
}
});
return db;
}
// start the connector so that flutter can communicate with the JavaScript process
setFlutterRxDatabaseConnector(
createDB
);Before you can use the JavaScript code, you have to bundle it into a single .js file. In this example we do that with webpack in a npm script here which bundles everything into the javascript/dist/index.js file.
To allow Flutter to access that file during runtime, add it to the assets inside of your pubspec.yaml:
flutter:
assets:
- javascript/dist/index.jsIn Flutter
First you have to use the rxdb dart package as a flutter dependency. Currently the package is not published at the dart pub.dev. Instead you have to install it from the local filesystem inside of your RxDB npm installation.
# inside of pubspec.yaml
dependencies:
rxdb:
path: path/to/your/node_modules/rxdb/src/plugins/flutter/dartAfterwards you can import the rxdb library in your dart code
and connect to the JavaScript process from there. For reference, check out the lib/main.dart file.
import 'package:rxdb/rxdb.dart';
// start the javascript process and connect to the database
RxDatabase myDatabase = await getRxDatabase("javascript/dist/index.js", databaseName);
// get a collection
RxCollection myCollection = database.getCollection('heroes');
// insert a document
RxDocument myDocument = await collection.insert({
"id": "zflutter-${DateTime.now()}",
"name": nameController.text,
"color": colorController.text
});
// create a query
RxQuery<RxHeroDocType> myQuery = RxDatabaseState.collection.find();
// subscribe to a query
myQuery.$().listen((newResults) {
setState(() {
documents = newResults;
});
});Run the example
First you have to bundle the JavaScript by running
npm run install && npm run buildin the /javascript directory. You have to rerun this command each time you change the JavaScript code.In your terminal, execute
flutter runto start the example application.Read more about using RxDB as Flutter Database
Overview (`examples/graphql/README.md`)
RxDB GraphQL example
This is an example usage of RxDB with with the graphql-replication-plugin.
It represents a simple hero-list which is two-way-replicated with the server.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install(you may need to runnpm install --legacy-peer-deps) - go to this folder
cd examples/graphql - run
npm run preinstall && npm install - run
npm start - Open http://127.0.0.1:8888/ IMPORTANT: do not use localhost
You can change the storage and disable sync via url parameter, e.g.:
Overview (`examples/ionic/README.md`)
RxDB Ionic + Capacitor Example
This is a hero list app that shows how to use RxDB inside of an Ionic app that runs in the browser and, via Capacitor, on Android and iOS.
The app uses:
- Ionic 8 with Angular standalone components
- Capacitor 8 for the Android and iOS builds
- The localstorage RxStorage which works in the browser and inside of the Capacitor webview
- Reactive queries, so the list updates whenever a hero is inserted, changed or deleted
- The multi-tab support of RxDB in the browser. Open the app in two tabs and watch both update at the same time
Try it out in the browser
- Clone the whole RxDB repo
- Go to this folder
cd examples/ionic - Run
npm install - Run
npm run dev - Open http://localhost:8100/
Run it on Android or iOS
The native projects are not checked in, so you have to create them once:
npm run build
npm run cap:add:android # or: npm run cap:add:iosAfterwards you can open the project in Android Studio or Xcode:
npm run cap:open:android # or: npm run cap:open:iosBoth scripts run the web build and npx cap sync before opening the native IDE.
Notice
For the GitHub CI this example installs the local RxDB build (rxdb-local.tgz) which is created by the preinstall script. In your own app you install rxdb from npm instead:
npm install rxdb rxjsStorage on native devices
The localstorage RxStorage is used here because it needs no native plugin and works everywhere. When you store many documents on a mobile device, you can switch to the SQLite RxStorage which stores the data outside of the webview. Switching storages is a configuration change, not a rewrite, so only the storage field in src/app/services/database.service.ts has to be changed.
Related
Overview (`examples/node/README.md`)
RxDB node example
This is an example usage of RxDB with node. It implements a simple heroes-list.
Try it out
- Make sure you have nodeJS version v7.10.0 or newer
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/node - run
npm install - run
npm start
Overview (`examples/react-native/README.md`)
RxDB React-Native example
This is an example usage of RxDB with React-native. It implements a simple heroes-list which can be filled by the user.
Syncing
For database replication and syncing you will need to input a public ip address as the app simulators will have a different context for localhost. One simple way is to run a server locally and expose the port via tunnelmole, a free and open source tunneling tool, or ngrok, a popular closed source tunneling tool.
Try it out
- follow installation instructions to setup your react-native environment (follow instructions for Expo)
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/react-native - run
npx yarn install - run
npm start- to run on ios or android specific emulators use
npm run iosandnpm run androidrespectively
- to run on ios or android specific emulators use
- It use current version RxDB (see preinstall script). Tested with [email protected]
Screenshot
Related
Overview (`examples/react/README.md`)
RxDB React example
This is an example usage of RxDB with Create React App. It implements a simple heroes-list which can be filled by the user.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/react - run
npm run preinstall && npm install - run
npm start - Open http://127.0.0.1:8888/ IMPORTANT: do not use localhost
Screenshot
Overview (`examples/supabase/README.md`)
RxDB Supabase example (Moved)
Moved: Since mid 2025, RxDB has an official Supabase replication plugin which made this example obsolete.
Overview (`examples/svelte/README.md`)
RxDB in Svelte Demo
This is a quick note-taking app that demonstrates how to use RxDB within a Svelte app.
It runs on Svelte 5 and uses runes:
- Shared state lives in
src/store.svelte.jsand is created with$state. src/NoteList.sveltesubscribes to the RxDB query inside an$effect, which also unsubscribes when the component is destroyed.src/main.jsstarts the app withmount()instead ofnew App().
npm run preinstall && npm i && npm run devThen open http://localhost:5000/
Overview (`examples/tauri/README.md`)
RxDB Tauri example
This is an example usage of RxDB with Tauri and the SQLite RxStorage. It implements a simple heroes-list which can be filled by the user.
It is built with the Tauri CLI and the SQL plugin.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install - go to this folder
cd examples/tauri - run
npm run preinstall && npm install -D - run
npm run tauri dev
Tests
To run the tests, you must install the Tauri Driver and other dependencies.
Overview (`examples/vite-vanilla-ts/README.md`)
RxDB in Vite Typescript Demo
This is a quick note-taking app that demonstrates how to use RxDB within a Vite Vanilla Typescript app.
Try it out
- clone the whole RxDB-repo
- go to this folder
(examples/vite-vanilla-ts) - run
npm installoryarn install - run
npm run devoryarn dev - Open http://localhost:5173/
Overview (`examples/vue/README.md`)
RxDB vuejs example
This is an example usage of RxDB with VueJS. It implements a simple heroes-list which can be filled by the user.
For easier mapping of observables.
Try it out
- clone the whole RxDB-repo
- go into project
cd rxdb - run
npm install && npm run build - go to this folder
cd examples/vue - run
npm run preinstall && npm install - run
npm start - Open http://127.0.0.1:8080/ IMPORTANT: do not use localhost
Overview (`orga/changelog/README.md`)
Changelog Entries
Add one JSON file per changelog entry to this folder. Each file must match the JSON schema in schema.json. The CI validates all entries with node ./scripts/check-changelog-entries.js.
File naming: Use a descriptive filename ending in .json, for example:
fix-count-query-with-limit.jsonadd-supabase-attachment-support.json
File content example:
{
"$schema": "./schema.json",
"text": "FIX memory storage `count()` returning incorrect results when the selector is not fully satisfied by the index and the query has a `limit` set",
"links": [
{
"url": "https://github.com/pubkey/rxdb/issues/1234",
"linkText": "#1234"
}
],
"priority": 6
}Fields:
text: The markdown text of the change, without the leading-and without the links. Starts with the type of the change likeFIX,ADD,CHANGEorPERFORMANCE.links: Linked URLs, for example the root issue or pull request, each withurlandlinkText. They are rendered as markdown links behind the text. Use[]when there is nothing to link.priority: How important the change is for RxDB users, from1to10. In the release notes and inCHANGELOG.mdthe entries with the highest priority are shown first. Rough guide:1: lint fixes, flaky tests, internal tooling2-3: typos, error message and docs fixes4-6: bugfixes and small behavior changes7-8: performance improvements, new features of existing plugins, breaking changes9-10: new plugins, new storages, new replication plugins
On release, all entry files in this folder are merged into CHANGELOG.md, sorted by priority, and deleted automatically by the scripts/set-version.mjs script.
This approach prevents merge conflicts in CHANGELOG.md when multiple PRs are open at the same time.
Overview (`src/plugins/query-builder/mquery/README.md`)
this is a copy from mquery
This was copied to prevent the usage of the unused imports and
because the collection-request-part is not used by rxdb.
https://github.com/aheckmann/mquery
Overview (`examples/flutter/ios/Runner/Assets.xcassets/LaunchImage.imageset/README.md`)
Launch Screen Assets
You can customize the launch screen with your own desired assets by replacing the image files in this directory.
You can also do it by opening your Flutter project's Xcode project with open ios/Runner.xcworkspace, selecting Runner/Assets.xcassets in the Project Navigator and dropping in the desired images.
Docs Src Docs Overview (`docs-src/docs/overview.md`)
title: RxDB Docs slug: overview.html image: /headers/overview.jpg
import { Overview } from '@site/src/components/overview';
RxDB Documentation
Docs Src Docs Quickstart (`docs-src/docs/quickstart.md`)
title: Quickstart slug: quickstart.html image: /headers/quickstart.jpg
import {Steps} from '@site/src/components/steps';
import {TriggerEvent} from '@site/src/components/trigger-event';
import {Tabs} from '@site/src/components/tabs';
import {NavbarDropdownSyncList} from '@site/src/components/navbar-dropdowns';
import { IconQuickstart } from '@site/src/components/icons/quickstart';
import { HeadlineWithIcon } from '@site/src/components/headline-with-icon';
RxDB Quickstart
Welcome to the RxDB Quickstart. Here we'll learn how to create a simple real-time app with the RxDB database that is able to store and query data persistently in a browser and does realtime updates to the UI on changes.
Installation
Install the RxDB library and the RxJS dependency:
npm install rxdb rxjsPick a Storage
RxDB is able to run in a wide range of JavaScript runtimes like browsers, mobile apps, desktop and servers. Therefore different storage engines exist that ensure the best performance depending on where RxDB is used.
LocalStorage
Use this for the simplest browser setup and very small datasets. It has a tiny bundle size and works anywhere localStorage is available, but is not optimized for large data or heavy writes.
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
let storage = getRxStorageLocalstorage();IndexedDB ๐
The premium IndexedDB storage is a high-performance, browser-native storage with a smaller bundle and faster startup compared to Dexie-based IndexedDB. Recommended when you have ๐ premium access and care about performance and bundle size.
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
let storage = getRxStorageIndexedDB();Dexie.js
Dexie.js is a friendly wrapper around IndexedDB and is a great default for browser apps when you don't use premium. It's reliable, works well for medium-sized datasets, and is free to use.
import {
getRxStorageDexie
} from 'rxdb/plugins/storage-dexie';
let storage = getRxStorageDexie();SQLite
SQLite is ideal for React Native, Capacitor, Electron, Node.js and other hybrid or native environments. It gives you a fast, durable database on disk. Use the ๐ premium storage for production; a trial version exists for quick experimentation.
Premium SQLite (Node.js example)
import {
getRxStorageSQLite,
getSQLiteBasicsNode
} from 'rxdb-premium/plugins/storage-sqlite';
// Provide the sqliteBasics adapter for your runtime, e.g. Node.js, React Native, etc.
// For example in Node.js you would derive
// sqliteBasics from a sqlite3-compatible library:
import sqlite3 from 'sqlite3';
const storage = getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNode(sqlite3)
});SQLite trial storage (Node.js, free)
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsNodeNative
} from 'rxdb/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
const storage = getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
});Expo Filesystem ๐
For React Native and Expo applications, the Expo Filesystem storage offers superior performance compared to SQLite and Async Storage by utilizing OPFS JSI bindings.
import {
getRxStorageExpoAsync
} from 'rxdb-premium/plugins/storage-filesystem-expo';
let storage = getRxStorageExpoAsync();And more...
There are many more storages such as MongoDB, DenoKV, Filesystem, Memory, Memory-Mapped, FoundationDB and more. Browse the full list of storages.
Which storage should I use?
RxDB provides a wide range of storages depending on your JavaScript runtime and performance needs.
<li>In the Browser: Use the [LocalStorage](/rx-storage-localstorage.html) storage for simple setup and small build size. For bigger datasets, use either the [dexie.js storage](/rx-storage-dexie.html) (free) or the [IndexedDB RxStorage](/rx-storage-indexeddb.html) if you have [๐ premium access](/premium/) which is a bit faster and has a smaller build size.</li>
<li>In [Electron](/electron-database.html) and [React Native](/react-native-database.html): Use the [SQLite RxStorage](./rx-storage-sqlite.html) if you have [๐ premium access](/premium/) or the [SQLite Trial RxStorage](/rx-storage-sqlite.html) for tryouts.</li>
<li>In Capacitor: Use the [SQLite RxStorage](/rx-storage-sqlite.html) if you have [๐ premium access](/premium/), otherwise use the [LocalStorage](/rx-storage-localstorage.html) storage.</li>Dev-Mode
When you use RxDB in development, you should always enable the dev-mode plugin, which adds helpful checks and validations, and tells you if you do something wrong.
import { addRxPlugin } from 'rxdb/plugins/core';
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
addRxPlugin(RxDBDevModePlugin);Schema Validation
Schema validation is required when using dev-mode and recommended (but optional) in production. Wrap your storage with the AJV schema validator to ensure all documents match your schema before being saved.
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
storage = wrappedValidateAjvStorage({ storage });Create a Database
A database is the topโlevel container in RxDB, responsible for managing collections, coordinating persistence, and providing reactive change streams.
import { createRxDatabase } from 'rxdb/plugins/core';
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: storage
});Add a Collection
An RxDatabase contains RxCollections for storing and querying data. A collection is similar to an SQL table, and individual records are stored in the collection as JSON documents. An RxDatabase can have as many collections as you need.
Add a collection with a schema to the database:
await myDatabase.addCollections({
// name of the collection
todos: {
// we use the JSON-schema standard
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have maxLength
},
name: {
type: 'string'
},
done: {
type: 'boolean',
default: false
},
dueAt: {
type: 'string',
format: 'date-time',
maxLength: 30
}
},
required: ['id', 'name', 'done', 'dueAt']
}
}
});Insert a Document
Now that we have an RxCollection we can store some documents in it.
const myDocument = await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
done: false,
dueAt: '2026-01-01T10:00:00.000Z'
});Run a Query
Execute a query that returns all found documents once:
const foundDocuments = await myDatabase.todos.find({
selector: {
done: {
$eq: false
}
}
}).exec();Update a Document
In the first found document, set done to true:
const firstDocument = foundDocuments[0];
await firstDocument.patch({
done: true
});Delete a Document
Delete the document so that it can no longer be found in queries:
await firstDocument.remove();Observe a Query
Subscribe to data changes so that your UI is always up-to-date with the data stored on disk. RxDB allows you to subscribe to data changes even when the change happens in another part of your application, another browser tab, or during database replication/synchronization:
const observable = myDatabase.todos.find({
selector: {
done: {
$eq: false
}
}
}).$ // get the observable via RxQuery.$;
observable.subscribe(notDoneDocs => {
console.log('Currently have ' + notDoneDocs.length + ' things to do');
// -> here you would re-render your app to show the updated document list
});Observe a Document Value
You can also subscribe to the fields of a single RxDocument. Add the $ sign to the desired field and then subscribe to the returned observable.
myDocument.done$.subscribe(isDone => {
console.log('done: ' + isDone);
});Sync the Client
RxDB has multiple replication plugins to replicate database state with a server.
HTTP
import {
replicateHTTP,
pullQueryBuilderFromRxSchema,
} from "rxdb/plugins/replication-http";
replicateHTTP({
collection: myDatabase.todos,
push: {
handler: async (rows) => {
return fetch("https://example.com/api/todos/push", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(rows),
}).then((res) => res.json());
},
},
pull: {
handler: async (lastCheckpoint) => {
return fetch(
"https://example.com/api/todos/pull?" +
new URLSearchParams({
checkpoint: JSON.stringify(lastCheckpoint)
}),
).then((res) => res.json());
},
},
});GraphQL
import { replicateGraphQL } from 'rxdb/plugins/replication-graphql';
replicateGraphQL({
collection: myDatabase.todos,
url: 'https://example.com/graphql',
push: { batchSize: 50 },
pull: { batchSize: 50 }
});WebRTC (P2P)
The easiest way to replicate data between your clients' devices is the WebRTC replication plugin that replicates data between devices without a centralized server. This makes it easy to try out replication without having to host anything:
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
replicateWebRTC({
collection: myDatabase.todos,
connectionHandlerCreator: getConnectionHandlerSimplePeer({}),
topic: '', // <- set any app-specific room id here.
secret: 'mysecret',
pull: {},
push: {}
})CouchDB
import { replicateCouchDB } from 'rxdb/plugins/replication-couchdb';
replicateCouchDB({
collection: myDatabase.todos,
url: 'http://example.com/todos/',
push: {},
pull: {}
});And more...
Explore all replication plugins, including advanced conflict handling and custom protocols.
Next steps
You are now ready to dive deeper into RxDB.
- Start reading the full documentation here.
- There is a full implementation of the quickstart guide so you can clone that repository and play with the code.
- For frameworks and runtimes like Angular, React Native and others, check out the list of example implementations.
- Also please continue reading the documentation, join the community on our Discord chat, and star the GitHub repo.
- If you are using RxDB in a production environment and are able to support its continued development, please take a look at the ๐ Premium package which includes additional plugins and utilities.
Docs Src Docs Adapters (`docs-src/docs/adapters.md`)
title: PouchDB Adapters slug: adapters.html image: /headers/adapters.jpg
PouchDB Adapters
When you use PouchDB RxStorage, there are many adapters that define where the data has to be stored.
Depending on which environment you work in, you can choose between different adapters. For example, in the browser you want to store the data inside of IndexedDB but on NodeJS you want to store the data on the filesystem.
This page is an overview over the different adapters with recommendations on what to use where.
:::warning
The PouchDB RxStorage is removed from RxDB and can no longer be used in new projects. You should switch to a different RxStorage.
:::
Please always ensure that your pouchdb adapter-version is the same as pouchdb-core in the rxdb package.json. Otherwise, you might have strange problems.
Any environment
Memory
In any environment, you can use the memory-adapter. It stores the data in the javascript runtime memory. This means it is not persistent and the data is lost when the process terminates.
Use this adapter when:
- You want to have really good performance
- You do not want persistent state, for example in your test suite
import {
createRxDatabase
} from 'rxdb'
import {
getRxStoragePouch
} from 'rxdb/plugins/pouchdb';
// npm install pouchdb-adapter-memory --save
addPouchPlugin(require('pouchdb-adapter-memory'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('memory')
});Memdown
With RxDB you can also use adapters that implement abstract-leveldown like the memdown-adapter.
// npm install memdown --save
// npm install pouchdb-adapter-leveldb --save
// leveldown adapters need the leveldb plugin
addPouchPlugin(require('pouchdb-adapter-leveldb'));
const memdown = require('memdown');
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch(memdown) // the full leveldown-module
});Browser
IndexedDB
The IndexedDB adapter stores the data inside of IndexedDB use this in browsers environments as default.
// npm install pouchdb-adapter-idb --save
addPouchPlugin(require('pouchdb-adapter-idb'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('idb')
});IndexedDB
A reimplementation of the indexeddb adapter which uses native secondary indexes. Should have a much better performance but can behave different on some edge cases.
:::note
Multiple users have reported problems with this adapter. It is not recommended to use this adapter.
:::
// npm install pouchdb-adapter-indexeddb --save
addPouchPlugin(require('pouchdb-adapter-indexeddb'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('indexeddb')
});Websql
This adapter stores the data inside of websql. It has a different performance behavior. Websql is deprecated. You should not use the websql adapter unless you have a really good reason.
// npm install pouchdb-adapter-websql --save
addPouchPlugin(require('pouchdb-adapter-websql'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('websql')
});NodeJS
leveldown
This adapter uses a LevelDB C++ binding to store that data on the filesystem. It has the best performance compared to other filesystem adapters. This adapter can not be used when multiple nodejs-processes access the same filesystem folders for storage.
// npm install leveldown --save
// npm install pouchdb-adapter-leveldb --save
// leveldown adapters need the leveldb plugin
addPouchPlugin(require('pouchdb-adapter-leveldb'));
const leveldown = require('leveldown');
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch(leveldown) // the full leveldown-module
});
// or use a specific folder to store the data
const database = await createRxDatabase({
name: '/root/user/project/mydatabase',
storage: getRxStoragePouch(leveldown) // the full leveldown-module
});Node-Websql
This adapter uses the node-websql-shim to store data on the filesystem. Its advantages are that it does not need a leveldb build and it can be used when multiple nodejs-processes use the same database-files.
// npm install pouchdb-adapter-node-websql --save
addPouchPlugin(require('pouchdb-adapter-node-websql'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('websql') // the name of your adapter
});
// or use a specific folder to store the data
const database = await createRxDatabase({
name: '/root/user/project/mydatabase',
storage: getRxStoragePouch('websql') // the name of your adapter
});React-Native
react-native-sqlite
Uses ReactNative SQLite as storage. Claims to be much faster than the asyncstorage adapter.
To use it, you have to do some steps from this tutorial.
First install pouchdb-adapter-react-native-sqlite and react-native-sqlite-2.
npm install pouchdb-adapter-react-native-sqlite react-native-sqlite-2Then you have to link the library.
react-native link react-native-sqlite-2You also have to add some polyfills which are need but not included in react-native.
npm install base-64 eventsimport { decode, encode } from 'base-64'
if (!global.btoa) {
global.btoa = encode;
}
if (!global.atob) {
global.atob = decode;
}
// Avoid using node dependent modules
process.browser = true;Then you can use it inside of your code.
import { createRxDatabase } from 'rxdb';
import { addPouchPlugin, getRxStoragePouch } from 'rxdb/plugins/pouchdb';
import SQLite from 'react-native-sqlite-2'
import SQLiteAdapterFactory from 'pouchdb-adapter-react-native-sqlite'
const SQLiteAdapter = SQLiteAdapterFactory(SQLite)
addPouchPlugin(SQLiteAdapter);
addPouchPlugin(require('pouchdb-adapter-http'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('react-native-sqlite') // the name of your adapter
});asyncstorage
Uses react-native's asyncstorage.
:::note
There are known problems with this adapter and it is not recommended to use it.
:::
// npm install pouchdb-adapter-asyncstorage --save
addPouchPlugin(require('pouchdb-adapter-asyncstorage'));
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch('node-asyncstorage') // the name of your adapter
});asyncstorage-down
A leveldown adapter that stores on asyncstorage.
// npm install pouchdb-adapter-asyncstorage-down --save
// leveldown adapters need the leveldb plugin
addPouchPlugin(require('pouchdb-adapter-leveldb'));
const asyncstorageDown = require('asyncstorage-down');
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch(asyncstorageDown) // the full leveldown-module
});Cordova / Phonegap / Capacitor
cordova-sqlite
Uses cordova's global cordova.sqlitePlugin. It can be used with cordova and capacitor.
// npm install pouchdb-adapter-cordova-sqlite --save
addPouchPlugin(require('pouchdb-adapter-cordova-sqlite'));
/**
* In capacitor/cordova you have to wait until
* all plugins are loaded and
* 'window.sqlitePlugin' can be accessed.
* This function waits until document deviceready
* is called which ensures everything is loaded.
* @link https://cordova.apache.org/docs/de/
* latest/cordova/events/events.deviceready.html
*/
export function awaitCapacitorDeviceReady(): Promise<void> {
return new Promise(res => {
document.addEventListener('deviceready', () => {
res();
});
});
}
async function getDatabase(){
// first wait until the deviceready event is fired
await awaitCapacitorDeviceReady();
const database = await createRxDatabase({
name: 'mydatabase',
storage: getRxStoragePouch(
'cordova-sqlite',
// pouch settings are passed as second parameter
{
// for ios, cordova-sqlite needs
// to know where to save data.
iosDatabaseLocation: 'Library'
}
)
});
}Docs Src Docs Alternatives (`docs-src/docs/alternatives.md`)
title: Alternatives for realtime local-first JavaScript applications and local databases slug: alternatives.html image: /headers/alternatives.jpg
Alternatives for realtime offline-first JavaScript applications
To give you an augmented view over the topic of client side JavaScript databases, this page contains all known alternatives to RxDB. Remember that you are reading this inside of the RxDB documentation, so everything is opinionated.
If you disagree with anything or think that something is missing, make a pull request to this file on the RxDB github repository.
:::note
RxDB has these main benefits:
- RxDB is a battle proven tool widely used by companies in real projects in production.
- RxDB is not VC funded and therefore does not require you to use a specific cloud service to rip you off. RxDB can be used with your own backend or no backend at all.
- RxDB has a working business model of selling premium plugins which ensures that RxDB will be maintained and improved continuously while many alternatives are dead already or seem to die soon.
- RxDB has years (since 2016) of performance optimization, bug fixing and feature adding. It is just working as is and there are close to zero open issues.
:::
Alternatives to RxDB
RxDB is an observable, replicating, local first, JavaScript database. So it makes only sense to list similar projects as alternatives, not just any database or JavaScript store library. However, I will list up some projects that RxDB is often compared with, even if it only makes sense for some use cases.
Here are the alternatives to RxDB:
Firebase
Firebase is a platform developed by Google for creating mobile and web applications. Firebase has many features and products, two of which are client side databases. The Realtime Database and the Cloud Firestore.
Firebase - Realtime Database
The firebase realtime database was the first database in firestore. It has to be mentioned that in this context, "realtime" means "realtime replication", not "realtime computing". The firebase realtime database stores data as a big unstructured JSON tree that is replicated between clients and the backend.
Firebase - Cloud Firestore
The firestore is the successor to the realtime database. The big difference is that it behaves more like a 'normal' database that stores data as documents inside of collections. The conflict resolution strategy of firestore is always last-write-wins which might or might not be suitable for your use case.
The biggest difference to RxDB is that firebase products are only able to be used on top of the Firebase cloud hosted backend, which creates a vendor lock-in. RxDB can replicate with any self hosted CouchDB server or custom GraphQL endpoints. You can even replicate Firestore to RxDB with the Firestore Replication Plugin.
Read more about why RxDB is a good alternative to Firebase.
Meteor
Meteor (since 2012) is one of the oldest technologies for JavaScript realtime applications. Meteor is not a library but a whole framework with its own package manager, database management and replication.
Because of how it works, it has proven to be hard to integrate it with other modern JavaScript frameworks like angular, vue.js or svelte.
Meteor uses MongoDB in the backend and can replicate with a Minimongo database in the frontend.
While testing, it has proven to be impossible to make a meteor app offline first capable. There are some projects that might do this, but all are unmaintained.
Read more about why RxDB is a good alternative to Meteor.
Minimongo
Forked in Jan 2014 from meteorJSs' minimongo package, Minimongo is a client-side, in-memory, JavaScript version of MongoDB with backend replication over HTTP. Similar to MongoDB, it stores data in documents inside of collections and also has the same query syntax. Minimongo has different storage adapters for IndexedDB, WebSQL, LocalStorage and SQLite.
Compared to RxDB, Minimongo has no concept of revisions or conflict handling, which might lead to undefined behavior when used with replication or in multiple browser tabs. Minimongo has no observable queries or changestream.
Read more about why RxDB is a good alternative to Minimongo.
WatermelonDB
WatermelonDB is a reactive & asynchronous JavaScript database. While originally made for React and React Native, it can also be used with other JavaScript frameworks. The main goal of WatermelonDB is performance within an application with lots of data.
In React Native, WatermelonDB uses the provided SQLite database. Also there is an Expo plugin for WatermelonDB. In a browser, WatermelonDB uses the LokiJS in-memory database to store and query data. WatermelonDB is one of the rare projects that support both Flow and Typescript at the same time.
Read more about why RxDB is a good alternative to WatermelonDB.
AWS Amplify
AWS Amplify is a collection of tools and libraries to develop web- and mobile frontend applications. Similar to firebase, it provides everything needed like authentication, analytics, a REST API, storage and so on. Everything hosted in the AWS Cloud, even when they state that "AWS Amplify is designed to be open and pluggable for any custom backend or service". For realtime replication, AWS Amplify can connect to an AWS App-Sync GraphQL endpoint.
AWS Datastore
Since December 2019 the Amplify library includes the AWS Datastore which is a document-based, client side database that is able to replicate data via AWS AppSync in the background.
The main difference to other projects is the complex project configuration via the amplify cli and the bit confusing query syntax that works over functions. Complex Queries with multiple OR/AND statements are not possible which might change in the future.
Local development is hard because the AWS AppSync mock does not support realtime replication. It also is not really offline-first because a user login is always required.
DataStore is deprecated. Amplify Gen 1 (which includes DataStore) entered maintenance mode in 2024 and reaches end-of-life on May 1, 2027. Amplify Gen 2 does not include a DataStore replacement, so teams must implement their own offline sync layer before that date.
// An AWS datastore OR query
const posts = await DataStore.query(Post, c => c.or(
c => c.rating("gt", 4).status("eq", PostStatus.PUBLISHED)
));
// An AWS datastore SORT query
const posts = await DataStore.query(Post, Predicates.ALL, {
sort: s => s.rating(SortDirection.ASCENDING).title(SortDirection.DESCENDING)
});The biggest difference to RxDB is that you have to use the AWS cloud backends. This might not be a problem if your data is at AWS anyway.
Read more about why RxDB is a good alternative to AWS Amplify DataStore.
RethinkDB
RethinkDB is a backend database that pushed dynamic JSON data to the client in realtime. It was founded in 2009 and the company shut down in 2016.
RethinkDB is not a client side database, it streams data from the backend to the client which of course does not work while offline.
Read more about why RxDB is a good alternative to RethinkDB.
Horizon
Horizon is the client side library for RethinkDB which provides useful functions like authentication, permission management and subscription to a RethinkDB backend. Offline support never made it to horizon.
Read more about why RxDB is a good alternative to Horizon.
Supabase
Supabase labels itself as "an open source Firebase alternative". It is a collection of open source tools that together mimic many Firebase features, most of them by providing a wrapper around a PostgreSQL database. While it has realtime queries that run over the wire, like with RethinkDB, Supabase has no client-side storage or replication feature and therefore is not offline first.
Read more about why RxDB is a good alternative to Supabase.
CouchDB
Apache CouchDB is a server-side, document-oriented database that is mostly known for its multi-master replication feature. Instead of having a master-slave replication, with CouchDB you can run replication in any constellation without having a master server as bottleneck where the server even can go off- and online at any time. This comes with the drawback of having a slow replication with much network overhead.
CouchDB has a changestream and a query syntax similar to MongoDB.
Read more about why RxDB is a good alternative to CouchDB.
PouchDB
PouchDB is a JavaScript database that is compatible with most of the CouchDB API. It has an adapter system that allows you to switch out the underlying storage layer. There are many adapters like for IndexedDB, SQLite, the Filesystem and so on. The main benefit is to be able to replicate data with any CouchDB compatible endpoint.
Because of the CouchDB compatibility, PouchDB has to do a lot of overhead in handling the revision tree of documents, which is why it can show bad performance for bigger datasets.
RxDB was originally build around PouchDB until the storage layer was abstracted out in version 10.0.0 so it now allows to use different RxStorage implementations. PouchDB has some performance issues because of how it has to store the document revision tree to stay compatible with the CouchDB API.
Read more about why RxDB is a good alternative to PouchDB.
Couchbase
Couchbase (originally known as Membase) is another NoSQL document database made for realtime applications.
It uses the N1QL query language which is more SQL like compared to other NoSQL query languages. In theory you can achieve replication of a Couchbase with a PouchDB database, but this has shown to be not that easy.
Read more about why RxDB is a good alternative to Couchbase.
Cloudant
Cloudant is a cloud-based service that is based on CouchDB and has mostly the same features.
It was originally designed for cloud computing where data can automatically be distributed between servers. But it can also be used to replicate with frontend PouchDB instances to create scalable web applications.
It was bought by IBM in 2014 and since 2018 the Cloudant Shared Plan is retired and migrated to IBM Cloud.
Read more about why RxDB is a good alternative to Cloudant.
Hoodie
Hoodie is a backend solution that enables offline-first JavaScript frontend development without having to write backend code. Its main goal is to abstract away configuration into simple calls to the Hoodie API.
It uses CouchDB in the backend and PouchDB in the frontend to enable offline-first capabilities.
The last commit for hoodie was one year ago and the website (hood.ie) is offline which indicates it is not an active project anymore.
Read more about why RxDB is a good alternative to Hoodie.
LokiJS
LokiJS is a JavaScript embeddable, in-memory database. And because everything is handled in-memory, LokiJS has awesome performance when mutating or querying data. You can still persist to a permanent storage (IndexedDB, Filesystem etc.) with one of the provided storage adapters. The persistence happens after a timeout is reached after a write, or before the JavaScript process exits. This also means you could loose data when the JavaScript process exits ungracefully like when the power of the device is shut down or the browser crashes.
While the project is not that active anymore, it is more finished than unmaintained.
In the past, RxDB supported using LokiJS as RxStorage but because the LokiJS is not maintained anymore and had too many issues, this storage option was removed in RxDB version 16.
Read more about why RxDB is a good alternative to LokiJS.
Gundb
GUN is a JavaScript graph database. While having many features, the decentralized replication is the main unique selling point. You can replicate data Peer-to-Peer without any centralized backend server. GUN has several other features that are useful on top of that, like encryption and authentication.
While testing it was really hard to get basic things running. GUN is open source, but because of how the source code is written, it is very difficult to understand what is going wrong.
Read more about why RxDB is a good alternative to Gundb.
sql.js
sql.js is a javascript library to run SQLite on the web. It uses a virtual database file stored in memory and does not have any persistence. All data is lost once the JavaScript process exits. sql.js is created by compiling SQLite to WebAssembly so it has about the same features as SQLite. For older browsers there is a JavaScript fallback.
Read more about why RxDB is a good alternative to sql.js.
absurd-sQL
Absurd-sql is a project that implements an IndexedDB-based persistence for sql.js. Instead of directly writing data into the IndexedDB, it treats IndexedDB like a disk and stores data in blocks there which shows to have a much better performance, mostly because of how performance expensive IndexedDB transactions are.
Read more about why RxDB is a good alternative to absurd-sQL.
NeDB
NeDB was a embedded persistent or in-memory database for Node.js, nw.js, Electron and browsers.
It is document-oriented and had the same query syntax as MongoDB.
Like LokiJS it has persistence adapters for IndexedDB etc. to persist the database state on the disc.
The last commit to NeDB was in 2016.
Read more about why RxDB is a good alternative to NeDB.
Dexie.js
Dexie.js is a minimalistic wrapper for IndexedDB. While providing a better API than plain IndexedDB, Dexie also improves performance by batching transactions and other optimizations. It also adds additional non-IndexedDB features like observable queries or multi tab support or react hooks.
Compared to RxDB, Dexie.js does not support complex (MongoDB-like) queries and requires a lot of fiddling when a document range of a specific index must be fetched.
Dexie.js is used by Whatsapp Web, Microsoft To Do and Github Desktop.
RxDB supports using Dexie.js as Database storage which enhances IndexedDB via dexie with RxDB features like MongoDB-like queries etc.
Read more about why RxDB is a good alternative to Dexie.js.
LowDB
LowDB is a small, local JSON database powered by the Lodash library. It is designed to be simple, easy to use, and straightforward. LowDB allows you to perform native JavaScript queries and persist data in a flat JSON file. Written in TypeScript, it's particularly well-suited for small projects, prototyping, or when you need a lightweight, file-based database.
As an alternative to LowDB, RxDB offers real-time reactivity, allowing developers to subscribe to database changes, a feature not natively available in LowDB. Additionally, RxDB provides robust query capabilities, including the ability to subscribe to query results for automatic UI updates. These features make RxDB a strong alternative to LowDB for more complex and dynamic applications.
Read more about why RxDB is a good alternative to LowDB.
localForage
localForage is a popular JavaScript library for offline storage that provides a simple, promise-based API. It abstracts over different storage mechanisms such as IndexedDB, WebSQL, or localStorage, making it easier to write code once and have it work seamlessly across various browsers. While localForage is great for storing data locally in a key-value manner, it doesn't provide the real-time reactive queries, conflict handling, or revision-based replication that RxDB does. This makes localForage a useful choice for straightforward caching or persistent storage needs, but not ideal for advanced offline-first scenarios requiring multi-user collaboration or complex querying.
Read more about why RxDB is a good alternative to localForage.
MongoDB Realm
Originally Realm was a mobile database for Android and iOS. Later they added support for other languages and runtimes, also for JavaScript.
It was meant as replacement for SQLite but is more like an object store than a full SQL database.
In 2019 MongoDB bought Realm and changed the projects focus.
Now Realm is made for replication with the MongoDB Realm Sync based on the MongoDB Atlas Cloud platform. This tight coupling to the MongoDB cloud service is a big downside for most use cases.
Read more about why RxDB is a good alternative to MongoDB Realm. If you plan to switch, follow the Realm to RxDB migration guide.
Apollo
The Apollo GraphQL platform is made to transfer data between a server to UI applications over GraphQL endpoints. It contains several tools like GraphQL clients in different languages or libraries to create GraphQL endpoints.
While it is has different caching features for offline usage, compared to RxDB it is not fully offline first because caching alone does not mean your application is fully usable when the user is offline.
Read more about why RxDB is a good alternative to Apollo.
Replicache
Replicache is a client-side sync framework for building realtime, collaborative, local-first web apps. It claims to work with most backend stacks. In contrast to other local first tools, replicache does not work like a local database. Instead it runs on so called mutators that unify behavior on the client and server side. So instead of implementing and calling REST routes on both sides of your stack, you will implement mutators that define a specific delta behavior based on the input data. To observe data in replicache, there are subscriptions that notify your frontend application about changes to the state.
Replicache can be used in most frontend technologies like browsers, React/Remix, NextJS/Vercel and React Native. While Replicache can be installed and used from npm, the Replicache source code is not open source and the Replicache github repo does not allow you to inspect or debug it. Still you can use replicache for in non-commercial projects, or for companies with < $200k revenue (ARR) and < $500k in funding. (2024: Replicache will be free and Rocicorp are working on a new Zerosync product to succeed Replicache and Reflect.)
Read more about why RxDB is a good alternative to Replicache.
InstantDB
InstantDB is designed for real-time data synchronization with built-in offline support, allowing changes to be queued locally and synced when the user reconnects. While it offers seamless optimistic updates and rollback capabilities, its offline-first design is not as mature or comprehensive as RxDB's - the offline data is more of a cache, not a full-database sync. The query language used is Datalog, and the backend sync service is written in Clojure. InstantDB is focused more on simplicity and real-time collaboration, with fewer customization options for storage or conflict resolution compared to RxDB, which supports various storage adapters and advanced conflict handling via CRDTs.
Read more about why RxDB is a good alternative to InstantDB.
Yjs
Yjs is a CRDT-based (Conflict-free Replicated Data Type) library focused on enabling real-time collaboration - particularly for text editing, although it can handle other data types as well. While it provides powerful conflict resolution and peer-to-peer synchronization out of the box, Yjs itself is not a full-fledged database. Instead, you typically combine Yjs with other storage or networking layers to achieve a local-first architecture. This flexibility allows for sophisticated real-time features, but also means you must handle indexing, queries, and persistence on your own if you need them. Compared to RxDB, Yjs does not offer built-in replication adapters or a query system, so developers who require a more complete solution for conflict resolution, data persistence, and offline-first capabilities may find RxDB more convenient.
Read more about why RxDB is a good alternative to Yjs.
ElectricSQL
2024: ElectricSQL is being rewritten in a new Electric-Next branch, which focuses on partial syncing of ("shapes", which makes is basically a NoSQL like document database) of data from a remote Postgres DB to a local clients written in TypeScript/JS or Elixir. The write path is not yet implemented, neither is client-side reactivity. The ElectricSQL backend is written in Elixir.
Read more about why RxDB is a good alternative to ElectricSQL.
SignalDB
SignalDB provides a reactive, in-memory local-first JavaScript database with real-time sync, but it doesn't offer the same level of multi-client replication or flexibility with storage backends that RxDB provides, and through a RxDB persistence adapters you can actually use SignalDB for the front-end reactivity while relying on RxDB for backend sync and persistence.
Read more about why RxDB is a good alternative to SignalDB.
Ditto
Ditto is a closed-source, edge-native database whose native SDKs sync device-to-device over Bluetooth LE, peer-to-peer WiFi, and LAN without a server, which is a capability RxDB does not have. But the mesh is native only: per the Ditto docs, web browsers cannot use those peer-to-peer transports and the Web SDK connects to Ditto Server over WebSockets, so for a JavaScript client the connection shape is the same as RxDB WebSocket replication. Ditto Cloud is metered per device connection, storage, and data transfer, while the RxDB core is Apache 2.0 with a backend you own.
Read more about why RxDB is a good alternative to Ditto.
PowerSync
PowerSync is a "framework" for implementing local-first solutions. It centralizes business logic and conflict resolution on a central, authoritative server (PostgreSQL or MongoDB), vs RxDB that also supports custom backends. Both RxDB and PowerSync can be used with a variety of storage backends, but PowerSync uses SQLite as the front-end database which has shown to be slow because the WASM-SQLite abstraction increases read and write latency. In terms of client SDKs, PowerSync offers Flutter, Kotlin, and Swift in addition to JS/TypeScript. PowerSync offers many client technologies, PowerSync is under a license that restricts commercial use that competes with PowerSync and the JourneyApps Platform.
Read more about why RxDB is a good alternative to PowerSync.
Read further
Docs Src Docs Backup (`docs-src/docs/backup.md`)
title: Backup slug: backup.html image: /headers/backup.jpg
import {DocsAnimation} from '@site/src/components/docs-animation';
import {BackupCopy} from '@site/src/components/animations/backup-copy';
๐ฅ Backup Plugin
With the backup plugin you can write the current database state and ongoing changes into folders on the filesystem.
The files are written in plain json together with their attachments so that you can read them out with any software or tools, without being bound to RxDB.
This is useful to:
- Consume the database content with other software that cannot replicate with RxDB
- Write a backup of the database to a remote server by mounting the backup folder on the other server.
The backup plugin works only in node.js, not in a browser. It is intended to have a backup strategy when using RxDB on the server side like with the RxServer. To run backups on the client side, you should use one of the replication plugins instead, or write the data into a single JSON dump that you can store or hand to the user.
Installation
import { addRxPlugin } from 'rxdb';
import { RxDBBackupPlugin } from 'rxdb/plugins/backup';
addRxPlugin(RxDBBackupPlugin);one-time backup
Write the whole database to the filesystem once.
When called multiple times, it will continue from the last checkpoint and not start all over again.
const backupOptions = {
// if false, a one-time backup will be written
live: false,
// the folder where the backup will be stored
directory: '/my-backup-folder/',
// if true, attachments will also be saved
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
await backupState.awaitInitialBackup();
// call again to run from the last checkpoint
const backupState2 = myDatabase.backup(backupOptions);
await backupState2.awaitInitialBackup();live backup
When live: true is set, the backup will write all ongoing changes to the backup directory.
const backupOptions = {
// set live: true to have an ongoing backup
live: true,
directory: '/my-backup-folder/',
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
// you can still await the initial backup write,
// but further changes will still be processed.
await backupState.awaitInitialBackup();writeEvents$
You can listen to the writeEvents$ Observable to get notified about written backup files.
const backupOptions = {
live: false,
directory: '/my-backup-folder/',
attachments: true
}
const backupState = myDatabase.backup(backupOptions);
const subscription = backupState.writeEvents$
.subscribe(writeEvent =>
console.dir(writeEvent)
);
/*
> {
collectionName: 'humans',
documentId: 'foobar',
files: [
'/my-backup-folder/foobar/document.json'
],
deleted: false
}
*/Limitations
- It is currently not possible to import from a written backup. If you need this functionality, please make a pull request.
Docs Src Docs Capacitor Database (`docs-src/docs/capacitor-database.md`)
title: Capacitor Database Guide - SQLite, RxDB & More slug: capacitor-database.html image: /headers/capacitor-database.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
Capacitor Database - SQLite, RxDB and others
Capacitor is an open source native JavaScript runtime to build Web based Native apps. You can use it to create cross-platform iOS, Android, and Progressive Web Apps with the web technologies JavaScript, HTML, and CSS.
It is developed by the Ionic Team and provides a great alternative to create hybrid apps. Compared to React Native, Capacitor is more Web-Like because the JavaScript runtime supports most Web APIs like IndexedDB, fetch, and so on.
To read and write persistent data in Capacitor, there are multiple solutions which are shown in the following.
Database Solutions for Capacitor
Preferences API
Capacitor comes with a native Preferences API which is a simple, persistent key->value store for lightweight data, similar to the browsers localstorage or React Native AsyncStorage.
To use it, you first have to install it from npm npm install @capacitor/preferences and then you can import it and write/read data.
Notice that all calls to the preferences API are asynchronous so they return a Promise that must be await-ed.
import { Preferences } from '@capacitor/preferences';
// write
await Preferences.set({
key: 'foo',
value: 'baar',
});
// read
const { value } = await Preferences.get({ key: 'foo' }); // > 'bar'
// delete
await Preferences.remove({ key: 'foo' });The preferences API is good when only a small amount of data needs to be stored and when no query capabilities besides the key access are required. Complex queries or other features like indexes or replication are not supported which makes the preferences API not suitable for anything more than storing simple data like user settings.
Localstorage/IndexedDB/WebSQL
Since Capacitor apps run in a web view, Web APIs like IndexedDB, Localstorage and WebSQL are available. But the default browser behavior is to clean up these storages regularly when they are not in use for a long time or the device is low on space. Therefore you cannot 100% rely on the persistence of the stored data and your application needs to expect that the data will be lost eventually.
Storing data in these storages can be done in browsers, because there is no other option. But in Capacitor iOS and Android, you should not rely on these.
SQLite
SQLite is a SQL based relational database written in C that was crafted to be embed inside of applications. Operations are written in the SQL query language and SQLite generally follows the PostgreSQL syntax.
To use SQLite in Capacitor, there are three options:
- The @capacitor-community/sqlite package
- The cordova-sqlite-storage package
- The non-free Ionic Secure Storage which comes at 999$ per month.
It is recommended to use the @capacitor-community/sqlite because it has the best maintenance and is open source. Install it first npm install --save @capacitor-community/sqlite and then set the storage location for iOS apps:
{
"plugins": {
"CapacitorSQLite": {
"iosDatabaseLocation": "Library/CapacitorDatabase"
}
}
}Now you can create a database connection and use the SQLite database.
import { Capacitor } from '@capacitor/core';
import {
CapacitorSQLite, SQLiteDBConnection, SQLiteConnection, capSQLiteSet,
capSQLiteChanges, capSQLiteValues, capEchoResult, capSQLiteResult,
capNCDatabasePathResult
} from '@capacitor-community/sqlite';
const sqlite = new SQLiteConnection(CapacitorSQLite);
const database: SQLiteDBConnection = await this.sqlite.createConnection(
databaseName,
encrypted,
mode,
version,
readOnly
);
let { rows } = database.query('SELECT name FROM todos');The downside of SQLite is that it is lacking many features that are handful when using a database together with an UI based application like your Capacitor app. For example it is not possible to observe queries or document fields. Also there is no realtime replication feature, you can only import json files. This makes SQLite a good solution when you just want to store data on the client, but when you want to sync data with a server or other clients or create big complex realtime applications, you have to use something else.
RxDB
RxDB is an local first, NoSQL database for JavaScript Applications like hybrid apps. Because it is reactive, you can subscribe to all state changes like the result of a query or even a single field of a document. This is great for UI-based realtime applications in a way that makes it easy to develop realtime applications like what you need in Capacitor.
Because RxDB is made for Web applications, most of the available RxStorage plugins can be used to store and query data in a Capacitor app. However it is recommended to use the SQLite RxStorage because it stores the data on the filesystem of the device, not in the JavaScript runtime (like IndexedDB). Storing data on the filesystem ensures it is persistent and will not be cleaned up by any process. Also the performance of SQLite is much faster compared to IndexedDB, because SQLite does not have to go through a browsers permission layers. For the SQLite binding you should use the @capacitor-community/sqlite package.
Because the SQLite RxStorage is part of the ๐ Premium Plugins which must be purchased, it is recommended to use the LocalStorage RxStorage while testing and prototyping your Capacitor app.
To use the SQLite RxStorage in Capacitor you have to install all dependencies via npm install rxdb rxjs rxdb-premium @capacitor-community/sqlite.
For iOS apps you should add a database location in your Capacitor settings:
{
"plugins": {
"CapacitorSQLite": {
"iosDatabaseLocation": "Library/CapacitorDatabase"
}
}
}Then you can assemble the RxStorage and create a database with it:
Import RxDB and SQLite
import {
createRxDatabase
} from 'rxdb/plugins/core';
import {
CapacitorSQLite,
SQLiteConnection
} from '@capacitor-community/sqlite';
import { Capacitor } from '@capacitor/core';
const sqlite = new SQLiteConnection(CapacitorSQLite);Import the RxDB SQLite Storage
RxDB Core
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsCapacitor
} from 'rxdb/plugins/storage-sqlite';RxDB Premium ๐
import {
getRxStorageSQLite,
getSQLiteBasicsCapacitor
} from 'rxdb-premium/plugins/storage-sqlite';Create a Database with the Storage
RxDB Core
// create database
const myDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsCapacitor(sqlite, Capacitor)
})
});RxDB Premium ๐
// create database
const myDatabase = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsCapacitor(sqlite, Capacitor)
})
});Add a Collection
// create collections
await myDatabase.addCollections({
todos: {
schema: {
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean', default: false },
priority: {
type: 'integer',
minimum: 0,
maximum: 10,
multipleOf: 1
}
},
required: ['id', 'name', 'done', 'priority']
}
}
});Insert a Document
await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
done: false,
priority: 8
});Run a Query
const result = await myDatabase.todos.find({
selector: {
done: false
}
}).exec();Observe a Query
myDatabase.todos.find({
selector: {
done: false
},
sort: [{ priority: 'desc' }]
}).$.subscribe(result => {/* ... */});FAQ
Capacitor is a lightweight, open-source JavaScript native runtime engineered by the Ionic team that bridges standard Web APIs into native iOS/Android builds. Unlike React Native, which bundles a proprietary JavaScript engine directly mapping to native UI components, Capacitor mounts a standard platform WebView capable of executing raw HTML/CSS/JS exactly as it runs in modern browsers. This allows seamless utilization of standard Web APIs like RxDB atop IndexedDB or OPFS without requiring platform-specific native C++ module compilation.
Follow up
- If you haven't done yet, you should start learning about RxDB with the Quickstart Tutorial.
- There is a followup list of other client side database alternatives.
Docs Src Docs Cleanup (`docs-src/docs/cleanup.md`)
title: Cleanup slug: cleanup.html image: /headers/cleanup.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {CleanupSweep} from '@site/src/components/animations/cleanup-sweep';
๐งน Cleanup
To make the replication work, and for other reasons, RxDB has to keep deleted documents in storage so that it can replicate their deletion state.
This ensures that when a client is offline, the deletion state is still known and can be replicated with the backend when the client goes online again.
Keeping too many deleted documents in the storage, can slow down queries or fill up too much disc space.
With the cleanup plugin, RxDB will run cleanup cycles that clean up deleted documents when it can be done safely.
Installation
import { addRxPlugin } from 'rxdb';
import { RxDBCleanupPlugin } from 'rxdb/plugins/cleanup';
addRxPlugin(RxDBCleanupPlugin);Create a database with cleanup options
You can set a specific cleanup policy when a RxDatabase is created. For most use cases, the defaults should be ok.
/* Detailed source-code truncated for AI context efficiency. */Calling cleanup manually
You can manually run a cleanup per collection by calling RxCollection.cleanup().
/**
* Manually run the cleanup with the
* minimumDeletedTime from the cleanupPolicy.
*/
await myRxCollection.cleanup();
/**
* Overwrite the minimumDeletedTime
* be setting it explicitly (time in milliseconds)
*/
await myRxCollection.cleanup(1000);
/**
* Purge all deleted documents no
* matter when they were deleted
* by setting minimumDeletedTime to zero.
*/
await myRxCollection.cleanup(0);Using the cleanup plugin to empty a collection
When you have a collection with documents and you want to empty it by purging all documents, the recommended way is to call myRxCollection.remove(). However, this will destroy the JavaScript class of the collection and stop all listeners and observables.
Sometimes the better option might be to manually delete all documents and then use the cleanup plugin to purge the deleted documents:
// delete all documents
await myRxCollection.find().remove();
// purge all deleted documents
await myRxCollection.cleanup(0);FAQ
The cleanup cycles are optimized to run only when the database is idle and it is unlikely that another database interaction performance will be decreased in the meantime. For example, by default, the cleanup does not run in the first 60 seconds of the creation of a collection to ensure the initial page load of your website does not slow down. Also, we use mechanisms like the requestIdleCallback() API to improve the correct timing of the cleanup cycle.
Docs Src Docs Contribute (`docs-src/docs/contribute.md`)
title: Contribute slug: contribution.html image: /headers/contribution.jpg
Contribution
We are open to, and grateful for, any contributions made by the community.
Developing
Requirements
Before you can start developing, do the following:
- Make sure you have installed nodejs with the version stated in the .nvmrc
- Clone the repository
git clone https://github.com/pubkey/rxdb.git - Install the dependencies
cd rxdb && npm install - Make sure that the tests work for you. At first, try it out with
npm run test:node:memorywhich tests the memory storage in node. In the package.json you can find more scripts to run the tests with different storages.
Adding tests
Before you start creating a bugfix or a feature, you should create a test to reproduce it. Tests are in the test/unit-folder.
If you want to reproduce a bug, you can modify the test in this file.
Making a PR
If you make a pull-request, ensure the following:
- Every feature or bugfix must be committed together with a unit-test which ensures everything works as expected.
- Do not commit build-files (anything in the
dist-folder) - Before you add non-trivial changes, create an issue to discuss if this will be merged and you don't waste your time.
- To run the unit and integration-tests, do
npm run testand ensure everything works as expected
Getting help
If you need help with your contribution, ask at discord.
No-Go
When reporting a bug, you need to make a PR with a test case that runs in the CI and reproduces your problem.
Sending a link with a repo does not help the maintainer because installing random peoples projects is time consuming and dangerous.
Also the maintainer will never go on a bug hunt based on your plain description. Either you report the bug with a test case, or the maintainer will likely not help you.
Docs
The source of the documentation is at the docs-src-folder.
To read the docs locally, run npm run docs:install && npm run docs:serve and open http://localhost:4000/
Thank you for contributing!
Docs Src Docs Crdt (`docs-src/docs/crdt.md`)
title: CRDT - Conflict-free replicated data type Database slug: crdt.html image: /headers/crdt.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {CrdtMerge} from '@site/src/components/animations/crdt-merge';
RxDB CRDT Plugin
Whenever there are multiple instances in a distributed system, data writes can cause conflicts. Two different clients could do a write to the same document at the same time or while they are both offline. When the clients replicate the document state with the server, a conflict emerges that must be resolved by the system.
In RxDB, conflicts are normally resolved by setting a conflictHandler when creating a collection. The conflict handler is a JavaScript function that gets the two conflicting states of the same document and it will return the resolved document state.
The default conflict handler will always drop the fork state and use the master state to ensure that clients that have been offline for a long time, do not overwrite other clients changes when they go online again.
With CRDTs (short for Conflict-free replicated data type), all document
writes are represented as CRDT operations in plain JSON. The CRDT operations are stored together with the document and each time a conflict arises, the CRDT conflict handler will automatically merge the operations in a deterministic way. Using CRDTs is an easy way to "magically" handle all conflict problems in your application by storing the deltas of writes together with the document data.
RxDB CRDT operations
In RxDB, a CRDT operation is defined with NoSQL update operators, like you might know them from MongoDB update operations or the RxDB update plugin.
To run the operators, RxDB uses the mingo library.
A CRDT operator example:
const myCRDTOperation = {
// increment the priority field by +1
$inc: {
priority: 1
},
// set the done field to true
$set: {
done: true
}
};Operators
At the moment, not all possible operators are implemented in mingo, if you need additional ones, you should make a pull request there.
The following operators can be used at this point in time:
$min$max$inc$set$unset$push$addToSet$pop$pullAll$rename
For the exact definition on how each operator behaves, check out the MongoDB documentation on update operators.
Installation
To use CRDTs with RxDB, you need the following:
- Add the CRDT plugin via
addRxPlugin. - Add a field to your schema that defines where to store the CRDT operations via
getCRDTSchemaPart() - Set the
crdtoptions in your schema. - Do NOT set a custom conflict handler, the plugin will use its own one.
// import the relevant parts from the CRDT plugin
import {
getCRDTSchemaPart,
RxDBcrdtPlugin
} from 'rxdb/plugins/crdt';
// add the CRDT plugin to RxDB
import { addRxPlugin } from 'rxdb';
addRxPlugin(RxDBcrdtPlugin);
// create a database
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageLocalstorage()
});
// create a schema with the CRDT options
const todoSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
priority: {
type: 'integer',
minimum: 0,
maximum: 10,
multipleOf: 1
},
crdts: getCRDTSchemaPart() // use this field to store the CRDT operations
},
required: ['id', 'name', 'priority'],
crdt: { // CRDT options
field: 'crdts'
}
}
// add a collection
await myDatabase.addCollections({
todos: {
schema: todoSchema
}
});
// insert a document
const myDocument = await myDatabase.todos.insert({
id: 'todo4',
name: 'Clean the kitchen',
priority: 1
});
// run a CRDT operation that increments the 'priority' by one
await myDocument.updateCRDT({
ifMatch: {
$inc: {
priority: 1
}
}
});Conditional CRDT operations
By default, all CRDTs operations will be run to build the current document state. But in many cases, more granular operations are required to better reflect the desired business logic. For these cases, conditional CRDTs can be used.
For example if you have a field priority with a maximum of 10, you might want to only run the $inc operation, if the priority value is less than 10.
In an conditional CRDT, you can specify a selector and the operation sets ifMatch and ifNotMatch. At each time the CRDT is applied to the document state, first the selector will run and evaluate which operations path must be used.
await myDocument.updateCRDT({
// only if the selector matches, the ifMatch operation will run
selector: {
priority: {
$lt: 10
}
},
// an operation that runs if the selector matches
ifMatch: {
$inc: {
priority: 1
}
},
// if the selector does NOT match, you could run a different operation instead
ifNotMatch: {
// ...
}
});Running multiples operations at once
By default, one CRDT operation is applied to the document in a single database write.
To represent more complex logic chains, it might make sense to use multiple CRDTs and write them at once inside of a single atomic document write.
For these cases, the updateCRDT() method allows to pass an array of operations.
await myDocument.updateCRDT([
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
},
{
selector: { /** ... **/ },
ifMatch: { /** ... **/ }
}
]);CRDTs on inserts
When CRDTs are enabled with the plugin, all insert operations are automatically mapped as CRDT operation with the $set operator.
// Calling RxCollection.insert()
await myDatabase.todos.insert({
id: 'todo4',
name: 'Clean the kitchen',
priority: 1
});
// is exactly equal to calling insertCRDT()
await myDatabase.todos.insertCRDT({
ifMatch: {
$set: {
id: 'todo4',
name: 'Clean the kitchen',
priority: 1
}
}
});When the same document is inserted in multiple client instances and then replicated, a conflict will emerge and the insert-CRDTs will overwrite each other in a deterministic order.
You can use insertCRDT() to make conditional insert operations with any logic. To check for the previous existence of a document, use the $exists query operation on the primary key of the document.
await myDatabase.todos.insertCRDT({
selector: {
// only run if the document did not exist before.
id: { $exists: false }
},
ifMatch: {
// if the document did not exist, insert it
$set: {
id: 'todo4',
name: 'Clean the kitchen',
priority: 1
}
},
ifNotMatch: {
// if document existed already, increment the priority by +1
$inc: {
priority: 1
}
}
});Deleting documents
You can delete a document with a CRDT operation by setting _deleted to true. Calling RxDocument.remove() will do exactly the same when CRDTs are activated.
await doc.updateCRDT({
ifMatch: {
$set: {
_deleted: true
}
}
});
// OR
await doc.remove();CRDTs with replication
CRDT operations are stored inside of a special field besides your 'normal' document fields.
When replicating document data with the RxDB replication or the CouchDB replication or even any custom replication, the CRDT operations must be replicated together with the document data as if they would be 'normal' a document property.
When any instances makes a write to the document, it is required to update the CRDT operations accordingly. For example if your custom backend updates a document, it must also do that by adding a CRDT operation. In dev-mode RxDB will refuse to store any document data where the document properties do not match the result of the CRDT operations.
Why not automerge.js or yjs?
There are already CRDT libraries out there that have been considered to be used with RxDB. The biggest ones are automerge and yjs. The decision was made to not use these but instead go for a more NoSQL way of designing the CRDT format because:
- Users do not have to learn a new syntax but instead can use the NoSQL query operations which they already know to manipulate the JSON data of a document.
- RxDB is often used to replicate data with any custom backend on an already existing infrastructure. Using NoSQL operators instead of binary data in CRDTs, makes it easy to implement the exact same logic on these backends so that the backend can also do document writes and still be compliant to the RxDB CRDT plugin.
So instead of using YJS or Automerge with a database, you can use RxDB with the CRDT plugin to have a more database specific CRDT approach. This gives you additional features for free such as schema validation or data migration.
When to not use CRDTs
CRDT can only be use when your business logic allows to represent document changes via static json operators.
If you can have cases where user interaction is required to correctly merge conflicting document states, you cannot use CRDTs for that.
Also when CRDTs are used, it is no longer allowed to do non-CRDT writes to the document properties.
CRDT Alternative
While the CRDT plugin can automatically merge concurrent document updates, it is not the only way to resolve conflicts in RxDB.
An alternative approach to CRDT is to use RxDB's built-in conflict handling system.
Why use conflict handlers instead of CRDT?
Conflict handlers offer a simpler and more flexible way to manage data conflicts. Instead of encoding changes as CRDT operations, you define how RxDB should decide which document version "wins" with plain JavaScript code. This approach is easier to reason about because it works directly with your domain logic. For example, you can compare timestamps, prioritize certain fields, or even involve user interaction to resolve conflicts.
Conflict handlers are:
- Easier to understand: you work with plain document states instead of CRDT operations.
- Fully customizable: you can define any merge strategy, from simple last-write-wins to complex rule-based logic.
- Compatible with all data types: unlike CRDTs, which are best suited for numeric or set-based updates.
- Transparent: you always know which state is being written and why.
Downsides of CRDTs
CRDTs are powerful for automatic conflict-free merging, but they also come with trade-offs:
- Higher conceptual complexity: CRDTs require understanding of operation semantics, version vectors, and merge determinism.
- Limited flexibility: you can only express changes that fit the supported JSON-style update operators.
- Difficult debugging: when merges don't behave as expected, it can be hard to trace the sequence of CRDT operations that led to a state.
- Overhead for simple cases: if your data rarely conflicts or needs human oversight, using CRDTs can add unnecessary complexity.
When to choose conflict handlers
Use conflict handlers as CRDT alternative if:
- You want full control over merge logic.
- Your data model includes contextual or user-specific decisions.
- You prefer a straightforward, rule-based resolution system over automatic merges.
Use CRDTs if:
- Your app performs frequent offline writes that can be merged deterministically.
- Your data can be represented as additive, numeric, or array-based updates.
- You want minimal manual intervention during replication.
Both methods are first-class citizens in RxDB. CRDTs focus on automatic, deterministic merging, while conflict handlers emphasize clarity, flexibility, and control.
Example: merging different fields with conflict handlers instead of CRDT
For example, imagine two users edit different fields of the same document at the same time. One updates the name, the other updates the priority. A custom conflict handler can merge both changes so no data is lost:
const mergeFieldsHandler = {
isEqual: (a, b) => JSON.stringify(a) === JSON.stringify(b),
resolve: (input) => {
return {
...input.realMasterState,
name: input.newDocumentState.name ?? input.realMasterState.name,
priority: Math.max(
input.newDocumentState.priority,
input.realMasterState.priority
)
};
}
};In this example, if the two versions change different properties, the final merged document includes both updates. This kind of logic is often easier to reason about than designing equivalent CRDT operations.
FAQ
RxDB provides a distributed database with conflict-free replication. You build offline-first applications using local data storage. RxDB synchronizes data across multiple client devices. The CRDT plugin resolves data conflicts automatically during replication. You maintain continuous data consistency without manual merge logic.
A Conflict-free Replicated Data Type (CRDT) works by transforming all data writes into atomic mathematical operations (like $inc or $set) rather than absolute state replacements. When two offline clients modify the exact same document simultaneously, a CRDT merges these operations in a guaranteed deterministic order upon reconnection. Because the merge logic relies on commutative mathematics, conflicts are resolved automatically without requiring manual developer intervention or user prompts.
The best CRDT databases for scaling fully distributed, masterless applications are those prioritizing dynamic topology mappings without rigid server connections. RxDB coupled with its dedicated CRDT Plugin operates flawlessly without a central authority. It allows seamless peer-to-peer data replication through WebRTC or WebSocket adapters, automatically running NoSQL-based CRDT operations locally to achieve absolute eventual consistency across all distributed nodes.
Finding a unified, production-ready CRDT database ecosystem is challenging, as many solutions are simply raw algorithmic libraries (like Yjs or Automerge). RxDB offers a comprehensive, reliable CRDT environment built natively on top of standard NoSQL query selectors. Rather than forcing developers to learn complex binary array manipulation, RxDB's CRDT plugin resolves conflict-free data sync using familiar MongoDB-style JSON update commands like $inc and $push.
{/*
TODOs
- Clean up old CRDT operations by crunching them together
- CRDT streaming replication
*/}
Docs Src Docs Data Migration (`docs-src/docs/data-migration.md`)
title: Data Migration slug: data-migration.html image: /headers/data-migration.jpg
This documentation page has been moved to here
Docs Src Docs Dev Mode (`docs-src/docs/dev-mode.md`)
title: Development Mode slug: dev-mode.html image: /headers/dev-mode.jpg
import {Steps} from '@site/src/components/steps';
Dev Mode
The dev-mode plugin adds many checks and validations to RxDB.
This ensures that you use the RxDB API properly and so the dev-mode plugin should always be used when
using RxDB in development mode.
- Adds readable error messages.
- Ensures that
readonlyJavaScript objects are not accidentally mutated. - Adds validation check for validity of schemas, queries, ORM methods and document fields.
- Notice that the
dev-modeplugin does not perform schema checks against the data see schema validation for that.
- Notice that the
:::warning
The dev-mode plugin will increase your build size and decrease the performance. It must always be used in development. You should never use it in production.
:::
Import the dev-mode Plugin
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { addRxPlugin } from 'rxdb/plugins/core';Add the Plugin to RxDB
addRxPlugin(RxDBDevModePlugin);Usage with Node.js
async function createDb() {
if (process.env.NODE_ENV !== "production") {
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
}Usage with [Angular](./articles/angular-database.md)
import { isDevMode } from '@angular/core';
async function createDb() {
if (isDevMode()){
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
// ...
}Usage with webpack
In the webpack.config.js:
module.exports = {
entry: './src/index.ts',
/* ... */
plugins: [
// set a global variable that can be accessed during runtime
new webpack.DefinePlugin({ MODE: JSON.stringify("production") })
]
/* ... */
};In your source code:
declare var MODE: 'production' | 'development';
async function createDb() {
if (MODE === 'development') {
await import('rxdb/plugins/dev-mode').then(
module => addRxPlugin(module.RxDBDevModePlugin)
);
}
const db = await createRxDatabase( /* ... */ );
// ...
}Disable the dev-mode warning
When the dev-mode is enabled, it will print a console.warn() message to the console so that you do not accidentally use the dev-mode in production. To disable this warning you can call the disableWarnings() function.
import { disableWarnings } from 'rxdb/plugins/dev-mode';
disableWarnings();Disable the tracking iframe
When used in localhost and in the browser, the dev-mode plugin can add a tracking iframe to the DOM. This is used to track the effectiveness of marketing efforts of RxDB.
If you have premium access and want to disable this iframe, you can call setPremiumFlag() before creating the database.
import { setPremiumFlag } from 'rxdb-premium/plugins/shared';
setPremiumFlag();Docs Src Docs Downsides Of Offline First (`docs-src/docs/downsides-of-offline-first.md`)
title: Downsides of Local First / Offline First slug: downsides-of-offline-first.html image: /headers/downsides-of-offline-first.jpg
import {QuoteBlock} from '@site/src/components/quoteblock';
import {CenteredImage} from '@site/src/components/centered-image';
Downsides of Local First / Offline First
So you have read all these things about how the local-first (aka offline-first) paradigm makes it easy to create realtime web applications that even work when the user has no internet connection.
But there is no free lunch. The offline first paradigm is not the perfect approach for all kinds of apps.
<QuoteBlock
author="Daniel"
year="2024"
sourceLink="https://github.com/pubkey"
You fully understood a technology when you know when not to use it
In the following I will point out the limitations you need to know before you decide to use RxDB or even before you decide to create an offline first application.
It only works with small datasets
Making data available offline means it must be loaded from the server and then stored at the clients device.
You need to load the full dataset on the first page load and on every ongoing load you need to download the new changes to that set.
While in theory you could download in infinite amount of data, in practice you have a limit how long the user can wait before having an up-to-date state.
You want to display chat messages like Whatsapp? No problem. Syncing all the messages a user could write, can be done with a few HTTP requests.
Want to make a tool that displays server logs? Good luck downloading terabytes of data to the client just to search for a single string. This will not work.
Besides the network usage, there is another limit for the size of your data.
In browsers you have some options for storage: Cookies, Localstorage, WebSQL and IndexedDB.
Because Cookies and Localstorage is slow and WebSQL is deprecated, you will use IndexedDB.
The limit of how much data you can store in IndexedDB depends on two factors: Which browser is used and how much disc space is left on the device. You can assume that at least a couple of hundred megabytes are available at least. The maximum is potentially hundreds of gigabytes or more, but the browser implementations vary. Chrome allows the browser to use up to 60% of the total disc space per origin. Firefox allows up to 50%. But on safari you can only store up to 1GB and the browser will prompt the user on each additional 200MB increment.
The problem is, that you have no chance to really predict how much data can be stored. So you have to make assumptions that are hopefully true for all of your users. Also, you have no way to increase that space like you would add another hard drive to your backend server. Once your clients reach the limit, you likely have to rewrite big parts of your applications.
UPDATE (2023): Newer versions of browsers can store way more data, for example firefox stores up to 10% of the total disk size. For an overview about how much can be stored, read this guide
Browser storage is not really persistent
When data is stored inside IndexedDB or one of the other storage APIs, it cannot be trusted to stay there forever.
Apple for example deletes the data when the website was not used in the last 7 days. The other browsers also have logic to clean up the stored data, and in the end the user itself could be the one that deletes the browsers local data.
The most common way to handle this, is to replicate everything from the backend to the client again.
Of course, this does not work for state that is not stored at the backend. So if you assume you can store the users private data inside the browser in a secure way, you are wrong.
There can be conflicts
Imagine two of your users modify the same JSON document, while both are offline. After they go online again, their clients replicate the modified document to the server. Now you have two conflicting versions of the same document, and you need a way to determine how the correct new version of that document should look like. This process is called conflict resolution.
The default in many offline first databases is a deterministic conflict resolution strategy. Both conflicting versions of the document are kept in the storage and when you query for the document, a winner is determined by comparing the hashes of the document and only the winning document is returned. Because the comparison is deterministic, all clients and servers will always pick the same winner. This kind of resolution only works when it is not that important that one of the document changes gets dropped. Because conflicts are rare, this might be a viable solution for some use cases.
A better resolution can be applied by listening to the changestream of the database. The changestream emits an event each time a write happens to the database. The event contains information about the written document and also a flag if there is a conflicting version. For each event with a conflict, you fetch all versions for that document and create a new document that contains the winning state. With that you can implement pretty complex conflict resolution strategies, but you have to manually code it for each collection of documents.
Instead of the solving conflict once at every client, it can be made a bit easier by solely relying on the backend. This can be done when all of your clients replicate with the same single backend server. With RxDB's Graphql Replication each client side change is sent to the server where conflicts can be resolved and the winning document can be sent back to the clients.
Sometimes there is no way to solve a conflict with code. If your users edit text based documents or images, often only the users themselves can decide how the winning revision has to look. For these cases, you have to implement complex UI parts where the users can inspect the conflict and manage its resolution.
You do not have to handle conflicts if they cannot happen in the first place. You can achieve that by designing a write only database where existing documents cannot be touched. Instead of storing the current state in a single document, you store all the events that lead to the current state. Sometimes called the "everything is a delta" strategy, others would call it Event Sourcing. Like an accountant that does not need an eraser, you append all changes and afterwards aggregate the current state at the client.
// create one new document for each change to the users balance
{id: new Date().toJSON(), change: 100} // balance increased by $100
{id: new Date().toJSON(), change: -50} // balance decreased by $50
{id: new Date().toJSON(), change: 200} // balance increased by $200- There is this thing called conflict-free replicated data type, short CRDT. Using a CRDT library like automerge will magically solve all of your conflict problems. Until you use it in production where you observe that implementing CRDTs has basically the same complexity as implementing conflict resolution strategies.
Realtime is a lie
So you replicate stuff between the clients and your backend. Each change on one side directly changes the state of the other sides in realtime. But this "realtime" is not the same as in realtime computing. In the offline first world, the word realtime was introduced by firebase and is more meant as a marketing slogan than a technical description.
There is an internet between your backend and your clients and everything you do on one machine takes at least once the latency until it can affect anything on the other machines. You have to keep this in mind when you develop anything where the timing is important, like a multiplayer game or a stock trading app.
Even when you run a query against the local database, there is no "real" realtime.
Client side databases run on JavaScript and JavaScript runs on a single CPU that might be partially blocked because the user is running some background processes. So you can never guarantee a response deadline which violates the time constraint of realtime computing.
Eventual consistency
An offline first app does not have a single source of truth. There is a source on the backend, one on the own client, and also each other client has its own definition of truth. At the moment your user starts the app, the local state is hopefully already replicated with the backend and all other clients. But this does not have to be true, the states can have converged and you have to plan for that.
The user could update a document based on wrong assumptions because it was not fully replicated at that point in time because the user is offline. A good way to handle this problem is to show the replication state in the UI and tell the user when the replication is running, stopped, paused or finished.
And some data is just too important to be "eventual consistent". Create a wire transfer in your online banking app while you are offline. You keep the smartphone laying at your night desk and when you use again in the next morning, it goes online and replicates the transaction. No thank you, do not use offline first for these kinds of things, or at least you have to display the replication state of each document in the UI.
Permissions and authentication
Every offline first app that goes beyond a prototype, does likely not have the same global state for all of its users. Each user has a different set of documents that are allowed to be replicated or seen by the user. So you need some kind of authentication and permission handling to divide the documents.
The easy way is to just create one database for each user on the backend and only allow to replicate that one. Creating that many databases is not really a problem with for example CouchDB, and it makes permission handling easy.
But as soon as you want to query all of your data in the backend, it will bite back. Your data is not at a single place, it is distributed between all of the user specific databases. This becomes even more complex as soon as you store information together with the documents that is not allowed to be seen by outsiders. You not only have to decide which documents to replicate, but also which fields of them.
So what you really want is a single datastore in the backend and then replicate only the allowed document parts to each of the users.
This always requires you to implement your custom replication endpoint like what you do with RxDBs GraphQL Replication.
You have to migrate the client database
While developing your app, sooner or later you want to change the data layout. You want to add some new fields to documents or change the format of them. So you have to update the database schema and also migrate the stored documents.
With 'normal' applications, this is already hard enough and often dangerous. You wait until midnight, stop the webserver, make a database backup, deploy the new schema and then you hope that nothing goes wrong while it updates that many documents.
With offline first applications, it is even more fun. You do not only have to migrate your local backend database, you also have to provide a migration strategy for all of these client databases out there. And you also cannot migrate everything at the same time. The clients can only migrate when the new code was updated from the appstore or the user visited your website again. This could be today or in a few weeks.
Performance is not native
When you create a web based offline first app, you cannot store data directly on the users filesystem. In fact there are many layers between your JavaScript code and the filesystem of the operation system. Let's say you insert a document in RxDB:
- You call the RxDB API to validate and store the data
- RxDB calls the underlying RxStorage, for example PouchDB.
- Pouchdb calls its underlying storage adapter
- The storage adapter calls IndexedDB
- The browser runs its internal handling of the IndexedDB API
- In most browsers IndexedDB is implemented on top of SQLite
- SQLite calls the OS to store the data in the filesystem
All these layers are abstractions. They are not build for exactly that one use case, so you lose some performance to tunnel the data through the layer itself, and you also lose some performance because the abstraction does not exactly provide the functions that are needed by the layer above and it will overfetch data.
You will not find a benchmark comparison between how many transactions per second you can run on the browser compared to a server based database. Because it makes no sense to compare them. Browsers are slower, JavaScript is slower.
Is it fast enough?
What you really care about is "Is it fast enough?". For most use cases, the answer is yes. Offline first apps are UI based and you do not need to process a million transactions per second, because your user will not click the save button that often. "Fast enough" means that the data is processed in under 16 milliseconds so that you can render the updated UI in the next frame. This is of course not true for all use cases, so you better think about the performance limit before starting with the implementation.
Nothing is predictable
You have a PostgreSQL database and run a query over 1000ths of rows, which takes 200 milliseconds. Works great, so you now want to do something similar at the client device in your offline first app. How long does it take? You cannot know because people have different devices, and even equal devices have different things running in the background that slow the CPUs. So you cannot predict performance and as described above, you cannot even predict the storage limit. So if your app does heavy data analytics, you might better run everything on the backend and just send the results to the client.
There is no relational data
I started creating RxDB many years ago and while still maintaining it, I often worked with all these other offline first databases out there. RxDB and all of these other ones, are based on some kind of document databases similar to NoSQL. Often people want to have a relational database like the SQL one they use at the backend.
So why are there no real relations in offline first databases? I could answer with these arguments like how JavaScript works better with document based data, how performance is better when having no joins or even how NoSQL queries are more composable. But the truth is, everything is NoSQL because it makes replication easy. An SQL query that mutates data in different tables based on some selects and joins, cannot be partially replicated without breaking the client. You have foreign keys that point to other rows and if these rows are not replicated yet, you have a problem. To implement a robust Sync Engine for relational data, you need some stuff like a reliable atomic clock and you have to block queries over multiple tables while a transaction replicated. Watch this guy implementing offline first replication on top of SQLite or read this discussion about implementing offline first in supabase.
So creating replication for an SQL offline first database is way more work than just adding some network protocols on top of PostgreSQL. It might not even be possible for clients that have no reliable clock.
Docs Src Docs Electron Database (`docs-src/docs/electron-database.md`)
title: Electron Database - Storage adapters for SQLite, Filesystem and In-Memory slug: electron-database.html image: /headers/electron-database.jpg
import {CenteredImage} from '@site/src/components/centered-image';
Electron Database - RxDB with different storage for SQLite, Filesystem and In-Memory
Electron (aka Electron.js) is a framework developed by github that is designed to create desktop applications with the Web technology stack consisting of HTML, CSS and JavaScript.
Because the desktop application runs on the client's device, it is suitable to use a database that can store and query data locally. This allows you to create so-called local first apps that store data locally and even work when the user has no internet connection.
While there are many options to store data in Electron, for complex realtime apps using RxDB is recommended because it is a database made for UI-based client-side application, not a server-side database.
Databases for Electron
An Electron runtime can be divided into two parts:
- The "main" process which is a Node.js JavaScript process that runs without a UI in the background.
- One or multiple "renderer" processes that consist of a Chrome browser engine and runs the user interface. Each renderer process represents one "browser tab".
This is important to understand because choosing the right database depends on your use case and on which of these JavaScript runtimes you want to keep the data.
Server Side Databases in Electron.js
Because Electron runs on a desktop computer, you might think that it should be possible to use a common "server" database like MySQL, PostgreSQL or MongoDB. In theory, you could ship the correct database server binaries with your electron application and start a process on the client's device that exposes a port to the database that can be consumed by Electron. In practice, this is not a viable way to go because shipping the correct binaries and opening ports is way to complicated and troublesome. Instead you should use a database that can be bundled and run inside of Electron, either in the main or in the renderer process.
Localstorage / IndexedDB / WebSQL as alternatives to SQLite in Electron
Because Electron uses a common Chrome web browser in the renderer process, you can access the common Web Storage APIs like Localstorage, IndexedDB and WebSQL. This is easy to set up and storing small sets of data can be achieved in a short span of time.
But as soon as your application goes beyond a simple todo-app, there are multiple obstacles that come in your way. One thing is the bad multi-tab support. If you have more than one renderer process, it becomes hard to manage database writes between them. Each browser tab could modify the database state while the others do not know of the changes and keep an outdated UI.
Another thing is performance. IndexedDB is slow, mostly because it has to go through layers of browser security and abstractions. Storing and querying a lot of data might become your performance bottleneck. Localstorage and WebSQL are even slower, by the way. Using these Web Storage APIs is generally only recommended when you know for sure that there will be always only one rendering process and performance is not that relevant. The main reason for that is the security- and abstraction layers that write- and read operations have to go through when using the browsers IndexedDB API. So instead of using IndexedDB in Electron in the renderer process, you should use something that runs in the "main" process in Node.js like the Filesystem RxStorage or the In Memory RxStorage.
RxDB
RxDB is a NoSQL database for JavaScript applications. It has many features that come in handy when RxDB is used with UI based applications like your Electron app. For example, it is able to subscribe to query results of single fields of documents. It has encryption and compression features and most important it has a battle tested Sync Engine that can be used to do a realtime sync with your backend.
Because of the flexible storage layer of RxDB, there are many options on how to use it with Electron:
- The memory RxStorage that stores the data inside of the JavaScript memory without persistence
- The SQLite RxStorage
- The IndexedDB RxStorage
- The LocalStorage RxStorage
- The Dexie.js RxStorage
- The Node.js Filesystem
It is recommended to use the SQLite RxStorage because it has the best performance and is the easiest to set up. However it is part of the ๐ Premium Plugins which must be purchased, so to try out RxDB with Electron, you might want to use one of the other options. To start with RxDB, I would recommend using the LocalStorage RxStorage in the renderer processes. Because RxDB is able to broadcast the database state between browser tabs, having multiple renderer processes is not a problem like it would be when you use plain IndexedDB without RxDB.
In production, you would always run the RxStorage in the main process with the RxStorage Electron IpcRenderer & IpcMain plugins.
First, you have to install all dependencies via npm install rxdb rxjs.
Then you can assemble the RxStorage and create a database with it:
import { createRxDatabase } from 'rxdb';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// create database
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageLocalstorage()
});
// create collections
const collections = await myRxDatabase.addCollections({
humans: {
/* ... */
}
});
// insert document
await collections.humans.insert({id: 'foo', name: 'bar'});
// run a query
const result = await collections.humans.find({
selector: {
name: 'bar'
}
}).exec();
// observe a query
await collections.humans.find({
selector: {
name: 'bar'
}
}).$.subscribe(result => {/* ... */});For better performance in the renderer tab, you can later switch to the IndexedDB RxStorage. But in production, it is recommended to use the SQLite RxStorage or the Filesystem RxStorage in the main process so that database operations do not block the rendering of the UI.
To learn more about using RxDB with Electron, you might want to check out this example project.
SQLite in Electron.js without RxDB
SQLite is a SQL based relational database written in the C programming language that was crafted to be embedded inside of applications and stores data locally. Operations are written in the SQL query language similar to the PostgreSQL syntax.
Using SQLite in Electron is not possible in the renderer process, only in the main process. To communicate data operations between your main and your renderer processes, you have to use either @electron/remote (not recommended) or the ipcRenderer (recommended). So you start up SQLite in your main process and whenever you want to read or write data, you send the SQL queries to the main process and retrieve the result back as JSON data.
To install SQLite, use the SQLite3 package which is a native Node.js module. You also need the @electron/rebuild package to rebuild the SQLite module against the currently installed Electron version.
Install them with npm install sqlite3 @electron/rebuild.
Then you can rebuild SQLite with ./node_modules/.bin/electron-rebuild -f -w sqlite3
In the JavaScript code of your main process you can now create a database:
const sqlite3 = require('sqlite3');
const db = new sqlite3.Database('/path/to/database/file.db');
// create a table and insert a row
db.serialize(() => {
db.run("CREATE TABLE Users (name, lastName)");
db.run("INSERT INTO Users VALUES (?, ?)", ['foo', 'bar']);
});Also you have to set up the ipcRenderer so that message from the renderer process are handled:
ipcMain.handle('db-query', async (event, sqlQuery) => {
return new Promise(res => {
db.all(sqlQuery, (err, rows) => {
res(rows);
});
});
});In your renderer process, you can now call the ipcHandler and fetch data from SQLite:
const rows = await ipcRenderer.invoke('db-query', "SELECT * FROM Users");The downside of SQLite (or SQL in general) is that it is lacking many features that are handful when using a database together with UI based applications. It is not possible to observe queries or document fields and there is no replication method to sync data with a server. This makes SQLite a good solution when you just want to store data on the client or process expensive SQL queries on the server, but it is not suitable for more complex operations like two-way replication, encryption, compression and so on. Also developer helpers like TypeScript type safety are totally out of reach.
Follow up
- Learn how to use RxDB as database in electron with the Quickstart Tutorial.
- Check out the RxDB Electron example
- There is a followup list of other client side database alternatives that you can try to use with Electron.
Docs Src Docs Electron (`docs-src/docs/electron.md`)
title: Seamless Electron Storage with RxDB slug: electron.html image: /headers/electron.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
Electron Plugin
RxStorage Electron IpcRenderer & IpcMain
To use RxDB in electron, it is recommended to run the RxStorage in the main process and the RxDatabase in the renderer processes. With the rxdb electron plugin you can create a remote RxStorage and consume it from the renderer process.
To do this in a convenient way, the RxDB electron plugin provides the helper functions exposeIpcMainRxStorage and getRxStorageIpcRenderer.
Similar to the Worker RxStorage, these wrap any other RxStorage once in the main process and once in each renderer process. In the renderer you can then use the storage to create a RxDatabase which communicates with the storage of the main process to store and query data.
:::notenodeIntegration must be enabled in Electron.
:::
// main.js
const { exposeIpcMainRxStorage } = require('rxdb/plugins/electron');
const { getRxStorageMemory } = require('rxdb/plugins/storage-memory');
app.on('ready', async function () {
exposeIpcMainRxStorage({
key: 'main-storage',
storage: getRxStorageMemory(),
ipcMain: electron.ipcMain
});
});// renderer.js
const { getRxStorageIpcRenderer } = require('rxdb/plugins/electron');
const { getRxStorageMemory } = require('rxdb/plugins/storage-memory');
const db = await createRxDatabase({
name,
storage: getRxStorageIpcRenderer({
key: 'main-storage',
ipcRenderer: electron.ipcRenderer
})
});
/* ... */FAQ
You securely create an offline-first Electron application by maintaining strict process isolation: the primary database connection runs securely within the hidden Node.js main process, while the vulnerable DOM execution runs in the heavily-restricted renderer process. RxDB automates this architecture precisely via its dedicated Electron IPC plugin (exposeIpcMainRxStorage), enabling seamless, non-blocking data synchronization across the IPC boundary while mitigating direct local filesystem exposure to malicious client payloads.
Related
Docs Src Docs Encryption (`docs-src/docs/encryption.md`)
title: Encryption slug: encryption.html image: /headers/encryption.jpg
import {Steps} from '@site/src/components/steps';
import {PremiumBlock} from '@site/src/components/premium-block';
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_ENCRYPTION, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {IconEncryption} from '@site/src/components/icons/encryption';
import {Faq, FaqItem} from '@site/src/components/faq';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {EncryptionLogo} from '@site/src/components/animations/encryption-logo';
Encrypted Local Storage with RxDB
The RxDB encryption plugin empowers developers to fortify their applications' data security. It seamlessly integrates with RxDB, allowing for the secure storage and retrieval of documents by encrypting them with a password. With encryption and decryption processes handled internally, it ensures that sensitive data remains confidential, making it a valuable tool for building robust, privacy-conscious applications. The encryption works on all RxDB supported devices types like the browser, ReactNative or Node.js.
Encrypting client-side stored data in RxDB offers numerous advantages:
- Enhanced Security: In the unfortunate event of a user's device being stolen, the encrypted data remains safeguarded on the hard drive, inaccessible without the correct password.
- Access Control: You can retain control over stored data by revoking access at any time simply by withholding the password.
- Tamper proof Other applications on the device cannot read out the stored data when the password is only kept in the process-specific memory
Querying encrypted data
RxDB handles the encryption and decryption of data internally. This means that when you work with a RxDocument, you can access the properties of the document just like you would with normal, unencrypted data. RxDB automatically decrypts the data for you when you retrieve it, making it transparent to your application code.
This means the encryption works with all RxStorage like SQLite, IndexedDB, OPFS and so on.
Keep in mind that everything which reads the documents through the collection sees the decrypted values. A JSON dump therefore contains your encrypted fields in plain text, so treat such a dump as sensitive data.
However, there's a limitation when it comes to querying encrypted fields. Encrypted fields cannot be used as operators in queries. This means you cannot perform queries like "find all documents where the encrypted field equals a certain value." RxDB does not expose the encrypted data in a way that allows direct querying based on the encrypted content. To filter or search for documents based on the contents of encrypted fields, you would need to first decrypt the data and then perform the query, which might not be efficient or practical in some cases.
You could however use the memory mapped RxStorage to replicate the encrypted documents into a non-encrypted in-memory storage and then query them like normal.
Password handling
RxDB does not define how you should store or retrieve the encryption password. It only requires you to provide the password on database creation which grants you flexibility in how you manage encryption passwords.
You could ask the user on app-start to insert the password, or you can retrieve the password from your backend on app start (or revoke access by no longer providing the password).
The password is also required when you delete an encrypted database with removeRxDatabase(), see removing an encrypted database.
Asymmetric encryption
The encryption plugin itself uses symmetric encryption with a password to guarantee best performance when reading and storing data.
It is not able to do Asymmetric encryption by itself. If you need Asymmetric encryption with a private/publicKey, it is recommended to encrypted the password itself with the asymmetric keys and store the encrypted password beside the other data. On app-start you can decrypt the password with the private key and use the decrypted password in the RxDB encryption plugin
Using the RxDB Encryption Plugins
RxDB currently has two plugins for encryption:
- The free
encryption-crypto-jsplugin that is based on theAESalgorithm of the crypto-js library encryption-web-cryptoplugin that is based on the native Web Crypto API which makes it faster and more secure to use. Document inserts are about 10x faster compared tocrypto-jsand it has a smaller build size because it uses the browsers API instead of bundling an npm module.
An RxDB encryption plugin is a wrapper around any other RxStorage.
Wrap your RxStorage with the encryption
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
// wrap the normal storage with the encryption plugin
const encryptedStorage = wrappedKeyEncryptionCryptoJsStorage({
storage: getRxStorageLocalstorage()
});Create a RxDatabase with the wrapped storage
Also you have to set a password when creating the database. The format of the password depends on which encryption plugin is used.
import { createRxDatabase } from 'rxdb/plugins/core';
// create an encrypted database
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: encryptedStorage,
password: 'sudoLetMeIn'
});Create an RxCollection with an encrypted property
To define a field as being encrypted, you have to add it to the encrypted fields list in the schema.
const todoSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
note: {
type: 'string'
}
},
required: ['id', 'name'],
encrypted: ['note']
};
await myDatabase.addCollections({
todos: {
schema: todoSchema
}
});
await myDatabase.todos.insert({
id: 'todo2',
name: 'Buy groceries',
note: 'Milk, eggs, bread' // <- stored encrypted
});
const doc = await myDatabase.todos.findOne('todo2').exec();
console.log(doc.note);
// > 'Milk, eggs, bread'Using the WebCrypto API
import {
wrappedKeyEncryptionWebCryptoStorage,
createPassword
} from 'rxdb-premium/plugins/encryption-web-crypto';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
// wrap the normal storage with the encryption plugin
const encryptedIndexedDbStorage = wrappedKeyEncryptionWebCryptoStorage({
storage: getRxStorageIndexedDB()
});
const myPasswordObject = {
// Algorithm can be oneOf: 'AES-CTR' | 'AES-CBC' | 'AES-GCM'
algorithm: 'AES-CTR',
password: 'myRandomPasswordWithMin8Length'
};
// create an encrypted database
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: encryptedIndexedDbStorage,
password: myPasswordObject
});
/* ... */Changing the password
The password is set database specific and it is not possible to change the password of a database. Opening an existing database with a different password will throw an error. To change the password you can either:
- Use the storage migration plugin to migrate the database state into a new database.
- Store a randomly created meta-password in a different RxDatabase as a value of a local document. Encrypt the meta password with the actual user password and read it out before creating the actual database.
Encrypted attachments
To store the attachments data encrypted, you have to set encrypted: true in the attachments property of the schema.
const todoSchema = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
/* ... */
},
attachments: {
// if true, the attachment-data will be
// encrypted with the db-password
encrypted: true
}
};Encryption and workers
If you are using Worker RxStorage or SharedWorker RxStorage with encryption, it's recommended to run encryption inside of the worker. Encryption can be very cpu intensive and would take away CPU-power from the main thread which is the main reason to use workers.
You do not need to worry about setting the password inside of the worker. The password will be set when calling createRxDatabase from the main thread, and will be passed internally to the storage in the worker automatically.
Using encryption inside the worker with OPFS
When you wrap a storage like OPFS with encryption inside of a worker, you have to set the usesRxDatabaseInWorker option on the OPFS storage. Without this option, the OPFS storage returns raw JSON strings instead of parsed objects as a performance optimization. The encryption wrapper cannot process these strings and will throw an error.
// inside of the worker.js file
import { getRxStorageOPFS } from 'rxdb-premium/plugins/storage-opfs';
import {
wrappedKeyEncryptionWebCryptoStorage
} from 'rxdb-premium/plugins/encryption-web-crypto';
const storage = wrappedKeyEncryptionWebCryptoStorage({
storage: getRxStorageOPFS({
// Required when wrapping OPFS with encryption inside a worker
usesRxDatabaseInWorker: true
})
});Encryption Performance
As shown in the chart, the WebCrypto based encryption plugins are generally 5 times faster than the crypto-js plugin.
FAQ
RxDB provides robust plugins for client side field encryption directly within your javascript database. You encrypt sensitive document properties transparently before they save to local storage. The encryption-crypto-js plugin utilizes AES algorithms for dependable security. The encryption-web-crypto plugin employs native browser APIs to achieve superior performance. You maintain data confidentiality across Web, React Native, and Node.js environments.
You can implement encryption in JavaScript by manually encrypting fields with the native WebCrypto API before storing them, but this breaks standard querying. Advanced databases like RxDB simplify this through schema-level encryption plugins (encryption-web-crypto). By flagging specific document fields as encrypted: true in your JSON Schema, RxDB automatically encrypts the data before writing to the storage engine (like IndexedDB or SQLite) and decrypts it instantly upon retrieval.
No, chrome.storage.local (and standard IndexedDB in the browser) is not encrypted at rest by default. Any user or potentially malicious extension with adequate local machine access can read the underlying data files. To properly secure sensitive data at rest in a browser extension or Web App, you must explicitly encrypt strings before saving them, a process seamlessly automated by using an encrypted RxStorage wrapper. See IndexedDB Encryption for the details.
Yes, libraries like crypto-js or wrappers over the native WebCrypto API provide robust open-source encryption. For developers building native mobile apps (React Native, Expo, Ionic) or browser applications, utilizing a database that ships with native encryption wrappers like RxDB's Encryption Plugins is the most reliable method. It ensures data is never written to disk in plain text while allowing you to effortlessly swap underlying storage layers without rewriting your cryptography logic.
No. When you encrypt a parent field, the entire object at that path is encrypted as a single string. You cannot also encrypt a child path of an already-encrypted parent. For example, if you encrypt assignee, you must not also add assignee.email to the encrypted array. Doing so will throw an error in dev-mode.
// NOT ALLOWED - 'assignee.email' is a child of 'assignee'
const todoSchema = {
encrypted: ['assignee', 'assignee.email']
};
// CORRECT - only encrypt the parent
const todoSchema = {
encrypted: ['assignee']
};Docs Src Docs Errors (`docs-src/docs/errors.md`)
title: Error Messages slug: errors.html image: /headers/errors.jpg
RxDB Error Messages
When RxDB has an error, an RxError object is thrown instead of a normal JavaScript Error. This RxError contains additional properties such as a code field and parameters. By default the full human readable error messages are not included into the RxDB build. This is because error messages have a high entropy and cannot be compressed well. Therefore only an error message with the correct error-code and parameters is thrown but without the full text.
When you enable the DevMode Plugin the full error messages are added to the RxError. This should only be done in development, not in production builds to keep a small build size.
All RxDB error messages
import { ErrorMessages } from '@site/src/components/error-messages';
Docs Src Docs Fulltext Search (`docs-src/docs/fulltext-search.md`)
title: Fulltext Search slug: fulltext-search.html image: /headers/fulltext-search.jpg
import {PremiumBlock} from '@site/src/components/premium-block';
import {Steps} from '@site/src/components/steps';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {FulltextSearchLens} from '@site/src/components/animations/fulltext-search-lens';
Fulltext Search
To run fulltext search queries on the local data, RxDB has a fulltext search plugin based on flexsearch and RxPipeline. On each write to a given source RxCollection, an indexer is running to map the written document data into a fulltext search index.
The index can then be queried efficiently with complex fulltext search operations.
Benefits of using a local fulltext search
- Efficient Search and Indexing
The plugin utilizes the FlexSearch library, known for its speed and memory efficiency. This ensures that search operations are performed quickly, even with large datasets. The search engine can handle multi-field queries, partial matching, and complex search operations, providing users with highly relevant results.
- Local Data Indexing
With the plugin, all search operations are performed on the local data stored within the RxDB collections. This means that users can execute fulltext search queries without the need for an external server or database, which is especially beneficial for offline-first applications. The local indexing ensures that search queries are executed quickly, reducing the latency typically associated with remote database queries. Also when used in multiple browser tabs, it is ensured that through Leader Election, only exactly one tabs is doing the work of indexing without having an overhead in the other browser tabs.
- Real-time Indexing
The plugin integrates seamlessly with RxDB's reactive nature. Every time a document is written to an RxCollection, an indexer updates the fulltext search index in real-time. This ensures that search results are always up-to-date, reflecting the most current state of the data without requiring manual reindexing.
- Persistent indexing
The fulltext search index is efficiently persisted within the RxCollection, ensuring that the index remains intact across app restarts. When documents are added or updated in the collection, the index is incrementally updated in real-time, meaning only the changes are processed rather than reindexing the entire dataset. This incremental approach not only optimizes performance but also ensures that subsequent app launches are quick, as there's no need to reindex all the data from scratch, making the search feature both reliable and fast from the moment the app starts. When using an encrypted storage the index itself and incremental updates to it are stored fully encrypted and are only decrypted in-memory.
- Complex Query Support
The FlexSearch-based plugin allows for sophisticated search queries, including multi-term and contextual searches. Users can perform complex searches that go beyond simple keyword matching, enabling more advanced use cases like searching for documents with specific phrases, relevance-based sorting, or even phonetic matching.
- Offline-First Support and Privacy
As RxDB is designed with offline-first applications in mind, the fulltext search plugin supports this paradigm by ensuring that all search operations can be performed offline. This is crucial for applications that need to function in environments with intermittent or no internet connectivity, offering users a consistent and reliable search experience with zero latency.
Using the RxDB Fulltext Search
Step 1: Add the `RxDBFlexSearchPlugin` to RxDB.
import { RxDBFlexSearchPlugin } from 'rxdb-premium/plugins/flexsearch';
import { addRxPlugin } from 'rxdb/plugins/core';
addRxPlugin(RxDBFlexSearchPlugin);Step 2: Create a `RxFulltextSearch` instance on top of a collection with the `addFulltextSearch()` function.
import { addFulltextSearch } from 'rxdb-premium/plugins/flexsearch';
const flexSearch = await addFulltextSearch({
// unique identifier. Used to store metadata
// and continue indexing on restarts/reloads.
identifier: 'my-search',
// The source collection on whose documents the search is based on
collection: myRxCollection,
/**
* Transforms the document data to a given searchable string.
* This can be done by returning a single string property of the document
* or even by concatenating and transforming multiple fields like:
* doc => doc.firstName + ' ' + doc.lastName
*/
docToString: doc => doc.firstName,
/**
* (Optional)
* Amount of documents to index at once.
* See https://rxdb.info/rx-pipeline.html
*/
batchSize: number;
/**
* (Optional)
* lazy: Initialize the in memory fulltext index at the first search query.
* instant: Directly initialize so that the
* index is already there on the first query.
* Default: 'instant'
*/
initialization: 'instant',
/**
* (Optional)
* @link https://github.com/nextapps-de/flexsearch#index-options
*/
indexOptions: {},
});Step 3: Run a search operation:
// find all documents whose searchstring contains "foobar"
const foundDocuments = await flexSearch.find('foobar');
/**
* You can also use search options as second parameter
* @link https://github.com/nextapps-de/flexsearch#search-options
*/
const foundDocuments = await flexSearch.find('foobar', { limit: 10 });Docs Src Docs Install (`docs-src/docs/install.md`)
title: Installation slug: install.html image: /headers/install.jpg
import {InstallTabs} from '@site/src/components/install-tabs';
import {Tabs} from '@site/src/components/tabs';
Install RxDB
npm
To install the latest release of rxdb and its dependencies and save it to your package.json, run:
peer-dependency
You also need to install the peer-dependency rxjs if you have not installed it before.
Use the ES2015 build of RxJS
RxJS 7 ships an ES5 build and an ES2015 build. The ES2015 build is only resolved when your bundler sets the es2015 export condition. Vite, webpack, Rollup and esbuild do not set it by default, so they bundle the ES5 build together with the tslib helpers. In an RxDB app built with Vite, adding the condition reduces the bundle by about 6.6 KB minified (1.6 KB gzipped). The Angular CLI already sets the condition, so Angular projects need no change.
Vite
Vite 6 or newer:
// vite.config.ts
import { defineConfig, defaultClientConditions } from 'vite';
export default defineConfig({
resolve: {
conditions: ['es2015', ...defaultClientConditions]
}
});webpack
webpack 5 or newer, '...' keeps the default conditions:
// webpack.config.js
module.exports = {
resolve: {
conditionNames: ['es2015', '...']
}
};Rollup
With @rollup/plugin-node-resolve:
// rollup.config.mjs
import { nodeResolve } from '@rollup/plugin-node-resolve';
export default {
plugins: [
nodeResolve({ browser: true, exportConditions: ['es2015'] })
]
};esbuild
Setting conditions replaces the default module condition, so add it again:
await esbuild.build({
conditions: ['es2015', 'module']
});polyfills
RxDB is coded with ES8 and transpiled to ES5. This means you have to install polyfills to support older browsers. For example you can use core-js with:
npm i core-js --saveIf you need polyfills, you have to import them in your code.
import 'core-js/stable';Polyfill the `global` variable
When you use RxDB with Angular or other Webpack based frameworks, you might get the error Uncaught ReferenceError: global is not defined.
This is because some dependencies of RxDB assume a Node.js-specific global variable that is not added to browser runtimes by some bundlers.
You have to add them manually, like we do here.
(window as any).global = window;
(window as any).process = {
env: { DEBUG: undefined },
};Project Setup and Configuration
In the examples folder you can find CI tested projects for different frameworks and use cases, while in the /config folder base configuration files for Webpack, Rollup, Mocha, Karma, TypeScript are exposed.
Consult package.json for the versions of the packages supported.
Installing the latest RxDB build
If you need the latest development state of RxDB, add it as git dependency into your package.json.
"dependencies": {
"rxdb": "git+https://[email protected]/pubkey/rxdb.git#commitHash"
}Replace commitHash with the hash of the latest build-commit.
Import
To import rxdb, add this to your JavaScript file to import the default bundle that contains the RxDB core:
import {
createRxDatabase, // ./rx-database.md
/* ... */
} from 'rxdb';Import Reference
RxDB has three kinds of import paths:
rxdb(orrxdb/plugins/core): the core functions and all TypeScript types.rxdb/plugins/<name>: the open-source plugins.rxdb-premium/plugins/<name>: the premium plugins ๐.
Core Functions
import {
createRxDatabase,
addRxPlugin,
removeRxDatabase,
isRxDatabase,
isRxCollection,
isRxDocument,
isRxQuery,
toTypedRxJsonSchema
} from 'rxdb';TypeScript Types
All types are exported from rxdb. Use import type so that they do not end up in your bundle.
import type {
RxDatabase,
RxCollection,
RxDocument,
RxQuery,
RxJsonSchema,
RxStorage,
RxConflictHandler,
ExtractDocumentTypeFromTypedRxJsonSchema,
ReplicationPullHandler,
ReplicationPushHandler
} from 'rxdb';How to combine these types is shown in the TypeScript tutorial.
Storages
| Storage | Import |
|---|---|
| Memory | import { getRxStorageMemory } from 'rxdb/plugins/storage-memory'; |
| LocalStorage | import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage'; |
| Dexie.js | import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie'; |
| SQLite Trial | import { getRxStorageSQLiteTrial } from 'rxdb/plugins/storage-sqlite'; |
| MongoDB | import { getRxStorageMongoDB } from 'rxdb/plugins/storage-mongodb'; |
| DenoKV | import { getRxStorageDenoKV } from 'rxdb/plugins/storage-denokv'; |
| FoundationDB | import { getRxStorageFoundationDB } from 'rxdb/plugins/storage-foundationdb'; |
| Remote | import { getRxStorageRemote } from 'rxdb/plugins/storage-remote'; |
| Electron IpcRenderer | import { getRxStorageIpcRenderer } from 'rxdb/plugins/electron'; |
| IndexedDB ๐ | import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb'; |
| OPFS ๐ | import { getRxStorageOPFS, getRxStorageOPFSMainThread } from 'rxdb-premium/plugins/storage-opfs'; |
| SQLite ๐ | import { getRxStorageSQLite } from 'rxdb-premium/plugins/storage-sqlite'; |
| Expo Filesystem ๐ | import { getRxStorageExpoAsync } from 'rxdb-premium/plugins/storage-filesystem-expo'; |
| Node.js Filesystem ๐ | import { getRxStorageFilesystemNode } from 'rxdb-premium/plugins/storage-filesystem-node'; |
| Worker ๐ | import { getRxStorageWorker } from 'rxdb-premium/plugins/storage-worker'; |
| SharedWorker ๐ | import { getRxStorageSharedWorker } from 'rxdb-premium/plugins/storage-worker'; |
| Sharding ๐ | import { getRxStorageSharding } from 'rxdb-premium/plugins/storage-sharding'; |
For unit tests, use the Memory RxStorage. It needs no setup and all data is gone when the process ends. When your tests need a fresh database per test, give each database a random name with randomToken() from rxdb. More testing patterns are on the testing page.
Common Plugins
import { RxDBDevModePlugin } from 'rxdb/plugins/dev-mode';
import { wrappedValidateAjvStorage } from 'rxdb/plugins/validate-ajv';
import { RxDBMigrationSchemaPlugin } from 'rxdb/plugins/migration-schema';
import { RxDBLocalDocumentsPlugin } from 'rxdb/plugins/local-documents';
import { RxDBLeaderElectionPlugin } from 'rxdb/plugins/leader-election';
import { RxDBAttachmentsPlugin } from 'rxdb/plugins/attachments';
import { RxDBQueryBuilderPlugin } from 'rxdb/plugins/query-builder';
import { RxDBUpdatePlugin } from 'rxdb/plugins/update';
import { RxDBCleanupPlugin } from 'rxdb/plugins/cleanup';
import { wrappedKeyCompressionStorage } from 'rxdb/plugins/key-compression';
import {
wrappedKeyEncryptionCryptoJsStorage
} from 'rxdb/plugins/encryption-crypto-js';
import { replicateRxCollection, RxReplicationState } from 'rxdb/plugins/replication';Docs Src Docs Json Import Export (`docs-src/docs/json-import-export.md`)
title: JSON Import and Export slug: json-import-export.html image: /headers/json-import-export.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
๐ค JSON Import and Export
With the json-dump plugin you can export the data of a RxDatabase or a single RxCollection into a plain JSON object and import that object back later.
The dump is a normal JavaScript object, so you can store it in a file, send it to a server, or keep it in localStorage. It contains the documents in their decrypted form and, when you ask for it, the attachments data as base64 strings.
This is useful to:
- Seed a database with a fixed set of documents in tests or demos.
- Move data between two databases, for example when a user switches devices and you have no replication backend.
- Let users download their own data and load it back into the app.
The plugin runs in every runtime that RxDB supports. For server side backups that stream to disk instead of building one big object in memory, use the backup plugin.
Installation
import { addRxPlugin } from 'rxdb';
import { RxDBJsonDumpPlugin } from 'rxdb/plugins/json-dump';
addRxPlugin(RxDBJsonDumpPlugin);Export a collection
exportJSON() creates a dump of every document in the collection.
const json = await myCollection.exportJSON();
console.dir(json);
/* > {
* name: 'heroes',
* schemaHash: '22b549e064b35a21494eadc441491eeb644bee28aadb0feb9303cb37c0d51b86',
* docs: [
* {
* id: 'alice',
* name: 'Alice',
* color: 'blue',
* _deleted: false,
* _meta: { lwt: 1787225698353.01 }
* }
* ]
* }
*/The schemaHash is written into the dump so that importJSON() can refuse a dump that was created from a different schema. The revision _rev is not part of the dump, because the importing database creates its own revisions.
Import into a collection
importJSON() writes the documents of a dump into the collection.
await myCollection.importJSON(json);The collection has to exist and its schema has to be the same one the dump was created from. When the hashes do not match, RxDB throws the error JD2.
Notice that importing fires a change event for each inserted document, so open queries and their subscriptions update on their own.
Export a database
On the database level, exportJSON() runs the export for every collection and puts the results into one object.
const json = await myDatabase.exportJSON();
console.dir(json);
/* > {
* name: 'heroesdb',
* instanceToken: 'ruxpmzxulz',
* collections: [
* {
* name: 'heroes',
* schemaHash: '22b549e064b35a...',
* docs: []
* }
* ]
* }
*/To export only some of the collections, pass their names as the first argument.
const json = await myDatabase.exportJSON(['heroes', 'villains']);Collections whose name starts with an underscore are internal and are always left out of the database export.
Import into a database
await myDatabase.importJSON(json);The collections must be created before the import, because the dump does not carry the collection settings like the schema, the migration strategies or the conflict handler. When the dump contains a collection that does not exist in the database, RxDB throws the error JD1 and nothing is written.
Attachments
By default the attachments of the documents are not part of the dump, because their binary data can be way bigger than the documents itself. Set the attachments option to true to include them. The data is then stored as a base64 string so that the dump stays JSON friendly.
const json = await myCollection.exportJSON({ attachments: true });
console.dir(json.docs[0]._attachments);
/* > {
* 'cat.txt': {
* type: 'text/plain',
* length: 4,
* data: 'bWVvdw=='
* }
* }
*/On the database level you can pass the options either as the first argument or after the list of collection names.
// export everything, including the attachments
const json = await myDatabase.exportJSON({ attachments: true });
// export only some collections, including the attachments
const partialJson = await myDatabase.exportJSON(['heroes'], { attachments: true });importJSON() turns the base64 strings back into Blobs and writes them together with the documents, so no extra option is needed on the import side. The attachment digest is calculated again with the hash function of the importing database, which means a dump can be imported into a database that uses a different hashFunction.
Base64 needs about 33% more space than the raw bytes. When your documents carry big images or videos, the dump grows accordingly and you might want to keep attachments: false and move the files with something else.
Limitations
Keep these in mind before you use a dump as your backup strategy:
- Encrypted fields are exported in plain text. The export reads the documents through the collection, so the encryption is already undone at that point. Treat the dump as sensitive data and never write it to a place where the plain values must not appear.
- Deleted documents are not part of the dump. The export runs a normal query, and queries skip deleted documents. Because of that, a dump cannot be used to replicate deletions to another database.
- Revisions are not part of the dump. On import, every document is written as a fresh insert with a new revision and a new
_meta.lwt. A dump is a snapshot of the current state, not a history. - The whole dump is built in memory. For a few thousand documents this is fine. For a large dataset, use the backup plugin or one of the replication plugins instead.
- The schema hash must match. A dump cannot be imported into a collection with a changed schema. To move data across a schema change, import it into a collection with the old schema version and let the schema migration run.
FAQ
Call exportJSON() on the RxDatabase and write the result with JSON.stringify(). The dump is a plain object without class instances, so it survives the round trip through a string. In Node.js you can write it with fs.writeFileSync(), in the browser you can offer it to the user as a download.
Only when you ask for it. The attachments option defaults to false so that dumps stay small. Call exportJSON({ attachments: true }) to get the attachments data as base64 strings, which importJSON() writes back together with the documents.
The dump stores the hash of the schema it was created from, and the import compares it with the hash of the target collection. Any change to the schema changes the hash, even one that looks harmless like a new optional field. Create the target collection with the exact schema the dump came from, then run a schema migration to reach the new version.
Use the json-dump plugin when you need one JSON object that you can move around, for example to seed a database or to hand a user their data. Use the backup plugin when you run on the server and want the state written to disk continuously. The backup plugin writes each document as its own file and continues from the last checkpoint, so it also works for datasets that do not fit into memory.
No. A dump has no revisions and no deletions, so importing it a second time overwrites the target state instead of merging it. For two databases that both accept writes, use the RxDB replication, which handles conflicts, deletions, and checkpoints.
Follow Up
- Learn how to back up a database to the filesystem on the server side.
- Learn how to replicate your data with a backend instead of moving dumps around.
- Read the attachments docs to learn how binary data is stored.
- Start with the RxDB Quickstart.
Docs Src Docs Key Compression (`docs-src/docs/key-compression.md`)
title: Key Compression slug: key-compression.html image: /headers/key-compression.jpg
import {Steps} from '@site/src/components/steps';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {KeyCompressionLogo} from '@site/src/components/animations/key-compression-logo';
Key Compression
With the key compression plugin, documents will be stored in a compressed format which saves up to 40% disk space.
For compression the npm module jsonschema-key-compression is used.
It compresses json-data based on its json-schema while still having valid json. It works by compressing long attribute-names into smaller ones and backwards.
The compression and decompression happens internally, so when you work with a RxDocument, you can access any property like normal.
Enable key compression
The key compression plugin is a wrapper around any other RxStorage.
Wrap your RxStorage with the key compression plugin
import { wrappedKeyCompressionStorage } from 'rxdb/plugins/key-compression';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const storageWithKeyCompression = wrappedKeyCompressionStorage({
storage: getRxStorageLocalstorage()
});Create an RxDatabase
import { createRxDatabase } from 'rxdb/plugins/core';
const myDatabase = await createRxDatabase({
name: 'mydatabase',
storage: storageWithKeyCompression
});Create a compressed RxCollection
const todoSchema = {
keyCompression: true, // set this to true, to enable the keyCompression
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100 // <- the primary key must have set maxLength
},
name: {
type: 'string'
},
done: {
type: 'boolean',
default: false
}
/* ... */
},
required: ['id', 'name', 'done']
};
await myDatabase.addCollections({
todos: {
schema: todoSchema
}
});
await myDatabase.todos.insert({ id: 'todo1', name: 'Learn RxDB' });
const doc = await myDatabase.todos.findOne('todo1').exec();
// the document data is decompressed when you access it
console.log(doc.name);
// > 'Learn RxDB'Docs Src Docs Leader Election (`docs-src/docs/leader-election.md`)
title: Leader Election slug: leader-election.html image: /headers/leader-election.jpg
import {CenteredImage} from '@site/src/components/centered-image';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {LeaderElectionTabs} from '@site/src/components/animations/leader-election-tabs';
Leader-Election
RxDB comes with a leader-election which elects a leading instance between different instances in the same javascript runtime.
Before you read this, please check out on how many of your open browser-tabs you have opened the same website more than once. Count them, I will wait..
So if you would now inspect the traffic that these open tabs produce, you can see that many of them send exact the same data over wire for every tab. No matter if the data is sent with an open websocket or by polling.
Use-case-example
Imagine we have a website which displays the current temperature of the visitors location in various charts, numbers or heatmaps. To always display the live-data, the website opens a websocket to our API-Server which sends the current temperature every 10 seconds. Using the way most sites are currently build, we can now open it in 5 browser-tabs and it will open 5 websockets which send data 6*5=30 times per minute. This will not only waste the power of your clients device, but also wastes your api-servers resources by opening redundant connections.
Solution
The solution to this redundancy is the usage of a leader-election-algorithm which makes sure that always exactly one tab is managing the remote-data-access. The managing tab is the elected leader and stays leader until it is closed. No matter how many tabs are opened or closed, there must be always exactly one leader.
You could now start implementing a messaging-system between your browser-tabs, hand out which one is leader, solve conflicts and reassign a new leader when the old one 'dies'.
Or just use RxDB which does all these things for you.
Add the leader election plugin
To enable the leader election, you have to add the leader-election plugin.
import { addRxPlugin } from 'rxdb';
import { RxDBLeaderElectionPlugin } from 'rxdb/plugins/leader-election';
addRxPlugin(RxDBLeaderElectionPlugin);Code-example
To make it easy, here is an example where the temperature is pulled every ten seconds and saved to a collection. The pulling starts at the moment where the opened tab becomes the leader.
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'weatherDB',
storage: getRxStorageLocalstorage(),
password: 'myPassword',
multiInstance: true
});
await db.addCollections({
temperature: {
schema: mySchema
}
});
db.waitForLeadership()
.then(() => {
console.log('Long lives the king!'); // <- runs when db becomes leader
setInterval(async () => {
const temp = await fetch('https://example.com/api/temp/');
db.temperature.insert({
degrees: temp,
time: new Date().getTime()
});
}, 1000 * 10);
});Handle Duplicate Leaders
On rare occasions, it can happen that more than one leader is elected. This can happen when the CPU is on 100% or for any other reason the JavaScript process is fully blocked for a long time.
For most cases this is not really a problem because on duplicate leaders, both browser tabs replicate with the same backend anyways.
To handle the duplicate leader event, you can access the leader elector and set a handler:
import {
getLeaderElectorByBroadcastChannel
} from 'rxdb/plugins/leader-election';
const leaderElector = getLeaderElectorByBroadcastChannel(broadcastChannel);
leaderElector.onduplicate = async () => {
// Duplicate leader detected -> reload the page.
location.reload();
}Live-Example
In this example the leader is marked with the crown โ
Try it out
Run the angular-example where the leading tab is marked with a crown on the top-right-corner.
Notice
The leader election is implemented via the broadcast-channel module.
The leader is elected between different processes on the same javascript-runtime. Like multiple tabs in the same browser or multiple Node.js processes on the same machine. It will not run between different replicated instances.
Docs Src Docs Logger (`docs-src/docs/logger.md`)
title: RxDB Logger Plugin - Track & Optimize slug: logger.html image: /headers/logger.jpg
import {PremiumBlock} from '@site/src/components/premium-block';
import {Steps} from '@site/src/components/steps';
RxDB Logger Plugin
With the logger plugin you can log all operations to the storage layer of your RxDatabase.
This is useful to debug performance problems and for monitoring with Application Performance Monitoring (APM) tools like Bugsnag, Datadog, Elastic, Sentry and others.
Using the logger plugin
The logger is a wrapper that can be wrapped around any RxStorage. Once your storage is wrapped, you can create your database with the wrapped storage and the logging will automatically happen.
Import Plugins
import {
wrappedLoggerStorage
} from 'rxdb-premium/plugins/logger';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';Wrap Storage
// wrap a storage with the logger
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({})
});Create Database
// create your database with the wrapped storage
const db = await createRxDatabase({
name: 'mydatabase',
storage: loggingStorage
});
// create collections etc...Specify what to be logged
By default, the plugin will log all operations and it will also run a console.time()/console.timeEnd() around each operation. You can specify what to log so that your logs are less noisy. For this you provide a settings object when calling wrappedLoggerStorage().
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({}),
settings: {
// can used to prefix all log strings, default=''
prefix: 'my-prefix',
/**
* Be default, all settings are true.
*/
// if true, it will log timings with console.time() and console.timeEnd()
times: true,
// if false, it will not log meta storage instances like used in replication
metaStorageInstances: true,
// operations
bulkWrite: true,
findDocumentsById: true,
query: true,
count: true,
info: true,
getAttachmentData: true,
getChangedDocumentsSince: true,
cleanup: true,
close: true,
remove: true
}
});Using custom logging functions
With the logger plugin you can also run custom log functions for all operations.
const loggingStorage = wrappedLoggerStorage({
storage: getRxStorageIndexedDB({}),
onOperationStart: (operationsName, logId, args) => void,
onOperationEnd: (operationsName, logId, args) => void,
onOperationError: (operationsName, logId, args, error) => void
});Docs Src Docs Middleware (`docs-src/docs/middleware.md`)
title: Streamlined RxDB Middleware slug: middleware.html image: /headers/middleware.jpg
import {DocsAnimation} from '@site/src/components/docs-animation';
import {MiddlewareHook} from '@site/src/components/animations/middleware-hook';
Middleware
RxDB middleware-hooks (also called pre and post hooks) are functions which are passed control during execution of asynchronous functions.
The hooks are specified on RxCollection-level and help to create a clear what-happens-when-structure of your code.
Hooks can be defined to run parallel or in series one after another.
Hooks can be synchronous or asynchronous when they return a Promise.
To stop the operation at a specific hook, throw an error.
List
RxDB supports the following hooks:
- preInsert
- postInsert
- preSave
- postSave
- preRemove
- postRemove
- postCreate
Why is there no validate-hook?
Different to mongoose, the validation on document-data is running on the field-level for every change to a document.
This means if you set the value lastName of a RxDocument, then the validation will only run on the changed field, not the whole document.
Therefore it is not useful to have validate-hooks when a document is written to the database.
Use Cases
Middleware is useful for atomizing model logic and avoiding nested blocks of async code.
Here are some other ideas:
- complex validation
- removing dependent documents
- asynchronous defaults
- asynchronous tasks that a certain action triggers
- triggering custom events
- notifications
Usage
All hooks have the plain data as first parameter, and all but preInsert also have the RxDocument-instance as second parameter. If you want to modify the data in the hook, change attributes of the first parameter.
All hook functions are also this-bound to the RxCollection-instance.
Insert
An insert-hook receives the data-object of the new document.
lifecycle
- RxCollection.insert is called
- preInsert series-hooks
- preInsert parallel-hooks
- schema validation runs
- new document is written to database
- postInsert series-hooks
- postInsert parallel-hooks
- event is emitted to RxDatabase and RxCollection
preInsert
// series
myCollection.preInsert(function(plainData){
// set age to 50 before saving
plainData.age = 50;
}, false);
// parallel
myCollection.preInsert(function(plainData){
}, true);
// async
myCollection.preInsert(function(plainData){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the insert-operation
myCollection.preInsert(function(plainData){
throw new Error('stop');
}, false);postInsert
// series
myCollection.postInsert(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postInsert(function(plainData, rxDocument){
}, true);
// async
myCollection.postInsert(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);Save
A save-hook receives the document which is saved.
lifecycle
- RxDocument.save is called
- preSave series-hooks
- preSave parallel-hooks
- updated document is written to database
- postSave series-hooks
- postSave parallel-hooks
- event is emitted to RxDatabase and RxCollection
preSave
// series
myCollection.preSave(function(plainData, rxDocument){
// modify anyField before saving
plainData.anyField = 'anyValue';
}, false);
// parallel
myCollection.preSave(function(plainData, rxDocument){
}, true);
// async
myCollection.preSave(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the save-operation
myCollection.preSave(function(plainData, rxDocument){
throw new Error('stop');
}, false);postSave
// series
myCollection.postSave(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postSave(function(plainData, rxDocument){
}, true);
// async
myCollection.postSave(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);Remove
A remove-hook receives the document which is removed.
lifecycle
- RxDocument.remove is called
- preRemove series-hooks
- preRemove parallel-hooks
- deleted document is written to database
- postRemove series-hooks
- postRemove parallel-hooks
- event is emitted to RxDatabase and RxCollection
preRemove
// series
myCollection.preRemove(function(plainData, rxDocument){
}, false);
// parallel
myCollection.preRemove(function(plainData, rxDocument){
}, true);
// async
myCollection.preRemove(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);
// stop the remove-operation
myCollection.preRemove(function(plainData, rxDocument){
throw new Error('stop');
}, false);postRemove
// series
myCollection.postRemove(function(plainData, rxDocument){
}, false);
// parallel
myCollection.postRemove(function(plainData, rxDocument){
}, true);
// async
myCollection.postRemove(function(plainData, rxDocument){
return new Promise(res => setTimeout(res, 100));
}, false);postCreate
This hook is called whenever a RxDocument is constructed.
You can use postCreate to modify every RxDocument-instance of the collection.
This adds a flexible way to add specific behavior to every document. You can also use it to add custom getter/setter to documents. PostCreate-hooks cannot be asynchronous.
myCollection.postCreate(function(plainData, rxDocument){
Object.defineProperty(rxDocument, 'myField', {
get: () => 'foobar',
});
});
const doc = await myCollection.findOne().exec();
console.log(doc.myField);
// 'foobar':::note
This hook does not run on already created or cached documents. Make sure to add postCreate-hooks before interacting with the collection.
:::
Docs Src Docs Migration Schema (`docs-src/docs/migration-schema.md`)
title: Seamless Schema Data Migration with RxDB slug: migration-schema.html image: /headers/migration-schema.jpg
import {DocsAnimation} from '@site/src/components/docs-animation';
import {SchemaMigrationLogo} from '@site/src/components/animations/schema-migration-logo';
Migrate Database Data on schema changes
The RxDB Data Migration Plugin helps developers easily update stored data in their apps when they make changes to the data structure by changing the schema of a RxCollection. This is useful when developers release a new version of the app with a different schema.
Imagine you have your awesome todo-app distributed to many users. After a while, you decide that in your new version, you want to change the schema of the todos collection. Instead of saving the dueAt date as a unix-timestamp like 1486940585000 you want to have an ISO 8601 string like 2017-02-12T23:03:05.000Z so that it is readable and can be used in an index with a maxLength. To accomplish this, you change the schema and increase the version-number and you also change your code where you save new todos. But one problem remains: what happens with the todos which are already stored in the database on the user's device in the old schema?
With RxDB you can provide migrationStrategies for your collections that automatically (or on call) transform your existing data from older to newer schemas. This assures that the client's data always matches your newest code-version.
Add the migration plugin
To enable the data migration, you have to add the migration-schema plugin.
import { addRxPlugin } from 'rxdb';
import { RxDBMigrationSchemaPlugin } from 'rxdb/plugins/migration-schema';
addRxPlugin(RxDBMigrationSchemaPlugin);Providing strategies
Upon creation of a collection, you have to provide migrationStrategies when your schema's version-number is greater than 0. To do this, you have to add an object to the migrationStrategies property where a function for every schema-version is assigned. A migrationStrategy is a function which gets the old document-data as a parameter and returns the new, transformed document-data. If the strategy returns null, the document will be removed instead of migrated.
myDatabase.addCollections({
todos: {
schema: todoSchemaV1,
migrationStrategies: {
// 1 means, this transforms data from version 0 to version 1
1: function(oldDoc){
oldDoc.dueAt = new Date(oldDoc.dueAt).toISOString(); // unix to string
return oldDoc;
}
}
}
});Asynchronous strategies can also be used:
myDatabase.addCollections({
todos: {
schema: todoSchemaV1,
migrationStrategies: {
1: function(oldDoc){
oldDoc.dueAt = new Date(oldDoc.dueAt).toISOString(); // unix to string
return oldDoc;
},
/**
* 2 means, this transforms data from version 1 to version 2
* this returns a promise which resolves with the new document-data
*/
2: function(oldDoc){
// in the new schema (version: 2) we defined
// 'priority' as required field (integer)
// so we must get the priority of the todo from the server
return fetch('https://example.com/api/todoPriority/' + oldDoc.id)
.then(response => response.json())
.then(priority => {
oldDoc.priority = priority;
return oldDoc;
});
}
}
}
});you can also filter which documents should be migrated:
myDatabase.addCollections({
todos: {
schema: todoSchemaV1,
migrationStrategies: {
// 1 means, this transforms data from version 0 to version 1
1: function(oldDoc){
oldDoc.dueAt = new Date(oldDoc.dueAt).toISOString(); // unix to string
return oldDoc;
},
/**
* this removes all todos due before 2017-02-12
* they will not appear in the new collection
*/
2: function(oldDoc){
if(oldDoc.dueAt < '2017-02-12T00:00:00.000Z') return null;
else return oldDoc;
}
}
}
});autoMigrate
By default, the migration automatically happens when the collection is created. Calling RxDatabase.addCollections() returns only when the migration has finished.
If you have lots of data or the migrationStrategies take a long time, it might be better to start the migration 'by hand' and show the migration-state to the user as a loading-bar.
:::warning No writes during a running migration
While a schema migration is running on a collection, writes to that collection are not allowed.
Calls that would write will throw a COL25 error until the migration finishes.
Wait for collection.migratePromise() to resolve (or observe collection.getMigrationState().$ until status is DONE)
before performing writes.
:::
await myDatabase.addCollections({
todos: {
schema: todoSchemaV1,
autoMigrate: false, // <- migration will not run at creation
migrationStrategies: {
1: async function(oldDoc){
...
anything that takes very long
...
return oldDoc;
}
}
}
});
const todosCollection = myDatabase.todos;
// check if migration is needed
const needed = await todosCollection.migrationNeeded();
if(needed === false) {
return;
}
// start the migration
// 10 is the batch-size, how many docs will run
// at parallel
todosCollection.startMigration(10);
const migrationState = todosCollection.getMigrationState();
// 'start' the observable
migrationState.$.subscribe({
next: state => console.dir(state),
error: error => console.error(error),
complete: () => console.log('done')
});
// the emitted states look like this:
{
status: 'RUNNING' // oneOf 'RUNNING' | 'DONE' | 'ERROR'
count: {
total: 50, // amount of documents which must be migrated
handled: 0, // amount of handled docs
percent: 0 // percentage [0-100]
}
}If you don't want to show the state to the user, you can also use .migratePromise():
const migrationPromise = todosCollection.migratePromise(10);
await migrationPromise;migrationStates()
RxDatabase.migrationStates() returns an Observable that emits all migration states of any collection of the database.
Use this when you add collections dynamically and want to show a loading-state of the migrations to the user.
const allStatesObservable = myDatabase.migrationStates();
allStatesObservable.subscribe(allStates => {
allStates.forEach(migrationState => {
console.log(
'migration state of ' +
migrationState.collection.name
);
});
});Default values are not auto-applied
Schema default values are not automatically filled in during migration. When you insert a document through the normal RxDB API (like insert() or bulkInsert()), fields with a default in the schema are auto-filled. Migration does not do this. Your migration strategy has full explicit control over the document data and must set every field that the new schema needs.
const todoSchemaV1 = {
version: 1,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
// new field in v1 with a default value
tags: {
type: 'array',
default: [],
items: { type: 'string' }
}
},
required: ['id', 'name', 'tags']
};
const migrationStrategies = {
1: function(oldDoc){
/**
* You must explicitly set 'tags' here.
* The schema default [] will NOT be
* applied automatically during migration.
*/
oldDoc.tags = [];
return oldDoc;
}
};Migrating attachments
When you store RxAttachments together with your document, they can also be changed, added or removed while running the migration.
You can do this by mutating the oldDoc._attachments property.
import { createBlob } from 'rxdb';
const migrationStrategies = {
1: async function(oldDoc){
// do nothing with _attachments to keep all
// attachments in the new collection version.
return oldDoc;
},
2: async function(oldDoc){
// set _attachments to an empty object to
// delete all existing ones during migration.
oldDoc._attachments = {};
return oldDoc;
},
3: async function(oldDoc){
// update the data field of a single attachment to change its data.
oldDoc._attachments.myFile.data = await createBlob(
'my new text',
oldDoc._attachments.myFile.content_type
);
return oldDoc;
}
}Migration on multi-tab in browsers
If you use RxDB in a multiInstance environment, like a browser, it will ensure that exactly one tab is running a migration of a collection.
Also the migrationState.$ events are emitted between browser tabs.
Migration and Replication
If you use any of the RxReplication plugins, the migration will also run on the internal replication-state storage. It will migrate all assumedMasterState documents
so that after the migration is done, you do not have to re-run the replication from scratch.
RxDB assumes that you run the exact same migration on the servers and the clients. Notice that the replication pull-checkpoint will not be migrated. Your backend must be compatible with pull-checkpoints of older versions.
Migration should be run on all database instances
If you have multiple database instances (for example, if you are running replication inside of a Worker or SharedWorker and have created a database instance inside of the worker), schema migration should be started on all database instances. All instances must know about all migration strategies and any updated schema versions.
Docs Src Docs Migration Storage (`docs-src/docs/migration-storage.md`)
title: Migration Storage slug: migration-storage.html image: /headers/migration-storage.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
Storage Migration
The storage migration plugin can be used to migrate all data from one existing RxStorage into another. This is useful when:
- You want to migrate from one RxStorage to another one.
- You want to migrate to a new major RxDB version while keeping the previous saved data. This function only works from the previous major version upwards. Do not use it to migrate like rxdb v9 to v14.
The storage migration drops deleted documents and filters them out during the migration.
:::warning Do never change the schema while doing a storage migration
When you migrate between storages, you might want to change the schema in the same process. You should never do that because it will lead to problems afterwards and might make your database unusable.
When you also want to change your schema, first run the storage migration and afterwards run a normal schema migration.
:::
Usage
Lets say you want to migrate from LocalStorage RxStorage to the IndexedDB RxStorage.
import { migrateStorage } from 'rxdb/plugins/migration-storage';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
import { getRxStorageLocalstorage } from 'rxdb-old/plugins/storage-localstorage';
// create the new RxDatabase
const db = await createRxDatabase({
name: dbLocation,
storage: getRxStorageIndexedDB(),
multiInstance: false
});
await migrateStorage({
database: db as any,
/**
* Name of the old database,
* using the storage migration requires that the
* new database has a different name.
*/
oldDatabaseName: 'myOldDatabaseName',
oldStorage: getRxStorageLocalstorage(), // RxStorage of the old database
batchSize: 500, // batch size
// true: migrate all collections in parallel
// false (default): migrate in serial
parallel: false,
afterMigrateBatch: (input: AfterMigrateBatchHandlerInput) => {
console.log('storage migration: batch processed');
}
});:::note
Only collections that exist in the new database at the time you call migrateStorage() will have their data migrated.
- If your old database had collections
['users', 'posts', 'comments']but your new database only defines['users', 'posts'], then only users and posts data will be migrated. - Any collections missing from the new database will simply be skipped - no data for them will be read or written.
This allows you to selectively migrate only certain collections if desired, by choosing which collections to define before invoking migrateStorage().
:::
Migrate from a previous RxDB major version
To migrate from a previous RxDB major version, you have to install the 'old' RxDB in the package.json
{
"dependencies": {
"rxdb-old": "npm:[email protected]",
}
}Then you can run the migration by providing the old storage:
/* ... */
import { migrateStorage } from 'rxdb/plugins/migration-storage';
// import from the old RxDB version
import {
getRxStorageLocalstorage
} from 'rxdb-old/plugins/storage-localstorage';
await migrateStorage({
database: db as any,
/**
* Name of the old database,
* using the storage migration requires that the
* new database has a different name.
*/
oldDatabaseName: 'myOldDatabaseName',
oldStorage: getRxStorageLocalstorage(), // RxStorage of the old database
batchSize: 500, // batch size
parallel: false,
afterMigrateBatch: (input: AfterMigrateBatchHandlerInput) => {
console.log('storage migration: batch processed');
}
});
/* ... */Disable Version Check on [RxDB Premium ๐](/premium/)
RxDB Premium has a check in place that ensures that you do not accidentally use the wrong RxDB core and ๐ Premium version together which could break your database state.
This can be a problem during migrations where you have multiple versions of RxDB in use and it will throw the error Version mismatch detected.
You can disable that check by importing and running the disableVersionCheck() function from RxDB Premium.
// RxDB Premium v15 or newer:
import {
disableVersionCheck
} from 'rxdb-premium-old/plugins/shared';
disableVersionCheck();
// RxDB Premium v14:
// for esm
import {
disableVersionCheck
} from 'rxdb-premium-old/dist/es/shared/version-check.js';
disableVersionCheck();
// for cjs
import {
disableVersionCheck
} from 'rxdb-premium-old/dist/lib/shared/version-check.js';
disableVersionCheck();FAQ
Storage migration involves physically shifting all existing documents from one underlying RxStorage adapter (e.g., IndexedDB) into an entirely different storage engine (e.g., SQLite), often required during platform upgrades. Unlike PouchDB which lacked robust native migration rails, RxDB enforces distinct boundaries between structural Data Migrations (changing schema formats via the migrationStrategy map) and underlying Storage Migrations (migrateStorage()). These two distinct mechanisms must never be executed simultaneously.
Docs Src Docs Nodejs Database (`docs-src/docs/nodejs-database.md`)
title: RxDB - The Real-Time Database for Node.js slug: nodejs-database.html image: /headers/nodejs-database.jpg
import { PerformanceChart } from '@site/src/components/performance-chart';
import { PERFORMANCE_DATA_NODE, PERFORMANCE_METRICS } from '@site/src/components/performance-data';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
Node.js Database
RxDB is a fast, reactive realtime NoSQL database made for JavaScript applications like Websites, hybrid Apps, Electron-Apps, Progressive Web Apps and Node.js. While RxDB was initially created to be used with UI applications, it has been matured and optimized to make it useful for pure server-side use cases. It can be used as embedded, local database inside of the Node.js JavaScript process, or it can be used similar to a database server that Node.js can connect to. The RxStorage layer makes it possible to switch out the underlying storage engine which makes RxDB a very flexible database that can be optimized for many scenarios.
Persistent Database
To get a "normal" database connection where the data is persisted to a file system, the RxDB real time database provides multiple storage implementations that work in Node.js.
The FoundationDB storage connects to a FoundationDB cluster which itself is just a distributed key-value engine. RxDB adds the NoSQL query-engine, indexes and other features on top of it.
It scales horizontally because you can always add more servers to the FoundationDB cluster to increase the capacity.
Setting up a RxDB database is pretty simple. You import the FoundationDB RxStorage and tell RxDB to use that when calling createRxDatabase:
import { createRxDatabase } from 'rxdb';
import { getRxStorageFoundationDB } from 'rxdb/plugins/storage-foundationdb';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageFoundationDB({
apiVersion: 620,
clusterFile: '/path/to/fdb.cluster'
})
});
// add a collection
await db.addCollections({
users: {
schema: mySchema
}
});
// run a query
const result = await db.users.find({
selector: {
name: 'foobar'
}
}).exec();Another alternative storage is the SQLite RxStorage that stores the data inside of a SQLite file-based database. The SQLite storage is faster than FoundationDB and does not require to set up a cluster or anything because SQLite directly stores and reads the data inside of the filesystem. The downside of that is that it only scales vertically.
import { createRxDatabase } from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsNode
} from 'rxdb-premium/plugins/storage-sqlite';
import sqlite3 from 'sqlite3';
const myRxDatabase = await createRxDatabase({
name: 'path/to/database/file/foobar.db',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsNode(sqlite3)
})
});Because the SQLite RxStorage is not free and you might not want to set up a FoundationDB cluster, there is also the SQLite trial storage which ships with RxDB Core. It uses the node:sqlite module that is built into Node.js. The trial storage is limited to 500 documents and does not use indexes, so it works for prototypes but is not made for production.
import { createRxDatabase } from 'rxdb';
import {
getRxStorageSQLiteTrial,
getSQLiteBasicsNodeNative
} from 'rxdb/plugins/storage-sqlite';
import { DatabaseSync } from 'node:sqlite';
const myRxDatabase = await createRxDatabase({
name: 'path/to/database/file/foobar.db',
storage: getRxStorageSQLiteTrial({
sqliteBasics: getSQLiteBasicsNodeNative(DatabaseSync)
})
});Here is a performance comparison chart of the different storages (lower is better):
RxDB as Node.js In-Memory Database
One of the easiest way to use RxDB in Node.js is to use the Memory RxStorage. As the name implies, it stores the data directly in-memory of the Node.js JavaScript process. This makes it really fast to read and write data but of course the data is not persisted and will be lost when the nodejs process exits. Often the in-memory option is used when RxDB is used in unit tests because it automatically cleans up everything afterwards.
import { createRxDatabase } from 'rxdb';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
const db = await createRxDatabase({
name: 'exampledb',
storage: getRxStorageMemory()
});Also notice that the default memory limit of Node.js is 4gb (might change of newer versions) so for bigger datasets you might want to increase the limit with the max-old-space-size flag:
# increase the Node.js memory limit to 8GB
node --max-old-space-size=8192 index.jsHybrid In-memory-persistence-synced storage
If you want to have the performance of an in-memory database but require persistency of the data, you can use the memory-mapped storage. On database creation it will load all data into the memory and on writes it will first write the data into memory and later also write it to the persistent storage in the background. In the following example the FoundationDB storage is used, but any other RxStorage can be used as persistence layer.
import { createRxDatabase } from 'rxdb';
import { getRxStorageFoundationDB } from 'rxdb/plugins/storage-foundationdb';
import { getMemoryMappedRxStorage } from 'rxdb-premium/plugins/storage-memory-mapped';
const db = await createRxDatabase({
name: 'exampledb',
storage: getMemoryMappedRxStorage({
storage: getRxStorageFoundationDB({
apiVersion: 620,
clusterFile: '/path/to/fdb.cluster'
})
})
});While this approach gives you a database with great performance and persistent, it has two major downsides:
- The database size is limited to the memory size
- Writes can be lost when the Node.js process exists between a write to the memory state and the background persisting.
Share database between microservices with RxDB
Using a local, embedded database in Node.js works great until you have to share the data with another Node.js process or another server at all.
To share the database state with other instances, RxDB provides two different methods. One is replication and the other is the remote RxStorage.
The replication copies over the whole database set to other instances live-replicates all ongoing writes. This has the benefit of scaling better because each of your microservice will run queries on its own copy of the dataset.
Sometimes however you might not want to store the full dataset on each microservice. Then it is better to use the remote RxStorage and connect it to the "main" database. The remote storage will run all operations the main database and return the result to the calling database.
FAQ
You must choose a database based on your project requirements. For simple server-side document storage you use MongoDB or CouchDB. For relational data you use PostgreSQL or MySQL. If you need realtime synchronization between your Node.js backend and client applications you use RxDB. RxDB provides local-first offline support and seamless data replication. You can combine RxDB with storage plugins like SQLite or FoundationDB to achieve high performance.
No, the LocalStorage API is strictly a browser-exclusive window object feature and does not exist in native Node.js V8 execution environments. Attempting to mount the getRxStorageLocalstorage plugin inside a pure Node server will immediately trigger ReferenceError: localStorage is not defined. To achieve persistent local storage within Node.js architecture blocks, you must implement explicitly file-driven adapters like the Filesystem Node Storage or the SQLite Storage.
Follow up on RxDB+Node.js
- Check out the RxDB Nodejs example.
- If you haven't done yet, you should start learning about RxDB with the Quickstart Tutorial.
- I created a list of embedded JavaSCript databases that you will help you to pick a database if you do not want to use RxDB.
- Check out the MongoDB RxStorage that uses MongoDB for the database connection from your Node.js application and runs the RxDB real time database on top of it.
Docs Src Docs Nosql Performance Tips (`docs-src/docs/nosql-performance-tips.md`)
title: RxDB NoSQL Performance Tips slug: nosql-performance-tips.html image: /headers/nosql-performance-tips.jpg
Performance tips for RxDB and other [NoSQL](./articles/in-memory-nosql-database.md) databases
In this guide, you'll find techniques to improve the performance of RxDB operations and queries. Notice that all your performance optimizations should be done with a correct tracking of the metrics, otherwise you might change stuff into the wrong direction.
Use bulk operations
When you run write operations on multiple documents, make sure you use bulk operations instead of single document operations.
// wrong โ
for(const docData of dataAr){
await myCollection.insert(docData);
}
// right โ๏ธ
await myCollection.bulkInsert(dataAr);Help the query planner by adding operators that better restrict the index range
Often on complex queries, RxDB (and other databases) do not pick the optimal index range when querying a result set.
You can add additional restrictive operators to ensure the query runs over a smaller index space and has a better performance.
Lets see some examples for different query types.
/**
* Adding a restrictive operator for an $or query
* so that it better limits the index space for the time-field.
*/
const orQuery = {
selector: {
$or: [
{
time: { $gt: 1234 },
},
{
time: { $eq: 1234 },
user: { $gt: 'foobar' }
},
],
time: { $gte: 1234 } // <- add restrictive operator
}
}
/**
* Adding a restrictive operator for an $regex query
* so that it better limits the index space for the user-field.
* We know that all matching fields start with 'foo' so we can
* tell the query to use that as lower constraint for the index.
*/
const regexQuery = {
selector: {
user: {
$regex: '^foo(.*)0-9$', // a complex regex with a ^ in the beginning
$gte: 'foo' // <- add restrictive operator
}
}
}
/**
* Adding a restrictive operator for a query on an enum field.
* so that it better limits the index space for the time-field.
*/
const enumQuery = {
selector: {
/**
* Here lets assume our status field has
* the enum type
* ['idle', 'in-progress', 'done']
* so our restrictive operator can exclude
* all documents with 'done' as status.
*/
status: {
$in: [
'idle',
'in-progress',
],
$gt: 'done' // <- add restrictive operator on status
}
}
}For $in queries on an indexed field, the RxDB query planner limits the scanned index space to the range between the smallest and the largest of the given values. Adding a restrictive operator next to an $in is only useful when you know a tighter bound than the min/max of the values.
Set a specific index
Sometime the query planner of the database itself has no chance in picking the best index of the possible given indexes.
For queries where performance is very important, you might want to explicitly specify which index must be used.
const myQuery = myCollection.find({
selector: {
/* ... */
},
// explicitly specify index
index: [
'fieldA',
'fieldB'
]
});Try different ordering of index fields
The order of the fields in a compound index is very important for performance. When optimizing index usage, you should try out different orders on the index fields and measure which runs faster. For that it is very important to run tests on real-world data where the distribution of the data is the same as in production.
For example when there is a query on a user collection with an age and a gender field, it depends if the index ['gender', 'age'] performance better as ['age', 'gender'] based on the distribution of data:
const query = myCollection
.findOne({
selector: {
age: {
$gt: 18
},
gender: {
$eq: 'm'
}
},
/**
* Because the developer knows that 50% of the documents are 'male',
* but only 20% are below age 18,
* it makes sense to enforce using the
* ['gender', 'age'] index to improve
* performance. This could not be known
* by the query planer which might have
* chosen ['age', 'gender'] instead.
*/
index: ['gender', 'age']
});Notice that RxDB has the Query Optimizer Plugin that can be used to automatically find the best indexes.
Make a Query "hot" to reduce load
Having a query where the up-to-date result set is needed more than once, you might want to make the query "hot" by permanently subscribing to it. This ensures that the query result is kept up to date by RxDB ant the EventReduce algorithm at any time so that at the moment you need the current results, it has them already.
For example when you use RxDB at Node.js for a webserver, you should use an outer "hot" query instead of running the same query again on every request to a route.
// wrong โ
app.get('/list', (req, res) => {
const result = await myCollection.find({/* ... */}).exec();
res.send(JSON.stringify(result));
});
// right โ๏ธ
const query = myCollection.find({/* ... */});
query.subscribe(); // <- make it hot
app.get('/list', (req, res) => {
const result = await query.exec();
res.send(JSON.stringify(result));
});Store parts of your document data as attachment
For in-app databases like RxDB, it does not make sense to partially parse the JSON of a document. Instead, always the whole document json is parsed and handled. This has a better performance because JSON.parse() in JavaScript directly calls a C++ binding which can parse really fast compared to a partial parsing in JavaScript itself. Also by always having the full document, RxDB can de-duplicate memory caches of document across multiple queries.
The downside is that very very big documents with a complex structure can increase query time significantly. Documents fields with complex that are mostly not in use, can be move into an attachment. This would lead RxDB to not fetch the attachment data each time the document is loaded from disc. Instead only when explicitly asked for.
const myDocument = await myCollection.insert({/* ... */});
const attachment = await myDocument.putAttachment(
{
id: 'otherStuff.json',
data: createBlob(JSON.stringify({/* ... */}), 'application/json'),
type: 'application/json'
}
);Process queries in a worker process
Moving database storage into a WebWorker can significantly improve performance in web applications that use RxDB or similar NoSQL databases. When database operations are executed in the main JavaScript thread, they can block or slow down the User Interface, especially during heavy or complex data operations. By offloading these operations to a WebWorker, you effectively separate the data processing workload from the UI thread. This means the main thread remains free to handle user interactions and render updates without delay, leading to a smoother and more responsive user experience. Additionally, WebWorkers allow for parallel data processing, which can expedite tasks like querying and indexing. This approach not only enhances UI responsiveness but also optimizes overall application performance by leveraging the multi-threading capabilities of modern browsers.
With RxDB you can use the Worker and SharedWorker plugin to move the query processing away from the main thread.
Use less plugins and hooks
Utilizing fewer hooks and plugins in RxDB or similar NoSQL database systems can lead to markedly better performance. Each additional hook or plugin introduces extra layers of processing and potential overhead, which can cumulatively slow down database operations. These extensions often execute additional code or enforce extra checks with each operation, such as insertions, updates, or deletions. While they can provide valuable functionalities or custom behaviors, their overuse can inadvertently increase the complexity and execution time of basic database operations. By minimizing their use and only employing essential hooks and plugins, the system can operate more efficiently. This streamlined approach reduces the computational burden on each transaction, leading to faster response times and a more efficient overall data handling process, especially critical in high-load or real-time applications where performance is paramount.
Docs Src Docs Offline First (`docs-src/docs/offline-first.md`)
title: Local First / Offline First slug: offline-first.html image: /headers/offline-first.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
Local First / Offline First
Local-First (aka offline first) is a software paradigm where the software stores data locally on the client's device and must work as well offline as it does online.
To implement this, you have to store data at the client side, so that your application can still access it when the internet connection is lost.
This can be either done with complex caching strategies, or by using a local-first, offline database (like RxDB) that stores the data inside of a local database like IndexedDB and replicates it from and to the backend in the background. This makes the local database, not the server, the gateway for all persistent changes in application state. In DDIL environments, where the connection is denied, disrupted, intermittent, or limited most of the time, this is not an optimization but the only architecture that works.
Offline first is not about having no internet connection
:::note
I wrote a follow-up version of offline/first local first about Why Local-First Is the Future and what are Its Limitations
:::
While in the past, internet connection was unstable, things are changing especially for mobile devices.
Mobile networks have become better and having no internet becomes less common even in remote locations.
So if we did not care about offline first applications in the past, why should we even care now?
In the following I will point out why offline first applications are better, not because they support offline usage, but because of other reasons.
UX is better without loading spinners
In 'normal' web applications, most user interactions like fetching, saving or deleting data, correspond to a request to the backend server. This means that each of these interactions require the user to await the unknown latency to and from a remote server while looking at a loading spinner.
In offline-first apps, the operations go directly against the local storage which happens almost instantly. There is no perceptible loading time and so it is not even necessary to implement a loading spinner at all. As soon as the user clicks, the UI represents the new state as if it was already changed in the backend.
Multi-tab usage just works
Many, even big websites like Amazon, Reddit and Stack Overflow do not handle multi tab usage correctly. When a user has multiple tabs of the website open and does a login on one of these tabs, the state does not change on the other tabs.
On offline first applications, there is always exactly one state of the data across all tabs. Offline first databases (like RxDB) store the data inside of IndexedDb and share the state between all tabs of the same origin.
Latency is more important than bandwidth
In the past, often the bandwidth was the limiting factor on determining the loading time of an application.
But while bandwidth has improved over the years, latency became the limiting factor.
You can always increase the bandwidth by setting up more cables or sending more Starlink satellites to space.
But reducing the latency is not so easy. It is defined by the physical properties of the transfer medium, the speed of light and the distance to the server. All of these three are hard to optimize.
Offline first applications benefit from that because sending the initial state to the client can be done much faster with more bandwidth. And once the data is there, we no longer have to care about the latency to the backend server because you can run near zero latency queries locally.
Realtime comes for free
Most websites lie to their users. They do not lie because they display wrong data, but because they display old data that was loaded from the backend at the time the user opened the site.
To overcome this, you could build a realtime website where you create a websocket that streams updates from the backend to the client. This means work. Your client needs to tell the server which page is currently opened and which updates the client is interested in. Then the server can push updates over the websocket and you can update the UI accordingly.
With offline first applications, you already have a realtime replication with the backend. Most offline first databases provide some concept of changestream or data subscriptions and with RxDB you can even directly subscribe to query results or single fields of documents. This makes it easy to have an always updated UI whenever data on the backend changes.
Scales with data size, not with the amount of user interaction
On normal applications, each user interaction can result in multiple requests to the backend server which increase its load.
The more users interact with your application, the more backend resources you have to provide.
Offline first applications do not scale up with the amount of user actions but instead they scale up with the amount of data.
Once that data is transferred to the client, the user can do as many interactions with it as required without connecting to the server.
Modern apps have longer runtimes
In the past you used websites only for a short time. You open it, perform some action and then close it again. This made the first load time the important metric when evaluating page speed.
Today web applications have changed and with it the way we use them. Single page applications are opened once and then used over the whole day. Chat apps, email clients, PWAs and hybrid apps. All of these were made to have long runtimes.
This makes the time for user interactions more important than the initial loading time. Offline first applications benefit from that because there is often no loading time on user actions while loading the initial state to the client is not that relevant.
You might not need REST
On normal web applications, you make different requests for each kind of data interaction.
For that you have to define a swagger route, implement a route handler on the backend and create some client code to send or fetch data from that route. The more complex your application becomes, the more REST routes you have to maintain and implement.
With offline first apps, you have a way to hack around all this cumbersome work. You just replicate the whole state from the server to the client. The replication does not only run once, you have a realtime replication and all changes at one side are automatically there on the other side. On the client, you can access every piece of state with a simple database query.
While this of course only works for amounts of data that the client can load and store, it makes implementing prototypes and simple apps much faster.
You might not need Redux
Data is hard, especially for UI applications where many things can happen at the same time.
The user is clicking around. Stuff is loaded from the server. All of these things interact with the global state of the app.
To manage this complexity it is common to use state management libraries like Redux or MobX. With them, you write all this lasagna code to wrap the mutation of data and to make the UI react to all these changes.
On offline first apps, your global state is already there in a single place stored inside of the local database.
You do not have to care whether this data came from the UI, another tab, the backend or another device of the same user. You can just make writes to the database and fetch data out of it.
FAQ
RxDB excels as an offline-first mobile database for unreliable network environments. You build mobile applications using the local-first paradigm to store data directly on the device. RxDB ensures full read and write capabilities regardless of current internet connection status. The sync engine automatically resolves conflicts and pushes local changes to the server once connectivity is restored. This approach eliminates loading spinners and guarantees a consistent user experience during network fluctuations.
Follow up
- Learn how to store and query data with RxDB in the RxDB Quickstart
- Downsides of Offline First
- I wrote a follow-up version of offline/first local first about Why Local-First Is the Future and what are Its Limitations
Docs Src Docs Orm (`docs-src/docs/orm.md`)
title: ORM slug: orm.html image: /headers/orm.jpg
Object-Data-Relational-Mapping
Like mongoose, RxDB has ORM capabilities which can be used to add specific behavior to documents and collections.
statics
Statics are defined collection-wide and can be called on the collection.
Add statics to a collection
To add static functions, pass a statics object when you create your collection. The object contains functions, mapped to their function names.
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
statics: {
scream: function(){
return 'AAAH!!';
}
}
}
});
console.log(heroes.scream());
// 'AAAH!!'You can also use the this keyword which resolves to the collection:
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
statics: {
whoAmI: function(){
return this.name;
}
}
}
});
console.log(heroes.whoAmI());
// 'heroes'Instance Methods
Instance methods are defined collection-wide. They can be called on the RxDocuments of the collection.
Add instance methods to a collection
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
methods: {
scream: function(){
return 'AAAH!!';
}
}
}
});
const doc = await heroes.findOne().exec();
console.log(doc.scream());
// 'AAAH!!'Here you can also use the this keyword:
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
methods: {
whoAmI: function(){
return 'I am ' + this.name + '!!';
}
}
}
});
await heroes.insert({
name: 'Skeletor'
});
const doc = await heroes.findOne().exec();
console.log(doc.whoAmI());
// 'I am Skeletor!!'attachment-methods
Attachment methods are defined collection-wide. They can be called on the RxAttachments of the RxDocuments of the collection.
const heroes = await myDatabase.addCollections({
heroes: {
schema: mySchema,
attachments: {
scream: function(){
return 'AAAH!!';
}
}
}
});
const doc = await heroes.findOne().exec();
const attachment = await doc.putAttachment({
id: 'cat.txt',
data: 'meow I am a kitty',
type: 'text/plain'
});
console.log(attachment.scream());
// 'AAAH!!'Docs Src Docs Partial Sync (`docs-src/docs/partial-sync.md`)
title: Partial Sync with RxDB slug: partial-sync.html image: /headers/partial-sync.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
import {PartialSyncChunks} from '@site/src/components/animations/partial-sync-chunks';
import {DocsAnimation} from '@site/src/components/docs-animation';
Partial Sync with RxDB
Partial sync is a replication pattern where a client only synchronizes a subset of the server's data instead of the full dataset. The subset is defined by a scope, for example a geographic region, a project, a tenant, a permission group, or a chunk of a game world. RxDB does not have a single "partial sync" switch; the pattern is built on top of the standard RxDB Sync Engine by running multiple replication states in parallel, each one filtered to a different scope, and starting or stopping them as the user's context changes.
This page explains what partial sync is, when to use it, and how to implement it correctly with RxDB.
When to use Partial Sync
A full sync, where every client downloads the entire collection, works well when the dataset is small or when every client truly needs every document. Partial sync becomes useful when one or more of the following is true:
- The full dataset is too large to fit on a client device.
- Each user only ever interacts with a small slice of the data (their accounts, their region, their project).
- You want to reduce bandwidth, initial load time, or storage usage on the client.
- The data is segmented by permissions and the server must not send rows the user is not allowed to see.
- The visible slice changes over time (the user moves between projects, regions, or game chunks).
Example Use Cases for Partial Sync
The pattern shows up in very different products, but the shape of the solution stays the same: pick a scope id, run one replication per active scope, stop replications when the scope is no longer in use.
Minecraft-like Voxel World
A voxel game (think Minecraft) divides the world into fixed-size chunks, where each chunk is a cube of blocks identified by its grid coordinates. The full world can be effectively infinite, so syncing every block to every client is not an option, both because of the amount of data and because the player only ever interacts with the blocks near them.
The natural scope is the chunk id. The client keeps one db.voxels collection and starts a replication for every chunk inside the player's render distance, for example a 5x5 area around the player. Each replication uses a replicationIdentifier like voxels-chunk-123 and a pull URL like /api/voxels/pull?chunkId=123. As the player walks, a new ring of chunks comes into range and an old ring leaves. The client starts replications for the new chunks and cancels replications for the chunks that fell out of range. Because every replication keeps its own checkpoint, walking back to a chunk visited five minutes ago only fetches the blocks that other players modified in the meantime instead of re-downloading the whole chunk.
The player can visit thousands of chunks per session, so old chunks should be removed from local storage when the scope is left to keep the synced dataset small.
Multi-Tenant SaaS Workspace
In a multi-tenant SaaS product (think Notion, Linear, or a B2B CRM), every user belongs to one or more workspaces or tenants, and each workspace has its own documents, tasks, customers, or notes. A user only ever works inside one workspace at a time, and they must never see data from a workspace they have no access to. Downloading every workspace's data to the client would waste bandwidth and would also be a serious permissions problem.
The scope is the workspace id. When the user opens workspace acme, the client starts a replication with replicationIdentifier: 'workspace-acme' against an endpoint like /api/sync?workspaceId=acme. The server enforces, on every pull and push, that the authenticated user is allowed to access that workspace and only returns rows that belong to it. When the user switches to workspace globex, the client starts a second replication for globex and, depending on product behavior, either keeps the acme replication running in the background (so notifications and counts stay live) or cancels it to save resources. Because each workspace has its own replicationIdentifier, switching back to acme later resumes from the last checkpoint instead of re-syncing from scratch.
This is also a good fit for the cleanup plugin: when a user leaves a workspace permanently, removing the locally cached documents for that scope keeps the on-device footprint proportional to the workspaces the user actually uses.
Field-Service Work Orders by Route
A field-service or last-mile delivery app (think a technician dispatch tool or a courier app) gives each worker a list of jobs assigned to them for the day. The backend has work orders for thousands of technicians across many regions, but a single technician only needs the orders on their current route, plus a small buffer for jobs that might be reassigned to them. They also need full offline support, because the app is used inside basements, elevators, and rural areas with no signal.
The scope here can be the route id, the assigned-technician id, or even a geo region like a city or postal code. The client starts a replication identified by that scope, for example routes-2026-05-28-tech-42, and the server returns only the work orders matching it. When dispatch reassigns the technician to a different route mid-day, the client cancels the old replication and starts a new one. Updates the technician made offline (status changes, signatures, photos) sit in the local collection until the device gets signal again, at which point the active scope's replication pushes them up and resolves any conflicts.
Compared to a full sync of all work orders, this keeps the device's storage tiny, makes the initial load fast even on a weak connection, and removes any risk of one technician seeing another technician's jobs.
Core Idea: One Collection, Many Replication States
Partial sync in RxDB is built on three properties of the Sync Engine:
replicateRxCollection()returns an independentRxReplicationStatethat can be started and cancelled at any time.- Each replication is identified by a unique
replicationIdentifier. RxDB stores a separate checkpoint per identifier, so two replications never share progress and never overwrite each other's metadata. - The
pull.handleris just a function. It can call any URL with any query parameters, so filtering the data by scope is done on the server using parameters that the client passes in.
The pattern is: keep one RxCollection on the client (for example db.voxels) and attach multiple replication states to it. Each replication uses a different replicationIdentifier (for example voxels-chunk-123) and a pull/push handler that talks to a scoped server endpoint (for example /api/voxels?chunkId=123). When the user enters a new scope, you start a new replication. When they leave, you cancel the old one.
RxCollection: db.voxels
|
+------------------------+------------------------+
| | |
replicationState replicationState replicationState
id: chunk-123 id: chunk-124 id: chunk-125
pull: ?chunkId=123 pull: ?chunkId=124 pull: ?chunkId=125All replication states share the same local storage, so queries against the collection return documents from every active scope at once. The user does not see a difference between data that came from one chunk or another, they just see the documents.
Implementation
The minimal implementation has three parts:
- A registry of active replication states, keyed by scope id.
- A
startScopeReplication(scopeId)function that creates a new replication if one does not exist. - A
stopScopeReplication(scopeId)function that cancels the replication when the scope is no longer needed.
/* Detailed source-code truncated for AI context efficiency. */Server side
The server must understand the scope parameter and only return documents that belong to it. A typical pull endpoint:
- Reads
chunkId,minUpdatedAtandlimitfrom the request. - Returns up to
limitdocuments from the scope, ordered byupdatedAt, that have changed afterminUpdatedAt. - Returns documents that are deleted with the deleted flag still set, so deletions replicate too.
A typical push endpoint:
- Accepts an array of
{ assumedMasterState, newDocumentState }rows. - For each row, compares
assumedMasterStateto the current server state. - If they match, applies
newDocumentState. If they do not, returns the current server state as a conflict.
This is the same contract as for a normal full RxDB replication, just with an extra scope filter applied to the query.
For real time updates, add a pull.stream$ that emits whenever the server has new writes in the scope. See Replication for the full handler contract.
Resumable Sync per Scope
Because each replication has its own replicationIdentifier, RxDB stores a separate checkpoint for every scope in its internal replication metadata. When the user leaves chunk-123 and comes back later, the new replication state for chunk-123 reuses the existing checkpoint and only fetches documents that changed since the last sync. The client never re-downloads the whole chunk after the first time.
This is the same "git-like" diff sync that the standard Sync Engine uses, applied independently per scope.
Tagging Documents with the Scope they came from
The pull.modifier runs on every pulled document before it is written to local storage. Use it to add a field like chunkId, projectId or tenantId that records which scope a document belongs to. The corresponding push.modifier can then return null for documents that do not belong to the current scope, so each replication only pushes the rows it is responsible for.
Tagging is what makes the next two things possible:
- Per-scope queries: the client can list "everything from this chunk" with a normal RxDB query, even though all scopes share one collection.
- Per-scope cleanup: when a scope is no longer needed, you can identify and remove exactly the documents that belong to it.
If the document already has a natural scope field on the server (for example chunkId is already part of the schema), the pull.modifier is not strictly needed, the field comes down with the document. The modifier is only required when the scope is implicit on the server side.
Cleaning up Data from Scopes the User Left
Cancelling a replication state stops the sync, but it does not remove the documents that were already written into the local collection. For long running apps, those documents accumulate. There are two patterns to deal with this:
Remove the documents from the local collection on scope exit. After
stopChunkReplication(chunkId), run a local query that finds all documents tagged with thatchunkIdand remove them. UseRxCollection.find({ selector: { chunkId } }).remove()for a bulk delete. Be aware that this also pushes deletes back to the server through any replication that still considers those documents in scope, so make sure the push modifiers filter correctly.Let the cleanup plugin reclaim space over time. Deleted documents are kept around for a configurable
minimumDeletedTimeso deletions can still replicate, then the plugin physically removes them. SetawaitReplicationsInSync: trueto make sure cleanup only runs when all active replication states are caught up.
For purely local "evict from cache" semantics (drop the data without telling the server), use RxCollection.bulkRemove() and combine it with a push modifier that returns null for those rows, so the local delete is not propagated as a server delete.
Real Time Updates with a Pull Stream
live: true replication also needs a pull.stream$ observable that emits new server-side writes as they happen. With partial sync, each replication state should subscribe to a stream that is already filtered to its scope, otherwise every client receives events for every scope and ignores most of them. Typical implementations:
- A WebSocket connection per scope, or one shared connection with a server-side subscription per scope.
- Server-Sent Events with a
chunkIdquery parameter. - A pub/sub channel named after the scope id.
When the stream emits a RESYNC event, RxDB falls back to the pull handler with the last checkpoint, which gives you a way to recover from missed events.
Tradeoffs
Partial sync is more flexible than full sync but adds complexity. Things to keep in mind:
- The server must support scoped queries. Pull and push endpoints need to enforce the scope, both for correctness and for permissions.
- Conflict resolution still runs on the client. If the same document is in two active scopes at once, both replications can try to push it. Use push modifiers to make exactly one replication responsible for each document.
- Checkpoint storage grows with the number of historical scopes. Each unique
replicationIdentifierkeeps its own metadata. If a user opens thousands of distinct scopes, callreplicationState.remove()instead ofcancel()to clear the metadata for scopes you do not plan to return to. - Real time streams need to be scoped too. A single global
pull.stream$defeats the bandwidth savings of partial sync. - Cleanup is explicit. RxDB will not automatically forget data from a scope you left, you have to remove it or rely on the cleanup plugin.
Partial Sync Beyond Games
The voxel example is just a convenient way to picture the pattern. The same shape applies to many real applications:
- Multi-tenant SaaS: one replication per workspace or tenant id. Switch when the user switches workspace.
- Permission-based sync: one replication per role or permission group, so users only download what they are allowed to see.
- Project / account scoped CRM: one replication per active account. Stop it when the account is closed.
- Region or branch scoped enterprise apps: one replication per region. Useful when the same database powers many geographically distinct sites.
- Time windowed sync: one replication for "last 30 days" and a separate one started on demand for older data.
In all of these, the building blocks are the same: a stable replicationIdentifier per scope, a scoped pull/push handler, a tag on every pulled document, and a clear rule for when a scope is started and stopped. With those four pieces, RxDB gives you the offline-first, conflict-aware, resumable replication of the standard Sync Engine, applied to exactly the slice of data the user actually needs.
FAQ
A single full replication still downloads every document in the collection to the client, even if your queries only read a small subset. Partial sync moves the filter to the server side so the documents are never sent over the wire and never written to local storage in the first place. Queries on the client then operate on a smaller dataset, which is faster and uses less disk.
Yes. RxDB does not limit the number of replication states attached to a single RxCollection. Each one keeps its own checkpoint metadata under its replicationIdentifier. In practice the limit is the number of concurrent network connections and the server's ability to handle parallel pull requests. If you need hundreds of active scopes, consider combining several into one replication on the server side and using a composite scope id.
No. You typically use one endpoint with a scope parameter, for example /api/sync/pull?scope=acme. The handler passes the scope as a query parameter on each pull and push call. The server uses that parameter to filter rows.
The document is stored once in the local collection because the primary key is shared. Without care, both replications would try to push local changes for that document. Use a push.modifier that returns null for documents that do not belong to the current scope, so exactly one replication is responsible for pushing each document. For pulls, RxDB de-duplicates by primary key, so receiving the same document from two scopes is safe.
No. As long as you reuse the same replicationIdentifier, RxDB resumes from the last stored checkpoint and only fetches documents that changed since then. This is the same diff sync that the standard Sync Engine uses, applied per scope.
The server should reject the next pull or push for that scope. On the client, cancel the replication and remove the locally cached documents that belong to it. The cleanup plugin can handle the physical removal of soft-deleted rows over time.
Yes, and it should. Each replication's pull.stream$ must only emit events for its scope, otherwise every client receives events for every scope. Implementations include one WebSocket per scope, a shared WebSocket with server-side subscriptions per scope, or Server-Sent Events with a scope query parameter.
Start the new scoped replications with their own replicationIdentifier values. The first pull for each scope will see no checkpoint and fetch from the beginning of that scope. Documents already in the local collection from the old full sync are matched by primary key and updated in place. Once the new replications are in sync, cancel the old full replication and call .remove() on it to clear its checkpoint metadata.
Not always. If the server can derive the scope from the document's own fields (for example a chunkId or workspaceId column), no client-side tagging is needed. If the scope is implicit on the server side, use a pull.modifier to attach the scope id to each pulled document so the client can identify and clean up data per scope later.
Docs Src Docs Plugins (`docs-src/docs/plugins.md`)
title: Creating Plugins slug: plugins.html image: /headers/plugins.jpg
Creating Plugins
Creating your own plugin is very simple. A plugin is basically a javascript-object which overwrites or extends RxDB's internal classes, prototypes, and hooks.
A basic plugin:
/* Detailed source-code truncated for AI context efficiency. */Properties
rxdb
The rxdb-property signals that this plugin is an rxdb-plugin. The value should always be true.
prototypes
The prototypes-property contains a function for each of RxDB's internal prototype that you want to manipulate. Each function gets the prototype-object of the corresponding class as parameter and then can modify it. You can see a list of all available prototypes here
overwritable
Some of RxDB's functions are not inside of a class-prototype but are static. You can set and overwrite them with the overwritable-object. You can see a list of all overwritables here.
hooks
Sometimes you don't want to overwrite an existing RxDB-method, but extend it. You can do this by adding hooks which will be called each time the code jumps into the hooks corresponding call. You can find a list of all hooks here.
options
RxDatabase and RxCollection have an additional options-parameter, which can be filled with any data required be the plugin.
const collection = myDatabase.addCollections({
foo: {
schema: mySchema,
options: { // anything can be passed into the options
foo: ()=>'bar'
}
}
})
// Afterwards you can use these options in your plugin.
collection.options.foo(); // 'bar'Docs Src Docs Population (`docs-src/docs/population.md`)
title: Populate and Link Docs in RxDB slug: population.html image: /headers/population.jpg
import {DocsAnimation} from '@site/src/components/docs-animation';
import {Population} from '@site/src/components/animations/population';
Population
There are no joins in RxDB but sometimes we still want references to documents in other collections. This is where population comes in. You can specify a relation from one RxDocument to another RxDocument in the same or another RxCollection of the same database.
Then you can get the referenced document with the population getter.
This works exactly like population with mongoose.
Schema with ref
The ref keyword in properties describes to which collection the field value belongs to (has a relationship).
export const todoSchemaWithParent = {
title: 'todo related to a parent todo',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
parent: {
ref: 'todos', // refers to collection todos
// ref-values must always be string
// or ['string', 'null']
// (primary of foreign RxDocument)
type: 'string'
}
},
required: ['id', 'name']
};You can also have a one-to-many reference by using a string array. The subtasks field of the example schema works this way.
export const todoSchemaWithSubtasks = {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
subtasks: {
type: 'array',
ref: 'todos',
items: {
type: 'string'
}
}
},
required: ['id', 'name']
};populate()
via method
To get the referred RxDocument, you can use the populate() method.
It takes the field path as attribute and returns a Promise which resolves to the foreign document or null if not found.
For an array field like subtasks, it resolves to an array of documents.
await myDatabase.todos.bulkInsert([
{
id: 'todo1',
name: 'Learn RxDB',
done: false,
dueAt: '2026-01-01T10:00:00.000Z',
priority: 8,
tags: ['learning', 'rxdb'],
subtasks: ['todo3']
},
{
id: 'todo3',
name: 'Set up replication',
dueAt: '2026-01-03T14:15:00.000Z',
priority: 10,
tags: ['rxdb', 'backend']
}
]);
const doc = await myDatabase.todos.findOne('todo1').exec();
const subtasks = await doc.populate('subtasks');
console.dir(subtasks); //> [RxDocument[todo3]]via getter
You can also get the populated RxDocument with the direct getter. To do this, you have to add an underscore suffix _ to the field name.
This also works on nested values.
const doc = await myDatabase.todos.findOne('todo1').exec();
const subtasks = await doc.subtasks_; // notice the underscore `_`
console.dir(subtasks); //> [RxDocument[todo3]]Example with nested reference
await myDatabase.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
relations: {
type: 'object',
properties: {
parent: {
type: 'string',
ref: 'todos'
}
}
}
},
required: ['id', 'name']
}
}
});
/**
* We assume myDocument is a document from the collection
*/
const parent = await myDocument.relations.parent_;
console.dir(parent); //> RxDocumentExample with array
await myDatabase.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
subtasks: {
type: 'array',
ref: 'todos',
items: {
type: 'string'
}
}
},
required: ['id', 'name']
}
}
});
//[insert the subtasks todo2, todo3 and todo4 here]
await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
subtasks: [
'todo2',
'todo3',
'todo4'
]
});
const doc = await myDatabase.todos.findOne('todo1').exec();
const subtasks = await doc.subtasks_;
console.dir(subtasks); //> Array.<RxDocument>Docs Src Docs Query Builder (`docs-src/docs/query-builder.md`)
title: Query Builder - Chained Queries in RxDB slug: query-builder.html image: /headers/query-builder.jpg
Query Builder
With the query-builder plugin you can write RxQuery selectors with chained methods instead of passing a JSON object. Each chained call adds one condition, and the result is a normal RxQuery with a normal Mango query underneath. This means everything on the RxQuery page (exec(), $, update(), remove(), indexes, the query cache) works the same for chained queries.
All examples on this page use the todos collection and the example documents from the example schema.
Add the Plugin
import { addRxPlugin } from 'rxdb';
import { RxDBQueryBuilderPlugin } from 'rxdb/plugins/query-builder';
addRxPlugin(RxDBQueryBuilderPlugin);Now you can chain query methods on any find() or findOne() call:
// selector: { priority: { $gt: 5 } }
const query = myDatabase.todos.find().where('priority').gt(5);
const results = await query.exec();
// > [RxDocument[todo1], RxDocument[todo3]]Selecting a Field with where()
where(path) sets the field that the following operator method applies to. The selected field stays active until the next where() call, so you can chain multiple operators on the same field.
// selector: { priority: { $gte: 3, $lte: 8 }, done: { $eq: false } }
const results = await myDatabase.todos.find()
.where('priority').gte(3).lte(8)
.where('done').eq(false)
.exec();
// > [RxDocument[todo1]]Nested fields are selected with the dot notation:
// selector: { 'assignee.name': { $eq: 'Alice' } }
const results = await myDatabase.todos.find()
.where('assignee.name').eq('Alice')
.exec();
// > [RxDocument[todo1]]You can also pass a plain selector object to where(). It is merged into the existing selector:
// selector: { done: false, priority: { $gt: 5 } }
const results = await myDatabase.todos.find()
.where({
done: false,
priority: { $gt: 5 }
})
.exec();
// > [RxDocument[todo1], RxDocument[todo3]]Comparison Operators
Each operator method maps to the mango operator with the same name:
| Method | Mango operator | Example | Result |
|---|---|---|---|
eq(value) / equals(value) |
$eq |
.where('done').eq(true) |
todo2 |
ne(value) |
$ne |
.where('done').ne(true) |
todo1, todo3, todo4 |
gt(value) |
$gt |
.where('priority').gt(5) |
todo1, todo3 |
gte(value) |
$gte |
.where('priority').gte(8) |
todo1, todo3 |
lt(value) |
$lt |
.where('priority').lt(3) |
todo4 |
lte(value) |
$lte |
.where('priority').lte(3) |
todo2, todo4 |
in(array) |
$in |
.where('tags').in(['home', 'backend']) |
todo2, todo3 |
nin(array) |
$nin |
.where('tags').nin(['rxdb']) |
todo2, todo4 |
exists(boolean) |
$exists |
.where('note').exists(true) |
todo2 |
regex(value) |
$regex |
.where('name').regex('.*RxDB.*') |
todo1 |
mod([divisor, remainder]) |
$mod |
.where('priority').mod([2, 0]) |
todo1, todo3 |
On an array field like tags, in() matches when at least one element of the array is in the given list, and nin() matches when no element is in the list.
exists(false) finds documents where an optional field is missing:
// selector: { assignee: { $exists: false } }
const results = await myDatabase.todos.find()
.where('assignee').exists(false)
.exec();
// > [RxDocument[todo3], RxDocument[todo4]]Notice that on an RxQuery, each chained method takes exactly one argument and applies to the field that was selected with where() before. The two-argument forms like .gt('priority', 5) or .exists('note') are not supported. Always select the field with where() first.
Regex Queries
regex() takes a regex string, or an object with $regex and $options when you need flags. Passing a RegExp instance throws an error, because a RegExp is not JSON-serializable and is mutable.
// sql equivalent of LIKE '%RxDB%'
const results = await myDatabase.todos.find()
.where('name').regex('.*RxDB.*')
.exec();
// > [RxDocument[todo1]]
// case insensitive: matches 'learn rxdb', 'LEARN RXDB', 'Learn RxDB' and so on
const caseInsensitiveResults = await myDatabase.todos.find()
.where('name').regex({ $regex: '^learn rxdb$', $options: 'i' })
.exec();
// > [RxDocument[todo1]]Array Fields
size() matches arrays with an exact number of elements:
// selector: { tags: { $size: 2 } }
const results = await myDatabase.todos.find()
.where('tags').size(2)
.exec();
// > [RxDocument[todo1], RxDocument[todo3]]elemMatch() matches documents where at least one element of the array fulfills all given conditions:
// todos with at least one tag that starts with 'back'
// selector: { tags: { $elemMatch: { $regex: '^back' } } }
const results = await myDatabase.todos.find()
.where('tags').elemMatch({ $regex: '^back' })
.exec();
// > [RxDocument[todo3]]For arrays of objects, elemMatch() also accepts a function instead of a selector. The function receives a nested query builder, so the element conditions can be chained with where() as well: .where('items').elemMatch(elem => { elem.where('name').eq('foo'); }).
To find documents where an array contains multiple values, use and() with one condition per value:
// todos that have both the 'rxdb' and the 'learning' tag
// selector: { $and: [ { tags: 'rxdb' }, { tags: 'learning' } ] }
const results = await myDatabase.todos.find()
.and([
{ tags: 'rxdb' },
{ tags: 'learning' }
])
.exec();
// > [RxDocument[todo1]]Logical Operators
or(), and() and nor() take an array of selectors and add them to the $or, $and and $nor lists of the query:
// todos that are done or have a priority of 9 or higher
// selector: { $or: [ { done: true }, { priority: { $gte: 9 } } ] }
const results = await myDatabase.todos.find()
.or([
{ done: true },
{ priority: { $gte: 9 } }
])
.exec();
// > [RxDocument[todo2], RxDocument[todo3]]They can be combined with where() conditions:
// open todos that are neither tagged 'home' nor have a priority below 2
/**
* selector: {
* done: { $eq: false },
* $nor: [ { tags: 'home' }, { priority: { $lt: 2 } } ]
* }
*/
const results = await myDatabase.todos.find()
.where('done').eq(false)
.nor([
{ tags: 'home' },
{ priority: { $lt: 2 } }
])
.exec();
// > [RxDocument[todo1], RxDocument[todo3]]Sorting
sort() accepts a space-separated string where a - prefix means descending, or an object with 'asc', 'desc', 1 or -1 as values. Both queries below produce the sort [{ priority: 'desc' }, { name: 'asc' }]:
const results = await myDatabase.todos.find()
.sort('-priority name')
.exec();
// > [RxDocument[todo3], RxDocument[todo1], RxDocument[todo2], RxDocument[todo4]]
const sameResults = await myDatabase.todos.find()
.sort({ priority: 'desc', name: 'asc' })
.exec();Same as with JSON queries, RxDB appends the primary key as last sort field to get a deterministic order.
Limit and Skip
limit() and skip() set the number of returned documents and the offset. Together with sort() this is the way to paginate:
// the second page with 2 todos per page
const results = await myDatabase.todos.find()
.sort('-priority')
.skip(2)
.limit(2)
.exec();
// > [RxDocument[todo2], RxDocument[todo4]]The chained methods also work with findOne():
// the todo with the highest priority above 5
const doc = await myDatabase.todos.findOne()
.where('priority').gt(5)
.sort('-priority')
.exec();
// > RxDocument[todo3]Chaining on an Existing Query
The query builder methods also work on queries that were created with a JSON selector. Each call returns a new RxQuery and does not modify the original one, because RxQuery instances are immutable. This makes it possible to derive multiple queries from a base query:
const openTodosQuery = myDatabase.todos.find({
selector: {
done: false
}
});
// selector: { done: false, priority: { $gte: 8 } }
const urgentOpenTodos = await openTodosQuery
.where('priority').gte(8)
.exec();
// > [RxDocument[todo1], RxDocument[todo3]]
// openTodosQuery is unchangedFull Example
The following example combines most of the methods. The comment shows the mango query that RxDB runs under the hood:
/**
* {
* selector: {
* done: { $eq: false },
* priority: { $gte: 5 },
* tags: { $in: ['rxdb'] },
* $or: [ { 'assignee.name': 'Alice' }, { note: { $exists: false } } ]
* },
* sort: [ { priority: 'desc' } ],
* limit: 10
* }
*/
const query = myDatabase.todos.find()
.where('done').eq(false)
.where('priority').gte(5)
.where('tags').in(['rxdb'])
.or([
{ 'assignee.name': 'Alice' },
{ note: { $exists: false } }
])
.sort('-priority')
.limit(10);
// emits the current results and again whenever they change
query.$.subscribe(todos => console.dir(todos));
// > [RxDocument[todo3], RxDocument[todo1]]You can always inspect the generated mango query with query.mangoQuery. This helps when a chained query does not return the documents you expect.
Docs Src Docs Query Cache (`docs-src/docs/query-cache.md`)
title: Efficient RxDB Queries via Query Cache slug: query-cache.html image: /headers/query-cache.jpg
QueryCache
RxDB uses a QueryCache which optimizes the reuse of queries at runtime. This makes sense especially when RxDB is used in UI-applications where people move for- and backwards on different routes or pages and the same queries are used many times. Because of the event-reduce algorithm cached queries are even valuable for optimization, when changes to the database occur between now and the last execution.
Cache Replacement Policy
To not let RxDB fill up all the memory, a cache replacement policy is defined that clears up the cached queries. This is implemented as a function which runs regularly, depending on when queries are created and the database is idle. The default policy should be good enough for most use cases but defining custom ones can also make sense.
The default policy
The default policy starts cleaning up queries depending on how much queries are in the cache and how much document data they contain.
- It will never uncache queries that have subscribers to their results
- It tries to always have less than 100 queries without subscriptions in the cache.
- It prefers to uncache queries that have never executed and are older than 30 seconds
- It prefers to uncache queries that have not been used for longer time
Other references to queries
With JavaScript, it is not possible to count references to variables. Therefore it might happen that an uncached RxQuery is still referenced by the users code and used to get results. This should never be a problem, uncached queries must still work. Creating the same query again however, will result in having two RxQuery instances instead of one.
Using a custom policy
A cache replacement policy is a normal JavaScript function according to the type RxCacheReplacementPolicy.
It gets the RxCollection as first parameter and the QueryCache as second. Then it iterates over the cached RxQuery instances and uncaches the desired ones with uncacheRxQuery(rxQuery). When you create your custom policy, you should have a look at the default.
To apply a custom policy to a RxCollection, add the function as attribute cacheReplacementPolicy.
const collection = await myDatabase.addCollections({
humans: {
schema: mySchema,
cacheReplacementPolicy: function(){ /* ... */ }
}
});Docs Src Docs Query Optimizer (`docs-src/docs/query-optimizer.md`)
title: Optimize Client-Side Queries with RxDB slug: query-optimizer.html image: /headers/query-optimizer.jpg
import {PremiumBlock} from '@site/src/components/premium-block';
Query Optimizer
The query optimizer can be used to determine which index is the best to use for a given query.
Because RxDB is used in client side applications, it cannot do any background checks or measurements to optimize the query plan because that would cause significant performance problems.
Usage
import {
findBestIndex
} from 'rxdb-premium/plugins/query-optimizer';
import {
getRxStorageIndexedDB
} from 'rxdb-premium/plugins/storage-indexeddb';
const bestIndexes = await findBestIndex({
schema: myRxJsonSchema, // see Schema Validation
/**
* In this example we use the IndexedDB RxStorage,
* but any other storage can be used for testing.
*/
storage: getRxStorageIndexedDB(),
/**
* Multiple queries can be optimized at the same time
* which decreases the overall runtime.
*/
queries: {
/**
* Queries can be mapped by a query id,
* here we use myFirstQuery as query id.
*/
myFirstQuery: {
selector: {
age: {
$gt: 10
}
},
},
mySecondQuery: {
selector: {
age: {
$gt: 10
},
lastName: {
$eq: 'Nakamoto'
}
},
}
},
testData: [/** data for the documents. **/]
});Important details
This is a build time tool. You should use it to find the best indexes for your queries during build time. Then you store these results and you application can use the best indexes during run time.
It makes no sense to run time optimization with a different RxStorage (+settings) that what you use in production. The result of the query optimizer is heavily dependent on the RxStorage and JavaScript runtime. For example it makes no sense to run the optimization in Node.js and then use the optimized indexes in the browser.
It is very important that you use production like
testData. Finding the best index heavily depends on data distribution and amount of stored/queried documents. For example if you store and query users with anagefield, it makes no sense to just use a random number for the age because in production theageof your users is not equally distributed.The higher you set
runs, the more test cycles will be performed and the more significant will be the time measurements which leads to a better index selection.
Docs Src Docs React Native Database (`docs-src/docs/react-native-database.md`)
title: React Native Database - Sync & Store Like a Pro slug: react-native-database.html image: /headers/react-native-database.jpg
import {Steps} from '@site/src/components/steps';
import {Tabs} from '@site/src/components/tabs';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
React Native Database
If you are looking for a React Native Database, you usually want three things:
- Persistence: Store data locally on the device so the app works offline.
- Reactivity: Automatically update the UI when data changes.
- Sync: Replicate data with a backend server in real-time.
RxDB covers all of these requirements out of the box. It is a local-first NoSQL database that runs deeply integrated with React Native, giving you the power of a full featured database engine inside your mobile app.
The Storage Layer
React Native does not have a native database engine. To store data persistently and efficiently, RxDB offers multiple storage options.
๐ Expo Filesystem (Highest Performance)
For the absolute best performance in React Native and Expo applications, the premium Expo Filesystem RxStorage is highly recommended. Built on expo-opfs, it completely bypasses the React Native bridge and delivers significantly faster read/write speeds than traditional SQLite.
SQLite
If you prefer a free solution or specifically need SQLite, RxDB fully supports SQLite. It works on all mobile platforms and abstracts the complex SQL commands into a simple, NoSQL JSON document API.
Depending on your environment, different SQLite adapters are recommended:
React Native CLI
For bare React Native projects, use react-native-quick-sqlite. It uses JSI (JavaScript Interface) to communicate directly with C++, effectively bypassing the slow React Native Bridge.
Installation:
npm install rxdb rxjs react-native-quick-sqliteConfiguration:
import { createRxDatabase } from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsQuickSQLite
} from 'rxdb-premium/plugins/storage-sqlite';
import { open } from 'react-native-quick-sqlite';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsQuickSQLite(open)
}),
multiInstance: false,
ignoreDuplicate: true
});Expo Go
If you are using Expo, use the official expo-sqlite module.
Installation:
npx expo install expo-sqlite
npm install rxdb rxjsConfiguration:
import { createRxDatabase } from 'rxdb';
import {
getRxStorageSQLite,
getSQLiteBasicsExpoSQLiteAsync
} from 'rxdb-premium/plugins/storage-sqlite';
import * as SQLite from 'expo-sqlite';
const db = await createRxDatabase({
name: 'mydatabase',
storage: getRxStorageSQLite({
sqliteBasics: getSQLiteBasicsExpoSQLiteAsync(SQLite.openDatabaseAsync)
}),
multiInstance: false,
ignoreDuplicate: true
});React Integration
RxDB is deeply integrated with React. It provides hooks that make fetching data and subscribing to changes effortless.
1. Provide the Database
Wrap your application with the RxDatabaseProvider.
import { RxDatabaseProvider } from 'rxdb/plugins/react';
export default function App() {
// ... create db instance
return (
<RxDatabaseProvider database={db}>
<MyComponent />
</RxDatabaseProvider>
);
}2. Observe Data
Use the useRxQuery hook (or useLiveRxQuery shortcut) to fetch data. The component will automatically re-render whenever the data in the database changes. You do not have to manage subscriptions or event listeners manually.
import { useRxCollection, useLiveRxQuery } from 'rxdb/plugins/react';
function TaskList() {
const collection = useRxCollection('tasks');
// This hook automatically updates 'tasks' whenever the query result changes
const { result: tasks } = useLiveRxQuery(
collection.find({
selector: {
done: { $eq: false }
},
sort: [{ createdAt: 'asc' }]
})
);
return (
<FlatList
data={tasks}
renderItem={({ item }) => <Text>{item.title}</Text>}
keyExtractor={item => item.id}
/>
);
}3. Signals (Performance Mode)
For high-performance applications with frequent data updates, re-rendering the entire React component might be too slow.
RxDB supports Signals (via @preact/signals-react or similar) to pinpoint updates directly to the DOM nodes.
// Enable the signals plugin once
import { addRxPlugin } from 'rxdb';
import {
RxDBReactivityPreactSignalsPlugin
} from 'rxdb/plugins/reactivity-preact-signals';
addRxPlugin(RxDBReactivityPreactSignalsPlugin);
// ... in your component
const signals = collection.find().$$; // Returns a Signal<Doc[]>Using signals allows you to update only the specific text node that changed, keeping your UI running at 60fps even with massive data flux.
Sync with Backend
A local database alone is useful. But most real-world apps also have to sync their data with a backend.
RxDB provides a robust replication protocol that can sync with any backend.
It has dedicated plugins for popular backend solutions:
For custom backends, you can implement the simple HTTP replication protocol.
Example: Sync with Supabase
Syncing is set-and-forget. You start the replication, and RxDB handles the rest (pulling changes, pushing writes, handling conflict resolution).
import { replicateSupabase } from 'rxdb/plugins/replication-supabase';
const replicationState = replicateSupabase({
replicationIdentifier: 'my-sync',
collection: db.tasks,
supabaseClient: supabase,
pull: {},
push: {},
});Because RxDB handles the sync layer, you can build your app as if it were a purely local application. All reads and writes happen against the local SQLite database instantly, while the replication happens in the background. This is the essence of Local-First development.
Comparison with Alternatives
In the following you can see how RxDB compares to the most common React Native storage and database solutions. Each of them has valid use cases. The trouble starts when you use a key-value store for document data or a cloud SDK for an offline-first app.
| Feature | AsyncStorage | MMKV | SQLite (Raw) | WatermelonDB | Realm | Firestore (SDK) | RxDB |
|---|---|---|---|---|---|---|---|
| Type | Key-Value Store | Key-Value Store | Relational (SQL) | ORM on SQLite | Object Store | Cloud Document Store | NoSQL Document Store |
| Reactivity | โ None | โ ๏ธ Per-key listeners | โ Manual events | โ Observables | โ Local listeners | โ Real-time listeners | โ Hooks / Signals / RxJS |
| Persistence | โ File (Slow) | โ File (memory-mapped) | โ File (Generic) | โ SQLite | โ Custom File | โ ๏ธ Partial Cache | โ SQLite / File |
| Sync | โ Manual | โ Manual | โ Manual | โ ๏ธ Client primitives only | โ Shut down 2025 | โ Firebase only | โ Any Backend |
| Query Engine | โ None | โ None | โ SQL Strings | โ Query builder | โ Custom API | โ Limited | โ Mango JSON Query |
| Schema | โ None | โ None | โ SQL Schema | โ Schema + Models | โ Class Schema | โ Loose | โ JSON Schema |
| Migration | โ Manual | โ Manual | โ Manual SQL | โ Migration API | โ Migration API | โ None | โ Automatic |
Summary
- AsyncStorage: Good for simple key-value pairs like settings and flags. On Android it stores everything in one SQLite-backed store with a default total size limit of
6 MBand a read limit of about2 MBper entry (known limits). Too slow and too limited for document data. - MMKV: react-native-mmkv is a fast, synchronous key-value store that uses JSI to skip the React Native bridge. It is a good AsyncStorage replacement for settings. But it is not a database: there are no queries, no indexes, and no sync.
- SQLite: Great foundation, but requires writing raw SQL and manual reactivity/sync. RxDB uses it as a storage layer instead.
- WatermelonDB: WatermelonDB is a reactive ORM on top of SQLite, built for large datasets with lazy loading, and it performs well at that job. Its sync feature only ships the client-side primitives, so you have to design and implement the pull/push endpoints on your backend yourself, and the relational schema requires hand-written migrations.
- Realm: Fast object store, but MongoDB deprecated it in September 2024 and shut down the Device Sync service on September 30, 2025 (deprecation notice, community discussion). The local database lives on as open source, but without sync you should not start new projects on it. The Realm migration guide shows how to move to RxDB.
- Firestore: Easy networked DB, but poor offline support (cannot start offline), vendor lock-in, and latency issues. RxDB can replicate with Firestore so that reads and writes stay local.
- RxDB: Combines the performance of local SQLite with the ease of NoSQL, automatic reactivity, and backend-agnostic synchronization.
Performance claims are cheap. You can find measured numbers for the different RxDB storages on the RxStorage performance page, and it is recommended to run your own measurements with the access patterns of your app.
FAQ
For small key-value data like settings, AsyncStorage or MMKV are enough. When your app stores documents, needs queries and indexes, or has to work offline, you have to use a real database. RxDB combines local SQLite persistence with automatic reactivity and replication to any backend, which is why it fits most offline-first React Native apps.
Yes. RxDB runs in Expo apps with the expo-sqlite adapter of the SQLite RxStorage, and for the best performance you can use the Expo Filesystem RxStorage which bypasses the React Native bridge. Both work with the managed Expo workflow.
No. AsyncStorage is an unencrypted key-value store without queries, indexes, or schemas. On Android its default total size limit is 6 MB (known limits), so storing your app's documents in it will bite back as soon as the dataset grows. Use it for settings and use a database for data.
MongoDB deprecated Realm in September 2024 and shut down its Device Sync service on September 30, 2025, so new projects should not be started on it. RxDB is the closest replacement because it is also a local, reactive, object-like database, and its Sync Engine works with your own backend instead of a proprietary cloud. The Realm migration guide describes the migration path.
Yes. The RxDB replication protocol is backend-agnostic and only requires you to expose pull and push handlers, for example over the simple HTTP replication. There are also prebuilt plugins for Supabase, Firestore, GraphQL, and CouchDB.
Ready to start?
Check out the React Native Example Project or read the Quickstart Guide.
Docs Src Docs React (`docs-src/docs/react.md`)
title: React RxDB integration slug: react.html image: /headers/react.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Using RxDB with React
RxDB provides first-class support for both React and React Native via a dedicated React integration. This integration makes it possible to use RxDB inside functional components using React Context and hooks, without manually subscribing to observables or managing cleanup logic.
The same APIs work in React for the web and in React Native. The only difference between platforms is the storage and environment setup. The React integration itself behaves identically in both.
General concept
RxDB is internally reactive and emits changes via RxJS observables. React and React Native, however, are render-driven frameworks that rely on component state and hooks. The RxDB React integration bridges these two models by exposing a small set of hooks that translate RxDB state into React renders.
Instead of hiding reactivity behind configuration flags, the integration uses explicit hooks. This makes component behavior predictable, avoids accidental subscriptions, and keeps performance characteristics easy to reason about.
Usage
Installation
Install RxDB and React as usual:
npm install rxdb react react-domDatabase creation
Database creation is not part of the React integration itself. RxDB is created in the same way as in non-React applications, including storage selection, plugins, replication, hooks, and schema definitions.
This separation is intentional. React components should never be responsible for creating or configuring the database. They should only consume it.
import {
createRxDatabase,
addRxPlugin
} from 'rxdb';
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
async function getDatabase() {
const db = await createRxDatabase({
name: 'heroesreactdb',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
heroes: {
schema: myRxSchema
}
});
/**
* Do other stuff here
* like setting up middleware
* or starting replication.
*/
return db;
}Providing the database
To use RxDB in a React or React Native application, the database instance must be provided via a context. This is done using RxDatabaseProvider.
The database itself is created outside of React, usually in a separate module. The provider is only responsible for making the database available to components once it has been initialized.
import React, { useEffect, useState } from 'react';
import { RxDatabaseProvider } from 'rxdb/plugins/react';
import { getDatabase } from './Database';
const App = () => {
const [database, setDatabase] = useState();
useEffect(() => {
const initDb = async () => {
const db = await getDatabase();
setDatabase(db);
};
initDb();
}, []);
if (database == null) {
return <span>
Loading <a href="https://rxdb.info/react.html">RxDB</a> database...
</span>;
}
return (
<RxDatabaseProvider database={database}>
{/* your application */}
</RxDatabaseProvider>
);
};
export default App;Accessing collections
import { useRxCollection } from 'rxdb/plugins/react';
const collection = useRxCollection('heroes');The hook returns the collection once it becomes available. During the initial render, the value may be undefined, so components must handle this case.
This hook does not subscribe to any data. It only provides access to the collection instance.
Queries
To render query results in your component, use the useRxQuery hook.
import { useRxQuery } from 'rxdb/plugins/react';
const query = {
collection: 'heroes',
query: {
selector: {},
sort: [{ name: 'asc' }]
}
};
const HeroCount = () => {
const { results, loading } = useRxQuery(query);
if (loading) {
return <span>Loading...</span>;
}
return <span>Total heroes: {results.length}</span>;
};The query is executed when the component renders. If the component re-renders, the query may be re-executed, but changes in the underlying data will not automatically trigger updates.
This hook is well suited for static views, server-side rendering, and cases where live updates are not required.
Live queries
Most React applications require views that automatically update when the database changes. For this purpose, RxDB provides the useLiveRxQuery hook.
The hook accepts a query description object and returns the current results together with a loading state.
import { useLiveRxQuery } from 'rxdb/plugins/react';
const query = {
collection: 'heroes',
query: {
selector: {},
sort: [{ name: 'asc' }]
}
};
const HeroList = () => {
const { results, loading } = useLiveRxQuery(query);
if (loading) {
return <span>Loading...</span>;
}
return (
<ul>
{results.map(hero => (
<li key={hero.name}>{hero.name}</li>
))}
</ul>
);
};The component automatically re-renders whenever the query result changes. Subscriptions are created when the component mounts and are cleaned up automatically when the component unmounts.
The returned documents are fully reactive RxDB documents and can be modified or removed directly.
Single document subscription
The useRxDocument hook subscribes to a single document by its primary key and re-renders whenever that document changes. It accepts an RxCollection instance directly for full type safety.
import { useRxCollection } from 'rxdb/plugins/react';
import { useRxDocument } from 'rxdb/plugins/react';
const HeroDetail = ({ heroId }: { heroId: string }) => {
const collection = useRxCollection('heroes');
const { result: hero, loading } = useRxDocument(collection, heroId);
if (loading || !hero) {
return <span>Loading...</span>;
}
return <span>{hero.name} ({hero.color})</span>;
};Replication status
The useReplicationStatus hook subscribes to an RxReplicationState and exposes its status as React state. This is useful for showing sync indicators in the UI.
import { useReplicationStatus } from 'rxdb/plugins/react';
const SyncIndicator = ({ replicationState }) => {
const status = useReplicationStatus(replicationState);
const { syncing, error, lastSyncedAt, canceled } = status;
if (canceled) return <span>Sync stopped</span>;
if (error) return <span>Sync error: {error.message}</span>;
if (syncing) return <span>Syncing...</span>;
if (lastSyncedAt) {
const time = new Date(lastSyncedAt).toLocaleTimeString();
return <span>Last synced: {time}</span>;
}
return <span>Idle</span>;
};React Native compatibility
All hooks and providers described on this page work the same way in React Native. The React integration does not rely on any browser-specific APIs.
The only platform-specific part of a React Native setup is database creation, where a different storage plugin is typically used. Once the database is created, the React integration behaves identically.
Signals
In addition to the React hooks shown on this page, RxDB also supports alternative reactivity models such as signals. RxDB's core reactivity system can be configured to expose reactive values using different primitives instead of RxJS observables, which makes it possible to integrate RxDB with signal-based approaches in React or other frameworks. This is an advanced capability and is independent of the React integration described here. For more details about how RxDB's reactivity system works and how custom reactivity can be configured, see the Reactivity documentation.
Follow Up
RxDB includes a full React example application that demonstrates the patterns described on this page, including database creation outside of React, usage of
RxDatabaseProvider, and data access viauseRxQuery,useLiveRxQuery, anduseRxCollection.A corresponding React Native example is also available and shows the same integration concepts applied in a mobile environment, with only the platform-specific storage and setup differing.
Docs Src Docs Reactivity (`docs-src/docs/reactivity.md`)
title: Signals & Custom Reactivity with RxDB slug: reactivity.html image: /headers/reactivity.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {ReactiveStream} from '@site/src/components/animations/reactive-stream';
Signals & Co. - Custom reactivity adapters instead of RxJS Observables
RxDB internally uses the rxjs library for observables and streams. All functionalities of RxDB like query results or document fields that expose values that change over time return a rxjs Observable that allows you to observe the values and update your UI accordingly depending on the changes to the database state.
However there are many reasons to use other reactivity libraries that use a different datatype to represent changing values. For example when you use signals in angular or react, the template refs of vue or state libraries like MobX and redux.
RxDB allows you to pass a custom reactivity factory on RxDatabase creation so that you can easily access values wrapped with your custom datatype in a convenient way.
Adding a reactivity factory
Angular
In angular we use Angular Signals as custom reactivity objects.
Import
import { createReactivityFactory } from 'rxdb/plugins/reactivity-angular';
import { Injectable, inject } from '@angular/core';Set the reactivity factory
Set the factory as reactivity option when calling createRxDatabase.
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: createReactivityFactory(inject(Injector))
});
// add collections/sync etc...Use the Signal in an Angular component
import { Component, inject } from '@angular/core';
import { CommonModule } from '@angular/common';
import { DbService } from '../db.service';
@Component({
selector: 'app-todos-list',
standalone: true,
imports: [CommonModule],
template: `
<ul>
<li
*ngFor="let t of todosSignal();"
>{{ t.title }}</li>
</ul>
`,
})
export class TodosListComponent {
private dbService = inject(DbService);
// RxDB query - Angular Signal
readonly todosSignal = this.dbService.db.todos.find().$$;
}An example of how signals are used in angular with RxDB, can be found at the RxDB Angular Example
React
For React, we use the Preact Signals for custom reactivity.
Install Preact Signals
npm install @preact/signals-core --saveImport
import {
PreactSignalsRxReactivityFactory
} from 'rxdb/plugins/reactivity-preact-signals';Set the reactivity factory
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: PreactSignalsRxReactivityFactory
});
// add collections/sync etc...Use the Signal in a React component
import { useEffect, useState } from 'preact/hooks';
import { getDatabase } from './db';
export function TodosList() {
const [db, setDb] = useState(null);
useEffect(() => {
getDatabase().then(setDb);
}, []);
if (!db) return null;
// RxQuery -> Preact Signal
const todosSignal = db.todos.find().$$;
return (
<ul>
{todosSignal.value.map((doc: any) => (
<li key={doc.primary}>
{doc.title}
</li>
))}
</ul>
);
}Vue
For Vue, we use the Vue Shallow Refs for custom reactivity.
Import
import { VueRxReactivityFactory } from 'rxdb/plugins/reactivity-vue';Set the reactivity factory
const database = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage(),
reactivity: VueRxReactivityFactory
});
// add collections/sync etc...Use the Shallow Ref in a Vue component
<script setup lang="ts">
import { getDatabase } from './db';
const db = getDatabase();
// RxQuery to Vue shallowRef signal
const todosSignal = db.todos.find().$$;
</script>
<template>
<ul>
<li v-for="t in todosSignal" :key="t.primary">
<label>{{ t.title }}</label>
</li>
</ul>
</template>Accessing custom reactivity objects
All observable data in RxDB is marked by the single dollar sign $ like RxCollection.$ for events or RxDocument.myField$ to get the observable for a document field. To make custom reactivity objects distinguishable, they are marked with double-dollar signs $$ instead. Here are some example on how to get custom reactivity objects from RxDB specific instances:
// RxDocument
// get signal that represents the document field 'foobar'
const signal = myRxDocument.get$$('foobar');
// same as above
const signal = myRxDocument.foobar$$;
// get signal that represents whole document over time
const signal = myRxDocument.$$;
// get signal that represents the deleted state of the document
const signal = myRxDocument.deleted$$;// RxQuery
// get signal that represents the query result set over time
const signal = collection.find().$$;
// get signal that represents the query result set over time
const signal = collection.findOne().$$;// RxLocalDocument
// get signal that represents the whole local document state
const signal = myRxLocalDocument.$$;
// get signal that represents the foobar field
const signal = myRxLocalDocument.get$$('foobar');Docs Src Docs Replication Appwrite (`docs-src/docs/replication-appwrite.md`)
title: Appwrite Realtime Sync for Local-First Apps slug: replication-appwrite.html image: /headers/replication-appwrite.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {SyncDiagram} from '@site/src/components/sync-diagram';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
RxDB Appwrite Replication
This replication plugin allows you to synchronize documents between RxDB and an Appwrite server. It supports both push and pull replication, live updates via Appwrite's real-time subscriptions, offline-capability and conflict resolution.
<VideoBox videoId="L07xPMyL8sY" title="Appwrite in 100 Seconds" duration="2:35" />Why you should use RxDB with Appwrite?
Appwrite is a secure, open-source backend server that simplifies backend tasks like user authentication, storage, database management, and real-time APIs.
RxDatabase is a reactive database for the frontend that offers offline-first capabilities and rich client-side data handling.
Combining the two provides several benefits:
Offline-First: RxDB keeps all data locally, so your application remains fully functional even when the network is unavailable. When connectivity returns, the RxDB โ Appwrite replication automatically resolves and synchronizes changes.
Real-Time Sync: With Appwriteโs real-time subscriptions and RxDBโs live replication, you can build collaborative features that update across all clients instantaneously.
Conflict Handling: RxDB offers flexible conflict resolution strategies, making it simpler to handle concurrent edits across multiple users or devices.
Scalable & Secure: Appwrite is built to handle production loads with granular access controls, while RxDB easily scales across various storage backends on the client side.
Simplicity & Modularity: RxDBโs plugin-based architecture, combined with Appwriteโs Cloud offering makes it one of the easiest way to build local-first realtime apps that scale.
<SyncDiagram server={false} backend={{ icon: "/files/icons/appwrite.svg" }} clientConnectionLabel="Realtime API" simulateOffline />
Preparing the Appwrite Server
You can either use the appwrite cloud or self-host the Appwrite server. In this tutorial we use the Cloud which is recommended for beginners because it is way easier to set up. You can later decide to self-host if needed.
Set up an Appwrite Endpoint and Project
Self-Hosted
Docker
Ensure docker and docker-compose is installed and your version are up to date:
docker-compose -vRun the installation script
The installation script runs inside of a docker container. It will create a docker-compose file and an .env file.
docker run -it --rm \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
--entrypoint="install" \
appwrite/appwrite:1.6.1Start/Stop
After the installation is done, you can manually stop and start the appwrite instance with docker compose:
# stop
docker-compose down
# start
docker-compose upAppwrite Cloud
Create a Cloud Account
Got to the Appwrite Console, create an account and login.
Create a Project
At the console click the + Create Project button to create a new project. Remember the project-id which will be used later.
Create an Appwrite Database and Collection
After creating an Appwrite project you have to create an Appwrite Database and a collection, you can either do this in code with the node-appwrite SDK or in the Appwrite Console as shown in this video:
<VideoBox videoId="HGlBpna17LQ" title="Appwrite Database Tutorial" duration="9:47" startAt={328} />Add your documents attributes
In the appwrite collection, create all attributes of your documents. You have to define all the fields that your document in your RxDB schema knows about. For the todos schema used below, these are name (string), done (boolean), and priority (integer). The id of the RxDB document is stored as the Appwrite document id. Notice that Appwrite does not allow for nested attributes. So when you use RxDB with Appwrite, you should also not have nested attributes in your RxDB schema.
Add a `deleted` attribute
Appwrite (natively) hard-deletes documents. But for offline-handling RxDB needs soft-deleted documents on the server so that the deletion state can be replicated with other clients.
In RxDB, _deleted indicates that a document is removed locally and you need a similar field in your Appwrite collection on the Server: You must define a deletedField with any name to mark documents as "deleted" in the remote collection. Mostly you would use a boolean field named deleted (set it to required). The plugin will treat any document with { [deletedField]: true } as deleted and replicate that state to local RxDB.
Set the Permission on the Appwrite Collection
Appwrite uses permissions to control data access on the collection level. Make sure that in the Console at Collection -> Settings -> Permissions you have set the permission according to what you want to allow your clients to do. For testing, just enable all of them (Create, Read, Update and Delete).
Setting up the RxDB - Appwrite Replication
Now that we have set up the Appwrite server, we can go to the client side code and set up RxDB and the replication:
Install the Appwrite SDK and RxDB:
npm install appwrite rxdbImport the Appwrite SDK and RxDB
import {
replicateAppwrite
} from 'rxdb/plugins/replication-appwrite';
import {
createRxDatabase,
addRxPlugin,
RxCollection
} from 'rxdb/plugins/core';
import {
getRxStorageLocalstorage
} from 'rxdb/plugins/storage-localstorage';
import { Client } from 'appwrite';Create a Database with a Collection
const myDatabase = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
const todoSchema = {
title: 'todo schema',
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: {
type: 'string',
maxLength: 100
},
name: {
type: 'string'
},
done: {
type: 'boolean',
default: false
},
priority: {
type: 'integer',
minimum: 0,
maximum: 10,
multipleOf: 1
}
},
required: ['id', 'name', 'done', 'priority']
};
await myDatabase.addCollections({
todos: {
schema: todoSchema
}
});
const collection = myDatabase.todos;Configure the Appwrite Client
Appwrite Cloud
const client = new Client();
client.setEndpoint('https://cloud.appwrite.io/v1');
client.setProject('YOUR_APPWRITE_PROJECT_ID');Self-Hosted
const client = new Client();
client.setEndpoint('http://localhost/v1');
client.setProject('YOUR_APPWRITE_PROJECT_ID');Start the Replication
const replicationState = replicateAppwrite({
replicationIdentifier: 'my-appwrite-replication',
client,
databaseId: 'YOUR_APPWRITE_DATABASE_ID',
collectionId: 'YOUR_APPWRITE_COLLECTION_ID',
deletedField: 'deleted', // Field that represents deletion in Appwrite
collection,
pull: {
batchSize: 10,
},
push: {
batchSize: 10
},
/*
* ...
* You can set all other options for RxDB replication states
* like 'live' or 'retryTime'
* ...
*/
});Do other things with the replication state
The RxAppwriteReplicationState which is returned from replicateAppwrite() allows you to run all functionality of the normal RxReplicationState.
FAQ
Yes, Appwrite supports creating multiple top-level databases within a single project, which cleanly partition collections. However, Appwrite is a rigid NoSQL document store that does not support nested subcollections (unlike Firebase). When utilizing the RxDB Appwrite Replication plugin, your local RxDB schema must mirror this flat topology precisely, keeping all documents completely devoid of complex nested relationships.
Appwrite uses MariaDB (a highly performant MySQL fork) as its core backing database driver. To offer developers a flat NoSQL experience, Appwrite abstracts the MariaDB relational complexity behind a unified Document API. This architectural mapping allows RxDB to replicate data effortlessly into Appwrite via standard REST endpoints without ever dealing with strict SQL table mappings or migrations.
Appwrite natively provides robust WebSocket subscriptions allowing clients to receive real-time document events while the network is active. However, Appwrite does not feature a built-in offline-first caching or background-sync engine. To achieve true offline capabilities, you must mount the RxDB Appwrite Replication plugin on the client. RxDB handles all local caching, queues offline writes securely, and automatically pushes local mutations to Appwrite when connectivity returns.
Limitations of the Appwrite Replication Plugin
- Appwrite primary keys only allow for the characters
a-z,A-Z,0-9, and underscore_(They cannot start with a leading underscore). Also the primary key has a max length of 36 characters. - The Appwrite replication only works on browsers. This is because the Appwrite SDK does not support subscriptions in Node.js.
- Appwrite does not allow for bulk write operations so on push one HTTP request will be made per document. Reads run in bulk so this is mostly not a problem.
- Appwrite does not allow for transactions or "update-if" calls which can lead to overwriting documents instead of properly handling conflicts when multiple clients edit the same document in parallel. This is not a problem for inserts because "insert-if-not" calls are made.
- Nested attributes in Appwrite collections are only possible via experimental relationship attributes, and compatibility with RxDB is not tested. Users opting to use these experimental relationship attributes with RxDB do so at their own risk.
Docs Src Docs Replication Couchdb (`docs-src/docs/replication-couchdb.md`)
title: RxDB's CouchDB Replication Plugin slug: replication-couchdb.html image: /headers/replication-couchdb.jpg
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Replication with CouchDB
A plugin to replicate between a RxCollection and a CouchDB server.
This plugin uses the RxDB Sync Engine to replicate with a CouchDB endpoint. This plugin does NOT use the official CouchDB replication protocol because the CouchDB protocol was optimized for server-to-server replication and is not suitable for fast client side applications, mostly because it has to run many HTTP-requests (at least one per document) and also it has to store the whole revision tree of the documents at the client. This makes initial replication and querying very slow.
Because the way RxDB handles revisions and documents is very similar to CouchDB, using the RxDB replication with a CouchDB endpoint is pretty straightforward.
Pros
- Faster initial replication.
- Works with any RxStorage, not just PouchDB.
- Easier conflict handling because conflicts are handled during replication and not afterwards.
- Does not have to store all document revisions on the client, only stores the newest version.
Cons
- Does not support the replication of attachments.
- Like all CouchDB replication plugins, this one is also limited to replicating 6 collections in parallel. Read this for workarounds
Usage
Start the replication via replicateCouchDB().
/* Detailed source-code truncated for AI context efficiency. */When you call replicateCouchDB() it returns a RxCouchDBReplicationState which can be used to subscribe to events, for debugging or other functions. It extends the RxReplicationState so any other method that can be used there can also be used on the CouchDB replication state.
Conflict handling
When conflicts appear during replication, the conflictHandler of the RxCollection is used, equal to the other replication plugins. Read more about conflict handling here.
Auth example
Lets say for authentication you need to add a bearer token as HTTP header to each request. You can achieve that by crafting a custom fetch() method that adds the header field.
const myCustomFetch = (url, options) => {
// flat clone the given options to not mutate the input
const optionsWithAuth = Object.assign({}, options);
// ensure the headers property exists
if(!optionsWithAuth.headers) {
optionsWithAuth.headers = {};
}
// add bearer token to headers
optionsWithAuth.headers['Authorization'] ='Basic S0VLU0UhIExFQ0...';
// call the original fetch function with our custom options.
return fetch(
url,
optionsWithAuth
);
};
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
/**
* Add the custom fetch function here.
*/
fetch: myCustomFetch,
pull: {},
push: {}
}
);Also when your bearer token changes over time, you can set a new custom fetch method while the replication is running:
replicationState.fetch = newCustomFetchMethod;Also there is a helper method getFetchWithCouchDBAuthorization() to create a fetch handler with authorization:
import {
replicateCouchDB,
getFetchWithCouchDBAuthorization
} from 'rxdb/plugins/replication-couchdb';
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
/**
* Add the custom fetch function here.
*/
fetch: getFetchWithCouchDBAuthorization('myUsername', 'myPassword'),
pull: {},
push: {}
}
);Limitations
Since CouchDB only allows synchronization through HTTP/1.1 long polling requests there is a limitation of 6 active synchronization connections before the browser prevents sending any further request. This limitation is at the level of browser per tab per domain (some browser, especially older ones, might have a different limit, see here).
Since this limitation is at the browser level there are several solutions:
- Use only a single database for all entities and set a "type" field for each of the documents
- Create multiple subdomains for CouchDB and use a max of 6 active synchronizations (or less) for each
- Use a proxy (ex: HAProxy) between the browser and CouchDB and configure it to use HTTP/2.0, since HTTP/2.0 multiplexes requests.
If you use nginx in front of your CouchDB, you can use these settings to enable http2-proxying to prevent the connection limit problem:
server {
http2 on;
location /db {
rewrite /db/(.*) /$1 break;
proxy_pass http://172.0.0.1:5984;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded;
proxy_set_header Connection "keep_alive";
}
}Known problems
Database missing
In contrast to PouchDB, this plugin does NOT automatically create missing CouchDB databases.
If your CouchDB server does not have a database yet, you have to create it by yourself by running a PUT request to the database name url:
// create a 'humans' CouchDB database on the server
const remoteDatabaseName = 'humans';
await fetch(
'http://example.com/db/' + remoteDatabaseName,
{
method: 'PUT'
}
);React Native
React Native does not have a global fetch method. You have to import fetch method with the cross-fetch package:
import crossFetch from 'cross-fetch';
const replicationState = replicateCouchDB(
{
replicationIdentifier: 'my-couchdb-replication',
collection: myRxCollection,
url: 'http://example.com/db/humans',
fetch: crossFetch,
pull: {},
push: {}
}
);Docs Src Docs Replication Firestore (`docs-src/docs/replication-firestore.md`)
title: Smooth Firestore Sync for Offline Apps slug: replication-firestore.html image: /headers/replication-firestore.jpg
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Replication with Firestore from Firebase
With the replication-firestore plugin you can do a two-way realtime replication
between your client side RxDB Database and a Cloud Firestore database that is hosted on the Firebase platform. It will use the RxDB Sync Engine to manage the replication streams, error- and conflict handling.
Replicating your Firestore state to RxDB can bring multiple benefits compared to using the Firestore directly:
- It can reduce your cloud fees because your queries run against the local state of the documents without touching a server and writes can be batched up locally and send to the backend in bulks. This is mostly the case for read heavy applications.
- You can run complex NoSQL queries on your documents because you are not bound to the Firestore Query handling. You can also use local indexes, compression and encryption and do things like fulltext search, fully locally.
- Your application can be truly Offline-First because your data is stored in a client side database. In contrast Firestore by itself only provides options to support offline also which more works like a cache and requires the user to be online at application start to run authentication.
- It reduces the vendor lock in because you can switch out the backend server afterwards without having to rebuild big parts of the application. RxDB supports replication plugins with multiple technologies and it is even easy to set up with your custom backend.
- You can use sophisticated conflict resolution strategies so you are not bound to the Firestore last-write-wins strategy which is not suitable for many applications.
- The initial load time of your application can be decreased because it will do an incremental replication on restarts.
Usage
Install the firebase package
npm install firebaseInitialize your Firestore Database
import * as firebase from 'firebase/app';
import {
getFirestore,
collection
} from 'firebase/firestore';
const projectId = 'my-project-id';
const app = firebase.initializeApp({
projectId,
databaseURL: 'http://localhost:8080?ns=' + projectId,
/* ... */
});
const firestoreDatabase = getFirestore(app);
const firestoreCollection = collection(firestoreDatabase, 'my-collection-name');Start the Replication
Start the replication by calling replicateFirestore() on your RxCollection.
const replicationState = replicateFirestore({
replicationIdentifier: `https://firestore.googleapis.com/${projectId}`,
collection: myRxCollection,
firestore: {
projectId,
database: firestoreDatabase,
collection: firestoreCollection
},
/**
* (required) Enable push and pull replication with firestore by
* providing an object with optional filter
* for each type of replication desired.
* [default=disabled]
*/
pull: {},
push: {},
/**
* Either do a live or a one-time replication
* [default=true]
*/
live: true,
/**
* (optional) likely you should just use the default.
*
* In firestore it is not possible to read out
* the internally used write timestamp of a document.
* Even if we could read it out, it is not indexed which
* is required for fetch 'changes-since-x'.
* So instead we have to rely on a custom user defined field
* that contains the server time
* which is set by firestore via serverTimestamp()
* Notice that the serverTimestampField MUST NOT be
* part of the collections RxJsonSchema!
* [default='serverTimestamp']
*/
serverTimestampField: 'serverTimestamp'
});To observe and cancel the replication, you can use any other methods from the ReplicationState like error$, cancel() and awaitInitialReplication().
Handling deletes
RxDB requires you to never fully delete documents. This is needed to be able to replicate the deletion state of a document to other instances. The firestore replication will set a boolean _deleted field to all documents to indicate the deletion state. You can change this by setting a different deletedField in the sync options.
Do not set `enableIndexedDbPersistence()`
Firestore has the enableIndexedDbPersistence() feature which caches document states locally to IndexedDB. This is not needed when you replicate your Firestore with RxDB because RxDB itself will store the data locally already.
Using the replication with an already existing Firestore Database State
If you have not used RxDB before and you already have documents inside of your Firestore database, you have
to manually set the _deleted field to false and the serverTimestamp to all existing documents.
import {
getDocs,
query,
where,
serverTimestamp
} from 'firebase/firestore';
const allDocsResult = await getDocs(query(firestoreCollection));
allDocsResult.forEach(doc => {
doc.update({
_deleted: false,
serverTimestamp: serverTimestamp()
})
});Also notice that if you do writes from non-RxDB applications, you have to keep these fields in sync. It is recommended to use the Firestore triggers to ensure that.
Filtered Replication
You might need to replicate only a subset of your collection, either to or from Firestore. You can achieve this using push.filter and pull.filter options.
const replicationState = replicateFirestore(
{
collection: myRxCollection,
firestore: {
projectId,
database: firestoreDatabase,
collection: firestoreCollection
},
pull: {
filter: [
where('ownerId', '==', userId)
]
},
push: {
filter: (item) => item.syncEnabled === true
}
}
);Keep in mind that you can not use inequality operators (<, <=, !=, not-in, >, or >=) in pull.filter since that would cause a conflict with ordering by serverTimestamp.
Docs Src Docs Replication Google Drive (`docs-src/docs/replication-google-drive.md`)
title: Google Drive Sync slug: replication-google-drive.html image: /headers/replication-google-drive.jpg
import {Steps} from '@site/src/components/steps';
import {BetaBlock} from '@site/src/components/beta-block';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
Replication with Google Drive
The replication-google-drive plugin allows you to replicate your client-side RxDB database to a folder in the user's Google Drive. This enables cross-device sync for single users without requiring any backend server.
Overview
The replication uses the Google Drive API v3 and v2.
- Offline-First: Users can work offline. Changes are synced when they go online.
- No Backend Required: You don't need to host your own database server.
- Cross-Device: Users can access their data from multiple devices by signing into the same Google account.
- Realtime Sync: Uses WebRTC for peer-to-peer signaling to achieve near real-time updates. Uses the same google-drive folder instead of a signaling-server.
Usage
Enable Google Drive API
You need to enable the Google Drive API in the Google Cloud Console and create credentials (OAuth 2.0 Client ID) for your application.
Authenticate the User
Your application must handle the OAuth flow to get an accessToken from Google. You can use libraries like @react-oauth/google or the Google Identity Services SDK.
Start Replication
Once you have the accessToken, you can start the replication.
import { replicateGoogleDrive } from 'rxdb/plugins/replication-google-drive';
const replicationState = await replicateGoogleDrive({
replicationIdentifier: 'my-app-drive-sync',
collection: myDatabase.todos, // RxCollection
googleDrive: {
oauthClientId: 'YOUR_GOOGLE_CLIENT_ID',
authToken: 'USER_ACCESS_TOKEN',
folderPath: 'my-app-data/user-1'
},
live: true,
pull: {
batchSize: 60,
modifier: doc => doc // (optional) modify invalid data
},
push: {
batchSize: 60,
modifier: doc => doc // (optional) modify before sending
}
});
// Observe replication states
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
replicationState.awaitInitialReplication().then(() => {
console.log('Initial replication done');
});Signaling & WebRTC
Google Drive does not provide real-time events for file changes. If a user changes data on User Device A, User Device B would not know about it until it periodically polls the Drive API.
To achieve real-time updates, this plugin uses WebRTC to signal changes between connected devices.
- Devices create "signal files" in a
signalingsubfolder on Google Drive. - Other devices detect these files, read the WebRTC connection data, and establish a direct P2P connection with each other.
- When a device makes a write, it sends a "RESYNC" signal via WebRTC to all connected peers to notify them about the change.
Polyfill for Node.js
WebRTC is native in browsers but requires a polyfill in Node.js. Use createSimplePeerWrtc() to wrap the polyfill for compatibility with simple-peer:
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
// ...
const replicationState = await replicateGoogleDrive({
// ...
signalingOptions: {
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill)
}
});Options
googleDrive
- oauthClientId
string: The OAuth 2.0 Client ID of your application. - authToken
string: The valid access token associated with the user. - folderPath
string: The path to the folder in Google Drive where data should be stored.- The plugin will ensure this folder exists.
- For the default
drivespace it must not be the root folder. - For the
appDataFolderspace it is optional and interpreted relative to the appDataFolder root. - It creates subfolders
docs(for data) andsignaling(for WebRTC).
- space
'drive' | 'appDataFolder'(optional): Defaults to'drive'. See the section below. - apiEndpoint
string(optional): Defaults tohttps://www.googleapis.com. Useful for mocking or proxies. - transactionTimeout
number(optional): Default10000(10s). The plugin uses atransactionfile in Drive to ensure data integrity during writes. This is the timeout after which a lock is considered stale.
Using the appDataFolder
By default the plugin stores data in the user visible "My Drive". Set space: 'appDataFolder' to store data in Google Drive's hidden, per-application data folder instead. This is useful on Android and other clients where you want app state synced across the user's devices without cluttering their Drive UI.
const replicationState = await replicateGoogleDrive({
replicationIdentifier: 'my-app-drive-sync',
collection: myDatabase.todos,
googleDrive: {
oauthClientId: 'YOUR_GOOGLE_CLIENT_ID',
authToken: 'USER_ACCESS_TOKEN',
space: 'appDataFolder'
// folderPath is optional here
},
live: true,
pull: {},
push: {}
});When using appDataFolder:
- Request the
https://www.googleapis.com/auth/drive.appdataOAuth scope when you authenticate the user. The regular Drive scopes do not grant access to this space. - The data is hidden from the user's Drive UI and cannot be browsed manually.
- The folder is isolated per OAuth client id. A debug build and a release build with different client ids will not see each other's data.
folderPathis optional. If omitted, thedocsandsignalingsubfolders are created directly in the appDataFolder root.
attachments
Controls whether binary attachment data is replicated along with documents.
- Default: enabled automatically when the collection schema has
attachments: {}defined. - Set
attachments: falseto disable attachment replication (only document fields are synced).
const replicationState = await replicateGoogleDrive({
// ...
attachments: false, // disable attachment replication
pull: {},
push: {}
});When attachment replication is enabled, attachment binary data is encoded as base64 and stored in a separate _attachments_data field inside the document's JSON file on Drive. The standard _attachments field only contains metadata stubs (digest, length, type). On pull, the base64 data is decoded back to Blob instances and written to local storage.
pull & push
Standard RxDB Replication Options for batch size, modifiers, etc.
Technical Details
File Mapping
- Each RxDB document corresponds to one JSON file in the
docssubfolder. - The filename is
[primaryKey].json. - This simple mapping makes it easy to inspect or backup data manually.
Checkpointing
- The replication relies on the
modifiedTimeof files in Google Drive.
Conflict Resolution
- Conflicts are handled using the standard RxDB conflict handling strategies.
- The plugin assumes a master-slave replication pattern where the client (RxDB) merges changes.
- If the
transactionfile is locked by another device, the write retries until the lock is released or times out.
Limitations
- Rate Limits: Google Drive API has strict rate limits. The plugin attempts to handle 429 errors with exponential backoff, but heavy concurrent writes might hit these limits.
- Latency: Changes take time to propagate and appear in listings (eventual consistency), which the plugin handles internally.
- Signaling Delay: The initial WebRTC handshake requires writing and reading files from Drive, which can take a few seconds. Once connected, signaling is instant.
Testing
For testing, it is recommended to use google-drive-mock. It simulates the Google Drive API so you can run tests without real credentials.
FAQ
Google Drive API rate limiting is based on quota units, not a flat request count. According to the official Google Drive API usage limits documentation, the default limits are:
- 1,000,000 quota units per minute per project
- 325,000 quota units per minute per user (within a project)
So the HTTP requests/minute depends on the endpoint cost:
- If an endpoint costs 1 quota unit/request, the theoretical maximum is 1,000,000 req/min/project and 325,000 req/min/user.
- If an endpoint costs 100 quota units/request (for example
files.listin many setups), that is about 10,000 req/min/project and 3,250 req/min/user.
When limits are exceeded, Google can return 403 (User rate limit exceeded) or 429 (Rate limit exceeded) responses.
The replication plugin already retries with exponential backoff, but for high-traffic apps you should monitor quota usage and tune your sync frequency/batch sizes.
- Official limits: https://developers.google.com/workspace/drive/api/guides/limits
- Error handling guidance: https://developers.google.com/workspace/drive/api/guides/handle-errors
Docs Src Docs Replication Graphql (`docs-src/docs/replication-graphql.md`)
title: GraphQL Replication slug: replication-graphql.html image: /headers/replication-graphql.jpg
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Replication with GraphQL
The GraphQL replication provides handlers for GraphQL to run replication with GraphQL as the transport layer.
The GraphQL replication is mostly used when you already have a backend that exposes a GraphQL API that can be adjusted to serve as a replication endpoint. If you do not already have a GraphQL endpoint, using the HTTP replication is an easier solution.
:::note
To play around, check out the full example of the RxDB GraphQL replication with server and client
:::
Usage
Before you use the GraphQL replication, make sure you've learned how the RxDB replication works.
Creating a compatible GraphQL Server
At the server-side, there must exist an endpoint which returns newer rows when the last checkpoint is used as input. For example lets say you create a Query pullHuman which returns a list of document writes that happened after the given checkpoint.
For the push-replication, you also need a Mutation pushHuman which lets RxDB update data of documents by sending the previous document state and the new client document state.
Also for being able to stream all ongoing events, we need a Subscription called streamHuman.
input HumanInput {
id: ID!,
name: String!,
lastName: String!,
updatedAt: Float!,
deleted: Boolean!
}
type Human {
id: ID!,
name: String!,
lastName: String!,
updatedAt: Float!,
deleted: Boolean!
}
input Checkpoint {
id: String!,
updatedAt: Float!
}
type HumanPullBulk {
documents: [Human]!
checkpoint: Checkpoint
}
type Query {
pullHuman(checkpoint: Checkpoint, limit: Int!): HumanPullBulk!
}
input HumanInputPushRow {
assumedMasterState: HeroInputPushRowT0AssumedMasterStateT0
newDocumentState: HeroInputPushRowT0NewDocumentStateT0!
}
type Mutation {
# Returns a list of all conflicts
# If no document write caused a conflict, return an empty list.
pushHuman(rows: [HumanInputPushRow!]): [Human]
}
# headers are used to authenticate the subscriptions
# over websockets.
input Headers {
AUTH_TOKEN: String!;
}
type Subscription {
streamHuman(headers: Headers): HumanPullBulk!
}The GraphQL resolver for the pullHuman would then look like:
/* Detailed source-code truncated for AI context efficiency. */For examples for the other resolvers, consult the GraphQL Example Project.
RxDB Client
Pull replication
For the pull-replication, you first need a pullQueryBuilder. This is a function that gets the last replication checkpoint and a limit as input and returns an object with a GraphQL-query and its variables (or a promise that resolves to the same object). RxDB will use the query builder to construct what is later sent to the GraphQL endpoint.
const pullQueryBuilder = (checkpoint, limit) => {
/**
* The first pull does not have a checkpoint
* so we fill it up with defaults
*/
if (!checkpoint) {
checkpoint = {
id: '',
updatedAt: 0
};
}
const query = `query PullHuman($checkpoint: CheckpointInput, $limit: Int!) {
pullHuman(checkpoint: $checkpoint, limit: $limit) {
documents {
id
name
age
updatedAt
deleted
}
checkpoint {
id
updatedAt
}
}
}`;
return {
query,
operationName: 'PullHuman',
variables: {
checkpoint,
limit
}
};
};With the queryBuilder, you can then setup the pull-replication.
/* Detailed source-code truncated for AI context efficiency. */Push replication
For the push-replication, you also need a queryBuilder. Here, the builder receives a changed document as input which has to be sent to the server. It also returns a GraphQL-Query and its data.
const pushQueryBuilder = rows => {
const query = `
mutation PushHuman($writeRows: [HumanInputPushRow!]) {
pushHuman(writeRows: $writeRows) {
id
name
age
updatedAt
deleted
}
}
`;
const variables = {
writeRows: rows
};
return {
query,
operationName: 'PushHuman',
variables
};
};With the queryBuilder, you can then setup the push-replication.
const replicationState = replicateGraphQL(
{
collection: myRxCollection,
// urls to the GraphQL endpoints
url: {
http: 'http://example.com/graphql'
},
push: {
queryBuilder: pushQueryBuilder, // the queryBuilder from above
/**
* batchSize (optional)
* Amount of document that will be pushed
* to the server in a single request.
*/
batchSize: 5,
/**
* modifier (optional)
* Modifies all pushed documents before
* they are sent to the GraphQL endpoint.
* Returning null will skip the document.
*/
modifier: doc => doc
},
headers: {
Authorization: 'Bearer abcde...'
},
pull: {
/* ... */
},
/* ... */
}
);Pull Stream
To create a realtime replication, you need to create a pull stream that pulls ongoing writes from the server.
The pull stream gets the headers of the RxReplicationState as input, so that it can be authenticated on the backend.
const pullStreamQueryBuilder = (headers) => {
const query = `subscription onStream($headers: Headers) {
streamHero(headers: $headers) {
documents {
id,
name,
age,
updatedAt,
deleted
},
checkpoint {
id
updatedAt
}
}
}`;
return {
query,
variables: {
headers
}
};
};With the pullStreamQueryBuilder you can then start a realtime replication.
/* Detailed source-code truncated for AI context efficiency. */:::note
If it is not possible to create a websocket server on your backend, you can use any other method to pull out the ongoing events from the backend and then you can send them into RxReplicationState.emitEvent().
:::
Transforming null to undefined in optional fields
GraphQL fills up non-existent optional values with null while RxDB required them to be undefined.
Therefore, if your schema contains optional properties, you have to transform the pulled data to switch out null to undefined
const replicationState: RxGraphQLReplicationState<RxDocType> = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {/* ... */},
pull: {
queryBuilder: pullQueryBuilder,
modifier: (doc => {
// We have to remove optional non-existent field values
// they are set as null by GraphQL but should be undefined
Object.entries(doc).forEach(([k, v]) => {
if (v === null) {
delete doc[k];
}
});
return doc;
})
},
/* ... */
}
);pull.responseModifier
With the pull.responseModifier you can modify the whole response from the GraphQL endpoint before it is processed by RxDB.
For example if your endpoint is not capable of returning a valid checkpoint, but instead only returns the plain document array, you can use the responseModifier to aggregate the checkpoint from the returned documents.
import {
} from 'rxdb';
const replicationState: RxGraphQLReplicationState<RxDocType> = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {/* ... */},
pull: {
responseModifier: async function(
plainResponse, // the exact response that was returned from the server
// either 'handler' if plainResponse
// came from the pull.handler,
// or 'stream' if it came from
// the pull.stream
origin,
// if origin==='handler', the
// requestCheckpoint contains the
// checkpoint that was sent to
// the backend
requestCheckpoint
) {
/**
* In this example we aggregate the
* checkpoint from the documents array
* that was returned from the graphql endpoint.
*/
const docs = plainResponse;
return {
documents: docs,
checkpoint: docs.length === 0 ? requestCheckpoint : {
name: lastOfArray(docs).name,
updatedAt: lastOfArray(docs).updatedAt
}
};
}
},
/* ... */
}
);push.responseModifier
It's also possible to modify the response of a push mutation. For example if your server returns more than just the conflicting docs:
type PushResponse {
conflicts: [Human]
conflictMessages: [ReplicationConflictMessage]
}
type Mutation {
# Returns a PushResponse type that contains
# the conflicts along with other information
pushHuman(rows: [HumanInputPushRow!]): PushResponse!
}import {} from "rxdb";
const replicationState: RxGraphQLReplicationState<RxDocType> = replicateGraphQL(
{
collection: myRxCollection,
url: {/* ... */},
headers: {/* ... */},
push: {
responseModifier: async function (plainResponse) {
/**
* In this example we aggregate the
* conflicting documents from a
* response object
*/
return plainResponse.conflicts;
},
},
pull: {/* ... */},
/* ... */
}
);Helper Functions
RxDB provides the helper functions graphQLSchemaFromRxSchema(), pullQueryBuilderFromRxSchema(), pullStreamBuilderFromRxSchema() and pushQueryBuilderFromRxSchema() that can be used to generate handlers and schemas from the RxJsonSchema. To learn how to use them, please inspect the GraphQL Example.
RxGraphQLReplicationState
When you call myCollection.syncGraphQL() it returns a RxGraphQLReplicationState which can be used to subscribe to events, for debugging or other functions. It extends the RxReplicationState with some GraphQL specific methods.
.setHeaders()
Changes the headers for the replication after it has been set up.
replicationState.setHeaders({
Authorization: `...`
});Sending Cookies
The underlying fetch framework uses a same-origin policy for credentials by default. That means, cookies and session data is only shared if you backend and frontend run on the same domain and port. Pass the credential parameter to include cookies in requests to servers from different origins via:
replicationState.setCredentials('include');or directly pass it in the replicateGraphQL function:
replicateGraphQL(
{
collection: myRxCollection,
/* ... */
credentials: 'include',
/* ... */
}
);See the fetch spec for more information about available options.
:::note
To play around, check out the full example of the RxDB GraphQL replication with server and client
:::
Docs Src Docs Replication Http (`docs-src/docs/replication-http.md`)
title: HTTP Replication slug: replication-http.html image: /headers/replication-http.jpg
import {Steps} from '@site/src/components/steps';
import {Tabs} from '@site/src/components/tabs';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
HTTP Replication from a custom server to RxDB clients
While RxDB has a range of backend-specific replication plugins (like GraphQL or Firestore), the replication is built in a way to make it very easy to replicate data from a custom server to RxDB clients.
Using HTTP as a transport protocol makes it simple to create a compatible backend on top of your existing infrastructure. For events that must be sent from the server to the client, we can use Server-Sent Events.
In this tutorial we will implement a HTTP replication between an RxDB client and a MongoDB express server. You can adapt this for any other backend database technology like PostgreSQL or even a non-Node.js server like go or java.
To create a compatible server for replication, we will start a server and implement the correct HTTP routes and replication handlers. We need a push-handler, a pull-handler and for the ongoing changes pull.stream we use Server-Sent Events.
Setup
Start the Replication on the RxDB Client
RxDB does not have a specific HTTP-replication plugin because the replication primitives plugin is simple enough to start a HTTP replication on top of it.
We import the replicateRxCollection function and start the replication from there for a single RxCollection.
// > client.ts
import { replicateRxCollection } from 'rxdb/plugins/replication';
const replicationState = await replicateRxCollection({
collection: myRxCollection,
replicationIdentifier: 'my-http-replication',
push: { /* add settings from below */ },
pull: { /* add settings from below */ }
});Start a Node.js process with Express and MongoDB
On the server side, we start an express server that has a MongoDB connection and serves the HTTP requests of the client.
// > server.ts
import { MongoClient } from 'mongodb';
import express from 'express';
const mongoClient = new MongoClient('mongodb://localhost:27017/');
const mongoConnection = await mongoClient.connect();
const mongoDatabase = mongoConnection.db('myDatabase');
const mongoCollection = await mongoDatabase.collection('myDocs');
const app = express();
app.use(express.json());
/* ... add routes from below */
app.listen(80, () => {
console.log(`Example app listening on port 80`)
});Implement the Pull Endpoint
As first HTTP Endpoint, we need to implement the pull handler. This is used by the RxDB replication to fetch all documents writes that happened after a given checkpoint.
The checkpoint format is not determined by RxDB, instead the server can use any type of changepoint that can be used to iterate across document writes. Here we will just use a unix timestamp updatedAt and a string id which is the most common used format.
When the pull endpoint is called, the server responds with an array of document data based on the given checkpoint and a new checkpoint.
Also the server has to respect the batchSize so that RxDB knows when there are no more new documents and the server returns a non-full array.
// > server.ts
import { lastOfArray } from 'rxdb/plugins/core';
app.get('/pull', async (req, res) => {
const id = req.query.id;
const updatedAt = parseFloat(req.query.updatedAt);
const documents = await mongoCollection.find({
$or: [
/**
* Notice that we have to compare the updatedAt AND the id field
* because the updateAt field is not unique and when two documents
* have the same updateAt, we can still "sort" them by their id.
*/
{
updatedAt: { $gt: updatedAt }
},
{
updatedAt: { $eq: updatedAt },
id: { $gt: id }
}
]
})
.sort({updatedAt: 1, id: 1})
.limit(parseInt(req.query.batchSize, 10)).toArray();
const newCheckpoint = documents.length === 0 ? { id, updatedAt } : {
id: lastOfArray(documents).id,
updatedAt: lastOfArray(documents).updatedAt
};
res.setHeader('Content-Type', 'application/json');
res.end(JSON.stringify({ documents, checkpoint: newCheckpoint }));
});Implement the Pull Handler
On the client we add the pull.handler to the replication setting. The handler requests the correct server url and fetches the documents.
// > client.ts
const replicationState = await replicateRxCollection({
/* ... */
pull: {
async handler(checkpointOrNull, batchSize){
const updatedAt = checkpointOrNull ? checkpointOrNull.updatedAt : 0;
const id = checkpointOrNull ? checkpointOrNull.id : '';
const url = 'https://localhost/pull'
+ `?updatedAt=${updatedAt}`
+ `&id=${id}`
+ `&limit=${batchSize}`;
const response = await fetch(url);
const data = await response.json();
return {
documents: data.documents,
checkpoint: data.checkpoint
};
}
}
/* ... */
});Implement the Push Endpoint
To send client side writes to the server, we have to implement the push.handler. It gets an array of change rows as input and has to return only the conflicting documents that have not been written to the server. Each change row contains a newDocumentState and an optional assumedMasterState.
For conflict detection, on the server we first have to detect if the assumedMasterState is correct for each row. If yes, we have to write the new document state to the database, otherwise we have to return the "real" master state in the conflict array.
The server also creates an event that is emitted to the pullStream$ which is later used in the pull.stream$.
/* Detailed source-code truncated for AI context efficiency. */:::note
For simplicity in this tutorial, we do not use transactions. In reality you should run the full push function inside of a MongoDB transaction to ensure that no other process can mix up the document state while the writes are processed. Also you should call batch operations on MongoDB instead of running the operations for each change row.
:::
Implement the Push Handler
With the push endpoint in place, we can add a push.handler to the replication settings on the client.
// > client.ts
const replicationState = await replicateRxCollection({
/* ... */
push: {
async handler(changeRows){
const rawResponse = await fetch('https://localhost/push', {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: JSON.stringify(changeRows)
});
const conflictsArray = await rawResponse.json();
return conflictsArray;
}
}
/* ... */
});Implement the pullStream$ Endpoint
While the normal pull handler is used when the replication is in iteration mode, we also need a stream of ongoing changes when the replication is in event observation mode. This brings the realtime replication to RxDB where changes on the server or on a client will directly get propagated to the other instances.
On the server we have to implement the pullStream route and emit the events. We use the pullStream$ observable from above to fetch all ongoing events and respond them to the client. Here we use Server-Sent-Events (SSE) which is the most commonly used way to stream data from the server to the client. Other method also exist like WebSockets or Long-Polling.
// > server.ts
app.get('/pullStream', (req, res) => {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache'
});
const subscription = pullStream$.subscribe(event => {
res.write('data: ' + JSON.stringify(event) + '\n\n');
});
req.on('close', () => subscription.unsubscribe());
});:::note
How to build the pullStream$ Observable is not part of this tutorial. This heavily depends on your backend and infrastructure. Likely you have to observe the MongoDB event stream.
:::
Implement the pullStream$ Handler
From the client we can observe this endpoint and create a pull.stream$ observable that emits all events that are sent from the server to the client.
The client connects to an url and receives server-sent-events that contain all ongoing writes.
// > client.ts
import { Subject } from 'rxjs';
const myPullStream$ = new Subject();
const eventSource = new EventSource(
'http://localhost/pullStream',
{ withCredentials: true }
);
eventSource.onmessage = event => {
const eventData = JSON.parse(event.data);
myPullStream$.next({
documents: eventData.documents,
checkpoint: eventData.checkpoint
});
};
const replicationState = await replicateRxCollection({
/* ... */
pull: {
/* ... */
stream$: myPullStream$.asObservable()
}
/* ... */
});pullStream$ RESYNC flag
In case the client loses the connection, the EventSource will automatically reconnect but there might have been some changes that have been missed out in the meantime. The replication has to be informed that it might have missed events by emitting a RESYNC flag from the pull.stream$.
The replication will then catch up by switching to the iteration mode until it is in sync with the server again.
// > client.ts
eventSource.onerror = () => myPullStream$.next('RESYNC');The purpose of the RESYNC flag is to tell the client that "something might have changed" and then the client can react on that information without having to run operations in an interval.
If your backend is not capable of emitting the actual documents and checkpoint in the pull stream, you could just map all events to the RESYNC flag. This would make the replication work with a slight performance drawback:
// > client.ts
import { Subject } from 'rxjs';
const myPullStream$ = new Subject();
const eventSource = new EventSource(
'http://localhost/pullStream',
{ withCredentials: true }
);
eventSource.onmessage = () => myPullStream$.next('RESYNC');
const replicationState = await replicateRxCollection({
pull: {
stream$: myPullStream$.asObservable()
}
});Missing implementation details
In this tutorial we only covered the basics of doing a HTTP replication between RxDB clients and a server. We did not cover the following aspects of the implementation:
- Authentication: To authenticate the client on the server, you might want to send authentication headers with the HTTP requests
- Skip events on the
pull.stream$for the client that caused the changes to improve performance. - Version upgrades: You should add a version-flag to the endpoint urls. If you then update the version of your endpoints in any way, your old endpoints should emit a
Code 426to outdated clients so that they can update their client version.
Docs Src Docs Replication Microsoft Onedrive (`docs-src/docs/replication-microsoft-onedrive.md`)
title: Microsoft OneDrive Sync slug: replication-microsoft-onedrive.html image: /headers/replication-microsoft-onedrive.jpg
import {Steps} from '@site/src/components/steps';
import {BetaBlock} from '@site/src/components/beta-block';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Replication with Microsoft OneDrive
The replication-microsoft-onedrive plugin allows you to replicate your client-side RxDB database to a folder in the user's Microsoft OneDrive. This enables cross-device sync for single users without requiring any backend server.
Overview
The replication uses the Microsoft Graph API.
- Offline-First: Users can work offline. Changes are synced when they go online.
- No Backend Required: You don't need to host your own database server.
- Cross-Device: Users can access their data from multiple devices by signing into the same Microsoft account.
- Realtime Sync: Uses WebRTC for peer-to-peer signaling to achieve near real-time updates. Uses the same onedrive folder instead of a signaling-server.
Usage
Enable Microsoft Graph API
You need to register your application in the Azure portal and create credentials (OAuth 2.0 Client ID) with Files.ReadWrite permissions for your application.
Authenticate the User
Your application must handle the OAuth flow to get an accessToken from Microsoft. You can use libraries like @azure/msal-browser or @azure/msal-react.
Start Replication
Once you have the accessToken, you can start the replication.
import {
replicateMicrosoftOneDrive
} from 'rxdb/plugins/replication-microsoft-onedrive';
const replicationState = await replicateMicrosoftOneDrive({
replicationIdentifier: 'my-app-onedrive-sync',
collection: myDatabase.todos, // RxCollection
oneDrive: {
authToken: 'USER_ACCESS_TOKEN',
folderPath: 'my-app-data/user-1'
},
live: true,
pull: {
batchSize: 60,
modifier: doc => doc // (optional) modify invalid data
},
push: {
batchSize: 60,
modifier: doc => doc // (optional) modify before sending
}
});
// Observe replication states
replicationState.error$.subscribe(err => {
console.error('Replication error:', err);
});
replicationState.awaitInitialReplication().then(() => {
console.log('Initial replication done');
});Signaling & WebRTC
Microsoft OneDrive does not provide real-time events for file changes that a client can easily subscribe to in the browser. If a user changes data on User Device A, User Device B would not know about it until it periodically polls the API.
To achieve real-time updates, this plugin uses WebRTC to signal changes between connected devices.
- Devices create "signal files" in a
signalingsubfolder on OneDrive. - Other devices detect these files, read the WebRTC connection data, and establish a direct P2P connection with each other.
- When a device makes a write, it sends a "RESYNC" signal via WebRTC to all connected peers to notify them about the change.
Polyfill for Node.js
WebRTC is native in browsers but requires a polyfill in Node.js. Use createSimplePeerWrtc() to wrap the polyfill for compatibility with simple-peer:
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
// ...
const replicationState = await replicateMicrosoftOneDrive({
// ...
signalingOptions: {
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill)
}
});Options
oneDrive
- authToken
string: The valid access token associated with the user. - folderPath
string: The path to the folder in Microsoft OneDrive where data should be stored.- The plugin will ensure this folder exists.
- It must not be the root folder.
- It creates subfolders
docs(for data) andsignaling(for WebRTC).
- apiEndpoint
string(optional): Defaults tohttps://graph.microsoft.com/v1.0/me/drive. Useful for mocking or proxies. - transactionTimeout
number(optional): Default10000(10s). The plugin uses atransaction.jsonfile in OneDrive to ensure data integrity during writes. This is the timeout after which a lock is considered stale.
pull & push
Standard RxDB Replication Options for batch size, modifiers, etc.
Technical Details
File Mapping
- Each RxDB document corresponds to one JSON file in the
docssubfolder. - The filename is
[primaryKey].json. - This simple mapping makes it easy to inspect or backup data manually.
Checkpointing
- The replication relies on the
lastModifiedDateTimeof files in Microsoft OneDrive.
Conflict Resolution
- Conflicts are handled using the standard RxDB conflict handling strategies.
- The plugin assumes a master-slave replication pattern where the client (RxDB) merges changes.
- If the
transaction.jsonfile is locked by another device, the write retries until the lock is released or times out.
Limitations
- Rate Limits: Microsoft Graph API has strict rate limits. The plugin attempts to handle 429 errors with exponential backoff, but heavy concurrent writes might hit these limits.
- Latency: Changes take time to propagate and appear in listings (eventual consistency), which the plugin handles internally.
- Signaling Delay: The initial WebRTC handshake requires writing and reading files from OneDrive, which can take a few seconds. Once connected, signaling is instant.
Testing
For testing, it is recommended to use microsoft-onedrive-mock. It simulates the Microsoft Graph API so you can run tests without real credentials.
Docs Src Docs Replication Mongodb (`docs-src/docs/replication-mongodb.md`)
title: MongoDB Realtime Sync Engine for Local-First Apps slug: replication-mongodb.html image: /headers/replication-mongodb.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {SyncDiagram} from '@site/src/components/sync-diagram';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
MongoDB Replication Plugin
The MongoDB Replication Plugin for RxDB delivers seamless, two-way synchronization between MongoDB and RxDB, enabling real-time updates and offline-first functionality for your applications. Built on MongoDB Change Streams, it supports both Atlas and self-hosted deployments, ensuring your data stays consistent across every device and service.
Behind the scenes, the plugin is powered by the RxDB Sync Engine, which manages the complexities of real-world data replication for you. It automatically handles conflict detection and resolution, maintains precise checkpoints for incremental updates, and gracefully manages transitions between offline and online states. This means you don't need to manually implement retry logic, reconcile divergent changes, or worry about data loss during connectivity drops, the Sync Engine ensures consistency and reliability in every sync cycle.
Key Features
- Two-way replication between MongoDB and RxDB collections
- Offline-first support with automatic incremental re-sync
- Incremental updates via MongoDB Change Streams
- Conflict resolution handled by the RxDB Sync Engine
- Atlas and self-hosted support for replica sets and sharded clusters
Architecture Overview
The plugin operates in a three-tier architecture: Clients connect to RxServer, which in turn connects to MongoDB. RxServer streams changes from MongoDB to connected clients and pushes client-side updates back to MongoDB.
For the client side, RxServer exposes a replication endpoint over WebSocket or HTTP, which your RxDB-powered applications can consume.
The following diagram illustrates the flow of updates between clients, RxServer, and MongoDB in a live synchronization setup:
<SyncDiagram backend={{ label: "MongoDB", icon: "/files/icons/mongodb-icon.svg" }} clientConnectionLabel="HTTP" backendConnectionLabel="Change Stream" simulateOffline />
:::note
The MongoDB Replication Plugin is optimized for Node.js environments (e.g., when RxDB runs within RxServer or other backend services). Direct connections from browsers or mobile apps to MongoDB are not supported because MongoDB does not use HTTP as its wire protocol and requires a driver-level connection to a replica set or sharded cluster.
:::
Setting up the Client-RxServer-MongoDB Sync
Install the Client Dependencies
In your JavaScript project, install the RxDB libraries and the MongoDB node.js driver:
npm install rxdb rxdb-server mongodb --save
Set up a MongoDB Server
As first step, you need access to a running
MongoDB Server. This can be done by either
running a server locally or using the Atlas
Cloud. Notice that we need to have a
replica set
because only on these, the MongoDB changestream
can be used.
Shell
If you have installed MongoDB locally, you can start the server with this command:
mongod --replSet rs0 --bind_ip_all
Docker
If you have docker installed, you can start a container that runs the MongoDB server:
docker run -p 27017:27017 -p 27018:27018 -p 27019:27019 --rm --name rxdb-mongodb mongo:8.0.4 mongod --replSet rs0 --bind_ip_all
MongoDB Atlas
Learn here how to create a MongoDB atlas
account and how to start a MongoDB cluster
that runs in the cloud:
<VideoBox
videoId="bBA9rUdqmgY"
title="Create MongoDB Atlas Server"
duration="19:55"
/>After this step you should have a valid
connection string that points to a running
MongoDB Server likemongodb://localhost:27017/.
Create a MongoDB Database and Collection
On your MongoDB server, make sure to create a database and a collection.
//> server.ts
import { MongoClient } from 'mongodb';
const mongoClient = new MongoClient(
'mongodb://localhost:27017/?directConnection=true'
);
const mongoDatabase = mongoClient.db('my-database');
await mongoDatabase.createCollection('my-collection', {
changeStreamPreAndPostImages: { enabled: true }
});:::note
To observe document deletions on the
changestream, changeStreamPreAndPostImages
must be enabled. This is not required if you
have an insert/update-only collection where
no documents are deleted ever.
:::
Create a RxDB Database and Collection
Now we create an RxDB
database and a
collection. In this
example the
memory storage,
in production you would use a
persistent storage instead.
//> server.ts
import { createRxDatabase, addRxPlugin } from 'rxdb';
import { getRxStorageMemory } from 'rxdb/plugins/storage-memory';
// Create server-side RxDB instance
const db = await createRxDatabase({
name: 'serverdb',
storage: getRxStorageMemory()
});
// Add your collection schema
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean', default: false }
},
required: ['id', 'name', 'done']
}
}
});Sync the Collection with the MongoDB Server
Now we can start a
replication that does a
two-way replication between the RxDB
Collection and the MongoDB Collection.
//> server.ts
import { replicateMongoDB } from 'rxdb/plugins/replication-mongodb';
const replicationState = replicateMongoDB({
mongodb: {
collectionName: 'my-collection',
connection: 'mongodb://localhost:27017',
databaseName: 'my-database'
},
collection: db.todos,
replicationIdentifier: 'todos-mongodb-sync',
pull: { batchSize: 50 },
push: { batchSize: 50 },
live: true
});:::note You can do many things with the replication state
The RxMongoDBReplicationState which is
returned from replicateMongoDB() allows
you to run all functionality of the normal
RxReplicationState like
observing errors or doing start/stop
operations.
:::
Start a RxServer
Now that we have a RxDatabase and Collection
that is replicated with MongoDB, we can
spawn a RxServer on top
of it. This server can then be used by
client devices to connect.
//> server.ts
import { createRxServer } from 'rxdb-server/plugins/server';
import { RxServerAdapterExpress } from 'rxdb-server/plugins/adapter-express';
const server = await createRxServer({
database: db,
adapter: RxServerAdapterExpress,
port: 8080,
cors: '*'
});
const endpoint = server.addReplicationEndpoint({
name: 'todos',
collection: db.todos
});
console.log('Replication endpoint:', `http://localhost:8080/${endpoint.urlPath}`);
// do not forget to start the server!
await server.start();Sync a Client with the RxServer
On the client-side we create the exact same
RxDatabase and collection and then replicate
it with the replication endpoint of the
RxServer.
//> client.ts
import { createRxDatabase } from 'rxdb';
import { getRxStorageDexie } from 'rxdb/plugins/storage-dexie';
import { replicateServer } from 'rxdb-server/plugins/replication-server';
const db = await createRxDatabase({
name: 'mydb-client',
storage: getRxStorageDexie()
});
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean', default: false }
},
required: ['id', 'name', 'done']
}
}
});
// Start replication to the RxServer endpoint printed by the server:
// e.g. http://localhost:8080/todos/0
const replicationState = replicateServer({
replicationIdentifier: 'todos-rxserver',
collection: db.todos,
url: 'http://localhost:8080/todos/0',
live: true,
pull: { batchSize: 50 },
push: { batchSize: 50 }
});Follow Up
- Try it out with the RxDB-MongoDB
example repository - Read From Local to Global: Scalable
Edge Apps with RxDB + MongoDB
- Replication API Reference
- RxServer Documentation
- Join our Discord Forum for questions and feedback
Docs Src Docs Replication Nats (`docs-src/docs/replication-nats.md`)
title: RxDB & NATS - Realtime Sync slug: replication-nats.html image: /headers/replication-nats.jpg
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
Replication with NATS
With this RxDB plugin you can run a two-way realtime replication with a NATS server.
The replication itself uses the RxDB Sync Engine which handles conflicts, errors and retries.
On the client side the official NATS npm package is used to connect to the NATS server.
NATS is a messaging system that by itself does not have a validation or granulary access control build in.
Therefore it is not recommended to directly replicate the NATS server with an untrusted RxDB client application. Instead you should replicated from NATS to your Node.js server side RxDB database.
Precondition
For the replication endpoint the NATS cluster must have enabled JetStream and store all message data as structured JSON.
The easiest way to start a compatible NATS server is to use the official docker image:
docker run --rm --name rxdb-nats -p 4222:4222 nats:2.9.17 -js
Usage
Install the nats package
npm install nats --saveStart the Replication
To start the replication, import thereplicateNats() method from the RxDB
plugin and call it with the collection
that must be replicated.
The replication runs
per RxCollection,
you can replicate multiple RxCollections by
starting a new replication for each of them.
import {
replicateNats
} from 'rxdb/plugins/replication-nats';
const replicationState = replicateNats({
collection: myRxCollection,
replicationIdentifier: 'my-nats-replication-collection-A',
// in NATS, each stream need a name
streamName: 'stream-for-replication-A',
/**
* The subject prefix determines how the documents are stored in NATS.
* For example the document with id 'alice'
* will have the subject 'foobar.alice'
*/
subjectPrefix: 'foobar',
connection: { servers: 'localhost:4222' },
live: true,
pull: {
batchSize: 30
},
push: {
batchSize: 30
}
});Handling deletes
RxDB requires you to never
fully delete documents.
This is needed to be able to replicate the
deletion state of a document to other
instances. The NATS replication will set a
boolean _deleted field to all documents to
indicate the deletion state. You can change
this by setting a different deletedField
in the sync options.
Docs Src Docs Replication P2p (`docs-src/docs/replication-p2p.md`)
title: Seamless P2P Data Sync slug: replication-p2p.html image: /headers/replication-p2p.jpg
The RxDB Plugin `replication-p2p` has been renamed to `replication-webrtc`
The new documentation page has been moved to here
Docs Src Docs Replication Server (`docs-src/docs/replication-server.md`)
title: RxDB Server Replication slug: replication-server.html image: /headers/replication-server.jpg
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
RxDB Server Replication
The Server Replication Plugin connects to the replication endpoint of an RxDB Server Replication Endpoint and replicates data between the client and the server.
Usage
The replication server plugin is imported from the rxdb-server npm package. Then you start the replication with a given collection and endpoint url by calling replicateServer().
import { replicateServer } from 'rxdb-server/plugins/replication-server';
const replicationState = await replicateServer({
collection: usersCollection,
replicationIdentifier: 'my-server-replication',
// endpoint url with the servers collection
// schema version at the end
url: 'http://localhost:80/users/0',
headers: {
Authorization: 'Bearer S0VLU0UhI...'
},
push: {},
pull: {},
live: true
});outdatedClient$
When you update your schema at the server and run a migration, you end up with a different replication url that has a new schema version number at the end.
Your clients might still be running an old version of your application that will no longer be compatible with the endpoint. Therefore when the client tries to call a server endpoint with an outdated schema version, the outdatedClient$ observable emits to tell your client that the application must be updated. With that event you can tell the client to update the application.
On browser application you might want to just reload the page on that event:
replicationState.outdatedClient$.subscribe(() => {
location.reload();
});unauthorized$
When you clients auth data is not valid (or no longer valid), the server will no longer accept any requests from you client and inform the client that the auth headers must be updated.
The unauthorized$ observable will emit and expects you to update the headers accordingly so that following requests will be accepted again.
replicationState.unauthorized$.subscribe(() => {
replicationState.setHeaders({
Authorization: 'Bearer S0VLU0UhI...'
});
});forbidden$
When you client behaves wrong in any case, like update non-allowed values or changing documents that it is not allowed to,
the server will drop the connection and the replication state will emit on the forbidden$ observable.
It will also automatically stop the replication so that your client does not accidentally DOS attack the server.
replicationState.forbidden$.subscribe(() => {
console.log('Client is behaving wrong');
});Custom EventSource implementation
For the server send events, the eventsource npm package is used instead of the native EventSource API. We need this because the native browser API does not support sending headers with the request which is required by the server to parse the auth data.
If the eventsource package does not work for you, you can set an own implementation when creating the replication.
const replicationState = await replicateServer({
/* ... */
eventSource: MyEventSourceConstructor
/* ... */
});Docs Src Docs Replication Supabase (`docs-src/docs/replication-supabase.md`)
title: Supabase Replication Plugin for RxDB - Real-Time, Offline-First Sync slug: replication-supabase.html image: /headers/replication-supabase.jpg
import {Tabs} from '@site/src/components/tabs';
import {Steps} from '@site/src/components/steps';
import {VideoBox} from '@site/src/components/video-box';
import {SyncDiagram} from '@site/src/components/sync-diagram';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
Supabase Replication Plugin
The Supabase Replication Plugin for RxDB delivers seamless, two-way synchronization between your RxDB collections and a Supabase (Postgres) table. It uses PostgREST for pull/push and Supabase Realtime (logical replication) to stream live updates, so your data stays consistent across devices with first-class local-first, offline-ready support.
Under the hood, the plugin is powered by the RxDB Sync Engine. It handles checkpointed incremental pulls, robust retry logic, and conflict detection/resolution for you. You focus on features, and RxDB takes care of sync.
<VideoBox videoId="zBZgdTb-dns" title="Supabase in 100 Seconds" duration="2:36" />Key Features of the RxDB-Supabase Plugin
- Cloud Only Backend: No self-hosted server required. Client devices directly sync with the Supabase Servers.
- Two-way replication between Supabase tables and RxDB collections
- Offline-first with resumable, incremental sync
- Live updates via Supabase Realtime channels
- Conflict resolution handled by the RxDB Sync Engine
- Works in browsers and Node.js with
@supabase/supabase-js
Architecture Overview
<SyncDiagram server={false} backend={{ label: "Supabase", icon: "/files/icons/supabase.svg" }} clientConnectionLabel="PostgREST + Realtime" simulateOffline />
Clients connect directly to Supabase using the official JS client. The plugin:
- Pulls documents over PostgREST using a checkpoint
(modified, id)and deterministic ordering. - Pushes inserts/updates using optimistic concurrency guards.
- Streams new changes using Supabase Realtime so live replication stays up to date.
:::note
Because Supabase exposes Postgres over HTTP/WebSocket, you can safely replicate from browsers and mobile apps. Protect your data with Row Level Security (RLS) policies; use the anon key on clients and the service role key only on trusted servers.
:::
Setting up RxDB โ Supabase Sync
Install Dependencies
npm install rxdb @supabase/supabase-jsCreate a Supabase Project & Table
In your supabase project, create a new table. Ensure that:
- The primary key must have the type text (Primary keys must always be strings in RxDB)
- You have an modified field which stores the last modification timestamp of a row (default is
_modified) - You have a boolean field which stores if a row should is "deleted". You should not hard-delete rows in Supabase, because clients would miss the deletion if they haven't been online at the deletion time. Instead, use a deleted
booleanto mark rows as deleted. This way all clients can still pull the deletion, and RxDB will hide the complexity on the client side. - Enable the realtime observation of writes to the table.
Here is an example for a "todos" table:
create extension if not exists moddatetime schema extensions;
create table "public"."todos" (
"id" text primary key,
"name" text not null,
"done" boolean DEFAULT false NOT NULL,
"note" text,
"_deleted" boolean DEFAULT false NOT NULL,
"_modified" timestamp with time zone DEFAULT now() NOT NULL
);
-- auto-update the _modified timestamp
CREATE TRIGGER update_modified_datetime BEFORE UPDATE ON public.todos FOR EACH ROW
EXECUTE FUNCTION extensions.moddatetime('_modified');
-- add a table to the publication so we can subscribe to changes
alter publication supabase_realtime add table "public"."todos";Create an RxDB Database & Collection
Create a normal RxDB database, then add a collection whose schema mirrors your Supabase table. The primary key must match (same column name and type), and fields should be top-level simple types (string/number/boolean). You donโt need to model server internals: the plugin maps the serverโs _deleted flag to doc._deleted automatically, and _modified is optional in your schema (the plugin strips it on push and will include it on pull only if you define it). For browsers use a persistent storage like Localstorage or IndexedDB. For tests you can use the in-memory storage.
// client
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
export const db = await createRxDatabase({
name: 'mydb',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
todos: {
schema: {
version: 0,
primaryKey: 'id',
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean', default: false },
note: { type: 'string' }
},
required: ['id', 'name', 'done']
}
}
});Create the Supabase Client
Make a single Supabase client and reuse it across your app. In the browser, use the anon key (RLS-protected). On trusted servers you may use the service role key, but never ship that to clients.
Production
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
'https://xyzcompany.supabase.co',
'eyJhbGciOi...'
);Vite
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
import.meta.env.VITE_SUPABASE_URL!, // e.g. https://xyzcompany.supabase.co
import.meta.env.VITE_SUPABASE_ANON_KEY! // anon key for browsers
// optional options object here
);Local Development
//> client
import { createClient } from '@supabase/supabase-js';
export const supabase = createClient(
'http://127.0.0.1:54321',
'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'
);Start Replication
Connect your RxDB collection to the Supabase table to start the replication.
//> client
import { replicateSupabase } from 'rxdb/plugins/replication-supabase';
const replication = replicateSupabase({
tableName: 'todos',
client: supabase,
collection: db.todos,
replicationIdentifier: 'todos-supabase',
live: true,
pull: {
batchSize: 50,
// optional: shape incoming docs
modifier: (doc) => {
// map nullable note-field
if (doc.note === null) delete doc.note;
return doc;
},
// optional: customize the pull query before fetching
queryBuilder: ({ query }) => {
// Add filters, joins, or other PostgREST query modifiers
// This runs before checkpoint filtering and ordering
return query.eq('done', false);
},
},
push: {
batchSize: 50
},
// optional overrides if your column names differ:
// modifiedField: '_modified',
// deletedField: '_deleted'
});
// (optional) observe errors and wait for the first sync barrier
replication.error$.subscribe(err => console.error('[replication]', err));
await replication.awaitInitialReplication();:::note Nullable values must be mapped
Supabase returns null for nullable columns, but in RxDB you often model those fields as optional (i.e., they can be undefined/missing). To avoid schema errors, map null โ undefined in the pull.modifier (usually by deleting the key).
:::
Using Joins
You can use the pull.queryBuilder to use joins and also pull data from related tables.
To do that, you have to create a new query object in the pull.queryBuilder with the .select() method and return it.
const replication = replicateSupabase({
pull: {
queryBuilder: (/* ignore the passed query instance from here */) => {
/**
* Create a totally new query instance
* and return that.
*/
return supabase.from('todos').select('*, comments(*)');
}
}
});Do other things with the replication state
The RxSupabaseReplicationState which is returned from replicateSupabase() allows you to run all functionality of the normal RxReplicationState.
FAQ
Supabase and RxDB offer the best of both worlds for Node.js and TypeScript applications. Supabase provides a powerful PostgreSQL backend. You can use PostgreSQL as a document database by storing JSON data. RxDB provides a reactive local document store. You achieve real-time synchronization between the local document store and the Supabase backend. You benefit from strong TypeScript typing on the client and robust SQL querying on the server.
You connect an anonymous key to a Supabase project by initializing the official @supabase/supabase-js client utilizing your project's SUPABASE_URL and SUPABASE_ANON_KEY. In frontend applications interacting with the RxDB Supabase Replication plugin, you must inject the anon key, while simultaneously configuring strict Row Level Security (RLS) policies within your Supabase PostgreSQL backend to prevent unauthorized data manipulation.
Natively, the Supabase JavaScript client does not support advanced offline-first synchronization pipelines or complex Conflict-free Replicated Data Type (CRDT) architectures. To implement full offline sync capable of continuous background disconnected writes, you must attach the RxDB Supabase Replication Plugin. RxDB acts as the offline-first local CRDT-like cache, deferring all local mutations into a unified outbound queue until the Supabase TCP connection is restored.
Yes, Row Level Security (RLS) is strictly mandatory whenever you expose a Supabase database directly to the frontend. Without RLS, the anonymous anon key used by the RxDB client grants full read and write access to your entire PostgreSQL cluster. You must configure RLS policies that enforce auth.uid() = user_id checks to guarantee clients only replicate and mutate their own specific documents.
Supabase Realtime acts as an Elixir-based WebSocket broadcasting server that taps directly into PostgreSQL's logical replication stream. When a row changes on the database, the Realtime server parses the WAL (Write-Ahead Log) and pushes the event down to subscribed clients. The RxDB Supabase Replication plugin leverages this WebSocket channel strictly for live change detection, triggering rapid localized pulls over PostgREST to guarantee no data is dropped during connection turbulence.
Follow Up
- Replication API Reference: Learn the core concepts and lifecycle hooks - Replication
- Offline-First Guide: Caching, retries, and conflict strategies - Local-First
- Supabase Crash Course: Build a React Native app with RxDB - Webinar Video
- Row Level Security (RLS) - https://supabase.com/docs/guides/auth/row-level-security
- Realtime - https://supabase.com/docs/guides/realtime
- Local dev with the Supabase CLI - https://supabase.com/docs/guides/cli
- Community: Questions or feedback? Join our Discord - Chat
Docs Src Docs Replication Webrtc (`docs-src/docs/replication-webrtc.md`)
title: WebRTC P2P Replication with RxDB - Sync Browsers and Devices slug: replication-webrtc.html image: /headers/replication-webrtc.jpg
import {Steps} from '@site/src/components/steps';
import {HeadlineWithIcon} from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
import {CenteredImage} from '@site/src/components/centered-image';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {P2pReplicationMesh} from '@site/src/components/animations/p2p-replication-mesh';
P2P WebRTC Replication with RxDB
WebRTC P2P data connections are revolutionizing real-time web and mobile development by eliminating central servers in scenarios where clients can communicate directly. With the RxDB Sync Engine, you can sync your local database state across multiple browsers or devices via WebRTC P2P (Peer-to-Peer) connections, ensuring scalable, secure, and low-latency data flows without traditional server bottlenecks.
What is WebRTC?
WebRTC stands for Web Real-Time Communication. It is an open standard that enables browsers and native apps to exchange audio, video, or arbitrary data directly between peers, bypassing a central server after the initial connection is established. WebRTC uses NAT traversal techniques like ICE (Interactive Connectivity Establishment) to punch through firewalls and establish direct links. This peer-to-peer nature drastically reduces latency while maintaining high security and end-to-end encryption capabilities.
For a deeper look at comparing WebRTC with WebSockets and WebTransport, you can read our comprehensive overview. While WebSockets or WebTransport often work in client-server contexts, WebRTC offers direct peer-to-peer connections ideal for fully decentralized data flows.
<a href="https://webrtc.org/" target="_blank">
</a>Benefits of P2P Sync with WebRTC Compared to Client-Server Architecture
- Reduced Latency - By skipping a central server hop, data travels directly from one client to another, minimizing round-trip times and improving responsiveness.
- Scalability - New peers can join without overloading a central infrastructure. The sync overhead increases linearly with the number of connections rather than requiring a massive server cluster.
- Privacy & Ownership - Data stays within the userโs devices, avoiding risks tied to storing data on third-party servers. This design aligns well with local-first or "zero-latency" apps.
- Resilience - In some scenarios, if the central server is unreachable, P2P connections remain operational (assuming a functioning signaling path). Apps can still replicate data among local networks like when they are in the same Wifi or LAN. Notice that the peers still have to find each other through a signaling server, so this is not a full offline mesh. For DDIL environments without any infrastructure at all, compare the transports in the Ditto alternative page.
- Cost Savings - Reducing the reliance on a high-bandwidth server can cut hosting and bandwidth expenses, particularly in high-traffic or IoT-style use cases.
Peer-to-Peer (P2P) WebRTC Replication with the RxDB JavaScript Database
Traditionally, real-time data synchronization depends on centralized servers to manage and distribute updates. In contrast, RxDBโs WebRTC P2P replication allows data to flow directly among clients, removing the server as a data store. This approach is live and fully decentralized, requiring only a signaling server for initial discovery:
- No master-slave concept - each peer hosts its own local RxDB.
- Clients (browsers, devices) connect to each other via WebRTC data channels.
- The RxDB replication protocol then handles pushing/pulling document changes across peers.
Because RxDB is a NoSQL database and the replication protocol is straightforward, setting up robust P2P sync is far easier than orchestrating a complex client-server database architecture.
Using RxDB with the WebRTC Replication Plugin
Before you use this plugin, make sure that you understand how WebRTC works. Here we build a todo-app that replicates todo-entries between clients:
You can find a fully build example of this at the RxDB Quickstart Repository which you can also try out online.
First you create the database and then you can configure the replication:
Create the Database and Collection
Here we create a database with the localstorage based storage that stores data inside of the LocalStorage API in a browser. RxDB has a wide range of storages for other JavaScript runtimes.
import { createRxDatabase } from 'rxdb/plugins/core';
import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage';
const db = await createRxDatabase({
name: 'myTodoDB',
storage: getRxStorageLocalstorage()
});
await db.addCollections({
todos: {
schema: {
title: 'todo schema',
version: 0,
type: 'object',
primaryKey: 'id',
properties: {
id: { type: 'string', maxLength: 100 },
name: { type: 'string' },
done: { type: 'boolean', default: false },
dueAt: { type: 'string', format: 'date-time', maxLength: 30 }
},
required: ['id', 'name', 'done', 'dueAt']
}
}
});
// insert an example document
await db.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
done: false,
dueAt: '2026-01-01T10:00:00.000Z'
});Import the WebRTC replication plugin
import {
replicateWebRTC,
getConnectionHandlerSimplePeer
} from 'rxdb/plugins/replication-webrtc';Start the P2P replication
To start the replication you have to call replicateWebRTC on the collection.
As options you have to provide a topic and a connection handler function that implements the P2PConnectionHandlerCreator interface. As default you should start with the getConnectionHandlerSimplePeer method which uses the simple-peer library and comes shipped with RxDB.
/* Detailed source-code truncated for AI context efficiency. */Notice that in difference to the other replication plugins, the WebRTC replication returns a replicationPool instead of a single RxReplicationState. The replicationPool contains all replication states of the connected peers in the P2P network.
Observe Errors
To ensure we log out potential errors, observe the error$ observable of the pool.
replicationPool.error$.subscribe(err => console.error('WebRTC Error:', err));Stop the Replication
You can also dynamically stop the replication.
replicationPool.cancel();Live replications
The WebRTC replication is always live because there can not be a one-time sync when it is always possible to have new Peers that join the connection pool. Therefore you cannot set the live: false option like in the other replication plugins.
Signaling Server
For P2P replication to work with the RxDB WebRTC Replication Plugin, a signaling server is required. The signaling server helps peers discover each other and establish connections.
RxDB ships with a default signaling server that can be used with the simple-peer connection handler. This server is made for demonstration purposes and tryouts. It is not reliable and might be offline at any time.
In production you must always use your own signaling server instead!
Creating a basic signaling server is straightforward. The provided example uses 'socket.io' for WebSocket communication. However, in production, you'd want to create a more robust signaling server with authentication and additional logic to suit your application's needs.
Here is a quick example implementation of a signaling server that can be used with the connection handler from getConnectionHandlerSimplePeer():
import {
startSignalingServerSimplePeer
} from 'rxdb/plugins/replication-webrtc';
const serverState = await startSignalingServerSimplePeer({
port: 8080 // <- port
});For custom signaling servers with more complex logic, you can check the source code of the default one.
Signaling over Nostr Relays
Instead of running your own signaling server, you can let the peers find each other over Nostr relays. Nostr is an open protocol where clients publish signed JSON events to relays, which are plain WebSocket servers. There are many public relays, and you can run your own with open source software like strfry or nostr-rs-relay.
The connection handler from getConnectionHandlerNostr() sends the WebRTC offers, answers, and ICE candidates as Nostr events. After the peers are connected, the replicated documents go directly over the WebRTC data channel and never touch a relay.
import {
replicateWebRTC,
getConnectionHandlerNostr
} from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC(
{
collection: myRxCollection,
topic: 'my-users-pool',
connectionHandlerCreator: getConnectionHandlerNostr({
// Public relays, all peers of a topic must share at least one.
relays: [
'wss://nos.lol',
'wss://relay.primal.net',
'wss://nostr.mom'
],
/**
* (optional) Nostr secret key (32 bytes) of this peer.
* If not set, a random key is generated on each start.
*/
// secretKey: mySecretKey,
/**
* (optional) kind of the Nostr events that are used for signaling.
* Must be an ephemeral kind between 20000 and 29999.
* [default=25050]
*/
// eventKind: 25050
}),
pull: {},
push: {}
}
);How the signaling works:
- Ephemeral events: All signaling events use an ephemeral kind, so relays forward them to the subscribers but do not store them.
- Hashed topic: The events are tagged with a SHA-256 hash of the
topic, so the topic name is not readable on the relays. - Presence: Each peer publishes a presence event every
10s. A peer that sends no presence for35sis removed from the room. - Encrypted signals: Offers, answers, and ICE candidates are encrypted with NIP-44 to the public key of the receiving peer. Only the receiving peer can read them.
- Signed events: Every event is signed with the key of the sending peer, and events with an invalid signature are ignored. Because the signals that set up the WebRTC connection are signed, the public key of a connected peer is verified.
- Batched signals: Many public relays rate limit how many events a client can send per second. Trickle ICE creates many signals at once, so the signals to the same peer are collected for
200msand sent together in one event. - Multiple relays: The handler connects to all given relays and reconnects each of them with the same backoff as the signaling server connection. Events that arrive from more than one relay are only processed once.
The same secret key can be used on multiple devices or browser tabs at the same time, because each connection handler adds its own random session id to the peer id.
The public relays wss://nos.lol, wss://relay.primal.net, and wss://nostr.mom from the example above were tested with this handler on September 30, 2026. Public relays are run by third parties and can be offline, change their policies, or rate limit your app at any time. For production, run your own relay or pick relays that you trust, and pass more than one relay url.
Peer Validation
By default the replication will replicate with every peer the signaling server tells them about.
You can prevent invalid peers from replication by passing a custom isPeerValid() function that either returns true on valid peers and false on invalid peers.
const replicationPool = await replicateWebRTC(
{
/* ... */
isPeerValid: async (peer) => {
return true;
}
pull: {},
push: {}
/* ... */
}
);With the Nostr connection handler, each peer has a verified Nostr public key. You can use it to only replicate with known peers. Give each device its own secretKey and store the public keys of the allowed devices:
import {
replicateWebRTC,
getConnectionHandlerNostr,
getNostrPublicKeyOfPeer
} from 'rxdb/plugins/replication-webrtc';
import { getPublicKey } from 'nostr-tools/pure';
// hex public keys of the devices that are allowed to replicate
const allowedPublicKeys = [
getPublicKey(mySecretKey),
publicKeyOfOtherDevice
];
const replicationPool = await replicateWebRTC(
{
collection: myRxCollection,
topic: 'my-users-pool',
connectionHandlerCreator: getConnectionHandlerNostr({
relays: ['wss://nos.lol', 'wss://relay.primal.net'],
secretKey: mySecretKey
}),
isPeerValid: (peer) => {
const publicKey = getNostrPublicKeyOfPeer(peer);
return allowedPublicKeys.includes(publicKey);
},
pull: {},
push: {}
}
);Connection Handling and Timeouts
The simple-peer connection handler reconnects on its own when the connection to the signaling server or to another peer breaks. Reconnects run with an exponential backoff that starts at 500ms and is capped at 15s, so that an unreachable signaling server does not cause a busy loop. A peer connection that is not established within 15s is dropped and a new attempt is started. Existing WebRTC connections keep replicating while the signaling server is offline.
Messages that are bigger than the message size limit of the WebRTC data channel (which can be as low as 64 KiB depending on the browser) are split into chunks, so you can replicate big documents.
Each request to another peer fails when no answer arrives in time. The replication then retries after retryTime. You can change the timeout with the requestTimeout option:
const replicationPool = await replicateWebRTC(
{
/* ... */
// (optional) time in milliseconds [default=20000]
requestTimeout: 30000,
pull: {},
push: {}
}
);Conflict detection in WebRTC replication
RxDB's conflict handling works by detecting and resolving conflicts that may arise when multiple clients in a decentralized database system attempt to modify the same data concurrently.
A custom conflict handler can be set up, which is a plain JavaScript function. The conflict handler is run on each replicated document write and resolves the conflict if required. Find out more about RxDB conflict handling here
Known problems
SimplePeer requires to have `process.nextTick()`
In the browser you might not have a process variable or process.nextTick() method. But the simple peer uses that so you have to polyfill it.
In webpack you can use the process/browser package to polyfill it:
const plugins = [
/* ... */
new webpack.ProvidePlugin({
process: 'process/browser',
})
/* ... */
];In angular or other libraries you can add the polyfill manually:
window.process = {
nextTick: (fn, ...args) => setTimeout(() => fn(...args)),
};Polyfill the WebSocket and WebRTC API in Node.js
While all modern browsers support the WebRTC and WebSocket APIs, they is missing in Node.js which will throw the error No WebRTC support: Specify opts.wrtc option in this environment. Therefore you have to polyfill it with a compatible WebRTC and WebSocket polyfill. It is recommended to use the node-datachannel package for WebRTC which does not come with RxDB but has to be installed before via npm install node-datachannel --save.
For the Websocket API use the ws package that is included into RxDB.
Because the node-datachannel/polyfill has read-only properties on RTCSessionDescription that are incompatible with simple-peer, you must use the createSimplePeerWrtc() wrapper:
import nodeDatachannelPolyfill from 'node-datachannel/polyfill';
import { WebSocket } from 'ws';
import { createSimplePeerWrtc } from 'rxdb/plugins/replication-webrtc';
const replicationPool = await replicateWebRTC(
{
/* ... */
connectionHandlerCreator: getConnectionHandlerSimplePeer({
signalingServerUrl: 'wss://example.com:8080',
wrtc: createSimplePeerWrtc(nodeDatachannelPolyfill),
webSocketConstructor: WebSocket
}),
pull: {},
push: {}
/* ... */
}
);Storing replicated data encrypted on client device
Storing replicated data encrypted on client devices using the RxDB Encryption Plugin is a pivotal step towards bolstering data security and user privacy.
The WebRTC replication plugin seamlessly integrates with the RxDB encryption plugins, providing a robust solution for encrypting sensitive information before it's stored locally. By doing so, it ensures that even if unauthorized access to the device occurs, the data remains protected and unintelligible without the encryption key (or password). This approach is particularly vital in scenarios where user-generated content or confidential data is replicated across devices, as it empowers users with control over their own data while adhering to stringent security standards. Read more about the encryption plugins here.
FAQ
WebRTC enables true peer-to-peer (P2P) communication by establishing direct UDP/TCP data channels between browsers, completely bypassing centralized database architectures. Because the WebRTC connection requires initial IP discovery, clients must briefly connect to a WebSocket signaling server or to Nostr relays to exchange SDP offers and ICE candidates. Once peered, the RxDB WebRTC Replication plugin streams NoSQL document diffs and CRDT operations instantly across the channel, providing decentralized real-time sync with absolute zero cloud latency.
RxDB offers comprehensive peer discovery and sync plugins for distributed applications. The WebRTC replication plugin facilitates direct peer-to-peer data synchronization. A signaling server or a set of Nostr relays handles initial peer discovery and connection establishment. You connect browsers and mobile apps without a central database server. The sync engine automatically replicates local changes across all discovered peers.
Very few databases support true decentralized peer-to-peer (P2P) synchronization. RxDB is one of the leading options for this architecture, offering a dedicated WebRTC replication plugin that allows direct, client-to-client data synchronization via WebRTC data channels without routing through a central cloud database. Other notable decentralized tools include Ditto, GunDB, and CRDT-based libraries like Yjs or Automerge (though these are often data structure libraries, not fully queryable databases).
Follow Up
- Check out the RxDB Quickstart to see how to set up your first RxDB database.
- Explore advanced features like Custom Conflict Handling or Offline-First Performance.
- Try an example at RxDB Quickstart GitHub to see a working P2P Sync setup.
- Join the RxDB Community on GitHub or Discord if you have questions or want to share your P2P WebRTC experiences.
Docs Src Docs Replication Websocket (`docs-src/docs/replication-websocket.md`)
title: Websocket Replication slug: replication-websocket.html image: /headers/replication-websocket.jpg
import {Faq, FaqItem} from '@site/src/components/faq';
Websocket Replication
With the websocket replication plugin, you can spawn a websocket server from a RxDB database in Node.js and replicate with it.
:::note
The websocket replication plugin does not have any concept for authentication or permission handling. It is designed to create an easy server-to-server replication. It is not made for client-server replication. Make a pull request if you need that feature.
:::
Starting the Websocket Server
import { createRxDatabase } from 'rxdb';
import {
startWebsocketServer
} from 'rxdb/plugins/replication-websocket';
// create a RxDatabase like normal
const myDatabase = await createRxDatabase({/* ... */});
// start a websocket server
const serverState = await startWebsocketServer({
database: myDatabase,
port: 1337,
path: '/socket'
});
// stop the server
await serverState.close();Connect to the Websocket Server
The replication has to be started once for each collection that you want to replicate.
import {
replicateWithWebsocketServer
} from 'rxdb/plugins/replication-websocket';
// start the replication
const replicationState = await replicateWithWebsocketServer({
/**
* To make the replication work,
* the client collection name must be equal
* to the server collection name.
*/
collection: myRxCollection,
url: 'ws://localhost:1337/socket'
});
// stop the replication
await replicationState.cancel();Customize
We use the ws npm library, so you can use all optional configuration provided by it.
This is especially important to improve performance by opting in of some optional settings.
FAQ
The WebSocket protocol operates over a persistent, full-duplex TCP connection, fundamentally differing from standard HTTP's stateless, unidirectional request-response model. While traditional HTTP requires the client to initiate every exchange and constantly re-send heavy headers, WebSockets remain open indefinitely, allowing the server to push real-time data events down to the client with mere bytes of overhead, which is vital for high-throughput database Replication.
WebSockets should be used when your application demands heavy, bidirectional communication such as live chat, multiplayer gaming, or synchronous database multi-master replication. You should opt for Server-Sent Events (SSE) if your communication is strictly unidirectional (server-to-client) like delivering live sports scores or stock tickers, as SSE works natively over traditional HTTP/1.1 connections without encountering common corporate firewall and proxy blocks.
OpenAI and ChatGPT rely heavily on Server-Sent Events (SSE) rather than WebSockets to stream their generative text responses. Because LLM generation is inherently a unidirectional operation (the server generating and streaming tokens down to the client after an initial prompt), SSE is the perfect fit. It drastically reduces server overhead by utilizing standard HTTP/1.1 connections and avoids the bidirectional complexities, strict statefulness, and proxy-blocking issues commonly associated with persistent WebSocket streams.
Traditional polling requests data at fixed intervals regardless of state changes, wasting bandwidth and battery. Long polling holds the HTTP connection open until the server has new data or a timeout occurs, simulating a push. Unlike WebSockets, which establish a single, continuous, bidirectional TCP stream, both polling methods are strictly unidirectional and incur the overhead of re-establishing TCP handshakes and heavy HTTP headers for every discrete data event.
Scaling WebSockets requires load balancers that support persistent TCP connections and protocol upgrades (like HAProxy, NGINX, or AWS ALB). Because WebSockets are heavily stateful, you must configure sticky sessions (Session Affinity) to ensure a client's continuous stream routes to the exact same backend node. For distributing real-time replication events globally across multiple horizontal backend nodes, integrating a secondary Pub/Sub mechanism (like Redis Pub/Sub) is highly recommended.
No, Service Worker fetch events do not intercept WebSocket connections. The fetch event handler in a Service Worker is strictly designed to intercept standard HTTP/HTTPS requests. If you require offline caching or Request interception for your real-time data stream, you must intercept the initial REST calls or implement a custom sync queue inside the Service Worker, which is an architectural pattern databases like RxDB handle inherently on the client side without relying on Service Workers for WS persistence.
WebSockets are significantly faster (lower latency) because they eliminate the HTTP request/response header overhead for every message, pushing data instantly down an open TCP pipe. However, they are more "expensive" on the server side in terms of memory utilization; each open WebSocket connection consumes a dedicated file descriptor and RAM on the server indefinitely, making massive horizontal scaling more complex and costly compared to completely stateless HTTP polling endpoints.
According to the HTTP/1.1 specification (RFC 2616), browsers traditionally limit clients to a maximum of 6 concurrent connections per domain. This limit is rigidly shared across all open tabs. If a user opens 7 tabs connecting to the same WebSocket or SSE endpoint, the 7th tab will stall indefinitely. To bypass this, sophisticated real-time architectures (like RxDB) utilize Leader Election via the BroadcastChannel API to designate a single tab to maintain the active Socket connection, sharing the data pipeline across all other passive tabs locally.
Docs Src Docs Replication (`docs-src/docs/replication.md`)
title: RxDB realtime Sync Engine for Local-First Apps slug: replication.html image: /headers/replication.jpg
import { IconGear } from '@site/src/components/icons/gear';
import { HeadlineWithIcon } from '@site/src/components/headline-with-icon';
import {Faq, FaqItem} from '@site/src/components/faq';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {ReplicationPushPull} from '@site/src/components/animations/replication-push-pull';
RxDB's realtime Sync Engine for Local-First Apps
The RxDB Sync Engine provides the ability to sync the database state in realtime between the clients and the server.
The backend server does not have to be an RxDB instance; you can build a replication with any infrastructure.
For example you can replicate with a custom GraphQL endpoint or an HTTP server on top of a PostgreSQL or MongoDB database.
The replication is made to support the Local-First paradigm, so that when the client goes offline, the RxDB database can still read and write locally and will continue the replication when the client goes online again.
Design Decisions of the Sync Engine
In contrast to other (server-side) database replication protocols, the RxDB Sync Engine was designed with these goals in mind:
- Easy to Understand: The sync engine works in a simple "git-like" way that is easy to understand for an average developer. You only have to understand how three simple endpoints work.
- Complex Parts are in RxDB, not in the Backend: The complex parts of the Sync Engine, like conflict handling or offline-online switches, are implemented inside of RxDB itself. This makes creating a compatible backend very easy.
- Compatible with any Backend: Because the complex parts are in RxDB, the backend can be "dumb" which makes the protocol compatible to almost every backend. No matter if you use PostgreSQL, MongoDB or anything else.
- Performance is optimized for Client Devices and Browsers: By grouping updates and fetches into batches, it is faster to transfer and easier to compress. Client devices and browsers can also process this data faster, for example running
JSON.parse()on a chunk of data is faster than calling it once per row. Same goes for how client side storage like IndexedDB or OPFS works where writing data in bulks is faster. - Offline-First Support: By incorporating conflict handling at the client side, the protocol fully supports offline-first apps. Users can continue making changes while offline, and those updates will sync seamlessly once a connection is reestablished - all without risking data loss or having undefined behavior.
- Multi-Tab Support: When RxDB is used in a browser and multiple tabs of the same application are opened, only exactly one runs the replication at any given time. This reduces client- and backend resources.
The Sync Engine on the document level
On the RxDocument level, the replication works like git, where the fork/client contains all new writes and must be merged with the master/server before it can push its new state to the master/server.
A---B-----------D master/server state
\ /
B---C---D fork/client state- The client pulls the latest state
Bfrom the master. - The client does some changes
C+D. - The client pushes these changes to the master by sending the latest known master state
Band the new client stateDof the document. - If the master state is equal to the latest master
Bstate of the client, the new client stateDis set as the latest master state. - If the master also had changes and so the latest master change is different than the one that the client assumes, we have a conflict that has to be resolved on the client.
The Sync Engine on the transfer level
When document states are transferred, all handlers use batches of documents for better performance.
The server must implement the following methods to be compatible with the replication:
- pullHandler Get the last checkpoint (or null) as input. Returns all documents that have been written after the given checkpoint. Also returns the checkpoint of the latest written returned document.
- pushHandler a method that can be called by the client to send client-side writes to the master. It gets an array with the
assumedMasterStateand thenewForkStateof each document write as input. It must return an array that contains the master document states of all conflicts. If there are no conflicts, it must return an empty array. - pullStream an observable that emits batches of all master writes and the latest checkpoint of the write batches.
+--------+ +--------+
| | pullHandler() | |
| |---------------------> | |
| | | |
| | | |
| Client | pushHandler() | Server |
| |---------------------> | |
| | | |
| | pullStream$ | |
| | <-------------------------| |
+--------+ +--------+The replication runs in two different modes:
Checkpoint iteration
On first initial replication, or when the client comes online again, a checkpoint based iteration is used to catch up with the server state.
A checkpoint is a subset of the fields of the last pulled document. When the checkpoint is sent to the backend via pullHandler(), the backend must be able to respond with all documents that have been written after the given checkpoint.
For example if your documents contain an id and an updatedAt field, these two can be used as checkpoint.
When the checkpoint iteration reaches the last checkpoint, where the backend returns an empty array because there are no newer documents, the replication will automatically switch to the event observation mode.
Event observation
While the client is connected to the backend, the events from the backend are observed via pullStream$ and persisted to the client.
If your backend for any reason is not able to provide a full pullStream$ that contains all events and the checkpoint, you can instead only emit RESYNC events that tell RxDB that anything unknown has changed on the server and it should run the pull replication via checkpoint iteration.
When the client goes offline and online again, it might happen that the pullStream$ has missed out some events. Therefore the pullStream$ should also emit a RESYNC event each time the client reconnects, so that the client can become in sync with the backend via the checkpoint iteration mode.
Data layout on the server
To use the replication you first have to ensure that:
documents are deterministically sortable by their last write time
deterministic means that even if two documents have the same last write time, they have a predictable sort order.
This is most often ensured by using the primaryKey as second sort parameter as part of the checkpoint.documents are never deleted, instead the
_deletedfield is set totrue.This is needed so that the deletion state of a document exists in the database and can be replicated to other instances. If your backend uses a different field to mark deleted documents, you have to transform the data in the push/pull handlers or with the modifiers.
For example if your documents look like this:
const docData = {
"id": "todo1",
"name": "Learn RxDB",
"done": false,
/**
* Contains the last write timestamp
* so all document writes can be sorted by that value
* when they are fetched from the remote instance.
*/
"updatedAt": 1564483474,
/**
* Instead of physically deleting documents,
* a deleted document gets replicated.
*/
"_deleted": false
}Then your data is always sortable by updatedAt. This ensures that when RxDB fetches 'new' changes via pullHandler(), it can send the latest updatedAt+id checkpoint to the remote endpoint and then receive all newer documents.
By default, the field is _deleted. If your remote endpoint uses a different field to mark deleted documents, you can set the deletedField in the replication options which will automatically map the field on all pull and push requests.
Conflict handling
When multiple clients (or the server) modify the same document at the same time (or when they are offline), it can happen that a conflict arises during the replication.
A---B1---C1---X master/server state
\ /
B1---C2 fork/client stateIn the case above, the client would tell the master to move the document state from B1 to C2 by calling pushHandler(). But because the actual master state is C1 and not B1, the master would reject the write by sending back the actual master state C1.
RxDB resolves all conflicts on the client so it would call the conflict handler of the RxCollection and create a new document state D that can then be written to the master.
A---B1---C1---X---D master/server state
\ / \ /
B1---C2---D fork/client stateThe default conflict handler will always drop the fork state and use the master state. This ensures that clients that are offline for a very long time, do not accidentally overwrite other peoples changes when they go online again.
You can specify a custom conflict handler by setting the property conflictHandler when calling addCollection().
Learn how to create a custom conflict handler.
replicateRxCollection()
You can start the replication of a single RxCollection by calling replicateRxCollection() like in the following:
/* Detailed source-code truncated for AI context efficiency. */Multi Tab support
For better performance, the replication runs only in one instance when RxDB is used in multiple browser tabs or Node.js processes.
By setting waitForLeadership: false you can enforce that each tab runs its own replication cycles.
If used in a multi instance setting, so when at database creation multiInstance: false was not set,
you need to import the leader election plugin so that RxDB can know how many instances exist and which browser tab should run the replication.
Error handling
When sending a document to the remote fails for any reason, RxDB will send it again in a later point in time.
This happens for all errors. The document write could have already reached the remote instance and be processed, while only the answering fails.
The remote instance must be designed to handle this properly and to not crash on duplicate data transmissions.
Depending on your use case, it might be ok to just write the duplicate document data again.
But for a more resilient error handling you could compare the last write timestamps or add a unique write id field to the document. This field can then be used to detect duplicates and ignore re-sent data.
Also the replication has an .error$ stream that emits all RxError objects that arise during replication.
Notice that these errors contain an inner .parameters.errors field that contains the original error. Also they contain a .parameters.direction field that indicates if the error was thrown during pull or push. You can use these to properly handle errors. For example when the client is outdated, the server might respond with a 426 Upgrade Required error code that can then be used to force a page reload.
replicationState.error$.subscribe((error) => {
if(
error.parameters.errors &&
error.parameters.errors[0] &&
error.parameters.errors[0].code === 426
) {
// client is outdated -> enforce a page reload
location.reload();
}
});Security
Be aware that client side clocks can never be trusted. When you have a client-backend replication, the backend should overwrite the updatedAt timestamp or use another field, when it receives the change from the client.
RxReplicationState
The function replicateRxCollection() returns a RxReplicationState that can be used to manage and observe the replication.
Observable
To observe the replication, the RxReplicationState has some Observable properties:
// emits each document that was received from the remote
myRxReplicationState.received$.subscribe(doc => console.dir(doc));
// emits each document that was sent to the remote
myRxReplicationState.sent$.subscribe(doc => console.dir(doc));
// emits all errors that happen when running the push- & pull-handlers.
myRxReplicationState.error$.subscribe(error => console.dir(error));
// emits true when the replication was canceled, false when not.
myRxReplicationState.canceled$.subscribe(bool => console.dir(bool));
// emits true when a replication cycle is running, false when not.
myRxReplicationState.active$.subscribe(bool => console.dir(bool));
// emits each conflict that was reported by the remote in the response
// of the push handler, together with the output of the conflictHandler
// that resolved it.
myRxReplicationState.conflict$.subscribe(conflict => console.dir(conflict));awaitInitialReplication()
With awaitInitialReplication() you can await the initial replication that is done when a full replication cycle was successfully finished for the first time. The returned promise will never resolve if you cancel the replication before the initial replication can be done.
The promise resolves only when both directions have finished their first cycle:
- Downstream (pull): all documents that the pull handler returns, up to the latest checkpoint, are written to the local collection.
- Upstream (push): all local writes that existed when the replication started are sent to the push handler. When the push handler reports conflicts, they are resolved and the upstream runs again before it counts as done.
When only one direction is configured, the other direction completes instantly. A pull-only replication resolves as soon as the pull is done, and a push-only replication resolves as soon as the push is done.
await myRxReplicationState.awaitInitialReplication();awaitInSync()
Returns a Promise that resolves when:
awaitInitialReplication()has emitted.- All local data is replicated with the remote.
- No replication cycle is running or in retry-state.
:::warning
When multiInstance: true and waitForLeadership: true and another tab is already running the replication, awaitInSync() will not resolve until the other tab is closed and the replication starts in this tab.
await myRxReplicationState.awaitInSync();:::
:::warning
`awaitInitialReplication()` and `awaitInSync()` should not be used to block the application
A common mistake in RxDB usage is when developers want to block the app usage until the application is in sync.
Often they just await the promise of awaitInitialReplication() or awaitInSync() and show a loading spinner until they resolve. This is dangerous and should not be done because:
- When
multiInstance: trueandwaitForLeadership: true (default)and another tab is already running the replication,awaitInitialReplication()will not resolve until the other tab is closed and the replication starts in this tab. - Your app can no longer be started when the device is offline because there
awaitInitialReplication()will never resolve and the app cannot be used.
Instead you should store the last in-sync time in a local document and observe its value on all instances.
For example if you want to block clients from using the app if they have not been in sync for the last 24 hours, you could use this code:
// update last-in-sync-flag each time replication is in sync
// ensure flag exists
await myCollection.insertLocal(
'last-in-sync', { time: 0 }
).catch();
myReplicationState.active$.pipe(
mergeMap(async() => {
await myReplicationState.awaitInSync();
await myCollection.upsertLocal('last-in-sync', { time: Date.now() })
})
);
// observe the flag and toggle loading spinner
await showLoadingSpinner();
const oneDay = 1000 * 60 * 60 * 24;
await firstValueFrom(
myCollection.getLocal$('last-in-sync').pipe(
filter(d => d.get('time') > (Date.now() - oneDay))
)
);
await hideLoadingSpinner();:::
awaitDocumentPushed()
Returns a Promise that resolves when a specific RxDocument instance was successfully pushed to the server.
While awaitInSync() waits for the whole collection to be in sync, awaitDocumentPushed() only waits for a single document. This is useful when you have a sensitive write (like a financial transaction or a value with a uniqueness constraint) and you want to confirm that exactly this write reached the backend, without blocking on every other unrelated change in the collection.
You pass the RxDocument instance you got back from a write operation:
const doc = await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
dueAt: '2026-01-01T10:00:00.000Z',
priority: 8
});
await myReplicationState.awaitDocumentPushed(doc);
// here we know that the document state was pushed to the serverA RxDocument represents the state of a document at a given point in time, not the document in general. An older or a newer RxDocument instance of the same document is not the exact same RxDocument, because each instance carries the field values and the internal write time of that specific state. awaitDocumentPushed() therefore resolves based on the exact state of the instance you pass in.
It works by comparing the given document state with the last state that was written to the server, which RxDB stores in the replication meta data. Once the given state (or a newer one) was written to the server, the promise resolves.
If the document was overwritten by a newer local write before it could be pushed, the promise resolves as soon as a later state of that document has reached the server.
awaitDocumentPushed() does not set a timeout on purpose. If you need one, combine it with Promise.race():
const doc = await myDatabase.todos.insert({
id: 'todo1',
name: 'Learn RxDB',
dueAt: '2026-01-01T10:00:00.000Z',
priority: 8
});
await Promise.race([
myReplicationState.awaitDocumentPushed(doc),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 5000))
]);reSync()
Triggers a RESYNC cycle where the replication goes into checkpoint iteration until the client is in sync with the backend. Used in unit tests or when no proper pull.stream$ can be implemented so that the client only knows that something has been changed but not what.
myRxReplicationState.reSync();If your backend is not capable of sending events to the client at all, you could run reSync() in an interval so that the client will automatically fetch server changes after some time at least.
// trigger RESYNC each 10 seconds.
setInterval(() => myRxReplicationState.reSync(), 10 * 1000);cancel()
Cancels the replication. Returns a promise that resolves when everything has been cleaned up.
await myRxReplicationState.cancel();A canceled RxReplicationState can never be started again. After cancel(), isStopped() returns true and calling start() rejects with the error RC_START_CANCELED. To resume the replication, call replicateRxCollection() again with the same replicationIdentifier. The new replication state reuses the stored checkpoints and continues where the canceled one stopped, so it does not download all documents again.
await replicationState.cancel();
await replicationState.start(); // throws RC_START_CANCELED
// resume by creating a new replication state with the same identifier
const newReplicationState = replicateRxCollection({
collection: myRxCollection,
replicationIdentifier: 'my-rest-replication-to-https://example.com/api/sync',
pull: { /* ... */ },
push: { /* ... */ }
});Use pause() instead when you want to stop the replication for a while and continue later with the same RxReplicationState.
pause()
Pauses a running replication. The replication can later be resumed with RxReplicationState.start(). In contrast to cancel(), a paused replication keeps its state. When start() is called after a pause, it runs one resync cycle and then continues the live replication.
await myRxReplicationState.pause();
await myRxReplicationState.start(); // restartremove()
Cancels the replication and deletes the metadata of the replication state. This can be used to restart the replication "from scratch". Calling .remove() will only delete the replication metadata, it will NOT delete the documents from the collection of the replication.
await myRxReplicationState.remove();isStopped()
Returns true if the replication is stopped. This can be if a non-live replication is finished or a replication got canceled.
replicationState.isStopped(); // true/falseisPaused()
Returns true if the replication is paused.
replicationState.isPaused(); // true/falseSetting a custom initialCheckpoint
By default, the push replication will start from the beginning of time and push all documents from there to the remote.
By setting a custom push.initialCheckpoint, you can tell the replication to only push writes that are newer than the given checkpoint.
// store the latest checkpoint of a collection
let lastLocalCheckpoint: any;
myCollection.checkpoint$.subscribe(checkpoint => lastLocalCheckpoint = checkpoint);
// start the replication but only push documents
// that are newer than the lastLocalCheckpoint
const replicationState = replicateRxCollection({
collection: myCollection,
replicationIdentifier: 'my-custom-replication-with-init-checkpoint',
/* ... */
push: {
handler: /* ... */,
initialCheckpoint: lastLocalCheckpoint
}
});The same can be done for the other direction by setting a pull.initialCheckpoint. Notice that here we need the remote checkpoint from the backend instead of the one from the RxDB storage.
// get the last pull checkpoint from the server
const lastRemoteCheckpoint = await (
await fetch('http://example.com/pull-checkpoint')
).json();
// start the replication but only pull documents
// that are newer than the lastRemoteCheckpoint
const replicationState = replicateRxCollection({
collection: myCollection,
replicationIdentifier: 'my-custom-replication-with-init-checkpoint',
/* ... */
pull: {
handler: /* ... */,
initialCheckpoint: lastRemoteCheckpoint
}
});toggleOnDocumentVisible
Ensures replication continues running when the document is visible. This helps avoid situations where the leader-elected tab becomes stale or is hibernated by the browser to save battery.
When the tab becomes hidden, replication is automatically paused; when the tab becomes visible again (or the instance becomes leader), replication resumes.
Default: true
const replicationState = replicateRxCollection({
toggleOnDocumentVisible: true,
/* ... */
});Replication Lifecycle
| Method | State afterwards | Can start() resume it? |
Replication metadata |
|---|---|---|---|
pause() |
isPaused() === true |
โ Yes | Kept |
cancel() |
isStopped() === true |
โ No, start() throws RC_START_CANCELED. Call replicateRxCollection() again with the same replicationIdentifier |
Kept, so a new replication continues from the last checkpoint |
remove() |
isStopped() === true |
โ No, start() throws RC_START_CANCELED. Call replicateRxCollection() again |
Deleted, so a new replication starts from scratch |
The replication is also canceled automatically when its RxCollection or RxDatabase is closed.
Using the Replication with TypeScript
replicateRxCollection() and RxReplicationState take two generic types: the document type and the checkpoint type of your backend.
function replicateRxCollection<RxDocType, CheckpointType>(
options: ReplicationOptions<RxDocType, CheckpointType>
): RxReplicationState<RxDocType, CheckpointType>;The handler types are exported from rxdb:
ReplicationPullHandler<RxDocType, CheckpointType>:(lastPulledCheckpoint: CheckpointType | undefined, batchSize: number) => Promise<{ documents: WithDeleted<RxDocType>[], checkpoint: CheckpointType | undefined }>ReplicationPushHandler<RxDocType>:(rows: RxReplicationWriteToMasterRow<RxDocType>[]) => Promise<WithDeleted<RxDocType>[]>. Each row has anewDocumentStateand an optionalassumedMasterState.
When you write a helper function that starts the replication for any collection, take the document type from the RxCollection generic so that the returned RxReplicationState stays typed:
import type {
RxCollection,
ReplicationPullHandler,
ReplicationPushHandler
} from 'rxdb';
import {
replicateRxCollection,
RxReplicationState
} from 'rxdb/plugins/replication';
type MyCheckpoint = { id: string; updatedAt: number; };
export function startReplication<RxDocType>(
collection: RxCollection<RxDocType>,
pullHandler: ReplicationPullHandler<RxDocType, MyCheckpoint>,
pushHandler: ReplicationPushHandler<RxDocType>
): RxReplicationState<RxDocType, MyCheckpoint> {
return replicateRxCollection<RxDocType, MyCheckpoint>({
collection,
replicationIdentifier: 'my-sync-' + collection.name,
live: true,
pull: { handler: pullHandler },
push: { handler: pushHandler }
});
}
// replicationState is typed as RxReplicationState<TodoDocType, MyCheckpoint>
const replicationState = startReplication(
myDatabase.todos,
myPullHandler,
myPushHandler
);Storing Local-Only Metadata Alongside Replicated Documents
Sometimes you want to store data on the client that belongs to a replicated document but must never be sent to the server, for example a "read" flag, a draft text or a UI state. Keep in mind how the replication treats document fields:
- Every field of the document schema is part of the replicated document state. When the server sends a newer version of a document, the local document is replaced with the server state. A field that only exists on the client is lost at that moment.
- The
_metaand_revfields are internal to RxDB. They are stripped before documents are pushed and they are not passed to the conflict handler, which only sees theWithDeleted<RxDocType>data. RxDB writes_metaitself, so you cannot use it for your own data.
There are two ways to store local-only data that survive replication:
1. A separate local collection
Create a second RxCollection that is not replicated and use the same primary key as the replicated document. This keeps the data queryable and reactive, and a remote update of the replicated document does not touch it.
await myDatabase.addCollections({
// replicated with the server
todos: { schema: todoSchema },
// never replicated, stores client-only state per todo
todoLocalState: {
schema: {
version: 0,
primaryKey: 'id', // same value as the todo id
type: 'object',
properties: {
id: { type: 'string', maxLength: 100 },
isRead: { type: 'boolean' },
draftText: { type: 'string' }
},
required: ['id']
}
}
});
await myDatabase.todoLocalState.upsert({ id: 'todo1', isRead: true });2. Local documents
For key-value data that does not need queries, use local documents. They are stored in a separate storage instance, are never replicated and are not part of the collection schema. They require the local-documents plugin and localDocuments: true on the collection.
import { addRxPlugin } from 'rxdb';
import { RxDBLocalDocumentsPlugin } from 'rxdb/plugins/local-documents';
addRxPlugin(RxDBLocalDocumentsPlugin);
await myDatabase.addCollections({
todos: {
schema: todoSchema,
localDocuments: true
}
});
await myDatabase.todos.upsertLocal('ui-state-todo1', { isRead: true });
const localDoc = await myDatabase.todos.getLocal('ui-state-todo1');A schema field that you strip in the push.modifier does not reach the server, but it does not survive a pull either: as soon as the server sends a newer state of the document, the stripped field is gone from the local document. Only use this pattern for data that can be recomputed.
Attachment replication
Attachment replication is supported in the RxDB Sync Engine itself. However not all replication plugins support it.
If you start the replication with a collection which has enabled RxAttachments attachment data will be added to all push- and write data.
The pushed documents will contain an _attachments object which contains:
- The attachment meta data (id, length, digest) of all non-attachments
- The full attachment data of all attachments that have been updated/added from the client.
- Deleted attachments are spared out in the pushed document.
With this data, the backend can decide onto which attachments must be deleted, added or overwritten.
Accordingly, the pulled document must contain the same data, if the backend has a new document state with updated attachments.
Pull-Only Replication
With the replication protocol it is possible to do pull only replications where data is pulled from a backend but not pushed from the client. RxDB implements some performance optimizations for these like not storing server metadata on pull only streams.
Partial Sync with RxDB
RxDB supports partial sync patterns where you dynamically manage multiple replication states for different data scopes. This keeps local storage lean and reduces network overhead. Learn more on the dedicated Partial Sync page.
FAQ
When you have infinite loops in your replication or random re-runs of http requests after some time, the reason is likely that your pull-handler
is crashing. To debug this, add a log to the error$ handler to debug it. `myRxReplicationState.error$.subscribe(err => console.log('error$', err))`.Docs Src Docs Rx Attachment (`docs-src/docs/rx-attachment.md`)
title: Attachments slug: rx-attachment.html image: /headers/rx-attachment.jpg
import { DefaultCompressibleTypes } from '@site/src/components/default-compressible-types';
import {DocsAnimation} from '@site/src/components/docs-animation';
import {AttachmentsMail} from '@site/src/components/animations/attachments-mail';
Attachments
Attachments are binary data files that can be attachment to an RxDocument, like a file that is attached to an email.
Using attachments instead of adding the data to the normal document, ensures that you still have a good performance when querying and writing documents, even when a big amount of data, like an image file has to be stored.
- You can store string, binary files, images and whatever you want side by side with your documents.
- Deleted documents automatically loose all their attachments data.
- Not all replication plugins support the replication of attachments.
- Attachments can be stored encrypted.
Internally, attachment data is stored as Blob objects. Blob is the canonical internal type because it is immutable, carries MIME type metadata via Blob.type, provides synchronous size via Blob.size, and is structured-cloneable (works with Worker/Electron postMessage and IndexedDB). Conversion to ArrayBuffer only happens at system boundaries that require it: encryption (Web Crypto), compression (CompressionStream), digest hashing, and WebSocket serialization.
Add the attachments plugin
To enable the attachments, you have to add the attachments plugin.
import { addRxPlugin } from 'rxdb';
import { RxDBAttachmentsPlugin } from 'rxdb/plugins/attachments';
addRxPlugin(RxDBAttachmentsPlugin);Enable attachments in the schema
Before you can use attachments, you have to ensure that the attachments-object is set in the schema of your RxCollection.
const mySchema = {
version: 0,
type: 'object',
properties: {
// .
// .
// .
},
attachments: {
// if true, the attachment-data will be
// encrypted with the db-password
encrypted: true
}
};
const myCollection = await myDatabase.addCollections({
humans: {
schema: mySchema
}
});putAttachment()
Adds an attachment to a RxDocument. Returns a Promise with the new attachment.
import { createBlob } from 'rxdb';
const attachment = await myDocument.putAttachment(
{
id: 'cat.txt', // (string) name of the attachment
data: createBlob('meowmeow', 'text/plain'), // (Blob) data of the attachment
type: 'text/plain' // (string) type of the attachment
// data like 'image/jpeg'
}
);:::warning
Expo/React-Native does not support the Blob API natively. Make sure you use your own polyfill that properly supports blob.arrayBuffer() when using RxAttachments or use the putAttachmentBase64() and getDataBase64() so that you do not have to create blobs.
:::
putAttachments()
Write multiple attachments to a RxDocument in a single atomic operation. This is more efficient than calling putAttachment() multiple times because it only performs one write to the storage. Returns a Promise with an array of the new attachments.
import { createBlob } from 'rxdb';
const attachments = await myDocument.putAttachments([
{
id: 'cat.txt',
data: createBlob('meowmeow', 'text/plain'),
type: 'text/plain'
},
{
id: 'dog.txt',
data: createBlob('woof', 'text/plain'),
type: 'text/plain'
}
]);putAttachmentBase64()
Same as putAttachment() but accepts a plain base64 string instead of a Blob.
const attachment = await doc.putAttachmentBase64({
id: 'cat.txt',
length: 4,
data: 'bWVvdw==',
type: 'text/plain'
});getAttachment()
Returns an RxAttachment by its id. Returns null when the attachment does not exist.
const attachment = myDocument.getAttachment('cat.jpg');allAttachments()
Returns an array of all attachments of the RxDocument.
const attachments = myDocument.allAttachments();allAttachments$
Gets an Observable which emits a stream of all attachments from the document. Re-emits each time an attachment gets added or removed from the RxDocument.
const all = [];
myDocument.allAttachments$.subscribe(
attachments => all = attachments
);RxAttachment
The attachments of RxDB are represented by the type RxAttachment which has the following attributes/methods.
doc
The RxDocument which the attachment is assigned to.
id
The id as string of the attachment.
type
The type as string of the attachment.
length
The length of the data of the attachment as number.
digest
The hash of the attachments data as string.
:::note
The digest is NOT calculated by RxDB, instead it is calculated by the RxStorage. The only guarantee is that the digest will change when the attachments data changes.
:::
rev
The revision-number of the attachment as number.
remove()
Removes the attachment. Returns a Promise that resolves when done.
const attachment = myDocument.getAttachment('cat.jpg');
await attachment.remove();getData()
Returns a Promise which resolves the attachment's data as Blob. (async)
const attachment = myDocument.getAttachment('cat.jpg');
const blob = await attachment.getData(); // BlobgetDataBase64()
Returns a Promise which resolves the attachment's data as base64 string.
const attachment = myDocument.getAttachment('cat.jpg');
const base64Database = await attachment.getDataBase64(); // 'bWVvdw=='getStringData()
Returns a Promise which resolves the attachment's data as string.
const attachment = await myDocument.getAttachment('cat.jpg');
const data = await attachment.getStringData(); // 'meow'Inline attachments on insert and upsert
Instead of inserting a document first and then calling putAttachment() separately, you can include attachments directly in the document data when using insert(), bulkInsert(), upsert(), bulkUpsert(), or incrementalUpsert(). Provide _attachments as an array of { id, type, data } objects.
import { createBlob } from 'rxdb';
// insert with inline attachments
const doc = await myCollection.insert({
name: 'foo',
_attachments: [
{
id: 'photo.jpg',
type: 'image/jpeg',
data: myJpegBlob
},
{
id: 'notes.txt',
type: 'text/plain',
data: createBlob('some notes', 'text/plain')
}
]
});
const attachment = doc.getAttachment('photo.jpg');Upsert behavior with attachments
When upserting a document that already exists, attachments from the new data are merged with the document's existing attachments by default. This means existing attachments not mentioned in the upsert data are preserved.
To replace all existing attachments instead, pass { deleteExistingAttachments: true } as the second argument:
// Merge (default): keeps existing attachments, adds/updates new ones
const doc = await myCollection.upsert(docData);
// Replace: only the attachments in docData will exist after the upsert
const doc2 = await myCollection.upsert(docData, { deleteExistingAttachments: true });This option works with upsert(), bulkUpsert(), and incrementalUpsert().
Attachment compression {#attachment-compression}
Storing many attachments can be a problem when the disc space of the device is exceeded.
Therefore it can make sense to compress the attachments before storing them in the RxStorage.
With the attachments-compression plugin you can compress the attachments data on write and decompress it on reads.
This happens internally and will not change how you use the API. The compression is run with the Compression Streams API which is only supported on newer browsers.
MIME-type-aware compression
The compression plugin is MIME-type-aware. It only compresses attachment types that benefit from compression (text, JSON, SVG, etc.) and passes through already-compressed formats (JPEG, PNG, MP4, etc.) as-is. This avoids wasting CPU cycles on files that won't shrink.
A built-in default list of compressible types is used automatically. You can override it with the compressibleTypes option in your schema:
import {
wrappedAttachmentsCompressionStorage
} from 'rxdb/plugins/attachments-compression';
import { getRxStorageIndexedDB } from 'rxdb-premium/plugins/storage-indexeddb';
// create a wrapped storage with attachment-compression.
const storageWithAttachmentsCompression = wrappedAttachmentsCompressionStorage({
storage: getRxStorageIndexedDB()
});
const db = await createRxDatabase({
name: 'mydatabase',
storage: storageWithAttachmentsCompression
});
// set the compression mode at the schema level
const mySchema = {
version: 0,
type: 'object',
properties: {
// ..
},
attachments: {
// Specify the compression mode.
// OneOf ['deflate', 'gzip']
compression: 'deflate',
// Optional: override which MIME types get compressed.
// Supports wildcard prefix matching
// (e.g. 'text/*' matches 'text/plain',
// 'text/html', etc.).
// If omitted, a built-in default list of compressible types is used.
compressibleTypes: [
'text/*',
'application/json',
'application/xml',
'image/svg+xml'
// ... add your own patterns
]
}
};
/* ... create your collections as usual and store attachments in them. */The default compressible types include the following MIME type patterns. Binary formats like image/jpeg, image/png, video/*, and audio/* are not in the default list and will be stored without re-compression.
Docs Src Docs Rx Collection (`docs-src/docs/rx-collection.md`)
title: Master Data - Create and Manage RxCollections slug: rx-collection.html image: /headers/rx-collection.jpg
import { NON_PREMIUM_COLLECTION_LIMIT } from '../src/constants';
import {Faq, FaqItem} from '@site/src/components/faq';
RxCollection
A collection stores documents of the same type.
Creating a Collection
To create one or more collections you need an RxDatabase object which has the .addCollections() method. Every collection needs a collection name and a valid RxJsonSchema. Other attributes are optional.
const myCollections = await myDatabase.addCollections({
// key = collectionName
todos: {
schema: todoSchema,
statics: {}, // (optional) ORM-functions
// for this collection
methods: {}, // (optional) ORM-functions for documents
attachments: {}, // (optional) ORM-functions for attachments
options: {}, // (optional) Custom parameters
// that might be used in plugins
migrationStrategies: {}, // (optional)
autoMigrate: true, // (optional) [default=true]
cacheReplacementPolicy: function(){}, // (optional) custom
// cache replacement policy
conflictHandler: function(){} // (optional) custom
// conflict handler
},
// you can create multiple collections at once
projects: {
// ...
}
});:::note
Without the Premium Plugins, RxDB allows up to {NON_PREMIUM_COLLECTION_LIMIT} open collections in parallel. If you hit that limit, see the FAQ on how to remove it.
:::
name
The name uniquely identifies the collection and should be used to refine the collection in the database. Two different collections in the same database can never have the same name. Collection names must match the following regex: ^[a-z][a-z0-9]*$.
schema
The schema defines how the documents of the collection are structured. RxDB uses a schema format, similar to [JSON schema](https://json-