## File: README.md

[Loro](https://loro.dev)

https://github.com/loro-dev/loro/assets/18425020/fe246c47-a120-44b3-91d4-1e7232a5b4ac

✨ Loro 1.0 is out! Read the [announcement](https://loro.dev/blog/v1.0).

Loro is a [CRDTs(Conflict-free Replicated Data Types)](https://crdt.tech/) library that makes building [local-first][local-first] and collaborative apps easier. You can now use it in Rust, JS (via WASM), and Swift. # Features **Features Provided by CRDTs** - P2P Synchronization - Automatic Merging - Local Availability - Scalability - Delta Updates **Supported CRDT Algorithms** - 📝 Text Editing with [Fugue] - 📙 [Rich Text CRDT](https://loro.dev/blog/loro-richtext) - 🌲 [Moveable Tree](https://loro.dev/docs/tutorial/tree) - 🚗 [Moveable List](https://loro.dev/docs/tutorial/list) - 🗺️ [Last-Write-Wins Map](https://loro.dev/docs/tutorial/map) - 🤝 Mergeable map-key children via `ensureMergeable*` / `ensure_mergeable_*` for lazy child container creation **Advanced Features in Loro** - 🚀 [Fast Document Loading](https://loro.dev/blog/v1.0) - ⏱️ Fast [Time Travel](https://loro.dev/docs/tutorial/time_travel) Through History - 🏛️ [Version Control with Real-Time Collaboration](https://loro.dev/blog/v1.0#version-control) - 📦 [Shallow Snapshot](https://loro.dev/docs/advanced/shallow_snapshot) that Works like Git Shallow Clone https://github.com/user-attachments/assets/68e0017a-4987-4f71-b2cf-4ed28a210987 > In this example, we demonstrate importing an entire Loro codebase into a Loro-powered > version controller, preserving the complete Git DAG history while enabling fast version switching. # Example [](https://stackblitz.com/edit/loro-basic-test?file=test%2Floro-sync.test.ts) ```ts import { expect, test } from "vitest"; import { LoroDoc, LoroList } from "loro-crdt"; test("sync example", () => { // Sync two docs with two rounds of exchanges // Initialize document A const docA = new LoroDoc(); const listA: LoroList = docA.getList("list"); listA.insert(0, "A"); listA.insert(1, "B"); listA.insert(2, "C"); // Export all updates from docA const bytes: Uint8Array = docA.export({ mode: "update" }); // Simulate sending `bytes` across the network to another peer, B const docB = new LoroDoc(); // Peer B imports the updates from A docB.import(bytes); // B's state matches A's state expect(docB.toJSON()).toStrictEqual({ list: ["A", "B", "C"], }); // Get the current version of docB const version = docB.oplogVersion(); // Simulate editing at B: delete item 'B' const listB: LoroList = docB.getList("list"); listB.delete(1, 1); // Export the updates from B since the last sync point const bytesB: Uint8Array = docB.export({ mode: "update", from: version }); // Simulate sending `bytesB` back across the network to A // A imports the updates from B docA.import(bytesB); // A has the same state as B expect(docA.toJSON()).toStrictEqual({ list: ["A", "C"], }); }); ``` # DevTools ## Loro Inspector You can use the [Loro Inspector](https://inspector.loro.dev) to inspect the state and history of a Loro document. https://github.com/user-attachments/assets/ceeb7450-80ce-42f2-aef4-2e08fa2d1f1b # Bindings You can find bindings for other programming languages in [loro-ffi](https://github.com/loro-dev/loro-ffi). # Blog - [Loro 1.0](https://loro.dev/blog/v1.0) - [Movable tree CRDTs and Loro's implementation](https://loro.dev/blog/movable-tree) - [Introduction to Loro's Rich Text CRDT](https://loro.dev/blog/loro-richtext) - [Loro: Reimagine State Management with CRDTs](https://loro.dev/blog/loro-now-open-source) # Sponsors [](https://github.com/sponsors/loro-dev) # Credits Loro draws inspiration from the innovative work of the following projects and individuals: - [Diamond-types](https://github.com/josephg/diamond-types): The [Event Graph Walker (Eg-walker)](https://loro.dev/docs/advanced/event_graph_walker) algorithm from @josephg has been adapted to reduce the computation and space usage of CRDTs. - [Automerge](https://github.com/automerge/automerge): Their use of columnar encoding for CRDTs has informed our strategies for efficient data encoding. - [Yjs](https://github.com/yjs/yjs): We have incorporated a similar algorithm for effectively merging collaborative editing operations, thanks to their pioneering work. - [Matthew Weidner](https://mattweidner.com/): His work on the [Fugue](https://arxiv.org/abs/2305.00583) algorithm has been invaluable, enhancing our text editing capabilities. - [Martin Kleppmann](https://martin.kleppmann.com/): His work on CRDTs has significantly influenced our comprehension of the field. [local-first]: https://www.inkandswitch.com/local-first/ [Fugue]: https://arxiv.org/abs/2305.00583 --- ## File: crates/loro-wasm/README.md

[Loro](https://loro.dev)

✨ Loro 1.0 is out! Read the [announcement](https://loro.dev/blog/v1.0).

Loro is a [CRDTs(Conflict-free Replicated Data Types)](https://crdt.tech/) library that makes building [local-first][local-first] and collaborative apps easier. You can now use it in Rust, JS (via WASM), and Swift. # Features **Features Provided by CRDTs** - P2P Synchronization - Automatic Merging - Local Availability - Scalability - Delta Updates **Supported CRDT Algorithms** - 📝 Text Editing with [Fugue] - 📙 [Rich Text CRDT](https://loro.dev/blog/loro-richtext) - 🌲 [Moveable Tree](https://loro.dev/docs/tutorial/tree) - 🚗 [Moveable List](https://loro.dev/docs/tutorial/list) - 🗺️ [Last-Write-Wins Map](https://loro.dev/docs/tutorial/map) - 🤝 Mergeable map-key children via `ensureMergeable*` for lazy child container creation **Advanced Features in Loro** - 🚀 [Fast Document Loading](https://loro.dev/blog/v1.0) - ⏱️ Fast [Time Travel](https://loro.dev/docs/tutorial/time_travel) Through History - 🏛️ [Version Control with Real-Time Collaboration](https://loro.dev/blog/v1.0#version-control) - 📦 [Shallow Snapshot](https://loro.dev/docs/advanced/shallow_snapshot) that Works like Git Shallow Clone > In this example, we demonstrate importing an entire Loro codebase into a Loro-powered > version controller, preserving the complete Git DAG history while enabling fast version switching. ## Debugging the Wasm build The standard build pipeline (`deno run -A ./scripts/build.ts dev|release`) now keeps DWARF debugging information through `wasm-bindgen` and emits two helper files alongside every `loro_wasm_bg.wasm` artifact: - `loro_wasm_bg.wasm.map` — a v3 source map derived from DWARF so that Chrome, Edge, and Firefox can show original Rust locations when inspecting stack traces. Load the source map in browser devtools; when devtools fetches the debug companion it can map instructions back to Rust source files and line numbers without inflating the shipped `.wasm`. ## Bundler entries Bare `import { LoroDoc } from "loro-crdt"` uses package conditional exports. Browser development builds can resolve the nested `browser` + `development` conditions to the bundler entry, while browser production builds can resolve the `browser` condition to a synchronous browser build that avoids Vite/Rolldown production chunk cycles around `.wasm` wrappers. A legacy `browser` string field also points to the browser entry for bundlers that still consult it. Runtimes that need native `.wasm` module imports can still use the `bundler` entry, and apps that prefer explicit async initialization can use `loro-crdt/web`. Vite and Webpack understand `new URL("./loro_wasm_bg.wasm", import.meta.url)` and emit the WASM asset automatically. Plain esbuild and plain Rollup do not copy that asset by default. For those tools, either import `loro-crdt/base64` to inline the WASM into the JS bundle without top-level await, or keep the default `loro-crdt` import and copy `node_modules/loro-crdt/browser/loro_wasm_bg.wasm` next to the emitted JS bundle as a build step. Next.js Turbopack can use the default browser entry. If a Next.js Webpack build resolves the `bundler` entry instead of the package `browser` remap, use `loro-crdt/base64`. # Example [](https://stackblitz.com/edit/loro-basic-test?file=test%2Floro-sync.test.ts) ```ts import { expect, test } from 'vitest'; import { LoroDoc, LoroList } from 'loro-crdt'; test('sync example', () => { // Sync two docs with two rounds of exchanges // Initialize document A const docA = new LoroDoc(); const listA: LoroList = docA.getList('list'); listA.insert(0, 'A'); listA.insert(1, 'B'); listA.insert(2, 'C'); // Export all updates from docA const bytes: Uint8Array = docA.export({ mode: 'update' }); // Simulate sending `bytes` across the network to another peer, B const docB = new LoroDoc(); // Peer B imports the updates from A docB.import(bytes); // B's state matches A's state expect(docB.toJSON()).toStrictEqual({ list: ['A', 'B', 'C'], }); // Get the current version of docB const version = docB.oplogVersion(); // Simulate editing at B: delete item 'B' const listB: LoroList = docB.getList('list'); listB.delete(1, 1); // Export the updates from B since the last sync point const bytesB: Uint8Array = docB.export({ mode: 'update', from: version }); // Simulate sending `bytesB` back across the network to A // A imports the updates from B docA.import(bytesB); // A has the same state as B expect(docA.toJSON()).toStrictEqual({ list: ['A', 'C'], }); }); ``` # Blog - [Loro 1.0](https://loro.dev/blog/v1.0) - [Movable tree CRDTs and Loro's implementation](https://loro.dev/blog/movable-tree) - [Introduction to Loro's Rich Text CRDT](https://loro.dev/blog/loro-richtext) - [Loro: Reimagine State Management with CRDTs](https://loro.dev/blog/loro-now-open-source) # Credits Loro draws inspiration from the innovative work of the following projects and individuals: - [Diamond-types](https://github.com/josephg/diamond-types): The [Event Graph Walker (Eg-walker)](https://loro.dev/docs/advanced/event_graph_walker) algorithm from @josephg has been adapted to reduce the computation and space usage of CRDTs. - [Automerge](https://github.com/automerge/automerge): Their use of columnar encoding for CRDTs has informed our strategies for efficient data encoding. - [Yjs](https://github.com/yjs/yjs): We have incorporated a similar algorithm for effectively merging collaborative editing operations, thanks to their pioneering work. - [Matthew Weidner](https://mattweidner.com/): His work on the [Fugue](https://arxiv.org/abs/2305.00583) algorithm has been invaluable, enhancing our text editing capabilities. - [Martin Kleppmann](https://martin.kleppmann.com/): His work on CRDTs has significantly influenced our comprehension of the field. [local-first]: https://www.inkandswitch.com/local-first/ [Fugue]: https://arxiv.org/abs/2305.00583 --- ## File: crates/loro-internal/docs/diff_calc.md # Internal Diff Calculation Diff calculation produces the state patch between two versions. Checkout, forking, import, and revert all use this path. ## Replay base and changed containers `OpLog::iter_from_replay_base_causally` first chooses a replay base. The safe base is a **critical version** in the Eg-walker sense (arXiv:2409.14252 §3.5): a version that no concurrency crosses. When a concurrent branch invalidates the candidate, the DAG retreats the base to the latest single-head critical version, which can be older than the meet of the two versions. An old base is not evidence that every container in the replay range changed. `DiffCalculator::calc_diff_internal` derives the changed container set from the version-vector difference and routes common history only to those calculators. Otherwise each unchanged List/Text/MovableList can trigger its own full-history tracker rebuild. The DAG walk follows explicit dependencies plus the implicit previous counter of the same peer. An implicit path may be redundant when an explicit relay dependency already contains that predecessor. The walk remembers the dependency tip where an unmatched path split and performs a targeted ancestor lookup from the candidate common frontiers: - a covered tip is another route into already-common history, so `from` remains the replay base; - an uncovered tip is a real concurrent branch, so the conservative base is retained. This lookup runs only for unmatched branch tips and prunes by visited DAG nodes and Lamport time. It does not calculate a causal version for every new peer and does not compare each one with the complete `from` version vector. ## Diff modes - `Checkout` is the general and slowest mode. It can move in either direction and may use `ContainerHistoryCache`. - `Import` requires `to > from`, but some imported operations may be concurrent with `from`. - `ImportGreaterUpdates` additionally guarantees that every imported operation is causally after `from`, so the replay base is `from`. - `Linear` additionally guarantees that imported operations are ordered, so diff calculation does not need to build CRDT trackers. List, Text, and MovableList still rebuild their trackers from CRDT IDs when retreating, when their source context is incomplete, or when shallow history requires it. That fallback is a correctness requirement, not a replay-base optimization. --- ## File: crates/loro-internal/README.md # loro-internal This is the internal implementation of Loro. Its interface subject to change. You should not use it as dependency directly. Consider use `loro` crate instead. --- ## File: crates/loro-common/README.md # loro-common This crate contains the common basic types for the Loro project. It's shared by `loro-internal` and `loro-preload`. --- ## File: crates/loro/README.md # Loro Loro is a high‑performance CRDT framework for local‑first apps that keeps state consistent across devices and users, works offline and in real time, automatically merges conflicts, and enables undo/redo and time travel. [Loro](https://loro.dev) is a high-performance CRDTs library offering Rust, JavaScript and Swift APIs. ## Common Tasks & Examples ### Getting Started - Create a document: [`LoroDoc::new()`](struct.LoroDoc.html#method.new) - Initialize a new collaborative document - Add containers: [`get_text`](struct.LoroDoc.html#method.get_text), [`get_list`](struct.LoroDoc.html#method.get_list), [`get_map`](struct.LoroDoc.html#method.get_map), [`get_tree`](struct.LoroDoc.html#method.get_tree), [`get_movable_list`](struct.LoroDoc.html#method.get_movable_list), [`get_counter` (feature "counter")](struct.LoroDoc.html#method.get_counter) - Listen to changes: [`subscribe`](struct.LoroDoc.html#method.subscribe) or [`subscribe_root`](struct.LoroDoc.html#method.subscribe_root) - React to document/container modifications - Export/Import state: [`export`](struct.LoroDoc.html#method.export) and [`import`](struct.LoroDoc.html#method.import) - Save and load documents ### Real-time Collaboration - Sync between peers: [`export(ExportMode::updates(&vv))`](struct.LoroDoc.html#method.export) + [`import`](struct.LoroDoc.html#method.import) - Exchange incremental updates (see [`ExportMode::updates`](enum.ExportMode.html#method.updates)) - Stream updates: [`subscribe_local_update`](struct.LoroDoc.html#method.subscribe_local_update) - Send changes over WebSocket/WebRTC - Set unique peer ID: [`set_peer_id`](struct.LoroDoc.html#method.set_peer_id) - Ensure each client has a unique identifier - Handle conflicts: Automatic - All Loro data types are CRDTs that merge concurrent edits ### Rich Text Editing - Create rich text: [`get_text`](struct.LoroDoc.html#method.get_text) - Initialize a collaborative text container - Edit text: [`LoroText::insert`](struct.LoroText.html#method.insert), [`LoroText::delete`](struct.LoroText.html#method.delete), [`LoroText::apply_delta`](struct.LoroText.html#method.apply_delta) - Apply formatting: [`LoroText::mark`](struct.LoroText.html#method.mark) - Add bold, italic, links, custom styles - Track cursor positions: [`LoroText::get_cursor`](struct.LoroText.html#method.get_cursor) + [`LoroDoc::get_cursor_pos`](struct.LoroDoc.html#method.get_cursor_pos) - Stable positions across edits - Configure styles: [`config_text_style`](struct.LoroDoc.html#method.config_text_style) / [`config_default_text_style`](struct.LoroDoc.html#method.config_default_text_style) - Define expand behavior for marks ### Data Structures - Ordered lists: [`get_list`](struct.LoroDoc.html#method.get_list) - Arrays with [`push`](struct.LoroList.html#method.push), [`insert`](struct.LoroList.html#method.insert), [`delete`](struct.LoroList.html#method.delete) - Key-value maps: [`get_map`](struct.LoroDoc.html#method.get_map) - Objects with [`insert`](struct.LoroMap.html#method.insert), [`get`](struct.LoroMap.html#method.get), [`delete`](struct.LoroMap.html#method.delete) - Mergeable map-key children: [`ensure_mergeable_text`](struct.LoroMap.html#method.ensure_mergeable_text), [`ensure_mergeable_map`](struct.LoroMap.html#method.ensure_mergeable_map), and related `ensure_mergeable_*` methods - Lazy child containers that converge when peers create the same map-key child concurrently - Hierarchical trees: [`get_tree`](struct.LoroDoc.html#method.get_tree) - Trees with [`create`](struct.LoroTree.html#method.create), [`mov`](struct.LoroTree.html#method.mov), [`mov_to`](struct.LoroTree.html#method.mov_to) - Reorderable lists: [`get_movable_list`](struct.LoroDoc.html#method.get_movable_list) - Drag-and-drop with [`mov`](struct.LoroMovableList.html#method.mov), [`set`](struct.LoroMovableList.html#method.set) - Counters: [`get_counter` (feature "counter")](struct.LoroDoc.html#method.get_counter) - Distributed counters with [`increment`](struct.LoroCounter.html#method.increment) ### Ephemeral State & Presence - Not currently provided in the Rust crate. Model presence in your app layer alongside CRDT updates (e.g., via your network transport). Cursors can be shared using [`get_cursor`](struct.LoroText.html#method.get_cursor) data if needed. ### Version Control & History - Undo/redo: [`UndoManager`](struct.UndoManager.html) - Local undo of user’s own edits - Time travel: [`checkout`](struct.LoroDoc.html#method.checkout) to any [`Frontiers`](struct.Frontiers.html) - Debug or review history - Version tracking: [`oplog_vv`](struct.LoroDoc.html#method.oplog_vv), [`state_frontiers`](struct.LoroDoc.html#method.state_frontiers), [`VersionVector`](struct.VersionVector.html) - Fork documents: [`fork`](struct.LoroDoc.html#method.fork) or [`fork_at`](struct.LoroDoc.html#method.fork_at) - Create branches for experimentation - Merge branches: [`import`](struct.LoroDoc.html#method.import) - Combine changes from forked documents ### Performance & Storage - Incremental updates: [`export(ExportMode::updates(&their_vv))`](struct.LoroDoc.html#method.export) - Send only changes (see [`ExportMode::updates`](enum.ExportMode.html#method.updates)) - Compact history: [`export(ExportMode::Snapshot)`](struct.LoroDoc.html#method.export) - Full state with compressed history (see [`ExportMode::Snapshot`](enum.ExportMode.html#variant.Snapshot)) - Shallow snapshots: [`export(ExportMode::shallow_snapshot(&frontiers))`](struct.LoroDoc.html#method.export) - State without partial history (see [`ExportMode::shallow_snapshot`](enum.ExportMode.html#method.shallow_snapshot)) ## Documentation - Start with the [Rust API docs for `LoroDoc`](struct.LoroDoc.html) (container management, versioning, import/export, events) That page hosts examples and details for most important methods you’ll use day-to-day. - [Loro Website](https://loro.dev) for more details and guides - [Loro Examples](https://github.com/loro-dev/loro-examples) for more examples and guides ## Getting Started Add to your `Cargo.toml`: ```toml [dependencies] loro = "^1" ``` ### LoroDoc quick tour - Containers: [`get_text`](struct.LoroDoc.html#method.get_text), [`get_map`](struct.LoroDoc.html#method.get_map), [`get_list`](struct.LoroDoc.html#method.get_list), [`get_movable_list`](struct.LoroDoc.html#method.get_movable_list), [`get_tree`](struct.LoroDoc.html#method.get_tree) - Import/Export: [`export(ExportMode::…)`](struct.LoroDoc.html#method.export), [`import`](struct.LoroDoc.html#method.import), [`from_snapshot`](struct.LoroDoc.html#method.from_snapshot) - Versioning: [`oplog_vv`](struct.LoroDoc.html#method.oplog_vv), [`state_frontiers`](struct.LoroDoc.html#method.state_frontiers), [`checkout`/`checkout_to_latest`](struct.LoroDoc.html#method.checkout), [`revert_to`](struct.LoroDoc.html#method.revert_to), [`fork`](struct.LoroDoc.html#method.fork) - Events: [`subscribe`](struct.LoroDoc.html#method.subscribe), [`subscribe_root`](struct.LoroDoc.html#method.subscribe_root), [`subscribe_local_update`](struct.LoroDoc.html#method.subscribe_local_update) (send deltas to peers) - Paths/JSON: [`get_path_to_container`](struct.LoroDoc.html#method.get_path_to_container), [`get_deep_value`](struct.LoroDoc.html#method.get_deep_value) / [`ToJson`](trait.ToJson.html) (`to_json_value()`), optional [`jsonpath` (feature)](struct.LoroDoc.html#method.jsonpath) Optional cargo features: ```toml [dependencies] loro = { version = "^1", features = ["jsonpath"] } ``` ## Quick Examples 1) Local edits, change events, and two-peer sync ```rust use loro::{LoroDoc, ExportMode}; use std::sync::Arc; let a = LoroDoc::new(); let b = LoroDoc::new(); // Listen for container diffs on `a` let _changes = a.subscribe_root(Arc::new(|e| { println!("changed containers: {}", e.events.len()); })); a.get_text("text").insert(0, "Hello, Loro!").unwrap(); a.commit(); // events fire on commit/export/import/checkout // Sync via export/import (send `updates` via your transport) let updates = a.export(ExportMode::all_updates()).unwrap(); b.import(&updates).unwrap(); assert_eq!(a.get_deep_value(), b.get_deep_value()); ``` 2) Time travel and revert ```rust use loro::LoroDoc; let doc = LoroDoc::new(); let text = doc.get_text("text"); text.insert(0, "Hello").unwrap(); let v0 = doc.state_frontiers(); text.insert(5, ", world").unwrap(); assert_eq!(text.to_string(), "Hello, world"); // Time travel to v0 (read-only) doc.checkout(&v0).unwrap(); assert_eq!(text.to_string(), "Hello"); // Return to latest and revert doc.checkout_to_latest(); doc.revert_to(&v0).unwrap(); assert_eq!(text.to_string(), "Hello"); ``` --- ## File: crates/fractional_index/README.md # Fractional Index This crate is forked from [drifting-in-space/fractional_index](https://github.com/drifting-in-space/fractional_index) --- ## File: docs/Glossary.md # Glossary | Name | Meaning | Note | |-------------------------|----------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| | DAG | Directed Acyclic Graph | The changes in Loro form a DAG. The relationship is expressed by deps field. | | [RLE](#rle) | Run Length Encoding | We not only use it on the encoding, but also in the vec and tree. | | Change | A merged batch of ops | | | Frontiers | The DAG nodes that no one has dependencies on them | They can be represented by a series of id | | Op | Operation | | | State (of a Container) | The | In the code, the state most refers to the visible state to users. For example, the text string of a text container, the key value pairs in the map container | | Effect (of an Op) | How the op affect the state | We use CRDT to calculate the effect (obviously) | | Tracker | A special data structure to calculate the effects | | | [Container](#container) | A unit of CRDT data structure | | | Content | Content of an Op | | | ID | A global unit id for an Op | It has the structure of {clientId, counter} | | Counter | The second field inside ID | | | Lamport | [Lamport timestamp](https://en.wikipedia.org/wiki/Lamport_timestamp) | | | Span | A series of continuous things | | | Causal Order | The order of happen-before relationship | The DAG expresses the causal order of the changes | | VV | [Version Vector](https://en.wikipedia.org/wiki/Version_vector) | In code, it has exclusive end, `VV {0: 8}` means the last op id of client 0 is `0-7` | ### RLE We not only use RLE on the encoding, but also in the vec and tree. i.e. the elements can be merged, sliced and have lengths. We call the element with length of 1 as atom element (cannot be sliced). We use a `Rle` trait to express it. So `Op`, `Change`, elements inside `RleTree` all implement `Rle`. This gives us a compact way to represent the mergeable elements. ### Container Each op is associated with one container. Different CRDT algorithms use different types of containers. There are hierarchical relationship between containers, but they cannot affect each other # Convention - There are methods like `id_last`, `id_end`, `lamport_last`, `lamport_end`. `last` refers to the last one, but `end` refers to the exclusive end of a range. For example, OpSpan ranging from 0-0 till 0-8 has id_last of 0-8 and id_end of 0-9 --- ## File: docs/JsonSchema.md # JSON Schema for Loro's OpLog ## Introduction Loro supports multiple data structures and introduces many new concepts. Having only binary export formats would make it difficult for developers to understand the underlying processes. Better transparency leads to better developer experience. A human-readable JSON representation enables users to better understand and operate the document and to develop related tools. To better understand this document, you may first need to understand how Loro stores historical editing data: - [OpLog](https://www.loro.dev/docs/advanced/doc_state_and_oplog) - [`Change`, `Operation`](https://www.loro.dev/docs/advanced/op_and_change) - [`Replayable Event Graph (REG)`](https://www.loro.dev/docs/advanced/replayable_event_graph) It should be noted that considering the usage scenario, JSON Schema only supports backward compatibility but not forward compatibility. # Specification ## Root object The root object contains all `Change`s, `Op`s, and critical metadata like start/end versions and schema version. We will also extract the 64-bit integer PeerID to the beginning of the document and replace it internally with incrementing numbers starting from zero: 0, 1, 2, 3... This significantly reduces the document size and enhances readability. ```ts { "schema_version": number, "start_version": Map, "peers": string[], "changes": Change[], } ``` - `schema_version`: the version of the schema that the document is encoded with. It's 1 for the current specification. - `start_version`: the start `Frontiers` version of the document. They are represented as a map from the decimal string representation of `PeerID` to `Counter`. - `peers`: the list of peers in the document. We represent all PeerIDs as decimal strings to avoid exceeding JavaScript's number limit. - `changes`: the list of changes in the document. ## Changes `Change`s are crucial in the OpLog. A REG([Replay event graph](https://www.loro.dev/docs/advanced/replayable_event_graph)) is a directed acyclic graph where each node is a `Change`, and each edge is a causal dependency between `Change`s. The metadata of the `Change`s helps us reconstruct the graph. You can also attach a commit message to a `Change` like you usually do with Git's commit. ```ts { "id": string, "timestamp": number, "deps": OpID[], "lamport": number, "msg": string, "ops": Op[] } type OpID = `${number}@${PeerID}`; ``` - `id`: the string representation of the unique `ID` of each `Change`, in the form of `{Counter}@{PeerID}` which is the `@` character connecting `Counter` and `PeerID`. Of course, This `PeerID` is the index of peers in the global context. - `timestamp`: the number of Unix timestamp when the change is committed. [Timestamp is not recorded by default](https://loro.dev/docs/advanced/timestamp) - `deps`: a list of causal dependency of this `Change`, each item is the `ID` represented by a string. - `lamport`: the lamport timestamp of the `Change`. - `msg`: the commit message. - `ops`: all of the `Op` in the `Change`. ## Operations Operation (abbreviated as `Op`) is the most complex part of the document. Loro currently supports multiple containers `List`, `Map`, `RichText`, `Movable List` and `Movable Tree`. Each data structure has several different `Op`s. But in general, each `Op` is composed of the `ContainerID` of the container that created it, a counter, and the corresponding content of the `Op`. ```ts type Op = { "container": ContainerID, "counter": number, "content": OpContent // Its detailed definition is elaborated below, with different types for different Containers. }; type OpContent = ListOp | TextOp | MapOp | TreeOp | MovableListOp | UnknownOp; type ContainerID = | `cid:root-${string}:${ContainerType}` | `cid:${number}@${PeerID}:${ContainerType}`; ``` Mergeable child containers use reserved synthetic root-name container IDs under the internal mergeable namespace, plus binary markers in parent map slots to control visibility. They still fit the `cid:root-${string}:${ContainerType}` shape in exported JSON. See [`mergeable-container-id.md`](../crates/loro-internal/docs/mergeable-container-id.md) for the reserved namespace and marker details. - `container`: the `ContainerID` of the container that created this `Op`, represented by a string starts with `cid:`. - `counter`: the counter part of the OpID - `content`: the semantic content of the `Op`, it is different for each field depending on the `Container`. The following is the **content** of each container。 ### List ```ts type ListOp = ListInsertOp | ListDeleteOp; ``` #### Insert ```ts type ListInsertOp = { "type": "insert", "pos": number, "value": LoroValue } ``` - `type`: `insert`. - `pos`: the index of the insert operation. - `value`: the insert content which is a list of `LoroValue` #### Delete ```ts type ListDeleteOp = { "type": "delete", "pos": number, "len": number, "start_id": OpID } ``` - `type`: `delete`. - `pos`: the start index of the deletion. - `len`: the length of deleted content. - `start_id`: the string id of start element deleted. ### MovableList ```ts type MovableListOp = ListInsertOp | ListDeleteOp | MovableListMoveOp | MovableListSetOp; ``` #### Insert ```ts type ListInsertOp = { "type": "insert", "pos": number, "value": LoroValue } ``` - `type`: `insert`, - `pos`: the index of the insert operation. - `value`: the insert content which is a list of `LoroValue` #### Delete ```ts type ListDeleteOp = { "type": "delete", "pos": number, "len": number, "start_id": OpID } ``` - `type`: `delete` - `pos`: the start index of the deletion. - `len`: the length of deleted content. - `start_id`: the string id of start element deleted. #### Move ```ts type MovableListMoveOp = { "type": "move", "from": number, "to": number, "elem_id": ElemID } type ElemID = `L${number}@${PeerID}` ``` - `type`:`insert`, `delete`, `move` or `set`. - `from`: the index of the element before is moved. - `to`: the index of the index moved to after moving out the element - `elem_id`: the ID (described by lamport@peer) of the element moved. #### Set ```ts type MovableListSetOp = { "type": "set", "elem_id": ElemID, "value": LoroValue } type ElemID = `L${number}@${PeerID}` ``` - `type`:`insert`, `delete`, `move` or `set`. - `elem_id`: the ID (described by lamport@peer) of the element replaced. - `value`: the value set. ### Map ```ts type MapOp = MapInsertOp | MapDeleteOp; ``` #### Insert ```ts type MapInsertOp = { "type": "insert", "key": string, "value": LoroValue } ``` - `type`: `insert`. - `key`: the key of the insertion. - `value`: the value of the insertion. #### Delete ```ts type MapDeleteOp = { "type": "delete", "key": string } ``` - `type`: `delete`. - `key`: the key of the deletion ### Text ```ts type TextOp = TextInsertOp | TextDeleteOp | TextMarkOp | TextMarkEndOp; ``` #### Insert ```ts type TextInsertOp = { "type": "insert", "pos": number, "text": string } ``` `type`: `insert`. `pos`: the index of the insert operation. The position is based on the Unicode code point length. `text`: the string of the insertion. #### Delete ```ts type TextDeleteOp = { "type": "delete", "pos": number, "len": number, "start_id": OpID } ``` `type`: `delete`. `pos`: the index of the deletion. The position is based on the Unicode code point length. `len`: the length of the text deleted. `start_id`: the string id of the beginning element deleted. #### Mark ```ts type TextMarkOp = { "type": "mark", "start": number, "end": number, "style_key": string, "style_value": LoroValue, "info": number } ``` `type`: `mark` `start`: the start index of text need to mark. The position is based on the Unicode code point length. `end`: the end index of text need to mark. The position is based on the Unicode code point length. `style_key`: the key of style, it is customizable. `style_value`: the value of style, it is customizable. `info`: the config of the style, whether to expand the style when inserting new text around it. #### MarkEnd ```ts type TextMarkEndOp = { "type": "mark_end" } ``` `type`: `mark_end`. ### Tree ```ts type TreeOp = TreeCreateOp | TreeMoveOp | TreeDeleteOp; ``` #### Create ```ts type TreeCreateOp = { "type": "create", "target": TreeID, "parent": TreeID | null, "fractional_index": string } type TreeID = `${number}@${PeerID}` ``` - `type`: `create`. - `target`: the string format of target `TreeID` moved. - `parent`: the string format of `TreeID` or `null`. If it is `null`, the target node will be a root node. - `fractional_index`: the fractional index with hex string format of the target node. #### Move ```ts type TreeMoveOp = { "type": "move", "target": TreeID, "parent": TreeID | null, "fractional_index": string } type TreeID = `${number}@${PeerID}` ``` - `type`: `move`. - `target`: the string format of target `TreeID` moved. - `parent`: the string format of `TreeID` or `null`. If it is `null`, the target node will be a root node. - `fractional_index`: the fractional index with hex string format of the target node. #### Delete ```ts type TreeDeleteOp = { "type": "delete", "target": TreeID } type TreeID = `${number}@${PeerID}` ``` - `type`: `delete`. - `target`: the string format of target `TreeID` deleted. ### Unknown To support forward compatibility, we have an unknown type. When an `Op` with a newly supported Container from a newer version is decoded into the older version, it will be treated as an unknown type in a more general form, such as binary and string. When the new version decodes an unknown `Op`, the newer version of Loro will know its true type and decode correctly. ```ts type UnknownOp = { "type": "unknown", "prop": number, "value_type": string, "value": `${EncodeValue}` } ``` - `type`: just an unknown type. - `prop`: a property of the encoded op, it's a number. - `value_type`: the type of `EncodeValue`. - `value`: common data types used in encoding with json string format. ## Value In this section, we will introduction two *Value* in Loro. One is `LoroValue`, it's an enum of data types supported by Loro, such as the value inserted by `List` or `Map`. The another is `EncodedValue`, it's just used in encoding module for unknown type. ### LoroValue These are data types supported by Loro and its json format: - `null`: `null` - `Bool`: `true` or `false` - `F64`: `number`(float) - `I64`: `number` or `bigint` (signed) - `Binary`: `UInt8Array` - `String`: `string` - `List`: `Array` - `Map`: `Map` - `Container`: the id of container. `🦜:cid:{Counter}@{PeerID}:{ContainerType}` or `🦜:cid:root-{Name}:{ContainerType}` Note: Compared with the string format, we add a prefix `🦜:` when encoding the json format of `ContainerID` to prevent users from saving the string format of `ContainerID` and misinterpreting it as `ContainerID` when decoding. ### EncodedValue The `EncodedValue` is the specific type used by Loro when encoding, it's an internal value, users do not need to get it clear. It is specially designed to handle the schema mismatch due to forward and backward compatibility. In JSON encoding schema, the `EncodedValue` will be encoded as an object.