Node-Media-Server Documentation

Technical Documentation: illuspas/Node-Media-Server

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`)

Node-Media-Server v4

Introduction

Node-Media-Server is a high-performance/low-latency/open-source Live Streaming Server developed based on Nodejs.
v4 is design to implement enhanced RTMP FLV v1 support for native HEVC, VP9, AV1.
v4 is no longer compatible with the cn_cdn extension id flv_265 standard.
v4 is incompatible with v2. Do not upgrade across major versions.

Installation

text

npm install node-media-server -g
node-media-server

or run directly

text

npx node-media-server

Cli Options

text

Options:
  -c, --config <path>    Path to the config file (default: bin/config.json)
  -b, --bind <addr>      Bind address, overrides the "bind" config value
      --rtmp-port <n>    RTMP port, overrides rtmp.port
      --rtmps-port <n>   RTMPS port, overrides rtmps.port
      --http-port <n>    HTTP/WebSocket port, overrides http.port
      --https-port <n>   HTTPS/WSS port, overrides https.port
      --data-path <path> Runtime data directory, overrides store.path
      --record-path <p>  Record output directory, overrides record.path
      --record-auto      Enable auto recording, forces record.auto on
      --notify-url <url> Event webhook URL, overrides notify.url
      --auth-play        Enable play authentication, forces auth.play on
      --auth-publish     Enable publish authentication, forces auth.publish on
      --no-admin         Disable the webadmin console (/admin), media only
  -h, --help             Show this help

Command line values take precedence over config file values and are
never written back to the config file.

WebAdmin

Features

HTTP/HTTP2-flv Push/Play
WS/WSS-flv Push/Play
RTMP/RTMPS Push/Play
GOP cache
Notification
Authentication
Static file server
Record to flv file
Enhanced RTMP v2 multitrack audio
REST API System (New in v4.2.0)
JWT-based Authentication (New in v4.2.0)
Real-time Monitoring & Statistics (New in v4.2.0)
Session Management (New in v4.2.0)
Session Deletion (New in v4.2.0)
Advanced Health Monitoring (New in v4.2.0)
RTSP/RTMP Relay Tasks with persistence and management API (New in v4.3.0)
Web Admin UI with Dashboard, Streams, Records, History, Relay and Settings (New in v4.3.0)
Username/Password Login with scrypt hashing and login rate limiting (New in v4.3.0)
Configuration Management API with validation (New in v4.3.0)
Change Password API (New in v4.3.0)
Record File Download (New in v4.3.0)
Record Session Resume within publish grace period (New in v4.3.0)
History Search by stream path, IP and time range (New in v4.3.0)
Network Bandwidth Statistics (New in v4.3.0)
Graceful Shutdown (New in v4.3.0)

Static file services

Node-Media-Server can provide static file services for a directory.

text

"static": {
    "router": "/",
    "root": "./html"
}

Record to flv file

Node-Media-Server can record live streams as FLV files.
When the static file server is enabled and recordings are saved in its directory.
It can provide video-on-demand services.

text

"record": {
    "path": "./html/record",
    "auto": true
}

Set "auto": false to disable auto-recording of every published stream; streams can then be recorded on demand via the webadmin streams page or POST /api/v1/streams/{app}/{name}/record.

text

http://server_ip:8000/record/live/stream/unix_time.flv
or
https://server_ip:8443/record/live/stream/unix_time.flv

REST API System (New in v4.2.0)

Node-Media-Server v4.2.0 provides a REST API system for server management and monitoring, featuring:

JWT-based authentication with challenge-response login
Stream and session management
Relay (RTSP pull / RTMP pull-push) task management
Real-time monitoring & statistics
Advanced health monitoring

See the full API documentation in docs/api.md.

Supported clients

Client	H.264	HEVC	VP9	AV1
OBS_29.1+	✅	✅	❌	✅
FFmpeg/FFplay_6.1+	✅	✅	✅	✅
NodePlayer.js_1.0+	✅	✅	❌	❌
NodeMediaClient_3.0+	✅	✅	❌	❌

[QLive](https://play.google.com/store/apps/details?id=cn.nodemedia.qlive) free Android Live Streaming App

Custom output
Portrait and landscape orientation support
Multiple codec support
Multiple filter settings

[NodePlayer.js](https://www.nodemedia.cn/product/nodeplayer-js/) pure javascript implementation live streaming player

Online Demo

ASM.js, WASM, SIMD, WebWorker, WebCodecs, MediaSource multiple technical implementations
H.264/H.265+AAC/G711 software and hardware decoder
Ultra-low latency, Under extreme conditions less than 100 milliseconds
Enhanced HTTP/WS-FLV Protocol, Natively support h.265
Android/iOS/HarmonyOS/Chrome/Edge/Firefox/Safari, All modern browsers or platforms

[NodePublisher.js](https://www.nodemedia.cn/demo/nodepublisher/) pure javascript implementation live streaming publisher

WebSocket-FLV Protocol
H.264+AAC hardware encoder
Only chrome or chromium based browsers are supported at the moment
wss is required

[NodeMediaClient-iOS](https://github.com/NodeMedia/NodeMediaClient-iOS) iOS live streaming player and publisher SDK

Objective-C/Swift
RTMP/HTTP-FLV/RTSP
H.264/H.265+AAC/OPUS/G711
Ultra-low latency, Under extreme conditions less than 100 milliseconds
Enhanced RTMP/FLV Protocol, Natively support H.265/OPUS
Built-in beauty filter

[NodeMediaClient-Android](https://github.com/NodeMedia/NodeMediaClient-Android) Android live streaming player and publisher SDK

JAVA/Kotlin
armv7/arm64/x86/x86_64
RTMP/HTTP-FLV/RTSP
H.264/H.265+AAC/OPUS/G711
Ultra-low latency, Under extreme conditions less than 100 milliseconds
Enhanced RTMP/FLV Protocol, Natively support H.265/OPUS
Built-in beauty filter

[expo-nodemediaclient](https://github.com/NodeMedia/expo-nodemediaclient) Expo module for NodeMediaClient

iOS and Android
player and publisher
Built-in beauty filter

Other server-side options

Commercial version: [NodeMediaServer v3](https://www.nodemedia.cn/product/node-media-server/)

Multi-core ultra-high concurrency
WebRTC / SRT / HLS / RTSP / KMP protocol
Real-time audio transcoding
Multi-resolution video transcoding
VOD slicing and services

Windows executable: [nginx-live-win32](https://github.com/illuspas/nginx-live-win32)

Enhanced rtmp v2 and http/websocket-flv
No dependencies, single exe deployment
Easy to deploy in offline environments
Based on the latest nginx version

License

Apache 2.0

Overview (`webadmin/README.md`)

React + TypeScript + Vite

This template provides a minimal setup to get React working in Vite with HMR and some Oxlint rules.

Currently, two official plugins are available:

@vitejs/plugin-react uses Oxc
@vitejs/plugin-react-swc uses SWC

React Compiler

The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see this documentation.

Expanding the Oxlint configuration

If you are developing a production application, we recommend enabling type-aware lint rules by installing oxlint-tsgolint and editing .oxlintrc.json:

json

{
  "$schema": "./node_modules/oxlint/configuration_schema.json",
  "plugins": ["react", "typescript", "oxc"],
  "options": {
    "typeAware": true
  },
  "rules": {
    "react/rules-of-hooks": "error",
    "react/only-export-components": ["warn", { "allowConstantExport": true }]
  }
}

See the Oxlint rules documentation for the full list of rules and categories.

Architecture Overview (`docs/architecture-overview.md`)

Node-Media-Server 架构总览

项目简介

Node-Media-Server 是一个高性能的 Node.js 实时流媒体服务器，支持多种流媒体协议和现代编解码器。本文档详细分析了整个项目的架构、工作流程和各个组件之间的关系。

核心架构

1. 系统层次结构

text

/* Detailed source-code truncated for AI context efficiency. */

2. 主要组件说明

2.1 核心组件 (Core Components)

Context (src/core/context.js)

全局共享状态管理器
管理活跃会话 (sessions)
管理广播流 (broadcasts)
事件发射器 (eventEmitter)

Logger (src/core/logger.js)

统一日志记录系统
支持不同日志级别
结构化日志输出

AVPacket (src/core/avpacket.js)

音视频数据包抽象
统一媒体数据格式
支持多种编解码器

2.2 服务器组件 (Server Components)

NodeRtmpServer (src/server/rtmp_server.js)

RTMP/RTMPS 服务器实现
TCP/SSL 连接处理
支持标准1935端口和安全1936端口

NodeHttpServer (src/server/http_server.js)

HTTP/HTTPS 服务器
静态文件服务
API 接口服务
WebSocket 支持

NodeRecordServer (src/server/record_server.js)

流录制功能
支持多种录制格式
自动分段录制

NodeNotifyServer (src/server/notify_server.js)

推送通知系统
支持HTTP回调
事件通知机制

2.3 会话组件 (Session Components)

BaseSession (src/session/base_session.js)

抽象会话基类
通用会话属性和方法
会话生命周期管理

RtmpSession (src/session/rtmp_session.js)

RTMP协议会话实现
推流和播放会话处理
RTMP协议状态管理

FlvSession (src/session/flv_session.js)

FLV协议会话实现
HTTP和WebSocket支持
FLV流数据处理

2.4 协议组件 (Protocol Components)

Rtmp (src/protocol/rtmp.js)

RTMP协议完整实现
握手、分块、消息处理
支持增强RTMP FLV v1

Flv (src/protocol/flv.js)

FLV格式处理
标签解析和生成
现代编解码器支持

Amf (src/protocol/amf.js)

Action Message Format
元数据序列化/反序列化
与Flash兼容的数据格式

工作流程

1. 服务器启动流程

mermaid

graph TD
    A[bin/app.js 启动] --> B[加载配置文件]
    B --> C[安全处理: 替换默认密码]
    C --> D[创建 NodeMediaServer]
    D --> E[初始化全局上下文]
    E --> F[启动各个服务器组件]
    F --> G[RTMP 服务器: 1935/1936端口]
    F --> H[HTTP 服务器: 8000/8443端口]
    F --> I[录制服务器]
    F --> J[通知服务器]
    G --> K[服务器运行中]
    H --> K
    I --> K
    J --> K

2. RTMP推流流程

mermaid

graph TD
    A[推流客户端连接] --> B[TCP握手]
    B --> C[RTMP协议握手]
    C --> D[connect命令处理]
    D --> E[createStream命令]
    E --> F[publish命令]
    F --> G[创建广播服务器]
    G --> H[身份验证]
    H --> I[开始接收媒体数据]
    I --> J[解析为AVPacket]
    J --> K[转换为FLV/RTMP格式]
    K --> L[分发给所有订阅者]
    L --> M[更新GOP缓存]

3. 多协议播放流程

mermaid

graph TD
    A[播放客户端] --> B{协议类型}
    B -->|RTMP| C[RTMP握手]
    B -->|HTTP-FLV| D[HTTP请求]
    B -->|WebSocket-FLV| E[WebSocket连接]
    C --> F[创建RtmpSession]
    D --> G[创建FlvSession HTTP]
    E --> H[创建FlvSession WS]
    F --> I[获取广播服务器]
    G --> I
    H --> I
    I --> J[身份验证]
    J --> K[发送流头部数据]
    K --> L[发送GOP缓存]
    L --> M[实时流数据传输]

4. API请求处理流程

mermaid

graph TD
    A[API请求] --> B[Express路由]
    B --> C{是否需要认证}
    C -->|是| D[JWT中间件]
    C -->|否| E[直接处理]
    D --> F{令牌验证}
    F -->|成功| G[API处理器]
    F -->|失败| H[401未授权]
    G --> I[访问Context数据]
    I --> J[格式化响应]
    J --> K[返回JSON数据]
    E --> G

关键特性

1. 多协议支持

RTMP/RTMPS: 标准推流协议，支持SSL加密
HTTP-FLV: HTTP长连接播放，低延迟
WebSocket-FLV: WebSocket播放，更好的浏览器兼容性

2. 现代编解码器支持

视频: H.264, H.265/HEVC, VP9, AV1
音频: AAC, MP3, Opus
增强RTMP FLV v1: 原生支持现代编解码器

3. 安全特性

HMAC-SHA256握手: 安全的RTMP握手验证
TLS/SSL支持: HTTPS、RTMPS加密传输
JWT认证: REST API访问控制
流认证: MD5签名的推流/播放认证

4. 监控和管理

REST API: 完整的流管理接口
实时统计: 会话、流、系统性能监控
事件系统: 全生命周期事件通知
日志系统: 结构化日志记录

5. 高性能设计

事件驱动: 基于Node.js事件循环
内存优化: GOP缓存管理，防止内存泄漏
并发处理: 支持大量并发连接
协议转换: 实时格式转换，低延迟

配置系统

配置文件结构 (`bin/config.json`)

json

{
  "rtmp": {
    "port": 1935,
    "rtmps": {
      "port": 1936,
      "key": "server.key",
      "cert": "server.crt"
    }
  },
  "http": {
    "port": 8000,
    "https": {
      "port": 8443,
      "key": "server.key",
      "cert": "server.crt"
    }
  },
  "auth": {
    "play": true,
    "publish": true,
    "secret": "your-secret-key"
  },
  "jwt": {
    "users": [
      {"username": "admin", "password": "admin", "role": "admin"}
    ]
  }
}

事件系统

生命周期事件

preConnect: 连接建立前
postConnect: 连接建立后
prePublish: 推流开始前
postPublish: 推流开始后
prePlay: 播放开始前
postPlay: 播放开始后
donePublish: 推流结束
donePlay: 播放结束

事件使用示例

javascript

nms.on('prePublish', (session) => {
  console.log(`Stream ${session.streamPath} starting to publish`);
});

nms.on('donePlay', (session) => {
  console.log(`Player ${session.id} stopped playing`);
});

性能优化

1. 内存管理

GOP缓存大小限制 (4096帧)
会话及时清理
流数据自动释放

2. 网络优化

TCP_NODELAY启用
缓冲区大小优化
零拷贝数据传输

3. 编解码优化

硬件编码支持
快速格式转换
延迟优化算法

这个架构设计确保了Node-Media-Server能够高效处理大规模实时流媒体需求，同时保持良好的可扩展性和维护性。

Api (`docs/api.md`)

REST API (New in v4.2.0)

Node-Media-Server v4.2.0 provides a REST API for server management and monitoring: stream/session management, relay task control, real-time statistics, and health checks, protected by JWT authentication.

Overview

Base URL: http://server_ip:8000/api/v1 (or https://server_ip:8443/api/v1)
The API service is activated automatically when the auth.jwt section is present in the configuration file (see Configuration).
All endpoints require a JWT token, except POST /login and GET /health.
All responses use a consistent format:

json

{
  "success": true,
  "data": {},
  "message": "Optional message"
}

Authentication

Submit the configured username and password directly:

bash

POST /api/v1/login
Content-Type: application/json

{
  "username": "your_username",
  "password": "your_password"
}

Response:

json

{
  "success": true,
  "data": {
    "token": "your_jwt_token",
    "user": {
      "username": "your_username"
    },
    "expiresIn": "24h"
  },
  "message": "Login successful"
}

Passwords are stored in the config file as scrypt hashes (scrypt$N$r$p$salt$hash). Legacy plaintext entries are transparently upgraded to hashes on the first successful login or at server startup. Use HTTPS in production — the plaintext password crosses the network during login.

Brute-force protection:

Per-IP rate limit: POST /api/v1/login is limited to 10 requests per minute per IP (HTTP 429 when exceeded).
Account lockout: after 5 consecutive failed logins for the same username from the same IP, further attempts are rejected with HTTP 429 for 15 minutes; a successful login resets the counter.

Using the API

Include the JWT token in your requests:

bash

Authorization: Bearer your_jwt_token

The token is also accepted as a query parameter: ?token=your_jwt_token.

Requests with a missing or invalid token receive:

json

{
  "success": false,
  "error": "Invalid or missing authentication token",
  "code": "UNAUTHORIZED"
}

Change Password

bash

POST /api/v1/password
Authorization: Bearer your_jwt_token
Content-Type: application/json

{
  "oldPassword": "your_current_password",
  "newPassword": "your_new_password"
}

Validates the old password against the configured user, then updates auth.jwt.users in the running config and persists it back to the config file (when started via the CLI). Requirements: newPassword ≥ 6 characters and different from the old one. A wrong oldPassword returns 400 (a validation error, not an auth failure — the JWT itself is valid). After a successful change, log in again with the new password — already-issued tokens stay valid until they expire.

json

{
  "success": true,
  "data": {},
  "message": "Password changed successfully, please log in again"
}

Endpoints

Method	Path	Description	Auth
POST	/api/v1/login	Username/password login	No
POST	/api/v1/password	Change the current user's password	Yes
GET	/api/v1/health	Server health check	No
GET	/api/v1/info	Server version and configuration information	Yes
GET	/api/v1/streams	List all active streams	Yes
GET	/api/v1/streams/:app/:name	Get details of a specific stream	Yes
GET	/api/v1/streams/:app/:name/urls	Generate ready-to-use play URLs (signed when auth.play is on)	Yes
GET	/api/v1/streams/:app/:name/record	Query the recording status of a stream	Yes
POST	/api/v1/streams/:app/:name/record	Manually start recording a stream	Yes
DELETE	/api/v1/streams/:app/:name/record	Manually stop recording a stream	Yes
GET	/api/v1/sessions	List all connected sessions	Yes
DELETE	/api/v1/sessions/:id	Terminate a specific session	Yes
GET	/api/v1/stats	Real-time server performance statistics	Yes
GET	/api/v1/events	Stream lifecycle events over SSE	Yes
GET	/api/v1/relay	List all relay tasks	Yes
GET	/api/v1/relay/:streamPath	Get status of a specific relay task	Yes
POST	/api/v1/relay	Add a relay (pull/push) task	Yes
DELETE	/api/v1/relay	Remove a relay task	Yes
GET	/api/v1/records	List recording metadata (persisted)	Yes
GET	/api/v1/records/:id	Get one recording	Yes
DELETE	/api/v1/records/:id	Delete a recording (?file=true also deletes the flv file)	Yes
GET	/api/v1/history	List persisted publish/play history	Yes
DELETE	/api/v1/history	Clear history (?streamPath= limits the scope)	Yes

Health Check

bash

GET /api/v1/health

Returns the server health status and version.

json

{
  "success": true,
  "data": {
    "status": "ok",
    "timestamp": "2026-08-22T00:00:00.000Z",
    "version": "4.0.0"
  },
  "message": "Server is healthy"
}

Server Information

bash

GET /api/v1/info

Returns server metadata, an overview of the active configuration (ports, static/record/auth switches), and uptime.

json

{
  "success": true,
  "data": {
    "server": {
      "name": "node-media-server",
      "version": "4.0.0",
      "homepage": "https://github.com/illuspas/Node-Media-Server",
      "license": "Apache-2.0",
      "author": {}
    },
    "config": {
      "bind": "0.0.0.0",
      "rtmp_port": 1935,
      "rtmps_port": 1936,
      "http_port": 8000,
      "https_port": 8443,
      "static_enabled": false,
      "record_enabled": false,
      "auth_enabled": false
    },
    "uptime": 3600,
    "node_version": "v18.0.0"
  },
  "message": "Server information retrieved successfully"
}

Stream Management

bash

GET /api/v1/streams

List all active streams with detailed information including codecs, resolution, framerate, and subscriber count. status is the real publish state: publishing (live), reconnecting (publisher dropped, the stream is held for the grace window awaiting the same client), or idle (no publisher, e.g. only waiting players).

Traffic fields: inBytes is the publisher's cumulative input; outBytes is the total sent to the currently attached subscribers; inBps/outBps are real-time publish/play rates in bytes per second (multiply by 8 for bit/s), averaged over a sliding ~5-second sampling window maintained server-side. A publisher swap rebases the window instead of producing negative rates. These three fields are available on both the list and the single-stream endpoints.

json

{
  "success": true,
  "data": {
    "streams": [
      {
        "key": "/live/stream",
        "app": "live",
        "name": "stream",
        "status": "publishing",
        "publisher": {
          "id": "session_id",
          "ip": "192.168.1.100",
          "protocol": "rtmp",
          "createTime": 1724280000000,
          "videoCodec": "h264",
          "videoWidth": 1920,
          "videoHeight": 1080,
          "videoFramerate": 30,
          "audioCodec": "aac",
          "audioChannels": 2,
          "audioSamplerate": 44100,
          "inBytes": 1048576
        },
        "subscribers": 3,
        "outBytes": 3145728,
        "inBps": 262144,
        "outBps": 524288,
        "recording": false
      }
    ],
    "total": 1
  },
  "message": "Streams retrieved successfully"
}

Get a single stream (404 if the stream does not exist):

bash

GET /api/v1/streams/live/stream

The response data contains the same stream object shown above.

Play URL Generation

bash

GET /api/v1/streams/{app}/{name}/urls[?ttl=seconds]

Generates ready-to-use play URLs for a stream across every configured listener: rtmp, rtmps, http-flv, ws-flv, https-flv and wss-flv. It works whether or not the stream is currently live (status reports the current publish state, idle when the stream does not exist yet), which makes it handy for handing out links before a broadcast starts:

Only listeners with a port in the configuration are included.
The host is taken from the request's Host header, so the returned URLs are directly reachable from wherever the API call was made.
Protocol default ports (rtmp 1935, http 80, https 443) are omitted from the URLs.

When auth.play is enabled with a non-empty auth.secret, every URL carries a sign query parameter (<expiryUnixSeconds>-<md5hex>) that the server's signed-URL verification accepts for RTMP, HTTP-FLV and WebSocket-FLV playback alike. The optional ttl query parameter bounds the signature lifetime in seconds (clamped to 60–86400, default 3600) and is ignored when signing is disabled.

json

{
  "success": true,
  "data": {
    "app": "live",
    "name": "cam1",
    "streamPath": "/live/cam1",
    "status": "publishing",
    "signed": true,
    "expiresAt": "2026-09-10T04:46:57.000Z",
    "urls": [
      { "protocol": "rtmp", "url": "rtmp://192.168.1.2/live/cam1?sign=1789015617-04ccea110655266f5941a4673e2d3646" },
      { "protocol": "http-flv", "url": "http://192.168.1.2:8000/live/cam1.flv?sign=1789015617-04ccea110655266f5941a4673e2d3646" },
      { "protocol": "ws-flv", "url": "ws://192.168.1.2:8000/live/cam1.flv?sign=1789015617-04ccea110655266f5941a4673e2d3646" }
    ]
  },
  "message": "Play URLs generated successfully"
}

Manual Recording

bash

POST /api/v1/streams/{app}/{name}/record

Manually start recording a publishing stream (the webadmin record button). Fails with 400 if the record path is not configured/writable, the stream has no publisher, or it is already recording. Response data is { recordId, filePath }.

With record.auto: false in the config, published streams are not recorded automatically and this endpoint is the only way to record — combine it with the DELETE endpoint for full manual control. Toggling record.auto takes effect immediately.

bash

DELETE /api/v1/streams/{app}/{name}/record

Manually stop the active recording of a stream. The recording metadata is finalized in the records store. Fails with 400 if the stream is not recording.

bash

GET /api/v1/streams/{app}/{name}/record

Query whether the stream currently has an active recording session. Response data is { recording, recordId, filePath, startTime } — when not recording, recording is false and the other fields are omitted. Fails with 400 if the record server is not available. For historical (finalized) recordings use the records endpoints below.

Session Management

bash

GET /api/v1/sessions

Monitor all connected clients (publishers and players) with session details: protocol, stream app/name, type, bytes in/out, and creation time. The response data contains { sessions: [...], total }.

bash

DELETE /api/v1/sessions/{sessionId}

Terminate a specific session by ID. This disconnects the associated client and stops their stream or playback. Returns 404 if the session does not exist.

Response:

json

{
  "success": true,
  "data": {
    "id": "sessionId"
  },
  "message": "Session deleted successfully"
}

Server Statistics

bash

GET /api/v1/stats

Real-time server performance metrics including:

CPU usage (process.cpuUsage())
Memory consumption (RSS, heap total, heap used)
Process uptime, Node.js version, platform, and PID
Connected client count, split into publishers and players (publishers equals the active stream count)
Cumulative streaming network traffic in/out bytes, accumulated by every publisher/player session over the process lifetime (record file writes excluded), plus real-time inBps/outBps rates in bytes per second averaged over a sliding ~5-second window

json

{
  "success": true,
  "data": {
    "server": {
      "uptime": 3600,
      "nodeVersion": "v18.0.0",
      "platform": "darwin",
      "arch": "arm64",
      "pid": 12345
    },
    "cpu": { "user": 1200000, "system": 300000 },
    "memory": { "rss": 104857600, "heapTotal": 52428800, "heapUsed": 31457280 },
    "sessions": { "total": 4, "publishers": 1, "players": 3 },
    "network": { "inBytes": 1048576000, "outBytes": 3145728000, "inBps": 262144, "outBps": 786432 },
    "timestamp": "2026-08-22T00:00:00.000Z"
  },
  "message": "Server statistics retrieved successfully"
}

Event Stream (SSE)

bash

GET /api/v1/events
Authorization: Bearer your_jwt_token

Server-Sent Events feed of stream lifecycle events — the same events the webhook notifier (notify.url) receives: prePublish, postPublish, donePublish, prePlay, postPlay, donePlay, postRecord and doneRecord. Each event carries a session summary as JSON in data:

json

{
  "id": 42,
  "ip": "192.168.1.100:51234",
  "app": "live",
  "name": "cam1",
  "streamPath": "/live/cam1",
  "protocol": "rtmp",
  "isPublisher": true,
  "createTime": 1724280000000,
  "endTime": null,
  "inBytes": 1048576,
  "outBytes": 0
}

The server sends a retry: 3000 hint on connect and a : ping comment every 15 seconds to keep proxies from buffering or closing the idle connection; event listeners are cleaned up when the client disconnects. Play events are only emitted for external clients (sessions with a real peer address), matching the webhook behavior.

bash

curl -N -H "Authorization: Bearer your_jwt_token" http://localhost:8000/api/v1/events

event: postPublish
data: {"id":42,"ip":"192.168.1.100:51234","app":"live","name":"cam1","streamPath":"/live/cam1","protocol":"rtmp","isPublisher":true,"createTime":1724280000000,"endTime":null,"inBytes":0,"outBytes":0}

Relay Management

Relay tasks pull an RTSP/RTMP source into the server, or push a local stream out to an RTMP destination.

List relay tasks

bash

GET /api/v1/relay

Returns { tasks: [...], count } with the status of every relay task.

Get a task's status

bash

GET /api/v1/relay/live/stream

Returns the status of the pull task bound to /live/stream (URL-encoded stream path). Returns 404 if the task does not exist.

Add a relay task

bash

POST /api/v1/relay
Content-Type: application/json

{
  "url": "rtsp://192.168.1.100:554/camera/1",
  "mode": "pull",
  "streamPath": "/live/camera1",
  "transport": "tcp",
  "reconnect": true,
  "reconnectInterval": 5000,
  "maxReconnectAttempts": 10
}

Field	Required	Description
url	Yes	RTSP or RTMP source/destination URL. `rtspUrl` is accepted as a legacy alias
mode	No	`pull` (default) or `push`. RTSP only supports pull; RTMP supports both pull and push
streamPath	Yes	Local stream path, must start with `/` (e.g. `/live/camera1`)
transport	No	RTSP transport, default `tcp`
reconnect	No	Reconnect on failure, default `true`
reconnectInterval	No	Reconnect interval in milliseconds
maxReconnectAttempts	No	Maximum reconnect attempts

Response contains the new task's status in data.

Remove a relay task

bash

DELETE /api/v1/relay
Content-Type: application/json

{
  "streamPath": "/live/camera1"
}

For push tasks, pass either the full task key (taskKey: "push:rtmp://dest/live/stream") or mode: "push" together with url. Returns 404 if the task does not exist.

Relay tasks are persisted in the lightweight store (store.path, default ./data/relay_tasks.json) and restored automatically on restart.

Recordings

List recordings

bash

GET /api/v1/records?status=done&streamPath=/live/camera1&page=1&pageSize=20

Query params: status (recording | done), streamPath, page (1-based), pageSize (max 100). Response data is a page object: { items, count, page, pageSize, totalDuration, totalSize }, where each item contains id, streamPath, app, name, filePath, publisherId, startTime, endTime, duration (ms), size (bytes), status and count/aggregates cover the whole filter.

Delete a recording

bash

DELETE /api/v1/records/<id>         # remove the metadata entry only
DELETE /api/v1/records/<id>?file=true  # also delete the flv file on disk

Deleting the file is only allowed for paths inside the configured record.path. Recordings currently in progress (status recording) are protected with 409 — kick the publisher session (publisherId) instead.

Session History

Publish history only: plays are not stored as individual rows. Each play increments the stream's cumulative counter (persisted), and every publish entry carries that stream's playCount (历史播放量) as of the moment the publish ended.

List history

bash

GET /api/v1/history?streamPath=/live/camera1&ip=1.2.3.4&page=1&pageSize=20

Query params: streamPath, ip, protocol (exact matches), search (substring match on streamPath or ip), page, pageSize. Response data is a page object: { items, count, page, pageSize }; each item contains id, protocol, streamPath, app, name, ip, startTime, endTime, duration, inBytes, outBytes, playCount. History is capped at store.maxHistory publish entries (default 10000, oldest evicted first).

Clear history

bash

DELETE /api/v1/history                   # clear everything (resets all play counters)
DELETE /api/v1/history?streamPath=/live/camera1  # clear one stream's history and its play counter

Configuration

The API system is configured through the auth.jwt section of the configuration file (e.g. bin/config.json):

json

"store": {
    "path": "./data",
    "maxHistory": 10000
},
"auth": {
    "play": false,
    "publish": false,
    "secret": "<random-secret-for-signed-play/publish-URLs>",
    "jwt": {
        "secret": "3e64abe6a00088e5039452d1ea1c854af7e4cc6ec30c129547b44f89604a6164",
        "expiresIn": "24h",
        "refreshExpiresIn": "7d",
        "algorithm": "HS256",
        "users": [
            {
                "username": "admin",
                "password": "your_password",
                "role": "admin"
            }
        ]
    }
}

Security Features

Password Storage

Passwords in auth.jwt.users are stored as scrypt hashes (scrypt$N$r$p$salt$hash, hashed with crypto.scryptSync and compared via crypto.timingSafeEqual). Legacy plaintext entries are migrated automatically at startup or on first login. Because the plaintext password is submitted during login, use HTTPS in production.

JWT Configuration

Configurable secret key for token signing
Support for different algorithms (HS256, HS384, HS512)
Configurable token expiration times
Refresh token support for extended sessions

Example Usage

Using curl

bash

# Login
curl -X POST http://localhost:8000/api/v1/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"your_password"}'

# Get server stats
curl -X GET http://localhost:8000/api/v1/stats \
  -H "Authorization: Bearer your_jwt_token"

# Get active streams
curl -X GET http://localhost:8000/api/v1/streams \
  -H "Authorization: Bearer your_jwt_token"

# Generate play URLs for a stream (signed when auth.play is enabled)
curl -X GET "http://localhost:8000/api/v1/streams/live/stream/urls?ttl=600" \
  -H "Authorization: Bearer your_jwt_token"

# Follow stream lifecycle events (SSE)
curl -N http://localhost:8000/api/v1/events \
  -H "Authorization: Bearer your_jwt_token"

# Get all sessions
curl -X GET http://localhost:8000/api/v1/sessions \
  -H "Authorization: Bearer your_jwt_token"

# Delete a specific session
curl -X DELETE http://localhost:8000/api/v1/sessions/abc123-def456-ghi789 \
  -H "Authorization: Bearer your_jwt_token"

# Add an RTSP pull relay task
curl -X POST http://localhost:8000/api/v1/relay \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your_jwt_token" \
  -d '{"url":"rtsp://192.168.1.100:554/camera/1","streamPath":"/live/camera1"}'

# Remove a relay task
curl -X DELETE http://localhost:8000/api/v1/relay \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your_jwt_token" \
  -d '{"streamPath":"/live/camera1"}'

Using JavaScript

javascript

// Login
const loginRes = await fetch('http://localhost:8000/api/v1/login', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ username: 'admin', password: 'your_password' })
});
const { data: { token } } = await loginRes.json();

// Get streams
const streamsResponse = await fetch('http://localhost:8000/api/v1/streams', {
  headers: { 'Authorization': `Bearer ${token}` }
});

const streams = await streamsResponse.json();
console.log('Active streams:', streams);

// Get all sessions
const sessionsResponse = await fetch('http://localhost:8000/api/v1/sessions', {
  headers: { 'Authorization': `Bearer ${token}` }
});

const sessions = await sessionsResponse.json();
console.log('Active sessions:', sessions);

// Delete a specific session
if (sessions.data.sessions.length > 0) {
  const sessionIdToDelete = sessions.data.sessions[0].id;
  const deleteResponse = await fetch(`http://localhost:8000/api/v1/sessions/${sessionIdToDelete}`, {
    method: 'DELETE',
    headers: { 'Authorization': `Bearer ${token}` }
  });

  const deleteResult = await deleteResponse.json();
  console.log('Session deletion result:', deleteResult);
}

Data Flow (`docs/data-flow.md`)

Node-Media-Server 数据流分析

数据流架构总览

Node-Media-Server 采用了分层的数据流架构，确保了高效的媒体数据处理和多协议兼容性。整个数据流从推流端到播放端经历了多个处理阶段。

核心数据结构

1. AVPacket - 统一媒体数据包

AVPacket 是整个系统中最核心的数据结构，用于在不同组件间传递音视频数据：

javascript

class AVPacket {
  constructor() {
    this.codec_type = 0;     // 编码器类型: 8=音频, 9=视频, 18=脚本
    this.codec_id = 0;       // 编码器ID
    this.pts = 0;           // 显示时间戳
    this.dts = 0;           // 解码时间戳
    this.size = 0;          // 数据大小
    this.flags = 0;         // 标志位(AVPacket.FLAG_*): 0=音频序列头, 1=音频帧, 2=视频序列头, 3=关键帧, 4=普通帧, 5=元数据, 6=视频元数据
    this.data = Buffer.alloc(0); // 原始媒体数据
  }
}

2. Session - 会话数据结构

javascript

class BaseSession {
  constructor() {
    this.id = '';              // 唯一会话ID
    this.ip = '';              // 客户端IP地址
    this.protocol = '';        // 协议类型: "rtmp", "flv"
    this.streamApp = '';       // 应用名称
    this.streamName = '';      // 流名称
    this.streamPath = '';      // 完整流路径
    this.isPublisher = false;  // 是否为发布者
    this.inBytes = 0;          // 输入字节数
    this.outBytes = 0;         // 输出字节数
    this.createTime = 0;       // 创建时间
    this.endTime = 0;          // 结束时间
  }
}

推流数据流分析

1. RTMP推流数据流

mermaid

flowchart TD
    A[推流客户端] --> B[TCP Socket]
    B --> C[RTMP协议解析]
    C --> D[分块重组]
    D --> E[RTMP消息解析]
    E --> F[AMF数据解析]
    F --> G[创建AVPacket]
    G --> H[BroadcastServer]
    H --> I[协议格式转换]
    I --> J[RTMP格式包]
    I --> K[FLV格式包]
    J --> L[RTMP播放器]
    K --> M[FLV播放器]

    style A fill:#e1f5fe
    style L fill:#e8f5e8
    style M fill:#fff3e0
    style H fill:#f3e5f5

详细数据转换过程：

步骤1: RTMP消息到AVPacket

javascript

// RTMP Protocol处理
static parserTag(type, time, size, data) {
  let packet = new AVPacket();
  packet.codec_type = type;
  packet.pts = time;
  packet.dts = time;
  packet.size = size;
  packet.data = data;

  // 解析视频帧类型和编码器
  if (type === FLV_MEDIA_TYPE_VIDEO) {
    const frameType = data[0] >> 4 & 0b0111;
    const codecID = data[0] & 0x0f;

    // 处理现代编码器 (H.265, VP9, AV1)
    if (isExHeader) {
      const fourCC = data.subarray(1, 5);
      if (fourCC === 'av01' || fourCC === 'vp09' || fourCC === 'hvc1') {
        packet.codec_id = fourCC.readUint32BE();
      }
    }

    // 设置标志位
    if (VideoPacketType === VideoPacketTypeSequenceStart) {
      packet.flags = 2; // 视频头
    } else if (frameType === FLV_FRAME_KEY) {
      packet.flags = 3; // 关键帧
    } else {
      packet.flags = 4; // 普通帧
    }
  }

  return packet;
}

步骤2: AVPacket到多协议格式

javascript

// BroadcastServer处理
broadcastMessage(packet) {
  // 转换为FLV格式
  const flvMessage = Flv.createMessage(packet);

  // 转换为RTMP格式
  const rtmpMessage = Rtmp.createMessage(packet);

  // 根据包类型进行缓存管理
  switch (packet.flags) {
    case 0: // 音频头
      this.flvAudioHeader = Buffer.from(flvMessage);
      this.rtmpAudioHeader = Buffer.from(rtmpMessage);
      break;
    case 2: // 视频头
      this.flvVideoHeader = Buffer.from(flvMessage);
      this.rtmpVideoHeader = Buffer.from(rtmpMessage);
      break;
    case 3: // 关键帧 - 清理并重建GOP缓存
      this.flvGopCache?.clear();
      this.rtmpGopCache?.clear();
      this.flvGopCache = new Set();
      this.rtmpGopCache = new Set();
      this.flvGopCache.add(flvMessage);
      this.rtmpGopCache.add(rtmpMessage);
      break;
  }

  // 广播给所有订阅者
  this.subscribers.forEach((session) => {
    switch (session.protocol) {
      case "flv":
        session.sendBuffer(flvMessage);
        break;
      case "rtmp":
        session.sendBuffer(rtmpMessage);
        break;
    }
  });
}

2. GOP缓存数据流

GOP (Group of Pictures) 缓存是新播放器快速获得关键帧的重要机制：

mermaid

flowchart LR
    A[关键帧] --> B[加入GOP缓存]
    B --> C[缓存限制检查]
    C --> D{超过4096帧?}
    D -->|是| E[移除最旧帧]
    E --> B
    D -->|否| F[新播放器连接]
    F --> G[发送完整GOP]
    G --> H[实时流数据]

    style A fill:#ffcdd2
    style G fill:#c8e6c9
    style H fill:#fff9c4

GOP缓存管理代码：

javascript

// 在BroadcastServer中管理GOP缓存
broadcastMessage(packet) {
  // ... 其他处理 ...

  if (packet.flags === 3) { // 关键帧
    // 清理现有缓存
    this.flvGopCache?.clear();
    this.rtmpGopCache?.clear();

    // 重新初始化缓存
    this.flvGopCache = new Set();
    this.rtmpGopCache = new Set();
    this.flvGopCache.add(flvMessage);
    this.rtmpGopCache.add(rtmpMessage);
  } else if (packet.flags === 4) { // 普通帧
    // 添加到缓存，检查大小限制
    if (this.flvGopCache?.size < 4096) {
      this.flvGopCache?.add(flvMessage);
      this.rtmpGopCache?.add(rtmpMessage);
    }
  }
}

播放数据流分析

1. 新播放器连接数据流

mermaid

sequenceDiagram
    participant Player as 播放器
    participant BroadcastServer as 广播服务器
    participant GopCache as GOP缓存
    participant LiveStream as 实时流

    Player->>+BroadcastServer: 请求播放
    BroadcastServer->>Player: 发送FLV头部
    BroadcastServer->>Player: 发送元数据
    BroadcastServer->>Player: 发送音视频头
    BroadcastServer->>+GopCache: 获取GOP缓存
    GopCache-->>-BroadcastServer: 返回缓存帧
    BroadcastServer->>Player: 发送GOP缓存
    BroadcastServer->>+LiveStream: 订阅实时流
    LiveStream-->>-Player: 实时数据传输

2. HTTP-FLV数据流

mermaid

flowchart TD
    A[HTTP请求] --> B[FlvSession创建]
    B --> C[流路径解析]
    C --> D[获取BroadcastServer]
    D --> E[身份验证]
    E --> F[发送FLV头部]
    F --> G[发送流元数据]
    G --> H[发送GOP缓存]
    H --> I[实时数据流]
    I --> J[HTTP响应流]

    style A fill:#e3f2fd
    style J fill:#e8f5e8
    style D fill:#f3e5f5

HTTP-FLV数据发送实现：

javascript

// FlvSession中的数据发送
sendBuffer(buffer) {
  if (this.res.writableEnded) {
    return;
  }
  this.res.write(buffer);
  this.outBytes += buffer.length;
}

// 从BroadcastServer接收数据
postPlay(session) {
  // 发送头部信息
  session.sendBuffer(this.flvHeader);
  if (this.flvMetaData !== null) {
    session.sendBuffer(this.flvMetaData);
  }

  // 发送GOP缓存
  if (this.flvGopCache !== null) {
    this.flvGopCache.forEach((frame) => {
      session.sendBuffer(frame);
    });
  }

  // 添加到订阅者列表
  this.subscribers.set(session.id, session);
}

3. WebSocket-FLV数据流

mermaid

flowchart TD
    A[WebSocket连接] --> B[FlvSession创建]
    B --> C[URL路径解析]
    C --> D[获取BroadcastServer]
    D --> E[身份验证]
    E --> F[发送FLV头部]
    F --> G[发送流元数据]
    G --> H[发送GOP缓存]
    H --> I[实时WebSocket消息]

    style A fill:#e3f2fd
    style I fill:#fff3e0
    style D fill:#f3e5f5

WebSocket-FLV数据发送实现：

javascript

// FlvSession中的WebSocket数据发送
sendBuffer(buffer) {
  if (this.res.readyState !== WebSocket.OPEN) {
    return;
  }
  this.res.send(buffer);
  this.outBytes += buffer.length;
}

协议转换数据流

1. RTMP到FLV转换

mermaid

flowchart LR
    A[RTMP数据包] --> B[提取媒体数据]
    B --> C[解析编码器类型]
    C --> D[创建FLV标签头]
    D --> E[设置时间戳]
    E --> F[添加媒体数据]
    F --> G[添加PreviousTagSize]
    G --> H[FLV数据包]

    style A fill:#ffebee
    style H fill:#e8f5e8

RTMP到FLV转换代码：

javascript

// Flv.createMessage实现
static createMessage(avpacket) {
  const buffer = Buffer.alloc(11 + avpacket.size + 4);

  // FLV标签头
  buffer[0] = avpacket.codec_type;
  buffer.writeUintBE(avpacket.size, 1, 3);          // 数据大小
  buffer.writeUintBE(avpacket.dts & 0x00ffffff, 4, 3); // 时间戳低24位
  buffer[7] = (avpacket.dts >> 24) & 0xff;         // 时间戳扩展
  buffer[8] = 0;  // StreamID (始终为0)
  buffer[9] = 0;
  buffer[10] = 0;

  // 媒体数据
  avpacket.data.copy(buffer, 11, 0, avpacket.size);

  // PreviousTagSize
  buffer.writeUint32BE(11 + avpacket.size, 11 + avpacket.size);

  return buffer;
}

2. FLV到RTMP转换

mermaid

flowchart LR
    A[FLV数据包] --> B[解析标签头]
    B --> C[提取时间戳]
    C --> D[提取媒体数据]
    D --> E[创建RTMP消息]
    E --> F[设置消息头]
    F --> G[添加媒体数据]
    G --> H[RTMP数据包]

    style A fill:#e8f5e8
    style H fill:#ffebee

元数据处理流程

1. 推流端元数据

mermaid

flowchart TD
    A[推流端元数据] --> B[AMF编码]
    B --> C[RTMP消息]
    C --> D[AVPacket创建]
    D --> E[解码AMF数据]
    E --> F[提取流信息]
    F --> G[更新Session信息]
    G --> H[存储元数据]
    H --> I[分发到播放器]

    style A fill:#fff3e0
    style G fill:#e1f5fe
    style I fill:#e8f5e8

元数据处理实现：

javascript

// BroadcastServer中的元数据处理
if (packet.flags == 5) { // 元数据包
  let metadata = decodeAmf0Data(packet.data);
  if (this.publisher && metadata.cmd === "@setDataFrame") {
    // 提取音频参数
    this.publisher.audioCodec = metadata.dataObj.audiocodecid;
    this.publisher.audioChannels = metadata.dataObj.stereo ? 2 : 1;
    this.publisher.audioSamplerate = metadata.dataObj.audiosamplerate;

    // 提取视频参数
    this.publisher.videoCodec = metadata.dataObj.videocodecid;
    this.publisher.videoWidth = metadata.dataObj.width;
    this.publisher.videoHeight = metadata.dataObj.height;
    this.publisher.videoFramerate = metadata.dataObj.framerate;
  }
}

统计和监控数据流

1. 会话统计数据流

mermaid

flowchart TD
    A[会话活动] --> B[字节计数更新]
    B --> C[实时状态更新]
    C --> D[Context.sessions]
    D --> E[API统计查询]
    E --> F[统计响应]
    F --> G[监控仪表板]

    style A fill:#e8eaf6
    style D fill:#f3e5f5
    style G fill:#e8f5e8

2. 系统性能数据流

mermaid

flowchart LR
    A[Node.js进程] --> B[内存使用统计]
    A --> C[CPU使用统计]
    B --> D[Process.memoryUsage]
    C --> E[Process.cpuUsage]
    D --> F[StatsHandler]
    E --> F
    F --> G[API响应]
    G --> H[监控系统]

    style A fill:#fff3e0
    style F fill:#e1f5fe
    style H fill:#c8e6c9

统计数据处理：

javascript

// StatsHandler中的统计收集
getStats() {
  const memoryUsage = process.memoryUsage();
  const cpuUsage = process.cpuUsage();

  // 会话统计
  const totalSessions = Context.sessions.size;
  const publishers = Array.from(Context.sessions.values())
    .filter(s => s.isPublisher).length;
  const players = totalSessions - publishers;

  // 流统计
  const totalStreams = Context.broadcasts.size;
  const activeStreams = Array.from(Context.broadcasts.values())
    .filter(b => b.publisher !== null).length;

  return {
    system: {
      uptime: process.uptime(),
      memory: memoryUsage,
      cpu: cpuUsage
    },
    sessions: {
      total: totalSessions,
      publishers: publishers,
      players: players
    },
    streams: {
      total: totalStreams,
      active: activeStreams
    }
  };
}

错误处理数据流

1. 会话错误处理

mermaid

flowchart TD
    A[会话异常] --> B[错误捕获]
    B --> C[日志记录]
    C --> D[资源清理]
    D --> E[会话移除]
    E --> F[事件通知]
    F --> G[状态恢复]

    style A fill:#ffebee
    style C fill:#fff3e0
    style G fill:#e8f5e8

2. 协议错误处理

mermaid

flowchart LR
    A[协议错误] --> B[错误分类]
    B --> C{致命错误?}
    C -->|是| D[关闭连接]
    C -->|否| E[恢复处理]
    D --> F[会话清理]
    E --> G[继续处理]

    style A fill:#ffebee
    style D fill:#ffcdd2
    style G fill:#c8e6c9

数据流优化策略

1. 内存优化

GOP缓存大小限制 (4096帧)
及时清理过期会话
Buffer复用机制

2. 网络优化

零拷贝数据传输
TCP_NODELAY设置
缓冲区大小优化

3. 并发优化

事件驱动架构
非阻塞I/O
负载均衡分发

这个数据流架构确保了Node-Media-Server能够高效处理大规模实时媒体数据，同时保持低延迟和高吞吐量。

Enhanced Rtmp Multitrack Audio (`docs/enhanced-rtmp-multitrack-audio.md`)

Enhanced RTMP v2 多轨音频：OBS 补丁与测试指南

本文档记录如何让 OBS 通过 RTMP 推送多轨音频（Enhanced RTMP v2 multitrack audio），
以及如何在 Node-Media-Server（NMS）侧验证该功能。

相关背景：GitHub Issue #707
（multitrack 封装内的音频 sequence header 未被识别，导致迟到订阅者丢失音轨配置）。
协议规范全文见 enhanced-rtmp-v2.md。

1. 多轨音频的线上格式速览

OBS / FFmpeg 推流时，多轨音频按 Enhanced RTMP v2 规则封装为 FLV audio tag：

轨 0（首轨）：传统 legacy 封装，保持对旧播放器的兼容
- sequence header：af 00 <AudioSpecificConfig>
- 音频帧：af 01 <AAC frame>
轨 N（N ≥ 1）：multitrack 封装（OneTrack 布局）
- sequence header：95 00 <FourCC> <AudioSpecificConfig>
- 音频帧：95 01 <FourCC> <AAC frame>
- 字节 0 低 4 位恒为 AudioPacketTypeMultitrack(5)；字节 1 低 4 位才是
  内层包类型（0=SequenceStart, 1=CodedFrames）；字节 2-5 为 FourCC（如
  mp4a）；字节 6 为 trackIndex

实测抓包（OBS 三音轨推流，迟到订阅者收到的头部）：

text

轨0 legacy AAC header ASC=1190                        (AAC-LC, 48kHz, 立体声)
轨1 multitrack header fourcc=mp4a trackIdx=1 ASC=1190
轨2 multitrack header fourcc=mp4a trackIdx=2 ASC=1190

2. OBS 补丁

2.1 背景：为什么原版 OBS 找不到多轨设置

OBS 的推流输出层（plugins/obs-outputs）已完整实现 multitrack 音频封装：

rtmp-stream.c send_audio_header() / 音频帧发送路径：
idx == 0 走 legacy plain 封装，idx > 0 走 send_audio_packet_ex()
（multitrack 封装）
flv-mux.c：AUDIO_PACKET_TYPE_MULTITRACK = 5、
MULTITRACKTYPE_ONE_TRACK、FourCC + trackIndex 布局，全部就绪

但前端在两处把多轨音频限制为 SRT / RIST 协议，RTMP 服务下设置界面
（设置 → 输出 → 推流 → 音频）只显示单轨单选框：

frontend/settings/OBSBasicSettings_Stream.cpp 约 L250 AllowsMultiTrack()
frontend/utility/AdvancedOutput.cpp 约 L281 allowsMultiTrack()

2.2 补丁内容

两个函数做相同修改，放行 RTMP 协议（基于 OBS 33.0.0-rc1，commit 1ad611018）：

diff

--- a/frontend/settings/OBSBasicSettings_Stream.cpp
+++ b/frontend/settings/OBSBasicSettings_Stream.cpp
@@ -250,7 +250,8 @@
 bool OBSBasicSettings::AllowsMultiTrack(const char *protocol)
 {
 	return astrcmpi_n(protocol, SRT_PROTOCOL, strlen(SRT_PROTOCOL)) == 0 ||
-	       astrcmpi_n(protocol, RIST_PROTOCOL, strlen(RIST_PROTOCOL)) == 0;
+	       astrcmpi_n(protocol, RIST_PROTOCOL, strlen(RIST_PROTOCOL)) == 0 ||
+	       astrcmpi_n(protocol, "RTMP", 4) == 0;
 }

--- a/frontend/utility/AdvancedOutput.cpp
+++ b/frontend/utility/AdvancedOutput.cpp
@@ -281,7 +281,8 @@
 inline bool AdvancedOutput::allowsMultiTrack()
 {
 	...
 	return astrcmpi_n(protocol, SRT_PROTOCOL, strlen(SRT_PROTOCOL)) == 0 ||
-	       astrcmpi_n(protocol, RIST_PROTOCOL, strlen(RIST_PROTOCOL)) == 0;
+	       astrcmpi_n(protocol, RIST_PROTOCOL, strlen(RIST_PROTOCOL)) == 0 ||
+	       astrcmpi_n(protocol, "RTMP", 4) == 0;
 }

补丁生效后，RTMP 服务下推流音频区从单选框变为 Track 1~6 复选框，
AdvOut/StreamMultiTrackAudioMixes 位掩码配置对 RTMP 输出生效。

2.3 编译（macOS）

前置条件：Xcode（含命令行工具）、CMake ≥ 3.28。OBS 的 macOS 构建强制使用
Xcode 生成器，且会自动下载预编译依赖（obs-deps、Qt6 macOS-universal、CEF、Sparkle）。

bash

cd obs-studio
cmake --preset macos          # 配置 + 下载依赖（约 1.5GB）
cmake --build build_macos --config Release --parallel 8
# 产物: build_macos/frontend/Release/OBS.app

注意事项（弱网/代理环境实测经验）：

CMake 内置下载器不认代理环境变量。若需走代理，先用 curl 手动把依赖包
下载到源码根的 .deps/ 目录（文件存在即跳过下载）。文件名与 SHA256 以
CMakePresets.json 的 dependencies 段为准，例如：
- macos-deps-<version>-universal.tar.xz
- macos-deps-qt6-<version>-universal.tar.xz
- cef_binary_<version>_macos_arm64.tar.xz（单架构构建取宿主架构）
Apple Git 走 HTTP 代理可能被 macOS 本地网络权限拒绝（curl 正常、git 报
No route to host）。子模块可用 curl 下载 tarball 解压替代：
https://codeload.github.com/obsproject/<repo>/tar.gz/<commit>，所需子模块为
plugins/obs-browser、plugins/obs-websocket、deps/libdshowcapture/src
（commit 见 git submodule status；CMake 只检查目录存在性）。
构建产物与正式版 OBS 共用 ~/Library/Application Support/obs-studio
配置目录，注意备份。

3. OBS 多轨推流配置

启动补丁版：open build_macos/frontend/Release/OBS.app
设置 → 推流：服务选自定义，服务器 rtmp://<nms-host>:1935/live，
推流码任选（如 obs）
设置 → 输出：输出模式改为高级 → 推流页签 → 音频区 →
勾选 Track 1 + Track 2（补丁后为复选框）
混音器 → 高级音频属性：把不同音源分配到不同轨道
（如桌面音频 → 轨道 1，麦克风 → 轨道 2）
音频编码保持 AAC，开始推流

4. 测试验证方法

4.1 不改 OBS 的快速验证（FFmpeg 8.0+）

FFmpeg 8.0 的 flv 封装器原生支持多轨音频输出（轨 0 legacy、后续轨
multitrack），可直接合成推流：

bash

ffmpeg -re -i video_with_audio.mp4 -i music.mp4 \
  -map 0:v:0 -map 1:a:0 -map 0:a:0 \
  -c copy -f flv rtmp://localhost:1935/live/mt

4.2 NMS 侧验证（含 #707 迟到订阅者场景）

bash

# 1) 流列表：应看到 N 路音频 + 1 路视频（在推流开始数秒后执行，
#    即为"迟到订阅者"，等价于 issue #707 的复现条件）
ffprobe -show_entries stream=index,codec_type,codec_name,sample_rate \
  rtmp://localhost:1935/live/mt

# 2) -c copy 逐轨拉取（修复前此处报 "sample rate not set"）
ffmpeg -i rtmp://localhost:1935/live/mt \
  -map 0:a:0 -c copy -f adts track0.aac \
  -map 0:a:1 -c copy -f adts track1.aac

# 3) 解码完整性：0 错误为通过
ffmpeg -v error -i track0.aac -f null -
ffmpeg -v error -i track1.aac -f null -

# 4) 试听指定音轨
ffplay -ast 1 rtmp://localhost:1935/live/mt

# 5) 转封装给通用播放器（VLC 等可切换音轨）
ffmpeg -i rtmp://localhost:1935/live/mt \
  -map 0:v -map 0:a -c copy -f mpegts out.ts

HTTP-FLV / WS-FLV 拉流同样有效（NMS 原样透传 multitrack tag）：
http://<nms-host>:8000/live/mt.flv

4.3 播放器支持矩阵（RTMP/HTTP-FLV 直连）

播放器	多轨支持	说明
ffplay（FFmpeg ≥ 8.0）	✅	`-ast N` 选轨或播放中 `a` 键切换音轨
mpv（链接 FFmpeg ≥ 8.0 编译）	✅	`#` 键切换音轨
FFmpeg 转封装 HLS/TS 后	✅	VLC / hls.js / ExoPlayer 等通用播放器
VLC 直连 RTMP	✅	更新到最新VLC, 可在播放中切换音轨
flv.js / nodeplayer.js	❌	Web 端 FLV 解复用仅支持单音轨

4.4 迟到订阅者收到的头部抓取（深度验证）

抓取 HTTP-FLV 并解析音频 sequence header，多轨场景应收到每轨各一条：

bash

curl -sN --max-time 4 http://localhost:8000/live/mt.flv -o late.flv

解析规则：audio tag 中 af 00 开头为轨 0 legacy header；
95 00 开头且字节 1 低 4 位为 0 的为轨 N multitrack header
（字节 6 = trackIndex，其后 2 字节为 AudioSpecificConfig）。

5. NMS 侧实现要点

src/protocol/flv.js：Flv.parserTag 对 AudioPacketTypeMultitrack
包检查字节 1 低 4 位的内层类型，正确识别 multitrack 封装内的
sequence header（issue #707 修复）
src/server/broadcast_server.js：音频 sequence header 按轨道缓存
（以 multitrack trackIndex 为 key 的 Map），订阅者接入时重放全部轨道
的 codec config；同轨道重发 header 时覆盖旧值
回归测试：test/flv.test.js

Enhanced Rtmp V1 (`docs/enhanced-rtmp-v1.md`)

Enhancing RTMP, FLV

With Additional Video Codecs And HDR Support

Document Status

Author: Slavik Lozben
Affiliation: Veovera Software Organization (VSO)

Contributors: Google, Jean-Baptiste Kempf (FFmpeg, VideoLAN), pkv (OBS), Dennis Sädtler (OBS), Xavier Hallade (Intel Corporation), Luxoft, SplitmediaLabs Limited (XSplit), Craig Barberich (VSO), Chris Hock (Adobe), Michael Thornburgh

Document Version: v1-2025-01-22-r2

General Disclaimer: The features, enhancements, and specifications described in this document are provided for informational purposes only. While every effort has been made to ensure the accuracy and reliability of the information contained herein, Veovera Software Organization (VSO) makes no guarantees regarding its completeness, accuracy, or suitability for any specific purpose.

This document represents the finalized version of the Enhanced RTMP specification. However, Veovera Software Organization (VSO) may update or enhance it in the future in response to technological advancements, community feedback, or evolving industry requirements.

VSO shall not be liable for any damages, direct or indirect, resulting from the use or reliance on this document. Users are solely responsible for validating the suitability of the specifications described for their specific needs and for ensuring compliance with all applicable standards and requirements.

text

/* Detailed source-code truncated for AI context efficiency. */

Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. Definitions below are pasted from [RFC 2119].

MUST - This word, or the terms "REQUIRED" or "SHALL", means that the definition is an absolute requirement of the specification.
MUST NOT - This phrase, or the phrase "SHALL NOT", means that the definition is an absolute prohibition of the specification.
SHOULD - This word, or the adjective "RECOMMENDED", means that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
SHOULD NOT - This phrase, or the phrase "NOT RECOMMENDED", means that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
MAY - This word, or the adjective "OPTIONAL", means that an item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)

Additionally we add the key word DEPRECATED to the set of key words above. We use the Wikipedia description for the key word.

DEPRECATED - This word means a discouragement of use of some terminology, feature, design, or practice, typically because it has been superseded or is no longer considered efficient or safe, without completely removing it or prohibiting its use. Typically, deprecated materials are not completely removed to ensure legacy compatibility or back-up practice in case new methods are not functional in an odd scenario. It can also imply that a feature, design, or practice will be removed or discontinued entirely in the future. \

Abstract

There are ongoing requests from the media streaming industry to enhance the RTMP/FLV solution by bringing the protocol up to date with the current state of the art streaming technologies. RTMP was released over 20 years ago (The first public release of RTMP was in 2002). Many streaming solutions use RTMP in their stack today. While RTMP has remained popular it has gone a bit stale in its evolution. As an example RTMP/FLV does not have support for popular video codecs like VP9, HEVC, AV1. This document outlines enhancements to the RTMP/FLV specification to help bring this protocol inline with the current streaming media technologies by adding new capabilities to it. The new capabilities are outlined here in this spec. The additional capabilities added to this enhancement spec for the RTMP solution are:

New video codec types: \

txt

+---------------------------------------+--------------------------------------------------+
¦        Additional Video Codecs        ¦                      Notes                       ¦
+---------------------------------------+--------------------------------------------------+
¦HEVC (H.265)                           ¦Popular within streaming hardware and software    ¦
+---------------------------------------+solutions.                                        ¦
¦VP9                                    ¦                                                  ¦
+---------------------------------------+--------------------------------------------------+
¦                                       ¦- Gaining popularity                              ¦
¦AV1                                    ¦- Codec agnostic services are asking for AV1      ¦
¦                                       ¦  support                                         ¦
+---------------------------------------+--------------------------------------------------+

HDR capability - To support new video codecs and the current range of displays
PacketTypeMetadata - to support various types of video metadata

Introduction

This document describes enhancements to the RTMP spec by adding support for additional media codecs and HDR capability. One of the key goals is to ensure that this enhancement does not define any breaking changes to legacy clients and the content that they stream. This means that legacy RTMP/FLV specifications and documentations continue to stay valid and important to the RTMP ecosystem. This enhancement specification tries to limit duplication of information from legacy specifications. Legacy specification plus this documentation here form one holistic information for the RTMP solution. Some of the legacy informative references that have been leveraged are:

The Enhancement Spec Usage License

This document is licensed under the Apache License, Version 2.0 (the "License");
You may not use this document except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, the specification distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. \

RTMP Message Format

Adobe’s Real Time Messaging Protocol (RTMP) describes RTMP as “an application-level protocol designed for multiplexing and packetizing multimedia transport streams (such as audio, video, and interactive content) over a suitable transport protocol (such as TCP)”. One of the most important features of RTMP is the Chunk Stream. Chunk Stream multiplexes, packetizes and prioritizes messages on the wire. Chunking and prioritizing messages is the “RT” (i.e. Real Time) within RTMP. RTMP Message has two sections. A message header followed by a message payload:

The format of the message header is describe below

txt

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Message Type  |                Payload length                 |
|    (1 byte)   |                   (3 bytes)                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Timestamp                           |
|                           (4 bytes)                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Stream ID                     |
|                 (3 bytes)                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

There are two message types reserved for media messages.
- The message type value of 8 is reserved for audio message
- The message type value of 9 is reserved for video messages
Message payload follows the header. The payload for example could contain compressed audio data or compressed video data. RTMP knows nothing about the payload, including how to process it. If we want to add new codec types they have to be defined where the actual payload internals are defined. Flash Video (FLV) is a container file format where AV payload internals, including the codecs, are defined.
Please see the original RTMP (in various locations) & FLV (in Annex E. on page 68) specification for the endianness (aka byte order) of the data format on the wire.

A TidBit About FLV File Format

FLV file is a container for AV data. The file consists of alternating back-pointers and tags. Each tag is accompanied by data related to that tag. Each TagType within an FLV file is defined by 5 bits. AUDIODATA has a tag type of 8 and VIDEODATA has a tag type of 9. Note: these Tag Types map to the same Message Type IDs in the RTMP spec. This is by design. Each Tag Types of 8 or 9 is accompanied by an AudioTagHeader or VideoTagHeader. It’s common to think of RTMP in conjunction with FLV. That said, RTMP is a protocol and FLV is a file container. This is why they are originally defined in separate specifications. This enhancement spec is enhancing both RTMP and FLV.

Current VideoTagHeader

Below is the VideoTagHeader format for the pre 2023 FLV spec (i.e. ver 10.1.2.01):
Table 2: Current VideoTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Note: We have 4 bits to define video CodecID, luckily not all values are taken. We will leverage available values to define additional video CodecIDs. Whew, we have room to define new video formats. We can leverage the unused bits to achieve unlimited codec expansion. Please see the ENHANCEMENT TO RTMP & FLV sections below for the enhancement descriptions.

ENHANCEMENT TO RTMP & FLV ARE DESCRIBED BELOW

Enhancing onMetaData

The FLV metadata object SHALL be carried in a SCRIPTDATA tag named onMetaData. The available properties (as described in the FLV file format specification 10.1) differ depending on the software creating the FLV file. The videocodecid property has been enhanced to support FOURCC (i.e. Four-character ASCII code, such as ‘av01’, encoded as UI32) values. Typical metadata object properties include:

Table: onMetaData Properties

text

/* Detailed source-code truncated for AI context efficiency. */

Defining Additional Video Codecs

Below VideoTagHeader is extended to define additional video codecs and the supporting signaling while keeping backwards compatibility intact. The ExVideoTagHeader is future proof for defining additional codecs and the accompanying signaling. During parsing, logic must gracefully fail if at any point important signaling/flags (ex. FrameType, IsExHeader, ExHeaderInfo) are not understood.
Table 4: Extended VideoTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Extending NetConnection connect Command

When a client connects to an RTMP server it sends a connect command to the server. The command structure sent from the client to the server contains a Command Object. The Command Object is made up of name-value pairs. This is where the client indicates what audio and video codecs it supports. This name-value pair list will need to be extended to declare newly defined codecs, or any other enhancements, that are supported by the client. Following is the description of a new name-value pair used in Command Object of the connect command.
Table 5: new name-value pair that can be set in the Command Object

txt

+------------------+---------------+--------------------------------------------------+----------------------------+
¦     Property     ¦     Type      ¦                   Description                    ¦       Example Value        ¦
+------------------+---------------+--------------------------------------------------+----------------------------+
¦                  ¦               ¦The enhanced list of supported codecs. It’s a     ¦                            ¦
¦                  ¦               ¦strict array of dense ordinal indices. Each entry ¦                            ¦
¦    fourCcList    ¦Strict Array of¦in the array is of String Type. Each entry is set ¦  [“av01”, “vp09”, “hvc1”]  ¦
¦                  ¦    Strings    ¦to a fourCC value (i.e. a string that is a        ¦                            ¦
¦                  ¦               ¦sequence of four bytes) representing a supported  ¦                            ¦
¦                  ¦               ¦video codec.                                      ¦                            ¦
+------------------+---------------+--------------------------------------------------+----------------------------+

Metadata Frame

To support various types of video metadata, the FLV container specification has been enhanced. The VideoTagHeader has been extended to define a new PacketTypeMetadata (see Table 4) whose payload will contain an AMF encoded metadata. The metadata will be represented by a series of [name, value] pairs. For now the only defined [name, value] pair is [“colorInfo”, Object]. When leveraging PacketTypeMetadata to deliver HDR metadata, the metadata MUST be sent prior to the video sequence, scene, frame or such that it affects. Each time a new colorInfo object is received it invalidates and replaces the current one. To reset to the original color state you can send colorInfo with a value of Undefined (the RECOMMENDED approach) or an empty object (i.e., “{}”).

It is intentional to leverage a video message to deliver PacketTypeMetadata instead of other RTMP Message types. One benefit of leveraging a video message is to avoid any racing conditions between video messages and other RTMP message types. Given this, once your colorInfo object is parsed, the read values MUST be processed in time to affect the first frame of the video section which follows the colorInfo object.

The colorInfo object provides HDR metadata to enable a higher quality image source conforming to BT.2020 (aka. Rec. 2020) standard. The properties of the colorInfo object, which are encoded in an AMF message format, are defined below.

Note:

For content creators: Whenever it behooves to add video hint information via metadata (ex. HDR) to the FLV container it is recommended to add it via PacketTypeMetadata. This may be done in addition (or instead) to encoding the metadata directly into the codec bitstream.
The object encoding format (i.e. AMF0 or AMF3) is signaled during the connect command.

text

/* Detailed source-code truncated for AI context efficiency. */

Table 6: Flag values for the videoFunction property

text

/* Detailed source-code truncated for AI context efficiency. */

Action Message Format (AMF0 vs AMF3)

Action Message Format (AMF) is a compact binary format that is used to serialize script data. AMF has two versions: AMF 0 [AMF0] and AMF 3 [AMF3]. One way AMF3 improves on AMF0 is by optimizing the payload size on the wire. To understand the full scope of the optimizations please see the links above. It is RECOMMENDED to support AMF3 in the RTMP solution. It is nice to have AMF3 support within the FLV file. You should understand the ecosystem before adding AMF3 data to your FLV files.

The way to insure support for AMF3 in RTMP is by:

Adding support for Command Message, Data Message and Shared Object Message and types which use AMF3 form.
Signaling in the connect command that the object encoding format is AMF3.

Note: RTMP has had AMF3 as part of its specification for some time now. During the handshake the client declares whether it has support for AMF3. \
The way to insure support for AMF3 in FLV is by:

Adding a new TagType 15 (i.e., not 18) which supports SCRIPTDATA which is encoded via AMF3 (i.e., similar to the way Data Message is handled)

Note: Prior to Y2023 FLV file format did not have AMF3 as part of its SCRIPTDATA specification.

Protocol Versioning

There is no need for a version bump within enhanced RTMP for either the RTMP handshake sequence or the FLV header file version field. All of the enhancements are triggered via the newly defined additions to the bitstream format which don’t break legacy implementations. Enhanced RTMP is self describing in its capabilities.

Documentation Versioning

Overview

This section outlines our standardized approach for versioning our specification documentation. Effective versioning ensures consistency, enables users to identify the latest version easily, and facilitates collaboration among team members.

File Naming Convention

We name the documentation files with a clear identifier and the major version number.

Example:
enhanced-rtmp-v2.pdf

Version Information Inside the Document

We include a dedicated section or metadata within each document to specify the version details which includes the major version number, date, and stage of development (alpha/beta/release).

Example:
Status: v2-2024-02-26-a1

Calendar Versioning Format Description

The format for versioning documents is structured as follows:

v#-yyy-mm-dd-[a|b|r]#:
- v#: Major version number for tracking the progression of the enhanced RTMP development.
- yyyy-mm-dd: Date when the document was updated.
- [a|b|r]: Suffix to distinguish between the alpha, beta, and release stage.
- #: Minor version number for a particular date. Increments for multiple versions on the same date.

This format provides a comprehensive overview of each version's status and chronological order, facilitating effective tracking and management of the enhanced RTMP specification development.

Enhanced Rtmp V2 (`docs/enhanced-rtmp-v2.md`)

Enhanced RTMP (V2)

Table of Contents
Document Status
Documentation Versioning
Version Stage Definitions
Release Version Disclaimer for Enhanced RTMP
Usage License
Terminology
Abstract
Introduction
Conventions
Simple Data Types
RTMP Message Format
FLV File Format Overview
Enhancements to RTMP and FLV
Enhancing onMetaData
Reconnect Request
Enhanced Audio
Enhanced Video
Metadata Frame
Multitrack Streaming via Enhanced RTMP
Enhancing NetConnection connect Command
Action Message Format (AMF): AMF0 and AMF3
Protocol Versioning
References

Document Status

Author: Slavik Lozben
Affiliation: Veovera Software Organization (VSO)

Contributors: Adobe, Google, Twitch, Jean-Baptiste Kempf (FFmpeg, VideoLAN), pkv (OBS), Dennis Sädtler (OBS), Xavier Hallade (Intel Corporation), Luxoft, SplitmediaLabs Limited (XSplit), Meta, Michael Thornburgh, Veovera Software Organization

Document Version: v2-2026-01-31-r2

General Disclaimer: The features, enhancements, and specifications described in this document are intended for informational purposes only and may not reflect the final implementation. Veovera Software Organization (VSO) does not guarantee the accuracy, completeness, or suitability of this information for any specific purpose. Users are solely responsible for any decisions or implementations based on this document.

VSO reserves the right to refine, update, or enhance any part of this document at its sole discretion, based on technological feasibility, market conditions, or community feedback. VSO shall not be liable for any damages, direct or indirect, resulting from the use of this document.

This document represents a Release Version of the enhanced RTMP (E-RTMP) specifications, which is comprehensively specified and stable enough for broad implementation. For a detailed explanation of the Release Version, including its purpose, features, and intended use, please refer to the Release Version Disclaimer for Enhanced RTMP section later in this document.

Documentation Versioning

Overview

This section outlines the versioning strategy for this specification. This standardized versioning system ensures consistency, traceability, and clarity for implementers by making it easy to determine whether a document introduces new behaviors or includes only editorial updates.

Version identifiers communicate major milestones, publication dates, non-editorial revisions, and the document’s development phase, all in a human-readable, chronologically sortable format.

File Naming Convention

Documentation files are named with a clear identifier and a major version number.

Example:
enhanced-rtmp-v2.pdf

Version Information Inside the Document

Each document includes a version identifier using the following format:

Example:
Document Version: v2-2025-04-15-r1

This version string reflects the major version milestone, publication date, non-editorial revision number, and development phase.

Version Format Description

The format for versioning documents is structured as follows:

v<**major**>-<**yyyy-mm-dd**>-<**phase**><**revision**>

Table: Version format description

text

/* Detailed source-code truncated for AI context efficiency. */

This format supports a clear, linear history. If you're comparing two versions and the revision number (r#) has increased, you can assume there are non-editorial changes that may affect implementation. Editorial-only changes retain the same r# but update the date.

Commit History in GitHub

The document and its commit history are maintained in the Enhanced RTMP GitHub repository. Although the document version string resets r# at each phase transition, all underlying commits and commit messages are preserved in GitHub, ensuring a complete and auditable record of all changes made.

Version Stage Definitions

We define distinct stages for the development of the E-RTMP protocol specification to indicate its maturity and readiness for implementation. Each stage serves a different purpose for those implementing the protocol:

Alpha Version

Purpose: The alpha stage represents an early, stable version of the protocol specification. It is intended for real-world implementation and feedback collection from developers and implementers who are beginning to build solutions based on the protocol.
Features: While the specification is mostly defined, some aspects may still evolve based on implementation feedback. Breaking changes are possible, but efforts are made to minimize them to ensure stability for early adopters.
Audience: Developers and implementers looking to integrate the protocol in real-world applications who are prepared to adapt to refinements or changes.
Stability: Moderate. The alpha version is considered stable enough for serious implementation, but it is still subject to potential changes that could affect backward compatibility.
Documentation Status: Indicated by the version identifier "a" (e.g., v2-2024-02-26-a1).

Beta Version

Purpose: The beta stage indicates that the protocol specification is nearing its final form, with all core features defined and ready for implementation across diverse environments. This stage focuses on verifying that the protocol works reliably at scale, with extensive real-world testing.
Features: The protocol is comprehensively specified, and any changes at this stage should ideally be non-breaking. These changes may involve optimizations or clarifications to ensure smooth, large-scale deployments, but no core elements of the protocol are expected to be altered.
Audience: Developers and organizations preparing for production use, who are looking to validate their implementations against a near-final version of the protocol.
Stability: High. The beta version is stable enough for broad implementation with the expectation that no significant breaking changes will be introduced.
Documentation Status: Indicated by the version identifier "b" (e.g., v2-2024-02-26-b1).

Release Version (General Availability)

Purpose: The release (General Availability) stage is the finalized version of the protocol specification, fully stable and ready for widespread production use.
Features: The specification is locked, and no breaking changes should occur. Any updates will focus on backward-compatible improvements or bug fixes.
Audience: Developers, implementers, and end-users who need a reliable, long-term stable version for production deployments.
Stability: Highest. The release version ensures stability for production environments with long-term support.
Documentation Status: Indicated by the version identifier "r" (e.g., v2-2024-02-26-r1).

Release Version Disclaimer for Enhanced RTMP

This document represents the Release Version of the enhanced Real-Time Messaging Protocol (E-RTMP) specification. All core features are finalized and the specification is considered fully stable for broad implementation across diverse environments.

The protocol is not expected to undergo breaking changes. Future updates to this specification are intended to be backward-compatible, focusing on clarifications, documentation improvements, or optional extensions that do not disrupt existing implementations.

Veovera Software Organization (VSO) provides this document “as is,” without warranties, express or implied, including but not limited to suitability for a particular purpose. Users should independently validate their implementations to meet the needs of their specific applications. VSO disclaims all liability for any direct, indirect, incidental, or consequential damages resulting from use of this document.

We welcome community feedback through GitHub issues or other established channels to guide the evolution of the protocol.

This specification is published under the Apache License, Version 2.0, which governs its use, distribution, and modification. Please refer to the Usage License section later in this document for full licensing terms.

The version identifier (e.g., v2-2024-02-26-r1) reflects the release stage status.

Usage License

Copyright 2022-2024 Veovera Software Organization

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

<https://www.apache.org/licenses/LICENSE-2.0>

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Terminology

MUST: This word, or the terms "REQUIRED" or "SHALL", means that the definition is an absolute requirement of the specification.
MUST NOT: This phrase, or the phrase "SHALL NOT", means that the definition is an absolute prohibition of the specification.
SHOULD: This word, or the adjective "RECOMMENDED", means that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
SHOULD NOT: This phrase, or the phrase "NOT RECOMMENDED", means that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.
MAY: This word, or the adjective "OPTIONAL", means that an item is truly optional. One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)

Additionally we add the keyword [DEPRECATED] to the set of keywords above.

DEPRECATED: This word means a discouragement of use of some terminology, feature, design, or practice, typically because it has been superseded or is no longer considered efficient or safe, without completely removing it or prohibiting its use. Typically, deprecated materials are not completely removed to ensure legacy compatibility or back-up practice in case new methods are not functional in an odd scenario. It can also imply that a feature, design, or practice will be removed or discontinued entirely in the future.

Abstract

In the rapidly evolving media streaming landscape, there is a pressing need to update legacy protocols to align with modern technological standards. The Real-Time Messaging Protocol [RTMP] and Flash Video [FLV] file format, introduced in 2002, have been pivotal and continue to be vital especially in live broadcasting. Despite RTMP widespread use, it has shown signs of aging, particularly in the lack of support for contemporary video codecs (e.g., VP8, VP9, HEVC, VVC, AV1) and audio codecs (Opus, FLAC, AC-3, E-AC-3). Recognizing this, Veovera Software Organization (VSO), in collaboration with industry giants like Adobe, YouTube, and Twitch, and other key stakeholders, has embarked on a mission to rejuvenate RTMP, ensuring it meets the demands of contemporary streaming needs.

This document details the comprehensive enhancements made to the RTMP and FLV specifications, aimed at revitalizing the technology for current and future media demands. Our strategic approach prioritizes innovation while maintaining backward compatibility, thereby augmenting RTMP's utility without undermining existing infrastructures. Some of the key advancements include:

Advanced Audio Codecs: Integration of codecs like AC-3, E-AC-3, Opus, and FLAC to meet diverse audio quality and compression needs, ensuring compatibility with modern systems.
Multichannel Audio Configurations: Support for multichannel audio to enhance auditory experiences without compromising existing setups.
Advanced Video Codecs: Introduction of codecs such as VP8, VP9, HEVC, VVC and AV1 with HDR support to meet modern display and content standards.
Video Metadata: Expansion of VideoPacketType.Metadata to support a broader range of video metadata types.
FourCC Signaling: Inclusion of FourCC signaling for advanced codecs mentioned above, as well as for legacy codecs such as AVC, AAC, and MP3.
Multitrack Capabilities: New audio and video multitrack capabilities for concurrent management and processing of multiple media streams, enhancing media experiences.
Reconnect Request Feature: A new Reconnect Request feature improves connection stability and resilience.
Timestamp Precision: Introduction of nanosecond precision offsets, ensuring enhanced synchronization and compatibility across diverse media formats such as MP4, M2TS, and Safari's Media Source Extensions, without altering the core RTMP timestamps.
ModEx Signal: A signaling mechanism that lets packets carry modifiers or extensions, enabling precise control over timing, metadata, and other media stream behaviors.

The additional audio and video codecs supported by enhanced RTMP (E-RTMP) are summarized in the following table:

Table: Additional audio and video codecs for E-RTMP

text

/* Detailed source-code truncated for AI context efficiency. */

These strategic enhancements position RTMP as a robust, future-proof standard in the streaming technology arena. Veovera is committed to open collaboration and values community input, believing that protocols and standards should be open and free to foster innovation and create a thriving ecosystem. Companies can capitalize on solutions built around open standards; the more popular and accessible a protocol is, the stronger the foundation for developing compelling solutions. A standard’s popularity fuels adoption, allowing companies to leverage its widespread use. In contrast, fragmentation caused by proprietary protocols hampers industry growth, while open standards empower everyone to innovate freely, creating a healthier marketplace. E-RTMP’s shift toward openness aligns with the principles of open standards, emphasizing its potential to become a foundational technology. We encourage participation in the ongoing development process through our GitHub repository, where you can access detailed documentation, contribute to the project, and share insights to foster a vibrant ecosystem around E-RTMP.

Introduction

This document describes enhancements to legacy [RTMP] and legacy [FLV], introducing support for new media codecs, HDR capability, and more. A primary objective is to ensure these enhancements do not introduce breaking changes for established clients or the content they stream. As such, legacy RTMP and legacy FLV specifications remain integral to the RTMP ecosystem. While this updated specification aims to minimize redundancy with previous versions, when combined with previous-generation documentation, it provides a comprehensive overview of the RTMP solution. We've drawn from several legacy references, which are as follows:

Adobe legacy [RTMP] specification
Adobe legacy [FLV] specification
Additional [LEGACY] specifications

Conventions

This document employs certain conventions to convey particular meanings and requirements. The following section outlines the notation, terminology, and symbols used throughout to ensure clarity and consistency. These conventions provide insight into the ethos of how the E-RTMP specification has been crafted and should be interpreted.

E-RTMP: refers to a series of improvements made to the legacy Real-Time Messaging Protocol [RTMP], originally developed by Adobe. While "enhanced RTMP" may be used descriptively, the preferred and consistent name for the protocol is E-RTMP, which serves as a consistent identifier that distinguishes the updated protocol from the legacy RTMP specification. Endorsed by Adobe and widely adopted across the industry, E-RTMP serves as the current standard for RTMP-based solutions and includes enhancements to both RTMP and the legacy [FLV] formats. The name E-RTMP refers to an evolving protocol and does not correspond to any single release or version. To avoid confusion, alternate forms such as ERTMP, eRTMP, Enhanced-RTMP, or EnhancedRTMP should be avoided. The lowercase hyphenated form e-rtmp is acceptable in URLs, folder names, and other technical contexts where lowercase formatting is conventional.
Pseudocode: Pseudocode has been provided to convey logic on how to interpret the E-RTMP binary format. The code style imitates a cross between TypeScript and C. The pseudocode was written in TypeScript and validated using VSCode to ensure correct syntax and catch any minor typographical errors. Below are some further explanations:
- Enumerations define valid values for fields that appear in the encoded bitstream. In contrast, script-level may use textual names or key/value maps, consistent with historical SCRIPTDATA and ActionScript object model.
- Pseudo variables are named in a self-descriptive manner. For instance:
  
  videoCommand = UI8 as VideoCommand
  
  The line above indicates that an unsigned 8-bit value is read from the bitstream. The legal values correspond to the enumerations within the VideoCommand set, and the pseudo variable videoCommand now holds that value.
- The pseudocode is written from the point of view of reading (a.k.a., parsing) the bitstream. If you are writing the bitstream, you can swap source with destination variables.
- E-RTMP typically employs camelCase naming conventions for variables. In contrast, the naming convention for legacy RTMP specification is usually preserved as is.
- Handshake and Enhancing NetConnection connect command: The E-RTMP specification generally prioritizes the client's perspective over that of the server. To shift this focus and view the interaction from the server's side, the server should echo back certain enhancement information.
  
  When the client informs the server of the enhancements it supports via the connect command, the server processes this command and responds using the same transaction ID. The server's response string will be one of the following: _result, _error, or a specific method name. A command string of _result or _error indicates a response rather than a new command.
  
  During this response, the server will include an object containing specific properties as one of the arguments to _result. It is at this point that the server should indicate its support for E-RTMP features. Specifically, the server should denote its capabilities through attributes such as videoFourCcInfoMap, capsEx, and other defined properties.
- The ethos of this pseudocode is to provide a high-level overview of the data structures and operations taking place on the wire. While it accurately represents the bytes being transmitted, it's important to note that the logic is not exhaustive. Specifically, this pseudocode does not cover all possible cases, nor does it always include items such as initialization logic, looping logic or error-handling mechanisms. It serves as a foundational guide that can be implemented in various ways, depending on specific needs and constraints.
Unrecognized value: If a value in the bitstream is not understood, the logic must fail gracefully in a manner appropriate for the implementation.
Table naming: Each table in the document is named according to the specific content or subject it is describing.
Bitstream optimization: One of the guiding principles of E-RTMP is to optimize the number of bytes transmitted over the wire. While minimizing payload overhead is a priority, it is sometimes more important to simplify the logic or enhance extensibility. For example, although more optimal methods for creating a codec ID than using FOURCC may exist, such approaches could render the enhancement non-standard and more challenging to extend and maintain in the future.
Capitalization rules: Another guiding principle in the E-RTMP is the standardization of capitalization for types. The original documentation capitalized types such as Number, String, and Boolean, and even included various other spellings. The E-RTMP adopts lowercase spelling for terms, such as number, string, and boolean. This change emphasizes that these types are simple, not objects.
ECMA Array vs Object: In the world of AMF (Action Message Format), both ECMA Array and Object are used to store collections of properties. A property is simply a pairing of a name with a value. In enhanced RTMP, the term Object is specifically used to indicate the Object Type. In the past, people have sometimes used ECMA Array and Object as if they were the same thing. However, for better coding practices, it's recommended to use Object when you're creating AMF data. When you're reading or decoding AMF data, you should be prepared to handle either ECMA Array or Object for greater flexibility and robustness.
Default values: Unless explicitly called out, there should be no assumptions made regarding default values, such as null or undefined.
Legacy vs. Enhanced Properties: In the documentation, an effort has been made to distinguish between legacy properties and newly defined ones through color coding, such as using bold text or different background colors for enhancements. While this color coding is not guaranteed to be consistent, the distinctions between values defined in E-RTMP should be readily apparent.
Capability flags: The capabilities flags, exchanged during a connect handshake, may not cover all possible functionalities. For instance, a client might indicate support for multitrack processing without specifying its ability to encode or decode multitrack streams. In scenarios where a client, capable of issuing a play command, declares multitrack support, it MUST be equipped to handle the playback of such streams. Similarly, if a client is aware of the server's multitrack capabilities, it MAY opt to publish a multitrack stream.
Quotation Marks and Emphasis Guidelines: Ultimately, the context should drive the meaning, but we make an effort to leverage quotation marks and emphasis (i.e., bold) to maintain readability. We aim to avoid syntactic sugar as much as possible to ensure the document remains straightforward, easy to read, scan, and understand. The conventions for using double quotes ("), back quotes (`), and emphasis in this document to ensure clarity and consistency are as follows:
- Double quotes are used for: direct quotations, titles of short works, and when referencing a specific term or phrase.
- Back quotes are used for: code snippets, commands, or technical terms.
- Bold is used for: emphasis on important terms or phrases. Sometimes, back quotes and bold can be interchanged for ease of reading.

Simple Data Types

The following data types are used in [RTMP] bitstreams and [FLV] files. FOURCC was introduced to support E-RTMP.

Table: Simple data types

text

/* Detailed source-code truncated for AI context efficiency. */

Note: Unless specifically called out, multi-byte integers SHALL be stored in big-endian byte order

RTMP Message Format

Adobe's Real-Time Messaging Protocol [RTMP] is an application-level protocol designed for the multiplexing and packetizing of multimedia streams—such as audio, video, and interactive content, for transmission over network protocols like TCP. A fundamental feature of RTMP is the Chunk Stream, which facilitates the multiplexing, packetizing, and prioritization of messages, integral to the protocol's real-time capabilities.

The legacy RTMP specification in Section 6.1 elaborates on the RTMP Message Format, providing precise encoding guidelines for the RTMP message header, inclusive of field widths and byte order. However, this portrayal might be somewhat confusing because RTMP messages, when transported over the Chunk Stream, don't literally conform to this depicted format. An RTMP Message is divided into two principal components: a message virtual header and a message payload. The "virtual" descriptor indicates that while RTMP messages are carried within the RTMP Chunk Stream, their headers are conceptually encoded as Chunk Message Headers. When these are decoded from the RTMP Chunk Stream, the underlying transport layer, the resulting format is to be understood as a virtual header. This abstract representation aligns with the structured format and semantics detailed in the legacy RTMP specification. Detailed next is the format of the message virtual header and some additional related information.

Message virtual header

txt

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|MessageType ID |                Payload length                 |
|    (1 byte)   |                   (3 bytes)                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Timestamp                           |
|                           (4 bytes)                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 Stream ID                     |
|                 (3 bytes)                     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

There are two message types reserved for media messages:
- The message type value of 8 is reserved for audio message
- The message type value of 9 is reserved for video messages
The message payload follows the header and may contain various types of content, such as compressed audio or video data. RTMP itself does not recognize or process the payload's content. If new codec types are to be added, they must be defined where the actual payload internals are outlined. FLV is a container file format where the specifics of the AV payload, including the codecs, are defined.
Please refer to the legacy RTMP specification (in various locations) and the legacy [FLV] specification (Annex E) for details on the endianness (a.k.a., byte order) of the data format on the wire.

FLV File Format Overview

An [FLV] file is a container for AV (Audio and Video) data. The file consists of alternating back-pointers and tags, each accompanied by data related to that tag. Each TagType within an FLV file is unsigned and defined by 5 bits. AUDIODATA has a TagType of 8, and VIDEODATA has a TagType of 9.

Note: Each TagType corresponds directly to the same MessageType ID, defined by UI8, in the [RTMP] specification. This alignment is intentional.

TagType values of 8 or 9 are accompanied by an AudioTagHeader or VideoTagHeader respectively. While RTMP is commonly associated with FLV, it is important to note that RTMP is a protocol, whereas FLV is a file container format. This distinction is why they were originally defined in separate specifications. This enhancement specification aims to improve both RTMP and FLV.

Pre 2023 AudioTagHeader Format

Below is the AudioTagHeader format for the legacy FLV specification:

Table: FLV specification AudioTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Pre 2023 VideoTagHeader Format

Below is the VideoTagHeader format for the legacy FLV specification:

Table: FLV specification VideoTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Enhancements to RTMP and FLV

Within the following sections, this document provides a comprehensive overview of the enhancements made to [RTMP] and [FLV]. Together, these improvements constitute the enhanced RTMP also known as E-RTMP. These enhancements are discussed in detail, highlighting their impact and benefits.

Enhancing onMetaData

[FLV] metadata SHALL be encapsulated within a [SCRIPTDATA] segment, which includes a [ScriptTagBody] encoded in the Action Message Format (AMF). Importantly, this metadata SHALL always remain unencrypted, even when the FLV content itself is encrypted. This design choice is essential for allowing various FLV parsers to successfully stream the FLV content and for enabling media players to provide contextual information to the user.

The ScriptTagBody is structured to encapsulate method invocations. It consists of an item containing a method name (e.g., onMetaData) along with a corresponding set of arguments.

To signal FLV metadata, the item within the ScriptTagBody MUST encapsulate the method name onMetaData, along with a single argument of type ECMA array. This array holds metadata properties, the availability of which may vary depending on the software used to create the FLV. Typical onMetaData argument properties include, but are not limited to:

Table: Typical properties found in the onMetaData argument object

text

/* Detailed source-code truncated for AI context efficiency. */

Note:

The properties audiocodecid and videocodecid have been enhanced to support FOURCC (Four-byte ASCII code) values. These values are interpreted as UI32 (e.g., "av01").

The properties audioTrackIdInfoMap and videoTrackIdInfoMap are new.

Reconnect Request

Objective

[RTMP] packetizes multimedia streams using a suitable transport protocol, typically a persistent TCP connection. There are instances when a streaming platform may request the streaming client to reconnect, such as:

When live streaming servers undergo updates.
When there's a need to redirect the client to a different server instance, ensuring optimal load balancing and precise geolocation mapping.

To accommodate these needs, a NetConnection.Connect.ReconnectRequest status event has been introduced as part of the NetConnection onStatus command.

NetConnection Commands

NetConnection establishes a bidirectional link between a client and a server, allowing for asynchronous Remote Procedure Calls (RPCs). The following commands (a.k.a., predefined RPCs) can be issued via NetConnection:

connect
createStream
deleteStream
onStatus

The onStatus command has been enhanced to include the capability to request a client to reconnect. Servers can issue an onStatus command to prompt clients to adapt to changes in NetConnection status. The structure of this command, as relayed from the server to the client, is outlined below:

Table: Server to client, NetConnection onStatus command

text

/* Detailed source-code truncated for AI context efficiency. */

The following is a description of AMF-encoded name-value pairs in the Info Object for the onStatus command when handling reconnect. It MAY contain other properties as appropriate to the client.

Table: Info Object parameter for onStatus command when handling reconnect

text

/* Detailed source-code truncated for AI context efficiency. */

Message Flow When Handling NetConnection.Connect.ReconnectRequest

Prior to the shutdown of the live streaming server or when the server intends to remap the client to another server instance, it dispatches an onStatus command to the client with a code of NetConnection.Connect.ReconnectRequest. If the server aims to remap the client, it MUST set the tcUrl property in the Info Object. In order to avoid a disruption, the server managing the original connection (commonly referred to as the "old server") SHOULD continue processing messages from the client until the client disconnects.
When the client receives the NetConnection.Connect.ReconnectRequest event, it persists in streaming to/from the current server up to the next appropriate media boundary, such as a keyframe. Subsequently, it establishes a connection with a new server and disconnects from the old server. If the Info Object includes the tcUrl property, the client uses this URL for the reconnection process. Absent this property, the client defaults to the tcUrl for the current connection.
While the client can establish a new connection before severing the original one, it SHOULD exercise caution to ensure the Quality of Service (QoS) is not compromised.

The capability to support the NetConnection.Connect.ReconnectRequest event becomes evident during the initial connect phase. Detailed guidelines for signaling reconnect ability can be found in the Enhancing NetConnection connect Command section.

Detailed Overview of the onStatus Command for NetConnection

The server-to-client onStatus command for NetConnection, serves a crucial function within the RTMP framework. Though the legacy RTMP specification may not have detailed this command, the goal here is to offer an overview for a better understanding.

Both clients and servers can initiate RPCs at the receiving end, with some RPCs being predefined as commands. onStatus stands out as one such essential command.

When using the onStatus command, the goal is to inform the client about the status of the connection. Each dispatched command message comprises the following elements:

Command Name: type string
Transaction ID: type number
Command Object (set to null when dispatching an onStatus command): type Object
Info Object (which can be viewed as Optional Arguments): type Object

Both the Command Object and the Info Object offer additional context and details for the command. The onStatus command is triggered whenever there's a status change or an error concerning the NetConnection. To handle this information, you should define a callback function.

// Sample pseudocode for the onStatus callback function
nc.onStatus = function(infoObject) {
  // Handle the status change or error here.
}

infoObject is an AMF-encoded object with properties that provide information about the status of a NetConnection. It contains at least the following three properties, but MAY contain other properties as appropriate to the client.

Table: infoObject for onStatus command

text

/* Detailed source-code truncated for AI context efficiency. */

The table below provides examples of code, level, and description property values. Please note that this is not an exhaustive list, and not all entries may apply to every type of client. Additionally, the description property values included are merely illustrative examples; developers are responsible for conveying the appropriate meaning in their specific solutions.

Table: code, level and description values for infoObject used by onStatus

text

/* Detailed source-code truncated for AI context efficiency. */

Enhanced Audio

The AudioTagHeader has been extended to define additional audio codecs, multichannel audio, multitrack capabilities, signaling support, and additional miscellaneous enhancements, while ensuring backward compatibility. This extension is termed the ExAudioTagHeader and is designed to be future-proof, allowing for the definition of additional audio codecs, features, and corresponding signaling.

During the parsing process, the logic MUST handle unexpected or unknown elements gracefully. Specifically, if any critical signaling or flags (e.g., AudioPacketType and AudioFourCc) are not recognized, the system MUST fail in a controlled and predictable manner.

Important: A single audio message for a unique timestamp may include a batch of AudioPacketType values (e.g., multiple trackId values). When parsing an audio message, the bitstream MUST be processed completely to ensure all payload data has been handled.

Table: Extended AudioTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Enhanced Video

The VideoTagHeader has been extended to define additional video codecs, multitrack capabilities, signaling support, and additional miscellaneous enhancements, while ensuring backward compatibility. This extension is termed the ExVideoTagHeader and is designed to be future-proof, allowing for the definition of additional video codecs, features, and corresponding signaling.

During the parsing process, the logic MUST handle unexpected or unknown elements gracefully. Specifically, if any critical signaling or flags (e.g., VideoFrameType, VideoPacketType, or VideoFourCc) are not recognized, the system MUST fail in a controlled and predictable manner.

Important: A single video message for a unique timestamp may include a batch of VideoPacketType values (e.g., multiple trackId values, Metadata values). When parsing a video message, the bitstream MUST be processed completely to ensure all payload data has been handled.

Table: Extended VideoTagHeader

text

/* Detailed source-code truncated for AI context efficiency. */

Metadata Frame

To support various types of video metadata, the legacy [FLV] specification has been enhanced. The VideoTagHeader has been extended to define a new VideoPacketType.Metadata (see ExVideoTagHeader table in Enhanced Video section) whose payload will contain an AMF-encoded metadata. The metadata will be represented by a series of [name, value] pairs. For now the only defined [name, value] pair is ["colorInfo", Object]. When leveraging VideoPacketType.Metadata to deliver HDR metadata, the metadata MUST be sent prior to the video sequence, scene, frame or such that it affects. Each time a new colorInfo object is received it invalidates and replaces the current one. To reset to the original color state you can send colorInfo with a value of Undefined (the RECOMMENDED approach) or an empty object (i.e., {}).

It is intentional to leverage a video message to deliver VideoPacketType.Metadata instead of other [RTMP] Message types. One benefit of leveraging a video message is to avoid any racing conditions between video messages and other RTMP message types. Given this, once your colorInfo object is parsed, the read values MUST be processed in time to affect the first frame of the video section which follows the colorInfo object.

The colorInfo object provides HDR metadata to enable a higher quality image source conforming to BT.2020 (a.k.a., Rec. 2020) standard. The properties of the colorInfo object, which are encoded in an AMF message format, are defined below.

Note:

For content creators: Whenever it behooves to add video hint information via metadata (e.g., HDR) to the FLV container it is RECOMMENDED to add it via VideoPacketType.Metadata. This may be done in addition (or instead) to encoding the metadata directly into the codec bitstream.

The object encoding format (i.e., AMF0 or AMF3) is signaled during the connect command.

text

/* Detailed source-code truncated for AI context efficiency. */

Table: Flag values for the videoFunction property

text

/* Detailed source-code truncated for AI context efficiency. */

HDR Formats (Informative)

E-RTMP supports transport of video encoded using a variety of HDR formats. The following table summarizes common HDR formats, their associated codecs, metadata characteristics, and signaling considerations. This section is informative and does not define protocol requirements.

Table: HDR formats (informative)

text

/* Detailed source-code truncated for AI context efficiency. */

Note:

HDR10+ with AVC is defined, but rarely deployed in practice.

AVC, HEVC, and VVC may carry HDR metadata in the codec bitstream via Supplemental Enhancement Information (SEI) messages.

For HDR10 and HDR10+, metadata presence in the codec bitstream is not guaranteed; therefore, it is RECOMMENDED to include HDR metadata in the FLV container.

For Dolby Vision, HDR dynamic metadata is read from the codec bitstream (for example, via NAL units for AVC/HEVC/VVC or OBUs for AV1). Accordingly, such metadata SHOULD NOT be duplicated in the FLV container.

Dolby Vision with HLG transfer characteristics remains Dolby Vision and requires Dolby Vision signaling and metadata. It MUST NOT be interpreted as baseline HLG.

Where applicable, HDR-related signaling and metadata may be carried via the colorInfo object as described above.

Multitrack Streaming via Enhanced RTMP

Introduction to Multitrack Capabilities

E-RTMP has introduced support for multitrack streaming, offering increased flexibility in audio and video streaming through the use of a track index (i.e., audioTrackId and videoTrackId). This feature allows for the serialization of multiple tracks over a single E-RTMP connection and stream channel.

It's important to note that multitrack support is designed to augment, not replace, the option of using multiple streams for streaming. While both multiple streams and multitrack can potentially address the same use cases, the choice between them will depend on the specific capabilities of your E-RTMP implementation and requirements. In certain cases, multitrack may not be the most efficient option.

Multitrack Sample Use Cases

Adaptive Bitrate Streaming: Multitrack support allows the client to send Adaptive Bitrate (ABR) ladders, thus avoiding the need for server-side transcoding and reducing quality loss. This also facilitates sending content with multiple codecs like AV1, HEVC, and VP9.
Device Specific Streaming: The feature allows for the streaming of different aspect ratios, tailored for various device profiles, enabling more dynamic and flexible presentations.
Frame-Level Synchronization: For example, you can synchronize multiple camera views in a concert.
Multi-Language Support: Support for multiple audio tracks in a single [FLV] file is now available, eliminating the need for multiple file versions.
Multi-codec publishing: Allows a publisher to encode media in multiple formats within a single stream and enabling delivery of the appropriate codec based on platform support

Multitrack Media Message Guidelines

Video Messages: Each video message MUST include a videoTrackId (refer to the videoPacketType.Multitrack entry in the ExVideoTagHeader table within the Enhanced Video section for video bitstream signaling) as it is not persistent across messages.
Audio Messages: Similarly, each audio message MUST include an audioTrackId (refer to the AudioPacketType.Multitrack in the ExAudioTagHeader table within the Enhanced Audio section for audio bitstream signaling).
Payload Parsing: All tracks within a single timestamp MUST be processed to ensure comprehensive media handling.
Track Ordering: To provide a consistent convention, it is RECOMMENDED that trackId 0 be used for the default track of a given media type. The default track is the representation that the publisher expects most receivers to select when no additional selection logic is applied, for example the primary or most broadly applicable presentation for the stream.

Additional variants, for example different bitrates, resolutions, codecs, languages, or camera angles, SHOULD use distinct positive trackId values (1, 2, 3, …). These values are identifiers only and do not imply any inherent ordering, priority, or quality ranking.

Encoders and ingesters SHOULD provide complete and accurate information in the onMetaData fields for each track. This includes codec identifiers and other descriptive track-level attributes such as bitrate, resolution, sample rate, channel count, language, and similar properties relevant to track selection and processing.

The authoritative details for each track are also present within the stream itself, for example through codec configuration records, and fully self-describing media packets. These in-stream signals can be used by receivers when determining the appropriate processing behavior.

The full definition of track selection or priority logic is beyond the scope of the E-RTMP specification. The guidance provided here is intended only to establish a consistent convention for trackId usage and related metadata structure. Individual implementations can use different approaches when combining onMetaData information with in-stream signaling to perform track selection or processing.

Implementations MUST NOT infer detailed quality or compatibility characteristics from the trackId alone. Instead, they SHOULD consider the metadata provided via onMetaData together with the information conveyed within the media stream to evaluate characteristics such as bitrate, resolution, codec, language, or device profile alignment when choosing a track.

SCRIPTDATA Multitrack Parameter Handling

trackId SHOULD be a Parameter: For methods within RTMP that involve [SCRIPTDATA] messages, the trackId can be a critical parameter for operations that pertain to specific media tracks. In such cases, the trackId SHOULD be passed in as an argument to the method, ensuring that the action or data manipulation is accurately applied to the correct track. When track namespaces include both audio and video tracks, the combination of trackId and mediaType uniquely identifies the target track.
Recommended Parameter Passing:
- Using a Map or Object Argument: The recommended way to pass the trackId to methods involving SCRIPTDATA is by including it within a map or as a property of an object argument. This approach aligns with practices such as those used in the Enhancing onMetaData section, enhancing consistency and scalability across various implementations.
- Function Signature Example: This method takes an object arg which contains the properties trackId and mediaType. This structure is effective for managing multiple parameters and enhances readability and maintainability.

// Invokes a media-type specific script for a given track.
function scriptMethodName(arg: {
  trackId: number,
  mediaType: "video" | "audio" | "data",
  // ... additional properties can be added here
}) {
  console.log(`Invoking ${arg.mediaType} script for trackId:`, arg.trackId);
}

Note: This example illustrates one possible object structure. Actual SCRIPTDATA payloads may vary depending on the encoder or tooling, and may use maps or other AMF object forms consistent with historical SCRIPTDATA usage.

Advantages of Parameter Passing Approach:
- Clarity and Structure: Using an object or map to pass arguments, including the trackId, organizes the parameters neatly and reduces the chances of errors or misalignment in parameter order.
- Enhanced Maintenance: It becomes easier to add more parameters in the future without altering the method signature drastically, thereby maintaining compatibility and easing future enhancements. The style of passing the trackId as part of a structured object or map ensures a robust framework for handling SCRIPTDATA operations in RTMP streams, providing clear, scalable, and error-free management of track-specific data.

Leveraging Multitrack Features in E-RTMP

Multitrack capabilities in E-RTMP offer a wide range of possibilities, from adaptive bitrate streaming to multi-language support. While this document doesn't prescribe specific encoding rules or manifest metadata, it aims to guide you through the complexities of leveraging multitrack features. Consider various parameters like codecs, frame rates, key frames, sampling rates, and resolutions to meet your unique objectives. Remember, media encoding settings are separate from E-RTMP configurations.

Enhancing NetConnection connect Command

When a client connects to an E-RTMP server, it sends a connect command to the server. The command structure sent from the client to the server contains a Command Object, comprising name-value pairs. This is where the client indicates the audio and video codecs it supports. To declare support for newly defined codecs or other enhancements supported by the client, this name-value pair list must be extended. Below is the description of a new name-value pair used in the Command Object of the connect command.

Table: New name-value pair that can be set in the Command Object

text

/* Detailed source-code truncated for AI context efficiency. */

As you can see, the client declares to the server what enhancements it supports. The server responds with a command, either _result or _error, to indicate whether the response is a result or an error. During the response, the server provides some properties within an Object as one of the parameters. This is where the server needs to state its support for E-RTMP. The server SHOULD state its support via attributes such as videoFourCcInfoMap, capsEx, and similar properties.

Action Message Format (AMF): AMF0 and AMF3

Action Message Format (AMF) is a compact binary format used to serialize SCRIPTDATA. It has two specifications: [AMF0] and [AMF3]. AMF3 improves on AMF0 by optimizing the payload size on the wire. To understand the full scope of these optimizations, please refer to the AMF0 and AMF3 specifications.

Supporting AMF3 in the [RTMP] and [FLV] is beneficial due to its optimization over AMF0. Understanding the ecosystem is crucial before adding AMF3 support to RTMP or FLV.

Enabling AMF3 in RTMP

To enable support for AMF3 in RTMP, the following steps are REQUIRED:

Adding support for Data Message, Shared Object Message and Command Message and their associated AMF3 message types (i.e., 15, 16 and 17).
Adding support for the AMF3 set of possible type markers (see AMF3 specification section 3.1).
Signaling in the connect command that the AMF3 encoding format is supported in addition to AMF0.

RTMP has had AMF3 as part of its specification for some time now. During the handshake, the client declares whether it has support for AMF3.

Enabling AMF3 in FLV

Prior to Y2023, the FLV file format did not have AMF3 as part of its SCRIPTDATA specification. To ensure support for AMF3 in FLV:

Add a new FLV TagType 15 (i.e., in addition to TagType 18), which supports SCRIPTDATA encoded via AMF3 (i.e., similar to the way Data Message is handled).

Important AMF3-encoded Historical Specification Clarification

Established, pre E-RTMP, specifications state the following:

Command Messages carry the AMF-encoded commands between the client and the server. Message type values:
- 20 for AMF0 encoding.
- 17 for AMF3 encoding.
Data Messages are sent by the client or server to send Metadata or user data to the peer, including details such as creation time, duration, theme, etc. Message type values:
- 18 for AMF0 encoding.
- 15 for AMF3 encoding.
The message types 19 for AMF0 and 16 for AMF3 are reserved for Shared Object events.
AMF0 was extended to allow an AMF0 encoding context to be switched to AMF3. A new type marker, avmplus-object-marker (byte 0x11), was added. The presence of this marker signifies that the following value is encoded in AMF3. Legacy AMF0 systems that haven't been updated to support AMF3 should throw an unknown type error.

Unfortunately, the above is incomplete and may be somewhat unclear. To clarify, in addition to the above:

Object Encoding property in the Command Object of the connect command indicates the type of serialization (a.k.a., encoding) supported by the client or server:
- A value of 0 (default and optional) indicates support for AMF0 encoding and message types of 18, 19 and 20.
- A value of 3 indicates support for both AMF0 and AMF3 encoding and message types of (18, 15), (19, 16) and (20, 17).
Message payload for message types of 15, 16 and 17 starts with a format selector byte. Currently, only format 0 is defined to indicate AMF0-encoded values. It's possible to signal a switch to AMF3 serialization by prefixing an AMF3 value with an AMF0 avmplus-object-marker (byte 0x11). The switch isn't sticky, and parsing MUST return to AMF0 encoding mode once the AMF3 value is serialized. This means that every AMF3 encoded value MUST be prefixed with an avmplus-object-marker (byte 0x11) as defined in AMF0.

Protocol Versioning

There is no need for a version bump within E-RTMP for either the [RTMP] handshake sequence or the FLV header file version field. All of the enhancements are triggered via the newly defined additions to the bitstream format which don’t break legacy implementations. E-RTMP is self describing in its capabilities.

References

[FourCC]

A sequence of four bytes (typically ASCII) used to uniquely identify data formats,
<https://en.wikipedia.org/wiki/FourCC>.

[LEGACY]

Legacy specifications for the RTMP solution,
<https://veovera.github.io/enhanced-rtmp/docs/legacy/>.

[RFC2119]

Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.

[RFC8174]

Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>.

[RTMP]

Parmar, H., Ed. and M. Thornburgh, Ed., "Adobe’s Real Time Messaging Protocol", December 2012,
<https://veovera.github.io/enhanced-rtmp/docs/legacy/rtmp-v1-0-spec.pdf>.

[ScriptTagBody]

"Adobe Flash Video File Format Specification, Version 10.1", August 2010,
<https://veovera.org/docs/legacy/video-file-format-v10-1-spec.pdf#page=80>.

[SCRIPTDATA]

"Adobe Flash Video File Format Specification, Version 10.1", August 2010,
<https://veovera.org/docs/legacy/video-file-format-v10-1-spec.pdf#page=80>.

[WebCodecs]

W3C, "WebCodecs"
<https://www.w3.org/TR/webcodecs/>.

Sequence Diagrams (`docs/sequence-diagrams.md`)

Node-Media-Server 系统架构时序图

1. 服务器启动时序图

mermaid

sequenceDiagram
    participant User as 用户
    participant App as bin/app.js
    participant NMS as NodeMediaServer
    participant Context as Context
    participant RtmpServer as RTMP Server
    participant HttpServer as HTTP Server
    participant RecordServer as Record Server
    participant NotifyServer as Notify Server

    User->>+App: 启动 node bin/app.js
    App->>App: 加载配置文件 config.json
    App->>App: 生成随机管理员密码
    App->>App: 解析SSL证书路径
    App->>+NMS: new NodeMediaServer(config)
    NMS->>NMS: 初始化日志系统
    NMS->>+Context: 设置全局上下文
    Context->>Context: 创建 sessions Map
    Context->>Context: 创建 broadcasts Map
    Context->>Context: 创建 eventEmitter
    NMS->>+RtmpServer: 创建 RTMP 服务器
    RtmpServer->>RtmpServer: 创建TCP监听器 (端口1935)
    RtmpServer->>RtmpServer: 创建TLS监听器 (端口1936)
    NMS->>+HttpServer: 创建 HTTP 服务器
    HttpServer->>HttpServer: 设置Express路由
    HttpServer->>HttpServer: 配置JWT中间件
    HttpServer->>HttpServer: 设置WebSocket服务器
    NMS->>+RecordServer: 创建录制服务器
    NMS->>+NotifyServer: 创建通知服务器
    NMS->>NMS: 注册事件监听器
    NMS->>NMS: 启动所有服务器
    NMS-->>-App: 服务器启动完成
    App-->>-User: Node-Media-Server 运行中

2. RTMP 推流时序图

text

/* Detailed source-code truncated for AI context efficiency. */

3. RTMP 播放时序图

text

/* Detailed source-code truncated for AI context efficiency. */

4. FLV 播放时序图 (HTTP/WebSocket)

text

/* Detailed source-code truncated for AI context efficiency. */

5. WebSocket-FLV 播放时序图

mermaid

sequenceDiagram
    participant Player as 播放器
    participant HttpServer as HTTP Server
    participant WsServer as WebSocket Server
    participant FlvSession as FLV Session
    participant BroadcastServer as Broadcast Server

    Player->>+HttpServer: HTTP升级请求
    HttpServer->>WsServer: WebSocket连接
    WsServer->>+FlvSession: 创建FLV会话
    FlvSession->>FlvSession: 解析WebSocket URL
    FlvSession->>FlvSession: 提取流路径信息
    FlvSession->>BroadcastServer: 获取广播服务器
    FlvSession->>Context: 注册会话
    FlvSession-->>-WsServer: 会话就绪
    WsServer-->>Player: WebSocket连接建立

    FlvSession->>BroadcastServer: 执行postPlay
    BroadcastServer->>BroadcastServer: 验证播放权限
    BroadcastServer-->>FlvSession: 播放授权

    BroadcastServer->>FlvSession: 发送FLV头部
    FlvSession->>Player: WebSocket消息(FLV头)

    BroadcastServer->>FlvSession: 发送元数据
    FlvSession->>Player: WebSocket消息(元数据)

    BroadcastServer->>FlvSession: 发送音频头部
    FlvSession->>Player: WebSocket消息(音频头)

    BroadcastServer->>FlvSession: 发送视频头部
    FlvSession->>Player: WebSocket消息(视频头)

    BroadcastServer->>FlvSession: 发送GOP缓存
    FlvSession->>Player: WebSocket消息(GOP帧)

    loop 实时流传输
        BroadcastServer->>FlvSession: 新的媒体包
        FlvSession->>Player: WebSocket消息(FLV数据)
    end

    Player->>FlvSession: WebSocket关闭
    FlvSession->>BroadcastServer: 执行donePlay
    FlvSession->>Context: 删除会话

6. API 请求处理时序图

text

/* Detailed source-code truncated for AI context efficiency. */

7. 多协议流分发时序图

text

/* Detailed source-code truncated for AI context efficiency. */

8. 会话生命周期管理时序图

mermaid

sequenceDiagram
    participant Session as Session会话
    participant Context as 全局上下文
    participant BroadcastServer as 广播服务器
    participant EventEmitter as 事件发射器
    participant Logger as 日志系统

    Note over Session,Logger: 会话创建阶段
    Session->>Session: 生成唯一会话ID
    Session->>Context: 注册到sessions Map
    Session->>Logger: 记录会话创建日志
    Session->>EventEmitter: 触发会话事件(可选)

    Note over Session,Logger: 运行阶段
    Session->>BroadcastServer: 建立流连接
    BroadcastServer->>Session: 返回流数据
    Session->>Session: 更新统计信息(字节计数等)
    Session->>Logger: 定期记录会话状态

    Note over Session,Logger: 会话结束阶段
    Session->>Session: 检测连接断开
    Session->>BroadcastServer: 通知会话结束
    BroadcastServer->>BroadcastServer: 清理订阅者/发布者
    BroadcastServer->>EventEmitter: 触发donePlay/donePublish事件
    Session->>Context: 从sessions Map移除
    Session->>Logger: 记录会话结束日志
    Session->>Session: 释放资源

--- METRICS ---

Files Extracted: 9
Estimated Token Budget: ~34152 tokens
Canonical Reference: https://codewiki.google/github.com/illuspas/Node-Media-Server

Docker & Self-Hosted Deployment Verified OCI

Technical Documentation: illuspas/Node-Media-Server

Overview (`README.md`) Copy Section

Node-Media-Server v4

Introduction Copy Chapter

Installation Copy Chapter

Cli Options Copy Chapter

WebAdmin Copy Chapter

Features Copy Chapter

Static file services Copy Chapter

Record to flv file Copy Chapter

REST API System (New in v4.2.0) Copy Chapter

Supported clients Copy Chapter

[QLive](https://play.google.com/store/apps/details?id=cn.nodemedia.qlive) free Android Live Streaming App Copy Section

[NodePlayer.js](https://www.nodemedia.cn/product/nodeplayer-js/) pure javascript implementation live streaming player Copy Section

[NodePublisher.js](https://www.nodemedia.cn/demo/nodepublisher/) pure javascript implementation live streaming publisher Copy Section

[NodeMediaClient-iOS](https://github.com/NodeMedia/NodeMediaClient-iOS) iOS live streaming player and publisher SDK Copy Section

[NodeMediaClient-Android](https://github.com/NodeMedia/NodeMediaClient-Android) Android live streaming player and publisher SDK Copy Section

[expo-nodemediaclient](https://github.com/NodeMedia/expo-nodemediaclient) Expo module for NodeMediaClient Copy Section

Other server-side options Copy Chapter

Commercial version: [NodeMediaServer v3](https://www.nodemedia.cn/product/node-media-server/) Copy Section

Windows executable: [nginx-live-win32](https://github.com/illuspas/nginx-live-win32) Copy Section

License Copy Chapter

Overview (`webadmin/README.md`) Copy Section

React + TypeScript + Vite

React Compiler Copy Chapter

Expanding the Oxlint configuration Copy Chapter

Architecture Overview (`docs/architecture-overview.md`) Copy Section

Node-Media-Server 架构总览

项目简介 Copy Chapter

核心架构 Copy Chapter

1. 系统层次结构 Copy Section

2. 主要组件说明 Copy Section

2.1 核心组件 (Core Components)

2.2 服务器组件 (Server Components)

2.3 会话组件 (Session Components)

2.4 协议组件 (Protocol Components)

工作流程 Copy Chapter

1. 服务器启动流程 Copy Section

2. RTMP推流流程 Copy Section

3. 多协议播放流程 Copy Section

4. API请求处理流程 Copy Section

关键特性 Copy Chapter

1. 多协议支持 Copy Section

2. 现代编解码器支持 Copy Section

3. 安全特性 Copy Section

4. 监控和管理 Copy Section

5. 高性能设计 Copy Section

配置系统 Copy Chapter

配置文件结构 (`bin/config.json`) Copy Section

事件系统 Copy Chapter

生命周期事件 Copy Section

事件使用示例 Copy Section

性能优化 Copy Chapter

1. 内存管理 Copy Section

2. 网络优化 Copy Section

3. 编解码优化 Copy Section

Api (`docs/api.md`) Copy Section

REST API (New in v4.2.0)

Overview Copy Chapter

Authentication Copy Chapter

Login (Username/Password) Copy Section

Using the API Copy Section

Change Password Copy Section

Endpoints Copy Chapter

Health Check Copy Section

Server Information Copy Section

Stream Management Copy Section

Play URL Generation Copy Section

Manual Recording Copy Section

Session Management Copy Section

Server Statistics Copy Section

Event Stream (SSE) Copy Section

Relay Management Copy Section

Recordings Copy Section

Session History Copy Section

Configuration Copy Chapter

Security Features Copy Chapter

Password Storage Copy Section

JWT Configuration Copy Section

Overview (`README.md`)

Introduction

Installation

Cli Options

WebAdmin

Features

Static file services

Record to flv file

REST API System (New in v4.2.0)

Supported clients

[QLive](https://play.google.com/store/apps/details?id=cn.nodemedia.qlive) free Android Live Streaming App

[NodePlayer.js](https://www.nodemedia.cn/product/nodeplayer-js/) pure javascript implementation live streaming player

[NodePublisher.js](https://www.nodemedia.cn/demo/nodepublisher/) pure javascript implementation live streaming publisher

[NodeMediaClient-iOS](https://github.com/NodeMedia/NodeMediaClient-iOS) iOS live streaming player and publisher SDK

[NodeMediaClient-Android](https://github.com/NodeMedia/NodeMediaClient-Android) Android live streaming player and publisher SDK

[expo-nodemediaclient](https://github.com/NodeMedia/expo-nodemediaclient) Expo module for NodeMediaClient

Other server-side options

Commercial version: [NodeMediaServer v3](https://www.nodemedia.cn/product/node-media-server/)

Windows executable: [nginx-live-win32](https://github.com/illuspas/nginx-live-win32)

License

Overview (`webadmin/README.md`)

React Compiler

Expanding the Oxlint configuration

Architecture Overview (`docs/architecture-overview.md`)

项目简介

核心架构

1. 系统层次结构

2. 主要组件说明

工作流程

1. 服务器启动流程

2. RTMP推流流程

3. 多协议播放流程

4. API请求处理流程

关键特性

1. 多协议支持

2. 现代编解码器支持

3. 安全特性

4. 监控和管理

5. 高性能设计

配置系统

配置文件结构 (`bin/config.json`)

事件系统

生命周期事件

事件使用示例

性能优化

1. 内存管理

2. 网络优化

3. 编解码优化

Api (`docs/api.md`)

Overview

Authentication

Login (Username/Password)

Using the API

Change Password

Endpoints

Health Check

Server Information

Stream Management

Play URL Generation

Manual Recording

Session Management

Server Statistics

Event Stream (SSE)

Relay Management

Recordings

Session History

Configuration

Security Features

Password Storage

JWT Configuration

Example Usage

Using curl

Using JavaScript

Data Flow (`docs/data-flow.md`)

数据流架构总览

核心数据结构

1. AVPacket - 统一媒体数据包

2. Session - 会话数据结构

推流数据流分析

1. RTMP推流数据流