# Technical Documentation: tinode/chat

> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/tinode/chat)

### Overview (`README.md`)

# Tinode Instant Messaging Server

<img src="docs/logo.svg" align="left" width=128 height=128> Instant messaging full stack. Backend in pure [Go](http://golang.org) (license [GPL 3.0](http://www.gnu.org/licenses/gpl-3.0.en.html)), clients for Android (Java), iOS (Swift), and web (ReactJS), as well as [gRPC](https://grpc.io/) client support for C++, C#, Go, Java, Node, PHP, Python, Ruby, Objective-C, etc (all clients licensed under [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)). Wire transport is JSON over websocket (long polling is also available) or [protobuf](https://developers.google.com/protocol-buffers/) with gRPC.

This is beta-quality software: feature-complete and stable but probably with a few bugs or missing features. Follow [instructions](INSTALL.md) to install and run or use one of the cloud services below. Read [API documentation](docs/API.md).

Tinode is *not* XMPP/Jabber. It is *not* compatible with XMPP. It's meant as a replacement for XMPP. On the surface, it's a lot like open source WhatsApp or Telegram.

<a href="https://apps.apple.com/us/app/tinode/id1483763538"><img src="docs/app-store.svg" height=36></a> <a href="https://play.google.com/store/apps/details?id=co.tinode.tindroidx"><img src="docs/play-store.svg" height=36></a> <a href="https://web.tinode.co/"><img src="docs/web-app.svg" height=36></a>

## Why?

The promise of [XMPP](http://xmpp.org/) was to deliver federated instant messaging: anyone would be able to spin up an IM server capable of exchanging messages with any other XMPP server in the world. Unfortunately, XMPP never delivered on this promise. Instant messengers are still a bunch of incompatible walled gardens, similar to what AoL of the late 1990s was to the open Internet.

The goal of this project is to deliver on XMPP's original vision: create a modern open platform for federated instant messaging with an emphasis on mobile communication. A secondary goal is to create a decentralized IM platform that is much harder to track and block by the governments.

An explicit NON-goal: we are not building yet another Slack replacement.

## Installing and running

See [general instructions](./INSTALL.md) or [docker-specific instructions](./docker/README.md).

## Getting support

* Read [API documentation](docs/API.md) and [FAQ](docs/faq.md). Read configuration instructions contained in the [`tinode.conf`](./server/tinode.conf) file.
* For support, general questions, discussions post to [https://groups.google.com/d/forum/tinode](https://groups.google.com/d/forum/tinode).
* For bugs and feature requests [open an issue](https://github.com/tinode/chat/issues/new/choose).
* Use https://tinode.co/contact for commercial inquiries.

## Helping out

* If you appreciate our work, please help spread the word! Sharing on Reddit, HN, and other communities helps more than you think.
* Consider buying paid support: https://tinode.co/support.html
* If you are a software developer, send us your pull requests with bug fixes and new features.
* If you use the app and discover bugs or missing features, let us know by filing bug reports and feature requests. Vote for existing [feature requests](https://github.com/tinode/chat/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc+label%3A%22feature+request%22) you find most valuable.
* If you speak a language other than English, [translate](docs/translations.md) the apps into your language. You may also review and improve existing translations.
* If you are a UI/UX expert, help us polish the app UI.
* Use it: install it for your colleagues or friends at work or at home.

## Public service

A [public Tinode service](https://web.tinode.co/) is available. You can use it just like any other instant messenger. Keep in mind that demo accounts present in [sandbox](https://sandbox.tinode.co/) are not available in the public service. You must register an account using valid email in order to use the service.

### Web

TinodeWeb, a single page web app, is available at https://web.tinode.co/ ([source](https://github.com/tinode/webapp/)). See screenshots below.

### Android

[Tinode for Android](https://play.google.com/store/apps/details?id=co.tinode.tindroidx) a.k.a Tindroid is stable and functional ([source](https://github.com/tinode/tindroid)). See the screenshots below. A [debug APK](https://github.com/tinode/tindroid/releases/latest) is also provided for convenience.

### iOS

[Tinode for iOS](https://apps.apple.com/us/app/tinode/id1483763538) a.k.a. Tinodios is stable and functional ([source](https://github.com/tinode/ios)). See the screenshots below.


## Demo/Sandbox

A sandboxed demo service is available at https://sandbox.tinode.co/.

Log in as one of `alice`, `bob`, `carol`, `dave`, `frank`. Password is `<login>123`, e.g. login for `alice` is `alice123`. You can discover other users by email or phone by prefixing them with `email:` or `tel:` respectively. Emails are `<login>@example.com`, e.g. `alice@example.com`, phones are `+17025550001` through `+17025550009`.

When you register a new account you are asked for an email address to send validation code to. For demo purposes you may use `123456` as a universal validation code. The code you get in the email is also valid.

### Sandbox Notes

* The sandbox server is reset (all data wiped) every night at 3:15am Pacific time. An error message `User not found or offline` means the server was reset while you were connected. If you see it on the web, reload and relogin. On Android log out and re-login. If the database was changed, delete the app then reinstall.
* Sandbox user `Tino` is a [basic chatbot](./chatbot) which responds with a [random quote](http://fortunes.cat-v.org/) to any message.
* As generally accepted, when you register a new account you are asked for an email address. The server will send an email with a verification code to that address and you can use it to validate the account. To make things easier for testing, the server will also accept `123456` as a verification code. Remove line `"debug_response": "123456"` from `tinode.conf` to disable this option.
* The sandbox server is configured to use [ACME](https://letsencrypt.org/) TLS [implementation](https://godoc.org/golang.org/x/crypto/acme) with hard-coded requirement for [SNI](https://en.wikipedia.org/wiki/Server_Name_Indication). If you are unable to connect then the most likely reason is your TLS client's missing support for SNI. Use a different client.
* The default web app loads a single minified javascript bundle and minified CSS. The un-minified version is also available at https://sandbox.tinode.co/index-dev.html
* [Docker images](https://hub.docker.com/u/tinode/) with the same demo are available.
* You are welcome to test your client software against the sandbox, hack it, etc. No DDoS-ing though please.

## Features

### Supported

* Multiple native platforms:
  * [Android](https://github.com/tinode/tindroid/) (Java)
  * [iOS](https://github.com/tinode/ios) (Swift)
  * [Web](https://github.com/tinode/webapp/) (React.js)
  * Scriptable [command line](tn-cli/) (Python)
* User features:
  * One-on-one and group messaging.
  * Video and voice calls. Voice messages.
  * Channels with unlimited number of read-only subscribers.
  * All chats are synchronized across all devices.
  * Granular access control with permissions for various actions.
  * User search/discovery.
  * Rich formatting of messages markdown-style: \*style\* &rarr; **style**, with inline images, videos, file attachments.
  * Forms and templated responses suitable for chatbots.
  * Verified/staff/untrusted account markers.
  * Leave notes to self, bookmark (save) messages.
  * Message status notifications: message delivery to server; received and read notifications; typing notifications.
  * Most recent message preview in contact list.
  * Server-generated presence notifications for people, group chats.
  * Forwarding and replying to messages.
  * Editing sent messages.
  * Pinned chats and messages.
  * Customizable message backgrounds (wallpapers).
  * Light/dark/system UI themes.
* Administration:
  * Granular access control with permissions for various actions.
  * Support for custom authentication backends.
  * Ability to block unwanted communication server-side.
  * Anonymous users (important for use cases related to tech support over chat).
  * Plugins to extend functionality, for example, to support moderation or chatbots.
  * Scriptable [command-line tool](tn-cli/) for server administration.
* Performance, reliability and development:
  * Sharded clustering with failover.
  * Storage and out of band transfer of large objects like images or document files using local file system or Amazon S3 (other storage systems can be supported with [media handlers](https://github.com/tinode/chat/blob/master/server/media/media.go#L21)).
  * JSON or [protobuf version 3](https://developers.google.com/protocol-buffers/) wire protocols.
  * Bindings for various programming languages:
    * Javascript with no external dependencies.
    * Java with dependencies on [Jackson](https://github.com/FasterXML/jackson), [Java-Websocket](https://github.com/TooTallNate/Java-WebSocket), [ICU4J](https://github.com/unicode-org/icu). Suitable for Android but with no Android SDK dependencies.
    * Swift with no external dependencies.
    * C/C++, C#, Go, Python, PHP, Ruby and many other languages using [gRPC](https://grpc.io/docs/languages/).
  * Choice of a database backend. Other databases can be added by writing [adapters](server/db/adapter.go).
    * MySQL (and MariaDB, Percona as long as they remain SQL and wire protocol compatible)
    * PostgreSQL
    * MongoDB
    * [RethinkDB](http://rethinkdb.com/). Support is deprecated and will be dropped in 2027 because RethinkDB is no longer being developed (unless its development resumes).

### Planned

* [Federation](https://en.wikipedia.org/wiki/Federation_(information_technology)).
* Location and contacts sharing.
* Previews of attached documents, links.
* Recording video messages.
* Video/audio broadcasting.
* Group video/audio calls.
* Attaching music/audio other than voice messages.
* Different levels of message persistence (from strict persistence to "store until delivered" to purely ephemeral messaging).
* Message encryption at rest.
* End to end encryption with [OTR](https://en.wikipedia.org/wiki/Off-the-Record_Messaging) for one-on-one messaging and undecided method for group messaging.
* Full text search in messages.

### Translations

All client software has support for [internationalization](docs/translations.md). The following translations are provided:

| Language | Server | Webapp | Android | iOS |
| --- | :---: | :---: | :---: | :---: |
| English | &check; | &check; | &check; | &check; |
| Arabic |   | &check; |   |   |
| Chinese simplified | &check; | &check; | &check; | &check; |
| Chinese traditional | &check; | &check; | &check; | &check; |
| French | &check; | &check; | &check; |   |
| German |   | &check; | &check; |   |
| Hindi |   |   | &check; |   |
| Indonesian | &check; | &check; | &check; | &check; |
| Italian |   | &check; | &check; | &check; |
| Korean |   | &check; | &check; |   |
| Portuguese | &check; | &check; | &check; |   |
| Romanian |   | &check; | &check; |   |
| Russian | &check; | &check; | &check; | &check; |
| Spanish | &check; | &check; | &check; | &check; |
| Thai |   | &check; |   |   |
| Ukrainian | &check; | &check; | &check; | &check; |
| Vietnamese | &check; | &check; |   |   |

More translations are [welcome](docs/translations.md). In addition to languages listed above, particularly interested in Bengali, Urdu, Japanese, Turkish, Persian.

## Third-Party

### Projects

* [Arango DB adapter](https://github.com/gfxlabs/chat/tree/master/server/db/arango) (outdated)
* [DynamoDB adapter](https://github.com/riandyrn/chat/tree/master/server/db/dynamodb) (outdated)

### Licenses

* Demo avatars and some other graphics are from https://www.pexels.com/ under [CC0 license](https://www.pexels.com/photo-license/) and https://pixabay.com/ under their [license](https://pixabay.com/service/license/).
* Web and Android background patterns are from http://subtlepatterns.com/ under [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) license.
* Android icons are from https://material.io/tools/icons/ under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) license.

## Screenshots

### [Android](https://github.com/tinode/tindroid/)

<p align="center">
<img src="docs/android-contacts.png" alt="Android screenshot: list of chats" width=250 />
<img src="docs/android-chat.png" alt="Android screenshot: one conversation" width=250 />
<img src="docs/android-video-call.png" alt="Android screenshot: video call" width=250 />
</p>

### [iOS](https://github.com/tinode/ios)

<p align="center">
<img src="docs/ios-contacts.png" alt="iOS screenshot: list of chats" width=250 /> <img src="docs/ios-chat.png" alt="iOS screenshot: one conversation" width=250 /> <img src="docs/ios-video-call.png" alt="iOS screenshot: video call" width="250" />
</p>

### [Desktop Web](https://github.com/tinode/webapp/)

<p align="center">
  <img src="docs/web-desktop.jpg" alt="Desktop web: full app" width=810 />
</p>

### [Mobile Web](https://github.com/tinode/webapp/)

<p align="center">
  <img src="docs/web-mob-contacts.png" alt="Mobile web: contacts" width=250 /> <img src="docs/web-mob-chat.png" alt="Mobile web: chat" width=250 /> <img src="docs/web-mob-video-call.png" alt="Mobile web: topic info" width=250 />
</p>


#### SEO Strings

Words 'chat' and 'instant messaging' in Chinese, Russian, Persian and a few other languages.

* 聊天室 即時通訊
* чат мессенджер
* インスタントメッセージ
* 인스턴트 메신저
* پیام رسان فوری
* تراسل فوري
* فوری پیغام رسانی
* Nhắn tin tức thời
* anlık mesajlaşma sohbet
* mensageiro instantâneo
* pesan instan
* mensajería instantánea
* চ্যাট ইন্সট্যান্ট মেসেজিং
* चैट त्वरित संदेश
* তাৎক্ষণিক বার্তা আদান প্রদান

---

### Overview (`chatbot/README.md`)

# Tinode ChatBot Examples

* [Python chatbot](python/)
* [Karuha](https://github.com/Visecy/Karuha) - third party chatbot framework.
* [C# .Net/.NetCore chatbot](https://github.com/tinode/csharpbot)

---

### Overview (`docker/README.md`)

# Using Docker to run Tinode

All images are available at https://hub.docker.com/r/tinode/

1. [Install Docker](https://docs.docker.com/install/) 1.8 or above. The provided dockerfiles are dependent on [Docker networking](https://docs.docker.com/network/) which may not work with the older Docker.

2. Create a bridge network. It's used to connect Tinode container with the database container.
	```
	$ docker network create tinode-net
	```

3. Decide which database backend you want to use: MySQL, PostgreSQL, MongoDB or RethinkDB. Run the selected database container, attaching it to `tinode-net` network:

	1. **MySQL**: If you've decided to use MySQL backend, run the official MySQL Docker container:
	```
	$ docker run --name mysql --network tinode-net --restart always --env MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql:5.7
	```
	See [instructions](https://hub.docker.com/_/mysql/) for more options. MySQL 5.7 or above is required.

	2. **PostgreSQL**: If you've decided to use PostgreSQL backend, run the official PostgreSQL Docker container:
	```
	$ docker run --name postgres --network tinode-net --restart always --env POSTGRES_PASSWORD=postgres -d postgres:13
	```
	See [instructions](https://hub.docker.com/_/postgres/) for more options. PostgresSQL 13 or above is required.

	The name `rethinkdb`, `mysql`, `mongodb` or `postgres` in the `--name` assignment is important. It's used by other containers as a database's host name.

	3. **MongoDB**: If you've decided to use MongoDB backend, run the official MongoDB Docker container and initialise it as single node replica set (you can change "rs0" if you wish):
   ```
   $ docker run --name mongodb --network tinode-net --restart always -d mongo:latest --replSet "rs0"
   $ docker exec -it mongodb mongosh

   # And inside mongo shell:
   > rs.initiate( {"_id": "rs0", "members": [ {"_id": 0, "host": "mongodb:27017"} ]} )
   > quit()
   ```
	See [instructions](https://hub.docker.com/_/mongo/) for more options. MongoDB 4.2 or above is required.

	4. **RethinkDB**: If you've decided to use RethinkDB backend, run the official RethinkDB Docker container:
	```
	$ docker run --name rethinkdb --network tinode-net --restart always -d rethinkdb:2.3
	```
	See [instructions](https://hub.docker.com/_/rethinkdb/) for more options.

4. Run the Tinode container for the appropriate database:

	1. **MySQL**:
	```
	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-mysql:latest
	```

	2. **PostgreSQL**:
	```
	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-postgres:latest
	```

	3. **MongoDB**:
	```
	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-mongodb:latest
	```

	4. **RethinkDB**:
	```
	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-rethinkdb:latest
	```

	You can also run Tinode with the `tinode/tinode` image (which has all of the above DB adapters compiled in). You will need to specify the database adapter via `STORE_USE_ADAPTER` environment variable. E.g. for `mysql`, the command line will look like
	```
	$ docker run -p 6060:6060 -d -e STORE_USE_ADAPTER mysql --name tinode-srv --network tinode-net tinode/tinode:latest
	```

	See [below](#supported-environment-variables) for more options.

	The port mapping `-p 5678:1234` tells Docker to map container's port 1234 to host's port 5678 making server accessible at http://localhost:5678/. The container will initialize the database with test data on the first run.

	You may replace `:latest` with a different tag. See all all available tags here:
	 * [MySQL tags](https://hub.docker.com/r/tinode/tinode-mysql/tags/)
	 * [PostgreSQL tags](https://hub.docker.com/r/tinode/tinode-postgresql/tags/) (beta version)
	 * [MongoDB tags](https://hub.docker.com/r/tinode/tinode-mongodb/tags/)
	 * [RethinkDB tags](https://hub.docker.com/r/tinode/tinode-rethink/tags/)
	 * [All bundle tags](https://hub.docker.com/r/tinode/tinode/tags/)

5. Test the installation by pointing your browser to [http://localhost:6060/](http://localhost:6060/).

## Optional

### External config file

The container comes with a built-in config file which can be customized with values from the environment variables (see [Supported environment variables](#supported_environment_variables) below). If changes are extensive it may be more convenient to replace the built-in config file with a custom one. In that case map the config file located on your host (e.g. `/users/jdoe/new_tinode.conf`) to container (e.g. `/tinode.conf`) using [Docker volumes](https://docs.docker.com/storage/volumes/) `--volume /users/jdoe/new_tinode.conf:/tinode.conf` then instruct the container to use the new config `--env EXT_CONFIG=/tinode.conf`:
```
$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net \
		--volume /users/jdoe/new_tinode.conf:/tinode.conf \
		--env EXT_CONFIG=/tinode.conf \
		tinode/tinode-mysql:latest
```
When `EXT_CONFIG` is set, most other environment variables are ignored. Consult [the table](#supported-environment-variables) below for a full list.


### Resetting or upgrading the database

The database schema may change from time to time. An error `Invalid database version 101. Expected 103` means the schema has changed and needs to be updated, in this case from version 101 to version 103. You need to either reset or upgrade the database to continue:

Shut down the Tinode container and remove it:
```
$ docker stop tinode-srv && docker rm tinode-srv
```
then repeat step 4 adding `--env RESET_DB=true` to reset or `--env UPGRADE_DB=true` to upgrade.

Also, the database is automatically created if missing.


### Enable push notifications

Tinode uses Google Firebase Cloud Messaging (FCM) to send pushes.
Follow [instructions](../docs/faq.md#q-how-to-setup-fcm-push-notifications) for obtaining the required FCM credentials.

* Download and save the [FCM service account credentials](https://cloud.google.com/docs/authentication/production) file.
* Obtain values for `apiKey`, `messagingSenderId`, `projectId`, `appId`, `messagingVapidKey`.
Assuming your Firebase credentials file is named `myproject-1234-firebase-adminsdk-abc12-abcdef012345.json`
and it's saved at `/Users/jdoe/`, web API key is `AIRaNdOmX4ULR-X6ranDomzZ2bHdRanDomq2tbQ`, Sender ID `141421356237`,
Project ID `myproject-1234`, App ID `1:141421356237:web:abc7de1234fab56cd78abc`, VAPID key (a.k.a. "Web Push certificates")
is `83_Or_So_Random_Looking_Characters`, start the container with the following parameters (using MySQL container as an example):

```
$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net \
		-v /Users/jdoe:/config \
		--env FCM_CRED_FILE=/config/myproject-1234-firebase-adminsdk-abc12-abcdef012345.json \
		--env FCM_API_KEY=AIRaNdOmX4ULR-X6ranDomzZ2bHdRanDomq2tbQ \
		--env FCM_APP_ID=1:141421356237:web:abc7de1234fab56cd78abc \
		--env FCM_PROJECT_ID=myproject-1234 \
		--env FCM_SENDER_ID=141421356237 \
		--env FCM_VAPID_KEY=83_Or_So_Random_Looking_Characters \
		tinode/tinode-mysql:latest
```

### Configure video calling

Tinode uses [WebRTC](https://webrtc.org/) for video and audio calls. WebRTC needs [Interactive Communication Establishment (ICE)](https://tools.ietf.org/id/draft-ietf-ice-rfc5245bis-13.html) [TURN(S)](https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT) and/or [STUN](https://en.wikipedia.org/wiki/STUN) servers to traverse [NAT](https://en.wikipedia.org/wiki/Network_address_translation), otherwise calls may not work. Tinode does not include TURN(S) or STUN out of the box. You need to obtain and configure your own service. Once you setup your TURN(S) and/or STUN service, save its configuration to a file, for example `/Users/jdoe/turn-config.json` and provide path to this file when starting the container:

```
$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net \
		-v /Users/jdoe:/config \
		--env ICE_SERVERS_FILE=/config/turn-config.json \
		< ... other config parameters ... >
		tinode/tinode-mysql:latest
```

The config file uses the following format:
```json
[
  {
    "urls": [
      "stun:stun.example.com"
    ]
  },
  {
    "username": "user-name-to-use-for-authentication-with-the-server",
    "credential": "your-password",
    "urls": [
      "turn:turn.example.com:80?transport=udp",
      "turn:turn.example.com:3478?transport=tcp",
      "turns:turn.example.com:443?transport=tcp",
    ]
  }
]
```

[XIRSYS](https://xirsys.com/) offers a free tier for developers. We are in no way affiliated with XIRSYS. We do not endorse or otherwise take any responsibility for your use of their services.

### Run the chatbot

See [instructions](../chatbot/python/).

The chatbot password is generated only when the database is initialized or reset. It's saved to `/botdata` directory in the container. If you want to keep the data available between container changes, such as image upgrades, make sure the `/botdata` is a mounted volume (i.e. you always launch the container with `--volume botdata:/botdata` option).

## Supported environment variables

You can specify the following environment variables when issuing `docker run` command:

| Variable | Type | Default | Purpose |
| --- | --- | --- | --- |
| `ACC_GC_ENABLED`[^2] | bool | `false` | Enable/diable automatic deletion of unfinished account registrations. |
| `AUTH_TOKEN_KEY`[^2] | string | `wfaY2RgF2S1OQI/ZlK+LS​rp1KB2jwAdGAIHQ7JZn+Kc=` | base64-encoded 32 random bytes used as salt for authentication tokens. |
| `AWS_ACCESS_KEY_ID`[^2] | string |  | AWS Access Key ID when using `s3` media handler. |
| `AWS_CORS_ORIGINS`[^2] | string | `["*"]` | Allowed origins ([CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin)) URL for downloads. Generally use your server URL and its aliases. |
| `AWS_REGION`[^2] | string |  | AWS Region when using `s3` media handler |
| `AWS_S3_BUCKET`[^2] | string |  | Name of the AWS S3 bucket when using `s3` media handler. |
| `AWS_S3_ENDPOINT`[^2] | string |  | An endpoint URL (hostname only or fully qualified URI) to override the default endpoint; can be of any S3-compatible service, such as `minio-api.x.io` |
| `AWS_SECRET_ACCESS_KEY`[^2] | string |  | AWS [Secret Access Key](https://aws.amazon.com/blogs/security/wheres-my-secret-access-key/) when using `s3` media handler. |
| `CLUSTER_SELF` | string |  | Node name if the server is running in a Tinode cluster. |
| `DEBUG_EMAIL_VERIFICATION_CODE`[^2] | string |  | Enable dummy email verification code, e.g. `123456`. Disabled by default (empty string). |
| `DEFAULT_COUNTRY_CODE`[^2] | string | `US` | 2-letter country code to assign to sessions by default when the country isn't specified by the client explicitly and it's impossible to infer it. |
| `EXT_CONFIG`[^1] | string |  | Path to external config file to use instead of the built-in one. If this parameter is used, most other variables are ignored[^1]. |
| `EXT_STATIC_DIR` | string |  | Path to external directory containing static data (e.g. Tinode Webapp files). |
| `FCM_CRED_FILE`[^2] | string |  | Path to JSON file with FCM server-side service account credentials which will be used to send push notifications. |
| `FCM_API_KEY` | string |  | Firebase API key; required for receiving push notifications in the web client. |
| `FCM_APP_ID` | string |  | Firebase web app ID; required for receiving push notifications in the web client. |
| `FCM_PROJECT_ID` | string |  | Firebase project ID; required for receiving push notifications in the web client. |
| `FCM_SENDER_ID` | string |  | Firebase FCM sender ID; required for receiving push notifications in the web client. |
| `FCM_VAPID_KEY` | string |  | Also called 'Web Client certificate' in the FCM console; required by the web client to receive push notifications. |
| `FCM_INCLUDE_ANDROID_NOTIFICATION`[^2] | boolean | true | If true, pushes a data + notification message, otherwise a data-only message. [More info](https://firebase.google.com/docs/cloud-messaging/concept-options). |
| `FCM_MEASUREMENT_ID` | string |  | Google Analytics ID of the form `G-123ABCD789`. |
| `FS_CORS_ORIGINS`[^2] | string | `["*"]` | Cors origins when media is served from the file system. See `AWS_CORS_ORIGINS` for details. |
| `ICE_SERVERS_FILE`[^2] | string |  | Path to JSON file with configuration of ICE servers to be used for video calls. |
| `MEDIA_HANDLER`[^2] | string | `fs` | Handler of large files, either `fs` or `s3`. |
| `MYSQL_DSN`[^2] | string | <code>'root@tcp(mysql)/tinode?&#8203;parseTime=true&#8203;&collation=utf8mb4_0900_ai_ci'</code> | MySQL [DSN](https://github.com/go-sql-driver/mysql#dsn-data-source-name). |
| `PLUGIN_PYTHON_CHAT_BOT_ENABLED`[^2] | bool | `false` | Enable calling into the plugin provided by Python chatbot. |
| `POSTGRES_DSN`[^2] | string | <code>'postgresql://postgres:postgres@&#8203;localhost:5432/tinode?&#8203;sslmode=disable&#8203;&connect_timeout=10'</code> |  PostgreSQL [DSN](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). |
| `RESET_DB` | bool | `false` | Drop and recreate the database. |
| `SAMPLE_DATA` | string |  _see comment →_ | File with sample data to load. Default `data.json` when resetting or generating new DB, none when upgrading. Use `` (empty string) to disable. |
| `SMTP_AUTH_MECHANISM`[^2] | string | `"plain"` | SMTP authentication mechanism to use; one of "login", "cram-md5", "plain". |
| `SMTP_DOMAINS`[^2] | string |  | White list of email domains; when non-empty, accept registrations with emails from these domains only (email verification). |
| `SMTP_HELO_HOST`[^2] | string | _see comment →_ | FQDN to use in SMTP HELO/EHLO command; if missing, the hostname from `SMTP_HOST_URL` is used. |
| `SMTP_HOST_URL`[^2] | string | `'http://localhost:6060/'` | URL of the host where the webapp is running (email verification). |
| `SMTP_LOGIN`[^2] | string |  | Optional login to use for authentication with the SMTP server (email verification). |
| `SMTP_PASSWORD`[^2] | string |  | Optional password to use for authentication with the SMTP server (email verification). |
| `SMTP_PORT`[^2] | number |  | Port number of the SMTP server to use for sending verification emails, e.g. `25` or `587`. |
| `SMTP_SENDER` | string |  | [RFC 5322](https://tools.ietf.org/html/rfc5322) email address to use in the `FROM` field of verification emails, e.g. `'"John Doe" <jdoe@example.com>'`. |
| `SMTP_SERVER`[^2] | string |  | Name of the SMTP server to use for sending verification emails, e.g. `smtp.gmail.com`. If SMTP_SERVER is not defined, email verification will be disabled. |
| `STORE_USE_ADAPTER`[^2] | string |  | DB adapter name (specify with `tinode/tinode` container only). |
| `TEL_HOST_URL`[^2] | string | `'http://localhost:6060/'` | URL of the host where the webapp is running for phone verification. |
| `TEL_SENDER`[^2] | string |  | Sender name to pass to SMS sending service. |
| `TLS_CONTACT_ADDRESS`[^2] | string |  | Optional email to use as contact for [LetsEncrypt](https://letsencrypt.org/) certificates, e.g. `jdoe@example.com`. |
| `TLS_DOMAIN_NAME`[^2] | string |  | If non-empty, enables TLS (http**s**) and configures domain name of your container, e.g. `www.example.com`. In order for TLS to work you have to expose your HTTPS port to the Internet and correctly configure DNS. It WILL FAIL with `localhost` or unroutable IPs. |
| `TNPG_AUTH_TOKEN` | string |  | Tinode Push Gateway authentication token. |
| `TNPG_ORG`[^2] | string |  | Tinode Push Gateway organization name as registered at https://console.tinode.co |
| `UID_ENCRYPTION_KEY`[^2] | string | `la6YsO+bNX/+XIkOqc5Svw==` | base64-encoded 16 random bytes used as an encryption key for user IDs. |
| `UPGRADE_DB` | bool | `false` | Upgrade database schema, if necessary. |
| `WAIT_FOR` | string |  | If non-empty, waits for the specified database `host:port` to be available before starting the server. |

[^1]: If set, variables marked with the footnote `[2]` are ignored.
[^2]: Ignored if `EXT_CONFIG` is set.

A convenient way to generate a desired number of random bytes and base64-encode them on Linux and Mac:
```
$ openssl rand -base64 <desired length>
```

## Metrics Exporter

See [monitoring/exporter/README](../monitoring/exporter/README.md) for information on the Exporter.
Container is also available as a part of the Tinode docker distribution: `tinode/exporter`.
Run it with

```
$ docker run -p 6222:6222 -d --name tinode-exporter --network tinode-net \
		--env SERVE_FOR=<prometheus|influxdb> \
		--env TINODE_ADDR=<tinode metrics endpoint> \
		... <monitoring service specific vars> \
		tinode/exporter:latest
```

Available variables:

| Variable | Type | Default | Function |
| --- | --- | --- | --- |
| `SERVE_FOR` | string | `` | Monitoring service: `prometheus` or `influxdb` |
| `TINODE_ADDR` | string | `http://localhost/stats/expvar/` | Tinode metrics path |
| `INFLUXDB_VERSION` | string | `1.7` | InfluxDB version (`1.7` or `2.0`) |
| `INFLUXDB_ORGANIZATION` | string | `org` | InfluxDB organization |
| `INFLUXDB_PUSH_INTERVAL` | int | `60` | Exporter metrics push interval in seconds |
| `INFLUXDB_PUSH_ADDRESS` | string | `https://mon.tinode.co/intake` | InfluxDB backend url |
| `INFLUXDB_AUTH_TOKEN` | string | `` | InfluxDB auth token |
| `PROM_NAMESPACE` | string | `tinode` | Prometheus namespace |
| `PROM_METRICS_PATH` | string | `/metrics` | Exporter webserver path that Prometheus server scrapes |

---

### Overview (`keygen/README.md`)

# keygen: API key generator

A command-line utility to generate an API key for [Tinode server](../server/)

**Parameters:**

 * `sequence`: Sequential number of the API key. This value can be used to reject previously issued keys.
 * `isroot`: Currently unused. Intended to designate key of a system administrator.
 * `validate`: Key to validate: check previously issued key for validity.
 * `salt`: [HMAC](https://en.wikipedia.org/wiki/HMAC) salt, 32 random bytes base64 standard encoded; must be present for key validation; optional when generating the key: if missing, a cryptographically-strong salt will be automatically generated.


## Usage

The API key is used to provide some protection from automatic scraping of server API and for identification of client applications.

* `API key` is used on the client side.
* `HMAC salt` is used on the server side to verify the API key.

Run the generator:

```sh
./keygen
```

Sample output:

```text
API key v1 seq1 [ordinary]: AQAAAAABAACGOIyP2vh5avSff5oVvMpk
HMAC salt: TC0Jzr8f28kAspXrb4UYccJUJ63b7CSA16n1qMxxGpw=
```

Copy `HMAC salt` to `api_key_salt` parameter in your server [config file](https://github.com/tinode/chat/blob/master/server/tinode.conf).
Copy `API key` to the client applications:

 * TinodeWeb: `API_KEY` in [config.js](https://github.com/tinode/webapp/blob/master/src/config.js)
 * Tindroid: `API_KEY` in [Cache.java](https://github.com/tinode/tindroid/blob/master/app/src/main/java/co/tinode/tindroid/Cache.java)
 * Tinodious: `kApiKey` in [SharedUtils.swift](https://github.com/tinode/ios/blob/master/TinodiosDB/SharedUtils.swift)

Rebuild the clients after changing the API key.

---

### Overview (`loadtest/README.md`)

# Tinode Load Testing

Content of this directory is for running rudimentary load tests of Tinode server. You need this only if you want to run your own load tests.

## Tsung

The `tsung.xml` is a configuration for [Tsung](http://tsung.erlang-projects.org/). The `tinode.beam` is an erlang binary required by the test to generate base64-encoded user-password pairs. The `tinode.erl` is the source for `tinode.beam` (`erlc tinode.erl` -> `tinode.beam`).

[Install Tsung](http://tsung.erlang-projects.org/user_manual/installation.html), then run the test
```
tsung -f ./tsung.xml start
```

## Gatling

A similar loadtest scenario is also available in Gatling. The configuration file is `loadtest.scala`.
Run it with (after [installing Gatling](https://gatling.io/docs/current/installation/)):
```
gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
```

Currently, three tests are available:

* `tinode.Loadtest`: after connecting to server, retrieves user's subscriptions, and publishes a few messages to them one by one.
* `tinode.MeLoadtest`: attempts to max out `me` topic connections.
* `tinode.SingleTopicLoadtest`: connects to and publishes messages to the specified topic (typically, a group topic).

The script supports passing params via the `JAVA_OPTS` envvar.

Parameter name | Default value | Description
-------------- | ------------- | -------------
`num_sessions` | 10000 | Total number of sessions to connect to the server
`ramp` | 300 | Time period in seconds over which to ramp up the load (`0` to `num_sessions`).
`publish_count` | 10 | Number of messages that a user will publish to a topic it subscribes to.
`publish_interval` | 100 | Maximum period of time a user will wait between publishing subsequent messages to a topic.
`accounts` | users.csv | `tinode.Loadtest` and `tinode.SingleTopicLoadtest` only: Path to CSV file containing user accounts to use in loadtest (in format `username,password[,token]` (`token` field is optional).
`topic` | | `tinode.SingleTopicLoadtest` only: topic name to send load to.
`username` | | `tinode.MeLoadtest` only: user to subscribe to `me` topic.
`password` | | `tinode.MeLoadtest` only: user password.

Examples:
```shell
JAVA_OPTS="-Daccounts=users.csv -Dnum_sessions=100 -Dramp=10" gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
```
Ramps up load to 100 sessions listed in `users.csv` file over 10 seconds.

```shell
JAVA_OPTS="-Dusername=user1 -Dpassword=user1123 -Dnum_sessions=10000 -Dramp=600" gatling.sh -sf . -rsf . -rd "na" -s tinode.MeLoadtest
```
Connects 10000 sessions to `me` topic for `user1` with password `user1123` over 600 seconds.

```shell
JAVA_OPTS="-Dtopic=grpYOrcDwORhPg -Daccounts=users.csv -Dnum_sessions=10000 -Dramp=1000 -Dpublish_count=2 -Dpublish_interval=300" gatling.sh -sf . -rsf . -rd "na" -s tinode.SingleTopicLoadtest
```
Connects 10000 users (specified in `users.csv` file) to `grpYOrcDwORhPg` topic over 1000 seconds. Each user will publish 2 messages with interval up to 300 seconds.

This will be eventually packaged into a docker container.

### Experiments

We have tested our single-server Tinode synthetic setup with 50000 accounts on a standard `t3.xlarge` AWS box (4 vCPUs, 16GiB, 5Gbps network) with the `mysql` backend.
As the load increases, before starting to drop:

* The server can sustain 50000 concurrently connected sessions.
* An individual group topic was able to sustain 1500 concurrent sessions.

---

### Overview (`monitoring/README.md`)

# Monitoring Support

This directory contains code related to monitoring Tinode server. Supported monitoring services are
* [Prometheus](https://prometheus.io/)
* [InfluxDB](https://www.influxdata.com/)

See [exporter/README](./exporter/README.md) for more details.

---

### Overview (`pbx/README.md`)

# Protocol Buffer and gRPC definitions

Definitions for Tinode [gRPC](https://grpc.io/) client and plugins.

Tinode gRPC clients must implement rpc service `Node`, Tinode plugins `Plugin`.

Generated `Go` and `Python` code is included. For a sample `Python` implementation of a command line client see [tn-cli](../tn-cli/).
For a partial plugin implementation see [chatbot](../chatbot/).

If you want to make changes, you have to install protobuffers tool chain and gRPC:
```
$ python -m pip install grpcio grpcio-tools googleapis-common-protos
```

To generate `Go` bindings add the following comment to your code and run `go generate` (your actual path to `/pbx` may be different):
```
//go:generate protoc --proto_path=../pbx --go_out=plugins=grpc:../pbx ../pbx/model.proto
```

To generate `Python` bindings:
```
python -m grpc_tools.protoc -I../pbx --python_out=. --grpc_python_out=. ../pbx/model.proto
```

---

### Overview (`py_grpc/README.md`)

# Generated Protocol Buffer and gRPC files for [Tinode](https://github.com/tinode)

Generated Python code for [gRPC](https://grpc.io/) client and plugins.

gRPC clients must implement rpc service `Node`, plugins must implement `Plugin`.

For a sample implementation of a command line client see [tn-cli](https://github.com/tinode/chat/tree/master/tn-cli/).
For a partial plugin implementation see [chatbot](https://github.com/tinode/chat/tree/master/chatbot).

## Installing

Install the package by executing
```
pip install tinode_grpc
```


## Generating files

Don't modify included files directly. If you want to make changes, you have to install protobuffers tool chain and gRPC then generate the Python bindings from [`pbx/model.proto`](https://github.com/tinode/chat/tree/master/pbx/model.proto) (your path to `model.proto` may be different):
```
python -m grpc_tools.protoc -I../pbx --python_out=. --pyi_out=. --grpc_python_out=. ../pbx/model.proto
```
The generated `model_pb2_grpc.py` imports `model_pb2.py` as a module instead of a package which is incompatible with python3 packaging system. Use `../pbx/py_fix.py` to apply a fix. This is only needed if you want to repackage the generated files.

---

### Overview (`rest-auth/README.md`)

# Example of a REST authenticator server.

This is an example of a server-side [REST authenticator](../server/auth/rest/). It's a basic Python script meant to be run as a web server. It implements the required endpoints. It responds to all requests with dummy data.

The service uses [Flask](http://flask.pocoo.org/), so make sure it's installed:
```
pip install flask
```
Run the service as
```
python auth.py
```

---

### Overview (`tinode-db/README.md`)

# Utility to initialize or upgrade `tinode` DB

This utility initializes the `tinode` database (or upgrades an existing DB from an earlier version) and optionally loads it with data. To force database reset use command line option `--reset=true`.

## Build the package:

 - **RethinkDB**
  `go build -tags rethinkdb` or `go build -i -tags rethinkdb` to automatically install missing dependencies.

 - **MySQL**
  `go build -tags mysql` or `go build -i -tags mysql` to automatically install missing dependencies.

 - **MongoDB**
  `go build -tags mongodb` or `go build -i -tags mongodb` to automatically install missing dependencies.

 - **PostgreSQL**
  `go build -tags postgres` or `go build -i -tags postgres` to automatically install missing dependencies.


## Run

Run from the command line.

`tinode-db [parameters]`

Command line parameters:
 - `--reset`: delete the database then re-create it in a blank state; it has no effect if the database does not exist.
 - `--upgrade`: upgrade database from an earlier version retaining all the data; make sure to backup the DB before upgrading.
 - `--no_init`: check that database exists but don't create it if missing.
 - `--data=FILENAME`: fill `tinode` database with data from the provided file. See [data.json](data.json).
 - `--config=FILENAME`: load configuration from FILENAME. Example config is included as [tinode.conf](tinode.conf).
 - `--make_root=USER_ID`: promote an existing user to root user, `USER_ID` of the form `usrAbCDef123`.
 - `--add_root=USERNAME[:PASSWORD]`: create a new user account and make it root; if password is missing, a strong password will be generated.

Configuration file options:
 - `uid_key` is a base64-encoded 16 byte XTEA encryption key to (weakly) encrypt object IDs so they don't appear sequential. You probably want to use your own key in production.
 - `store_config.adapters.mysql` and `store_config.adapters.rethinkdb` are database-specific sections:
  - `database` is the name of the database to generate.
  - `addresses` is RethinkDB/MongoDB's host and port number to connect to. An array of hosts can be provided as well `["host1", "host2"]`.
  - `dsn` is MySQL's Data Source Name.
  - `replica_set` is MongoDB's Replicaset name.

The `uid_key` is only used if the sample data is being loaded. It should match the key of a production server and should be kept private.

The default `data.json` file creates six users with user names `alice`, `bob`, `carol`, `dave`, `frank`, and `tino` (chat bot user). Passwords are the same as the user names with 123 appended, e.g. user `alice` gets password `alice123`; `tino` gets a randomly generated password. It also creates three group topics, and multiple peer to peer topics. Users are subscribed to topics and to each other. All topics are randomly filled with messages.

Avatar photos curtesy of https://www.pexels.com/ under [CC0 license](https://www.pexels.com/photo-license/).

## Links:

* [RethinkDB schema](https://github.com/tinode/chat/tree/master/server/db/rethinkdb/schema.md)
* [MySQL schema](https://github.com/tinode/chat/tree/master/server/db/mysql/schema.sql)
* [MongoDB schema](https://github.com/tinode/chat/tree/master/server/db/mongodb/schema.md)
* [PostgreSQL schema](https://github.com/tinode/chat/tree/master/server/db/postgres/schema.sql)

---

### Overview (`tn-cli/README.md`)

# Command Line Client for Tinode

This is a scriptable command line chat client. It's written in Python and can be used to extend Tinode using [gRPC](https://grpc.io) [API](../pbx/).

Python 2.7 or 3.4+ is required. PIP 9.0.1 or newer is required.

Install dependencies:
```
$ python -m pip install -r requirements.txt
```

Run the client from the command line:
```
python tn-cli.py --login-basic=alice:alice123
```

If you are updating an existent installation, make sure the `tinode_grpc` version matches the [server](../server/) version. Upgrade `tinode_grpc` if needed:
```
python -m pip install --upgrade tinode_grpc==X.XX.XX
```
where `X.XX.XX` is the version number which must match the server version number.

The client takes optional parameters:

 * `--host` is the address of the gRPC server to connect to; default `localhost:16060`.
 * `--web-host` is the address of Tinode web server, used for file uploads only; default `localhost:6060`.
 * `--ssl` the server requires a secure connection (SSL)
 * `--ssl-host` the domain name to use for SNI if different from the `--host` domain name.
 * `--login-basic` is the `login:password` to be authenticated with.
 * `--login-token` is the token to be authenticated with.
 * `--login-cookie` direct the client to read the token from the cookie file `.tn-cli-cookie` generated during an earlier login.
 * `--no-login` do not login even if cookie file is present; this is the default in non-interactive (scripted) mode.
 * `--no-cookie` do not save cookie on successful login; this is the default in non-interactive (scripted) mode.
 * `--api-key` web API key for file uploads; default `AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K`
 * `--load-macros` path to a macro file.
 * `--verbose` log incoming and outgoing messages as JSON.
 * `--background` start interactive session in background; non-interactive sessions are always started in background.

If multiple `login-XYZ` are provided, `login-cookie` is considered first, then `login-token` then `login-basic`. Authentication with token (and cookie) is much faster than with the username-password pair.

## Commands

Type `<command> -h` for help

See some of these commands in use in the [sample-script.txt](sample-script.txt). Try it as
```
python tn-cli.py < sample-script.txt
```

### Local (non-networking)

* `.await` - issue a gRPC call and wait for completion, optionally assign result to a variable.
* `.delmark` - use custom delete marker instead of default `DEL!`; needed when some value is to be removed rather than set to blank.
* `.exit` - terminate execution and exit the CLI; also `.quit`.
* `.log` - write a value of a variable to `stdout`.
* `.must` - issue a gRPC call and wait for completion, optionally assign result to a variable; raise an exception if result is not a success.
* `.quit` - terminate execution and exit the CLI; also `.exit`.
* `.sleep` - suspend the process for a number of milliseconds.
* `.use` - set default user (on_behalf_of user) or topic.
* `.verbose` - toggle logging verbosity.

### gRPC calls

* `acc` - create  or modify an account
* `login` - authenticate current session
* `sub` - subscribe to topic
* `leave` - detach or unsubscribe from topic
* `pub` - post message to topic
* `get` - query topic for metadata or messages
* `set` - update topic metadata
* `del` - delete message(s), topic, subscription, or user
* `note` - send notification
* `file` - upload or download large file out of band

### HTTP requests

* `upload` - (deprecated, use `file`) upload file out of band

### Macros

Macros are high-level wrappers for series of gRPC calls. Currently, the following macros are [available](macros.py):

* `chacs` - change default permissions/acs for a user (requires root privileges)
* `chcred` - add or delete a credential for a user (requires root privileges)
* `passwd` - set user's password (requires root privileges)
* `resolve` - resolve login and print the corresponding user id
* `useradd` - create a new user account
* `userdel` - delete user account (requires root privileges)
* `usermod` - modify user account (requires root privileges)
* `thecard` - print user's public and private info (requires root privileges)

You can define your own macros in [macros.py](macros.py) or create a separate python module (you can load it via `--load-macros`).
Refer to [macros.py](macros.py) for examples.

## Connecting to secure (HTTPS) server

If the server is configured to use TLS, i.e. running as `httpS://my-server.example.com/`, the gRPC endpoint also uses the same SSL certificate. In that case add the `--ssl` option.

If you want to connect to the secure gRPC endpoint over a local network or under a different name i.e. as `localhost` instead of  `my-server.example.com` in this example, you must specify the SSL domain name to use, otherwise the server will not be able to find the right SSL certificate:
```
python tn-cli.py --host=localhost:6001 --ssl --ssl-host=my-server.example.com
```
The `--ssl-host` option makes the connection susceptible to the [Man-in-the-middle attack](https://en.wikipedia.org/wiki/Man-in-the-middle_attack), so don't do it over public networks.

## Crash on shutdown

Python 3.6 sometimes crashes on shutdown with a message `Fatal Python error: PyImport_GetModuleDict: no module dictionary!`. That happens because Python is buggy: https://bugs.python.org/issue26153

---

### Overview (`chatbot/csharp/README.md`)

# Tinode Chatbot Example for .Net or .NetCore

Moved to a separate repo: https://github.com/tinode/csharpbot

---

### Overview (`chatbot/python/README.md`)

# Tinode Chatbot

This is a simple chatbot for Tinode using [gRPC API](../../pbx/). It's written in Python as a demonstration
that the API is language-independent.

The chat bot subscribes to events stream using Plugin API and logs in to Tinode server as a regular user over gRPC interface (see `grpc_listen` in [tinode.conf](../../server/tinode.conf) file). The event stream API is used to listen for creation of new accounts. When a new account is created, the bot initiates a p2p topic with the new user. Then it listens for messages sent to the topic and responds to each with a random quote from `quotes.txt` file.

Generated files are provided for convenience in a [separate folder](../../py_grpc/tinode_grpc). You may re-generate them if needed:
```
python -m pip install grpcio-tools
python -m grpc_tools.protoc -../../pbx --python_out=. --grpc_python_out=. ../../pbx/model.proto
```

Chatbot expects gRPC binding to be provided as `tinode-grpc`. If you want to use them locally, first copy `model_pb2.py` and `model_pb2_grpc.py` to the same folder as `chatbot.py` then find the lines
```
from tinode_grpc import pb
from tinode_grpc import pbx
```
in `chatbot.py` and replace them with
```
import model_pb2 as pb
import model_pb2_grpc as pbx
```

## Installing and running

### Using PIP

#### Prerequisites

[gRPC](https://grpc.io/) requires [python](https://www.python.org/) 2.7 or 3.4 or higher.
Make sure [pip](https://pip.pypa.io/en/stable/installing/) 9.0.1 or higher is installed.
```
$ python -m pip install --upgrade pip
```
If you cannot upgrade pip due to a system-owned installation, you can run install it in a `virtualenv`:
```
$ python -m pip install virtualenv
$ virtualenv venv
$ source venv/bin/activate
$ python -m pip install --upgrade pip
```

#### Install dependencies:
```
$ python -m pip install -r requirements.txt
```

On El Capitan OSX, you may get the following error:
```
$ OSError: [Errno 1] Operation not permitted: '/tmp/pip-qwTLbI-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
```
You can work around this using:
```
$ python -m pip install tinode_grpc --ignore-installed
```

### Run the chatbot

Start the [tinode server](../../INSTALL.md) first. Then start the chatbot with credentials of the user you want to be your bot, `alice` in this example:
```
python chatbot.py --login-basic=alice:alice123
```
If you want to run the bot in the background, start it as
```
nohup python chatbot.py --login-basic=alice:alice123 &
```
Run `python chatbot.py -h` for more options.

If you are using python 2, keep in mind that `condition.wait()` [is forever buggy](https://bugs.python.org/issue8844). As a consequence of this bug the bot cannot be terminated with a SIGINT. It has to be stopped with a SIGKILL.

You can use cookie file to store credentials. Sample cookie files are provided as `basic-cookie.sample` and `token-cookie.sample`. Once authenticated the bot will store the token in the cookie file, `.tn-cookie` by default. If you have a cookie file with the desired credentials, you can run the bot with no parameters:
```
python chatbot.py
```

If the server is configured to use TLS, i.e. running as `httpS://my-server.example.com/`, the gRPC endpoint also uses the same SSL certificate. In that case add the `--ssl` option when starting the chatbot. If you want the chatbot to connect to the secure server over a local network or under a different name rather than the `my-server.example.com`, for instance as `localhost`, you must specify the SSL domain name to use, otherwise the server will not be able to find the right SSL certificate:
```
python chatbot.py --host=localhost:16060 --ssl --ssl-host=my-server.example.com
```

Quotes are read from `quotes.txt` by default. The file is plain text with one quote per line.


### Using Docker

**Warning!** Although the chatbot itself is less than 11KB, the chatbot Docker image is 175MB: the `:slim` Python 3 image is about 140MB, gRPC adds another ~30MB.

1. Follow [instructions](../../docker/README.md) to build and run dockerized Tinode chat server up to and including _step 3_.

2. In _step 4_ run the server adding `--env PLUGIN_PYTHON_CHAT_BOT_ENABLED=true` and `--volume botdata:/botdata` to the command line:
	1. **RethinkDB**:
	```
	$ docker run -p 6060:18080 -d --name tinode-srv --env PLUGIN_PYTHON_CHAT_BOT_ENABLED=true --volume botdata:/botdata --network tinode-net tinode/tinode-rethink:latest
	```
	2. **MySQL**:
	```
	$ docker run -p 6060:18080 -d --name tinode-srv --env PLUGIN_PYTHON_CHAT_BOT_ENABLED=true --volume botdata:/botdata --network tinode-net tinode/tinode-mysql:latest
	```
	3. **MongoDB**:
	```
	$ docker run -p 6060:18080 -d --name tinode-srv --env PLUGIN_PYTHON_CHAT_BOT_ENABLED=true --volume botdata:/botdata --network tinode-net tinode/tinode-mongodb:latest
	```

3. Run the chatbot
	```
	$ docker run -d --name tino-chatbot --network tinode-net --volume botdata:/botdata tinode/chatbot:latest
	```

4. Test that the bot is functional by pointing your browser to http://localhost:6060/, login and talk to user `Tino`. The user should respond to every message with a random quote.


You may replace the `:latest` with a different tag. See all available tags here:
 * [Tinode-MySQL tags](https://hub.docker.com/r/tinode/tinode-mysql/tags/)
 * [Tinode-RethinkDB tags](https://hub.docker.com/r/tinode/tinode-rethink/tags/)
 * [Tinode-MongoDB tags](https://hub.docker.com/r/tinode/tinode-mongodb/tags/)
 * [Chatbot tags](https://hub.docker.com/r/tinode/chatbot/tags/)

In general try to use docker images all with the same tag.

---

### Overview (`docker/docker-compose/README.md`)

# Docker compose for end-to-end setup.

These reference docker-compose files will run Tinode with the MySql backend either as [a single-instance](single-instance.yml) or [a 3-node cluster](cluster.yml) setup.

```
docker-compose -f <name of the file> [-f <name of override file>] up -d
```

By default, this command starts up a mysql instance, Tinode server(s) and Tinode exporter(s).
Tinode server(s) is(are) configured similar to [Tinode Demo/Sandbox](../../README.md#demosandbox) and
maps its web port to the host's port 6060 (6061, 6062). Tinode exporter(s) serve(s) metrics for InfluxDB.

Reference configuration for the following databases is also available in the override files:
* [PostgreSQL 15.2](https://hub.docker.com/_/postgres/tags)
* [MongoDB 4.2.3](https://hub.docker.com/_/mongo/tags)
* [RethinkDB 2.4.2](https://hub.docker.com/_/rethinkdb/tags)


## Commands

### Full stack
To bring up the full stack, you can use the following commands:
* MySql:
  - Single-instance setup: `docker-compose -f single-instance.yml up -d`
  - Cluster: `docker-compose -f cluster.yml up -d`
* PostgreSQL:
  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.postgres.yml up -d`
  - Cluster: `docker-compose -f cluster.yml -f cluster.postgres.yml up -d`
* MongoDb:
  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.mongodb.yml up -d`
  - Cluster: `docker-compose -f cluster.yml -f cluster.mongodb.yml up -d`
* RethinkDb:
  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.rethinkdb.yml up -d`
  - Cluster: `docker-compose -f cluster.yml -f cluster.rethinkdb.yml up -d`

You can run individual/separate components of the setup by providing their names to the `docker-compose` command.
E.g. to start the Tinode server in the single-instance MySql setup,
```
docker-compose -f single-instance.yml up -d tinode-0
```

### Database resets and/or version upgrades
To reset the database or upgrade the database version, you can set `RESET_DB` or `UPGRADE_DB` environment variable to true when starting Tinode with docker-compose.
E.g. for upgrading the database in MongoDb cluster setup, use:
```
UPGRADE_DB=true docker-compose -f cluster.yml -f cluster.mongodb.yml up -d tinode-0
```

For resetting the database in RethinkDb single-instance setup, run:
```
RESET_DB=true docker-compose -f single-instance.yml -f single-instance.rethinkdb.yml up -d tinode-0
```

## Troubleshooting
Print out and verify your docker-compose configuration by running:
```
docker-compose -f <name of the file> [-f <name of override file>] config
```

If the Tinode server(s) are failing, you can print the job's stdout/stderr with:
```
docker logs tinode-<instance number>
```

Additionally, you can examine the jobs `tinode.log` file. To download it from the container, run:
```
docker cp tinode-<instance number>:/var/log/tinode.log .
```

---

### Overview (`monitoring/exporter/README.md`)

# Tinode Metric Exporter

This is a simple service which reads JSON monitoring data exposed by Tinode server using [expvar](https://golang.org/pkg/expvar/) and re-publishes it in other formats. Currently the supported formats are:

* [InfluxDB](https://www.influxdata.com/) [exporter](https://docs.influxdata.com/influxdb/v1.7/tools/api/#write-http-endpoint) **pushes** data to its target backend. This is the default mode.
* [Prometheus](https://prometheus.io/) [exporter](https://prometheus.io/docs/instrumenting/exporters/) which exports data in [prometheus format](https://prometheus.io/docs/concepts/data_model/). The Prometheus monitoring service is expected to **pull/scrape** data from the  exporter.

## Usage

Exporters are intended to run next to (pair with) Tinode servers: one Exporter per one Tinode server, i.e. a single Exporter provides metrics from a single Tinode server. 

## Configuration

The exporters are configured by command-line flags:

### Common flags
* `serve_for` specifies which monitoring service the Exporter will gather metrics for; accepted values: `influxdb`, `prometheus`; default: `influxdb`.
* `tinode_addr` is the address where the Tinode instance publishes `expvar` data to scrape; default: `http://localhost:6060/stats/expvar`.
* `listen_at` is the hostname to bind to for serving the metrics; default: `:6222`.
* `instance` is the Exporter instance name (it may be exported to the upstream backend); default: `exporter`.
* `metric_list` is a comma-separated list of metrics to export; default: `Version,LiveTopics,TotalTopics,LiveSessions,ClusterLeader,TotalClusterNodes,LiveClusterNodes,memstats.Alloc`.

### InfluxDB
* `influx_push_addr` is the address of InfluxDB target server where the data gets sent; default: `http://localhost:9999/write`.
* `influx_db_version` is the version of InfluxDB (only 1.7 and 2.0 are supported); default: `1.7`.
* `influx_organization` specifies InfluxDB organization to push metrics as; default: `test`;
* `influx_bucket` is the name of InfluxDB storage bucket to store data in (used only in InfluxDB 2.0); default: `test`.
* `influx_auth_token` - InfluxDB authentication token; no default value.
* `influx_push_interval` - InfluxDB push interval in seconds; default: `30`.

#### Example

Run InfluxDB Exporter as
```
./exporter \
    --serve_for=influxdb \
    --tinode_addr=http://localhost:6060/stats/expvar \
    --listen_at=:6222 \
    --instance=exp-0 \
    --influx_push_addr=http://my-influxdb-backend.net/write \
    --influx_db_version=1.7 \
    --influx_organization=myOrg \
    --influx_auth_token=myAuthToken123 \
    --influx_push_interval=30
```

This exporter will push the collected metrics to the specified backend once every 30 seconds.


### Prometheus
* `prom_namespace` is a prefix to use for metrics names. If you are monitoring multiple tinode instances you may want to use different namespaces; default: `tinode`.
* `prom_metrics_path` is the path under which to expose the metrics for scraping; default: `/metrics`.
* `prom_timeout` is the Tinode connection timeout in seconds in response to Prometheus scrapes; default: `15`.

#### Example
Run Prometheus Exporter as
```
./exporter \
    --serve_for=prometheus \
    --tinode_addr=http://localhost:6060/stats/expvar \
    --listen_at=:6222 \
    --instance=exp-0 \
    --prom_namespace=tinode \
    --prom_metrics_path=/metrics \
    --prom_timeout=15
```

This exporter will serve data at path /metrics, on port 6222.
Once running, configure your Prometheus monitoring installation to collect data from this exporter.

---

### Overview (`server/auth/rest/README.md`)

# REST or JSON-RPC authenticator

This authenticator permits authentication of Tinode users and creation of Tinode accounts using a separate process as a
source of truth. For instance, if accounts are managed by corporate LDAP, this service allows handling of Tinode
authentication using the same LDAP service.

This authenticator calls a designated authentication service over HTTP(S) POST. A skeleton implementation of a server
is provided for reference at [rest-auth](../../../rest-auth/). The requests may be handled either by a single endpoint
or by separate per-request endpoints.

Request and response payloads are formatted as JSON. Some of the request or response fields are context-dependent and
may be skipped.



- [REST or JSON-RPC authenticator](#rest-or-json-rpc-authenticator)
	- [Configuration](#configuration)
	- [Request](#request)
	- [Response](#response)
	- [Recognized error responses](#recognized-error-responses)
	- [The server must implement the following endpoints:](#the-server-must-implement-the-following-endpoints)
		- [`add` Add new authentication record](#add-add-new-authentication-record)
			- [Sample request](#sample-request)
			- [Sample response (rec values may change)](#sample-response-rec-values-may-change)
		- [`auth` Request for authentication](#auth-request-for-authentication)
			- [Sample request](#sample-request)
			- [Sample response when the account already exists (optional challenge included)](#sample-response-when-the-account-already-exists-optional-challenge-included)
			- [Sample response when the account needs to be created by Tinode](#sample-response-when-the-account-needs-to-be-created-by-tinode)
		- [`checkunique` Checks if provided authentication record is unique.](#checkunique-checks-if-provided-authentication-record-is-unique)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)
		- [`del` Requests to delete authentication record.](#del-requests-to-delete-authentication-record)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)
		- [`gen` Generate authentication secret.](#gen-generate-authentication-secret)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)
		- [`link` Requests server to link new account ID to authentication record.](#link-requests-server-to-link-new-account-id-to-authentication-record)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)
		- [`upd` Update authentication record.](#upd-update-authentication-record)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)
		- [`rtagns` Get a list of restricted tag namespaces.](#rtagns-get-a-list-of-restricted-tag-namespaces)
			- [Sample request](#sample-request)
			- [Sample response](#sample-response)



## Configuration

Add the following section to the `auth_config` in [tinode.conf](../../tinode.conf):

```js
...
"auth_config": {
  ...
  "rest": {
    // ServerUrl is the URL of the authentication server to call. The URL must be absolute:
    // it must include the scheme, such as http or https, and the host name.
    "server_url": "http://127.0.0.1:5000/",
    // Authentication server is allowed to create new accounts.
    "allow_new_accounts": true,
    // Use separate endpoints, i.e. add request name to serverUrl path when making requests:
    // http://127.0.0.1:5000/add
    "use_separate_endpoints": true
  },
  ...
},
```
If you want to use your authenticator **instead** of stock `basic` (login-password) authentication,
add logical renaming and disable `rest` at the original name:
```js
...
"auth_config": {
  "logical_names": ["basic:rest", "rest:"],
  "rest": { ... },
  ...
},
...
```

## Request

```js
{
  "endpoint": "auth",       // string, one of the endpoints as described below, optional.
  "secret": "Ym9iOmJvYjEyMw==", // authentication secret as provided by the client,
                            // base64-encoded bytes, optional.
  "addr": "2001:0db8:85a3:0000:0000:8a2e:0370:7334", // string, IPv4 or IPv6 address of
                            // the client making the request, optional.
  "rec": {    // authentication record, optional.
    {
      "uid": "LELEQHDWbgY", // user ID, int64 base64-encoded
      "authlvl": "auth",    // authentication level
      "lifetime": "10000s", // lifetime of this record in seconds or as time.Duration string;
                            // see https://golang.org/pkg/time/#Duration for format.
       "features": 1,       // bitmap of features as integer or as a string of feature characters:
                            // "validated" (V) or "no login" (L)".
       "tags": ["email:alice@example.com"], // Tags associated with this authentication record.
       "state": "ok",       // optional account state.
    }
  }
}
```

## Response

```js
{
  "err": "internal", // string, error message in case of an error.
  "rec": {           // authentication record.
    ...              // the same as `request.rec`
  },
  "byteval": "Ym9iOmJvYjEyMw==",    // array of bytes, optional
  "ts": "2018-12-04T15:17:02.627Z", // time stamp, optional
  "boolval": true,                  // boolean value, optional
  "strarr": ["abc", "def"],         // array of strings, optoional
  "newacc": {        // data to use for creating a new account.
    // Default access mode
    "auth": "JRWPS",
    "anon": "N",
    "public": {...}, // user's public data, see /docs/API.md#trusted-public-and-private-fields
    "trusted": {...}, // user's trusted data, see /docs/API.md#trusted-public-and-private-fields
    "private": {...} // user's private data, see /docs/API.md#trusted-public-and-private-fields
  }
}
```

## Recognized error responses

The error is returned as json:

```json
{ "err": "error-message" }
```

See [here](../../store/types/types.go#L24) for an up to date list of supported error messages.

* "internal": database failure or other internal catch-all failure.
* "malformed": request cannot be parsed or otherwise wrong.
* "failed": authentication failed (wrong login or password, etc).
* "duplicate value": duplicate credential, i.e. attempt to create a record with a non-unique login.
* "unsupported": the operation is not supported.
* "expired": the secret has expired.
* "policy": policy violation, e.g. password too weak.
* "credentials": credentials like email or captcha must be validated.
* "not found": the object was not found.
* "denied": the operation is not permitted.

## The server must implement the following endpoints:

### `add` Add new authentication record

This endpoint requests server to add a new authentication record. This endpoint is generally used for account creation.
If accounts are managed externally, it's likely to be unused and should generally return an error `"unsupported"`.

#### Sample request
```json
{
  "endpoint": "add",
  "secret": "Ym9iOmJvYjEyMw==",
  "addr": "111.22.33.44",
  "rec": {
    "uid": "LELEQHDWbgY",
    "lifetime": "10000s",
    "features": 2,
    "tags": ["email:alice@example.com"]
  }
}
```

#### Sample response (rec values may change)
```json
{
  "rec": {
    "uid": "LELEQHDWbgY",
    "authlvl": "auth",
    "lifetime": "5000s",
    "features": 1,
    "tags": ["email:alice@example.com", "uname:alice"]
  }
}
```

### `auth` Request for authentication

Request to authenticate a user. Client (Tinode) provides a secret, authentication server responds with a user record.
If this is a very first login and the server manages the accounts, the server may return `newacc` object which will
be used by client (Tinode) to create the account. The server may optionally return a challenge as `byteval`.

#### Sample request
```json
{
  "endpoint": "auth",
  "secret": "Ym9iOmJvYjEyMw==",
  "addr": "111.22.33.44"
}
```

#### Sample response when the account already exists (optional challenge included)
```json
{
  "rec": {
    "uid": "LELEQHDWbgY",
    "authlvl": "auth",
    "state": "ok"
  },
  "byteval": "9X6m3tWeBEMlDxlcFAABAAEAbVs"
}
```

#### Sample response when the account needs to be created by Tinode
```js
{
  "rec": {
    "state": "suspended", // Or "ok".
    "authlvl": "auth",
    "lifetime": "5000s",
    "features": 1,
    "tags": ["email:alice@example.com", "uname:alice"]
  },
  "newacc": {
    "auth": "JRWPS",
    "anon": "N",
    "public": {/* see /docs/API.md#trusted-public-and-private-fields */},
    "trusted": {/* see /docs/API.md#trusted-public-and-private-fields */},
    "private": {/* see /docs/API.md#trusted-public-and-private-fields */}
  }
}
```

### `checkunique` Checks if provided authentication record is unique.

Request is used for account creation. If accounts are managed by the server, the server should respond with
an error `"unsupported"`.

#### Sample request
```json
{
  "endpoint": "checkunique",
  "secret": "Ym9iOmJvYjEyMw==",
  "addr": "111.22.33.44"
}
```

#### Sample response
```json
{
  "boolval": true
}
```

### `del` Requests to delete authentication record.

If accounts are managed by the server, the server should respond with an error `"unsupported"`.

#### Sample request
```json
{
  "endpoint": "del",
  "rec": {
    "uid": "LELEQHDWbgY",
  }
}
```

#### Sample response
```json
{}
```


### `gen` Generate authentication secret.

If accounts are managed by the server, the server should respond with an error `"unsupported"`.

#### Sample request
```json
{
  "endpoint": "gen",
  "rec": {
    "uid": "LELEQHDWbgY",
    "authlvl": "auth",
  }
}
```

#### Sample response
```json
{
  "byteval": "9X6m3tWeBEMlDxlcFAABAAEAbVs",
  "ts": "2018-12-04T15:17:02.627Z",
}
```


### `link` Requests server to link new account ID to authentication record.

If server requested Tinode to create a new account, this endpoint is used to link the new Tinode user ID with the
server's authentication record. If linking is successful, the server should respond with a non-empty json.

#### Sample request
```json
{
  "endpoint": "link",
  "secret": "Ym9iOmJvYjEyMw==",
  "rec": {
    "uid": "LELEQHDWbgY",
    "authlvl": "auth",
  },
}
```

#### Sample response
```json
{}
```


### `upd` Update authentication record.

If accounts are managed by the server, the server should respond with an error `"unsupported"`.

#### Sample request
```json
{
  "endpoint": "upd",
  "secret": "Ym9iOmJvYjEyMw==",
  "addr": "111.22.33.44",
  "rec": {
    "uid": "LELEQHDWbgY",
    "authlvl": "auth",
  }
}
```

#### Sample response
```json
{}
```

### `rtagns` Get a list of restricted tag namespaces.

Server may enforce certain tag namespaces (tag prefixes) to be restricted, i.e. not editable by the user.
These are also used in Tinode discovery mechanism (e.g. searching for users, contact sync). See
[API docs](/docs/API.md#fnd-and-tags-finding-users-and-topics) for details.

The server may optionally provide a regular expression to validate search tokens before rewriting them as prefixed tags.
I.e. if server allows only logins of 3-8 ASCII letters and numbers then the regexp could be `^[a-z0-9_]{3,8}$`
which is base64-encoded as `XlthLXowLTlfXXszLDh9JA==`.

#### Sample request
```json
{
  "endpoint": "rtagns",
}
```

#### Sample response
```json
{
  "strarr": ["basic", "email", "tel"],
  "byteval": "XlthLXowLTlfXXszLDh9JA=="
}
```

---

### Overview (`server/push/fcm/README.md`)

# FCM push adapter

This adapter sends push notifications to mobile clients and web browsers using [Google FCM](https://firebase.google.com/docs/cloud-messaging/). As of the time of this writing it supports Android with [Play Services](https://developers.google.com/android/guides/overview), iOS devices, and all major web browsers [excluding Safari](https://caniuse.com/#feat=push-api).

This adapter requires you to obtain your own credentials from Goole Firebase. If you want to use iOS and Android mobile apps with your service, they must be recompiled with your credentials obtained from Google. If you do not want to recompile mobile clients, consider using TNPG adapter instead.


## Configuring FCM adapter

### Server and TinodeWeb

1. Create a project at https://firebase.google.com/ if you have not done so already.
2. Follow instructions at https://cloud.google.com/iam/docs/creating-managing-service-account-keys to download the credentials file.
3. Update the server config [`tinode.conf`](../server/tinode.conf#L255), section `"push"` -> `"name": "fcm"`. Do _ONE_ of the following:
  * _Either_ enter the path to the downloaded credentials file into `"credentials_file"`.
  * _OR_ copy the file contents to `"credentials"`.<br/><br/>
    Remove the other entry. I.e. if you have updated `"credentials_file"`, remove `"credentials"` and vice versa.
4. Update [TinodeWeb](/tinode/webapp/) config [`firebase-init.js`](https://github.com/tinode/webapp/blob/master/firebase-init.js): update `apiKey`, `messagingSenderId`, `projectId`, `appId`, `messagingVapidKey`. See more info at https://github.com/tinode/webapp/#push_notifications

### iOS and Android

1. If you are using an Android client, add `google-services.json` to [Tindroid](/tinode/tindroid/) by following instructions at https://developers.google.com/android/guides/google-services-plugin and recompile the client. You may also optionally submit it to Google Play Store.
See more info at https://github.com/tinode/tindroid/#push_notifications
2. If you are using an iOS client, add `GoogleService-Info.plist` to [Tinodios](/tinode/ios/) by following instructions at https://firebase.google.com/docs/cloud-messaging/ios/client) and recompile the client. You may optionally submit the app to Apple AppStore.
See more info at https://github.com/tinode/ios/#push_notifications

---

### Overview (`server/push/stdout/README.md`)

# `stdout` push adapter

This is an adapter which logs push notifications to `STDOUT` where they can be redirected to file or processed by some other service.
This adapter is primarily intended for debugging and logging.

---

### Overview (`server/push/tnpg/README.md`)

# TNPG: Push Gateway

This is a push notifications adapter which communicates with Tinode Push Gateway (TNPG).

TNPG is a proprietary service intended to simplify deployment of on-premise installations.
Deploying a Tinode server without TNPG requires [configuring Google FCM](../fcm/) with your own credentials, recompiling Android and iOS clients, releasing them to PlayStore and AppStore under your own accounts. It's usually time consuming and relatively complex.

TNPG solves this problem by letting Tinode LLC (the company behind Tinode) to send push notifications on your behalf: you hand a notification over to TNPG, TNPG sends it to the clients using its own credentials and certificates. Internally it uses [Google FCM](https://firebase.google.com/docs/cloud-messaging/) and as such supports the same platforms as FCM. The main advantage of using TNPG over FCM is simplicity of configuration: you can use stock mobile clients with your custom Tinode server, all is needed is a configuration update on the server.

## Configuring TNPG adapter

### Obtain TNPG token

1. Register at https://console.tinode.co and create an organization.
2. Get the TPNG token from the _Self hosting_ &rarr; _Push Gateway_ section by following the instructions there.

### Configure the server
Update the server config [`tinode.conf`](../../tinode.conf#L413), section `"push"` -> `"name": "tnpg"`:
```js
{
  "enabled": true,
  "org": "myorg", // Short name (URL) of the organization you registered at console.tinode.co
  "token": "SoMe_LonG.RaNDoM-StRiNg.12345" // authentication token obtained from console.tinode.co
}
```
Make sure the `fcm` section is disabled `"enabled": false` or removed altogether.

---

### API (`docs/API.md`)

- [Server API](#server-api)
  - [How it Works?](#how-it-works)
  - [General Considerations](#general-considerations)
  - [Connecting to the Server](#connecting-to-the-server)
    - [gRPC](#grpc)
    - [WebSocket](#websocket)
    - [Long Polling](#long-polling)
    - [Out of Band Large Files](#out-of-band-large-files)
    - [Running Behind a Reverse Proxy](#running-behind-a-reverse-proxy)
  - [Users](#users)
    - [Authentication](#authentication)
      - [Creating an Account](#creating-an-account)
      - [Logging in](#logging-in)
      - [Changing Authentication Parameters](#changing-authentication-parameters)
      - [Resetting a Password, i.e. "Forgot Password"](#resetting-a-password-ie-forgot-password)
    - [Suspending a User](#suspending-a-user)
    - [Credential Validation](#credential-validation)
    - [Access Control](#access-control)
  - [Topics](#topics)
    - [me Topic](#me-topic)
    - [fnd and Tags: Finding Users and Topics](#fnd-and-tags-finding-users-and-topics)
      - [Query Language](#query-language)
      - [Incremental Updates to Queries](#incremental-updates-to-queries)
      - [Query Rewrite](#query-rewrite)
      - [Possible Use Cases](#possible-use-cases)
    - [Peer to Peer Topics](#peer-to-peer-topics)
    - [Group Topics](#group-topics)
    - [sys Topic](#sys-topic)
  - [Using Server-Issued Message IDs](#using-server-issued-message-ids)
  - [User Agent and Presence Notifications](#user-agent-and-presence-notifications)
  - [Trusted, Public, Private, Auxiliary Fields](#trusted-public-private-auxiliary-fields)
    - [Trusted](#trusted)
    - [Public](#public)
    - [Private](#private)
    - [Auxiliary](#auxiliary)
  - [Format of Content](#format-of-content)
  - [Out-of-Band Handling of Large Files](#out-of-band-handling-of-large-files)
    - [Uploading](#uploading)
    - [Downloading](#downloading)
  - [Push Notifications](#push-notifications)
    - [Tinode Push Gateway](#tinode-push-gateway)
    - [Google FCM](#google-fcm)
    - [Stdout](#stdout)
  - [Video Calls](#video-calls)
  - [Link Previews](#link-previews)
  - [Messages](#messages)
    - [Client to Server Messages](#client-to-server-messages)
      - [{hi}](#hi)
      - [{acc}](#acc)
      - [{login}](#login)
      - [{sub}](#sub)
      - [{leave}](#leave)
      - [{pub}](#pub)
      - [{get}](#get)
      - [{set}](#set)
      - [{del}](#del)
      - [{note}](#note)
    - [Server to Client Messages](#server-to-client-messages)
      - [{data}](#data)
      - [{ctrl}](#ctrl)
      - [{meta}](#meta)
      - [{pres}](#pres)
      - [{info}](#info)



# Server API

## How it Works?

Tinode is an IM router and a store. Conceptually it loosely follows a [publish-subscribe](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) model.

Server connects sessions, users, and topics. Session is a network connection between a client application and the server. User represents a human being who connects to the server with a session. Topic is a named communication channel which routes content between sessions.

Users and topics are assigned unique IDs. User ID is a string with 'usr' prefix followed by base64-URL-encoded pseudo-random 64-bit number, e.g. `usr2il9suCbuko`. Topic IDs are described below.

Clients such as mobile or web applications create sessions by connecting to the server over a websocket or through long polling. Client authentication is required in order to perform most operations. Client authenticates the session by sending a `{login}` packet. See [Authentication](#authentication) section for details. Once authenticated, the client receives a token which is used for authentication later. Multiple simultaneous sessions may be established by the same user. Logging out is not supported (and not needed).

Once the session is established, the user can start interacting with other users through topics. The following
topic types are available:

* `me` is a topic for managing one's profile and receiving notifications about other topics; `me` topic exists for every user.
* `fnd` topic is used for finding other users and topics; `fnd` topic also exists for every user.
* Peer to peer topic is a communication channel strictly between two users. Each participant sees topic name as the ID of the other participant: 'usr' prefix followed by a base64-URL-encoded numeric part of user ID, e.g. `usr2il9suCbuko`.
* Group topic is a channel for multi-user communication. It's named as 'grp' followed by 11 pseudo-random characters, i.e. `grpYiqEXb4QY6s`. Group topics must be explicitly created.

Session joins a topic by sending a `{sub}` packet. Packet `{sub}` serves three functions: creating a new topic, subscribing user to a topic, and attaching session to a topic. See [`{sub}`](#sub) section below for details.

Once the session has joined the topic, the user may start generating content by sending `{pub}` packets. The content is delivered to other attached sessions as `{data}` packets.

The user may query or update topic metadata by sending `{get}` and `{set}` packets.

Changes to topic metadata, such as changes in topic description, or when other users join or leave the topic, is reported to live sessions with `{pres}` (presence) packet. The `{pres}` packet is sent either to the topic being affected or to the `me` topic.

When user's `me` topic comes online (i.e. an authenticated session attaches to `me` topic), a `{pres}` packet is sent to `me` topics of all other users, who have peer to peer subscriptions with the first user.

## General Considerations

Timestamps are always represented as [RFC 3339](http://tools.ietf.org/html/rfc3339)-formatted string with precision up to milliseconds and timezone always set to UTC, e.g. `"2015-10-06T18:07:29.841Z"`.

Whenever base64 encoding is mentioned, it means base64 URL encoding with padding characters stripped, see [RFC 4648](http://tools.ietf.org/html/rfc4648).

The `{data}` packets have server-issued sequential IDs: base-10 numbers starting at 1 and incrementing by one with every message. They are guaranteed to be unique per topic.

In order to connect requests to responses, client may assign message IDs to all packets set to the server. These IDs are strings defined by the client. Client should make them unique at least per session. The client-assigned IDs are not interpreted by the server, they are returned to the client as is.

## Connecting to the Server

There are three ways to access the server over the network: websocket, long polling, and [gRPC](https://grpc.io/).

When the client establishes a connection to the server over HTTP(S), such as over a websocket or long polling, the server offers the following endpoints:
 * `/v0/channels` for websocket connections
 * `/v0/channels/lp` for long polling
 * `/v0/file/u` for file uploads
 * `/v0/file/s` for serving files (downloads)

`v0` denotes API version (currently zero). Every HTTP(S) request must include the API key. The server checks for the API key in the following order:
* HTTP header `X-Tinode-APIKey`
* URL query parameter `apikey` (/v0/file/s/abcdefg.jpeg?apikey=...)
* Form value `apikey`
* Cookie `apikey`

A default API key is included with every demo app for convenience. Generate your own key for production using [`keygen` utility](../keygen).

Once the connection is opened, the client must issue a `{hi}` message to the server. Server responds with a `{ctrl}` message which indicates either success or an error. The `params` field of the response contains server's protocol version `"params":{"ver":"0.15"}` and may include other values.

### gRPC

See definition of the gRPC API in the [proto file](../pbx/model.proto). gRPC API has slightly more functionality than the API described in this document: it allows the `root` user to send messages on behalf of other users as well as delete users.

The `bytes` fields in protobuf messages expect JSON-encoded UTF-8 content. For example, a string should be quoted before being converted to bytes as UTF-8: `[]byte("\"some string\"")` (Go), `'"another string"'.encode('utf-8')` (Python 3).

### WebSocket

Messages are sent in text frames, one message per frame. Binary frames are reserved for future use. By default server allows connections with any value in the `Origin` header.

### Long Polling

Long polling works over `HTTP POST` (preferred) or `GET`. In response to client's very first request server sends a `{ctrl}` message containing `sid` (session ID) in `params`. Long polling client must include `sid` in every subsequent request either in the URL or in the request body.

Server allows connections from all origins, i.e. `Access-Control-Allow-Origin: *`

### Out of Band Large Files

Large files are sent out of band using `HTTP POST` as `Content-Type: multipart/form-data`. See [below](#out-of-band-handling-of-large-files) for details.

### Running Behind a Reverse Proxy

Tinode server can be set up to run behind a reverse proxy, such as NGINX. For efficiency it can accept client connections from Unix sockets by setting `listen` and/or `grpc_listen` config parameters to the path of the Unix socket file, e.g. `unix:/run/tinode.sock`. The server may also be configured to read peer's IP address from `X-Forwarded-For` HTTP header by setting `use_x_forwarded_for` config parameter to `true`.

## Users

User is meant to represent a person, an end-user: producer and consumer of messages.

Users are generally assigned one of the two authentication levels: authenticated `auth` or anonymous `anon`. The third level `root` is only accessible over `gRPC` where it permits the `root` to send messages on behalf of other users.

When a connection is first established, the client application can send either an `{acc}` or a `{login}` message which authenticates the user at one the levels.

Each user is assigned a unique ID. The IDs are composed as `usr` followed by base64-encoded 64-bit numeric value, e.g. `usr2il9suCbuko`. Users also have the following properties:

* `created`: timestamp when the user record was created
* `updated`: timestamp of when user's `public` or `trusted` was last updated
* `status`: state of the account
* `username`: unique string used in `basic` authentication; username is not accessible to other users
* `defacs`: object describing user's default access mode for peer to peer conversations with authenticated and anonymous users; see [Access control](#access-control) for details
  * `auth`: default access mode for authenticated `auth` users
  * `anon`: default access for anonymous `anon` users
* `trusted`: an application-defined object issued by the system administration. Anyone can read it but only system administrators can change it.
* `public`: an application-defined object that describes the user. Anyone can query user for `public` data.
* `private`: an application-defined object that is unique to the current user and accessible only by the user.
* `tags`: [discovery](#fnd-and-tags-finding-users-and-topics) and credentials.

User's account has a state. The following states are defined:
 * `ok` (normal): the default state which means the account is not restricted in any way and can be used normally;
 * `susp` (suspended): the user is prevented from accessing the account as well as not found through [search](#fnd-and-tags-finding-users-and-topics); the state can be assigned by the administrator and fully reversible.
 * `del` (soft-deleted): user is marked as deleted but user's data is retained; un-deleting the user is not currenly supported.
 * `undef` (undefined): used internally by authenticators; should not be used elsewhere.

A user may maintain multiple simultaneous connections (sessions) with the server. Each session is tagged with a client-provided `User Agent` string intended to differentiate client software.

Logging out is not supported by design. If an application needs to change the user, it should open a new connection and authenticate it with the new user credentials.


### Authentication

Authentication is conceptually similar to [SASL](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer): it's provided as a set of adapters each implementing a different authentication method. Authenticators are used during account registration [`{acc}`](#acc) and during [`{login}`](#login). The server comes with the following authentication methods out of the box:

 * `token` provides authentication by a cryptographic token.
 * `basic` provides authentication by a login-password pair.
 * `anonymous` is designed for cases where users are temporary, such as handling customer support requests through chat.
 * `rest` is a [meta-method](../server/auth/rest/) which allows use of external authentication systems by means of JSON RPC.

Any other authentication method can be implemented using adapters.

The `token` is intended to be the primary means of authentication. Tokens are designed in such a way that token authentication is light weight. For instance, token authenticator generally does not make any database calls, all processing is done in-memory. All other authentication methods are intended to be used only to obtain or refresh the token. Once the token is obtained, subsequent logins should use it.

The `basic` authentication scheme expects `secret` to be a base64-encoded string of a string composed of a user name followed by a colon `:` followed by a plan text password. User name in the `basic` scheme must not contain the colon character `:` (ASCII 0x3A).

The `anonymous` scheme can be used to create accounts, it cannot be used for logging in: a user creates an account using `anonymous` scheme and obtains a cryptographic token which it uses for subsequent `token` logins. If the token is lost or expired, the user is no longer able to access the account.

Compiled-in authenticator names may be changed by using `logical_names` configuration feature. For example, a custom `rest` authenticator may be exposed as `basic` instead of default one or `token` authenticator could be hidden from users. The feature is activated by providing an array of mappings in the config file: `logical_name:actual_name` to rename or `actual_name:` to hide. For instance, to use a `rest` service for basic authentication use `"logical_names": ["basic:rest"]`.


#### Creating an Account

When a new account is created, the user must inform the server which authentication method will be later used to gain access to this account as well as provide shared secret, if appropriate. Only `basic` and `anonymous` can be used during account creation. The `basic` requires the user to generate and send a unique login and password to the server. The `anonymous` does not exchange secrets.

User may optionally set `{acc login=true}` to use the new account for immediate authentication. When `login=false` (or not set), the new account is created but the authentication status of the session which created the account remains unchanged. When `login=true` the server will attempt to authenticate the session with the new account, the `{ctrl}` response to the `{acc}` request will contain the authentication token on success. This is particularly important for the `anonymous` authentication because that's the only time when the authentication token can be retrieved.

#### Logging in

Logging in is performed by issuing a `{login}` request. Logging in is possible with `basic` and `token` only. Response to any login is a `{ctrl}` message with either a code 200 and a token which can be used in subsequent logins with `token` authentication, or a code 300 request for additional information, such as verifying credentials or responding to a method-dependent challenge in multi-step authentication, or a code 4xx error.

Token has server-configured expiration time so it needs to be periodically refreshed.

#### Changing Authentication Parameters

User may change authentication parameters, such as changing login and password, by issuing an `{acc}` request. Only `basic` authentication currently supports changing parameters:
```js
acc: {
  id: "1a2b3", // string, client-provided message id, optional
  user: "usr2il9suCbuko", // user being affected by the change, optional
  token: "XMg...g1Gp8+BO0=", // authentication token if the session
                             // is not yet authenticated, optional.
  scheme: "basic", // authentication scheme being updated.
  secret: base64encode("new_username:new_password") // new parameters
}
```
In order to change just the password, `username` should be left empty, i.e. `secret: base64encode(":new_password")`.

If the session is not authenticated, the request must include a `token`. It can be a regular authentication token obtained during login, or a restricted token received through [Resetting a Password](#resetting-a-password) process. If the session is authenticated, the token must not be included. If the request is authenticated for access level `ROOT`, then the `user` may be set to a valid ID of another user. Otherwise it must be blank (defaulting to the current user) or equal to the ID of the current user.


#### Resetting a Password, i.e. "Forgot Password"

To reset login or password, (or any other authentication secret, if such action is supported by the authenticator), one sends a `{login}` message with the `scheme` set to `reset` and the `secret` containing a base64-encoded string "`authentication scheme to reset secret for`:`reset method`:`reset method value`". Most basic case of resetting a password by email is
```js
login: {
  id: "1a2b3",
  scheme: "reset",
  secret: base64encode("basic:email:jdoe@example.com")
}
```
where `jdoe@example.com` is an earlier validated user's email.

If the email matches the registration, the server will send a message using specified method and address with instructions for resetting the secret. The email contains a restricted security token which the user can include into an `{acc}` request with the new secret as described in [Changing Authentication Parameters](#changing-authentication-parameters).

### Suspending a User

User's account can be suspended by service administrator. Once the account is suspended, the user is no longer able to login and use the service.

Only the `root` user may suspend the account. To suspend the account the root user sends the following message:
```js
acc: {
  id: "1a2b3", // string, client-provided message id, optional
  user: "usr2il9suCbuko", // user being affected by the change
  status: "susp"
}
```
Sending the same message with `status: "ok"` un-suspends the account. A root user may check account status by executing `{get what="desc"}` command against user's `me` topic.


### Credential Validation

Server may be optionally configured to require validation of certain credentials associated with the user accounts and authentication scheme. For instance, it's possible to require user to provide a unique email or a phone number, or to solve a captcha as a condition of account registration.

The server supports verification of email out of the box with just a configuration change. is mostly functional, verification of phone numbers is not functional because a commercial subscription is needed in order to be able to send text messages (SMS).

If certain credentials are required, then user must maintain them in validated state at all times. It means if a required credential has to be changed, the user must first add and validate the new credential and only then remove the old one.

Credentials are initially assigned at registration time by sending an `{acc}` message, added using `{set topic="me"}`, deleted using `{del topic="me"}`, and queries by `{get topic="me"}` messages. Credentials are verified by the client by sending either a `{login}` or an `{acc}` message.


### Access Control

Access control manages user's access to topics through access control lists (ACLs). The access is assigned individually to each user-topic pair (subscription).

Access control is mostly usable for group topics. Its usability for `me` and P2P topics is limited to managing presence notifications and banning uses from initiating or continuing P2P conversations. All channel readers are given the same permissions.

User's access to a topic is defined by two sets of permissions: user's desired permissions "want", and permissions granted to user by topic's manager(s) "given". Each permission is represented by a bit in a bitmap. It can be either present or absent. The actual access is determined as a bitwise AND of wanted and given permissions. The permissions are communicated in messages as a set of ASCII characters, where presence of a character means a set permission bit:

* No access: `N` is not a permission per se but an indicator that permissions are explicitly cleared/not set. It usually indicates that the default permissions should *not* be applied.
* Join: `J`, permission to subscribe to a topic
* Read: `R`, permission to receive `{data}` packets
* Write: `W`, permission to `{pub}` to topic
* Presence: `P`, permission to receive presence updates `{pres}`
* Approve: `A`, permission to approve requests to join a topic, remove and ban members; a user with such permission is topic's administrator
* Sharing: `S`, permission to invite other people to join the topic
* Delete: `D`, permission to hard-delete messages; only owners can completely delete topics
* Owner: `O`, user is the topic owner; the owner can assign any other permission to any topic member, change topic description, delete topic; topic may have a single owner only; some topics have no owner

When a user subscribes to a topic or starts a chat with another user, the access permissions are either set explicitly or assigned by default `defacs`. Access permissions can be modified by sending `{set}` messages.

A client may set explicit permissions in `{sub}` and `{set}` messages. If the permissions are missing or set to an empty string (not `N`!), Tinode will use default permissions `defacs` assigned earlir. If no default permissions are found, the authenticated users in group topics will receive a `JRWPS` access, in P2P topics will get `JRWPA`; anonymous users will receive `N` (no access) which means every subscription request must be explicitly approved by the topic manager.

Default access is defined for two categories of users: authenticated and anonymous. The default access value is applied as a "given" permission to all new subscriptions. Topic's default access is established at the topic creation time by `{sub.desc.defacs}` and can be subsequently modified by the owner by sending `{set}` messages. Likewise, user's default access is established at the account creation time by `{acc.desc.defacs}` and can be modified by the user by sending a `{set}` message to `me` topic.


## Topics

Topic is a named communication channel for one or more people. Topics have persistent properties. These topic properties can be queried by `{get what="desc"}` message.

Topic properties independent of the user making the query:
* `created`: timestamp of topic creation time
* `updated`: timestamp of when topic's `trusted`, `public`, or `private` was last updated
* `touched`: timestamp of the last message sent to the topic
* `defacs`: object describing topic's default access mode for authenticated and anonymous users; see [Access control](#access-control) for details
 * `auth`: default access mode for authenticated users
 * `anon`: default access for anonymous users
* `seq`: integer server-issued sequential ID of the latest `{data}` message sent through the topic
* `trusted`: an application-defined object issued by the system administrators. Anyone can read it but only administrators can change it.
* `public`: an application-defined object that describes the topic. Anyone who can subscribe to topic can receive topic's `public` data, only topic `owner` can change it.

User-dependent topic properties:
* `acs`: object describing given user's current access permissions; see [Access control](#access-control) for details
 * `want`: access permission requested by this user
 * `given`: access permissions given to this user
* `private`: an application-defined object that is unique to the current user (topic subscriber).

Topic usually have subscribers. One of the subscribers may be designated as topic owner (`O` access permission) with full access permissions. The list of subscribers can be queries with a `{get what="sub"}` message. The list of subscribers is returned in a `sub` section of a `{meta}` message.

### `me` Topic

Topic `me` is automatically created for every user at the account creation time. It serves as means of managing account information, receiving presence notification from people and topics of interest. Topic `me` has no owner. The topic cannot be deleted or unsubscribed from. One can `leave` the topic which will stop all relevant communication and indicate that the user is offline (although the user may still be logged in and may continue to use other topics).

Joining or leaving `me` generates a `{pres}` presence update sent to all users who have peer to peer topics with the given user and `P` permissions set.

Topic `me` is read-only. `{pub}` messages to `me` are rejected.

Message `{get what="desc"}` to `me` is automatically replied with a `{meta}` message containing `desc` section with the topic parameters (see intro to [Topics](#topics) section). The `public` parameters of `me` topic is data that the user wants to show to his/her connections. Changing it changes `public` not just for the `me` topic, but also everywhere where user's `public` is shown, such as `public` of all user's peer to peer topics.

Message `{get what="sub"}` to `me` is different from any other topic as it returns the list of topics that the current user is subscribed to as opposite to the expected user's subscription to `me`.
* seq: server-issued numeric id of the last message in the topic
* recv: seq value self-reported by the current user as received
* read: seq value self-reported by the current user as read
* seen: for P2P subscriptions, timestamp of user's last presence and User Agent string are reported
 * when: timestamp when the user was last online
 * ua: user agent string of the user's client software last used

Message `{get what="data"}` to `me` is rejected.

Internally the `me` topics are not persisted separately from the users. The `me` topics don't exist in the `topics` table or collection, they are created in memory from the `users` database record.

### `slf` Topic

Topic `slf` (self) provides mechanism for storing information, like bookmarks or saved messages. Messages sent to `slf` are accessible only by the user who sent them.

This topic is created automatically when the user subscribes to it for the first time.

### `fnd` and Tags: Finding Users and Topics

Topic `fnd` is automatically created for every user at the account creation time. It serves as an endpoint for discovering other users and group topics. Users and group topics can be discovered by `tags`. Tags are optionally assigned at the topic or user creation time then can be updated by using `{set what="tags"}` against a `me` or a group topic.

A tag is an arbitrary case-insensitive Unicode string (forced to lowercase on the server) up to 96 characters long which may contain characters from `Letter` and `Number` Unicode [classes/categories](https://en.wikipedia.org/wiki/Unicode_character_property#General_Category) as well as any of the following ASCII characters: `_`, `.`, `+`, `-`, `@`, `#`, `!`, `?`.

Tag may have a prefix which serves as a namespace. The prefix is a 2-16 character string which starts with a letter [a-z] and may contain lowercase ASCII letters and numbers followed by a colon `:`, ex. prefixed phone tag `tel:+14155551212` or prefixed email tag `email:alice@example.com`. Some prefixed tags are optionally enforced to be unique. In that case only one user or topic may have such a tag. Certain tags may be forced to be immutable to the user, i.e. user's attempts to add or remove an immutable tag will be rejected by the server.

The tags are indexed server-side and used in user and topic discovery. Search returns users and topics sorted by the number of matched tags in descending order.

In order to find users or topics, a user sets either `public` or `private` parameter of the `fnd` topic to a search query (see [Query language](#query-language)) then issues a `{get topic="fnd" what="sub"}` request. If both `public` and `private` are set, the `public` query is used. The `private` query is persisted across sessions and devices, i.e. all user's sessions see the same `private` query. The value of the `public` query is ephemeral, i.e. it's not saved to database and not shared between user's sessions. The `private` query is intended for large queries which do not change often, such as finding matches for everyone in user's contact list on a mobile phone. The `public` query is intended to be short and specific, such as finding some topic or a user who is not in the contact list.

The system responds with a `{meta}` message with the `sub` section listing details of the found users or topics formatted as subscriptions.

Topic `fnd` is read-only. `{pub}` messages to `fnd` are rejected.

_CURRENTLY UNSUPPORTED_ When a new user registers with tags matching the given query, the `fnd` topic will receive `{pres}` notification for the new user.

[Plugins](../pbx) support `Find` service which can be used to replace default search with a custom one.

Internally the `fnd` topics are not persisted separately from the users. The `fnd` topics don't exist in the `topics` table or collection, they are created in memory from the `users` database record.

#### Query Language

Tinode query language is used to define search queries for finding users and topics. The query is a string containing atomic terms separated by spaces or commas. The individual query terms are matched against user's or topic's tags. The individual terms may be written in an RTL language but the query as a whole is parsed left to right. Spaces are treated as the `AND` operator, commas (as well as commas preceded and/or followed by a space) as the `OR` operator. The order of operators is ignored: all `AND` tags are grouped together, all `OR` tags are grouped together. `OR` takes precedence over `AND`: if a tag is preceded of followed by a comma, it's an `OR` tag, otherwise an `AND`. For example, `aaa bbb, ccc` (`aaa AND bbb OR ccc`) is interpreted as `(bbb OR ccc) AND aaa`.

Query terms containing spaces must convert spaces to underscores ` ` -> `_`, e.g. `new york` -> `new_york`.

**Some examples:**
* `flowers`: find topics or users which contain tag `flowers`.
* `flowers travel`: find topics or users which contain both tags `flowers` and `travel`.
* `flowers, travel`: find topics or users which contain either tag `flowers` or `travel` (or both).
* `flowers travel, puppies`: find topics or users which contain `flowers` and either `travel` or `puppies`, i.e. `(travel OR puppies) AND flowers`.
* `flowers, travel puppies, kittens`: find topics or users which contain either one of `flowers`, `travel`, `puppies`, or `kittens`, i.e. `flowers OR travel OR puppies OR kittens`. The space between `travel` and `puppies` is treated as `OR` due to `OR` taking precedence over `AND`.

#### Incremental Updates to Queries

_CURRENTLY UNSUPPORTED_ Queries, particularly `fnd.private` could be arbitrarily large, limited only by the limits on the message size, and by the limits on the query size in the underlying database. Instead of rewriting the entire query to add or remove a term, terms can be added or removed incrementally.

The incremental update request is processed left to right. It may contain the same term multiple times, i.e. `-a_tag+a_tag` is a valid request.

#### Query Rewrite

Finding users by login, phone or email requires query terms to be written with prefixes, i.e. `email:alice@example.com` instead of `alice@example.com`. This may present a problem to end users because it requires them to learn the query language. Tinode solves this problem by implementing _query rewrite_ on the server: if query term (tag) does not contain a prefix, server rewrites it by adding the appropriate prefix. In queries to `fnd.public` the original term is also kept (query `alice@example.com` is rewritten as `email:alice@example.com OR alice@example.com`), in queries to `fnd.private` only the rewritten term is kept (`alice@example.com` is rewritten as `email:alice@example.com`). All terms that look like email, for instance, `alice@example.com` are rewritten to `email:alice@example.com OR alice@example.com`. Terms which look like phone numbers are converted to [E.164](https://en.wikipedia.org/wiki/E.164) and also rewritten as `tel:+14155551212 OR +14155551212`. In addition, in queries to `fnd.public` all other unprefixed terms which look like logins are rewritten as logins: `alice` -> `basic:alice OR alice`.

As described above, tags which look like phone numbers are converted to E.164 format. Such conversion requires an ISO 3166-1 alpha-2 country code. The following logic is used when converting phone number tags to E.164:
* If the tag already contains a country calling code, it's used as is: `+1(415)555-1212` -> `+14155551212`.
* If the tag has no prefix, country code is taken from the locale value set by the client in `lang` field of the `{hi}` message.
* If client has not provided the code in the `hi.lang`, the country code is taken from `default_country_code` field of the `tinode.conf`.
* If no `default_country_code` is set in `tinode.conf`, `US` country code is used.

#### Possible Use Cases
* Restricting users to organisations.
  An immutable tag(s) may be assigned to the user which denotes the organisation the user belongs to. When the user searches for other users or topics, the search can be restricted to always contain the tag. This approach can be used to segment users into organisations with limited visibility into each other.

* Search by geographical location.
  Client software may periodically assign a [geohash](https://en.wikipedia.org/wiki/Geohash) tag to the user based on current location. Searching for users in a given area would mean matching on geohash tags.

* Search by numerical range, such as age range.
  The approach is similar to geohashing. The entire range of numbers is covered by the smallest possible power of 2, for instance the range of human ages is covered by 2<sup>7</sup>=128 years. The entire range is split in two halves: the range 0-63 is denoted by 0, 64-127 by 1. The operation is repeated with each subrange, i.e. 0-31 is 00, 32-63 is 01, 0-15 is 000, 32-47 is 010. Once completed, the age 30 will belong to the following ranges: 0 (0-63), 00 (0-31), 001 (16-31), 0011 (24-31), 00111 (28-31), 001111 (30-31), 0011110 (30). A 30 y.o. user is assigned a few tags to indicate the age, i.e. `age:00111`, `age:001111`,  and `age:0011110`. Technically, all 7 tags may be assigned but usually it's impractical. To query for anyone in the age range 28-35 convert the range into a minimal number of tags: `age:00111` (28-31), `age:01000` (32-35). This query will match the 30 y.o. user by tag `age:00111`.


### Peer to Peer Topics

Peer to peer (P2P) topics represent communication channels between strictly two users. The name of the topic is different for each of the two participants. Each of them sees the name of the topic as the user ID of the other participant: `usr` followed by base64 URL-encoded ID of the user. For example, if two users `usrOj0B3-gSBSs` and `usrIU_LOVwRNsc` start a P2P topic, the first one will see it as `usrIU_LOVwRNsc`, the second as `usrOj0B3-gSBSs`. The P2P topic has no owner.

A P2P topic is created by one user subscribing to topic with the name equal to the ID of the other user. For instance, user `usrOj0B3-gSBSs` can establish a P2P topic with user `usrIU_LOVwRNsc` by sending a `{sub topic="usrIU_LOVwRNsc"}`. Tinode will respond with a `{ctrl}` packet with the name of the newly created topic as described above. The other user will receive a `{pres}` message on `me` topic with updated access permissions.

Internally, P2P topics are stored as `p2p` followed by base64 URL encoded concatenation of two 64-bit user IDs, with the lower numeric value ID first: `p2pm7PvMGmdcx_uVkDRaSTbwA`.

The 'public' parameter of P2P topics is user-dependent. For instance a P2P topic between users A and B would show user A's 'public' to user B and vice versa. If a user updates 'public', all user's P2P topics will automatically update 'public' too.

The 'private' parameter of a P2P topic is defined by each participant individually as with any other topic type.

### Group Topics

Group topic represents a communication channel between multiple users. The name of a group topic is `grp` or `chn` followed by a string of characters from base64 URL-encoding set. No other assumptions can be made about internal structure or length of the group name.

Group topics support limited number of subscribers (controlled by a `max_subscriber_count` parameter in configuration file) with access permissions of each subscriber managed individually. Group topics may also be enabled to support any number of read-only users - `readers`. All `readers` have the same access permissions. Group topics with enabled `readers` are called `channels`.

A group topic is created by sending a `{sub}` message with the topic field set to string `new` or `nch` optionally followed by any characters, e.g. `new` or `newAbC123` are equivalent. Tinode will respond with a `{ctrl}` message with the name of the newly created topic, i.e. `{sub topic="new"}` is replied with `{ctrl topic="grpmiKBkQVXnm3P"}`. If topic creation fails, the error is reported on the original topic name, i.e. `new` or `newAbC123`. The user who created the topic becomes topic owner. Ownership can be transferred to another user with a `{set}` message but one user must remain the owner at all times.

A `channel` topic is different from the non-channel group topic in the following ways:

 * Channel topic is created by sending `{sub topic="nch"}`. Sending `{sub topic="new"}` will create a group topic without enabling channel functionality.
 * Sending `{sub topic="chnAbC123"}` will create a `reader` subscription to a channel. A non-channel topic will reject such subscription request.
 * When searching for topics using [`fnd`](#fnd-and-tags-finding-users-and-topics), channels will show addresses with `chn` prefixes, non-channel topic will show with `grp` prefixes.
 * Messages received by readers on channels have no `From` field. Normal subscribers will receive messages with `From` containing ID of the sender.
 * Default permissions for a channel and non-channel group topics are different: channel group topic grants no permissions at all.
 * A subscriber joining or leaving the topic (regular or channel-enabled) generates a `{pres}` message to all other subscribers who are currently in the joined state with the topic and have appropriate permissions. Reader joining or leaving the channel generates no `{pres}` message.

### `sys` Topic

The `sys` topic serves as an always available channel of communication with the system administrators. A normal non-root user cannot subscribe to `sys` but can publish to it without subscription. Existing clients use this channel to report abuse by sending a Drafty-formatted `{pub}` message with the report as JSON attachment. A root user can subscribe to `sys` topic. Once subscribed, the root user will receive messages sent to `sys` topic by other users.

## Using Server-Issued Message IDs

Tinode provides basic support for client-side caching of `{data}` messages in the form of server-issued sequential message IDs. The client may request the last message id from the topic by issuing a `{get what="desc"}` message. If the returned ID is greater than the ID of the latest received message, the client knows that the topic has unread messages and their count. The client may fetch these messages using `{get what="data"}` message. The client may also paginate history retrieval by using message IDs.

## User Agent and Presence Notifications

A user is reported as being online when one or more of user's sessions are attached to the `me` topic. Client-side software identifies itself to the server using `ua` (user agent) field of the `{login}` message. The _user agent_ is published in `{meta}` and `{pres}` messages in the following way:

 * When user's first session attaches to `me`, the _user agent_ from that session is broadcast in the `{pres what="on" ua="..."}` message.
 * When multiple user sessions are attached to `me`, the _user agent_ of the session where the most recent action has happened is reported in `{pres what="ua" ua="..."}`; the 'action' in this context means any message sent by the client. To avoid potentially excessive traffic, user agent changes are broadcast no more frequently than once a minute.
 * When user's last session detaches from `me`, the _user agent_ from that session is recorded together with the timestamp; the user agent is broadcast in the `{pres what="off"  ua="..."}` message and subsequently reported as the last online timestamp and user agent.

An empty `ua=""` _user agent_ is not reported. I.e. if user attaches to `me` with non-empty _user agent_ then does so with an empty one, the change is not reported. An empty _user agent_ may be disallowed in the future.

## Trusted, Public, Private, Auxiliary Fields

Topics have `trusted`, `public`, `aux` fields, subscriptions have `private` fields. The primary difference between these fields is in access control:

 * `trusted`: writable by `ROOT` users, readable by anyone.
 * `public`: writable by the `owner` or the user, readable by anyone.
 * `aux`: writable by topic administrators, readable by subscribers.
 * `private`: readable and writable only by the user who created the subscription.

Generally, the fields are application-defined. The server does not enforce any particular structure of these fields except for `fnd` topic. At the same time, client software should use the same format for interoperability reasons. The following sections describe the format of these fields as they are implemented by all official clients.

Although it's not yet enforced, if a third-party application defines custom keys, the key names should start with an `x-` followed by the application's fully qualified domain name, e.g. `x-example.com-value: "abc"`. The fields should contain primitive types only, i.e. `string`, `boolean`, `number`, or `null`.

### Trusted

The format of the optional `trusted` field in group and peer to peer topics is a set of key-value pairs; `fnd` and `sys` topics do not have the `trusted`. The field is writable by `ROOT` users, readable by anyone who has access to the topic or user. The following optional keys are currently defined:
```js
trusted: {
  verified: true, // boolean, an indicator of a verified/trustworthy user or topic.
  staff: true,    // boolean, an indicator that the user or topic
                  // is a part of/belongs to the server administration.
  danger: true    // boolean, an indicator that the user or topic are untrustworthy.
}
```

### Public

The format of the `public` field in group, peer to peer, systems topics is expected to be [theCard](./thecard.md). The field is writable by by the user for users, the topic owner for topics. The field is readable by anyone who has access to topic or user.

The `fnd` topic expects `public` to be a string representing a [search query](#query-language)).

### Private

The format of the `private` field in group and peer to peer topics is a set of key-value pairs. The field is writable and readable by the user only. The following keys are currently defined:
```js
private: {
  comment: "some comment", // string, optional user comment about a topic or a peer user
  arch: true, // boolean, indicator that the topic is archived by the user, i.e.
              // should not be shown in the UI with other non-archived topics.
  accepted: "JRWS", // string, 'given' mode accepted by the user.
  tpins: ["grpmiKBkQVXnm3P", "usrIU_LOVwRNsc"] // array of topic IDs to pin to the top of
              // the contacts list; 'me' topic only.
}
```

The `fnd` topic expects `private` to be a string representing a [search query](#query-language)).

### Auxiliary

The format of the `aux` field is a set of key-value pairs. The `aux` is writable by topic admins and readable by all topic subscribers. The following keys are currently defined:

```js
aux: {
  pins: [1001, 23456] // array of integer message IDs to pin to the top of the message list.
}
```

## Format of Content

Format of `content` field in `{pub}` and `{data}` is application-defined and as such the server does not enforce any particular structure of the field. At the same time, client software should use the same format for interoperability reasons. Currently the following two types of `content` are supported:
 * Plain text
 * [Drafty](./drafty.md)

If Drafty is used, a message header `"head": {"mime": "text/x-drafty"}` must be set.


## Out-of-Band Handling of Large Files

Large files create problems when sent in-band for multiple reasons:
 * limits on database storage as in-band messages are stored in database fields
 * in-band messages must be downloaded completely as a part of downloading chat history

Tinode provides two endpoints for handling large files: `/v0/file/u` for uploading files and `v0/file/s` for downloading. The endpoints require the client to provide both [API key](#connecting-to-the-server) and login credentials. The server checks credentials in the following order:

**Login credentials**
 * HTTP header `Authorization` (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization)
 * URL query parameters `auth` and `secret` (/v0/file/s/abcdefg.jpeg?auth=...&secret=...)
 * Form values `auth` and `secret`
 * Cookies `auth` and `secret`

### Uploading

To upload a file first create an RFC 2388 multipart request then send it to the server using HTTP POST. The server responds to the request either with a `307 Temporary Redirect` with the new upload URL, or with a `200 OK` and a `{ctrl}` message in response body:

```js
ctrl: {
  params: {
    url: "/v0/file/s/mfHLxDWFhfU.pdf"
  },
  code: 200,
  text: "ok",
  ts: "2018-07-06T18:47:51.265Z"
}
```
If `307 Temporary Redirect` is returned, the client must retry the upload at the provided URL. The URL returned in `307` response should be used for just this one upload. All subsequent uploads should try the default URL first.

The `ctrl.params.url` contains the path to the uploaded file at the current server. It could be either the full path like `/v0/file/s/mfHLxDWFhfU.pdf`, a relative path like `./mfHLxDWFhfU.pdf`, or just the file name `mfHLxDWFhfU.pdf`. Anything but the full path is interpreted against the default *download* endpoint `/v0/file/s/`. For instance, if `mfHLxDWFhfU.pdf` is returned then the file is located at `http(s)://current-tinode-server/v0/file/s/mfHLxDWFhfU.pdf`.

Once the URL of the file is received, either immediately or after following the redirect, the client may use the URL to send a `{pub}` message with the uploaded file as an attachment, or, if the file is an image, as an avatar image for a topic or user profile (see [theCard](./thecard.md)). For example, the URL can be used in a [Drafty](./drafty.md)-formatted `pub.content` field:

```js
{
  pub: {
    id: "121103",
    topic: "grpnG99YhENiQU",
    head: {
      mime: "text/x-drafty"
    },
    content: {
      ent: [
      {
        data: {
        mime: "image/jpeg",
        name: "roses-are-red.jpg",
        ref:  "/v0/file/s/sJOD_tZDPz0.jpg",
        size: 437265
      },
        tp: "EX"
      }
    ],
    fmt: [
      {
        at: -1,
      key:0,
      len:1
      }
    ]
    }
  },
  extra: {
    attachments: ["/v0/file/s/sJOD_tZDPz0.jpg"]
  }
}
```

It's important to list the used URLs in the `extra: attachments[...]` field. Tinode server uses this field to maintain the uploaded file's use counter. Once the counter drops to zero for the given file (for instance, because a message with the shared URL was deleted or because the client failed to include the URL in the `extra.attachments` field), the server will garbage collect the file. Only relative URLs should be used. Absolute URLs in the `extra.attachments` field are ignored. The URL value is expected to be the `ctrl.params.url` returned in response to upload.

### Downloading

The serving endpoint `/v0/file/s` serves files in response to HTTP GET requests. The client must evaluate relative URLs against this endpoint, i.e. if it receives a URL `mfHLxDWFhfU.pdf` or `./mfHLxDWFhfU.pdf` it should interpret it as a path `/v0/file/s/mfHLxDWFhfU.pdf` at the current Tinode HTTP server.

_Important!_ As a security measure, the client should not send security credentials if the download URL is absolute and leads to another server.

## Push Notifications

Tinode uses compile-time adapters for handling push notifications. The server comes with [Tinode Push Gateway](../server/push/tnpg/), [Google FCM](https://firebase.google.com/docs/cloud-messaging/), and `stdout` adapters. Tinode Push Gateway and Google FCM support Android with [Play Services](https://developers.google.com/android/guides/overview) (may not be supported by some Chinese phones), iOS devices and all major web browsers excluding Safari. The `stdout` adapter does not actually send push notifications. It's mostly useful for debugging, testing and logging. Other types of push notifications such as [TPNS](https://intl.cloud.tencent.com/product/tpns) can be handled by writing appropriate adapters.

If you are writing a custom plugin, the notification payload is the following:
```js
{
  topic: "grpnG99YhENiQU", // Topic which received the message.
  xfrom: "usr2il9suCbuko", // ID of the user who sent the message.
  ts: "2019-01-06T18:07:30.038Z", // message timestamp in RFC3339 format.
  seq: "1234", // sequential ID of the message (integer value sent as text).
  mime: "text/x-drafty", // optional message MIME-Type.
  content: "Lorem ipsum dolor sit amet, consectetur adipisci", // The first 80 characters of the message content as plain text.
}
```

### Tinode Push Gateway

Tinode Push Gateway (TNPG) is a proprietary Tinode service which sends push notifications on behalf of Tinode. Internally it uses Google FCM and as such supports the same platforms as FCM. The main advantage of using TNPG over FCM is simplicity of configuration: mobile clients do not need to be recompiled, all is needed is a [configuration update](../server/push/tnpg/) on a server.

### Google FCM

[Google FCM](https://firebase.google.com/docs/cloud-messaging/) supports Android with [Play Services](https://developers.google.com/android/guides/overview), iPhone and iPad devices, and all major web browsers excluding Safari. In order to use FCM mobile clients (iOS, Android) must be recompiled with credentials obtained from Google. See [instructions](../server/push/fcm/) for details.

### Stdout

The `stdout` adapter is mostly useful for debugging and logging. It writes push payload to `STDOUT` where it can be redirected to file or read by some other process.

## Video Calls

[See separate document](call-establishment.md).

## Link Previews

Tinode provides an optional service which helps client applications generate link (URL) previews for inclusion into messages. The enpoint of this service (if enabled) is located at `/v0/urlpreview`. The service takes a single parameter `url`:

```
/v0/urlpreview?url=https%3A%2F%2Ftinode.co
```
The first several kilobytes of the document at the given URL is fetched by issuing an HTTP(S) GET request. If the returned document has content-type `text/html`, the HTML is parsed for page title, description, and image URL. The result is formatted as JSON and returned as

```json
{"title": "Page title", "description": "This is a page description", "image_url": "https://tinode.co/img/logo64x64.png"}
```

The link preview service requires authentication. It's exactly the same as authentication for [Out of Band Large Files](#out-of-band-handling-of-large-files).

## Messages

A message is a logically associated set of data. Messages are passed as JSON-formatted UTF-8 text.

All client to server messages may have an optional `id` field. It's set by the client as means to receive an acknowledgement from the server that the message was received and processed. The `id` is expected to be a session-unique string but it can be any string. The server does not attempt to interpret it other than to check JSON validity. The `id` is returned unchanged by the server when it replies to the client message.

Server requires strictly valid JSON, including double quotes around field names. For brevity the notation below omits double quotes around field names as well as outer curly brackets. Examples use `//` comments only for expressiveness. The comments cannot be used in actual communication with the server.

For messages that update application-defined data, such as `{set}` `private` or `public` fields, when server-side
data needs to be cleared, use a string with a single Unicode DEL character "&#x2421;" (`\u2421`). I.e. sending `"public": null` will not clear the field, but sending `"public": "␡"` will.

Any unrecognized fields are silently ignored by the server.

### Client to Server Messages

Every client to server message contains the main payload described in the sections below and an optional top-level field `extra`:
```js
{
  abc: { ... }, // Main payload, see sections below.
  extra: {
    attachments: ["/v0/file/s/sJOD_tZDPz0.jpg"], // Array of out-of-band attachments which have to be exempted from GC.
    obo: "usr2il9suCbuko", // Alternative user ID set by the root user (obo = On Behalf Of).
    authlevel: "auth"  // Altered authentication level set by the root user.
  }
}
```
The `attachments` array lists URLs of files uploaded out of band. Such listing increments use counter of these files. Once the use counter drops to 0, the files will be automatically deleted.
The `obo` (On Behalf Of) can be set by the `root` user. If the `obo` is set, the server will treat the message as if it came from the specified user as opposite to the actual sender.
The `authlevel` is supplementary to the `obo` and permits setting custom authentication level for the user. A an `"auth"` level is used if the field is unset.

#### `{hi}`

Handshake message client uses to inform the server of its version and user agent. This message must be the first that
the client sends to the server. Server responds with a `{ctrl}` which contains server build `build`, wire protocol version `ver`,
session ID `sid` in case of long polling, as well as server constraints, all in `ctrl.params`.

```js
hi: {
  id: "1a2b3",     // string, client-provided message id, optional
  ver: "0.15.8-rc2", // string, version of the wire protocol supported by the client, required
  ua: "JS/1.0 (Windows 10)", // string, user agent identifying client software,
                   // optional
  dev: "L1iC2...dNtk2", // string, unique value which identifies this specific
                   // connected device for the purpose of push notifications; not
                   // interpreted by the server.
                   // see [Push notifications support](#push-notifications-support); optional
  platf: "android", // string, underlying OS for the purpose of push notifications, one of
                   // "android", "ios", "web"; if missing, the server will try its best to
                   // detect the platform from the user agent string; optional
  lang: "en-US"    // human language of the client device; optional
}
```
The user agent `ua` is expected to follow [RFC 7231 section 5.5.3](http://tools.ietf.org/html/rfc7231#section-5.5.3) recommendation but the format is not enforced. The message can be sent more than once to update `ua`, `dev` and `lang` values. If sent more than once, the `ver` field of the second and subsequent messages must be either unchanged or not set.

#### `{acc}`

Message `{acc}` creates users or updates `tags` or authentication credentials `scheme` and `secret` of exiting users. To create a new user set `user` to the string `new` optionally followed by any character sequence, e.g. `newr15gsr`. Either authenticated or anonymous session can send an `{acc}` message to create a new user. To update authentication data or validate a credential of the current user leave `user` unset.

The `{acc}` message **cannot** be used to modify `desc` or `cred` of an existing user. Update user's `me` topic instead.

```
/* Detailed source-code truncated for AI context efficiency. */
```

Server responds with a `{ctrl}` message with `params` containing details of the new user account such as user ID and, in case of `login: true`, authentication token. If `desc.defacs` is missing, the server will assign server-default access permissions to new account.

The only supported authentication schemes for account creation are `basic` and `anonymous`.

#### `{login}`

Login is used to authenticate the current session.

```js
login: {
  id: "1a2b3",     // string, client-provided message id, optional
  scheme: "basic", // string, authentication scheme; "basic",
                   // "token", and "reset" are currently supported
  secret: base64encode("username:password"), // string, base64-encoded secret for the chosen
                  // authentication scheme, required
  cred: [
    {
      meth: "email", // string, verification method, e.g. "email", "tel", "captcha", etc, required
      resp: "178307" // string, verification response, required
    },
  ...
  ],   // response to a request for credential verification, optional
}
```

Server responds to a `{login}` packet with a `{ctrl}` message. The `params` of the message contains the id of the logged in user as `user`. The `token` contains an encrypted string which can be used for authentication. Expiration time of the token is passed as `expires`.

#### `{sub}`

The `{sub}` packet serves the following functions:
 * creating a new topic
 * subscribing user to an existing topic
 * attaching session to a previously subscribed topic
 * fetching topic data

User creates a new group topic by sending `{sub}` packet with the `topic` field set to `new12321` (regular topic) or `nch12321` (channel) where `12321` denotes any string including an empty string. Server will create a topic and respond back to the session with the name of the newly created topic.

User creates a new peer to peer topic by sending `{sub}` packet with `topic` set to peer's user ID.

The user is always subscribed to and the session is attached to the newly created topic.

If the user had no relationship with the topic, sending `{sub}` packet creates it. Subscribing means to establish a relationship between session's user and the topic where no relationship existed in the past.

Joining (attaching to) a topic means for the session to start consuming content from the topic. Server automatically differentiates between subscribing and joining/attaching based on context: if the user had no prior relationship with the topic, the server subscribes the user then attaches the current session to the topic. If relationship existed, the server only attaches the session to the topic. When subscribing, the server checks user's access permissions against topic's access control list. It may grant immediate access, deny access, may generate a request for approval from topic managers.

Server replies to the `{sub}` with a `{ctrl}`.

The `{sub}` message may include a `get` and `set` fields which mirror `{get}` and `{set}` messages. If included, server will treat them as a subsequent `{set}` and `{get}` messages on the same topic. If the `get` is set, the reply may include `{meta}` and `{data}` messages.


```
/* Detailed source-code truncated for AI context efficiency. */
```

See [Trusted, Public, and Private Fields](#trusted-public-and-private-fields) for `trusted`, `private`, and `public` format considerations.

#### `{leave}`

This is a counterpart to `{sub}` message. It also serves two functions:
* leaving the topic without unsubscribing (`unsub=false`)
* unsubscribing (`unsub=true`)

Server responds to `{leave}` with a `{ctrl}` packet. Leaving without unsubscribing affects just the current session. Leaving with unsubscribing will affect all user's sessions.

```js
leave: {
  id: "1a2b3",  // string, client-provided message id, optional
  topic: "grp1XUtEhjv6HND",   // string, topic to leave, unsubscribe, or
                              // delete, required
  unsub: true // boolean, leave and unsubscribe, optional, default: false
}
```

#### `{pub}`

The message is used to distribute content to topic subscribers.

```js
pub: {
  id: "1a2b3", // string, client-provided message id, optional
  topic: "grp1XUtEhjv6HND", // string, topic to publish to, required
  noecho: false, // boolean, suppress echo (see below), optional
  head: { key: "value", ... }, // set of string key-value pairs, optional
  content: { ... }  // object, application-defined content to publish
               // to topic subscribers, required
}
```

Topic subscribers receive the `content` in the [`{data}`](#data) message. By default the originating session gets a copy of `{data}` like any other session currently attached to the topic. If for some reason the originating session does not want to receive the copy of the data it just published, set `noecho` to `true`.

See [Format of Content](#format-of-content) for `content` format considerations.

The following values are currently defined for the `head` field:

 * `attachments`: an array of paths indicating media attached to this message `["/v0/file/s/sJOD_tZDPz0.jpg"]`.
 * `auto`: `true` when the message was sent automatically, i.e. by a chatbot or an auto-responder.
 * `forwarded`: an indicator that the message is a forwarded message, a unique ID of the original message, `"grp1XUtEhjv6HND:123"`.
 * `mentions`: an array of user IDs mentioned (`@alice`) in the message: `["usr1XUtEhjv6HND", "usr2il9suCbuko"]`.
 * `mime`: MIME-type of the message content, `"text/x-drafty"`; a `null` or a missing value is interpreted as `"text/plain"`.
 * `replace`: an indicator that the message is a correction/replacement for another message, a topic-unique ID of the message being updated/replaced, `":123"`
 * `reply`: an indicator that the message is a reply to another message, a unique ID of the original message, `"grp1XUtEhjv6HND:123"`.
 * `sender`: a user ID of the sender added by the server when the message is sent on behalf of another user, `"usr1XUtEhjv6HND"`.
 * `thread`: an indicator that the message is a part of a conversation thread, a topic-unique ID of the first message in the thread, `":123"`; `thread` is intended for tagging a flat list of messages as opposite to creating a tree.
 * `webrtc`: a string representing the state of the video call the message represents. Possible values:
   * `"started"`: call has been initiated and being established
   * `"accepted"`: call has been accepted and established
   * `"finished"`: previously successfully established call has been ended
   * `"missed"`: call timed out before getting established
   * `"declined"`: call was hung up by the callee before getting established
   * `"busy"`: the call was declined due to the callee being in another call.
   * `"disconnected"`: call was terminated by the server for other reasons (e.g. due to an error)
 * `webrtc-duration`: a number representing a video call duration (in milliseconds).

Application-specific fields should start with an `x-<application-name>-`. Although the server does not enforce this rule yet, it may start doing so in the future.

The unique message ID should be formed as `<topic_name>:<seqId>` whenever possible, such as `"grp1XUtEhjv6HND:123"`. If the topic is omitted, i.e. `":123"`, it's assumed to be the current topic.

#### `{get}`

Query topic for metadata, such as description or a list of subscribers, or query message history. The requester must be [subscribed and attached](#sub) to the topic to receive the full response. Some limited `desc` and `sub` information is available without being attached.

```
/* Detailed source-code truncated for AI context efficiency. */
```

* `{get what="desc"}`

Query topic description. Server responds with a `{meta}` message containing requested data. See `{meta}` for details.
If `ims` is specified and data has not been updated, the message will skip `trusted`, `public`, and `private` fields.

Limited information is available without [attaching](#sub) to topic first.

See [Trusted, Public, and Private Fields](#trusted-public-and-private-fields) for `trusted`, `private`, and `public` format considerations.

* `{get what="sub"}`

Get a list of subscribers. Server responds with a `{meta}` message containing a list of subscribers. See `{meta}` for details.
For `me` topic the request returns a list of user's subscriptions. If `ims` is specified and data has not been updated,
responds with a `{ctrl}` "not modified" message.

Only user's own subscription is returned without [attaching](#sub) to topic first.

* `{get what="tags"}`

Query indexed tags. Server responds with a `{meta}` message containing an array of string tags. See `{meta}` and `fnd` topic for details.
Supported only for `me` and group topics.

* `{get what="data"}`

Query message history. Server sends `{data}` messages matching parameters provided in the `data` field of the query.
The `id` field of the data messages is not provided as it's common for data messages. When all `{data}` messages are transmitted, a `{ctrl}` message is sent.

* `{get what="del"}`

Query message deletion history. Server responds with a `{meta}` message containing a list of deleted message ranges.

* `{get what="cred"}`

Query [credentials](#credentail-validation). Server responds with a `{meta}` message containing an array of credentials. Supported for `me` topic only.

* `{get what="aux"}`

Query auxiliary topic data. Server responds with a `{meta}` message containing an object with auxiliary key-value pairs.


#### `{set}`

Update topic metadata, delete messages or topic. The requester is generally expected to be [subscribed and attached](#sub) to the topic. Only `desc.private` and requester's `sub.mode` can be updated without attaching first.

```
/* Detailed source-code truncated for AI context efficiency. */
```

#### `{del}`

Delete messages, subscriptions, topics, users.

```js
del: {
  id: "1a2b3", // string, client-provided message id, optional
  topic: "grp1XUtEhjv6HND", // string, topic affected, required for "topic", "sub",
               // "msg"
  what: "msg", // string, one of "topic", "sub", "msg", "user", "cred"; what
               // to delete - the entire topic, a subscription, some or all messages,
               // a user, a credential; optional, default: "msg"
  hard: false, // boolean, request to hard-delete vs mark as deleted; in case of
               // what="msg" delete for all users vs current user only;
               // optional, default: false
  delseq: [{low: 123, hi: 125}, {low: 156}], // array of ranges of message IDs
               // to delete, inclusive-exclusive, i.e. [low, hi), optional
  user: "usr2il9suCbuko" // string, user being deleted (what="user") or whose
               // subscription is being deleted (what="sub"), optional
  cred: { // credential to delete ('me' topic only).
    meth: "email", // string, verification method, e.g. "email", "tel", etc.
    val: "alice@example.com" // string, credential being deleted
  }
}
```

`what="msg"`

User can soft-delete `hard=false` (default) or hard-delete `hard=true` messages. Soft-deleting messages hides them from the requesting user but does not delete them from storage. An `R` permission is required to soft-delete messages. Hard-deleting messages deletes message content from storage (`head`, `content`) leaving a message stub. It affects all users. A `D` permission is needed to hard-delete messages. Messages can be deleted in bulk by specifying one or more message ID ranges in `delseq` parameter. Each delete operation is assigned a unique `delete ID`. The greatest `delete ID` is reported back in the `clear` of the `{meta}` message.

`what="sub"`

Deleting a subscription removes specified user from topic subscribers. It requires an `A` permission. A user cannot delete own subscription. A `{leave}` should be used instead. If the subscription is soft-deleted (default), it's marked as deleted without actually deleting a record from storage.

`what="topic"`

Deleting a topic deletes the topic including all subscriptions, and all messages. Only the owner can delete a topic.

`what="user"`

Deleting a user is a very heavy operation. Use caution.

`what="cred"`

Delete credential. Validated credentials and those with no attempts at validation are hard-deleted. Credentials with failed attempts at validation are soft-deleted which prevents their reuse by the same user.


#### `{note}`

Client-generated ephemeral notification for forwarding to other clients currently attached to the topic, such as typing notifications or delivery receipts. The message is "fire and forget": not stored to disk per se and not acknowledged by the server. Messages deemed invalid are silently dropped.
The `{note.recv}` and `{note.read}` do alter persistent state on the server. The value is stored and reported back in the corresponding fields of the `{meta.sub}` message.

```js
note: {
  topic: "grp1XUtEhjv6HND", // string, topic to notify, required
  what: "kp", // string, action type of the notification.
  seq: 123,   // integer, ID of the message being acknowledged, required for
              // 'recv' & 'read'.
  unread: 10, // integer, client-reported total count of unread messages, optional.
  event: "ringing", // string, subaction; surrently used only by video/audio calls,
                    // when what="call".
  payload: {  // object, required payload for 'call' and 'data'.
    ...
  }
}
```

The following actions types are currently defined:
 * call: a video call status update.
 * data: a generic packet of structured data, usually a form response.
 * kp: key press, i.e. a typing notification. The client should use it to indicate that the user is composing a new message.
 * kpa: audio message is in the process of recording.
 * kpv: video message is in the process of recording.
 * read: a `{data}` message is seen (read) by the user. It implies `recv` as well.
 * recv: a `{data}` message is received by the client software but may not yet seen by user.

The `read` and `recv` notifications may optionally include `unread` value which is the total count of unread messages as determined by this client. The per-user `unread` count is maintained by the server: it's incremented when new `{data}` messages are sent to user and reset to the values reported by the `{note unread=...}` message. The `unread` value is never decremented by the server. The value is included in push notifications to be shown on a badge on iOS:
<p align="center">
  <img src="./ios-pill-128.png" alt="Tinode iOS icon with a pill counter" width=64 height=64 />
</p>


### Server to Client Messages

Messages to a session generated in response to a specific request contain an `id` field equal to the id of the
originating message. The `id` is not interpreted by the server.

Most server to client messages have a `ts` field which is a timestamp when the message was generated by the server.

#### `{data}`

Content published in the topic. These messages are the only messages persisted in database; `{data}` messages are
broadcast to all topic subscribers with an `R` permission.

```js
data: {
  topic: "grp1XUtEhjv6HND", // string, topic which distributed this message,
                            // always present
  from: "usr2il9suCbuko", // string, id of the user who published the
                          // message; could be missing if the message was
                          // generated by the server
  head: { key: "value", ... }, // set of string key-value pairs, passed
                               // unchanged from {pub}, optional
  ts: "2015-10-06T18:07:30.038Z", // string, timestamp
  seq: 123, // integer, server-issued sequential ID
  content: { ... } // object, application-defined content exactly as published
              // by the user in the {pub} message
}
```

Data messages have a `seq` field which holds a sequential numeric ID generated by the server. The IDs are guaranteed to be unique within a topic. IDs start from 1 and sequentially increment with every successful [`{pub}`](#pub) message received by the topic.

See [Format of Content](#format-of-content) for `content` format considerations.

See [`{pub}`](#pub) message for the possible values of the `head` field.


#### `{ctrl}`

Generic response indicating an error or a success condition. The message is sent to the originating session.

```js
ctrl: {
  id: "1a2b3", // string, client-provided message id, optional
  topic: "grp1XUtEhjv6HND", // string, topic name, if this is a response in context
                            // of a topic, optional
  code: 200, // integer, code indicating success or failure of the request, follows
             // the HTTP status codes model, always present
  text: "OK", // string, text with more details about the result, always present
  params: { ... }, // object, generic response parameters, context-dependent,
                   // optional
  ts: "2015-10-06T18:07:30.038Z", // string, timestamp
}
```

#### `{meta}`

Information about topic metadata or subscribers, sent in response to `{get}`, `{set}` or `{sub}` message to the originating session.

```
/* Detailed source-code truncated for AI context efficiency. */
```

#### `{pres}`

Tinode uses `{pres}` message to inform clients of important events. A separate [document](https://docs.google.com/spreadsheets/d/e/2PACX-1vStUDHb7DPrD8tF5eANLu4YIjRkqta8KOhLvcj2precsjqR40eDHvJnnuuS3bw-NcWsP1QKc7GSTYuX/pubhtml?gid=1959642482&single=true) explains all possible use cases.

```js
pres: {
  topic: "me", // string, topic which receives the notification, always present
  src: "grp1XUtEhjv6HND", // string, topic or user affected by the change, always present
  what: "on", // string, action type, what's changed, always present
  seq: 123, // integer, "what" is "msg", a server-issued ID of the message,
            // optional
  clear: 15, // integer, "what" is "del", an update to the delete transaction ID.
  delseq: [{low: 123}, {low: 126, hi: 136}], // array of ranges, "what" is "del",
             // ranges of IDs of deleted messages, optional
  ua: "Tinode/1.0 (Android 2.2)", // string, a User Agent string identifying the client
             // software if "what" is "on" or "ua", optional
  act: "usr2il9suCbuko",  // string, user who performed the action, optional
  tgt: "usrRkDVe0PYDOo",  // string, user affected by the action, optional
  acs: {want: "+AS-D", given: "+S"} // object, changes to access mode, "what" is "acs",
                          // optional
}
```

The following action types are currently defined:

 * on: topic or user came online
 * off: topic or user went offline
 * ua: user agent changed, for example user was logged in with one client, then logged in with another
 * upd: topic description has changed
 * tags: topic tags have changed
 * aux: topic aux data has changed
 * acs: access permissions have changed
 * gone: topic is no longer available, for example, it was deleted or you were unsubscribed from it
 * term: subscription to topic has been terminated, you may try to resubscribe
 * msg: a new message is available
 * read: one or more messages have been read by the recipient
 * recv: one or more messages have been received by the recipient
 * del: messages were deleted


The `{pres}` messages are purely transient: they are not stored and no attempt is made to deliver them later if the destination is temporarily unavailable.

Timestamp is not present in `{pres}` messages.


#### `{info}`

Forwarded client-generated notification `{note}`. Server guarantees that the message complies with this specification and that content of `topic` and `from` fields is correct. The other content is copied from the `{note}` message verbatim and may potentially be incorrect or misleading if the originator  so desires.

```js
info: {
  topic: "grp1XUtEhjv6HND", // string, topic affected, always present
  src: "usrRkDVe0PYDOo",  // string, topic where the even has occurred;
                          // present only when "topic": "me"
  from: "usr2il9suCbuko", // string, id of the user who published the
                          // message, always present
  what: "read", // string, one of "kp", "recv", "read", "data", see client-side {note},
                // always present
  seq: 123, // integer, ID of the message that client has acknowledged,
            // guaranteed 0 < read <= recv <= {ctrl.params.seq}; present for recv &
            // read
  event: "ringing", // string, used by video/audio calls
  payload: { ... }  // object, arbitrary payload, used by video calls
}
```

---

### CLA (`docs/CLA.md`)

# Tinode Individual Contributor License Agreement

In order to clarify the intellectual property license granted with Contributions from any person or entity,
[Tinode LLC](https://tinode.co) must have a Contributor License Agreement ("CLA") on file that has been signed by each Contributor,
indicating agreement to the license terms below. This license is for your protection as a Contributor as
well as the protection of Tinode LLC; it does not change your rights to use your own Contributions for any
other purpose.

You accept and agree to the following terms and conditions for Your present and future Contributions
submitted to Tinode. Except for the license granted herein to Tinode LLC and recipients of software distributed
by Tinode LLC, You reserve all right, title, and interest in and to Your Contributions.

1. Definitions.
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with Tinode LLC. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to Tinode LLC for inclusion in, or documentation of, any of the products owned or managed by Tinode LLC (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to Tinode LLC or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, Tinode LLC for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution."

2. Grant of Copyright License.
Subject to the terms and conditions of this Agreement, You hereby grant to Tinode LLC and to recipients of software distributed by Tinode LLC a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.

3. Grant of Patent License.
Subject to the terms and conditions of this Agreement, You hereby grant to Tinode LLC and to recipients of software distributed by Tinode LLC a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.

4. You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to Tinode, or that your employer has executed a separate Corporate CLA with Tinode LLC.

5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.

6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.

7. Should You wish to submit work that is not Your original creation, You may submit it to Tinode separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]".

8. You agree to notify Tinode LLC of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect.

---

## [SIGN NOW](https://docs.google.com/forms/d/e/1FAIpQLSfmtJDHzFOJTzIv5jZ-gHRxVU0ysTdIMJakv1xgUUCu_RGeKQ/formResponse)

---

### Call Establishment (`docs/call-establishment.md`)

# Video Call Establishment Flow

Tinode supports peer to peer video calls over [WebRTC](https://webrtc.org/). The diagram below illustrates a call establishment flow between two users `Alice` and `Bob`. The flow is conceptually similar to [SIP](https://en.wikipedia.org/wiki/Session_Initiation_Protocol), but uses native Tinode messages for transport.

Notes:
- All communication is proxied by the Tinode server.
- Client-to-server events are dispatched in `{note}` messages with the call's `topic` and `seq` fields set.
- Server-to-client data are routed in `{info}` messages on `me` topic with the call's `src` (call topic) and `seq` fields set (and/or in data push notifications).
- It's assumed that both Alice and Bob may have multiple devices.

## Details
### Call phases
The flow may be broken down into 4 phases:
* Steps 1-5: call initiation
* Steps 6-7: call acceptance
* Steps 8-15: metadata exchange
* Steps 16-17: call termination

```
/* Detailed source-code truncated for AI context efficiency. */
```

### Call Establishment & Termination steps

#### Call initiation
1. `Alice` initiates a call by posting a video call message (with `webrtc=started` header)
2. Server replies with a `{ctrl}` message containing the `seq` id of the call.
3. Server routes an `invite` event message to `Bob` (all clients).
  - Additionally, server sends data push notifications containing a `webrtc=started` field to `Bob`.
  - Upon receiving either of the above, `Bob` displays the incoming call UI.
4. `Bob` replies with a `ringing` event.
5. Server relays the `ringing` event to `Alice`. The latter now plays the ringing sound.
  - Note that `Alice` may receive multiple `ringing` events as each separate instance of `Bob` acknowldges receipt of the call invitation separately.
  - `Alice` and server will wait for up to a server configured timeout for `Bob` to accept the call and then hang up.
  - At this point, the call is officially **initiated**.

#### Call acceptance
6. `Bob` accepts the call by sending an `accept` event.
7. (a) and (b): Server routes `accept` event to `Alice` and `Bob`.
  - Additionally, the server broadcasts a replacement for the call data message with `webrtc=accepted` header.
  - Push notifications for the replacement message are sent as well.
  - `Bob`'s sessions except the one that accepted the call may silently dismiss the incoming call UI.
  - At this point, the call is officially **accepted**.

#### Metadata exchange
8. `Alice` sends an `offer` event containing an SDP payload.
9. Server routes the `offer` to `Bob`.
10. Upon receiving the `offer`, `Bob` replies with an `answer` event containing an SDP payload.
11. Server forwards `Bob`'s `answer` event to `Alice`.

Steps 12-15 are Ice candidate exchange between `Alice` and `Bob`.
At this point the call is officially **established**. `Alice` and `Bob` can see and hear each other.

#### Call termination
16. `Alice` sends a `hang-up` event to server.
17. Server routes a `hang-up` event to `Bob`.

Additionally, the server broadcasts a replacement for the call data message with `webrtc=finished` header.
Push notifications for the replacement message are sent as well.

---

### Drafty (`docs/drafty.md`)

# Drafty: Rich Message Format

Drafty is a text format used by Tinode to style messages. The intent of Drafty is to be expressive just enough without opening too many possibilities for security issues. One may think of it as JSON-encapsulated [markdown](https://en.wikipedia.org/wiki/Markdown). Drafty is influenced by FB's [draft.js](https://draftjs.org/) specification. As of the time of this writing [Javascript](https://github.com/tinode/tinode-js/blob/master/src/drafty.js), [Java](https://github.com/tinode/tindroid/blob/master/tinodesdk/src/main/java/co/tinode/tinodesdk/model/Drafty.java) and [Swift](https://github.com/tinode/ios/blob/master/TinodeSDK/model/Drafty.swift) implementations exist. A [Go implementation](https://github.com/tinode/chat/blob/master/server/drafty/drafty.go) can convert Drafy to plain text and previews.

## Example

> this is **bold**, `code` and _italic_, ~~strike~~<br/>
>  combined **bold and _italic_**<br/>
>  an url: https://www.example.com/abc#fragment and another _[https://web.tinode.co](https://web.tinode.co)_<br/>
>  this is a [@mention](#) and a [#hashtag](#) in a string<br/>
> second [#hashtag](#)<br/>

Sample Drafty-JSON representation of the text above:
```js
{
   "txt":  "this is bold, code and italic, strike combined bold and italic an url: https://www.example.com/abc#fragment and another www.tinode.co this is a @mention and a #hashtag in a string second #hashtag",
   "fmt": [
       { "at":8, "len":4,"tp":"ST" },{ "at":14, "len":4, "tp":"CO" },{ "at":23, "len":6, "tp":"EM"},
       { "at":31, "len":6, "tp":"DL" },{ "tp":"BR", "len":1, "at":37 },{ "at":56, "len":6, "tp":"EM" },
       { "at":47, "len":15, "tp":"ST" },{ "tp":"BR", "len":1, "at":62 },{ "at":120, "len":13, "tp":"EM" },
       { "at":71, "len":36, "key":0 },{ "at":120, "len":13, "key":1 },{ "tp":"BR", "len":1, "at":133 },
       { "at":144, "len":8, "key":2 },{ "at":159, "len":8, "key":3 },{ "tp":"BR", "len":1, "at":179 },
       { "at":187, "len":8, "key":3 },{ "tp":"BR", "len":1, "at":195 }
   ],
   "ent": [
       { "tp":"LN", "data":{ "url":"https://www.example.com/abc#fragment" } },
       { "tp":"LN", "data":{ "url":"http://www.tinode.co" } },
       { "tp":"MN", "data":{ "val":"mention" } },
       { "tp":"HT", "data":{ "val":"hashtag" } }
   ]
}
```

## Structure

Drafty object has three fields: plain text `txt`, inline markup `fmt`, and entities `ent`.

### Plain Text

The message to be sent is converted to plain Unicode text with all markup stripped and kept in `txt` field. In general, a valid Drafy may contain just the `txt` field.

### Inline Formatting `fmt`

Inline formatting is an array of individual styles in the `fmt` field. Each style is represented by an object with at least `at` and `len` fields. The `at` value means 0-based offset into `txt`, `len` is the number of characters to apply formatting to. The third value of style is either `tp` or `key`.

If `tp` is provided, it means the style is a basic text decoration:
 * `BR`: line break.
 * `CO`: code or monotyped text, possibly with different background: `monotype`.
 * `DL`: deleted or strikethrough text: ~~strikethrough~~.
 * `EM`: emphasized text, usually represented as italic: _italic_.
 * `FM`: form / set of fields; may also be represented as an entity.
 * `HD`: hidden content.
 * `HL`: highlighted text, such as text in a different color or with a different background; the color cannot be specified.
 * `RW`: logical grouping of formats, a row; may also be represented as an entity.
 * `ST`: strong or bold text: **bold**.

If key is provided, it's a 0-based index into the `ent` array which contains extended style parameters such as an image or an URL:
 * `AU`: embedded audio.
 * `BN`: interactive button.
 * `EX`: generic attachment.
 * `FM`: form / set of fields; may also be represented as a basic decoration.
 * `HT`: hashtag, e.g. [#hashtag](#).
 * `IM`: inline image.
 * `LN`: link (URL) [https://api.tinode.co](https://api.tinode.co).
 * `MN`: mention such as [@tinode](#).
 * `RW`: logical grouping of formats, a row; may also be represented as a basic decoration.
 * `VC`: video (and audio) calls.
 * `VD`: inline video.

Examples:
 * `{ "at":8, "len":4, "tp":"ST"}`: apply formatting `ST` (strong/bold) to 4 characters starting at offset 8 into `txt`.
 * `{ "at":144, "len":8, "key":2 }`: insert entity `ent[2]` into position 144, the entity spans 8 characters.
 * `{ "at":-1, "len":0, "key":4 }`: show the `ent[4]` as a file attachment, don't apply any styling to text.

The clients should be able to handle missing `at`, `key`, and `len` values. Missing values are assumed to be equal to `0`.

The indexes `at` and `len` are measured in [Unicode code points](https://developer.mozilla.org/en-US/docs/Glossary/Code_point), not bytes or characters. The behavior of multi-codepoint glyphs such as emojis with Fitzpatrick skin tone modifiers, variation selectors, or grouped with `ZWJ` is currently undefined.

#### `FM`: a form, an ordered set or fields

Form provides means to add paragraph-level formatting to a logical group of elements. It may be represented as a text decoration or as an entity.
<table>
<tr><th>Do you agree?</th></tr>
<tr><td><a href="">Yes</a></td></tr>
<tr><td><a href="">No</a></td></tr>
</table>

```js
{
 "txt": "Do you agree? Yes No",
 "fmt": [
   {"len": 20, "tp": "FM"}, // missing 'at' is zero: "at": 0
   {"len": 13, "tp": "ST"}
   {"at": 13, "len": 1, "tp": "BR"},
   {"at": 14, "len": 3}, // missing 'key' is zero: "key": 0
   {"at": 17, "len": 1, "tp": "BR"},
   {"at": 18, "len": 2, "key": 1},
 ],
 "ent": [
   {"tp": "BN", "data": {"name": "yes", "act": "pub", "val": "oh yes!"}},
   {"tp": "BN", "data": {"name": "no", "act": "pub"}}
 ]
}
```
If a `Yes` button is pressed in the example above, the client is expected to send a message to the server with the following content:
```js
{
 "txt": "Yes",
 "fmt": [{
   "at":-1
 }],
 "ent": [{
   "tp": "EX",
   "data": {
     "mime": "text/x-drafty-fr", // drafty form-response.
     "val": {
       "seq": 15, // seq id of the message containing the form.
       "resp": {"yes": "oh yes!"}
     }
   }
 }]
}
```

The form may be optionally represented as an entity:
```js
{
  "tp": "FM",
  "data": {
    "su": true
  }
}
```
The `data.su` describes how interactive form elements behave after the click. An `"su": true` indicates that the form is `single use`: the form should change after the first interaction to show that it's no longer accepting input.

### Entities `ent`

In general, an entity is a text decoration which requires additional (possibly large) data. An entity is represented by an object with two fields: `tp` indicates type of the entity, `data` is type-dependent styling information. Unknown fields are ignored.

#### `AU`: embedded audio record
`AU` is an audio record. The `data` contains the following fields:
```js
{
  "tp": "AU",
  "data": {
    "mime": "audio/aac",
    "val": "Rt53jUU...iVBORw0KGgoA==",
    "ref": "/v0/file/s/e769gvt1ILE.m4v",
    "preview": "Aw4JKBkAAAAKMSM...vHxgcJhsgESAY"
    "duration": 180000,
    "name": "ding_dong.m4a",
    "size": 595496
  }
}
```
 * `mime`: data type, such as 'audio/ogg'.
 * `val`: optional in-band audio data: base64-encoded audio bits.
 * `ref`: optional reference to out-of-band audio data. Either `val` or `ref` must be present.
 * `preview`: base64-encoded array of bytes to generate a visual preview; each byte is an amplitude bar.
 * `duration`: duration of the record in milliseconds.
 * `name`: optional name of the original file.
 * `size`: optional size of the file in bytes.

To create a message with just a single audio record and no text, use the following Drafty:
```js
{
  txt: " ",
  fmt: [{len: 1}],
  ent: [{tp: "AU", data: {<your audio data here>}]}
}
```

_IMPORTANT Security Consideration_: the `val` and `ref` fields may contain malicious payload. The client should restrict URL scheme in the `ref` field to `http` and `https` only. The client should present content of `val` field to the user only if it's correctly converted to an audio.


#### `BN`: interactive button
`BN` offers an option to send data to a server, either origin server or another one. The `data` contains the following fields:
```js
{
  "tp": "BN",
  "data": {
    "name": "confirmation",
    "act": "url",
    "val": "some-value",
    "ref": "https://www.example.com/path/?foo=bar"
  }
}
```
* `act`: type of action in response to button click:
  * `pub`: send a Drafty-formatted `{pub}` message to the current topic with the form data as an attachment:
  ```js
  { "tp":"EX", "data":{ "mime":"text/x-drafty-fr", "val": { "seq": 3, "resp": { "confirmation": "some-value" } } } }
  ```
  * `url`: issue an `HTTP GET` request to the URL from the `data.ref` field. The following query parameters are appended to the URL: `<name>=<val>`, `uid=<current-user-ID>`, `topic=<topic name>`, `seq=<message sequence ID>`.
  * `note`: send a `{note}` message to the current topic with `what` set to `data` (not currently implemented, contact us if you need it).
  ```js
  { "what": "data", "data": { "mime": "text/x-drafty-fr", "val": { "seq": 3, "resp": { "confirmation": "some-value" } } }
  ```
* `name`: optional name of the button which is reported back to the server.
* `val`: additional opaque data.
* `ref`: the URL for the `url` action.

If the `name` is provided but `val` is not, `val` is assumed to be `1`. If `name` is undefined then nether `name` nor `val` are sent.

The button in the example above will send an HTTP GET to https://www.example.com/path/?foo=bar&confirmation=some-value&uid=usrFsk73jYRR&topic=grpnG99YhENiQU&seq=3 assuming the current user ID is `usrFsk73jYRR`, the topic is `grpnG99YhENiQU`, and the sequence ID of message with the button is `3`.

_IMPORTANT Security Consideration_: the client should restrict URL scheme in the `ref` field to `http` and `https` only.


#### `EX`: file attachment
`EX` is an attachment which the client should not try to interpret. The `data` contains the following fields:
```js
{
  "tp": "EX",
  "data": {
    "mime", "text/plain",
    "val", "Q3l0aG9uPT0w...PT00LjAuMAo=",
    "ref": "/v0/file/s/abcdef12345.txt",
    "name", "requirements.txt",
    "size": 1234
  }
}
```
* `mime`: data type, such as 'application/octet-stream'.
* `val`: optional in-band base64-encoded file data.
* `ref`: optional reference to out-of-band file data. Either `val` or `ref` must be present.
* `name`: optional name of the original file.
* `size`: optional size of the file in bytes.

To generate a message with the file attachment shown as a downloadable file, use the following format:
```js
{
  at: -1,
  len: 0,
  key: <EX entity reference>
}
```

_IMPORTANT Security Consideration_: the `ref` fields may contain malicious payload. The client should restrict URL scheme in the `ref` field to `http` and `https` only.


#### `IM`: inline image or attached image with inline preview
`IM` is an image. The `data` contains the following fields:
```js
{
  "tp": "IM",
  "data": {
    "mime": "image/png",
    "val": "Rt53jUU...iVBORw0KGgoA==",
    "ref": "/v0/file/s/abcdef12345.jpg",
    "width": 512,
    "height": 512,
    "name": "sample_image.png",
    "size": 123456
  }
}
```
 * `mime`: data type, such as 'image/jpeg'.
 * `val`: optional in-band image data: base64-encoded image bits.
 * `ref`: optional reference to out-of-band image data. Either `val` or `ref` must be present.
 * `width`, `height`: linear dimensions of the image in pixels.
 * `name`: optional name of the original file.
 * `size`: optional size of the file in bytes.

To create a message with just a single image and no text, use the following Drafty:
```js
{
  txt: " ",
  fmt: [{len: 1}],
  ent: [{tp: "IM", data: {<your image data here>}]}
}
```

_IMPORTANT Security Consideration_: the `val` and `ref` fields may contain malicious payload. The client should restrict URL scheme in the `ref` field to `http` and `https` only. The client should present content of `val` field to the user only if it's correctly converted to an image.


#### `LN`: link (URL)
`LN` is an URL. The `data` contains a single `url` field:
`{ "tp": "LN", "data": { "url": "https://www.example.com/abc#fragment" } }`
The `url` could be any valid URL that the client knows how to interpret, for instance it could be email or phone URL too: `email:alice@example.com` or `tel:+17025550001`.

_IMPORTANT Security Consideration_: the `url` field may be maliciously constructed. The client should disable certain URL schemes such as `javascript:` and `data:`.


#### `MN`: mention such as [@alice](#)
Mention `data` contains a single `val` field with ID of the mentioned user:
```js
{ "tp":"MN", "data":{ "val":"usrFsk73jYRR" } }
```


#### `HT`: hashtag, e.g. [#tinode](#)
Hashtag `data` contains a single `val` field with the hashtag value which the client software needs to interpret, for instance it could be a search term:
```js
{ "tp":"HT", "data":{ "val":"tinode" } }
```

#### `VC`: video call control message
Video call `data` contains current state of the call and its duration:
```js
{
  "tp": "VC",
  "data": {
    "duration": 10000,
    "state": "disconnected",
    "incoming": false,
    "aonly": true
  }
}
```

* `duration`: call duration in milliseconds.
* `state`: surrent call state; supported states:
	* `accepted`: a call is established (ongoing).
	* `busy`: a call cannot be established because the callee is already in another call.
	* `finished`: a previously establied call has successfully finished.
	* `disconnected`: the call is dropped, for example because of an error.
	* `missed`: the call is missed, i.e. the callee didn't pick up the phone.
	* `declined`: the call is declined, i.e. the callee hung up before picking up.
* `incoming`: true if the call is incoming, otherwise the call is outgoing.
* `aonly`: true if this is an audio-only call (no video).

The `VC` may also be represented as a format `"fmt": [{"len": 1, "tp": "VC"}]` with no entity. In such a case all call information is contained in the `head` fields of the enclosing message.

#### `VD`: video with preview
`VD` represents a video recording. The `data` contains the following fields:
```js
{
  "tp": "VD",
  "data": {
    "mime": "video/webm",
    "ref": "/v0/file/s/abcdef12345.webm",
    "preview": "AsTrsU...k86n00Ggo=="
    "preref": "/v0/file/s/abcdef54321.jpeg",
    "premime": "image/jpeg",
    "width": 640,
    "height": 360,
    "duration": 32000,
    "name": " bigbuckbunny.webm",
    "size": 1234567
  }
}
```
 * `mime`: data type of the video, such as 'video/webm'.
 * `val`: optional in-band video data: base64-encoded video bits, usually not present (null).
 * `ref`: optional reference to an out-of-band video data. Either `val` or `ref` must be present.
 * `preview`: optional base64-encoded screencapture image from the video (poster).
 * `preref`: optional reference to an out-of-band screencapture image from the video (poster).
 * `premime`: data type of the optional screencapture image (poster); assumed 'image/jpeg' if missing.
 * `width`, `height`: linear dimensions of the video and poster in pixels.
 * `duration`: duration of the video in milliseconds.
 * `name`: optional name of the original file.
 * `size`: optional size of the file in bytes.

To create a message with just a single video and no text, use the following Drafty:
```js
{
  txt: " ",
  fmt: [{len: 1}],
  ent: [{tp: "VD", data: {<your video data here>}]}
}
```

_IMPORTANT Security Consideration_: the `val`, `ref`, `preview` fields may contain malicious payload. The client should restrict URL scheme in the `ref` and `preview` fields to `http` and `https` only. The client should present content of `val` field to the user only if it's correctly converted to a video.

---

### Faq (`docs/faq.md`)

# Frequently Asked Questions

### Q: Where can I find server logs when running in Docker?<br/>
**A**: The log is in the container at `/var/log/tinode.log`. Attach to a running container with command
```
docker exec -it name-of-the-running-container /bin/bash
```
Then, for instance, see the log with `tail -50 /var/log/tinode.log`

If the container has stopped already, you can copy the log out of the container (saving it to `./tinode.log`):
```
docker cp name-of-the-container:/var/log/tinode.log ./tinode.log
```

Alternatively, you can instruct the docker container to save the logs to a directory on the host by mapping a host directory to `/var/log/` in the container. Add `-v /where/to/save/logs:/var/log` to the `docker run` command.


### Q: What are the options for enabling push notifications?<br/>
**A**: You can use [Tinode Push Gateway (TNPG)](https://github.com/tinode/chat/tree/master/server/push/tnpg) or you can use [Google FCM](https://firebase.google.com/docs/cloud-messaging):
 * _Tinode Push Gateway_ uses Tinode servers to send pushes on your behalf. It requires minumum setup: your server sends request to TNPG, which forwards it to Google FCM or Apple APNS.
 * _Google FCM_ does not rely on Tinode infrastructure for pushes but requires you to build and release your own mobile apps (iOS and Android).


### Q: How to setup push notifications with Tinode Push Gateway?<br/>
**A**: Enabling TNPG push notifications requires two steps:
 * register at [console.tinode.co](https://console.tinode.co) and obtain a TNPG token.
 * configure server with the token.
See detailed instructions [here](../server/push/tnpg/).


### Q: How to setup push notifications with Google FCM?<br/>
**A**: This option requires you to build and release your own mobile apps. If you do not want to do it, use the TNPG option above.

Enabling FCM push notifications requires the following steps:
 * enable push sending from the server.
 * enable receiving pushes in the clients.

#### Server and TinodeWeb

1. Create a project at https://firebase.google.com/ if you have not done so already.
2. Follow instructions at https://cloud.google.com/iam/docs/creating-managing-service-account-keys to download the credentials file.
3. Update the server config [`tinode.conf`](../server/tinode.conf#L255), section `"push"` -> `"name": "fcm"`. Do _ONE_ of the following:
  * _Either_ enter the path to the downloaded credentials file into `"credentials_file"`.
  * _OR_ copy the file contents to `"credentials"`.<br/><br/>
    Remove the other entry. I.e. if you have updated `"credentials_file"`, remove `"credentials"` and vice versa.
4. Update [TinodeWeb](/tinode/webapp/) config [`firebase-init.js`](https://github.com/tinode/webapp/blob/master/firebase-init.js): update `apiKey`, `messagingSenderId`, `projectId`, `appId`, `messagingVapidKey`. See more info at https://github.com/tinode/webapp/#push_notifications

#### iOS and Android
1. If you are using an Android client, add `google-services.json` to [Tindroid](/tinode/tindroid/) by following instructions at https://developers.google.com/android/guides/google-services-plugin and recompile the client. You may also optionally submit it to Google Play Store.
See more info at https://github.com/tinode/tindroid/#push_notifications
2. If you are using an iOS client, add `GoogleService-Info.plist` to [Tinodios](/tinode/ios/) by following instructions at https://firebase.google.com/docs/cloud-messaging/ios/client) and recompile the client. You may optionally submit the app to Apple AppStore.
See more info at https://github.com/tinode/ios/#push_notifications


### Q: How to add new users?<br/>
**A**: There are three ways to create accounts:
* A user can create a new account using one of the applications (web, Android, iOS).
* A new account can be created using [tn-cli](../tn-cli/) (`acc` command or `useradd` macro). The process can be scripted.
* If the user already exists in an external database, the Tinode account can be automatically created on the first login using the [rest authenticator](../server/auth/rest/).


### Q: How do I make my installation private?<br/>
**A**: If you want to restrict registrations to only those people whom you approve, then the simplest way is to restrict Tinode registrations to an email domain you control: register a custom domain, set up a catch-all email forwarding service at your domain registrar (usually free). Then use your domain name in Tinode config (`"acc_validation" -> "email" -> "domains"`, for example `"domains": ["my-domain.com"]`). You will receive registration emails at your catch-all email box and you will be able to forward validation codes to your users manually. Alternatively, if you have a lot of users, you can use [rest authenticator](../server/auth/rest/).


### Q: How to create a `root` user?<br/>
**A**: Starting with Tinode version 0.18 the `root` access can be granted to a user by running the following command:
```sh
./tinode-db -auth=ROOT -uid=usrAbcDef123 -scheme=basic
```
Starting with 0.21 you can use a simpler command:
```sh
./tinode-db -make_root=usrAbcDef123
```
Where `usrAbcDef123` is the ID of the user to update.

In version 0.17 and older the `root` access can be granted to a user only by executing a database query.
First create or choose the user you want to promote to `root` then execute the query:
* RethinkDB:
```js
r.db('tinode').table('auth').get('basic:login-of-the-user-to-make-root').update({authLvl: 30})
```
* MySQL, PostgreSQL:
```sql
USE 'tinode';
UPDATE auth SET authlvl=30 WHERE uname='basic:login-of-the-user-to-make-root';
```
* MongoDB:
```js
db.getCollection('auth').updateOne({_id: 'basic:login-of-the-user-to-make-root'}, {$set: {authlvl: 30}})
```
The test database has a stock user `xena` which has root access.


### Q: Once the number of network connections reaches about 1000 per node, all kinds of problems start. Is this a bug?<br/>
**A**: It is likely not a bug. To ensure good server performance Linux limits the total number of open file descriptors (live network connections, open files) for each process at the kernel level. The default limit is usually 1024. There are other possible restrictions on the number of file descriptors. The problems you are experiencing are likely caused by exceeding one of the Linux-imposed limits. Please seek assistance of a system administrator.


### Q: What is the difference between a group topic and a channel?<br/>
**A**: Channel is a special case of a group topic. Normal group topics allow limited number of subscribers (128 by default). Each subscriber can be managed individually: invited, removed, banned, promoted to administrator or owner, other access permissions can be personally adjusted. Group topics with enabled channel functionality additionally permit an unlimited number of `readers`. The readers have read-only access to the topic, they cannot be managed individually, cannot be invited or removed, they cannot post messages. Readers do not generate presence notifications when joining or un-joining the topic and do not receive presence notifications from normal group members. Readers receive channel messages with `From` field set to `null`, i.e. they do not know who personally posted any given message to the channel. Readers cannot delete channel messages.


### Q: What is the proper way to format gRPC {pub content}?<br/>
**A**: The gPRC sends `content` field of a `{pub}` message as a byte array while the client applications expect it to be valid JSON. Consequently, you have to format the field to be valid JSON before passing it to gRPC. For example, to send a plain text `Hello world` message you have to send a quoted string `"Hello world"`. In most cases the string you pass to the gRPC call would look like `"\"Hello world\""` or `'"Hello world"'`.


### Q: How to fix PostgreSQL initialization failing with 'missing database' error?<br/>
**A**: PostgreSQL has a (mis)feature: a DB connection must always select a database. If the connection tries to use a database which does not exist (even with intent to create it), the connection fails. When Tinode is started for the first time, it tries to create a database, usually `tinode` (see `tinode.conf`, `"store_config": {"adapters": {"postgres": {"DBName": "tinode"}}}`. The database `tinode` obviously does not exist, so Tinode connection falls back to 'default' database which has the same name as the name of the connecting PostgreSQL user. The default configuration specifies user as `postgres` (`"User": "postgres"`), the database `postgres` always exists, so the connection succeeds and everything works as expected. But if you change the user to anything other than `postgres`, let's say `tinodeadmin`, then trouble starts: the database with the name `tinodeadmin` does not exist and the connection fails. If you want to change the user name to anything other than `postgres`, then you must create either a database `tinode` (or whatever you named your Tinode database) or an empty database with the same name as your user `tinodeadmin`. For example:
```
$ psql
	postgres=# create database tinode;
	exit
```

---

### Monitoring (`docs/monitoring.md`)

# Monitoring Tinode server

Tinode server can optionally expose runtime statistics as a json document at a configurable HTTP(S) endpoint. The feature is enabled by adding a string parameter `expvar` to the config file. The value of the `expvar` is the URL path pointing to the location where the variables are served. In addition to the config file, the feature can be enabled from the command line by adding an `--expvar` parameter. The feature is disabled if the value of `expvar` is an empty string `""` or a dash `"-"`. A non-blank value of the command line parameter overrides the config file value.

The feature is enabled in the default config file to publish stats at `/debug/vars`.

As of the time of this writing the following stats are published:

* `memstats`: Go's memory statistics as described at https://golang.org/pkg/runtime/#MemStats
* `cmdline`: server's command line parameters as an array of strings.
* `TotalSessions`: the count of all sessions which were created during server's life time.
* `LiveSessions`: the number of sessions currently live, regardless of authentication status.
* `TotalTopics`: the count of all topics activated during servers's life time.
* `LiveTopics`: the number of currently active topics.

---

### Thecard (`docs/thecard.md`)

# theCard: Person/Topic Description Format

Tinode uses `theCard` to store and transmit descriptions of people and topics. The format is conceptually similar to [vCard](https://www.rfc-editor.org/rfc/rfc6350.txt) 3.0.

When `JSON` is used to represent `theCard` data, it does it differently than [jCard](https://tools.ietf.org/html/rfc7095). `theCard` and `jCard` are incompatible. The main difference is that `theCard` uses objects to represent logically related data while `jCard` uses ordered arrays.

`theCard` is structured as an object:

```
/* Detailed source-code truncated for AI context efficiency. */
```

All fields are optional. Tinode clients currently use only `fn`, `photo`, `org`, `note`, `comm` fields. If other fields are needed in the future, then they will be adopted from the correspondent [vCard](https://www.rfc-editor.org/rfc/rfc6350.txt) fields.

---

### Translations (`docs/translations.md`)

# Localizing Tinode

**IMPORTANT!** Please use `devel` branches for translations.

## Server

The server sends emails or SMS to users upon creation of a new account and when the user requests to reset the password:

* [/server/templ/email-validation-en.templ](../server/templ/email-validation-en.templ)
* [/server/templ/email-password-reset-en.templ](../server/templ/email-password-reset-en.templ)
* [/server/templ/sms-validation-en.templ](../server/templ/sms-validation-en.templ)

Create a copy of the files naming them `email-password-reset-XX.teml`, `email-validation-XX.templ`, `sms-validation-XX.templ` where `XX` is the [ISO-631-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) code of the new language. Translate the content and send a pull request with the new files. If you don't know how to create a pull request then just sent the translated files in any way you can.


## Webapp

The translations are located in two places: [/src/i18n/](https://github.com/tinode/webapp/tree/devel/src/i18n/) and [/service-worker.js](https://github.com/tinode/webapp/blob/devel/service-worker.js#L11).

In order to add a translation, copy `/src/i18n/en.json` to a file named `/src/i18n/XX.json` where `XX` is the [BCP-47](https://tools.ietf.org/rfc/bcp/bcp47.txt) code of the new language. If in doubt how to choose the BCP-47 language code, use a two letter [ISO-631-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes). Only the `"translation":` line has to be translated, the `"defaultMessage"`, `"description"` etc should NOT be translated, they serve as help only, i.e.:

```js
"action_block_contact": {
  "translation": "Bloquear contacto", // <<<---- Only this string needs to be translated
  "defaultMessage": "Block Contact",  // This is the default message in English
  "description": "Flat button [Block Contact]", // This is an explanation where/how the string is used.
  "missing": false,
  "obsolete": false
},
```

When translating the `service-worker.js`, just add the strings directly to the file. Only two strings need to be translated "New message" and "New chat":

```js
const i18n = {
  ...
  'XX': {
    'new_message': "New message",
    'new_chat': "New chat",
  },
  ...
```

Please send a pull request with the new files. If you don't know how to create a pull request just sent the files in any way you can.


## Android

A single file needs to be translated: [/tinode/tindroid/app/src/main/res/values/strings.xml](https://github.com/tinode/tindroid/blob/devel/app/src/main/res/values/strings.xml)

Create a new directory `values-XX` in [app/src/main/res](https://github.com/tinode/tindroid/tree/devel/app/src/main/res), where `XX` is a two-letter [ISO-631-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) code , optionally followed by a two letter [ISO 3166-1-alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) region code (preceded by lowercase r). For example `values-pt.xml` would contain Portuguese translations while `values-pt-rBR.xml` is for _Brazilian_ Portuguese translations.

Make a copy of the file with English strings, place it to the new directory. Translate all the strings not marked with `translatable="false"` (the strings with `translatable="false"` don't need to be included at all) then send a pull request with the new file. If you don't know how to create a pull request then just sent the file in any way you can.


## iOS

Unfortunately iOS localization process is very convoluted and generally requires `Xcode` which runs only on Mac. Unless you are familiar with the iOS development, please create a [feature request](https://github.com/tinode/ios/issues/new?assignees=&labels=&template=feature_request.md&title=) for the desired language and we will send you the file for translation.

If you feel brave enough, you can translate the [following .xliff file](https://github.com/tinode/ios/blob/devel/Localizations/en.xcloc/Localized%20Contents/en.xliff) and then send it to us in any way you can. Translate all the strings between the `<target>` tags:

```xml
<trans-unit id="Action failed: %@" xml:space="preserve"> 
  <source>Action failed: %@</source> 
  <target>Se ha producido un error al realizar la acción: %@</target> 
  <note>Toast notification</note> 
</trans-unit>
```

If you are familiar with Xcode and localization for iOS, the exported localizations are located at [/Localizations](https://github.com/tinode/ios/tree/devel/Localizations).

---

--- METRICS ---
- Files Extracted: 27
- Estimated Token Budget: ~48708 tokens
- Canonical Reference: https://codewiki.google/github.com/tinode/chat