redis Documentation

1. Project Overview & Quickstart (redis/redis) Copy Chapter

File: README.md Copy Chapter

This document serves as both a quick start guide to Redis and a detailed resource for building it from source.

• New to Redis? Start with What is Redis and Getting Started
• Ready to build from source? Jump to Build Redis from Source
• Want to contribute? See the Code contributions section
  and CONTRIBUTING.md
• Looking for detailed documentation? Navigate to redis.io/docs

Table of contents Copy Chapter

• What is Redis?
  • Key use cases
• Why choose Redis?
• What is Redis Open Source?
• Getting started
  • Redis starter projects
  • Using Redis with client libraries
  • Using Redis with redis-cli
  • Using Redis with Redis Insight
• Redis data types, processing engines, and capabilities
• Cloud hosted Redis
• Community
• Build Redis from source
  • Install dependencies and build
  • Building Redis - flags and general notes
  • Fixing build problems with dependencies or cached build options
  • Fixing problems building 32 bit binaries
  • Allocator
  • Monotonic clock
  • Verbose build
  • Running Redis with TLS
• Code contributions
• Redis Trademarks

What is Redis? Copy Chapter

For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.

Key use cases Copy Section

Redis excels in various applications, including:

• Caching: Supports multiple eviction policies, key expiration, and hash-field expiration.
• Distributed Session Store: Offers flexible session data modeling (string, JSON, hash).
• Data Structure Server: Provides low-level data structures (strings, lists, sets, hashes, sorted sets, JSON, etc.) with high-level semantics (counters, queues, leaderboards, rate limiters) and supports transactions & scripting.
• NoSQL Data Store: Key-value, document, and time series data storage.
• Search and Query Engine: Indexing for hash/JSON documents, supporting vector search, full-text search, geospatial queries, ranking, and aggregations via Redis Search.
• Event Store & Message Broker: Implements queues (lists), priority queues (sorted sets), event deduplication (sets), streams, and pub/sub with probabilistic stream processing capabilities.
• Vector Store for GenAI: Integrates with AI applications (e.g. LangGraph, mem0) for short-term memory, long-term memory, LLM response caching (semantic caching), and retrieval augmented generation (RAG).
• Real-Time Analytics: Powers personalization, recommendations, fraud detection, and risk assessment.

Why choose Redis? Copy Chapter

Redis is a popular choice for developers worldwide due to its combination of speed, flexibility, and rich feature set. Here's why people choose Redis for:

• Performance: Because Redis keeps data primarily in memory and uses efficient data structures, it achieves extremely low latency (often sub-millisecond) for both read and write operations. This makes it ideal for applications demanding real-time responsiveness.
• Flexibility: Redis isn't just a key-value store, it provides native support for a wide range of data structures and capabilities listed in What is Redis?
• Extensibility: Redis is not limited to the built-in data structures, it has a modules API that makes it possible to extend Redis functionality and rapidly implement new Redis commands
• Simplicity: Redis has a simple, text-based protocol and well-documented command set
• Ubiquity: Redis is battle tested in production workloads at a massive scale. There is a good chance you indirectly interact with Redis several times daily
• Versatility: Redis is the de facto standard for use cases such as:
  • Caching: quickly access frequently used data without needing to query your primary database
  • Session management: read and write user session data without hurting user experience or slowing down every API call
  • Querying, sorting, and analytics: perform deduplication, full text search, and secondary indexing on in-memory data as fast as possible
  • Messaging and interservice communication: job queues, message brokering, pub/sub, and streams for communicating between services
  • Vector operations: Long-term and short-term LLM memory, RAG content retrieval, semantic caching, semantic routing, and vector similarity search

In summary, Redis provides a powerful, fast, and flexible toolkit for solving a wide variety of data management challenges. If you want to know more, here is a list of starting points:

• Introduction to Redis data types
• The full list of Redis commands
• Redis for AI
• Redis documentation

What is Redis Open Source? Copy Chapter

Redis Community Edition (Redis CE) was renamed Redis Open Source with the v8.0 release.

Redis Ltd. also offers Redis Software, a self-managed software with additional compliance, reliability, and resiliency for enterprise scaling,
and Redis Cloud, a fully managed service integrated with Google Cloud, Azure, and AWS for production-ready apps.

Read more about the differences between Redis Open Source and Redis here.

Getting started Copy Chapter

If you want to get up and running with Redis quickly without needing to build from source, use one of the following methods:

• Redis Cloud
• Official Redis Docker images (Alpine/Debian)
  sh Copy

  docker run -d -p 6379:6379 redis:latest

• Redis binary distributions
  • Snap
  • Homebrew
  • RPM
  • Debian
• Redis quick start guides

If you prefer to build Redis from source - see instructions below.

Redis starter projects Copy Section

To get started as quickly as possible in your language of choice, use one of the following starter projects:

• Python (redis-py)
• C#/.NET (NRedisStack/StackExchange.Redis)
• Go (go-redis)
• JavaScript (node-redis)
• Java/Spring (Jedis)

Using Redis with client libraries Copy Section

To connect your application to Redis, you will need a client library. Redis has documented client libraries in most popular languages, with community-supported client libraries in additional languages.

• Python (redis-py)
• Python (RedisVL)
• C#/.NET (NRedisStack/StackExchange.Redis)
• JavaScript (node-redis)
• Java (Jedis)
• Java (Lettuce)
• Go (go-redis)
• PHP (Predis)
• C (hiredis)
• Full list of client libraries

Using Redis with redis-cli Copy Section

redis-cli is Redis' command line interface. It is available as part of all the binary distributions and when you build Redis from source.

You can start a redis-server instance, and then, in another terminal try the following:

cd src
./redis-cli

redis> ping
PONG
redis> set foo bar
OK
redis> get foo
"bar"
redis> incr mycounter
(integer) 1
redis> incr mycounter
(integer) 2
redis>

Using Redis with Redis Insight Copy Section

For a more visual and user-friendly experience, use Redis Insight - a tool that lets you explore data, design, develop, and optimize your applications while also serving as a platform for Redis education and onboarding. Redis Insight integrates Redis Copilot, a natural language AI assistant that improves the experience when working with data and commands.

• Redis Insight documentation
• Redis Insight GitHub repository

Redis data types, processing engines, and capabilities Copy Chapter

Redis provides a variety of data types, processing engines, and capabilities to support a wide range of use cases:

• String: Sequences of bytes, including text, serialized objects, and binary arrays used for caching, counters, and bitwise operations.
• JSON: Nested JSON documents that are indexed and searchable using JSONPath expressions and with Redis Search
• Array: Sparse, index-addressable collection of string values
• Hash: Field-value maps used to represent basic objects and store groupings of key-value pairs with support for hash field expiration (TTL)
• Redis Search: Use Redis as a document database, a vector database, a secondary index, and a search engine. Define indexes for hash and JSON documents and then use a rich query language for vector search, full-text search, geospatial queries, and aggregations.
• List: Linked lists of string values used as stacks, queues, and for queue management.
• Set: Unordered collection of unique strings used for tracking unique items, relations, and common set operations (intersections, unions, differences).
• Sorted set: Collection of unique strings ordered by an associated score used for leaderboards and rate limiters.
• Vector set (beta): Collection of vector embeddings used for semantic similarity search, semantic caching, semantic routing, and Retrieval Augmented Generation (RAG).
• Geospatial indexes: Coordinates used for finding nearby points within a given radius or bounding box.
• Bitmap: A set of bit-oriented operations defined on the string type used for efficient set representations and object permissions.
• Bitfield: Binary-encoded strings that let you set, increment, and get integer values of arbitrary bit length used for limited-range counters, numeric values, and multi-level object permissions such as role-based access control (RBAC)
• Hyperloglog: A probabilistic data structure for approximating the cardinality of a set used for analytics such as counting unique visits, form fills, etc.
• *Bloom filter: A probabilistic data structure to check if a given value is present in a set. Used for fraud detection, ad placement, and unique column (i.e. username/email/slug) checks.
• *Cuckoo filter: A probabilistic data structure for checking if a given value is present in a set while also allowing limited counting and deletions used in targeted advertising and coupon code validation.
• *t-digest: A probabilistic data structure used for estimating the percentile of a large dataset without having to store and order all the data points. Used for hardware/software monitoring, online gaming, network traffic monitoring, and predictive maintenance.
• *Top-k: A probabilistic data structure for finding the most frequent values in a data stream used for trend discovery.
• *Count-min sketch: A probabilistic data structure for estimating how many times a given value appears in a data stream used for sales volume calculations.
• Time series: Data points indexed in time order used for monitoring sensor data, asset
  tracking, and predictive analytics
• Pub/sub: A lightweight messaging capability. Publishers send messages to a channel, and subscribers receive messages from that channel.
• Stream: An append-only log with random access capabilities and complex consumption strategies such as consumer groups. Used for event sourcing, sensor monitoring, and notifications.
• Transaction: Allows the execution of a group of commands in a single step. A request sent by another client will never be served in the middle of the execution of a transaction. This guarantees that the commands are executed as a single isolated operation.
• Programmability: Upload and execute Lua scripts on the server. Scripts can employ programmatic control structures and use most of the commands while executing to access the database. Because scripts are executed on the server, reading and writing data from scripts is very efficient.

Cloud hosted Redis Copy Chapter

Fully-managed Redis with real-time performance at scale.

Redis Cloud

Community Copy Chapter

Redis Community Resources

Build Redis from source Copy Chapter

This section refers to building Redis from source. If you want to get up and running with Redis quickly without needing to build from source see the Getting started section.

These instructions apply to Redis 8.10 and above. For versions lower than 8.10, see the 8.8 build instructions.

Configuration files: the build steps below tell you to run
./src/redis-server redis.conf. Release tarballs bake the bundled modules'
loadmodule lines and per-module settings directly into redis.conf during
packaging, so an extracted release tarball is ready to run as-is. When
building from a git checkout instead, that module config lives in the
auto-generated redis-full.conf produced by make modules-update (and
regenerated by make sync-redis-conf) — run ./src/redis-server redis-full.conf there. Edit Redis-core settings in
redis.conf. See modules/MODULES.md
for the full config flow.

Install dependencies and build Copy Section

Building Redis with all data structures (JSON, time series, Bloom / cuckoo / count-min / top-k, t-digest, and the Query Engine) needs a build toolchain plus a few version-sensitive dependencies — GCC/Clang, LLVM 21, CMake 3.25–3.31.6, Rust 1.94, OpenSSL, Python 3, and assorted -dev libraries. Instead of a per-OS package list, the repo installs them for you with make bootstrap, which detects your OS and installs each bundled module's prerequisites.

CMake version range matters. The modules require 3.25 ≤ CMake ≤ 3.31.6 — CMake 4.x is not supported and the build will fail with it. On distros that ship CMake 4.x by default (e.g. Ubuntu 26.04), pin a supported version, e.g. pip3 install 'cmake==3.31.6'. Note make bootstrap only installs CMake when it's missing or too old; it won't downgrade a pre-installed 4.x, so remove/pin that yourself.

1. Get the source

Either works — the release tarball already bundles the module sources; a git checkout needs one extra step to fetch them:

# A) Release tarball (recommended for building/running a release).
#    Replace <version>, e.g. 8.10.0 — extracts into redis-<version>/:
wget -O redis-<version>.tar.gz https://github.com/redis/redis/releases/download/<version>/redis-full.tar.gz
tar xvf redis-<version>.tar.gz && cd redis-<version>

# B) git checkout — clone the bundled modules once:
git clone https://github.com/redis/redis.git && cd redis
make modules-update

2. Install the build dependencies

Pick whichever option fits your environment:

1. Build inside the Docker build environment — recommended.
   The repo ships docker/Dockerfile.noble (Ubuntu 24.04) with every prerequisite baked in, so you build inside the container and never touch your host toolchain:

   bash Copy

   docker build -f docker/Dockerfile.noble -t redis-build:noble .
   
   # Multi-arch (requires `docker buildx` configured):
   docker buildx build --platform linux/amd64,linux/arm64 \
       -f docker/Dockerfile.noble -t redis-build:noble .
   
   # Build with the working tree mounted:
   docker run --rm -it -v "$PWD":/workspace -w /workspace redis-build:noble \
       bash -lc 'make -j"$(nproc)" && make run'

2. Install everything on a fresh machine or container.
   On a clean environment (for example a throwaway ubuntu:24.04 container), let bootstrap install every prerequisite for Redis core and all cloned modules:

   sh Copy

   make bootstrap

   ⚠️ make bootstrap installs system packages and may override existing versions of shared tools (compiler, CMake, LLVM, …). Prefer option 1, or run it in a disposable container, if that matters on your machine.

3. See only what's missing.
   To inspect which prerequisites are absent before installing anything, print one deduped list across Redis core and all modules:

   sh Copy

   make bootstrap list

   Then install just the reported packages yourself. (Version-gated deps are shown as name (>= X); optional test/coverage deps are listed separately and don't fail the check.)

4. Get the exact install commands to copy-paste.
   To run exactly what make bootstrap would, but only for the missing dependencies, use dry-run — it prints the precise install command for each missing dependency and installs nothing:

   sh Copy

   make bootstrap dry-run

   The commands are printed per module, so a dependency shared by several
   modules appears once for each. Work through them iteratively:

   1. Copy-paste the commands for a module to install its dependencies.
   2. Re-run make bootstrap dry-run — the deps you just installed no longer
      show, so you now see only what's still missing for the remaining modules.
   3. Repeat until make bootstrap dry-run prints no install commands.

Manual, per-OS install (no Docker, and you'd rather not let make bootstrap touch your host): follow the per-OS dependency instructions in the 8.8 README, which still lists them explicitly — https://github.com/redis/redis/tree/8.8#readme.

3. Build and run

export BUILD_TLS=yes            # optional — TLS support (needs OpenSSL dev libs)
make -j "$(nproc)"

# Release tarball (module config is baked into redis.conf):
./src/redis-server redis.conf
# From a git checkout, use the auto-generated module config instead:
./src/redis-server redis-full.conf

make (same as make build / make all) builds whatever is cloned under modules/*/src alongside Redis core. To build just the core data structures — even with modules cloned — use make build redis.

Building Redis - flags and general notes Copy Section

Redis can be compiled and used on Linux, OSX, OpenBSD, NetBSD, FreeBSD.
We support big endian and little endian architectures, and both 32 bit and 64-bit systems.

It may compile on Solaris derived systems (for instance SmartOS) but our support for this platform is best effort and Redis is not guaranteed to work as well as on Linux, OSX, and *BSD.

To build Redis with all the data structures (including JSON, time series, Bloom filter, cuckoo filter, count-min sketch, top-k, and t-digest) and with Redis Query Engine, make sure first that all the prerequisites are installed (see Install dependencies and build above), then clone the bundled modules once and build:

make modules-update
make

make (same as make build / make all) always builds whatever's cloned under modules/*/src alongside Redis core — there's no separate flag to opt in. If nothing is cloned yet, you get a core-only build.

To build Redis with just the core data structures — even if modules are already cloned — use:

make build redis

To build with TLS support, you need OpenSSL development libraries (e.g. libssl-dev on Debian/Ubuntu) and the following flag in the make command:

make BUILD_TLS=yes

To build with systemd support, you need systemd development libraries (such as libsystemd-dev on Debian/Ubuntu or systemd-devel on CentOS), and the following flag:

make USE_SYSTEMD=yes

To append a suffix to Redis program names, add the following flag:

make PROG_SUFFIX="-alt"

You can build a 32 bit Redis binary using:

make 32bit

After building Redis, it is a good idea to test it using:

make test

If TLS is built, running the tests with TLS enabled (you will need tcl-tls installed):

./utils/gen-test-certs.sh
./runtest --tls

Redis supports compression of replication stream via zstd as of 8.10. To build with compression support you have to install zstd development libraries (e.g libzstd-dev on Debian/Ubuntu) and use the following flag when invoking the make command:

make BUILD_COMPRESSION=yes

Fixing build problems with dependencies or cached build options Copy Section

Redis has some dependencies which are included in the deps directory. make does not automatically rebuild dependencies even if something in the source code of dependencies changes.

When you update the source code with git pull or when code inside the dependencies tree is modified in any other way, make sure to use the following command in order to really clean everything and rebuild from scratch:

make distclean

This will clean: jemalloc, lua, hiredis, linenoise and other dependencies.

Also, if you force certain build options like 32bit target, no C compiler optimizations (for debugging purposes), and other similar build time options, those options are cached indefinitely until you issue a make distclean
command.

Fixing problems building 32 bit binaries Copy Section

If after building Redis with a 32 bit target you need to rebuild it
with a 64 bit target, or the other way around, you need to perform a make distclean in the root directory of the Redis distribution.

In case of build errors when trying to build a 32 bit binary of Redis, try the following steps:

• Install the package libc6-dev-i386 (also try g++-multilib).
• Try using the following command line instead of make 32bit:
  make CFLAGS="-m32 -march=native" LDFLAGS="-m32"

Allocator Copy Section

Selecting a non-default memory allocator when building Redis is done by setting the MALLOC environment variable. Redis is compiled and linked against libc malloc by default, except for jemalloc being the default on Linux systems. This default was picked because jemalloc has proven to have fewer fragmentation problems than libc malloc.

To force compiling against libc malloc, use:

make MALLOC=libc

To compile against jemalloc on Mac OS X systems, use:

make MALLOC=jemalloc

Monotonic clock Copy Section

By default, Redis will build using the POSIX clock_gettime function as the monotonic clock source. On most modern systems, the internal processor clock can be used to improve performance. Cautions can be found here: http://oliveryang.net/2015/09/pitfalls-of-TSC-usage/

On ARM aarch64 systems, the hardware clock is enabled by default because the ARM Generic Timer is architecturally guaranteed to be available and monotonic on all ARMv8-A processors (see the “The Generic Timer in AArch64 state” section of the Arm Architecture Reference Manual for Armv8-A).

To build with support for the processor's internal instruction clock on other architectures, use:

make CFLAGS="-DUSE_PROCESSOR_CLOCK"

Verbose build Copy Section

Redis will build with a user-friendly colorized output by default.
If you want to see a more verbose output, use the following:

make V=1

Running Redis with TLS Copy Section

Please consult the TLS.md file for more information on how to use Redis with TLS.

Running Redis with the Query Engine and optional proprietary Intel SVS-VAMANA optimisations Copy Section

License Disclaimer
If you are using Redis Open Source under AGPLv3 or SSPLv1, you cannot use it together with the Intel Optimizations (Leanvec and LVQ binaries). The reason is that the Intel SVS license is not compatible with those licenses.
The Leanvec and LVQ techniques are closed source and are only available for use with Redis Open Source when distributed under the RSALv2 license.
For more details, please refer to the information provided by Intel here.

By default, Redis with the Redis Query Engine supports SVS-VAMANA index with global 8-bit quantisation. To compile Redis with the Intel SVS-VAMANA optimisations, LeanVec and LVQ, use the following:

make BUILD_INTEL_SVS_OPT=yes

Alternatively, you can export the variable before running the build step for your platform:

export BUILD_INTEL_SVS_OPT=yes
make

Code contributions Copy Chapter

By contributing code to the Redis project in any form, including sending a pull request via GitHub, a code fragment or patch via private email or public discussion groups, you agree to release your code under the terms of the Redis Software Grant and Contributor License Agreement. Please see the CONTRIBUTING.md file in this source distribution for more information. For security bugs and vulnerabilities, please see SECURITY.md and the description of the ability of users to backport security patches under Redis Open Source 7.4+ under BSDv3. Open Source Redis releases are subject to the following licenses:

1. Version 7.2.x and prior releases are subject to BSDv3. These contributions to the original Redis core project are owned by their contributors and licensed under the 3BSDv3 license as referenced in the REDISCONTRIBUTIONS.txt file. Any copy of that license in this repository applies only to those contributions;

2. Versions 7.4.x to 7.8.x are subject to your choice of RSALv2 or SSPLv1; and

3. Version 8.0.x and subsequent releases are subject to the tri-license RSALv2/SSPLv1/AGPLv3 at your option as referenced in the LICENSE.txt file.

Redis Trademarks Copy Chapter

The purpose of a trademark is to identify the goods and services of a person or company without causing confusion. As the registered owner of its name and logo, Redis accepts certain limited uses of its trademarks, but it has requirements that must be followed as described in its Trademark Guidelines available at: https://redis.io/legal/trademark-policy/.

File: deps/hiredis/README.md Copy Chapter

This Readme reflects the latest changed in the master branch. See v1.0.0 for the Readme and documentation for the latest release (API/ABI history).

HIREDIS

Hiredis is a minimalistic C client library for the Redis database.

It is minimalistic because it just adds minimal support for the protocol, but
at the same time it uses a high level printf-alike API in order to make it
much higher level than otherwise suggested by its minimal code base and the
lack of explicit bindings for every Redis command.

Apart from supporting sending commands and receiving replies, it comes with
a reply parser that is decoupled from the I/O layer. It
is a stream parser designed for easy reusability, which can for instance be used
in higher level language bindings for efficient reply parsing.

Hiredis only supports the binary-safe Redis protocol, so you can use it with any
Redis version >= 1.2.0.

The library comes with multiple APIs. There is the
synchronous API, the asynchronous API and the reply parsing API.

Upgrading to `1.1.0` Copy Chapter

Almost all users will simply need to recompile their applications against the newer version of hiredis.

NOTE: Hiredis can now return nan in addition to -inf and inf in a REDIS_REPLY_DOUBLE.
Applications that deal with RESP3 doubles should make sure to account for this.

Upgrading to `1.0.2` Copy Chapter

NOTE: v1.0.1 erroneously bumped SONAME, which is why it is skipped here.

Version 1.0.2 is simply 1.0.0 with a fix for CVE-2021-32765. They are otherwise identical.

Upgrading to `1.0.0` Copy Chapter

Version 1.0.0 marks the first stable release of Hiredis.
It includes some minor breaking changes, mostly to make the exposed API more uniform and self-explanatory.
It also bundles the updated sds library, to sync up with upstream and Redis.
For code changes see the Changelog.

Note: As described below, a few member names have been changed but most applications should be able to upgrade with minor code changes and recompiling.

IMPORTANT: Breaking changes from `0.14.1` -> `1.0.0` Copy Chapter

• redisContext has two additional members (free_privdata, and privctx).
• redisOptions.timeout has been renamed to redisOptions.connect_timeout, and we've added redisOptions.command_timeout.
• redisReplyObjectFunctions.createArray now takes size_t instead of int for its length parameter.

IMPORTANT: Breaking changes when upgrading from 0.13.x -> 0.14.x Copy Chapter

Bulk and multi-bulk lengths less than -1 or greater than LLONG_MAX are now
protocol errors. This is consistent with the RESP specification. On 32-bit
platforms, the upper bound is lowered to SIZE_MAX.

Change redisReply.len to size_t, as it denotes the the size of a string

User code should compare this to size_t values as well. If it was used to
compare to other values, casting might be necessary or can be removed, if
casting was applied before.

Upgrading from `<0.9.0` Copy Chapter

Version 0.9.0 is a major overhaul of hiredis in every aspect. However, upgrading existing
code using hiredis should not be a big pain. The key thing to keep in mind when
upgrading is that hiredis >= 0.9.0 uses a redisContext* to keep state, in contrast to
the stateless 0.0.1 that only has a file descriptor to work with.

Synchronous API Copy Chapter

To consume the synchronous API, there are only a few function calls that need to be introduced:

redisContext *redisConnect(const char *ip, int port);
void *redisCommand(redisContext *c, const char *format, ...);
void freeReplyObject(void *reply);

Connecting Copy Section

The function redisConnect is used to create a so-called redisContext. The
context is where Hiredis holds state for a connection. The redisContext
struct has an integer err field that is non-zero when the connection is in
an error state. The field errstr will contain a string with a description of
the error. More information on errors can be found in the Errors section.
After trying to connect to Redis using redisConnect you should
check the err field to see if establishing the connection was successful:

redisContext *c = redisConnect("127.0.0.1", 6379);
if (c == NULL || c->err) {
    if (c) {
        printf("Error: %s\n", c->errstr);
        // handle error
    } else {
        printf("Can't allocate redis context\n");
    }
}

One can also use redisConnectWithOptions which takes a redisOptions argument
that can be configured with endpoint information as well as many different flags
to change how the redisContext will be configured.

redisOptions opt = {0};

/* One can set the endpoint with one of our helper macros */
if (tcp) {
    REDIS_OPTIONS_SET_TCP(&opt, "localhost", 6379);
} else {
    REDIS_OPTIONS_SET_UNIX(&opt, "/tmp/redis.sock");
}

/* And privdata can be specified with another helper */
REDIS_OPTIONS_SET_PRIVDATA(&opt, myPrivData, myPrivDataDtor);

/* Finally various options may be set via the `options` member, as described below */
opt->options |= REDIS_OPT_PREFER_IPV4;

If a connection is lost, int redisReconnect(redisContext *c) can be used to restore the connection using the same endpoint and options as the given context.

Configurable redisOptions flags Copy Section

There are several flags you may set in the redisOptions struct to change default behavior. You can specify the flags via the redisOptions->options member.

Note: A redisContext is not thread-safe.

Other configuration using socket options Copy Section

The following socket options are applied directly to the underlying socket.
The values are not stored in the redisContext, so they are not automatically applied when reconnecting using redisReconnect().
These functions return REDIS_OK on success.
On failure, REDIS_ERR is returned and the underlying connection is closed.

To configure these for an asyncronous context (see Asynchronous API below), use ac->c to get the redisContext out of an asyncRedisContext.

int redisEnableKeepAlive(redisContext *c);
int redisEnableKeepAliveWithInterval(redisContext *c, int interval);

Enables TCP keepalive by setting the following socket options (with some variations depending on OS):

• SO_KEEPALIVE;
• TCP_KEEPALIVE or TCP_KEEPIDLE, value configurable using the interval parameter, default 15 seconds;
• TCP_KEEPINTVL set to 1/3 of interval;
• TCP_KEEPCNT set to 3.

int redisSetTcpUserTimeout(redisContext *c, unsigned int timeout);

Set the TCP_USER_TIMEOUT Linux-specific socket option which is as described in the tcp man page:

When the value is greater than 0, it specifies the maximum amount of time in milliseconds that trans mitted data may remain unacknowledged before TCP will forcibly close the corresponding connection and return ETIMEDOUT to the application.
If the option value is specified as 0, TCP will use the system default.

Sending commands Copy Section

There are several ways to issue commands to Redis. The first that will be introduced is
redisCommand. This function takes a format similar to printf. In the simplest form,
it is used like this:

reply = redisCommand(context, "SET foo bar");

The specifier %s interpolates a string in the command, and uses strlen to
determine the length of the string:

reply = redisCommand(context, "SET foo %s", value);

When you need to pass binary safe strings in a command, the %b specifier can be
used. Together with a pointer to the string, it requires a size_t length argument
of the string:

reply = redisCommand(context, "SET foo %b", value, (size_t) valuelen);

Internally, Hiredis splits the command in different arguments and will
convert it to the protocol used to communicate with Redis.
One or more spaces separates arguments, so you can use the specifiers
anywhere in an argument:

reply = redisCommand(context, "SET key:%s %s", myid, value);

Using replies Copy Section

The return value of redisCommand holds a reply when the command was
successfully executed. When an error occurs, the return value is NULL and
the err field in the context will be set (see section on Errors).
Once an error is returned the context cannot be reused and you should set up
a new connection.

The standard replies that redisCommand are of the type redisReply. The
type field in the redisReply should be used to test what kind of reply
was received:

RESP2 Copy Section

• REDIS_REPLY_STATUS:

  • The command replied with a status reply. The status string can be accessed using reply->str.
    The length of this string can be accessed using reply->len.

• REDIS_REPLY_ERROR:

  • The command replied with an error. The error string can be accessed identical to REDIS_REPLY_STATUS.

• REDIS_REPLY_INTEGER:

  • The command replied with an integer. The integer value can be accessed using the
    reply->integer field of type long long.

• REDIS_REPLY_NIL:

  • The command replied with a nil object. There is no data to access.

• REDIS_REPLY_STRING:

  • A bulk (string) reply. The value of the reply can be accessed using reply->str.
    The length of this string can be accessed using reply->len.

• REDIS_REPLY_ARRAY:

  • A multi bulk reply. The number of elements in the multi bulk reply is stored in
    reply->elements. Every element in the multi bulk reply is a redisReply object as well
    and can be accessed via reply->element[..index..].
    Redis may reply with nested arrays but this is fully supported.

RESP3 Copy Section

Hiredis also supports every new RESP3 data type which are as follows. For more information about the protocol see the RESP3 specification.

• REDIS_REPLY_DOUBLE:

  • The command replied with a double-precision floating point number.
    The value is stored as a string in the str member, and can be converted with strtod or similar.

• REDIS_REPLY_BOOL:

  • A boolean true/false reply.
    The value is stored in the integer member and will be either 0 or 1.

• REDIS_REPLY_MAP:

  • An array with the added invariant that there will always be an even number of elements.
    The MAP is functionally equivalent to REDIS_REPLY_ARRAY except for the previously mentioned invariant.

• REDIS_REPLY_SET:

  • An array response where each entry is unique.
    Like the MAP type, the data is identical to an array response except there are no duplicate values.

• REDIS_REPLY_PUSH:

  • An array that can be generated spontaneously by Redis.
    This array response will always contain at least two subelements. The first contains the type of PUSH message (e.g. message, or invalidate), and the second being a sub-array with the PUSH payload itself.

• REDIS_REPLY_ATTR:

  • An array structurally identical to a MAP but intended as meta-data about a reply.
    As of Redis 6.0.6 this reply type is not used in Redis

• REDIS_REPLY_BIGNUM:

  • A string representing an arbitrarily large signed or unsigned integer value.
    The number will be encoded as a string in the str member of redisReply.

• REDIS_REPLY_VERB:

  • A verbatim string, intended to be presented to the user without modification.
    The string payload is stored in the str member, and type data is stored in the vtype member (e.g. txt for raw text or md for markdown).

Replies should be freed using the freeReplyObject() function.
Note that this function will take care of freeing sub-reply objects
contained in arrays and nested arrays, so there is no need for the user to
free the sub replies (it is actually harmful and will corrupt the memory).

Important: the current version of hiredis (1.0.0) frees replies when the
asynchronous API is used. This means you should not call freeReplyObject when
you use this API. The reply is cleaned up by hiredis after the callback
returns. We may introduce a flag to make this configurable in future versions of the library.

Cleaning up Copy Section

To disconnect and free the context the following function can be used:

void redisFree(redisContext *c);

This function immediately closes the socket and then frees the allocations done in
creating the context.

Sending commands (cont'd) Copy Section

Together with redisCommand, the function redisCommandArgv can be used to issue commands.
It has the following prototype:

void *redisCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);

It takes the number of arguments argc, an array of strings argv and the lengths of the
arguments argvlen. For convenience, argvlen may be set to NULL and the function will
use strlen(3) on every argument to determine its length. Obviously, when any of the arguments
need to be binary safe, the entire array of lengths argvlen should be provided.

The return value has the same semantic as redisCommand.

Pipelining Copy Section

To explain how Hiredis supports pipelining in a blocking connection, there needs to be
understanding of the internal execution flow.

When any of the functions in the redisCommand family is called, Hiredis first formats the
command according to the Redis protocol. The formatted command is then put in the output buffer
of the context. This output buffer is dynamic, so it can hold any number of commands.
After the command is put in the output buffer, redisGetReply is called. This function has the
following two execution paths:

1. The input buffer is non-empty:
   • Try to parse a single reply from the input buffer and return it
   • If no reply could be parsed, continue at 2
2. The input buffer is empty:
   • Write the entire output buffer to the socket
   • Read from the socket until a single reply could be parsed

The function redisGetReply is exported as part of the Hiredis API and can be used when a reply
is expected on the socket. To pipeline commands, the only thing that needs to be done is
filling up the output buffer. For this cause, two commands can be used that are identical
to the redisCommand family, apart from not returning a reply:

void redisAppendCommand(redisContext *c, const char *format, ...);
void redisAppendCommandArgv(redisContext *c, int argc, const char **argv, const size_t *argvlen);

After calling either function one or more times, redisGetReply can be used to receive the
subsequent replies. The return value for this function is either REDIS_OK or REDIS_ERR, where
the latter means an error occurred while reading a reply. Just as with the other commands,
the err field in the context can be used to find out what the cause of this error is.

The following examples shows a simple pipeline (resulting in only a single call to write(2) and
a single call to read(2)):

redisReply *reply;
redisAppendCommand(context,"SET foo bar");
redisAppendCommand(context,"GET foo");
redisGetReply(context,(void**)&reply); // reply for SET
freeReplyObject(reply);
redisGetReply(context,(void**)&reply); // reply for GET
freeReplyObject(reply);

This API can also be used to implement a blocking subscriber:

reply = redisCommand(context,"SUBSCRIBE foo");
freeReplyObject(reply);
while(redisGetReply(context,(void *)&reply) == REDIS_OK) {
    // consume message
    freeReplyObject(reply);
}

Errors Copy Section

When a function call is not successful, depending on the function either NULL or REDIS_ERR is
returned. The err field inside the context will be non-zero and set to one of the
following constants:

• REDIS_ERR_IO:
  There was an I/O error while creating the connection, trying to write
  to the socket or read from the socket. If you included errno.h in your
  application, you can use the global errno variable to find out what is
  wrong.

• REDIS_ERR_EOF:
  The server closed the connection which resulted in an empty read.

• REDIS_ERR_PROTOCOL:
  There was an error while parsing the protocol.

• REDIS_ERR_OTHER:
  Any other error. Currently, it is only used when a specified hostname to connect
  to cannot be resolved.

In every case, the errstr field in the context will be set to hold a string representation
of the error.

Asynchronous API Copy Chapter

Hiredis comes with an asynchronous API that works easily with any event library.
Examples are bundled that show using Hiredis with libev
and libevent.

Connecting Copy Section

The function redisAsyncConnect can be used to establish a non-blocking connection to
Redis. It returns a pointer to the newly created redisAsyncContext struct. The err field
should be checked after creation to see if there were errors creating the connection.
Because the connection that will be created is non-blocking, the kernel is not able to
instantly return if the specified host and port is able to accept a connection.
In case of error, it is the caller's responsibility to free the context using redisAsyncFree()

Note: A redisAsyncContext is not thread-safe.

An application function creating a connection might look like this:

void appConnect(myAppData *appData)
{
    redisAsyncContext *c = redisAsyncConnect("127.0.0.1", 6379);
    if (c->err) {
        printf("Error: %s\n", c->errstr);
        // handle error
        redisAsyncFree(c);
        c = NULL;
    } else {
        appData->context = c;
        appData->connecting = 1;
        c->data = appData; /* store application pointer for the callbacks */
        redisAsyncSetConnectCallback(c, appOnConnect);
        redisAsyncSetDisconnectCallback(c, appOnDisconnect);
    }
}

The asynchronous context should hold a connect callback function that is called when the connection
attempt completes, either successfully or with an error.
It can also hold a disconnect callback function that is called when the
connection is disconnected (either because of an error or per user request). Both callbacks should
have the following prototype:

void(const redisAsyncContext *c, int status);

On a connect, the status argument is set to REDIS_OK if the connection attempt succeeded. In this
case, the context is ready to accept commands. If it is called with REDIS_ERR then the
connection attempt failed. The err field in the context can be accessed to find out the cause of the error.
After a failed connection attempt, the context object is automatically freed by the library after calling
the connect callback. This may be a good point to create a new context and retry the connection.

On a disconnect, the status argument is set to REDIS_OK when disconnection was initiated by the
user, or REDIS_ERR when the disconnection was caused by an error. When it is REDIS_ERR, the err
field in the context can be accessed to find out the cause of the error.

The context object is always freed after the disconnect callback fired. When a reconnect is needed,
the disconnect callback is a good point to do so.

Setting the connect or disconnect callbacks can only be done once per context. For subsequent calls the
api will return REDIS_ERR. The function to set the callbacks have the following prototype:

/* Alternatively you can use redisAsyncSetConnectCallbackNC which will be passed a non-const
   redisAsyncContext* on invocation (e.g. allowing writes to the privdata member). */
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);

ac->data may be used to pass user data to both callbacks. A typical implementation
might look something like this:

void appOnConnect(redisAsyncContext *c, int status)
{
    myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
    appData->connecting = 0;
    if (status == REDIS_OK) {
        appData->connected = 1;
    } else {
        appData->connected = 0;
        appData->err = c->err;
        appData->context = NULL; /* avoid stale pointer when callback returns */
    }
    appAttemptReconnect();
}

void appOnDisconnect(redisAsyncContext *c, int status)
{
    myAppData *appData = (myAppData*)c->data; /* get my application specific context*/
    appData->connected = 0;
    appData->err = c->err;
    appData->context = NULL; /* avoid stale pointer when callback returns */
    if (status == REDIS_OK) {
        appNotifyDisconnectCompleted(mydata);
    } else {
        appNotifyUnexpectedDisconnect(mydata);
        appAttemptReconnect();
    }
}

Sending commands and their callbacks Copy Section

In an asynchronous context, commands are automatically pipelined due to the nature of an event loop.
Therefore, unlike the synchronous API, there is only a single way to send commands.
Because commands are sent to Redis asynchronously, issuing a command requires a callback function
that is called when the reply is received. Reply callbacks should have the following prototype:

void(redisAsyncContext *c, void *reply, void *privdata);

The privdata argument can be used to curry arbitrary data to the callback from the point where
the command is initially queued for execution.

The functions that can be used to issue commands in an asynchronous context are:

int redisAsyncCommand(
  redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
  const char *format, ...);
int redisAsyncCommandArgv(
  redisAsyncContext *ac, redisCallbackFn *fn, void *privdata,
  int argc, const char **argv, const size_t *argvlen);

Both functions work like their blocking counterparts. The return value is REDIS_OK when the command
was successfully added to the output buffer and REDIS_ERR otherwise. Example: when the connection
is being disconnected per user-request, no new commands may be added to the output buffer and REDIS_ERR is
returned on calls to the redisAsyncCommand family.

If the reply for a command with a NULL callback is read, it is immediately freed. When the callback
for a command is non-NULL, the memory is freed immediately following the callback: the reply is only
valid for the duration of the callback.

All pending callbacks are called with a NULL reply when the context encountered an error.

For every command issued, with the exception of SUBSCRIBE and PSUBSCRIBE, the callback is
called exactly once. Even if the context object id disconnected or deleted, every pending callback
will be called with a NULL reply.

For SUBSCRIBE and PSUBSCRIBE, the callbacks may be called repeatedly until an unsubscribe
message arrives. This will be the last invocation of the callback. In case of error, the callbacks
may receive a final NULL reply instead.

Disconnecting Copy Section

An asynchronous connection can be terminated using:

void redisAsyncDisconnect(redisAsyncContext *ac);

When this function is called, the connection is not immediately terminated. Instead, new
commands are no longer accepted and the connection is only terminated when all pending commands
have been written to the socket, their respective replies have been read and their respective
callbacks have been executed. After this, the disconnection callback is executed with the
REDIS_OK status and the context object is freed.

The connection can be forcefully disconnected using

void redisAsyncFree(redisAsyncContext *ac);

In this case, nothing more is written to the socket, all pending callbacks are called with a NULL
reply and the disconnection callback is called with REDIS_OK, after which the context object
is freed.

Hooking it up to event library *X* Copy Section

There are a few hooks that need to be set on the context object after it is created.
See the adapters/ directory for bindings to libev and libevent.

Reply parsing API Copy Chapter

Hiredis comes with a reply parsing API that makes it easy for writing higher
level language bindings.

The reply parsing API consists of the following functions:

redisReader *redisReaderCreate(void);
void redisReaderFree(redisReader *reader);
int redisReaderFeed(redisReader *reader, const char *buf, size_t len);
int redisReaderGetReply(redisReader *reader, void **reply);

The same set of functions are used internally by hiredis when creating a
normal Redis context, the above API just exposes it to the user for a direct
usage.

Usage Copy Section

The function redisReaderCreate creates a redisReader structure that holds a
buffer with unparsed data and state for the protocol parser.

Incoming data -- most likely from a socket -- can be placed in the internal
buffer of the redisReader using redisReaderFeed. This function will make a
copy of the buffer pointed to by buf for len bytes. This data is parsed
when redisReaderGetReply is called. This function returns an integer status
and a reply object (as described above) via void **reply. The returned status
can be either REDIS_OK or REDIS_ERR, where the latter means something went
wrong (either a protocol error, or an out of memory error).

The parser limits the level of nesting for multi bulk payloads to 7. If the
multi bulk nesting level is higher than this, the parser returns an error.

Customizing replies Copy Section

The function redisReaderGetReply creates redisReply and makes the function
argument reply point to the created redisReply variable. For instance, if
the response of type REDIS_REPLY_STATUS then the str field of redisReply
will hold the status as a vanilla C string. However, the functions that are
responsible for creating instances of the redisReply can be customized by
setting the fn field on the redisReader struct. This should be done
immediately after creating the redisReader.

For example, hiredis-rb
uses customized reply object functions to create Ruby objects.

Reader max buffer Copy Section

Both when using the Reader API directly or when using it indirectly via a
normal Redis context, the redisReader structure uses a buffer in order to
accumulate data from the server.
Usually this buffer is destroyed when it is empty and is larger than 16
KiB in order to avoid wasting memory in unused buffers

However when working with very big payloads destroying the buffer may slow
down performances considerably, so it is possible to modify the max size of
an idle buffer changing the value of the maxbuf field of the reader structure
to the desired value. The special value of 0 means that there is no maximum
value for an idle buffer, so the buffer will never get freed.

For instance if you have a normal Redis context you can set the maximum idle
buffer to zero (unlimited) just with:

context->reader->maxbuf = 0;

This should be done only in order to maximize performances when working with
large payloads. The context should be set back to REDIS_READER_MAX_BUF again
as soon as possible in order to prevent allocation of useless memory.

Reader max array elements Copy Section

By default the hiredis reply parser sets the maximum number of multi-bulk elements
to 2^32 - 1 or 4,294,967,295 entries. If you need to process multi-bulk replies
with more than this many elements you can set the value higher or to zero, meaning
unlimited with:

context->reader->maxelements = 0;

SSL/TLS Support Copy Chapter

Building Copy Section

SSL/TLS support is not built by default and requires an explicit flag:

make USE_SSL=1

This requires OpenSSL development package (e.g. including header files to be
available.

When enabled, SSL/TLS support is built into extra libhiredis_ssl.a and
libhiredis_ssl.so static/dynamic libraries. This leaves the original libraries
unaffected so no additional dependencies are introduced.

Using it Copy Section

First, you'll need to make sure you include the SSL header file:

#include <hiredis/hiredis.h>
#include <hiredis/hiredis_ssl.h>

You will also need to link against libhiredis_ssl, in addition to
libhiredis and add -lssl -lcrypto to satisfy its dependencies.

Hiredis implements SSL/TLS on top of its normal redisContext or
redisAsyncContext, so you will need to establish a connection first and then
initiate an SSL/TLS handshake.

Hiredis OpenSSL Wrappers

Before Hiredis can negotiate an SSL/TLS connection, it is necessary to
initialize OpenSSL and create a context. You can do that in two ways:

1. Work directly with the OpenSSL API to initialize the library's global context
   and create SSL_CTX * and SSL * contexts. With an SSL * object you can
   call redisInitiateSSL().
2. Work with a set of Hiredis-provided wrappers around OpenSSL, create a
   redisSSLContext object to hold configuration and use
   redisInitiateSSLWithContext() to initiate the SSL/TLS handshake.

/* An Hiredis SSL context. It holds SSL configuration and can be reused across
 * many contexts.
 */
redisSSLContext *ssl_context;

/* An error variable to indicate what went wrong, if the context fails to
 * initialize.
 */
redisSSLContextError ssl_error = REDIS_SSL_CTX_NONE;

/* Initialize global OpenSSL state.
 *
 * You should call this only once when your app initializes, and only if
 * you don't explicitly or implicitly initialize OpenSSL it elsewhere.
 */
redisInitOpenSSL();

/* Create SSL context */
ssl_context = redisCreateSSLContext(
    "cacertbundle.crt",     /* File name of trusted CA/ca bundle file, optional */
    "/path/to/certs",       /* Path of trusted certificates, optional */
    "client_cert.pem",      /* File name of client certificate file, optional */
    "client_key.pem",       /* File name of client private key, optional */
    "redis.mydomain.com",   /* Server name to request (SNI), optional */
    &ssl_error);

if(ssl_context == NULL || ssl_error != REDIS_SSL_CTX_NONE) {
    /* Handle error and abort... */
    /* e.g.
    printf("SSL error: %s\n",
        (ssl_error != REDIS_SSL_CTX_NONE) ?
            redisSSLContextGetError(ssl_error) : "Unknown error");
    // Abort
    */
}

/* Create Redis context and establish connection */
c = redisConnect("localhost", 6443);
if (c == NULL || c->err) {
    /* Handle error and abort... */
}

/* Negotiate SSL/TLS */
if (redisInitiateSSLWithContext(c, ssl_context) != REDIS_OK) {
    /* Handle error, in c->err / c->errstr */
}

RESP3 PUSH replies Copy Chapter

Redis 6.0 introduced PUSH replies with the reply-type >. These messages are generated spontaneously and can arrive at any time, so must be handled using callbacks.

Default behavior Copy Section

Hiredis installs handlers on redisContext and redisAsyncContext by default, which will intercept and free any PUSH replies detected. This means existing code will work as-is after upgrading to Redis 6 and switching to RESP3.

Custom PUSH handler prototypes Copy Section

The callback prototypes differ between redisContext and redisAsyncContext.

redisContext

void my_push_handler(void *privdata, void *reply) {
    /* Handle the reply */

    /* Note: We need to free the reply in our custom handler for
             blocking contexts.  This lets us keep the reply if
             we want. */
    freeReplyObject(reply);
}

redisAsyncContext

void my_async_push_handler(redisAsyncContext *ac, void *reply) {
    /* Handle the reply */

    /* Note:  Because async hiredis always frees replies, you should
              not call freeReplyObject in an async push callback. */
}

Installing a custom handler Copy Section

There are two ways to set your own PUSH handlers.

1. Set push_cb or async_push_cb in the redisOptions struct and connect with redisConnectWithOptions or redisAsyncConnectWithOptions.

   c Copy

   redisOptions = {0};
   REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
   options->push_cb = my_push_handler;
   redisContext *context = redisConnectWithOptions(&options);

2. Call redisSetPushCallback or redisAsyncSetPushCallback on a connected context.

   c Copy

   redisContext *context = redisConnect("127.0.0.1", 6379);
   redisSetPushCallback(context, my_push_handler);

   Note redisSetPushCallback and redisAsyncSetPushCallback both return any currently configured handler, making it easy to override and then return to the old value.

Specifying no handler Copy Section

If you have a unique use-case where you don't want hiredis to automatically intercept and free PUSH replies, you will want to configure no handler at all. This can be done in two ways.

1. Set the REDIS_OPT_NO_PUSH_AUTOFREE flag in redisOptions and leave the callback function pointer NULL.

   c Copy

   redisOptions = {0};
   REDIS_OPTIONS_SET_TCP(&options, "127.0.0.1", 6379);
   options->options |= REDIS_OPT_NO_PUSH_AUTOFREE;
   redisContext *context = redisConnectWithOptions(&options);

2. Call redisSetPushCallback with NULL once connected.

   c Copy

   redisContext *context = redisConnect("127.0.0.1", 6379);
   redisSetPushCallback(context, NULL);

   Note: With no handler configured, calls to redisCommand may generate more than one reply, so this strategy is only applicable when there's some kind of blocking redisGetReply() loop (e.g. MONITOR or SUBSCRIBE workloads).

Allocator injection Copy Chapter

Hiredis uses a pass-thru structure of function pointers defined in alloc.h that contain the currently configured allocation and deallocation functions. By default they just point to libc (malloc, calloc, realloc, etc).

Overriding Copy Section

One can override the allocators like so:

hiredisAllocFuncs myfuncs = {
    .mallocFn = my_malloc,
    .callocFn = my_calloc,
    .reallocFn = my_realloc,
    .strdupFn = my_strdup,
    .freeFn = my_free,
};

// Override allocators (function returns current allocators if needed)
hiredisAllocFuncs orig = hiredisSetAllocators(&myfuncs);

To reset the allocators to their default libc function simply call:

hiredisResetAllocators();

AUTHORS Copy Chapter

Salvatore Sanfilippo (antirez at gmail),
Pieter Noordhuis (pcnoordhuis at gmail)
Michael Grunder (michael dot grunder at gmail)

Hiredis is released under the BSD license.

File: deps/hdr_histogram/README.md Copy Chapter

HdrHistogram_c: 'C' port of High Dynamic Range (HDR) Histogram

HdrHistogram Copy Chapter

This port contains a subset of the functionality supported by the Java
implementation. The current supported features are:

• Standard histogram with 64 bit counts (32/16 bit counts not supported)
• All iterator types (all values, recorded, percentiles, linear, logarithmic)
• Histogram serialisation (encoding version 1.2, decoding 1.0-1.2)
• Reader/writer phaser and interval recorder

Features not supported, but planned

• Auto-resizing of histograms

Features unlikely to be implemented

• Double histograms
• Atomic/Concurrent histograms
• 16/32 bit histograms

Simple Tutorial

Recording values Copy Chapter

#include <hdr_histogram.h>

struct hdr_histogram* histogram;

// Initialise the histogram
hdr_init(
    1,  // Minimum value
    INT64_C(3600000000),  // Maximum value
    3,  // Number of significant figures
    &histogram)  // Pointer to initialise

// Record value
hdr_record_value(
    histogram,  // Histogram to record to
    value)  // Value to record

// Record value n times
hdr_record_values(
    histogram,  // Histogram to record to
    value,  // Value to record
    10)  // Record value 10 times

// Record value with correction for co-ordinated omission.
hdr_record_corrected_value(
    histogram,  // Histogram to record to
    value,  // Value to record
    1000)  // Record with expected interval of 1000.

// Print out the values of the histogram
hdr_percentiles_print(
    histogram,
    stdout,  // File to write to
    5,  // Granularity of printed values
    1.0,  // Multiplier for results
    CLASSIC);  // Format CLASSIC/CSV supported.

More examples Copy Chapter

For more detailed examples of recording and logging results look at the
hdr_decoder
and hiccup
examples. You can run hiccup and decoder
and pipe the results of one into the other.

$ ./examples/hiccup | ./examples/hdr_decoder

File: deps/fpconv/README.md Copy Chapter

libfpconv

Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm[1].

This port contains a subset of the 'C' version of Fast and accurate double to string conversion based on Florian Loitsch's Grisu-algorithm available at github.com/night-shift/fpconv).

[1] https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf

File: deps/xxhash/build/make/README.md Copy Chapter

multiconf.make

multiconf.make is a self-contained Makefile include that lets you build the same targets under many different flag sets. For example debug vs release, ASan vs UBSan, GCC vs Clang.
Each different set of flags generates object files into its own dedicated cache directory, so objects compiled with one configuration are never reused by another.
Object files from previous configurations are preserved, so swapping back to a previous configuration only requires compiling objects which have actually changed.

Benefits at a glance Copy Chapter

Quick Start Copy Chapter

1 · List your sources Copy Section

C_SRCDIRS   := src src/cdeps    # all .c are in these directories
CXX_SRCDIRS := src src/cxxdeps  # all .cpp are in these directories

2 · Add and include Copy Section

# root/Makefile
include multiconf.make

3 · Declare targets Copy Section

app:
$(eval $(call c_program,app,app.o cdeps/obj.o))

test:
$(eval $(call cxx_program,test, test.o cxxdeps/objcxx.o))

lib.a:
$(eval $(call static_library,lib.a, lib.o cdeps/obj.o))

lib.so:
$(eval $(call c_dynamic_library,lib.so, lib.o cdeps/obj.o))

4 · Build any config you like Copy Section

# Release with GCC
make CFLAGS="-O3"

# Debug with Clang + AddressSanitizer (new cache dir)
make CC=clang CFLAGS="-g -O0 -fsanitize=address"

# Switch back to GCC release (objects still valid, relink only)
make CFLAGS="-O3"

Objects for each command live in different sub-folders; nothing overlaps.

File: deps/README.md Copy Chapter

This directory contains all Redis dependencies, except for the libc that
should be provided by the operating system.

• Jemalloc is our memory allocator, used as replacement for libc malloc on Linux by default. It has good performances and excellent fragmentation behavior. This component is upgraded from time to time.
• hiredis is the official C client library for Redis. It is used by redis-cli, redis-benchmark and Redis Sentinel. It is part of the Redis official ecosystem but is developed externally from the Redis repository, so we just upgrade it as needed.
• linenoise is a readline replacement. It is developed by the same authors of Redis but is managed as a separated project and updated as needed.
• lua is Lua 5.1 with minor changes for security and additional libraries.
• hdr_histogram Used for per-command latency tracking histograms.

How to upgrade the above dependencies

Jemalloc Copy Chapter

Jemalloc is modified with changes that allow us to implement the Redis
active defragmentation logic. However this feature of Redis is not mandatory
and Redis is able to understand if the Jemalloc version it is compiled
against supports such Redis-specific modifications. So in theory, if you
are not interested in the active defragmentation, you can replace Jemalloc
just following these steps:

1. Remove the jemalloc directory.
2. Substitute it with the new jemalloc source tree.
3. Edit the Makefile located in the same directory as the README you are
   reading, and change the --with-version in the Jemalloc configure script
   options with the version you are using. This is required because otherwise
   Jemalloc configuration script is broken and will not work nested in another
   git repository.

However note that we change Jemalloc settings via the configure script of Jemalloc using the --with-lg-quantum option, setting it to the value of 3 instead of 4. This provides us with more size classes that better suit the Redis data structures, in order to gain memory efficiency.

If you want to upgrade Jemalloc while also providing support for
active defragmentation, in addition to the above steps you need to perform
the following additional steps:

5. In Jemalloc tree, file include/jemalloc/jemalloc_macros.h.in, make sure
   to add #define JEMALLOC_FRAG_HINT.
6. Implement the function je_get_defrag_hint() inside src/jemalloc.c. You
   can see how it is implemented in the current Jemalloc source tree shipped
   with Redis, and rewrite it according to the new Jemalloc internals, if they
   changed, otherwise you could just copy the old implementation if you are
   upgrading just to a similar version of Jemalloc.

Updating/upgrading jemalloc

The jemalloc directory is pulled as a subtree from the upstream jemalloc github repo. To update it you should run from the project root:

1. git subtree pull --prefix deps/jemalloc https://github.com/jemalloc/jemalloc.git <version-tag> --squash
   This should hopefully merge the local changes into the new version.
2. In case any conflicts arise (due to our changes) you'll need to resolve them and commit.
3. Reconfigure jemalloc:

rm deps/jemalloc/VERSION deps/jemalloc/configure
cd deps/jemalloc
./autogen.sh --with-version=<version-tag>-0-g0

4. Update jemalloc's version in deps/Makefile: search for "--with-version=<old-version-tag>-0-g0" and update it accordingly.
5. Commit the changes (VERSION,configure,Makefile).

Hiredis Copy Chapter

Hiredis is used by Sentinel, redis-cli and redis-benchmark. Like Redis, uses the SDS string library, but not necessarily the same version. In order to avoid conflicts, this version has all SDS identifiers prefixed by hi.

1. git subtree pull --prefix deps/hiredis https://github.com/redis/hiredis.git <version-tag> --squash
   This should hopefully merge the local changes into the new version.
2. Conflicts will arise (due to our changes) you'll need to resolve them and commit.

Linenoise Copy Chapter

Linenoise is rarely upgraded as needed. The upgrade process is trivial since
Redis uses a non modified version of linenoise, so to upgrade just do the
following:

1. Remove the linenoise directory.
2. Substitute it with the new linenoise source tree.

Lua Copy Chapter

We use Lua 5.1 and no upgrade is planned currently, since we don't want to break
Lua scripts for new Lua features: in the context of Redis Lua scripts the
capabilities of 5.1 are usually more than enough, the release is rock solid,
and we definitely don't want to break old scripts.

So upgrading of Lua is up to the Redis project maintainers and should be a
manual procedure performed by taking a diff between the different versions.

Currently we have at least the following differences between official Lua 5.1
and our version:

1. Makefile is modified to allow a different compiler than GCC.
2. We have the implementation source code, and directly link to the following external libraries: lua_cjson.o, lua_struct.o, lua_cmsgpack.o and lua_bit.o.
3. There is a security fix in ldo.c, line 498: The check for LUA_SIGNATURE[0] is removed in order to avoid direct bytecode execution.

Hdr_Histogram Copy Chapter

Updated source can be found here: https://github.com/HdrHistogram/HdrHistogram_c
We use a customized version based on master branch commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42.

1. Compare all changes under /hdr_histogram directory to upstream master commit e4448cf6d1cd08fff519812d3b1e58bd5a94ac42
2. Copy updated files from newer version onto files in /hdr_histogram.
3. Apply the changes from 1 above to the updated files.

File: modules/vector-sets/examples/cli-tool/README.md Copy Chapter

This tool is similar to redis-cli (but very basic) but allows
to specify arguments that are expanded as vectors by calling
ollama to get the embedding.

Whatever is passed as !"foo bar" gets expanded into
VALUES ... embedding ...

You must have ollama running with the mxbai-emb-large model
already installed for this to work.

Example:

redis> KEYS *
1) food_items
2) glove_embeddings_bin
3) many_movies_mxbai-embed-large_BIN
4) many_movies_mxbai-embed-large_NOQUANT
5) word_embeddings
6) word_embeddings_bin
7) glove_embeddings_fp32

redis> VSIM food_items !"drinks with fruit"
1) (Fruit)Juices,Lemonade,100ml,50 cal,210 kJ
2) (Fruit)Juices,Limeade,100ml,128 cal,538 kJ
3) CannedFruit,Canned Fruit Cocktail,100g,81 cal,340 kJ
4) (Fruit)Juices,Energy-Drink,100ml,87 cal,365 kJ
5) Fruits,Lime,100g,30 cal,126 kJ
6) (Fruit)Juices,Coconut Water,100ml,19 cal,80 kJ
7) Fruits,Lemon,100g,29 cal,122 kJ
8) (Fruit)Juices,Clamato,100ml,60 cal,252 kJ
9) Fruits,Fruit salad,100g,50 cal,210 kJ
10) (Fruit)Juices,Capri-Sun,100ml,41 cal,172 kJ

redis> vsim food_items !"barilla"
1) Pasta&Noodles,Spirelli,100g,367 cal,1541 kJ
2) Pasta&Noodles,Farfalle,100g,358 cal,1504 kJ
3) Pasta&Noodles,Capellini,100g,353 cal,1483 kJ
4) Pasta&Noodles,Spaetzle,100g,368 cal,1546 kJ
5) Pasta&Noodles,Cappelletti,100g,164 cal,689 kJ
6) Pasta&Noodles,Penne,100g,351 cal,1474 kJ
7) Pasta&Noodles,Shells,100g,353 cal,1483 kJ
8) Pasta&Noodles,Linguine,100g,357 cal,1499 kJ
9) Pasta&Noodles,Rotini,100g,353 cal,1483 kJ
10) Pasta&Noodles,Rigatoni,100g,353 cal,1483 kJ

File: modules/vector-sets/README.md Copy Chapter

IMPORTANT: Please note that this is a merged module, it's part of the Redis binary now, and you don't need to build it and load it into Redis. Compiling Redis version 8 or greater will result into having the Vector Sets commands available. However, you could compile this module as a shared library in order to load it in older versions of Redis.

This module implements Vector Sets for Redis, a new Redis data type similar
to Sorted Sets but having string elements associated to a vector instead of
a score. The fundamental goal of Vector Sets is to make possible adding items,
and later get a subset of the added items that are the most similar to a
specified vector (often a learned embedding), or the most similar to the vector
of an element that is already part of the Vector Set.

Moreover, Vector sets implement optional filtered search capabilities: it is possible to associate attributes to all or to a subset of elements in the set, and then, using the FILTER option of the VSIM command, to ask for items similar to a given vector but also passing a filter specified as a simple mathematical expression (Like ".year > 1950" or similar). This means that you can have vector similarity and scalar filters at the same time.

Installation Copy Chapter

WARNING: If you are running Redis 8.0 RC1 or greater you don't need to install anything, just compile Redis, and the Vector Sets commands will be part of the default install. Otherwise to test Vector Sets with older Redis versions follow the following instructions.

Build with:

make

Then load the module with the following command line, or by inserting the needed directives in the redis.conf file.

./redis-server --loadmodule vset.so

To run tests, I suggest using this:

./redis-server --save "" --enable-debug-command yes

The execute the tests with:

./test.py

Reference of available commands Copy Chapter

VADD: add items into a vector set

VADD key [REDUCE dim] FP32|VALUES vector element [CAS] [NOQUANT | Q8 | BIN]
         [EF build-exploration-factor] [SETATTR <attributes>] [M <numlinks>]

Add a new element into the vector set specified by the key.
The vector can be provided as FP32 blob of values, or as floating point
numbers as strings, prefixed by the number of elements (3 in the example):

VADD mykey VALUES 3 0.1 1.2 0.5 my-element

Meaning of the options:

REDUCE implements random projection, in order to reduce the
dimensionality of the vector. The projection matrix is saved and reloaded
along with the vector set. Please note that the REDUCE option must be passed immediately before the vector, like in REDUCE 50 VALUES ....

CAS performs the operation partially using threads, in a
check-and-set style. The neighbor candidates collection, which is slow, is
performed in the background, while the command is executed in the main thread.

NOQUANT forces the vector to be created (in the first VADD call to a given key) without integer 8 quantization, which is otherwise the default.

BIN forces the vector to use binary quantization instead of int8. This is much faster and uses less memory, but has impacts on the recall quality. The distance is computed as normalized Hamming distance (hamming_bits * 2 / dim), yielding values in [0, 2] consistent with cosine distance semantics, not raw Hamming bit counts.

Q8 forces the vector to use signed 8 bit quantization. This is the default, and the option only exists in order to make sure to check at insertion time if the vector set is of the same format.

EF plays a role in the effort made to find good candidates when connecting the new node to the existing HNSW graph. The default is 200. Using a larger value, may help to have a better recall. To improve the recall it is also possible to increase EF during VSIM searches.

SETATTR associates attributes to the newly created entry or update the entry attributes (if it already exists). It is the same as calling the VSETATTR attribute separately, so please check the documentation of that command in the filtered search section of this documentation.

M defaults to 16 and is the HNSW famous M parameters. It is the maximum number of connections that each node of the graph have with other nodes: more connections mean more memory, but a better ability to explore the graph. Nodes at layer zero (every node exists at least at layer zero) have M*2 connections, while the other layers only have M connections. This means that, for instance, an M of 64 will use at least 1024 bytes of memory for each node! That is, 64 links * 2 times * 8 bytes pointers, and even more, since on average each node has something like 1.33 layers (but the other layers have just M connections, instead of M*2). If you don't have a recall quality problem, the default is fine, and uses a limited amount of memory.

VSIM: return elements by vector similarity

VSIM key [ELE|FP32|VALUES] <vector or element> [WITHSCORES] [WITHATTRIBS] [COUNT num] [EPSILON delta] [EF search-exploration-factor] [FILTER expression] [FILTER-EF max-filtering-effort] [TRUTH] [NOTHREAD]

The command returns similar vectors, for simplicity (and verbosity) in the following example, instead of providing a vector using FP32 or VALUES (like in VADD), we will ask for elements having a vector similar to a given element already in the sorted set:

> VSIM word_embeddings ELE apple
 1) "apple"
 2) "apples"
 3) "pear"
 4) "fruit"
 5) "berry"
 6) "pears"
 7) "strawberry"
 8) "peach"
 9) "potato"
10) "grape"

It is possible to specify a COUNT and also to get the similarity score (from 1 to 0, where 1 is identical, 0 is opposite vector) between the query and the returned items.

> VSIM word_embeddings ELE apple WITHSCORES COUNT 3
1) "apple"
2) "0.9998867657923256"
3) "apples"
4) "0.8598527610301971"
5) "pear"
6) "0.8226882219314575"

It is also possible to specify a EPSILON, that is a floating point number between 0 and 1 in order to only return elements that have a distance that is no further than the specified one. In vector sets, the returned elements have a similarity score (when compared to the query vector) that is between 1 and 0, where 1 means identical, 0 opposite vectors. If for instance the EPSILON option is specified with an argument of 0.2, it means that we will get only elements that have a similarity of 0.8 or better (a distance < 0.2). This is useful when a large COUNT is specified, yet we don't want elements that are too far away our query vector.

The EF argument is the exploration factor: the higher it is, the slower the command becomes, but the better the index is explored to find nodes that are near to our query. Sensible values are from 50 to 1000.

The TRUTH option forces the command to perform a linear scan of all the entries inside the set, without using the graph search inside the HNSW, so it returns the best matching elements (the perfect result set) that can be used in order to easily calculate the recall. Of course the linear scan is O(N), so it is much slower than the log(N) (considering a small COUNT) provided by the HNSW index.

The NOTHREAD option forces the command to execute the search on the data structure in the main thread. Normally VSIM spawns a thread instead. This may be useful for benchmarking purposes, or when we work with extremely small vector sets and don't want to pay the cost of spawning a thread. It is possible that in the future this option will be automatically used by Redis when we detect small vector sets. Note that this option blocks the server for all the time needed to complete the command, so it is a source of potential latency issues: if you are in doubt, never use it.

The WITHSCORES option returns, for each returned element, a floating point number representing how near the element is from the query, as a similarity between 0 and 1, where 0 means the vectors are opposite, and 1 means they are pointing exactly in the same direction (maximum similarity).

The WITHATTRIBS option returns, for each element, the JSON attribute associated with the element, or NULL for the elements missing an attribute.

For FILTER and FILTER-EF options, please check the filtered search section of this documentation.

Note that when WITHSCORES and WITHATTRIBS are provided at the same time, the RESP2 reply guarantees that the returned elements are always in the sequence ele,score,attribs, while RESP3 replies will be in the form ele > score|attrib when just one is provided, or ele -> [score,attrib] when both are provided, that is, when both options are used and RESP3 is used the score and attribute will be a two-items array associated to the element key.

VDIM: return the dimension of the vectors inside the vector set

VDIM keyname

Example:

> VDIM word_embeddings
(integer) 300

Note that in the case of vectors that were populated using the REDUCE
option, for random projection, the vector set will report the size of
the projected (reduced) dimension. Yet the user should perform all the
queries using full-size vectors.

VCARD: return the number of elements in a vector set

VCARD key

Example:

> VCARD word_embeddings
(integer) 3000000

VREM: remove elements from vector set

VREM key element

Example:

> VADD vset VALUES 3 1 0 1 bar
(integer) 1
> VREM vset bar
(integer) 1
> VREM vset bar
(integer) 0

VREM does not perform thumstone / logical deletion, but will actually reclaim
the memory from the vector set, so it is save to add and remove elements
in a vector set in the context of long running applications that continuously
update the same index.

VEMB: return the approximated vector of an element

VEMB key element

Example:

> VEMB word_embeddings SQL
  1) "0.18208661675453186"
  2) "0.08535309880971909"
  3) "0.1365649551153183"
  4) "-0.16501599550247192"
  5) "0.14225517213344574"
  ... 295 more elements ...

Because vector sets perform insertion time normalization and optional
quantization, the returned vector could be approximated. VEMB will take
care to de-quantized and de-normalize the vector before returning it.

It is possible to ask VEMB to return raw data, that is, the internal representation used by the vector: fp32, int8, or a bitmap for binary quantization. This behavior is triggered by the RAW option of of VEMB:

VEMB word_embedding apple RAW

In this case the return value of the command is an array of three or more elements:

1. The name of the quantization used, that is one of: "fp32", "bin", "q8".
2. The a string blob containing the raw data, 4 bytes fp32 floats for fp32, a bitmap for binary quants, or int8 bytes array for q8 quants.
3. A float representing the l2 of the vector before normalization. You need to multiply by this vector if you want to de-normalize the value for any reason.

For q8 quantization, an additional elements is also returned: the quantization
range, so the integers from -127 to 127 represent (normalized) components
in the range -range, +range.

VISMEMBER: test if a given element already exists

This command will return 1 (or true) if the specified element is already in the vector set, otherwise 0 (or false) is returned.

VISMEMBER key element

As with other existence check Redis commands, if the key does not exist it is considered as if it was empty, thus the element is reported as non existing.

**VRANGE: return elements in a lexicographical range

VRANGE key start end count

The VRANGE command has many different use cases, but its main goal is to
provide a stateless iterator for the elements inside a vector set: that is,
it allows to retrieve all the elements inside a vector set in small amounts
for each call, without an explicit cursor, and with guarantees about what
the user will miss in case the vector set is changing (elements added and/or
removed) during the iteration.

The command usage is straightforward:

> VRANGE word_embeddings_int8 [Redis + 10
 1) "Redis"
 2) "Rediscover"
 3) "Rediscover_Ashland"
 4) "Rediscover_Northern_Ireland"
 5) "Rediscovered"
 6) "Rediscovered_Bookshop"
 7) "Rediscovering"
 8) "Rediscovering_God"
 9) "Rediscovering_Lost"
10) "Rediscovers"

The above command returns 10 (or less, if less are available in the specified range) elements from "Redis" (inclusive) to the maximum possible element. The comparison is performed byte by byte, as memcmp() would do, in this way the elements have a total order. The start and end range can be either a string, prefixed by [ or ( (the prefix is mandatory) to tell the command if the range is inclusive or exclusive, or can be the special symbols - and + that means the maximum and minimum element.

So for instance if I want to iterate all the elements, ten elements for each call, I'll proceed as such:

> VRANGE mykey - + 10
 1) "a"
 2) "a-league"
 3) "a."
 4) "a.d."
 5) "a.k.a."
 6) "a.m."
 7) "a1"
 8) "a2"
 9) "a3"
10) "a7"

This will give me the first 10 elements. Then I want the next ten elements
starting from the last element in the previous result, but excluding it,
so the next range will use the ( prefix with the last element of the
previous call, that was "a7":

> VRANGE mykey (a7 + 10
 1) "a930913"
 2) "aa"
 3) "aaa"
 4) "aaron"
 5) "ab"
 6) "aba"
 7) "abandon"
 8) "abandoned"
 9) "abandoning"
10) "abandonment"

And so forth.

The command count is mandatory, however a negative count means to return all the elements in the set. This means that VRANGE mykey - + -1 will return every element. Of course, iterating like that means that it is possible to block the server for a long time.

The command time complexity is O(1) to seek to the element (considering the element would be of reasonable size), since we use a Radix Tree in the underlying implementation, plus the time to yield "M" elements. So if M is small, each call is just executed in constant time. However the iteration of a total set (via multiple calls) of N elements is O(N). Basically: this command, with a small count, will never produce latency issues in the Redis server.

In case the elements are changing continuously as the set is iterated, the guarantees are very simple: each range will produce exactly the elements that were present in the range in the moment the VRANGE command was executed. In other words, an iteration performed in this way is guaranteed to return all the elements that stayed within the vector set from the start to the end of the iteration. Elements removed or added in the meantime may be returned or not depending on the moment they were added or removed.

VLINKS: introspection command that shows neighbors for a node

VLINKS key element [WITHSCORES]

The command reports the neighbors for each level.

VINFO: introspection command that shows info about a vector set

VINFO key

Example:

> VINFO word_embeddings
 1) quant-type
 2) int8
 3) vector-dim
 4) (integer) 300
 5) size
 6) (integer) 3000000
 7) max-level
 8) (integer) 12
 9) vset-uid
10) (integer) 1
11) hnsw-max-node-uid
12) (integer) 3000000

VSETATTR: associate or remove the JSON attributes of elements

VSETATTR key element "{... json ...}"

Each element of a vector set can be optionally associated with a JSON string
in order to use the FILTER option of VSIM to filter elements by scalars
(see the filtered search section for more information). This command can set,
update (if already set) or delete (if you set to an empty string) the
associated JSON attributes of an element.

The command returns 0 if the element or the key don't exist, without
raising an error, otherwise 1 is returned, and the element attributes
are set or updated.

VGETATTR: retrieve the JSON attributes of elements

VGETATTR key element

The command returns the JSON attribute associated with an element, or
null if there is no element associated, or no element at all, or no key.

VRANDMEMBER: return random members from a vector set

VRANDMEMBER key [count]

Return one or more random elements from a vector set.

The semantics of this command are similar to Redis's native SRANDMEMBER command:

• When called without count, returns a single random element from the set, as a single string (no array reply).
• When called with a positive count, returns up to count distinct random elements (no duplicates).
• When called with a negative count, returns count random elements, potentially with duplicates.
• If the count value is larger than the set size (and positive), only the entire set is returned.

If the key doesn't exist, returns a Null reply if count is not given, or an empty array if a count is provided.

Examples:

> VADD vset VALUES 3 1 0 0 elem1
(integer) 1
> VADD vset VALUES 3 0 1 0 elem2
(integer) 1
> VADD vset VALUES 3 0 0 1 elem3
(integer) 1

# Return a single random element
> VRANDMEMBER vset
"elem2"

# Return 2 distinct random elements
> VRANDMEMBER vset 2
1) "elem1"
2) "elem3"

# Return 3 random elements with possible duplicates
> VRANDMEMBER vset -3
1) "elem2"
2) "elem2"
3) "elem1"

# Return more elements than in the set (returns all elements)
> VRANDMEMBER vset 10
1) "elem1"
2) "elem2"
3) "elem3"

# When key doesn't exist
> VRANDMEMBER nonexistent
(nil)
> VRANDMEMBER nonexistent 3
(empty array)

This command is particularly useful for:

1. Selecting random samples from a vector set for testing or training.
2. Performance testing by retrieving random elements for subsequent similarity searches.

When the user asks for unique elements (positev count) the implementation optimizes for two scenarios:

• For small sample sizes (less than 20% of the set size), it uses a dictionary to avoid duplicates, and performs a real random walk inside the graph.
• For large sample sizes (more than 20% of the set size), it starts from a random node and sequentially traverses the internal list, providing faster performances but not really "random" elements.

The command has O(N) worst-case time complexity when requesting many unique elements (it uses linear scanning), or O(M*log(N)) complexity when the users asks for M random elements in a sorted set of N elements, with M much smaller than N.

Filtered search

Each element of the vector set can be associated with a set of attributes specified as a JSON blob:

> VADD vset VALUES 3 1 1 1 a SETATTR '{"year": 1950}'
(integer) 1
> VADD vset VALUES 3 -1 -1 -1 b SETATTR '{"year": 1951}'
(integer) 1

Specifying an attribute with the SETATTR option of VADD is exactly equivalent to adding an element and then setting (or updating, if already set) the attributes JSON string. Also the symmetrical VGETATTR command returns the attribute associated to a given element.

> VADD vset VALUES 3 0 1 0 c
(integer) 1
> VSETATTR vset c '{"year": 1952}'
(integer) 1
> VGETATTR vset c
"{\"year\": 1952}"

At this point, I may use the FILTER option of VSIM to only ask for the subset of elements that are verified by my expression:

> VSIM vset VALUES 3 0 0 0 FILTER '.year > 1950'
1) "c"
2) "b"

The items will be returned again in order of similarity (most similar first), but only the items with the year field matching the expression is returned.

The expressions are similar to what you would write inside the if statement of JavaScript or other familiar programming languages: you can use and, or, the obvious math operators like +, -, /, >=, <, ... and so forth (see the expressions section for more info). The selectors of the JSON object attributes start with a dot followed by the name of the key inside the JSON objects.

Elements with invalid JSON or not having a given specified field are considered as not matching the expression, but will not generate any error at runtime.

FILTER expressions capabilities Copy Chapter

FILTER expressions allow you to perform complex filtering on vector similarity results using a JavaScript-like syntax. The expression is evaluated against each element's JSON attributes, with only elements that satisfy the expression being included in the results.

Expression Syntax Copy Section

Expressions support the following operators and capabilities:

1. Arithmetic operators: +, -, *, /, % (modulo), ** (exponentiation)
2. Comparison operators: >, >=, <, <=, ==, !=
3. Logical operators: and/&&, or/||, !/not
4. Containment operator: in
5. Parentheses for grouping: (...)

Selector Notation Copy Section

Attributes are accessed using dot notation:

• .year references the "year" attribute
• .movie.year would NOT reference the "year" field inside a "movie" object, only keys that are at the first level of the JSON object are accessible.

JSON and expressions data types Copy Section

Expressions can work with:

• Numbers (double precision floats)
• Strings (enclosed in single or double quotes)
• Booleans (no native type: they are represented as 1 for true, 0 for false)
• Arrays (for use with the in operator: value in [1, 2, 3])

JSON attributes are converted in this way:

• Numbers will be converted to numbers.
• Strings to strings.
• Booleans to 0 or 1 number.
• Arrays to tuples (for "in" operator), but only if composed of just numbers and strings.

Any other type is ignored, and accessig it will make the expression evaluate to false.

The IN operator Copy Section

The IN operator works in two ways, it can test for membership in an array, like in:

5 in [1, 2, 3]
"foo" in [1, "foo", "bar"]

But can also check for substrings, in case the A and B operators are both strings.

"foo" in "barfoobar" # Will evaluate to true
"zap" in "foobar" # Will evaluate to false

Examples Copy Section

# Find items from the 1980s
VSIM movies VALUES 3 0.5 0.8 0.2 FILTER '.year >= 1980 and .year < 1990'

# Find action movies with high ratings
VSIM movies VALUES 3 0.5 0.8 0.2 FILTER '.genre == "action" and .rating > 8.0'

# Find movies directed by either Spielberg or Nolan
VSIM movies VALUES 3 0.5 0.8 0.2 FILTER '.director in ["Spielberg", "Nolan"]'

# Complex condition with numerical operations
VSIM movies VALUES 3 0.5 0.8 0.2 FILTER '(.year - 2000) ** 2 < 100 and .rating / 2 > 4'

Error Handling Copy Section

Elements with any of the following conditions are considered not matching:

• Missing the queried JSON attribute
• Having invalid JSON in their attributes
• Having a JSON value that cannot be converted to the expected type

This behavior allows you to safely filter on optional attributes without generating errors.

FILTER effort Copy Section

The FILTER-EF option controls the maximum effort spent when filtering vector search results.

When performing vector similarity search with filtering, Vector Sets perform the standard similarity search as they apply the filter expression to each node. Since many results might be filtered out, Vector Sets may need to examine a lot more candidates than the requested COUNT to ensure sufficient matching results are returned. Actually, if the elements matching the filter are very rare or if there are less than elements matching than the specified count, this would trigger a full scan of the HNSW graph.

For this reason, by default, the maximum effort is limited to a reasonable amount of nodes explored.

Modifying the FILTER effort Copy Section

1. By default, Vector Sets will explore up to COUNT * 100 candidates to find matching results.
2. You can control this exploration with the FILTER-EF parameter.
3. A higher FILTER-EF value increases the chances of finding all relevant matches at the cost of increased processing time.
4. A FILTER-EF of zero will explore as many nodes as needed in order to actually return the number of elements specified by COUNT.
5. Even when a high FILTER-EF value is specified the implementation will do a lot less work if the elements passing the filter are very common, because of the early stop conditions of the HNSW implementation (once the specified amount of elements is reached and the quality check of the other candidates trigger an early stop).

VSIM key [ELE|FP32|VALUES] <vector or element> COUNT 10 FILTER '.year > 2000' FILTER-EF 500

In this example, Vector Sets will examine up to 500 potential nodes. Of course if count is reached before exploring 500 nodes, and the quality checks show that it is not possible to make progresses on similarity, the search is ended sooner.

Performance Considerations Copy Section

• If you have highly selective filters (few items match), use a higher FILTER-EF, or just design your application in order to handle a result set that is smaller than the requested count. Note that anyway the additional elements may be too distant than the query vector.
• For less selective filters, the default should be sufficient.
• Very selective filters with low FILTER-EF values may return fewer items than requested.
• Extremely high values may impact performance without significantly improving results.

The optimal FILTER-EF value depends on:

1. The selectivity of your filter.
2. The distribution of your data.
3. The required recall quality.

A good practice is to start with the default and increase if needed when you observe fewer results than expected.

Testing a large-ish data set Copy Section

To really see how things work at scale, you can download the following dataset:

wget https://antirez.com/word2vec_with_attribs.rdb

It contains the 3 million words in Word2Vec having as attribute a JSON with just the length of the word. Because of the length distribution of words in large amounts of texts, where longer words become less and less common, this is ideal to check how filtering behaves with a filter verifying as true with less and less elements in a vector set.

For instance:

> VSIM word_embeddings_bin ele "pasta" FILTER ".len == 6"
 1) "pastas"
 2) "rotini"
 3) "gnocci"
 4) "panino"
 5) "salads"
 6) "breads"
 7) "salame"
 8) "sauces"
 9) "cheese"
10) "fritti"

This will easily retrieve the desired amount of items (COUNT is 10 by default) since there are many items of length 6. However:

> VSIM word_embeddings_bin ele "pasta" FILTER ".len == 33"
1) "skinless_boneless_chicken_breasts"
2) "boneless_skinless_chicken_breasts"
3) "Boneless_skinless_chicken_breasts"

This time even if we asked for 10 items, we only get 3, since the default filter effort will be 10*100 = 1000. We can tune this giving the effort in an explicit way, with the risk of our query being slower, of course:

> VSIM word_embeddings_bin ele "pasta" FILTER ".len == 33" FILTER-EF 10000
 1) "skinless_boneless_chicken_breasts"
 2) "boneless_skinless_chicken_breasts"
 3) "Boneless_skinless_chicken_breasts"
 4) "mozzarella_feta_provolone_cheddar"
 5) "Greatfood.com_R_www.greatfood.com"
 6) "Pepperidge_Farm_Goldfish_crackers"
 7) "Prosecuted_Mobsters_Rebuilt_Dying"
 8) "Crispy_Snacker_Sandwiches_Popcorn"
 9) "risultati_delle_partite_disputate"
10) "Peppermint_Mocha_Twist_Gingersnap"

This time we get all the ten items, even if the last one will be quite far from our query vector. We encourage to experiment with this test dataset in order to understand better the dynamics of the implementation and the natural tradeoffs of filtered search.

Keep in mind that by default, Redis Vector Sets will try to avoid a likely very useless huge scan of the HNSW graph, and will be more happy to return few or no elements at all, since this is almost always what the user actually wants in the context of retrieving similar items to the query.

Single Instance Scalability and Latency

Vector Sets implement a threading model that allows Redis to handle many concurrent requests: by default VSIM is always threaded, and VADD is not (but can be partially threaded using the CAS option). This section explains how the threading and locking mechanisms work, and what to expect in terms of performance.

Threading Model Copy Chapter

• The VSIM command runs in a separate thread by default, allowing Redis to continue serving other commands.
• A maximum of 32 threads can run concurrently (defined by HNSW_MAX_THREADS).
• When this limit is reached, additional VSIM requests are queued - Redis remains responsive, no latency event is generated.
• The VADD command with the CAS option also leverages threading for the computation-heavy candidate search phase, but the insertion itself is performed in the main thread. VADD always runs in a sub-millisecond time, so this is not a source of latency, but having too many hundreds of writes per second can be challenging to handle with a single instance. Please, look at the next section about multiple instances scalability.
• Commands run within Lua scripts, MULTI/EXEC blocks, or from replication are executed in the main thread to ensure consistency.

> VSIM vset VALUES 3 1 1 1 FILTER '.year > 2000'  # This runs in a thread.
> VADD vset VALUES 3 1 1 1 element CAS            # Candidate search runs in a thread.

Locking Mechanism Copy Chapter

Vector Sets use a read/write locking mechanism to coordinate access:

• Reads (VSIM, VEMB, etc.) acquire a read lock, allowing multiple concurrent reads.
• Writes (VADD, VREM, etc.) acquire a write lock, temporarily blocking all reads.
• When a write lock is requested while reads are in progress, the write operation waits for all reads to complete.
• Once a write lock is granted, all reads are blocked until the write completes.
• Each thread has a dedicated slot for tracking visited nodes during graph traversal, avoiding contention. This improves performances but limits the maximum number of concurrent threads, since each node has a memory cost proportional to the number of slots.

DEL latency Copy Chapter

Deleting a very large vector set (millions of elements) can cause latency spikes, as deletion rebuilds connections between nodes. This may change in the future.
The deletion latency is most noticeable when using DEL on a key containing a large vector set or when the key expires.

Performance Characteristics Copy Chapter

• Search operations (VSIM) scale almost linearly with the number of CPU cores available, up to the thread limit. You can expect a Vector Set composed of million of items associated with components of dimension 300, with the default int8 quantization, to deliver around 50k VSIM operations per second in a single host.
• Insertion operations (VADD) are more computationally expensive than searches, and can't be threaded: expect much lower throughput, in the range of a few thousands inserts per second.
• Binary quantization offers significantly faster search performance at the cost of some recall quality, while int8 quantization, the default, seems to have very small impacts on recall quality, while it significantly improves performances and space efficiency.
• The EF parameter has a major impact on both search quality and performance - higher values mean better recall but slower searches.
• Graph traversal time scales logarithmically with the number of elements, making Vector Sets efficient even with millions of vectors

Loading / Saving performances Copy Chapter

Vector Sets are able to serialize on disk the graph structure as it is in memory, so loading back the data does not need to rebuild the HNSW graph. This means that Redis can load millions of items per minute. For instance 3 million items with 300 components vectors can be loaded back into memory into around 15 seconds.

Scaling vector sets to multiple instances

The fundamental way vector sets can be scaled to very large data sets
and to many Redis instances is that a given very large set of vectors
can be partitioned into N different Redis keys, that can also live into
different Redis instances.

For instance, I could add my elements into key0, key1, key2, by hashing
the item in some way, like doing crc32(item)%3, effectively splitting
the dataset into three different parts. However once I want all the vectors
of my dataset near to a given query vector, I could simply perform the
VSIM command against all the three keys, merging the results by
score (so the commands must be called using the WITHSCORES option) on
the client side: once the union of the results are ordered by the
similarity score, the query is equivalent to having a single key key1+2+3
containing all the items.

There are a few interesting facts to note about this pattern:

1. It is possible to have a logical sorted set that is as big as the sum of all the Redis instances we are using.
2. Deletion operations remain simple, we can hash the key and select the key where our item belongs.
3. However, even if I use 10 different Redis instances, I'm not going to reach 10x the read operations per second, compared to using a single server: for each logical query, I need to query all the instances. Yet, smaller graphs are faster to navigate, so there is some win even from the point of view of CPU usage.
4. Insertions, so write queries, will be scaled linearly: I can add N items against N instances at the same time, splitting the insertion load evenly. This is very important since vector sets, being based on HNSW data structures, are slower to add items than to query similar items, by a very big factor.
5. While it cannot guarantee always the best results, with proper timeout management this system may be considered highly available, since if a subset of N instances are reachable, I'll be still be able to return similar items to my query vector.

Notably, this pattern can be implemented in a way that avoids paying the sum of the round trip time with all the servers: it is possible to send the queries at the same time to all the instances, so that latency will be equal the slower reply out of of the N servers queries.

Optimizing memory usage

Vector Sets, or better, HNSWs, the underlying data structure used by Vector Sets, combined with the features provided by the Vector Sets themselves (quantization, random projection, filtering, ...) form an implementation that has a non-trivial space of parameters that can be tuned. Despite to the complexity of the implementation and of vector similarity problems, here there is a list of simple ideas that can drive the user to pick the best settings:

• 8 bit quantization (the default) is almost always a win. It reduces the memory usage of vectors by a factor of 4, yet the performance penalty in terms of recall is minimal. It also reduces insertion and search time by around 2 times or more.
• Binary quantization is much more extreme: it makes vector sets a lot faster, but increases the recall error in a sensible way, for instance from 95% to 80% if all the parameters remain the same. Yet, the speedup is really big, and the memory usage of vectors, compaerd to full precision vectors, 32 times smaller.
• Vectors memory usage are not the only responsible for Vector Set high memory usage per entry: nodes contain, on average M*2 + M*0.33 pointers, where M is by default 16 (but can be tuned in VADD, see the M option). Also each node has the string item and the optional JSON attributes: those should be as small as possible in order to avoid contributing more to the memory usage.
• The M parameter should be increased to 32 or more only when a near perfect recall is really needed.
• It is possible to gain space (less memory usage) sacrificing time (more CPU time) by using a low M (the default of 16, for instance) and a high EF (the effort parameter of VSIM) in order to scan the graph more deeply.
• When memory usage is seriosu concern, and there is the suspect the vectors we are storing don't contain as much information - at least for our use case - to justify the number of components they feature, random projection (the REDUCE option of VADD) could be tested to see if dimensionality reduction is possible with acceptable precision loss.

Random projection tradeoffs Copy Chapter

Sometimes learned vectors are not as information dense as we could guess, that
is there are components having similar meanings in the space, and components
having values that don't really represent features that matter in our use case.

At the same time, certain vectors are very big, 1024 components or more. In this cases, it is possible to use the random projection feature of Redis Vector Sets in order to reduce both space (less RAM used) and space (more operations per second). The feature is accessible via the REDUCE option of the VADD command. However, keep in mind that you need to test how much reduction impacts the performances of your vectors in term of recall and quality of the results you get back.

What is a random projection? Copy Chapter

The concept of Random Projection is relatively simple to grasp. For instance, a projection that turns a 100 components vector into a 10 components vector will perform a different linear transformation between the 100 components and each of the target 10 components. Please note that each of the target components will get some random amount of all the 100 original components. It is mathematically proved that this process results in a vector space where elements still have similar distances among them, but still some information will get lost.

Examples of projections and loss of precision Copy Chapter

To show you a bit of a extreme case, let's take Word2Vec 3 million items and compress them from 300 to 100, 50 and 25 components vectors. Then, we check the recall compared to the ground truth against each of the vector sets produced in this way (using different REDUCE parameters of VADD). This is the result, obtained asking for the top 10 elements.

----------------------------------------------------------------------
Key                            Average Recall % Std Dev
----------------------------------------------------------------------
word_embeddings_int8           95.98           12.14
  ^ This is the same key used for ground truth, but without TRUTH option
word_embeddings_reduced_100    40.20           20.13
word_embeddings_reduced_50     24.42           16.89
word_embeddings_reduced_25     14.31           9.99

Here the dimensionality reduction we are using is quite extreme: from 300 to 100 means that 66.6% of the original information is lost. The recall drops from 96% to 40%, down to 24% and 14% for even more extreme dimension reduction.

Reducing the dimension of vectors that are already relatively small, like the above example, of 300 components, will provide only relatively small memory savings, especially because by default Vector Sets use int8 quantization, that will use only one byte per component:

> MEMORY USAGE word_embeddings_int8
(integer) 3107002888
> MEMORY USAGE word_embeddings_reduced_100
(integer) 2507122888

Of course going, for example, from 2048 component vectors to 1024 would provide a much more sensible memory saving, even with the int8 quantization used by Vector Sets, assuming the recall loss is acceptable. Other than the memory saving, there is also the reduction in CPU time, translating to more operations per second.

Another thing to note is that, with certain embedding models, binary quantization (that offers a 8x reduction of memory usage compared to 8 bit quants, and a very big speedup in computation) performs much better than reducing the dimension of vectors of the same amount via random projections:

word_embeddings_bin            35.48           19.78

Here in the same test did above: we have a 35% recall which is not too far than the 40% obtained with a random projection from 300 to 100 components. However, while the first technique reduces the size by 3 times, the size reduced of binary quantization is by 8 times.

> memory usage word_embeddings_bin
(integer) 2327002888

In this specific case the key uses JSON attributes and has a graph connection overhead that is much bigger than the 300 bits each vector takes, but, as already said, for big vectors (1024 components, for instance) or for lower values of M (see VADD, the M parameter connects the level of connectivity, so it changes the amount of pointers used per node) the memory saving is much stronger.

Vector Sets troubleshooting and understandability

Debugging poor recall or unexpected results Copy Chapter

Vector graphs and similarity queries pose many challenges mainly due to the following three problems:

1. The error due to the approximated nature of Vector Sets is hard to evaluate.
2. The error added by the quantization is often depends on the exact vector space (the embedding we are using and how far apart the elements we represent into such embeddings are).
3. We live in the illusion that learned embeddings capture the best similarity possible among elements, which is obviously not always true, and highly application dependent.

The only way to debug such problems, is the ability to inspect step by step what is happening inside our application, and the structure of the HNSW graph itself. To do so, we suggest to consider the following tools:

1. The TRUTH option of the VSIM command is able to return the ground truth of the most similar elements, without using the HNSW graph, but doing a linear scan.
2. The VLINKS command allows to explore the graph to see if the connections among nodes make sense, and to investigate why a given node may be more isolated than expected. Such command can also be used in a different way, when we want very fast "similar items" without paying the HNSW traversal time. It exploits the fact that we have a direct reference from each element in our vector set to each node in our HNSW graph.
3. The WITHSCORES option, in the supported commands, return a value that is directly related to the cosine similarity between the query and the items vectors, the interval of the similarity is simply rescaled from the -1, 1 original range to 0, 1, otherwise the metric is identical.

Clients, latency and bandwidth usage Copy Chapter

During Vector Sets testing, we discovered that often clients introduce considerable latecy and CPU usage (in the client side, not in Redis) for two main reasons:

1. Often the serialization to VALUES ... list of floats ... can be very slow.
2. The vector payload of floats represented as strings is very large, resulting in high bandwidth usage and latency, compared to other Redis commands.

Switching from VALUES to FP32 as a method for transmitting vectors may easily provide 10-20x speedups.

Implementation details

Vector sets are based on the hnsw.c implementation of the HNSW data structure with extensions for speed and functionality.

The main features are:

• Proper nodes deletion with relinking.
• 8 bits and binary quantization.
• Threaded queries.
• Filtered search with predicate callback.

File: src/commands/README.md Copy Chapter

This directory contains JSON files, one for each of Redis commands.

Each JSON contains all the information about the command itself, but these JSON files are not to be used directly!
Any third party who needs access to command information must get it from COMMAND INFO and COMMAND DOCS.
The output can be extracted in a JSON format by using redis-cli --json, in the same manner as in utils/generate-commands-json.py.

The JSON files are used to generate commands.def (and https://github.com/redis/docs/blob/main/data/commands.json) in Redis, and
despite looking similar to the output of COMMAND there are some fields and flags that are implicitly populated, and that's the
reason one shouldn't rely on the raw files.

The structure of each JSON is somewhat documented in https://redis.io/commands/command-docs/ and https://redis.io/commands/command/

The reply_schema section is a standard JSON Schema (see https://json-schema.org/) that describes the reply of each command.
It is designed to someday be used to auto-generate code in client libraries, but is not yet mature and is not exposed externally.

2. Official Technical Reference & Guides (redis/docs) Copy Chapter

Redis Docs

Note: if you are an AI agent, please see AI_AGENT_DEVELOPER_GUIDE.md for more information.

PRs are merged to the main branch of this repo. Merging to main automatically builds and publishes your changes to the live docs site (under /latest) — there is no separate manual publish step, so allow a little time for the build and CDN cache to refresh after your PR is merged.
Before your PR is merged, you can preview your changes on a per-branch staging URL; the docs team's bot posts the staging links as a comment on your PR.

Site template files and folders Copy Chapter

• /archetypes: A Markdown file needs to have some front matter. An archetype defines which front matter is used when using hugo new content. Right now, the only supported archetype is the default one. Note: We might want to add additional archetypes in the future because most of our pages contain additional meta data properties like linkTitle.
• /content: This folder contains the markdown files. We will have the subfolders like /develop, /integrate, and /operate
• /assets: CSS files, site-wide icons, and images.
• /data: Data files that are accessed by Hugo and rendered with the help of short codes or partials.
• /layouts/partials: HTML templates that are used across sites. Examples are TOCs, breadcrumbs, or headers.
• /layouts/$type: Each page type has at least the following templates to implement single.html and list.html. The single template is used to render a discrete page. The list template is used to render a collection of related pages (e.g., all sub-pages).
• /layouts/home.html: The home page of the site, that is, the page that is displayed when you open the root path.
• /layouts/404.html: The default 404 page.
• /layouts/shortcodes HTML template snippets that you want to leverage within Markdown files. Examples are tabbed code examples or notification blocks.
• /public: This is the folder that is generated by Hugo. This content needs to be published by site maintainers.
• /static: Any static files that need to be accessed by the site, e.g., CSS or JavaScript.
• /package.json: Node.js dependencies. Tailwind, for example, is installed via the Node package manager (npm).
• /config.toml: Hugo's site configuration, like the root path and menu items. Hugo can access configuration elements when rendering the site. So you can define custom configuration settings here.
• /Makefile: We use make to wrap some Hugo commands and to add additional build steps.
• /tailwind.config.js: This is the Tailwind CSS framwork's configuration file.
• /postcss.config.js: Needed to make Tailwind statically accessible to the site.

Requirements Copy Chapter

You will need the following tools to build the site locally:

• python3: Python >= 3.8.
• node and npm: Node.js >= 19.7.0, and the Node.js package manager >= 9.5.0.
• hugo: Hugo site generator, v0.143.0; the extended edition.
• make: To run the makefile.
• git: To manage the source code of this repo.

Build script and data files Copy Chapter

The site can be built using the command make all. Here is what's executed:

1. Clean the current folder by deleting the generated output of a previous build.
2. Install the necessary Node.js and Python dependencies.
3. Build the components that were fetched from external Github repositories (e.g., client examples). This fetches source code files and generates data files.
4. Execute the hugo command to generate the HTML pages. The build result can be found within the folder /public.

The command make serve performs the same steps, but serves the web pages on the local host for development purposes.

The Makefile contains details about the executed commands.

N.B. once you've built the site once, you can continuing working without rebuilding every time you make a change.
Use hugo serve to serve the docs locally.
The only time you really need to rebuild the site locally is when new code samples need to be pulled from the client sites.

Build pipeline Copy Chapter

The build pipeline that is defined within .github/workflows/main.yml builds the documentation and uploads it into a GCS bucket:

1. Install Hugo
2. Check the branch out
3. Perform the build as stated before by using make all
4. Install the Google Cloud CLI
5. Authenticate to the GCS bucket
6. Validate the branch name
7. Sync the branch with a GCS folder

Hugo specifics Copy Chapter

Relative links Copy Section

We are using the following syntax for Hugo relrefs:

[Link title]({{< relref "link relative to the site's base url" >}})

Here is an example:

[Data structure store]({{< relref "/develop/get-started/data-store" >}})

It's strongly advised to use relref because it provides the following advantages:

1. Links are checked at build time. Any broken references within the site are reported, stopping the build.
2. References are prefixed with the site's base URL, which means that they work in builds with a different base URL.

The following needs to be taken into account when using relref: The reference /develop/get-started/data-store and /develop/get-started/data-store/ aren't the same. You must use the trailing slash if the referenced article is an _index.md file within a folder (e.g., .../data-store/ for .../data-store/_index.md). Otherwise, you should not use the trailing slash (e.g., .../get-started/data-store.md).

Images Copy Section

The image shortcode doesn't need to be closed anymore. Here is an example;

{{< image filename="/images/rc/icon-database-update-status-pending.png" alt="Database update" >}}

The filename property value can be any file name path which is relative to the site's base path.

We added a new property class which allows you to override the CSS class of the image. Images have by default a block display in Tailwind. You can change this by setting the class property to inline. The following example shows two images that are in a single line:

{{< image filename="/images/rc/icon-database-update-status-pending.png#no-click" alt="Pending database status" class="inline" >}} &nbsp; {{< image filename="/images/rc/icon-database-status-active.png#no-click" alt="Active database status" class="inline" >}}

Templating Copy Section

Variables are scoped in the context of their block. Overriding a variable within an if block doesn't change the variable value as soon as you leave that block. This is a specific limitation of the Go templating language.

The following will give you the / outside of the condition block if the path was initially set to /:

{{ $path := $parsed.Path }}
{{ if (eq $path "/") }}
	{{ $path = "" }}
{{ end }}

{{- $path -}}

The solution is to use a scratchpad for storing the 'global' state:

{{ $path := $parsed.Path }}
{{ if (eq $path "/") }}
	{{ .Scratch.Set "path" ""}}
{{ else }}
    {{ .Scratch.Set "path" $path}}
{{ end }}

{{- .Scratch.Get "path" -}}

Child pages Copy Section

The redis.io template adds a list of links to child pages at the bottom of each section page. You can prevent this by adding the following front matter property:

hideListLinks: true

Tailwind specifics Copy Chapter

Fonts Copy Section

The fonts are located in the folder static/fonts. This folder contains typically file of the formats ttf, woff, or woff2. You can perform the following steps to add a font:

• Copy your font files to the fonts folder static/fonts.

• Ensure that the font is loaded by adding it to the partial layouts/partials/fonts.html. This partial defines the font using a style sheet and ensures that the font resources are pre-loaded.

  Here is an example that shows how to define the font face 'Geist Mono' with a regular font weight:

  text Copy

  @font-face {
      font-family: 'Geist Mono';
      src: url('{{ $basePath }}/fonts/GeistMono-Regular.woff2') format('woff2');
      font-weight: 400;
      font-style: normal;
      font-display: swap;
  }

• Make your font available via Tailwind by adding it to the tailwind.config.js file.

  text Copy

  module.exports = {
    content: ["content/**/*.md", "layouts/**/*.html", "./tailwindcss.whitelist.txt"],
    theme: {
      extend: {
        fontFamily: {
                                  sans: [ 'Space Grotesk', ...defaultTheme.fontFamily.sans ],
                                  mono: [ 'Space Mono', 'SF Mono', ...defaultTheme.fontFamily.mono ],
                                  monogeist: ['Geist Mono', ...defaultTheme.fontFamily.mono ],
        },
        ...

• Use a font-* CSS class within HTML or your style sheets, whereby the * needs to be substituted with one of the Tailwind fonts that you defined in the Tailwind configuration file (tailwind.config.js).

  Here is an example how to use font-monogeist in the global CSS file assets/css/index.css to change the font of code blocks:

  text Copy

  .prose pre > code {
    @apply bg-none font-monogeist;
  }

Troubleshooting Copy Chapter

NIL pointer error when rendering a section Copy Section

You might sometimes see the following error message:

nil pointer evaluating page.Page.Params

This happened for us when Hugo executed the bannerText logic, which inspected all parent pages by checking if they have a banner text set. The problem was that one of the parent folders was missing an _index.md file, which meant the folder couldn't be identified as a page. The solution was to add an _index.md file to the folder that didn't have it yet.