ossec-hids Documentation

Technical Documentation: ossec/ossec-hids

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`)

OSSEC v4.4.0

Information about OSSEC

OSSEC is a full platform to monitor and control your systems. It mixes together
all the aspects of HIDS (host-based intrusion detection), log monitoring and
SIM/SIEM together in a simple, powerful and open source solution.

Visit our website for the latest information. www.ossec.net

Current Releases

The current stable releases are available on the ossec website.

Releases can be downloaded from: Downloads
Release documentation is available at: docs

Development

The development version is hosted on GitHub and just a simple git clone away.

Screenshots

File Integrity Monitoring

Attack Detection

Help / Support

Join us on slack, ossec.slack.com: Invites to [email protected]

Join us on Discord: https://discord.gg/BXzM75Xzq7

Credits and Thanks

OSSEC comes with a modified version of zlib and a small part
of openssl (sha1 and blowfish libraries)
This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/)
This product includes cryptographic software written by Eric
Young ([email protected])
This product include software developed by the zlib project
(Jean-loup Gailly and Mark Adler)
This product include software developed by the cJSON project
(Dave Gamble)
Atomicorp hosting the annual OSSEC conference. Presentations for the 2019 conference can be found at https://www.atomicorp.com/ossec-con2019/

License

OSSEC is licensed under the GNU General Public License version 2 (GPLv2).

Overview (`.oss-scanner/README.md`)

Anthropic OSS Scanner support

Files in this directory enroll OSSEC HIDS with
Anthropic’s OSS Scanner:

File	Purpose
`Dockerfile`	Offline-audit image: deps, DEBUG server + agent builds, unit tests
`threat_model.md`	Scope, severity rubric, and exercise hints for the scanner

Enrollment (separate PR)

Core maintainers enroll by opening a PR against
anthropics/oss-scanner that adds
projects/ossec-hids/project.yaml (start from their templates/project.yaml).

Suggested project.yaml once this branch is on ossec/ossec-hids main
(or pin a tag/branch with #…):

yaml

repo: https://github.com/ossec/ossec-hids#main
primary_contact: [email protected]
auto_ccs:
  - [email protected]
homepage: https://www.ossec.net
disabled: false
dockerfile: .oss-scanner/Dockerfile
threat_model: .oss-scanner/threat_model.md

Before submitting that PR:

Merge this .oss-scanner/ support into the scanned branch.
Locally clone anthropics/oss-scanner, add the yaml above, then run
tools/validate.py and tools/check ossec-hids (needs Docker + PyYAML).

Local smoke-test of the Dockerfile

From the OSSEC repo root:

bash

docker build -f .oss-scanner/Dockerfile -t ossec-oss-scanner .
docker run --rm --network=none -it ossec-oss-scanner bash
# inside: ls src/ossec-analysisd .oss-scanner/bin/server .oss-scanner/bin/agent

Overview (`.github/workflows/README.md`)

GitHub Actions workflows

codeql.yml – CodeQL analysis (push/PR to master, weekly).
make-multi-platform.yml – Build with Make on Rocky Linux 9 (server/agent) and Windows agent cross-compile. Based on the multi-platform CI idea from PR #2158 by @cmac9203; adapted for this project’s Make build.

Overview (`contrib/selinux/README.md`)

OSSEC SELinux module

Optional SELinux policy for confining the OSSEC agent under
/var/ossec. Server/manager confinement is not covered yet (see #1948).

Quick fix for logrotate AVCs (#1948)

If SELinux blocks logrotate on /var/ossec/logs/* and you are not
using this module, label the logs like other system logs:

bash

semanage fcontext -a -t var_log_t '/var/ossec/logs(/.*)?'
restorecon -Rv /var/ossec/logs

Do not use logrotate_exec_t / logrotate_tmp_t / similar — those
are for logrotate's own files, not for application logs.

RPM packages apply the var_log_t mapping automatically when this
module is not loaded.

Installing this module

Requires checkpolicy, policycoreutils, and selinux-policy-devel
(package names vary by distro). On Fedora/RHEL also install
policycoreutils-python-utils for semanage.

bash

cd contrib/selinux/ossec_agent
make
semodule -i ossec_agent.pp
# or: semodule -i ../ossec_agent.pp.bz2
restorecon -Rv /var/ossec
systemctl restart ossec   # or ossec-hids / ossec-hids-agent
ps -eZ | grep ossec

If semodule -i ossec_agent.pp.bz2 fails looking for
/usr/libexec/selinux/hll/bz2, decompress first:

bash

bunzip2 -k ossec_agent.pp.bz2
semodule -i ossec_agent.pp

After the module is installed, logs use ossec_log_t (see .fc).
Remove any conflicting local mapping first:

bash

semanage fcontext -d '/var/ossec/logs(/.*)?' 2>/dev/null || true
restorecon -Rv /var/ossec/logs

Custom install prefix: adjust paths in ossec_agent.fc and re-run
make / restorecon.

Building

bash

cd contrib/selinux/ossec_agent && make

Ship/update the compressed module from the repo root with
make -C contrib/selinux/ossec_agent pp-bz2.

Bug reports & contribution

Original author: [email protected]

Overview (`src/external/cJSON/README.md`)

cJSON

Ultralightweight JSON parser in ANSI C.

License
Usage

License

MIT License

Copyright (c) 2009-2017 Dave Gamble and cJSON contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Usage

Welcome to cJSON.

cJSON aims to be the dumbest possible parser that you can get your job done with.
It's a single file of C, and a single header file.

JSON is described best here: http://www.json.org/
It's like XML, but fat-free. You use it to move data around, store things, or just
generally represent your program's state.

As a library, cJSON exists to take away as much legwork as it can, but not get in your way.
As a point of pragmatism (i.e. ignoring the truth), I'm going to say that you can use it
in one of two modes: Auto and Manual. Let's have a quick run-through.

I lifted some JSON from this page: http://www.json.org/fatfree.html
That page inspired me to write cJSON, which is a parser that tries to share the same
philosophy as JSON itself. Simple, dumb, out of the way.

Building

There are several ways to incorporate cJSON into your project.

copying the source

Because the entire library is only one C file and one header file, you can just copy cJSON.h and cJSON.c to your projects source and start using it.

cJSON is written in ANSI C (C89) in order to support as many platforms and compilers as possible.

CMake

With CMake, cJSON supports a full blown build system. This way you get the most features. CMake with an equal or higher version than 2.8.5 is supported. With CMake it is recommended to do an out of tree build, meaning the compiled files are put in a directory separate from the source files. So in order to build cJSON with CMake on a Unix platform, make a build directory and run CMake inside it.

text

mkdir build
cd build
cmake ..

This will create a Makefile and a bunch of other files. You can then compile it:

text

make

And install it with make install if you want. By default it installs the headers /usr/local/include/cjson and the libraries to /usr/local/lib. It also installs files for pkg-config to make it easier to detect and use an existing installation of CMake. And it installs CMake config files, that can be used by other CMake based projects to discover the library.

You can change the build process with a list of different options that you can pass to CMake. Turn them on with On and off with Off:

-DENABLE_CJSON_TEST=On: Enable building the tests. (on by default)
-DENABLE_CJSON_UTILS=On: Enable building cJSON_Utils. (off by default)
-DENABLE_TARGET_EXPORT=On: Enable the export of CMake targets. Turn off if it makes problems. (on by default)
-DENABLE_CUSTOM_COMPILER_FLAGS=On: Enable custom compiler flags (currently for Clang, GCC and MSVC). Turn off if it makes problems. (on by default)
-DENABLE_VALGRIND=On: Run tests with valgrind. (off by default)
-DENABLE_SANITIZERS=On: Compile cJSON with AddressSanitizer and UndefinedBehaviorSanitizer enabled (if possible). (off by default)
-DENABLE_SAFE_STACK: Enable the SafeStack instrumentation pass. Currently only works with the Clang compiler. (off by default)
-DBUILD_SHARED_LIBS=On: Build the shared libraries. (on by default)
-DBUILD_SHARED_AND_STATIC_LIBS=On: Build both shared and static libraries. (off by default)
-DCMAKE_INSTALL_PREFIX=/usr: Set a prefix for the installation.
-DENABLE_LOCALES=On: Enable the usage of localeconv method. ( on by default )
-DCJSON_OVERRIDE_BUILD_SHARED_LIBS=On: Enable overriding the value of BUILD_SHARED_LIBS with -DCJSON_BUILD_SHARED_LIBS.
-DENABLE_CJSON_VERSION_SO: Enable cJSON so version. ( on by default )

If you are packaging cJSON for a distribution of Linux, you would probably take these steps for example:

text

mkdir build
cd build
cmake .. -DENABLE_CJSON_UTILS=On -DENABLE_CJSON_TEST=Off -DCMAKE_INSTALL_PREFIX=/usr
make
make DESTDIR=$pkgdir install

On Windows CMake is usually used to create a Visual Studio solution file by running it inside the Developer Command Prompt for Visual Studio, for exact steps follow the official documentation from CMake and Microsoft and use the online search engine of your choice. The descriptions of the the options above still generally apply, although not all of them work on Windows.

Makefile

NOTE: This Method is deprecated. Use CMake if at all possible. Makefile support is limited to fixing bugs.

If you don't have CMake available, but still have GNU make. You can use the makefile to build cJSON:

Run this command in the directory with the source code and it will automatically compile static and shared libraries and a little test program (not the full test suite).

text

make all

If you want, you can install the compiled library to your system using make install. By default it will install the headers in /usr/local/include/cjson and the libraries in /usr/local/lib. But you can change this behavior by setting the PREFIX and DESTDIR variables: make PREFIX=/usr DESTDIR=temp install. And uninstall them with: make PREFIX=/usr DESTDIR=temp uninstall.

Meson

To make cjson work in a project using meson, the libcjson dependency has to be included:

meson

project('c-json-example', 'c')

cjson = dependency('libcjson')

example = executable(
    'example',
    'example.c',
    dependencies: [cjson],
)

Vcpkg

You can download and install cJSON using the vcpkg dependency manager:

text

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
vcpkg install cjson

The cJSON port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Including cJSON

If you installed it via CMake or the Makefile, you can include cJSON like this:

#include <cjson/cJSON.h>

Data Structure

cJSON represents JSON data using the cJSON struct data type:

/* The cJSON structure: */
typedef struct cJSON
{
    struct cJSON *next;
    struct cJSON *prev;
    struct cJSON *child;
    int type;
    char *valuestring;
    /* writing to valueint is DEPRECATED, use cJSON_SetNumberValue instead */
    int valueint;
    double valuedouble;
    char *string;
} cJSON;

An item of this type represents a JSON value. The type is stored in type as a bit-flag (this means that you cannot find out the type by just comparing the value of type).

To check the type of an item, use the corresponding cJSON_Is... function. It does a NULL check followed by a type check and returns a boolean value if the item is of this type.

The type can be one of the following:

cJSON_Invalid (check with cJSON_IsInvalid): Represents an invalid item that doesn't contain any value. You automatically have this type if you set the item to all zero bytes.
cJSON_False (check with cJSON_IsFalse): Represents a false boolean value. You can also check for boolean values in general with cJSON_IsBool.
cJSON_True (check with cJSON_IsTrue): Represents a true boolean value. You can also check for boolean values in general with cJSON_IsBool.
cJSON_NULL (check with cJSON_IsNull): Represents a null value.
cJSON_Number (check with cJSON_IsNumber): Represents a number value. The value is stored as a double in valuedouble and also in valueint. If the number is outside of the range of an integer, INT_MAX or INT_MIN are used for valueint.
cJSON_String (check with cJSON_IsString): Represents a string value. It is stored in the form of a zero terminated string in valuestring.
cJSON_Array (check with cJSON_IsArray): Represent an array value. This is implemented by pointing child to a linked list of cJSON items that represent the values in the array. The elements are linked together using next and prev, where the first element has prev.next == NULL and the last element next == NULL.
cJSON_Object (check with cJSON_IsObject): Represents an object value. Objects are stored same way as an array, the only difference is that the items in the object store their keys in string.
cJSON_Raw (check with cJSON_IsRaw): Represents any kind of JSON that is stored as a zero terminated array of characters in valuestring. This can be used, for example, to avoid printing the same static JSON over and over again to save performance. cJSON will never create this type when parsing. Also note that cJSON doesn't check if it is valid JSON.

Additionally there are the following two flags:

cJSON_IsReference: Specifies that the item that child points to and/or valuestring is not owned by this item, it is only a reference. So cJSON_Delete and other functions will only deallocate this item, not its child/valuestring.
cJSON_StringIsConst: This means that string points to a constant string. This means that cJSON_Delete and other functions will not try to deallocate string.

Working with the data structure

For every value type there is a cJSON_Create... function that can be used to create an item of that type.
All of these will allocate a cJSON struct that can later be deleted with cJSON_Delete.
Note that you have to delete them at some point, otherwise you will get a memory leak.
Important: If you have added an item to an array or an object already, you mustn't delete it with cJSON_Delete. Adding it to an array or object transfers its ownership so that when that array or object is deleted,
it gets deleted as well. You also could use cJSON_SetValuestring to change a cJSON_String's valuestring, and you needn't to free the previous valuestring manually.

Basic types

null is created with cJSON_CreateNull
booleans are created with cJSON_CreateTrue, cJSON_CreateFalse or cJSON_CreateBool
numbers are created with cJSON_CreateNumber. This will set both valuedouble and valueint. If the number is outside of the range of an integer, INT_MAX or INT_MIN are used for valueint
strings are created with cJSON_CreateString (copies the string) or with cJSON_CreateStringReference (directly points to the string. This means that valuestring won't be deleted by cJSON_Delete and you are responsible for its lifetime, useful for constants)

Arrays

You can create an empty array with cJSON_CreateArray. cJSON_CreateArrayReference can be used to create an array that doesn't "own" its content, so its content doesn't get deleted by cJSON_Delete.

To add items to an array, use cJSON_AddItemToArray to append items to the end.
Using cJSON_AddItemReferenceToArray an element can be added as a reference to another item, array or string. This means that cJSON_Delete will not delete that items child or valuestring properties, so no double frees are occurring if they are already used elsewhere.
To insert items in the middle, use cJSON_InsertItemInArray. It will insert an item at the given 0 based index and shift all the existing items to the right.

If you want to take an item out of an array at a given index and continue using it, use cJSON_DetachItemFromArray, it will return the detached item, so be sure to assign it to a pointer, otherwise you will have a memory leak.

Deleting items is done with cJSON_DeleteItemFromArray. It works like cJSON_DetachItemFromArray, but deletes the detached item via cJSON_Delete.

You can also replace an item in an array in place. Either with cJSON_ReplaceItemInArray using an index or with cJSON_ReplaceItemViaPointer given a pointer to an element. cJSON_ReplaceItemViaPointer will return 0 if it fails. What this does internally is to detach the old item, delete it and insert the new item in its place.

To get the size of an array, use cJSON_GetArraySize. Use cJSON_GetArrayItem to get an element at a given index.

Because an array is stored as a linked list, iterating it via index is inefficient (O(n²)), so you can iterate over an array using the cJSON_ArrayForEach macro in O(n) time complexity.

Objects

You can create an empty object with cJSON_CreateObject. cJSON_CreateObjectReference can be used to create an object that doesn't "own" its content, so its content doesn't get deleted by cJSON_Delete.

To add items to an object, use cJSON_AddItemToObject. Use cJSON_AddItemToObjectCS to add an item to an object with a name that is a constant or reference (key of the item, string in the cJSON struct), so that it doesn't get freed by cJSON_Delete.
Using cJSON_AddItemReferenceToArray an element can be added as a reference to another object, array or string. This means that cJSON_Delete will not delete that items child or valuestring properties, so no double frees are occurring if they are already used elsewhere.

If you want to take an item out of an object, use cJSON_DetachItemFromObjectCaseSensitive, it will return the detached item, so be sure to assign it to a pointer, otherwise you will have a memory leak.

Deleting items is done with cJSON_DeleteItemFromObjectCaseSensitive. It works like cJSON_DetachItemFromObjectCaseSensitive followed by cJSON_Delete.

You can also replace an item in an object in place. Either with cJSON_ReplaceItemInObjectCaseSensitive using a key or with cJSON_ReplaceItemViaPointer given a pointer to an element. cJSON_ReplaceItemViaPointer will return 0 if it fails. What this does internally is to detach the old item, delete it and insert the new item in its place.

To get the size of an object, you can use cJSON_GetArraySize, this works because internally objects are stored as arrays.

If you want to access an item in an object, use cJSON_GetObjectItemCaseSensitive.

To iterate over an object, you can use the cJSON_ArrayForEach macro the same way as for arrays.

cJSON also provides convenient helper functions for quickly creating a new item and adding it to an object, like cJSON_AddNullToObject. They return a pointer to the new item or NULL if they failed.

Parsing JSON

Given some JSON in a zero terminated string, you can parse it with cJSON_Parse.

cJSON *json = cJSON_Parse(string);

Given some JSON in a string (whether zero terminated or not), you can parse it with cJSON_ParseWithLength.

cJSON *json = cJSON_ParseWithLength(string, buffer_length);

It will parse the JSON and allocate a tree of cJSON items that represents it. Once it returns, you are fully responsible for deallocating it after use with cJSON_Delete.

The allocator used by cJSON_Parse is malloc and free by default but can be changed (globally) with cJSON_InitHooks.

If an error occurs a pointer to the position of the error in the input string can be accessed using cJSON_GetErrorPtr. Note though that this can produce race conditions in multithreading scenarios, in that case it is better to use cJSON_ParseWithOpts with return_parse_end.
By default, characters in the input string that follow the parsed JSON will not be considered as an error.

If you want more options, use cJSON_ParseWithOpts(const char *value, const char **return_parse_end, cJSON_bool require_null_terminated).
return_parse_end returns a pointer to the end of the JSON in the input string or the position that an error occurs at (thereby replacing cJSON_GetErrorPtr in a thread safe way). require_null_terminated, if set to 1 will make it an error if the input string contains data after the JSON.

If you want more options giving buffer length, use cJSON_ParseWithLengthOpts(const char *value, size_t buffer_length, const char **return_parse_end, cJSON_bool require_null_terminated).

Printing JSON

Given a tree of cJSON items, you can print them as a string using cJSON_Print.

char *string = cJSON_Print(json);

It will allocate a string and print a JSON representation of the tree into it. Once it returns, you are fully responsible for deallocating it after use with your allocator. (usually free, depends on what has been set with cJSON_InitHooks).

cJSON_Print will print with whitespace for formatting. If you want to print without formatting, use cJSON_PrintUnformatted.

If you have a rough idea of how big your resulting string will be, you can use cJSON_PrintBuffered(const cJSON *item, int prebuffer, cJSON_bool fmt). fmt is a boolean to turn formatting with whitespace on and off. prebuffer specifies the first buffer size to use for printing. cJSON_Print currently uses 256 bytes for its first buffer size. Once printing runs out of space, a new buffer is allocated and the old gets copied over before printing is continued.

These dynamic buffer allocations can be completely avoided by using cJSON_PrintPreallocated(cJSON *item, char *buffer, const int length, const cJSON_bool format). It takes a buffer to a pointer to print to and its length. If the length is reached, printing will fail and it returns 0. In case of success, 1 is returned. Note that you should provide 5 bytes more than is actually needed, because cJSON is not 100% accurate in estimating if the provided memory is enough.

Example

In this example we want to build and parse the following JSON:

json

{
    "name": "Awesome 4K",
    "resolutions": [
        {
            "width": 1280,
            "height": 720
        },
        {
            "width": 1920,
            "height": 1080
        },
        {
            "width": 3840,
            "height": 2160
        }
    ]
}

Printing

Let's build the above JSON and print it to a string:

text

/* Detailed source-code truncated for AI context efficiency. */

Alternatively we can use the cJSON_Add...ToObject helper functions to make our lives a little easier:

//NOTE: Returns a heap allocated string, you are required to free it after use.
char *create_monitor_with_helpers(void)
{
    const unsigned int resolution_numbers[3][2] = {
        {1280, 720},
        {1920, 1080},
        {3840, 2160}
    };
    char *string = NULL;
    cJSON *resolutions = NULL;
    size_t index = 0;

    cJSON *monitor = cJSON_CreateObject();

    if (cJSON_AddStringToObject(monitor, "name", "Awesome 4K") == NULL)
    {
        goto end;
    }

    resolutions = cJSON_AddArrayToObject(monitor, "resolutions");
    if (resolutions == NULL)
    {
        goto end;
    }

    for (index = 0; index < (sizeof(resolution_numbers) / (2 * sizeof(int))); ++index)
    {
        cJSON *resolution = cJSON_CreateObject();

        if (cJSON_AddNumberToObject(resolution, "width", resolution_numbers[index][0]) == NULL)
        {
            goto end;
        }

        if (cJSON_AddNumberToObject(resolution, "height", resolution_numbers[index][1]) == NULL)
        {
            goto end;
        }

        cJSON_AddItemToArray(resolutions, resolution);
    }

    string = cJSON_Print(monitor);
    if (string == NULL)
    {
        fprintf(stderr, "Failed to print monitor.\n");
    }

end:
    cJSON_Delete(monitor);
    return string;
}

Parsing

In this example we will parse a JSON in the above format and check if the monitor supports a Full HD resolution while printing some diagnostic output:

/* return 1 if the monitor supports full hd, 0 otherwise */
int supports_full_hd(const char * const monitor)
{
    const cJSON *resolution = NULL;
    const cJSON *resolutions = NULL;
    const cJSON *name = NULL;
    int status = 0;
    cJSON *monitor_json = cJSON_Parse(monitor);
    if (monitor_json == NULL)
    {
        const char *error_ptr = cJSON_GetErrorPtr();
        if (error_ptr != NULL)
        {
            fprintf(stderr, "Error before: %s\n", error_ptr);
        }
        status = 0;
        goto end;
    }

    name = cJSON_GetObjectItemCaseSensitive(monitor_json, "name");
    if (cJSON_IsString(name) && (name->valuestring != NULL))
    {
        printf("Checking monitor \"%s\"\n", name->valuestring);
    }

    resolutions = cJSON_GetObjectItemCaseSensitive(monitor_json, "resolutions");
    cJSON_ArrayForEach(resolution, resolutions)
    {
        cJSON *width = cJSON_GetObjectItemCaseSensitive(resolution, "width");
        cJSON *height = cJSON_GetObjectItemCaseSensitive(resolution, "height");

        if (!cJSON_IsNumber(width) || !cJSON_IsNumber(height))
        {
            status = 0;
            goto end;
        }

        if ((width->valuedouble == 1920) && (height->valuedouble == 1080))
        {
            status = 1;
            goto end;
        }
    }

end:
    cJSON_Delete(monitor_json);
    return status;
}

Note that there are no NULL checks except for the result of cJSON_Parse because cJSON_GetObjectItemCaseSensitive checks for NULL inputs already, so a NULL value is just propagated and cJSON_IsNumber and cJSON_IsString return 0 if the input is NULL.

Caveats

Zero Character

cJSON doesn't support strings that contain the zero character '\0' or \u0000. This is impossible with the current API because strings are zero terminated.

Character Encoding

cJSON only supports UTF-8 encoded input. In most cases it doesn't reject invalid UTF-8 as input though, it just propagates it through as is. As long as the input doesn't contain invalid UTF-8, the output will always be valid UTF-8.

C Standard

cJSON is written in ANSI C (or C89, C90). If your compiler or C library doesn't follow this standard, correct behavior is not guaranteed.

NOTE: ANSI C is not C++ therefore it shouldn't be compiled with a C++ compiler. You can compile it with a C compiler and link it with your C++ code however. Although compiling with a C++ compiler might work, correct behavior is not guaranteed.

Floating Point Numbers

cJSON does not officially support any double implementations other than IEEE754 double precision floating point numbers. It might still work with other implementations but bugs with these will be considered invalid.

The maximum length of a floating point literal that cJSON supports is currently 63 characters.

Deep Nesting Of Arrays And Objects

cJSON doesn't support arrays and objects that are nested too deeply because this would result in a stack overflow. To prevent this cJSON limits the depth to CJSON_NESTING_LIMIT which is 1000 by default but can be changed at compile time.

Thread Safety

In general cJSON is not thread safe.

However it is thread safe under the following conditions:

cJSON_GetErrorPtr is never used (the return_parse_end parameter of cJSON_ParseWithOpts can be used instead)
cJSON_InitHooks is only ever called before using cJSON in any threads.
setlocale is never called before all calls to cJSON functions have returned.

Case Sensitivity

When cJSON was originally created, it didn't follow the JSON standard and didn't make a distinction between uppercase and lowercase letters. If you want the correct, standard compliant, behavior, you need to use the CaseSensitive functions where available.

Duplicate Object Members

cJSON supports parsing and printing JSON that contains objects that have multiple members with the same name. cJSON_GetObjectItemCaseSensitive however will always only return the first one.

Enjoy cJSON!

Dave Gamble (original author)
Max Bruckner and Alan Wang (current maintainer)
and the other cJSON contributors

Overview (`src/tests/fuzz/README.md`)

OSSEC fuzz harnesses (OSS-Fuzz)

First slice for issue #1789:
fuzz_os_xml exercises OS_ReadXML and node walks in os_xml.

ClusterFuzz findings from OSS-Fuzz (security and non-exploitable memory
bugs) are treated as bugs to fix in this repository.

Local build (clang + libFuzzer + ASan)

bash

cd src
make clean-internals
make fuzz_os_xml CC=clang USE_HARDENING=no \
  CFLAGS='-O1 -g -fPIC -fsanitize=fuzzer,address -fno-omit-frame-pointer'

OSS-Fuzz sets LIB_FUZZING_ENGINE and sanitizer flags in the
environment; the same target links against that engine when present.

Run over the seed corpus

bash

./fuzz_os_xml -runs=5000 tests/fuzz/corpus_os_xml

Broken or invalid XML must return from the harness without crashing
(failed parse is a normal input).

Overview (`src/tests/regressions/README.md`)

Analysisd / pipeline regression tests

From `src/` after building the server target:

make TARGET=server -j$(nproc)

make regression

Or directly:

make -f tests/regressions/Makefile

make -f tests/regressions/Makefile check

Shared queue/thread unit tests (automated):

make -C shared/tests check

Interactive helpers (prime/hash/merge/ip) are optional:

make -C shared/tests legacy_helpers

Covered binaries (make check):

- issue_1079_last_events_uaf frequency last_events ownership

- issue_correlation_agent_shard location EventList isolation +

if_matched_sid cross-location

- issue_1748_sid_list_crash sid_prev_matched overflow + free callback

- issue_group_list_uaf Free_Eventinfo unlinks its group_prev_matched nodes

- issue_prev_matched_race if_matched_sid list lock across match threads

- issue_1274_auth_ipv6_key auth IPv6 key handling

- issue_2065_sender_counter sender_counter isolated from agent 0 rids

- issue_1611_local_ip_bind remoted local_ip / ipv6 bind family

- issue_1668_agent_hostname agent name vs device hostname split

- issue_1668_ossecalert_hostname OSSECAlert decoder vs owned hostname (double free)

- jsonout_default jsonout_output on when the XML tag is omitted

- issue_1814_control_chars strip controls, keep UTF-8

- issue_cleanevent_date_oob umlaut/Suricata date-probe bounds

- issue_1953_xml_recursion OS_ReadXML recursion limit

- issue_2100_authd_ssl authd SSL context flags

- getalert_empty_filename GetAlertData empty syscheck path

- checksum_short_message CheckSum rejects sub-digest lengths

- issue_empty_ciphertext ReadSecMSG rejects empty/oversized body

- issue_win_execd_cmd_injection WinExecd AR args safe for cmd.exe

- issue_logs_symlink_nofollow open_log_append refuses log symlinks

- issue_ar_location_injection AR hdr truncates location at "->"

- issue_ads_stream_overflow ADS BackupRead buffer fits NTFS names

Also invoked from `make regression` (own Makefiles):

- src/os_maild/tests mail list, SMS queue, shutdown gate, mail utils

- src/os_auth/tests auth connection drop

Additional scripts without unified Make targets (manual / historical):

issue_1817_uaf_fix.c, issue_1818_syscheck_uaf.c, issue_2056_repro_crash.c

Notes for new tests:

- Define TLS time globals with `__thread` (`c_time`, `__crt_hour`, `__crt_wday`)

so they link against analysisd objects that use thread-local clock state.

- Call `os_mutex_init(&rule->mutex, NULL)` on any RuleInfo used with Search_*.

- Do not stub `OS_GetLastEvent` when linking `eventinfo_list-live.o`.

Correlation / shard semantics (pipeline rule_matching_threads > 1):

- EventList history used by Search_LastEvents is per location-hashed shard.

- Frequency/context that must span multiple agents should use if_matched_sid

or if_matched_group (those lists remain global under prev_matched_mutex).

- Migration checklist: inventory multi-agent frequency rules → convert to

if_matched_sid/group → keep pure Search_LastEvents only for local correlation.

- Ops: events_received, events_dropped, alerts_dropped, archives_dropped*,

raw_input_queue_, fts_queue_, and *_delta fields in ossec-analysisd.state.

Raise queue sizes, lower EPS, or tune analysisd.*_push_wait_ms /

input_demux_threads / raw_input_queue_size — no sync fallback for mid-pipe

drops (alerts still sync-fall back).

- Rate-limited merror WARN "pipeline queue pressure" also fires under drop load.

Soak SLO / Nested replacement checklist (rule_matching_threads >= 2):

1. Build/install ossec-analysisd; refresh etc/internal_options.conf knobs.

2. ossec-control stop; kill leftover ossec-analysisd PIDs; confirm none remain.

3. ossec-control start; confirm exactly one analysisd PID and pipeline log line

(input_demux, shards); expect shard correlation reminder when shards > 1.

4. .state: events_received increasing; statistical/raw/fts queue_* present;

events_dropped_delta and events_dropped_shard_delta ~0 after settle;

alerts flowing; Check_Hour uses statistical writer when stats enabled.

5. Short burst should be absorbed by raw/demux wait_ms without sustained

mid-pipe deltas; sustained overload may increment drops (expected).

Wave 4 product planes (SCA/syscollector/winevt/dbsync/upgrade): deferred until

Waves 1–3 soak SLOs hold and product names a cutover plane.

Doc Active Response Internal (`doc/active-response-internal.txt`)

OSSEC HIDS 0.6
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

How the active response works internally:

Read active-response.txt for details on configuration

1 - The analysis server receives an event that matches the
active response policy.

2 - The analysis server verifies that all required fields
are provided with the event. It means that the analysis
server was able to decode the event and extract the
necessary information. One example is if it was able
to extract the IP address from the event to send to
the firewall to be blocked.

3 - If the active response policy specify that the action
must be executed locally on the AS, a message is sent
to the execd directly.

4 - If the active response policy specify that the action
must be executed remotely, a message is sent to the
"Active response forwarder" (remoted) to forward the
event to the specified agent.

Doc Active Response (`doc/active-response.txt`)

OSSEC HIDS v0.7
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

http://www.ossec.net/docs/docs/manual/ar/index.html

Doc Logs (`doc/logs.txt`)

OSSEC v0.9
Copyright (C) 2009 Trend Micro Inc.

OSSEC Logging

== Introduction ==

Ossec supports three types of logs. Alert logging, firewall
logging and event (archiving) logging.

Every message received is treated as an event.
Any log message, integrity report, system information will be treated
as such. Event logging is very expensive for the system because
it will archive every event. However, they can be usefull to get
the big picture if some attack happens.

Alert logging is the most useful one. An alert is generated when
an event is matched against one of the detection rules. In addition
to the logging, OSSEC can also generate e-mail notifications or
execute external commands for them.

== Event logging ==

Inside the OSSEC default log directory (by default /var/ossec/logs)
there is an entry for "archives" (/var/ossec/logs/archives). Inside this
directory, all events will be stored by date.
For example, all events received on May 22 of 2004, will be stored on:

/var/ossec/logs/archives/2004/May/events-22.log

After each day, a hash will be created for this specific day at

/var/ossec/logs/archives/2004/May/events-22.log.md5

This hash will be the hash of the file from the day 22 plus the hash
from the day 21.

The hash from the day 1, will be the hash from the day 31 (or 30 or 28)
from the previous month.

This will ensure that no log will be modified. Also, for this to happen,
all the logs (since the first day) will need to be modified.

== Alert logging ==

There will be a "alerts" directory on the OSSEC default logging directory.
It will be organized on the same way the event logging is. Please read
above to understand it.

Doc Manage Agents (`doc/manage_agents.txt`)

== How to add an agent without any keyboard input ==

By default, to add an agent from server side, you must provide your agent
information to manage_agents program, by using its interactive mode.
This is really tedious if you have many servers / agents to add. Luckily,
you can use following environment variables as responses

text

| variable name          | value   | description          |
 +------------------------+---------+----------------------+
 | OSSEC_ACTION           | A/a     | add an agent         |
 | OSSEC_AGENT_NAME       | string  | name of agent        |
 | OSSEC_AGENT_IP         | CIDR    | ip address of agent  |
 | OSSEC_AGENT_ID         | integer | max length = 8       |
 | OSSEC_AGENT_KEY        | string  | base64 format        | (*)
 | OSSEC_ACTION_CONFIRMED | y/Y/n/N | y -> confirmed       |

(*) OSSEC_AGENT_KEY is used only on agent (when key is being imported)

Please note that it's your duty to ensure that name, ip,... of agent are
valid. Otherwise, the program will fall back to interactive mode. In most
case, you should ensure that you new agent has an unique name/id. You can
simply know that by using manage_agents -l to list all known agents.

For more details, please refer to OSSEC document
https://www.ossec.net/docs/manual/agent/agent-management.html

== Notes ==

You can also use some tools (expect) to send strings to manage_agents,
instead of using the above environment variables.

Doc Manager (`doc/manager.txt`)

OSSEC v0.9
Copyright (C) 2009 Trend Micro Inc.

How do the server manager the agents.

-The server will open port 1514 (by default) and listen for
messages from the clients. Only the IP of the clients will be
allowed.

-Every 10 minutes, the client will send an status notification
to the server. This status message contain some information
about the agent system and information about the files it
has on the shared directory.

-The server will receive the status message, update the agent
status file and check if it has any file to be sent to the
agent. If it has, it will connect to the agent and send
the file.

-Every message will be encrypted.

Doc Nmap (`doc/nmap.txt`)

OSSEC
Copyright (C) 2009 Trend Micro Inc.

** Nmap correlation **

Ossec can read nmap grepable output files to use as a
correlation tool and also to alert based on host information
changes. Follow the step by step below on how to configure
ossec:

1- Add the nmap output file on ossec.conf (generally
at /var/ossec/etc/ossec.conf):

nmapg /var/log/nmap-out.log

2- If the file does not exist, touch it:

ossec-test# touch /var/log/nmap-out.log

3- Restart ossec:

ossec-test# /var/ossec/bin/ossec-control restart

4- Run your nmap scans (example scanning 192.168.2.0/24 network):

ossec-test# nmap --append_output -sU -sT -oG /var/log/nmap-out.log 192.168.2.0-255

*** Example of alert when a new host is found:

** Alert 1152058913.238: mail
2006 Jul 04 20:21:53 /var/log/nmap-out.log
Rule: 15 (level 8) -> 'New host information added.'
Src IP: (none)
User: (none)
Host: 192.168.2.10, open ports: 21(tcp) 22(tcp) 80(tcp) 113(tcp) 514(udp) 1514(udp) 4500(udp)

*** Example of alert when a new a host information is changed:

** Alert 1152058983.487: mail
2006 Jul 04 20:23:03 /var/log/nmap-out.log
Rule: 15 (level 8) -> 'Host information changed.'
Src IP: (none)
User: (none)
Host: 192.168.2.1, open ports: 54(udp) 8080(tcp) 161(udp) 520(udp) 1025(udp) 1900(udp)
Previously open ports: 53(udp) 80(tcp) 161(udp) 520(udp) 1025(udp) 1900(udp)

Doc Rootcheck (`doc/rootcheck.txt`)

Rootkit detection techniques used by the OSSEC HIDS
by Daniel B. Cid, [email protected]

Starting on version 0.4, the OSSEC HIDS will perform
rootkit detection on every system where the agent is
installed. The rootcheck (rootkit detection engine) will
be executed every X minutes (user specified --by default
every 2 hours) to detect any possible rootkit installed.
Used witht the log analysis and the integrity checking
engine, it will become a very powerful monitoring solution
(the OSSEC HIDS performs log analysis and integrity
checking since version 0.1).

Other feature included on version 0.4 is that the analysis
server will automatically forward the rootkit detection
signatures to the agents, reducing the administration
overhead for the system admin. The agents and server will
keep contact every 10 minutes and if the server is
updated with a new signature file, it will forward them
to all configured agents. Take a look at the management
documentation for more information.

The rootcheck will perform the following steps on the
system trying to find rootkits:

1- Read the rootkit_files.txt which contains a big database
of rootkits and files used by them. It will try to stats,
fopen and opendir each specified file. We use all these
system calls, because some kernel-level rootkits, hide
files from some system calls. The more system calls we
try, the better the detection. This method is more like
an anti-virus rule that needs to be updated constantly.
The chances of false-positives are small, but false
negatives can be produced by modifying the rootkits.

2- Read the rootkit_trojans.txt which contains a database
of signatures of files trojaned by rootkits. This
technique of modifying binaries with trojaned versions
was commonly used by most of the popular rootkits
available. This detection method will not find any
kernel level rootkit or any unknown rootkit.

3- Scan the /dev directory looking for anomalies. The /dev
should only have device files and the Makedev script.
A lot of rootkits use the /dev to hide files. This
technique can detect even non-public rootkits.

4- Scan the whole filesystem looking for unusual files and
permission problems. Files owned by root, with written
permission to others are very dangerous and the rootkit
detection will look for them. Suid files, hidden directories
and files will also be inspected.

5- Look for the presence of hidden processes. We use getsid()
and kill() to check if any pid is being used or not. If
the pid is being used, but "ps" can't see it, it is the
indication of kernel-level rootkit or a trojaned version
of "ps". We also verify the output of kill and getsid that
should be the same.

6- Look for the presence of hidden ports. We use bind() to
check every tcp and udp port on the system. If we can't
bind to the port (it's being used), but netstat does not
show it, we probably have a rootkit installed.

7- Scan all interfaces on the system and look for the ones
with "promisc" mode enabled. If the interface is in promiscuous
mode, the output of "ifconfig" should show that. If not,
we probably have a rootkit installed.

EOF

Doc Rule Ids (`doc/rule_ids.txt`)

ossec Rules ids.

Ossec official rules should be under some of these

assignments.

Local rules should go from 100000 to 120000.

Every rule will also have a revision attribute (if modified).

*default revision is 0 (when first added).

00000 - 00999 Internally reserved for ossec
01000 - 01999 General syslog
02100 - 02299 NFS
02300 - 02499 Xinetd
02500 - 02699 Access control
02700 - 02729 Mail/procmail
02800 - 02829 Smartd
02830 - 02859 Crond
02860 - 02899 Mount/Automount

03100 - 03299 Sendmail
03300 - 03499 Postfix
03500 - 03599 Spamd
03600 - 03699 Imapd
03700 - 03799 MailScanner

04100 - 04299 Generic Firewall
04300 - 04499 Cisco PIX Firewall
04500 - 04699 Netscreen Firewall

05100 - 05299 Kernels (Linux, Unix, etc)
05300 - 05399 Su
05400 - 05499 sudo
05500 - 05599 Pam unix
05600 - 05699 Telnetd
05700 - 05899 sshd
05900 - 05999 Adduser or user deletion.

07100 - 07199 Tripwire
07200 - 07299 Arpwatch
07300 - 07399 Symantec Anti Virus

09100 - 09199 PPTP
09200 - 09299 Squid syslog
09300 - 09399 Horde IMP

10100 - 10199 FTS

11100 - 11199 FTPd
11200 - 11299 ProFTPD
11300 - 11399 Pure-FTPD
11400 - 11499 vs-FTPD

12100 - 12299 Named (bind DNS)

13100 - 13299 Samba (smbd)

14100 - 14199 Racoon SSL
14200 - 14299 Cisco VPN Concentrator

17100 - 17399 Policy

18100 - 18499 Windows system
18500 - 18650 Sysmon rules
18651 - 18750 MS IPSec rules
20100 - 20299 IDS
20300 - 20499 IDS (Snort specific)
20500 - 20509 Windows PowerShell

30100 - 30999 Apache error log
31100 - 31199 Web access log

31501 - 32000 Web Appsec rules

35000 - 35999 Squid

40100 - 40499 Attack patterns
40500 - 40599 Privilege escalation

40600 - 40699 Scan patterns
40700 - 40899 Systemd
40900 - 40999 Firewalld

51500 - 51999 OpenBSD rules
52000 - 52499 Apparmor rules
52500 - 53199 clam av rules
53200 - 53499 nsd rules
53500 - 53299 opensmtpd rules
53300 - 53399 owncloud rules
53400 - 53500 proxmox ve rules
53501 - 53550 OpenSMTPd rules
53551 - 53599 dnsmasq
53600 - 53625 linux usb detection rules
53626 - 53630 ms usb detection rules
53631 - 53699 ms firewall rules
53700 - 53749 PSAD rules
53750 - 53799 unbound rules
53800 - 53825 Kaspersky Endpoint Security 10 for Linux rules
53826 - 53829 MHN - Dionaea
53830 - 53840 MHN - Cowrie
56000 - 56200 FreeBSD rules

100000 - 109999 User defined rules

Doc Rules (`doc/rules.txt`)

OSSEC HIDS v0.9
Copyright (C) 2009 Trend Micro Inc.

--- Rules Classification ---

-- Classification --

The rules are classified in multiple levels. From the lowest (00) to the maximum
level 16. Some levels are not used right now. Other levels can be added between
them or after them.

**The rules will be read from the highest to the lowest level. **

00 - Ignored - No action taken. Used to avoid false positives. These rules
are scanned before all the others. They include events with no
security relevance.
01 - None -
02 - System low priority notification - System notification or
status messages. They have no security relevance.
03 - Successful/Authorized events - They include successful login attempts,
firewall allow events, etc.
04 - System low priority error - Errors related to bad configurations or
unused devices/applications. They have no security relevance and
are usually caused by default installations or software testing.
05 - User generated error - They include missed passwords, denied
actions, etc. By itself they have no security relevance.
06 - Low relevance attack - They indicate a worm or a virus that have
no affect to the system (like code red for apache servers, etc).
They also include frequently IDS events and frequently errors.
07 - "Bad word" matching. They include words like "bad", "error", etc.
These events are most of the time unclassified and may have
some security relevance.
08 - First time seen - Include first time seen events. First time
an IDS event is fired or the first time an user logged in.
If you just started using OSSEC HIDS these messages will
probably be frequently. After a while they should go away.
It also includes security relevant actions (like the starting
of a sniffer or something like that).
09 - Error from invalid source - Include attempts to login as
an unknown user or from an invalid source. May have security
relevance (specially if repeated). They also include errors
regarding the "admin" (root) account.
10 - Multiple user generated errors - They include multiple bad
passwords, multiple failed logins, etc. They may indicate an
attack or may just be that a user just forgot his credentials.
11 - Integrity checking warning - They include messages regarding
the modification of binaries or the presence of rootkits (by
rootcheck). If you just modified your system configuration
you should be fine regarding the "syscheck" messages. They
may indicate a successful attack. Also included IDS events
that will be ignored (high number of repetitions).
12 - High importancy event - They include error or warning messages
from the system, kernel, etc. They may indicate an attack against
a specific application.
13 - Unusual error (high importance) - Most of the times it matches a
common attack pattern.
14 - High importance security event. Most of the times done with
correlation and it indicates an attack.
15 - Severe attack - No chances of false positives. Immediate
attention is necessary.

== Rules Group ==

-We can specify groups for specific rules. It's used for active
response reasons and for correlation.

We currently use the following groups:
invalid_login
authentication_success
authentication_failed
connection_attempt
attacks
adduser
sshd
ids
firewall
squid
apache
syslog

== Rules Config ==

http://www.ossec.net/docs/docs/manual/rules-decoders/index.html

Doc Br Active Response Internal (`doc/br/active-response-internal.txt`)

OSSEC HIDS 0.6
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

Como a resposta ativa trabalha internamente:

Leia o arquivo active-respose-doc.txt para maiores detalhes
de configura��o

1 - O servidor de an�lize recebe um evento que acione a
pol�tica de resposta ativa.

2 - O servidor de an�lize verifica todos os campos requeridos
fornecidos pelo evento. Isto significa que o servidor
de an�lise decodificar� o evento e ir� extrair as informa��es
necess�rias. Um exemplo disto � que pode-se extrair o n�mero
de IP de um evento e envi�lo para o firewall bloquear.

3 - Se a pol�tica de resposta ativa especificar esta a��o, e esta
dever ser executada localmente no AS, uma mensagem deve ser
enviada diretamente.

4 - Se a pol�tica de resposta ativa especificar uma a��o,
e esta deve ser executada remotamente, uma mensagem � enviada para
o servidor (remoted) para enviar o evento ao cliente especificado.

Doc Br Active Response (`doc/br/active-response.txt`)

OSSEC HIDS v0.7
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

http://www.ossec.net/docs/docs/manual/ar/index.html

Doc Br Logs (`doc/br/logs.txt`)

OSSEC HIDS 0.5
Copyright (c) 2004,2005 Daniel B. Cid [email protected]
[email protected]

OSSEC Hids Logging

== Introdu��o ==

O sistema suporta dois tipos de logs. Logs de alertas e eventos
ou arquivos de logs.

Todas as mensagens recebidas s�o tratadas como um envento. Todas as
mensagens de log, avisos de integridade, informa��es de sistema s�o
tratadas como tal. Fazer log de � muito pesado para o sistema pois
o sistema tem de arquivar cada evendo. Entretanto, � muito bom para
se ter uma vis�o geral de tentativas de ataque.

Criar logs de alertas � muito importante. Pode ser gerado quando um evento
vai de encontro com uma regra de detec��o. Adicionalmente o OSSEC hids
suporta envio de em-ails e execu��o de comandos esternos como m�todo de alerta.

== Criando logs de eventos ==

O diret�rio de logs do OSSEC (by default /var/ossec/logs)
poss�i entrada para arquivos (/var/ossec/logs/archives). Dentro deste diret�rio,
todos os enventos s�o classificados por data.
Por exemplo, todos os eventos recebidos em 22 de maio de 2004, ser�o guardados em:

/var/ossec/logs/archives/2004/May/events-22.log

Ap�s cada dia, uma combina��o ser� criada para cada dia espec�fico.

/var/ossec/logs/archives/2004/May/events-22.log.md5

Este ser� uma combina��o do arquivo do dia 22 com o do dia 21.

A combina��o do dia 1, ser� feita com o dia 31(ou 30 ou 28) do mes anterior.

Isto assegurar� que o log n�o seja modificado. Para que isto aconte�a,
todos os logs (desde o primeiro dia) dever�o ser modificados.

== Criando logs de alerta ==

Haver� um diret�rio de alerta dentro do diret�rio default do OSSEC.
Este ser� organizado da mesma forma que os logs de envetos. Por facor leia
a explica��o acima.

Doc Br Manager (`doc/br/manager.txt`)

OSSEC HIDS 0.5
Copyright (c) 2004,2005 Daniel B. Cid [email protected]
[email protected]

Como usar o gerenciamento de clientes do servidor.

-A m�quina cliente deve abrir a porta 1514 (por default) para enviar�
mensagens para o server. Somente o IP do servidor ser� liberado.

-A cada 10 minutos, o cliente envia uma notifica��o com o status para
o servidor. Esta mensagem de status contem informa��es sobre o
sistema do cliente e sobre os arquivos do diret�rio compartilhado.

-O servidor recebe a mensagem de status, atualiza o arquivo de status
do cliente e checa se h� algum arquivo para ser enviado para o cliente.
Se ouver, este conectar� com o cliente e enviar� o arquivo.

-Todas as mensagens s�o criptografadas.

Doc Br Rootcheck (`doc/br/rootcheck.txt`)

T�cnicas de detec��o de Rootkit usadas pelo OSSEC HIDS
por Daniel B. Cid, [email protected]

Comen�ando na vers�o 0.4, o OSSEC HIDS executar� a detec��o
de rootkit em todos os sistemas que o cliente estiver instalado.
O rootcheck (Sistema de detec��o de rootkit) ser� executado
a cada X minutos (Especificado pelo usu�rio --por default
a cada 2 horas) para detectar possiveis rootkits instalados.
Em conjunto com a an�lize de logs e o sistema de checagem de integridade,
ele torna-se uma solu��o poderosa de monitoramento (o OSSEC HIDS executa
a an�lize de logs e verifica��o de integridade desde a vers�o 0.1).

Outra caracter�stica inclu�da na vers�o 0.4 � que o servidor de an�lize
enviar� automaticamente as assinaturas de detec��o de rootkit aos
clientes, reduzindo o trabalho com administra��o para o administrador
de sistemas. Os clientes e servidor manter�o contato a cada 10 minutos
e se o servidor for atualizado com um novo arquivo de assinatura, ele enviar�
para todos os clientes configurados. De uma olhada na documenta��o para maiores
informa��es.

O rootcheck executar� as seguintes etapas para verificar a presen�a de
rootkits:

1- Ler� o arquivo rootkit_files.txt que contem um grande banco de dados
de rootkits e arquivos usados pelo mesmo. Tentar� usar as fun��es
fopen e opendir no arquivo especificado. N�s usamos todas a chamadas do
sistema,por causa de que rootkits que rodam a nivel de kernel, escondem as
chamadas dos sistema. N�s tentamos melhoras as coutras chamadas do sistema,
para uma melhor detec��o. Este m�todo � mais parecido com uma regra de
ant�-virus, que necessita ser atualizado contantemente. As chances de
falso-positivos s�o pequenas, mas falso-negativos podem ser produzidos
modificando os rootkits.

2- Ler� o arquivo rootkit_trojans.txt que contem um banco de dados de
assinaturas de arquivos infectados por rootkits. Esta t�cnica verifica
bin�rios modificados por trojans conhecidos. Este m�todo de detec��o
n�o encontra nenhum rootkit a nivel de kernel ou nenhum rootkit desconhecido.

3- Escaneia o diret�rio /dev procurando por anomalias. O /dev cont�m todos os
arquivos de device e scripts de Makedev. V�rios rootkits usam arquivos
escondidoso no /dev. Esta t�cnica pode detectar rootkits desconhecidos.

4- Escaneia o sistema de arquivos a procura por arquivos incomuns e problema de
permi��o. Arquivos setados para root, com permi��o de escrita para outros
s�o muito perigosos e a detec��o de rootkit olha isso. Arquivos com Suid,
diret�rios e arquivos escondidos s�o inspecionados.

5- Procura pela presen�a de processos ocultos. N�s usamos as fun��es getsid e
kill para checar se algum PID est� sendo usado ou n�o. Se estiver, mas a fun��o
n�o pode ve-lo, � uma indica��o de rootkit a nivel de kernel ou o ps foi
infectado. N�s verificamos tamb�m a sa�da do kill e do getid que deve ser a mesma.

6- Olha a presen�a de portas escondidas. N�s usamos a fun��o bind para checar todas
as portas TCP e UDP to do sistema. Se n�o podemos usar o bind na porta (ele est�
sendo usado), mas o netsatat n�o mostra, provavelmente h� um rootkit instalado.

7- Escaneia todas a interfaces do sistema e procura por alguma com m�do "promisc"
abilitado. Se a interface est� neste modo, a sa�da do ifconfig mostrar�. Se n�o,
h� a possibilidade de haver um rootkit instalado.

EOF

Doc Br Rule Ids (`doc/br/rule_ids.txt`)

0100 - 0199 Mensagens de erro do Syslog
0200 - 0299 Mensagens NFS
0300 - 0399 Mensagens do Xinetd
0400 - 0499 Mensagens do controle de acesso
0500 - 0599 Mensagens do Sendmail
0600 - 0699 Mensagens do Ftp
0700 - 0799 Mensagens do Smartd
0800 - 0899 Mensagens do Kernel
0900 - 0949 Mensagens do Crond
0950 - 0999 Mensagens autom�ticas
1000 - 1099 Mensagens do Syslog a respeito do Squid
1100 - 1199 Mensagens do su
1200 - 1299 Mensagens do tripwire
1300 - 1399 Mensagens do adduser
1400 - 1499 Mensagens do sudo
1500 - 1599 Mensagens do sshd
1600 - 1699 Testes padr�es de ataques
1700 - 1799 Testes padr�es da escala de privil�gios
1800 - 1899 Testes padr�es de scaners
1900 - 1999 Regras de DNS
2000 - 2499 Regras de PIX
2500 - 2599 Regras do proftpd
2600 - 2699 Mensagens do PPTP
2700 - 2799 Regras do FTS
2800 - 2899 Regras do Pure-FTPD
3000 - 3099 Regras do log de erro do Apache
3100 - 3199 Regras do log de acesso do Apache
4000 - 4999 Regras de IDS
5000 - 5999 Regras do Squid

Doc Br Rules (`doc/br/rules.txt`)

OSSEC HIDS v0.7
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

--- Classidica��o das regras ---

-- Classifica��o --

As regras s�o classificadas em v�rios niveis, indo do menos (00) para o maior (16).
Alguns n�veis n�o est�o sendo usados. Voc� pode adicionar mais entre ou ap�s estes.

**As regras ser�o lidas partindo do menor nivel para o maior. **

00 - Ignorado - N�o h� nehum a��o. Usado para evitar falso positivos. Estas regras
s�o escaneadas antes das demias e incluem eventos sem relev�ncia de seguran�a.
01 - Nenhuma -
02 - Notifica��o do sistema com baixa prioridade - Notifica��es do sistema ou mensagens
de status. Estas regras n�o posseum relev�ncia de seguran�a.
03 - Eventos de sucesso/Autorizado - Estes incluem tentativas de logins com sucesso,
eventos permitidos pelo, etc.
04 - Erros de sistema com baixa prioridade - Erros relatados por m� configura��o ou
devices/aplica��es n�o usadas. Estas n�o apresentam relev�ncia de seguran�a e
normalmente s�o causadas por instala��es padr�es ou software em testes.
05 - Erros gerados pelo usu�rio - Este incluem senhas erradas, a��es n�o permitidas,
etc, mas n�o tem relev�ncia de seguran�a.
06 - Ataque de baixa relev�ncia - Estas indicam que um worm ou v�rus que n�o afetam
o sistema (como o code red para servidores apache, etc). Incluem tamb�m eventos
de IDS ou erros frequentes.
07 - Combina��o de "Bad word". Estas incluem palavras como "bad", "error", etc. Estes
eventos normalmente n�o s�o classificados e possuem alguma relev�ncia de seguran�a.
08 - Visto pela primeira vez - Incluem eventos vistos pela primeira vez, primeira vez
de ventos de IDS ou primeira vez que um usu�rio loga. Se voc� estiver come�ando a
usar o OSSEC HIDS, estas mensagens ser�o frequ�ntes mas depois devem diminuir.
Estas incluem tamb�m a��es de seguran�a relevantes (como iniciar um sniffer ou
algo do g�nero).
09 - Erro de fonte inv�lida - Inclue tentativas de login com um usu�rio desconhecido
ou de uma fonte inv�lida. Pode haver relev�ncia de seguran�a (princilmente se ocorrer
repetidamente). Incluem tamb�m erros a respeito do usu�rio "root".
10 - Erros gerador por v�rios usu�rios - Estas incluem v�rios bad passwords,
v�rios logins falhos, etc. Podem indicar um ataque ou um usu�rio que esqueceu
suas credenciais.
11 - Advert�ncia de checagem de integridade -Estas incluem mensagens a respeito da
modifica��o de bin�rios ou a presen�a de rootkits (pelo rootcheck). Se voc�
houver modificado apenas a configura��o do seu sistema, deve ficar atento as mensagens
do "syscheck". Pode indicar um ataque com sucesso.
12 - Evento de alta import�ncia - Estas incluem erros ou avisos vindos do sistema, kernel,
etc. Podem indicar um ataque a uma aplica��o espec�fica.
13 - Erros incomuns (alta import�ncia) - A maioria das vezes mostra um ataque padr�o.
14 - Evendo de seguran�a de alta import�ncia - Na maioria das vezes s�o eventos de
corela��o e indicam um ataque.
15 - Ataque severo - N�o h� chanses de falso positivo. � necess�ria aten��o imediata.

== Grupos de regras ==

-N�s podemos especificar grupos para regras espec�ficas. Isto � usado para
raz�es de resposta ativa ou para corela��o.

== Configura��o das regras ==

http://www.ossec.net/docs/docs/manual/rules-decoders/index.html

Doc Pl Active Response Internal (`doc/pl/active-response-internal.txt`)

OSSEC HIDS 0.6
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

Jak działa aktywa ochrona (od środka):

Przeczytaj active-respose-doc.txt aby dowiedzieć się więcej
o konfiguracji

1 - Zdarzenie które pasuje do polityki aktywnej ochrony jest
wysyłane do serwera analizującego.

2 - Serwer weryfikuje czy wszystkie wymagane pola są są
dostarczone w zdarzeniu. Oznacza to, że serwer zdekodował
zdarzenie i odczytał wszystkie pożądane informacje. Np.
musi odczytac adres IP aby podjąć akcję o zablokwaniu go
na firewall-u.

3 - Jeśli aktywna ochrona jest skonfigurowana aby akcje były
wykonywane lokalnie, widomość zostaje bezpośrednio wysyłana
do execd.

4 - Jeśli aktywna ochrona jest skonfigurowana aby akcje były
wykonywane zdalnie, wiadomość jest wysyłana do remoted
(forwarder aktywnej ochrony), który przekazuje ją dalej
do odpowiedniego agenta.

Doc Pl Active Response (`doc/pl/active-response.txt`)

OSSEC HIDS v0.7
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

http://www.ossec.net/docs/docs/manual/ar/index.html

Doc Pl Logs (`doc/pl/logs.txt`)

OSSEC HIDS 0.5
Copyright (c) 2004,2005 Daniel B. Cid [email protected]
[email protected]

Logi OSSEC Hids

== Wprowadzenie ==

System wspiera dwa typy logowania. Logowanie alarmów i zdarzeń lub
logowanie z archiwizacją.

Każda otrzymana wiadomość jest traktowana jako zdarzenie.
Raporty integralności systemu, informacje systemowe lub zwykłe
wiadomości logów są traktowane jako zdarzenia. Logowanie zdarzeń bardzo
obciąża system, ponieważ archiwizuje każde zdarzenie. Mimo to, jest
przydatne, ponieważ może dać nam dokładny obraz wydarzeń po włamaniu.

Logowanie alarmów jest najbardziej użyteczne. Alarm jest generowany
kiedy zdarzenie zostaje dopasowane do jednej z reguł wykrywania. Poza
logowaniem OSSEC hids posiada powiadomnienie poprzez e-mail oraz
zewnętrzne wykonywanie komend jako metodę alarmu.

== Logowanie zdarzeń ==

W domyślnym katalogu logów OSSEC (/var/ossec/logs) jeden jest
przezaczony na archiwa (/var/ossec/logs/archives). W tym
katalogu wszystkie zdarzenia są przechowywane według daty.
Np. wszystkie zdarzenia otrzymane 22.05.2005, będą przechowywane w:

/var/ossec/logs/archives/2004/May/events-22.log

Na koniec każdego dnia, jest tworzony hash i przechowywany w:

/var/ossec/logs/archives/2004/May/events-22.log.md5

Jest to hash z pliku z dnia 22 plus hash z dnia 21.

Hash z dnia pierwszego, jest hashem z dnia 31 (30 lub 28) poprzedniego
miesiąca.

Hash jest tworzony aby mieć pewność, że logi nie były modyfikowane.
Dodatkowo zabezpiecza wszystkie poprzednie logi począwszy od pierwszego.

== Logowanie alarmów ==

W domyślnym katalogu logów OSSEC jest katalogo alarmów
(/var/ossec/logs/alerts). Zorganizowany w taki sam sposób jak
katalog zdarzeń. Przeczytaj powyżej aby zrozumieć.

Doc Pl Manager (`doc/pl/manager.txt`)

OSSEC HIDS 0.5
Copyright (c) 2004,2005 Daniel B. Cid [email protected]
[email protected]

Jak serwer zarządza agentami.

-Klient otwiera port 1514 (domyślnie) i nasłuchuje na wiadomości od
serwera. Wyłącznie IP serwera jest akceptowane.

-Co 10 minut, klient wysyła powiadomienie o stanie do serwera.
Powiadomienie zawiera niektóre informacje o systemie klienckim oraz
informacje o plikach które posiada w katalogu shared.

-Serwer otrzymuje wiadomości statusu, aktualizuje plik statusu agenta
oraz sprawdza czy posiada jakiś plik do wysłania dla agenta. Jeśli ma,
to łączy się z klientem i wysyła go.

-Każda wiadomość jest szyfrowana.

Doc Pl Rootcheck (`doc/pl/rootcheck.txt`)

Rootkit detection techniques used by the OSSEC HIDS
by Daniel B. Cid, [email protected]

Zaczynając od wersji 0.4, OSSEC HIDS dokonuje wykrywania rootkitów na
każdym systemnie na którym jest zainstalowany agent. Mechanizm
wykrywania rootkitów (rootcheck) jest wykonywany co X minut
(wyspecyfikowane przez użytkownika - domyśnie co dwie godziny) w celu
wykrycia zinstalowanych rootkitów. Używany razem z analizatorem logów
oraz mechanizmem sprawdzania integralności, staje się bardzo wydajnym
systemem monitorowania (OSSEC HIDS dokonuje analizy logów i sprawdzania
integralności od wersji 0.1).

Inną właściwością dołączoną do wersji 0.4 jest automatyczne rozsyłanie
sygnatur wykrywania rootkitów do wszystkich agentów przez serwer.
Redukuje to ilość obowiązków administratora. Serwer z agentami utrzymuje
kontakt co 10 minut. Dodatkowo jeśli serwer zostanie zaktualizowany
nowymi sygnaturami, przekazuje je wszystkim skonfigurowanym agentom.
Aby uzyskać więcej informacji zajżyj do dokumentacji zarządzania.

Kroki wykonywane przez rootcheck w celu wykrycia rootkitów:

1- Czyta plik rootkit_files.txt, który zawiera sporą baze danych o
rootkitach i używanych przez nie plików. Próbuje wykonać: stats,
fopen i opendir na każdym z takich plików. Używa wszystkich tych
wywołań systemowych, ponieważ istnieją rootkity pracujące na poziomie
jądra, które ukrywają pliki przed niektórymi wywołaniami. Im więcej
użyjemy wywołań, tym lepsza będzie detekcja. Metoda jest podobna do
reguły anty-virusa, potrzebuje nieustanntgo uaktualniania. Szanse
detekcji rootkita którego nie ma (false-positive) są małe, za to już
szanse nie wykrycie rootkita (false-negative) rosną wraz z
modyfikacjami rootkitów.

2- Czyta plik rootkit_trojans.txt który zawiera baze danych sygnatur
plików modyfikowanych na trojany przez rootkity. Technika modyfikacji
binariów na wersje z trojanami jest powszechnie używana przez
większość dostępnych rootkitów. Nastepująca metoda nie wykrywa
rootkitów poziomu jądra ani żadnych jeszcze nie znanych.

3- Skanuje katalog /dev wyszukując anomali. /dev powinien tylko zawierać
pliki urządzeń oraz skrypt Makedev. Duża część rootkitów używa
katalogu /dev w celu ukrycia plików. Ta metoda może wykryć nieznane
dotąd rootkity.

4- Skanuje cały system plików szukając niespotykanych plików oraz
tych z podejrzanymi uprawnieiami. Pliki użytkownika root z
możliwością zapisu dla innych są bardzo niebezpieczne, dlatego
rootkit wyszukuje takie. Pliki z ustawionym suid oraz ukryte katalogi
również są sprawdane.

5- Szuja ukrytych procesów. Używa getsid() oraz kill() do sprawdzenia
czy dany pid jest używany. Jeśli dany pid jest używany, a "ps" nie
widzi go, wskazuje to na obecność rootkita poziomu jądra lub
zmodyfikowaną wersję "ps". Weryfikuje także wyniki kill i getsid,
które powinny być identyczne.

6- Wyszukuje ukrytych portów. Używa bind() do sprawdzenia każdego portu
tcp i udp w systemie. Jeśli nie możemy wykonać bind na danym porcie
(czyli jest używany), a "netstat" go nie pokazuje, to prawdopodobnie
mamy zainstalowanego rootkita.

7- Skanuje wszystkie interfejsy w systemnie i wyszukuje te z włączonym
trybem "promisc". Jeśli interfejs jest w trybie "promisc", "ifconfig"
tego powinien pokazać to, jeśli nie prawdopodobnie mamy
zainstalowanego rootkita.

EOF

Doc Pl Rule Ids (`doc/pl/rule_ids.txt`)

0100 - 0199 Wiadomości błędów Syslog
0200 - 0299 Wiadomości NFS
0300 - 0399 Wiadomości Xinetd
0400 - 0499 Wiadomości kontroli dostępu
0500 - 0599 Wiadomości Sendmail
0600 - 0699 Wiadomości Ftp
0700 - 0799 Wiadomości Smartd
0800 - 0899 Wiadomości Kernel
0900 - 0949 Wiadomości Crond
0950 - 0999 Wiadomości Automount
1000 - 1099 Wiadomości Squid syslog
1100 - 1199 Wiadomości Su
1200 - 1299 Wiadomości Tripwire
1300 - 1399 Wiadomości Adduser
1400 - 1499 Wiadomości Sudo
1500 - 1599 Wiadomości sshd
1600 - 1699 Wzorce ataków
1700 - 1799 Wzorce przejmowania uprawnień
1800 - 1899 Wzorce skanowań
1900 - 1999 Reguły DNS
2000 - 2499 Reguły PIX
2500 - 2599 Reguły Proftpd
2600 - 2699 Wiadomości PPTP
2700 - 2799 Reguły Reguły FTS
2800 - 2899 Reguły Pure-FTPD
2900 - 2999 Reguły Spamd
3000 - 3099 Reguły błędów dla Apache'a (dla logów)
3100 - 3199 Reguły dostępu dla Apache'a (dla logów)
4000 - 4999 Reguły IDS
5000 - 5999 Reguły Squid
6000 - 6199 Reguły Postfix
6200 - 6499 Reguły Sendmail
7000 - 7499 Reguły Firewall
8000 - 8999 Reguły Windows
100000 - 199999 Reguły zdefiniowane przez użytkownika

Doc Pl Rules (`doc/pl/rules.txt`)

OSSEC HIDS v0.7
Copyright (c) 2004-2006 Daniel B. Cid [email protected]
[email protected]

--- Klasyfikacja reguł ---

-- Klasyfikacja --

Reguły są podzielone na 16 poziomów. Niektóre poziomy są aktualnie nie używane.
Inne mogą być dodane pomiędzy nimi lub za.

**Reguły są czytane od najwyższego do najniższego poziomu. **

00 - Ignorowane - Nie jest podejmowana żadna akcja. Używane aby uniknąć
fałszywych ataków. Te reguły są przetwarzane przed wszystkimi pozostałymi.
Zawierają zdarzenia nie związane z bezpieczeństwem.
01 - Brak -
02 - Niski priorytet powiadomień systemowych - Powiadomienia systemowe lub
wiadomości statusu. Nie związane z bezpieczeństwem.
03 - Pomyślne/Autoryzowane akcje - Zawiera pomyślne próby logowania, dozwolone
akcje firewall'a, itd.
04 - Niski priorytet błędów systemowych - Błędy związane ze złą konfiguracją lub
nieużywanymi urządzeniami/aplikacjami. Nie są związane z bezpieczeństwem,
zazwyczaj są powodowane poprzez instalacje lub testowanie aplikacji.
05 - Błędy wygenerowane przez użytkownika - Zawierają błędne logowania,
zabronione akcje, itp. Nie mają wpływu na bezpieczeństwo, ponieważ już są
blokowane.
06 - Atak o niskim znaczeniu - Powodowane przez robaki lub wirusy które nie mają
wpływu na system (takie jak code red dla serwera apache, itp). Zawierają
także często akcje IDS oraz błędy.
07 - Dopasowanie bad word - Zawierają słowa "bad', "error", itp. W większości
niesklasyfikowane akcje, które mogą mieć wpływ na bezpieczeństwo.
08 - Widziane poraz pierwszy - Zawierają akcje widziane poraz pierwszy. Np.
pierwszy raz jest generowana akcja IDS albo pierwsze logowanie użytkownika.
Jeśli dopiero zacząłeś używać OSSEC HIDS, takie akcje najprawsopodobniej
będą występować często. Lecz po jakimś czasie znikną. Zawierają także akcje
związane z bezpieczeństwem (takie jak uruchomienie sniffera lub podobnego
narzędzia).
09 - Błędy z nieznanego źródła - Zawierają próby logowania jako nieznany
użytkownik lub z niepoprawnego źródła. Mogą mieć wpływ na bezpieczeństwo
(szczególnie gdy powtarzane). Obejmują także błędy związane z kontem
"administratora", root'a.
10 - Powtarzalne błędy generowane przez użytkownika - Zawierają wielokrotne
nieudane próby logowania, błędy autoryzacji itp. Mogą oznaczać atak lub
poprostu użytkownik zapomiał swoich danych autoryzujących.
11 - Ostrzeżenie, sprawdzanie spójności - Zawiera wiadomości oznaczające
modyfikacje plików binarnych lub obecność rootkit'ów (generowane przez
rootcheck). Jeśli aktualizowałeś system nie powinieneś się przejmować tymi
ostrzeżeniami. Mogą oznaczać udany atak.
12 - Wysoce istotne zdarzenia - Zawierają wiadomości błądów lub ostrzeżeń od
systemu, jądra, itp. Moga oznaczać atak na konkretną aplikacje.
13 - Niespotykane błędy (wysoce istotne) - W większości dopasowane do wzorców
znanych ataków.
14 - Wysoce istotne zdarzenia bezpieczeństwa - Akcje wykrywane poprzez związki
ze sobą, zazwyczaj oznaczają atak.
15 - Poważny atak - Natychmiastowa reakcja jest potrzebna (nie może być mowy tu
o fałszywym ataku).

== Grupa reguł ==

-Możemy wyspecyfikować grupy dla specyficznych reguł. Jest to używane do
aktywnej ochrony oraz do relacji.
-Aktualnie używamy następujących grup:

invalid_login
authentication_success
authentication_failed
connection_attempt
attacks
adduser
sshd
ids
firewall
squid
apache
syslog

== Konfiguracja reguł ==

http://www.ossec.net/docs/docs/manual/rules-decoders/index.html

Src External Zlib 1.3.1 Doc Algorithm (`src/external/zlib-1.3.1/doc/algorithm.txt`)

Compression algorithm (deflate)

The deflation algorithm used by gzip (also zip and zlib) is a variation of
LZ77 (Lempel-Ziv 1977, see reference below). It finds duplicated strings in
the input data. The second occurrence of a string is replaced by a
pointer to the previous string, in the form of a pair (distance,
length). Distances are limited to 32K bytes, and lengths are limited
to 258 bytes. When a string does not occur anywhere in the previous
32K bytes, it is emitted as a sequence of literal bytes. (In this
description, `string' must be taken as an arbitrary sequence of bytes,
and is not restricted to printable characters.)

Literals or match lengths are compressed with one Huffman tree, and
match distances are compressed with another tree. The trees are stored
in a compact form at the start of each block. The blocks can have any
size (except that the compressed data for one block must fit in
available memory). A block is terminated when deflate() determines that
it would be useful to start another block with fresh trees. (This is
somewhat similar to the behavior of LZW-based compress.)

Duplicated strings are found using a hash table. All input strings of
length 3 are inserted in the hash table. A hash index is computed for
the next 3 bytes. If the hash chain for this index is not empty, all
strings in the chain are compared with the current input string, and
the longest match is selected.

The hash chains are searched starting with the most recent strings, to
favor small distances and thus take advantage of the Huffman encoding.
The hash chains are singly linked. There are no deletions from the
hash chains, the algorithm simply discards matches that are too old.

To avoid a worst-case situation, very long hash chains are arbitrarily
truncated at a certain length, determined by a runtime option (level
parameter of deflateInit). So deflate() does not always find the longest
possible match but generally finds a match which is long enough.

deflate() also defers the selection of matches with a lazy evaluation
mechanism. After a match of length N has been found, deflate() searches for
a longer match at the next input byte. If a longer match is found, the
previous match is truncated to a length of one (thus producing a single
literal byte) and the process of lazy evaluation begins again. Otherwise,
the original match is kept, and the next match search is attempted only N
steps later.

The lazy match evaluation is also subject to a runtime parameter. If
the current match is long enough, deflate() reduces the search for a longer
match, thus speeding up the whole process. If compression ratio is more
important than speed, deflate() attempts a complete second search even if
the first match is already long enough.

The lazy match evaluation is not performed for the fastest compression
modes (level parameter 1 to 3). For these fast modes, new strings
are inserted in the hash table only when no match was found, or
when the match is not too long. This degrades the compression ratio
but saves time since there are both fewer insertions and fewer searches.

Decompression algorithm (inflate)

2.1 Introduction

The key question is how to represent a Huffman code (or any prefix code) so
that you can decode fast. The most important characteristic is that shorter
codes are much more common than longer codes, so pay attention to decoding the
short codes fast, and let the long codes take longer to decode.

inflate() sets up a first level table that covers some number of bits of
input less than the length of longest code. It gets that many bits from the
stream, and looks it up in the table. The table will tell if the next
code is that many bits or less and how many, and if it is, it will tell
the value, else it will point to the next level table for which inflate()
grabs more bits and tries to decode a longer code.

How many bits to make the first lookup is a tradeoff between the time it
takes to decode and the time it takes to build the table. If building the
table took no time (and if you had infinite memory), then there would only
be a first level table to cover all the way to the longest code. However,
building the table ends up taking a lot longer for more bits since short
codes are replicated many times in such a table. What inflate() does is
simply to make the number of bits in the first table a variable, and then
to set that variable for the maximum speed.

For inflate, which has 286 possible codes for the literal/length tree, the size
of the first table is nine bits. Also the distance trees have 30 possible
values, and the size of the first table is six bits. Note that for each of
those cases, the table ended up one bit longer than the ``average'' code
length, i.e. the code length of an approximately flat code which would be a
little more than eight bits for 286 symbols and a little less than five bits
for 30 symbols.

2.2 More details on the inflate table lookup

Ok, you want to know what this cleverly obfuscated inflate tree actually
looks like. You are correct that it's not a Huffman tree. It is simply a
lookup table for the first, let's say, nine bits of a Huffman symbol. The
symbol could be as short as one bit or as long as 15 bits. If a particular
symbol is shorter than nine bits, then that symbol's translation is duplicated
in all those entries that start with that symbol's bits. For example, if the
symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a
symbol is nine bits long, it appears in the table once.

If the symbol is longer than nine bits, then that entry in the table points
to another similar table for the remaining bits. Again, there are duplicated
entries as needed. The idea is that most of the time the symbol will be short
and there will only be one table look up. (That's whole idea behind data
compression in the first place.) For the less frequent long symbols, there
will be two lookups. If you had a compression method with really long
symbols, you could have as many levels of lookups as is efficient. For
inflate, two is enough.

So a table entry either points to another table (in which case nine bits in
the above example are gobbled), or it contains the translation for the symbol
and the number of bits to gobble. Then you start again with the next
ungobbled bit.

You may wonder: why not just have one lookup table for how ever many bits the
longest symbol is? The reason is that if you do that, you end up spending
more time filling in duplicate symbol entries than you do actually decoding.
At least for deflate's output that generates new trees every several 10's of
kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code
would take too long if you're only decoding several thousand symbols. At the
other extreme, you could make a new table for every bit in the code. In fact,
that's essentially a Huffman tree. But then you spend too much time
traversing the tree while decoding, even for short symbols.

So the number of bits for the first lookup table is a trade of the time to
fill out the table vs. the time spent looking at the second level and above of
the table.

Here is an example, scaled down:

The code being decoded, with 10 symbols, from 1 to 6 bits long:

A: 0
B: 10
C: 1100
D: 11010
E: 11011
F: 11100
G: 11101
H: 11110
I: 111110
J: 111111

Let's make the first table three bits long (eight entries):

000: A,1
001: A,1
010: A,1
011: A,1
100: B,2
101: B,2
110: -> table X (gobble 3 bits)
111: -> table Y (gobble 3 bits)

Each entry is what the bits decode as and how many bits that is, i.e. how
many bits to gobble. Or the entry points to another table, with the number of
bits to gobble implicit in the size of the table.

Table X is two bits long since the longest code starting with 110 is five bits
long:

00: C,1
01: C,1
10: D,2
11: E,2

Table Y is three bits long since the longest code starting with 111 is six
bits long:

000: F,2
001: F,2
010: G,2
011: G,2
100: H,2
101: H,2
110: I,3
111: J,3

So what we have here are three tables with a total of 20 entries that had to
be constructed. That's compared to 64 entries for a single table. Or
compared to 16 entries for a Huffman tree (six two entry tables and one four
entry table). Assuming that the code ideally represents the probability of
the symbols, it takes on the average 1.25 lookups per symbol. That's compared
to one lookup for the single table, or 1.66 lookups per symbol for the
Huffman tree.

There, I think that gives you a picture of what's going on. For inflate, the
meaning of a particular symbol is often more than just a letter. It can be a
byte (a "literal"), or it can be either a length or a distance which
indicates a base value and a number of bits to fetch after the code that is
added to the base value. Or it might be the special end-of-block code. The
data structures created in inftrees.c try to encode all that information
compactly in the tables.

Jean-loup Gailly Mark Adler
[email protected] [email protected]

References:

[LZ77] Ziv J., Lempel A., ``A Universal Algorithm for Sequential Data
Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3,
pp. 337-343.

``DEFLATE Compressed Data Format Specification'' available in
http://tools.ietf.org/html/rfc1951

Src External Zlib 1.3.1 Doc Rfc1950 (`src/external/zlib-1.3.1/doc/rfc1950.txt`)

Network Working Group P. Deutsch
Request for Comments: 1950 Aladdin Enterprises
Category: Informational J-L. Gailly
Info-ZIP
May 1996

text

ZLIB Compressed Data Format Specification version 3.3

Status of This Memo

This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

IESG Note:

The IESG takes no position on the validity of any Intellectual
Property Rights statements contained in this document.

Notices

Permission is granted to copy and distribute this document for any
purpose and without charge, including translations into other
languages and incorporation into compilations, provided that the
copyright notice and this notice are preserved, and that any
substantive changes or deletions from the original are clearly
marked.

A pointer to the latest version of this and related documentation in
HTML format can be found at the URL
ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html.

Abstract

This specification defines a lossless compressed data format. The
data can be produced or consumed, even for an arbitrarily long
sequentially presented input data stream, using only an a priori
bounded amount of intermediate storage. The format presently uses
the DEFLATE compression method but can be easily extended to use
other compression methods. It can be implemented readily in a manner
not covered by patents. This specification also defines the ADLER-32
checksum (an extension and improvement of the Fletcher checksum),
used for detection of data corruption, and provides an algorithm for
computing it.

Deutsch & Gailly Informational [Page 1]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

Table of Contents

Introduction ................................................... 2
1.1. Purpose ................................................... 2
1.2. Intended audience ......................................... 3
1.3. Scope ..................................................... 3
1.4. Compliance ................................................ 3
1.5. Definitions of terms and conventions used ................ 3
1.6. Changes from previous versions ............................ 3
Detailed specification ......................................... 3
2.1. Overall conventions ....................................... 3
2.2. Data format ............................................... 4
2.3. Compliance ................................................ 7
References ..................................................... 7
Source code .................................................... 8
Security Considerations ........................................ 8
Acknowledgements ............................................... 8
Authors' Addresses ............................................. 8
Appendix: Rationale ............................................ 9
Appendix: Sample code ..........................................10

Introduction

1.1. Purpose

The purpose of this specification is to define a lossless
compressed data format that:

text

* Is independent of CPU type, operating system, file system,
     and character set, and hence can be used for interchange;

   * Can be produced or consumed, even for an arbitrarily long
     sequentially presented input data stream, using only an a
     priori bounded amount of intermediate storage, and hence can
     be used in data communications or similar structures such as
     Unix filters;

   * Can use a number of different compression methods;

   * Can be implemented readily in a manner not covered by
     patents, and hence can be practiced freely.

The data format defined by this specification does not attempt to
allow random access to compressed data.

Deutsch & Gailly Informational [Page 2]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

1.2. Intended audience

text

This specification is intended for use by implementors of software
  to compress data into zlib format and/or decompress data from zlib
  format.

  The text of the specification assumes a basic background in
  programming at the level of bits and other primitive data
  representations.

1.3. Scope

text

The specification specifies a compressed data format that can be
  used for in-memory compression of a sequence of arbitrary bytes.

1.4. Compliance

text

Unless otherwise indicated below, a compliant decompressor must be
  able to accept and decompress any data set that conforms to all
  the specifications presented here; a compliant compressor must
  produce data sets that conform to all the specifications presented
  here.

1.5. Definitions of terms and conventions used

text

byte: 8 bits stored or transmitted as a unit (same as an octet).
  (For this specification, a byte is exactly 8 bits, even on
  machines which store a character on a number of bits different
  from 8.) See below, for the numbering of bits within a byte.

1.6. Changes from previous versions

text

Version 3.1 was the first public release of this specification.
  In version 3.2, some terminology was changed and the Adler-32
  sample code was rewritten for clarity.  In version 3.3, the
  support for a preset dictionary was introduced, and the
  specification was converted to RFC style.

Detailed specification

2.1. Overall conventions

In the diagrams below, a box like this:
text
```
+---+
  |   | <-- the vertical bars might be missing
  +---+
```

Deutsch & Gailly Informational [Page 3]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

text

represents one byte; a box like this:

     +==============+
     |              |
     +==============+

  represents a variable number of bytes.

  Bytes stored within a computer do not have a "bit order", since
  they are always treated as a unit.  However, a byte considered as
  an integer between 0 and 255 does have a most- and least-
  significant bit, and since we write numbers with the most-
  significant digit on the left, we also write bytes with the most-
  significant bit on the left.  In the diagrams below, we number the
  bits of a byte so that bit 0 is the least-significant bit, i.e.,
  the bits are numbered:

     +--------+
     |76543210|
     +--------+

  Within a computer, a number may occupy multiple bytes.  All
  multi-byte numbers in the format described here are stored with
  the MOST-significant byte first (at the lower memory address).
  For example, the decimal number 520 is stored as:

         0     1
     +--------+--------+
     |00000010|00001000|
     +--------+--------+
      ^        ^
      |        |
      |        + less significant byte = 8
      + more significant byte = 2 x 256

2.2. Data format

text

A zlib stream has the following structure:

       0   1
     +---+---+
     |CMF|FLG|   (more-->)
     +---+---+

Deutsch & Gailly Informational [Page 4]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

text

(if FLG.FDICT set)

       0   1   2   3
     +---+---+---+---+
     |     DICTID    |   (more-->)
     +---+---+---+---+

     +=====================+---+---+---+---+
     |...compressed data...|    ADLER32    |
     +=====================+---+---+---+---+

  Any data which may appear after ADLER32 are not part of the zlib
  stream.

  CMF (Compression Method and flags)
     This byte is divided into a 4-bit compression method and a 4-
     bit information field depending on the compression method.

        bits 0 to 3  CM     Compression method
        bits 4 to 7  CINFO  Compression info

  CM (Compression method)
     This identifies the compression method used in the file. CM = 8
     denotes the "deflate" compression method with a window size up
     to 32K.  This is the method used by gzip and PNG (see
     references [1] and [2] in Chapter 3, below, for the reference
     documents).  CM = 15 is reserved.  It might be used in a future
     version of this specification to indicate the presence of an
     extra field before the compressed data.

  CINFO (Compression info)
     For CM = 8, CINFO is the base-2 logarithm of the LZ77 window
     size, minus eight (CINFO=7 indicates a 32K window size). Values
     of CINFO above 7 are not allowed in this version of the
     specification.  CINFO is not defined in this specification for
     CM not equal to 8.

  FLG (FLaGs)
     This flag byte is divided as follows:

        bits 0 to 4  FCHECK  (check bits for CMF and FLG)
        bit  5       FDICT   (preset dictionary)
        bits 6 to 7  FLEVEL  (compression level)

     The FCHECK value must be such that CMF and FLG, when viewed as
     a 16-bit unsigned integer stored in MSB order (CMF*256 + FLG),
     is a multiple of 31.

Deutsch & Gailly Informational [Page 5]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

text

FDICT (Preset dictionary)
     If FDICT is set, a DICT dictionary identifier is present
     immediately after the FLG byte. The dictionary is a sequence of
     bytes which are initially fed to the compressor without
     producing any compressed output. DICT is the Adler-32 checksum
     of this sequence of bytes (see the definition of ADLER32
     below).  The decompressor can use this identifier to determine
     which dictionary has been used by the compressor.

  FLEVEL (Compression level)
     These flags are available for use by specific compression
     methods.  The "deflate" method (CM = 8) sets these flags as
     follows:

        0 - compressor used fastest algorithm
        1 - compressor used fast algorithm
        2 - compressor used default algorithm
        3 - compressor used maximum compression, slowest algorithm

     The information in FLEVEL is not needed for decompression; it
     is there to indicate if recompression might be worthwhile.

  compressed data
     For compression method 8, the compressed data is stored in the
     deflate compressed data format as described in the document
     "DEFLATE Compressed Data Format Specification" by L. Peter
     Deutsch. (See reference [3] in Chapter 3, below)

     Other compressed data formats are not specified in this version
     of the zlib specification.

  ADLER32 (Adler-32 checksum)
     This contains a checksum value of the uncompressed data
     (excluding any dictionary data) computed according to Adler-32
     algorithm. This algorithm is a 32-bit extension and improvement
     of the Fletcher algorithm, used in the ITU-T X.224 / ISO 8073
     standard. See references [4] and [5] in Chapter 3, below)

     Adler-32 is composed of two sums accumulated per byte: s1 is
     the sum of all bytes, s2 is the sum of all s1 values. Both sums
     are done modulo 65521. s1 is initialized to 1, s2 to zero.  The
     Adler-32 checksum is stored as s2*65536 + s1 in most-
     significant-byte first (network) order.

Deutsch & Gailly Informational [Page 6]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

2.3. Compliance

text

A compliant compressor must produce streams with correct CMF, FLG
  and ADLER32, but need not support preset dictionaries.  When the
  zlib data format is used as part of another standard data format,
  the compressor may use only preset dictionaries that are specified
  by this other data format.  If this other format does not use the
  preset dictionary feature, the compressor must not set the FDICT
  flag.

  A compliant decompressor must check CMF, FLG, and ADLER32, and
  provide an error indication if any of these have incorrect values.
  A compliant decompressor must give an error indication if CM is
  not one of the values defined in this specification (only the
  value 8 is permitted in this version), since another value could
  indicate the presence of new features that would cause subsequent
  data to be interpreted incorrectly.  A compliant decompressor must
  give an error indication if FDICT is set and DICTID is not the
  identifier of a known preset dictionary.  A decompressor may
  ignore FLEVEL and still be compliant.  When the zlib data format
  is being used as a part of another standard format, a compliant
  decompressor must support all the preset dictionaries specified by
  the other format. When the other format does not use the preset
  dictionary feature, a compliant decompressor must reject any
  stream in which the FDICT flag is set.

References

[1] Deutsch, L.P.,"GZIP Compressed Data Format Specification",
available in ftp://ftp.uu.net/pub/archiving/zip/doc/

[2] Thomas Boutell, "PNG (Portable Network Graphics) specification",
available in ftp://ftp.uu.net/graphics/png/documents/

[3] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification",
available in ftp://ftp.uu.net/pub/archiving/zip/doc/

[4] Fletcher, J. G., "An Arithmetic Checksum for Serial
Transmissions," IEEE Transactions on Communications, Vol. COM-30,
No. 1, January 1982, pp. 247-252.

[5] ITU-T Recommendation X.224, Annex D, "Checksum Algorithms,"
November, 1993, pp. 144, 145. (Available from
gopher://info.itu.ch). ITU-T X.244 is also the same as ISO 8073.

Deutsch & Gailly Informational [Page 7]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

Source code

Source code for a C language implementation of a "zlib" compliant
library is available at ftp://ftp.uu.net/pub/archiving/zip/zlib/.
Security Considerations

A decoder that fails to check the ADLER32 checksum value may be
subject to undetected data corruption.
Acknowledgements

Trademarks cited in this document are the property of their
respective owners.

Jean-Loup Gailly and Mark Adler designed the zlib format and wrote
the related software described in this specification. Glenn
Randers-Pehrson converted this document to RFC and HTML format.
Authors' Addresses

L. Peter Deutsch
Aladdin Enterprises
203 Santa Margarita Ave.
Menlo Park, CA 94025

Phone: (415) 322-0103 (AM only)
FAX: (415) 322-1734
EMail: [email protected]

Jean-Loup Gailly

EMail: [email protected]

Questions about the technical content of this specification can be
sent by email to

Jean-Loup Gailly [email protected] and
Mark Adler [email protected]

Editorial comments on this specification can be sent by email to

L. Peter Deutsch [email protected] and
Glenn Randers-Pehrson [email protected]

Deutsch & Gailly Informational [Page 8]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

Appendix: Rationale

8.1. Preset dictionaries

A preset dictionary is specially useful to compress short input
sequences. The compressor can take advantage of the dictionary
context to encode the input in a more compact manner. The
decompressor can be initialized with the appropriate context by
virtually decompressing a compressed version of the dictionary
without producing any output. However for certain compression
algorithms such as the deflate algorithm this operation can be
achieved without actually performing any decompression.

The compressor and the decompressor must use exactly the same
dictionary. The dictionary may be fixed or may be chosen among a
certain number of predefined dictionaries, according to the kind
of input data. The decompressor can determine which dictionary has
been chosen by the compressor by checking the dictionary
identifier. This document does not specify the contents of
predefined dictionaries, since the optimal dictionaries are
application specific. Standard data formats using this feature of
the zlib specification must precisely define the allowed
dictionaries.

8.2. The Adler-32 algorithm

The Adler-32 algorithm is much faster than the CRC32 algorithm yet
still provides an extremely low probability of undetected errors.

The modulo on unsigned long accumulators can be delayed for 5552
bytes, so the modulo operation time is negligible. If the bytes
are a, b, c, the second sum is 3a + 2b + c + 3, and so is position
and order sensitive, unlike the first sum, which is just a
checksum. That 65521 is prime is important to avoid a possible
large class of two-byte errors that leave the check unchanged.
(The Fletcher checksum uses 255, which is not prime and which also
makes the Fletcher check insensitive to single byte changes 0 <->
255.)

The sum s1 is initialized to 1 instead of zero to make the length
of the sequence part of s2, so that the length does not have to be
checked separately. (Any sequence of zeroes has a Fletcher
checksum of zero.)

Deutsch & Gailly Informational [Page 9]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

Appendix: Sample code

The following C code computes the Adler-32 checksum of a data buffer.
It is written for clarity, not for speed. The sample code is in the
ANSI C programming language. Non C users may find it easier to read
with these hints:

& Bitwise AND operator.
text
```
Bitwise right shift operator. When applied to an
```
#define BASE 65521 /* largest prime smaller than 65536 */

/*
Update a running Adler-32 checksum with the bytes buf[0..len-1]
and return the updated checksum. The Adler-32 checksum should be
initialized to 1.

Usage example:

unsigned long adler = 1L;

while (read_buffer(buffer, length) != EOF) {
adler = update_adler32(adler, buffer, length);
}
if (adler != original_adler) error();

*/
unsigned long update_adler32(unsigned long adler,
unsigned char *buf, int len)

{
unsigned long s1 = adler & 0xffff;
unsigned long s2 = (adler >> 16) & 0xffff;
int n;

for (n = 0; n < len; n++) {
s1 = (s1 + buf[n]) % BASE;
s2 = (s2 + s1) % BASE;
}
return (s2 << 16) + s1;

}

/* Return the adler32 of the bytes buf[0..len-1] */

Deutsch & Gailly Informational [Page 10]

RFC 1950 ZLIB Compressed Data Format Specification May 1996

text

unsigned long adler32(unsigned char *buf, int len)
  {
    return update_adler32(1L, buf, len);
  }

Deutsch & Gailly Informational [Page 11]

Src External Zlib 1.3.1 Doc Rfc1951 (`src/external/zlib-1.3.1/doc/rfc1951.txt`)

Network Working Group P. Deutsch
Request for Comments: 1951 Aladdin Enterprises
Category: Informational May 1996

text

DEFLATE Compressed Data Format Specification version 1.3

Status of This Memo

This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

IESG Note:

The IESG takes no position on the validity of any Intellectual
Property Rights statements contained in this document.

Notices

A pointer to the latest version of this and related documentation in
HTML format can be found at the URL
ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html.

Abstract

This specification defines a lossless compressed data format that
compresses data using a combination of the LZ77 algorithm and Huffman
coding, with efficiency comparable to the best currently available
general-purpose compression methods. The data can be produced or
consumed, even for an arbitrarily long sequentially presented input
data stream, using only an a priori bounded amount of intermediate
storage. The format can be implemented readily in a manner not
covered by patents.

Deutsch Informational [Page 1]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

Table of Contents

Introduction ................................................... 2
1.1. Purpose ................................................... 2
1.2. Intended audience ......................................... 3
1.3. Scope ..................................................... 3
1.4. Compliance ................................................ 3
1.5. Definitions of terms and conventions used ................ 3
1.6. Changes from previous versions ............................ 4
Compressed representation overview ............................. 4
Detailed specification ......................................... 5
3.1. Overall conventions ....................................... 5
3.1.1. Packing into bytes .................................. 5

3.2. Compressed block format ................................... 6
3.2.1. Synopsis of prefix and Huffman coding ............... 6
3.2.2. Use of Huffman coding in the "deflate" format ....... 7
3.2.3. Details of block format ............................. 9
3.2.4. Non-compressed blocks (BTYPE=00) ................... 11
3.2.5. Compressed blocks (length and distance codes) ...... 11
3.2.6. Compression with fixed Huffman codes (BTYPE=01) .... 12
3.2.7. Compression with dynamic Huffman codes (BTYPE=10) .. 13

3.3. Compliance ............................................... 14
Compression algorithm details ................................. 14
References .................................................... 16
Security Considerations ....................................... 16
Source code ................................................... 16
Acknowledgements .............................................. 16
Author's Address .............................................. 17
Introduction

1.1. Purpose

The purpose of this specification is to define a lossless
compressed data format that:
* Is independent of CPU type, operating system, file system,
and character set, and hence can be used for interchange;
* Can be produced or consumed, even for an arbitrarily long
sequentially presented input data stream, using only an a
priori bounded amount of intermediate storage, and hence
can be used in data communications or similar structures
such as Unix filters;
* Compresses data with efficiency comparable to the best
currently available general-purpose compression methods,
and in particular considerably better than the "compress"
program;
* Can be implemented readily in a manner not covered by
patents, and hence can be practiced freely;

Deutsch Informational [Page 2]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

* Is compatible with the file format produced by the current
        widely used gzip utility, in that conforming decompressors
        will be able to read data produced by the existing gzip
        compressor.

  The data format defined by this specification does not attempt to:

      * Allow random access to compressed data;
      * Compress specialized data (e.g., raster graphics) as well
        as the best currently available specialized algorithms.

  A simple counting argument shows that no lossless compression
  algorithm can compress every possible input data set.  For the
  format defined here, the worst case expansion is 5 bytes per 32K-
  byte block, i.e., a size increase of 0.015% for large data sets.
  English text usually compresses by a factor of 2.5 to 3;
  executable files usually compress somewhat less; graphical data
  such as raster images may compress much more.

1.2. Intended audience

text

This specification is intended for use by implementors of software
  to compress data into "deflate" format and/or decompress data from
  "deflate" format.

  The text of the specification assumes a basic background in
  programming at the level of bits and other primitive data
  representations.  Familiarity with the technique of Huffman coding
  is helpful but not required.

1.3. Scope

text

The specification specifies a method for representing a sequence
  of bytes as a (usually shorter) sequence of bits, and a method for
  packing the latter bit sequence into bytes.

1.4. Compliance

text

Unless otherwise indicated below, a compliant decompressor must be
  able to accept and decompress any data set that conforms to all
  the specifications presented here; a compliant compressor must
  produce data sets that conform to all the specifications presented
  here.

1.5. Definitions of terms and conventions used

text

Byte: 8 bits stored or transmitted as a unit (same as an octet).
  For this specification, a byte is exactly 8 bits, even on machines

Deutsch Informational [Page 3]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

which store a character on a number of bits different from eight.
  See below, for the numbering of bits within a byte.

  String: a sequence of arbitrary bytes.

1.6. Changes from previous versions

text

There have been no technical changes to the deflate format since
  version 1.1 of this specification.  In version 1.2, some
  terminology was changed.  Version 1.3 is a conversion of the
  specification to RFC style.

Compressed representation overview

A compressed data set consists of a series of blocks, corresponding
to successive blocks of input data. The block sizes are arbitrary,
except that non-compressible blocks are limited to 65,535 bytes.

Each block is compressed using a combination of the LZ77 algorithm
and Huffman coding. The Huffman trees for each block are independent
of those for previous or subsequent blocks; the LZ77 algorithm may
use a reference to a duplicated string occurring in a previous block,
up to 32K input bytes before.

Each block consists of two parts: a pair of Huffman code trees that
describe the representation of the compressed data part, and a
compressed data part. (The Huffman trees themselves are compressed
using Huffman encoding.) The compressed data consists of a series of
elements of two types: literal bytes (of strings that have not been
detected as duplicated within the previous 32K input bytes), and
pointers to duplicated strings, where a pointer is represented as a
pair <length, backward distance>. The representation used in the
"deflate" format limits distances to 32K bytes and lengths to 258
bytes, but does not limit the size of a block, except for
uncompressible blocks, which are limited as noted above.

Each type of value (literals, distances, and lengths) in the
compressed data is represented using a Huffman code, using one code
tree for literals and lengths and a separate code tree for distances.
The code trees for each block appear in a compact form just before
the compressed data for that block.

Deutsch Informational [Page 4]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

Detailed specification

3.1. Overall conventions In the diagrams below, a box like this:
text
```
+---+
  |   | <-- the vertical bars might be missing
  +---+
```
represents one byte; a box like this:
text
```
+==============+
  |              |
  +==============+
```
represents a variable number of bytes.

Bytes stored within a computer do not have a "bit order", since
they are always treated as a unit. However, a byte considered as
an integer between 0 and 255 does have a most- and least-
significant bit, and since we write numbers with the most-
significant digit on the left, we also write bytes with the most-
significant bit on the left. In the diagrams below, we number the
bits of a byte so that bit 0 is the least-significant bit, i.e.,
the bits are numbered:
text
```
+--------+
  |76543210|
  +--------+
```
Within a computer, a number may occupy multiple bytes. All
multi-byte numbers in the format described here are stored with
the least-significant byte first (at the lower memory address).
For example, the decimal number 520 is stored as:
text
```
0        1
  +--------+--------+
  |00001000|00000010|
  +--------+--------+
   ^        ^
   |        |
   |        + more significant byte = 2 x 256
   + less significant byte = 8
```
3.1.1. Packing into bytes
text
```
This document does not address the issue of the order in which
  bits of a byte are transmitted on a bit-sequential medium,
  since the final data format described here is byte- rather than
```

Deutsch Informational [Page 5]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

bit-oriented.  However, we describe the compressed block format
     in below, as a sequence of data elements of various bit
     lengths, not a sequence of bytes.  We must therefore specify
     how to pack these data elements into bytes to form the final
     compressed byte sequence:

         * Data elements are packed into bytes in order of
           increasing bit number within the byte, i.e., starting
           with the least-significant bit of the byte.
         * Data elements other than Huffman codes are packed
           starting with the least-significant bit of the data
           element.
         * Huffman codes are packed starting with the most-
           significant bit of the code.

     In other words, if one were to print out the compressed data as
     a sequence of bytes, starting with the first byte at the
     *right* margin and proceeding to the *left*, with the most-
     significant bit of each byte on the left as usual, one would be
     able to parse the result from right to left, with fixed-width
     elements in the correct MSB-to-LSB order and Huffman codes in
     bit-reversed order (i.e., with the first bit of the code in the
     relative LSB position).

3.2. Compressed block format

text

3.2.1. Synopsis of prefix and Huffman coding

     Prefix coding represents symbols from an a priori known
     alphabet by bit sequences (codes), one code for each symbol, in
     a manner such that different symbols may be represented by bit
     sequences of different lengths, but a parser can always parse
     an encoded string unambiguously symbol-by-symbol.

     We define a prefix code in terms of a binary tree in which the
     two edges descending from each non-leaf node are labeled 0 and
     1 and in which the leaf nodes correspond one-for-one with (are
     labeled with) the symbols of the alphabet; then the code for a
     symbol is the sequence of 0's and 1's on the edges leading from
     the root to the leaf labeled with that symbol.  For example:

Deutsch Informational [Page 6]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

/\              Symbol    Code
                     0  1             ------    ----
                    /    \                A      00
                   /\     B               B       1
                  0  1                    C     011
                 /    \                   D     010
                A     /\
                     0  1
                    /    \
                   D      C

     A parser can decode the next symbol from an encoded input
     stream by walking down the tree from the root, at each step
     choosing the edge corresponding to the next input bit.

     Given an alphabet with known symbol frequencies, the Huffman
     algorithm allows the construction of an optimal prefix code
     (one which represents strings with those symbol frequencies
     using the fewest bits of any possible prefix codes for that
     alphabet).  Such a code is called a Huffman code.  (See
     reference [1] in Chapter 5, references for additional
     information on Huffman codes.)

     Note that in the "deflate" format, the Huffman codes for the
     various alphabets must not exceed certain maximum code lengths.
     This constraint complicates the algorithm for computing code
     lengths from symbol frequencies.  Again, see Chapter 5,
     references for details.

  3.2.2. Use of Huffman coding in the "deflate" format

     The Huffman codes used for each alphabet in the "deflate"
     format have two additional rules:

         * All codes of a given bit length have lexicographically
           consecutive values, in the same order as the symbols
           they represent;

         * Shorter codes lexicographically precede longer codes.

Deutsch Informational [Page 7]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

We could recode the example above to follow this rule as
     follows, assuming that the order of the alphabet is ABCD:

        Symbol  Code
        ------  ----
        A       10
        B       0
        C       110
        D       111

     I.e., 0 precedes 10 which precedes 11x, and 110 and 111 are
     lexicographically consecutive.

     Given this rule, we can define the Huffman code for an alphabet
     just by giving the bit lengths of the codes for each symbol of
     the alphabet in order; this is sufficient to determine the
     actual codes.  In our example, the code is completely defined
     by the sequence of bit lengths (2, 1, 3, 3).  The following
     algorithm generates the codes as integers, intended to be read
     from most- to least-significant bit.  The code lengths are
     initially in tree[I].Len; the codes are produced in
     tree[I].Code.

     1)  Count the number of codes for each code length.  Let
         bl_count[N] be the number of codes of length N, N >= 1.

     2)  Find the numerical value of the smallest code for each
         code length:

            code = 0;
            bl_count[0] = 0;
            for (bits = 1; bits <= MAX_BITS; bits++) {
                code = (code + bl_count[bits-1]) << 1;
                next_code[bits] = code;
            }

     3)  Assign numerical values to all codes, using consecutive
         values for all codes of the same length with the base
         values determined at step 2. Codes that are never used
         (which have a bit length of zero) must not be assigned a
         value.

            for (n = 0;  n <= max_code; n++) {
                len = tree[n].Len;
                if (len != 0) {
                    tree[n].Code = next_code[len];
                    next_code[len]++;
                }

Deutsch Informational [Page 8]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

}

     Example:

     Consider the alphabet ABCDEFGH, with bit lengths (3, 3, 3, 3,
     3, 2, 4, 4).  After step 1, we have:

        N      bl_count[N]
        -      -----------
        2      1
        3      5
        4      2

     Step 2 computes the following next_code values:

        N      next_code[N]
        -      ------------
        1      0
        2      0
        3      2
        4      14

     Step 3 produces the following code values:

        Symbol Length   Code
        ------ ------   ----
        A       3        010
        B       3        011
        C       3        100
        D       3        101
        E       3        110
        F       2         00
        G       4       1110
        H       4       1111

  3.2.3. Details of block format

     Each block of compressed data begins with 3 header bits
     containing the following data:

        first bit       BFINAL
        next 2 bits     BTYPE

     Note that the header bits do not necessarily begin on a byte
     boundary, since a block does not necessarily occupy an integral
     number of bytes.

Deutsch Informational [Page 9]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

BFINAL is set if and only if this is the last block of the data
     set.

     BTYPE specifies how the data are compressed, as follows:

        00 - no compression
        01 - compressed with fixed Huffman codes
        10 - compressed with dynamic Huffman codes
        11 - reserved (error)

     The only difference between the two compressed cases is how the
     Huffman codes for the literal/length and distance alphabets are
     defined.

     In all cases, the decoding algorithm for the actual data is as
     follows:

        do
           read block header from input stream.
           if stored with no compression
              skip any remaining bits in current partially
                 processed byte
              read LEN and NLEN (see next section)
              copy LEN bytes of data to output
           otherwise
              if compressed with dynamic Huffman codes
                 read representation of code trees (see
                    subsection below)
              loop (until end of block code recognized)
                 decode literal/length value from input stream
                 if value < 256
                    copy value (literal byte) to output stream
                 otherwise
                    if value = end of block (256)
                       break from loop
                    otherwise (value = 257..285)
                       decode distance from input stream

                       move backwards distance bytes in the output
                       stream, and copy length bytes from this
                       position to the output stream.
              end loop
        while not last block

     Note that a duplicated string reference may refer to a string
     in a previous block; i.e., the backward distance may cross one
     or more block boundaries.  However a distance cannot refer past
     the beginning of the output stream.  (An application using a

Deutsch Informational [Page 10]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

preset dictionary might discard part of the output stream; a
     distance can refer to that part of the output stream anyway)
     Note also that the referenced string may overlap the current
     position; for example, if the last 2 bytes decoded have values
     X and Y, a string reference with <length = 5, distance = 2>
     adds X,Y,X,Y,X to the output stream.

     We now specify each compression method in turn.

  3.2.4. Non-compressed blocks (BTYPE=00)

     Any bits of input up to the next byte boundary are ignored.
     The rest of the block consists of the following information:

          0   1   2   3   4...
        +---+---+---+---+================================+
        |  LEN  | NLEN  |... LEN bytes of literal data...|
        +---+---+---+---+================================+

     LEN is the number of data bytes in the block.  NLEN is the
     one's complement of LEN.

  3.2.5. Compressed blocks (length and distance codes)

     As noted above, encoded data blocks in the "deflate" format
     consist of sequences of symbols drawn from three conceptually
     distinct alphabets: either literal bytes, from the alphabet of
     byte values (0..255), or <length, backward distance> pairs,
     where the length is drawn from (3..258) and the distance is
     drawn from (1..32,768).  In fact, the literal and length
     alphabets are merged into a single alphabet (0..285), where
     values 0..255 represent literal bytes, the value 256 indicates
     end-of-block, and values 257..285 represent length codes
     (possibly in conjunction with extra bits following the symbol
     code) as follows:

Deutsch Informational [Page 11]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

Extra               Extra               Extra
        Code Bits Length(s) Code Bits Lengths   Code Bits Length(s)
        ---- ---- ------     ---- ---- -------   ---- ---- -------
         257   0     3       267   1   15,16     277   4   67-82
         258   0     4       268   1   17,18     278   4   83-98
         259   0     5       269   2   19-22     279   4   99-114
         260   0     6       270   2   23-26     280   4  115-130
         261   0     7       271   2   27-30     281   5  131-162
         262   0     8       272   2   31-34     282   5  163-194
         263   0     9       273   3   35-42     283   5  195-226
         264   0    10       274   3   43-50     284   5  227-257
         265   1  11,12      275   3   51-58     285   0    258
         266   1  13,14      276   3   59-66

     The extra bits should be interpreted as a machine integer
     stored with the most-significant bit first, e.g., bits 1110
     represent the value 14.

              Extra           Extra               Extra
         Code Bits Dist  Code Bits   Dist     Code Bits Distance
         ---- ---- ----  ---- ----  ------    ---- ---- --------
           0   0    1     10   4     33-48    20    9   1025-1536
           1   0    2     11   4     49-64    21    9   1537-2048
           2   0    3     12   5     65-96    22   10   2049-3072
           3   0    4     13   5     97-128   23   10   3073-4096
           4   1   5,6    14   6    129-192   24   11   4097-6144
           5   1   7,8    15   6    193-256   25   11   6145-8192
           6   2   9-12   16   7    257-384   26   12  8193-12288
           7   2  13-16   17   7    385-512   27   12 12289-16384
           8   3  17-24   18   8    513-768   28   13 16385-24576
           9   3  25-32   19   8   769-1024   29   13 24577-32768

  3.2.6. Compression with fixed Huffman codes (BTYPE=01)

     The Huffman codes for the two alphabets are fixed, and are not
     represented explicitly in the data.  The Huffman code lengths
     for the literal/length alphabet are:

               Lit Value    Bits        Codes
               ---------    ----        -----
                 0 - 143     8          00110000 through
                                        10111111
               144 - 255     9          110010000 through
                                        111111111
               256 - 279     7          0000000 through
                                        0010111
               280 - 287     8          11000000 through
                                        11000111

Deutsch Informational [Page 12]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

The code lengths are sufficient to generate the actual codes,
     as described above; we show the codes in the table for added
     clarity.  Literal/length values 286-287 will never actually
     occur in the compressed data, but participate in the code
     construction.

     Distance codes 0-31 are represented by (fixed-length) 5-bit
     codes, with possible additional bits as shown in the table
     shown in Paragraph 3.2.5, above.  Note that distance codes 30-
     31 will never actually occur in the compressed data.

  3.2.7. Compression with dynamic Huffman codes (BTYPE=10)

     The Huffman codes for the two alphabets appear in the block
     immediately after the header bits and before the actual
     compressed data, first the literal/length code and then the
     distance code.  Each code is defined by a sequence of code
     lengths, as discussed in Paragraph 3.2.2, above.  For even
     greater compactness, the code length sequences themselves are
     compressed using a Huffman code.  The alphabet for code lengths
     is as follows:

           0 - 15: Represent code lengths of 0 - 15
               16: Copy the previous code length 3 - 6 times.
                   The next 2 bits indicate repeat length
                         (0 = 3, ... , 3 = 6)
                      Example:  Codes 8, 16 (+2 bits 11),
                                16 (+2 bits 10) will expand to
                                12 code lengths of 8 (1 + 6 + 5)
               17: Repeat a code length of 0 for 3 - 10 times.
                   (3 bits of length)
               18: Repeat a code length of 0 for 11 - 138 times
                   (7 bits of length)

     A code length of 0 indicates that the corresponding symbol in
     the literal/length or distance alphabet will not occur in the
     block, and should not participate in the Huffman code
     construction algorithm given earlier.  If only one distance
     code is used, it is encoded using one bit, not zero bits; in
     this case there is a single code length of one, with one unused
     code.  One distance code of zero bits means that there are no
     distance codes used at all (the data is all literals).

     We can now define the format of the block:

           5 Bits: HLIT, # of Literal/Length codes - 257 (257 - 286)
           5 Bits: HDIST, # of Distance codes - 1        (1 - 32)
           4 Bits: HCLEN, # of Code Length codes - 4     (4 - 19)

Deutsch Informational [Page 13]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

text

(HCLEN + 4) x 3 bits: code lengths for the code length
              alphabet given just above, in the order: 16, 17, 18,
              0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15

              These code lengths are interpreted as 3-bit integers
              (0-7); as above, a code length of 0 means the
              corresponding symbol (literal/length or distance code
              length) is not used.

           HLIT + 257 code lengths for the literal/length alphabet,
              encoded using the code length Huffman code

           HDIST + 1 code lengths for the distance alphabet,
              encoded using the code length Huffman code

           The actual compressed data of the block,
              encoded using the literal/length and distance Huffman
              codes

           The literal/length symbol 256 (end of data),
              encoded using the literal/length Huffman code

     The code length repeat codes can cross from HLIT + 257 to the
     HDIST + 1 code lengths.  In other words, all code lengths form
     a single sequence of HLIT + HDIST + 258 values.

3.3. Compliance

text

A compressor may limit further the ranges of values specified in
  the previous section and still be compliant; for example, it may
  limit the range of backward pointers to some value smaller than
  32K.  Similarly, a compressor may limit the size of blocks so that
  a compressible block fits in memory.

  A compliant decompressor must accept the full range of possible
  values defined in the previous section, and must accept blocks of
  arbitrary size.

Compression algorithm details

While it is the intent of this document to define the "deflate"
compressed data format without reference to any particular
compression algorithm, the format is related to the compressed
formats produced by LZ77 (Lempel-Ziv 1977, see reference [2] below);
since many variations of LZ77 are patented, it is strongly
recommended that the implementor of a compressor follow the general
algorithm presented here, which is known not to be patented per se.
The material in this section is not part of the definition of the

Deutsch Informational [Page 14]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

specification per se, and a compressor need not follow it in order to
be compliant.

The compressor terminates a block when it determines that starting a
new block with fresh trees would be useful, or when the block size
fills up the compressor's block buffer.

The compressor uses a chained hash table to find duplicated strings,
using a hash function that operates on 3-byte sequences. At any
given point during compression, let XYZ be the next 3 input bytes to
be examined (not necessarily all different, of course). First, the
compressor examines the hash chain for XYZ. If the chain is empty,
the compressor simply writes out X as a literal byte and advances one
byte in the input. If the hash chain is not empty, indicating that
the sequence XYZ (or, if we are unlucky, some other 3 bytes with the
same hash function value) has occurred recently, the compressor
compares all strings on the XYZ hash chain with the actual input data
sequence starting at the current point, and selects the longest
match.

The compressor searches the hash chains starting with the most recent
strings, to favor small distances and thus take advantage of the
Huffman encoding. The hash chains are singly linked. There are no
deletions from the hash chains; the algorithm simply discards matches
that are too old. To avoid a worst-case situation, very long hash
chains are arbitrarily truncated at a certain length, determined by a
run-time parameter.

To improve overall compression, the compressor optionally defers the
selection of matches ("lazy matching"): after a match of length N has
been found, the compressor searches for a longer match starting at
the next input byte. If it finds a longer match, it truncates the
previous match to a length of one (thus producing a single literal
byte) and then emits the longer match. Otherwise, it emits the
original match, and, as described above, advances N bytes before
continuing.

Run-time parameters also control this "lazy match" procedure. If
compression ratio is most important, the compressor attempts a
complete second search regardless of the length of the first match.
In the normal case, if the current match is "long enough", the
compressor reduces the search for a longer match, thus speeding up
the process. If speed is most important, the compressor inserts new
strings in the hash table only when no match was found, or when the
match is not "too long". This degrades the compression ratio but
saves time since there are both fewer insertions and fewer searches.

Deutsch Informational [Page 15]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

References

[1] Huffman, D. A., "A Method for the Construction of Minimum
Redundancy Codes", Proceedings of the Institute of Radio
Engineers, September 1952, Volume 40, Number 9, pp. 1098-1101.

[2] Ziv J., Lempel A., "A Universal Algorithm for Sequential Data
Compression", IEEE Transactions on Information Theory, Vol. 23,
No. 3, pp. 337-343.

[3] Gailly, J.-L., and Adler, M., ZLIB documentation and sources,
available in ftp://ftp.uu.net/pub/archiving/zip/doc/

[4] Gailly, J.-L., and Adler, M., GZIP documentation and sources,
available as gzip-*.tar in ftp://prep.ai.mit.edu/pub/gnu/

[5] Schwartz, E. S., and Kallick, B. "Generating a canonical prefix
encoding." Comm. ACM, 7,3 (Mar. 1964), pp. 166-169.

[6] Hirschberg and Lelewer, "Efficient decoding of prefix codes,"
Comm. ACM, 33,4, April 1990, pp. 449-459.
Security Considerations

Any data compression method involves the reduction of redundancy in
the data. Consequently, any corruption of the data is likely to have
severe effects and be difficult to correct. Uncompressed text, on
the other hand, will probably still be readable despite the presence
of some corrupted bytes.

It is recommended that systems using this data format provide some
means of validating the integrity of the compressed data. See
reference [3], for example.
Source code

Source code for a C language implementation of a "deflate" compliant
compressor and decompressor is available within the zlib package at
ftp://ftp.uu.net/pub/archiving/zip/zlib/.
Acknowledgements

Trademarks cited in this document are the property of their
respective owners.

Phil Katz designed the deflate format. Jean-Loup Gailly and Mark
Adler wrote the related software described in this specification.
Glenn Randers-Pehrson converted this document to RFC and HTML format.

Deutsch Informational [Page 16]

RFC 1951 DEFLATE Compressed Data Format Specification May 1996

Author's Address

L. Peter Deutsch
Aladdin Enterprises
203 Santa Margarita Ave.
Menlo Park, CA 94025

Phone: (415) 322-0103 (AM only)
FAX: (415) 322-1734
EMail: [email protected]

Questions about the technical content of this specification can be
sent by email to:

Jean-Loup Gailly [email protected] and
Mark Adler [email protected]

Editorial comments on this specification can be sent by email to:

L. Peter Deutsch [email protected] and
Glenn Randers-Pehrson [email protected]

Deutsch Informational [Page 17]

Src External Zlib 1.3.1 Doc Rfc1952 (`src/external/zlib-1.3.1/doc/rfc1952.txt`)

Network Working Group P. Deutsch
Request for Comments: 1952 Aladdin Enterprises
Category: Informational May 1996

text

GZIP file format specification version 4.3

Status of This Memo

This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.

IESG Note:

The IESG takes no position on the validity of any Intellectual
Property Rights statements contained in this document.

Notices

A pointer to the latest version of this and related documentation in
HTML format can be found at the URL
ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html.

Abstract

This specification defines a lossless compressed data format that is
compatible with the widely used GZIP utility. The format includes a
cyclic redundancy check value for detecting data corruption. The
format presently uses the DEFLATE method of compression but can be
easily extended to use other compression methods. The format can be
implemented readily in a manner not covered by patents.

Deutsch Informational [Page 1]

RFC 1952 GZIP File Format Specification May 1996

Table of Contents

Introduction ................................................... 2
1.1. Purpose ................................................... 2
1.2. Intended audience ......................................... 3
1.3. Scope ..................................................... 3
1.4. Compliance ................................................ 3
1.5. Definitions of terms and conventions used ................. 3
1.6. Changes from previous versions ............................ 3
Detailed specification ......................................... 4
2.1. Overall conventions ....................................... 4
2.2. File format ............................................... 5
2.3. Member format ............................................. 5
2.3.1. Member header and trailer ........................... 6
2.3.1.1. Extra field ................................... 8
2.3.1.2. Compliance .................................... 9
1. References .................................................. 9
2. Security Considerations .................................... 10
3. Acknowledgements ........................................... 10
4. Author's Address ........................................... 10
5. Appendix: Jean-Loup Gailly's gzip utility .................. 11
6. Appendix: Sample CRC Code .................................. 11

Introduction

1.1. Purpose

The purpose of this specification is to define a lossless
compressed data format that:

text

* Is independent of CPU type, operating system, file system,
     and character set, and hence can be used for interchange;
   * Can compress or decompress a data stream (as opposed to a
     randomly accessible file) to produce another data stream,
     using only an a priori bounded amount of intermediate
     storage, and hence can be used in data communications or
     similar structures such as Unix filters;
   * Compresses data with efficiency comparable to the best
     currently available general-purpose compression methods,
     and in particular considerably better than the "compress"
     program;
   * Can be implemented readily in a manner not covered by
     patents, and hence can be practiced freely;
   * Is compatible with the file format produced by the current
     widely used gzip utility, in that conforming decompressors
     will be able to read data produced by the existing gzip
     compressor.

Deutsch Informational [Page 2]

RFC 1952 GZIP File Format Specification May 1996

text

The data format defined by this specification does not attempt to:

      * Provide random access to compressed data;
      * Compress specialized data (e.g., raster graphics) as well as
        the best currently available specialized algorithms.

1.2. Intended audience

text

This specification is intended for use by implementors of software
  to compress data into gzip format and/or decompress data from gzip
  format.

  The text of the specification assumes a basic background in
  programming at the level of bits and other primitive data
  representations.

1.3. Scope

text

The specification specifies a compression method and a file format
  (the latter assuming only that a file can store a sequence of
  arbitrary bytes).  It does not specify any particular interface to
  a file system or anything about character sets or encodings
  (except for file names and comments, which are optional).

1.4. Compliance

text

Unless otherwise indicated below, a compliant decompressor must be
  able to accept and decompress any file that conforms to all the
  specifications presented here; a compliant compressor must produce
  files that conform to all the specifications presented here.  The
  material in the appendices is not part of the specification per se
  and is not relevant to compliance.

1.5. Definitions of terms and conventions used

text

byte: 8 bits stored or transmitted as a unit (same as an octet).
  (For this specification, a byte is exactly 8 bits, even on
  machines which store a character on a number of bits different
  from 8.)  See below for the numbering of bits within a byte.

1.6. Changes from previous versions

text

There have been no technical changes to the gzip format since
  version 4.1 of this specification.  In version 4.2, some
  terminology was changed, and the sample CRC code was rewritten for
  clarity and to eliminate the requirement for the caller to do pre-
  and post-conditioning.  Version 4.3 is a conversion of the
  specification to RFC style.

Deutsch Informational [Page 3]

RFC 1952 GZIP File Format Specification May 1996

Detailed specification

2.1. Overall conventions

In the diagrams below, a box like this:
text
```
+---+
  |   | <-- the vertical bars might be missing
  +---+
```
represents one byte; a box like this:
text
```
+==============+
  |              |
  +==============+
```
represents a variable number of bytes.

Bytes stored within a computer do not have a "bit order", since
they are always treated as a unit. However, a byte considered as
an integer between 0 and 255 does have a most- and least-
significant bit, and since we write numbers with the most-
significant digit on the left, we also write bytes with the most-
significant bit on the left. In the diagrams below, we number the
bits of a byte so that bit 0 is the least-significant bit, i.e.,
the bits are numbered:
text
```
+--------+
  |76543210|
  +--------+
```
This document does not address the issue of the order in which
bits of a byte are transmitted on a bit-sequential medium, since
the data format described here is byte- rather than bit-oriented.

Within a computer, a number may occupy multiple bytes. All
multi-byte numbers in the format described here are stored with
the least-significant byte first (at the lower memory address).
For example, the decimal number 520 is stored as:
text
```
0        1
  +--------+--------+
  |00001000|00000010|
  +--------+--------+
   ^        ^
   |        |
   |        + more significant byte = 2 x 256
   + less significant byte = 8
```

Deutsch Informational [Page 4]

RFC 1952 GZIP File Format Specification May 1996

2.2. File format

text

A gzip file consists of a series of "members" (compressed data
  sets).  The format of each member is specified in the following
  section.  The members simply appear one after another in the file,
  with no additional information before, between, or after them.

2.3. Member format

text

Each member has the following structure:

     +---+---+---+---+---+---+---+---+---+---+
     |ID1|ID2|CM |FLG|     MTIME     |XFL|OS | (more-->)
     +---+---+---+---+---+---+---+---+---+---+

  (if FLG.FEXTRA set)

     +---+---+=================================+
     | XLEN  |...XLEN bytes of "extra field"...| (more-->)
     +---+---+=================================+

  (if FLG.FNAME set)

     +=========================================+
     |...original file name, zero-terminated...| (more-->)
     +=========================================+

  (if FLG.FCOMMENT set)

     +===================================+
     |...file comment, zero-terminated...| (more-->)
     +===================================+

  (if FLG.FHCRC set)

     +---+---+
     | CRC16 |
     +---+---+

     +=======================+
     |...compressed blocks...| (more-->)
     +=======================+

       0   1   2   3   4   5   6   7
     +---+---+---+---+---+---+---+---+
     |     CRC32     |     ISIZE     |
     +---+---+---+---+---+---+---+---+

Deutsch Informational [Page 5]

RFC 1952 GZIP File Format Specification May 1996

text

2.3.1. Member header and trailer

     ID1 (IDentification 1)
     ID2 (IDentification 2)
        These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139
        (0x8b, \213), to identify the file as being in gzip format.

     CM (Compression Method)
        This identifies the compression method used in the file.  CM
        = 0-7 are reserved.  CM = 8 denotes the "deflate"
        compression method, which is the one customarily used by
        gzip and which is documented elsewhere.

     FLG (FLaGs)
        This flag byte is divided into individual bits as follows:

           bit 0   FTEXT
           bit 1   FHCRC
           bit 2   FEXTRA
           bit 3   FNAME
           bit 4   FCOMMENT
           bit 5   reserved
           bit 6   reserved
           bit 7   reserved

        If FTEXT is set, the file is probably ASCII text.  This is
        an optional indication, which the compressor may set by
        checking a small amount of the input data to see whether any
        non-ASCII characters are present.  In case of doubt, FTEXT
        is cleared, indicating binary data. For systems which have
        different file formats for ascii text and binary data, the
        decompressor can use FTEXT to choose the appropriate format.
        We deliberately do not specify the algorithm used to set
        this bit, since a compressor always has the option of
        leaving it cleared and a decompressor always has the option
        of ignoring it and letting some other program handle issues
        of data conversion.

        If FHCRC is set, a CRC16 for the gzip header is present,
        immediately before the compressed data. The CRC16 consists
        of the two least significant bytes of the CRC32 for all
        bytes of the gzip header up to and not including the CRC16.
        [The FHCRC bit was never set by versions of gzip up to
        1.2.4, even though it was documented with a different
        meaning in gzip 1.2.4.]

        If FEXTRA is set, optional extra fields are present, as
        described in a following section.

Deutsch Informational [Page 6]

RFC 1952 GZIP File Format Specification May 1996

text

If FNAME is set, an original file name is present,
        terminated by a zero byte.  The name must consist of ISO
        8859-1 (LATIN-1) characters; on operating systems using
        EBCDIC or any other character set for file names, the name
        must be translated to the ISO LATIN-1 character set.  This
        is the original name of the file being compressed, with any
        directory components removed, and, if the file being
        compressed is on a file system with case insensitive names,
        forced to lower case. There is no original file name if the
        data was compressed from a source other than a named file;
        for example, if the source was stdin on a Unix system, there
        is no file name.

        If FCOMMENT is set, a zero-terminated file comment is
        present.  This comment is not interpreted; it is only
        intended for human consumption.  The comment must consist of
        ISO 8859-1 (LATIN-1) characters.  Line breaks should be
        denoted by a single line feed character (10 decimal).

        Reserved FLG bits must be zero.

     MTIME (Modification TIME)
        This gives the most recent modification time of the original
        file being compressed.  The time is in Unix format, i.e.,
        seconds since 00:00:00 GMT, Jan.  1, 1970.  (Note that this
        may cause problems for MS-DOS and other systems that use
        local rather than Universal time.)  If the compressed data
        did not come from a file, MTIME is set to the time at which
        compression started.  MTIME = 0 means no time stamp is
        available.

     XFL (eXtra FLags)
        These flags are available for use by specific compression
        methods.  The "deflate" method (CM = 8) sets these flags as
        follows:

           XFL = 2 - compressor used maximum compression,
                     slowest algorithm
           XFL = 4 - compressor used fastest algorithm

     OS (Operating System)
        This identifies the type of file system on which compression
        took place.  This may be useful in determining end-of-line
        convention for text files.  The currently defined values are
        as follows:

Deutsch Informational [Page 7]

RFC 1952 GZIP File Format Specification May 1996

text

0 - FAT filesystem (MS-DOS, OS/2, NT/Win32)
             1 - Amiga
             2 - VMS (or OpenVMS)
             3 - Unix
             4 - VM/CMS
             5 - Atari TOS
             6 - HPFS filesystem (OS/2, NT)
             7 - Macintosh
             8 - Z-System
             9 - CP/M
            10 - TOPS-20
            11 - NTFS filesystem (NT)
            12 - QDOS
            13 - Acorn RISCOS
           255 - unknown

     XLEN (eXtra LENgth)
        If FLG.FEXTRA is set, this gives the length of the optional
        extra field.  See below for details.

     CRC32 (CRC-32)
        This contains a Cyclic Redundancy Check value of the
        uncompressed data computed according to CRC-32 algorithm
        used in the ISO 3309 standard and in section 8.1.1.6.2 of
        ITU-T recommendation V.42.  (See http://www.iso.ch for
        ordering ISO documents. See gopher://info.itu.ch for an
        online version of ITU-T V.42.)

     ISIZE (Input SIZE)
        This contains the size of the original (uncompressed) input
        data modulo 2^32.

  2.3.1.1. Extra field

     If the FLG.FEXTRA bit is set, an "extra field" is present in
     the header, with total length XLEN bytes.  It consists of a
     series of subfields, each of the form:

        +---+---+---+---+==================================+
        |SI1|SI2|  LEN  |... LEN bytes of subfield data ...|
        +---+---+---+---+==================================+

     SI1 and SI2 provide a subfield ID, typically two ASCII letters
     with some mnemonic value.  Jean-Loup Gailly
     <[email protected]> is maintaining a registry of subfield
     IDs; please send him any subfield ID you wish to use.  Subfield
     IDs with SI2 = 0 are reserved for future use.  The following
     IDs are currently defined:

Deutsch Informational [Page 8]

RFC 1952 GZIP File Format Specification May 1996

text

SI1         SI2         Data
        ----------  ----------  ----
        0x41 ('A')  0x70 ('P')  Apollo file type information

     LEN gives the length of the subfield data, excluding the 4
     initial bytes.

  2.3.1.2. Compliance

     A compliant compressor must produce files with correct ID1,
     ID2, CM, CRC32, and ISIZE, but may set all the other fields in
     the fixed-length part of the header to default values (255 for
     OS, 0 for all others).  The compressor must set all reserved
     bits to zero.

     A compliant decompressor must check ID1, ID2, and CM, and
     provide an error indication if any of these have incorrect
     values.  It must examine FEXTRA/XLEN, FNAME, FCOMMENT and FHCRC
     at least so it can skip over the optional fields if they are
     present.  It need not examine any other part of the header or
     trailer; in particular, a decompressor may ignore FTEXT and OS
     and always produce binary output, and still be compliant.  A
     compliant decompressor must give an error indication if any
     reserved bit is non-zero, since such a bit could indicate the
     presence of a new field that would cause subsequent data to be
     interpreted incorrectly.

References

[1] "Information Processing - 8-bit single-byte coded graphic
character sets - Part 1: Latin alphabet No.1" (ISO 8859-1:1987).
The ISO 8859-1 (Latin-1) character set is a superset of 7-bit
ASCII. Files defining this character set are available as
iso_8859-1.* in ftp://ftp.uu.net/graphics/png/documents/

[2] ISO 3309

[3] ITU-T recommendation V.42

[4] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification",
available in ftp://ftp.uu.net/pub/archiving/zip/doc/

[5] Gailly, J.-L., GZIP documentation, available as gzip-*.tar in
ftp://prep.ai.mit.edu/pub/gnu/

[6] Sarwate, D.V., "Computation of Cyclic Redundancy Checks via Table
Look-Up", Communications of the ACM, 31(8), pp.1008-1013.

Deutsch Informational [Page 9]

RFC 1952 GZIP File Format Specification May 1996

[7] Schwaderer, W.D., "CRC Calculation", April 85 PC Tech Journal,
pp.118-133.

[8] ftp://ftp.adelaide.edu.au/pub/rocksoft/papers/crc_v3.txt,
describing the CRC concept.

Security Considerations

Any data compression method involves the reduction of redundancy in
the data. Consequently, any corruption of the data is likely to have
severe effects and be difficult to correct. Uncompressed text, on
the other hand, will probably still be readable despite the presence
of some corrupted bytes.

It is recommended that systems using this data format provide some
means of validating the integrity of the compressed data, such as by
setting and checking the CRC-32 check value.
Acknowledgements

Trademarks cited in this document are the property of their
respective owners.

Jean-Loup Gailly designed the gzip format and wrote, with Mark Adler,
the related software described in this specification. Glenn
Randers-Pehrson converted this document to RFC and HTML format.
Author's Address

L. Peter Deutsch
Aladdin Enterprises
203 Santa Margarita Ave.
Menlo Park, CA 94025

Phone: (415) 322-0103 (AM only)
FAX: (415) 322-1734
EMail: [email protected]

Questions about the technical content of this specification can be
sent by email to:

Jean-Loup Gailly [email protected] and
Mark Adler [email protected]

Editorial comments on this specification can be sent by email to:

L. Peter Deutsch [email protected] and
Glenn Randers-Pehrson [email protected]

Deutsch Informational [Page 10]

RFC 1952 GZIP File Format Specification May 1996

Appendix: Jean-Loup Gailly's gzip utility

The most widely used implementation of gzip compression, and the
original documentation on which this specification is based, were
created by Jean-Loup Gailly [email protected]. Since this
implementation is a de facto standard, we mention some more of its
features here. Again, the material in this section is not part of
the specification per se, and implementations need not follow it to
be compliant.

When compressing or decompressing a file, gzip preserves the
protection, ownership, and modification time attributes on the local
file system, since there is no provision for representing protection
attributes in the gzip file format itself. Since the file format
includes a modification time, the gzip decompressor provides a
command line switch that assigns the modification time from the file,
rather than the local modification time of the compressed input, to
the decompressed output.
Appendix: Sample CRC Code

The following sample code represents a practical implementation of
the CRC (Cyclic Redundancy Check). (See also ISO 3309 and ITU-T V.42
for a formal specification.)

The sample code is in the ANSI C programming language. Non C users
may find it easier to read with these hints:

& Bitwise AND operator.
^ Bitwise exclusive-OR operator.
text
```
Bitwise right shift operator. When applied to an
```
/* Table of CRCs of all 8-bit messages. */
unsigned long crc_table[256];

/* Flag: has the table been computed? Initially false. */
int crc_table_computed = 0;

/* Make the table for a fast CRC. */
void make_crc_table(void)
{
unsigned long c;

Deutsch Informational [Page 11]

RFC 1952 GZIP File Format Specification May 1996

text

int n, k;
    for (n = 0; n < 256; n++) {
      c = (unsigned long) n;
      for (k = 0; k < 8; k++) {
        if (c & 1) {
          c = 0xedb88320L ^ (c >> 1);
        } else {
          c = c >> 1;
        }
      }
      crc_table[n] = c;
    }
    crc_table_computed = 1;
  }

  /*
     Update a running crc with the bytes buf[0..len-1] and return
   the updated crc. The crc should be initialized to zero. Pre- and
   post-conditioning (one's complement) is performed within this
   function so it shouldn't be done by the caller. Usage example:

     unsigned long crc = 0L;

     while (read_buffer(buffer, length) != EOF) {
       crc = update_crc(crc, buffer, length);
     }
     if (crc != original_crc) error();
  */
  unsigned long update_crc(unsigned long crc,
                  unsigned char *buf, int len)
  {
    unsigned long c = crc ^ 0xffffffffL;
    int n;

    if (!crc_table_computed)
      make_crc_table();
    for (n = 0; n < len; n++) {
      c = crc_table[(c ^ buf[n]) & 0xff] ^ (c >> 8);
    }
    return c ^ 0xffffffffL;
  }

  /* Return the CRC of the bytes buf[0..len-1]. */
  unsigned long crc(unsigned char *buf, int len)
  {
    return update_crc(0L, buf, len);
  }

Deutsch Informational [Page 12]

Src External Zlib 1.3.1 Doc Txtvsbin (`src/external/zlib-1.3.1/doc/txtvsbin.txt`)

A Fast Method for Identifying Plain Text Files

Introduction

Given a file coming from an unknown source, it is sometimes desirable
to find out whether the format of that file is plain text. Although
this may appear like a simple task, a fully accurate detection of the
file type requires heavy-duty semantic analysis on the file contents.
It is, however, possible to obtain satisfactory results by employing
various heuristics.

Previous versions of PKZip and other zip-compatible compression tools
were using a crude detection scheme: if more than 80% (4/5) of the bytes
found in a certain buffer are within the range [7..127], the file is
labeled as plain text, otherwise it is labeled as binary. A prominent
limitation of this scheme is the restriction to Latin-based alphabets.
Other alphabets, like Greek, Cyrillic or Asian, make extensive use of
the bytes within the range [128..255], and texts using these alphabets
are most often misidentified by this scheme; in other words, the rate
of false negatives is sometimes too high, which means that the recall
is low. Another weakness of this scheme is a reduced precision, due to
the false positives that may occur when binary files containing large
amounts of textual characters are misidentified as plain text.

In this article we propose a new, simple detection scheme that features
a much increased precision and a near-100% recall. This scheme is
designed to work on ASCII, Unicode and other ASCII-derived alphabets,
and it handles single-byte encodings (ISO-8859, MacRoman, KOI8, etc.)
and variable-sized encodings (ISO-2022, UTF-8, etc.). Wider encodings
(UCS-2/UTF-16 and UCS-4/UTF-32) are not handled, however.

The Algorithm

The algorithm works by dividing the set of bytecodes [0..255] into three
categories:

The allow list of textual bytecodes:
9 (TAB), 10 (LF), 13 (CR), 32 (SPACE) to 255.
The gray list of tolerated bytecodes:
7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB), 27 (ESC).
The block list of undesired, non-textual bytecodes:
0 (NUL) to 6, 14 to 31.

If a file contains at least one byte that belongs to the allow list and
no byte that belongs to the block list, then the file is categorized as
plain text; otherwise, it is categorized as binary. (The boundary case,
when the file is empty, automatically falls into the latter category.)

Rationale

The idea behind this algorithm relies on two observations.

The first observation is that, although the full range of 7-bit codes
[0..127] is properly specified by the ASCII standard, most control
characters in the range [0..31] are not used in practice. The only
widely-used, almost universally-portable control codes are 9 (TAB),
10 (LF) and 13 (CR). There are a few more control codes that are
recognized on a reduced range of platforms and text viewers/editors:
7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB) and 27 (ESC); but these
codes are rarely (if ever) used alone, without being accompanied by
some printable text. Even the newer, portable text formats such as
XML avoid using control characters outside the list mentioned here.

The second observation is that most of the binary files tend to contain
control characters, especially 0 (NUL). Even though the older text
detection schemes observe the presence of non-ASCII codes from the range
[128..255], the precision rarely has to suffer if this upper range is
labeled as textual, because the files that are genuinely binary tend to
contain both control characters and codes from the upper range. On the
other hand, the upper range needs to be labeled as textual, because it
is used by virtually all ASCII extensions. In particular, this range is
used for encoding non-Latin scripts.

Since there is no counting involved, other than simply observing the
presence or the absence of some byte values, the algorithm produces
consistent results, regardless what alphabet encoding is being used.
(If counting were involved, it could be possible to obtain different
results on a text encoded, say, using ISO-8859-16 versus UTF-8.)

There is an extra category of plain text files that are "polluted" with
one or more block-listed codes, either by mistake or by peculiar design
considerations. In such cases, a scheme that tolerates a small fraction
of block-listed codes would provide an increased recall (i.e. more true
positives). This, however, incurs a reduced precision overall, since
false positives are more likely to appear in binary files that contain
large chunks of textual data. Furthermore, "polluted" plain text should
be regarded as binary by general-purpose text detection schemes, because
general-purpose text processing algorithms might not be applicable.
Under this premise, it is safe to say that our detection method provides
a near-100% recall.

Experiments have been run on many files coming from various platforms
and applications. We tried plain text files, system logs, source code,
formatted office documents, compiled object code, etc. The results
confirm the optimistic assumptions about the capabilities of this
algorithm.

--
Cosmin Truta
Last updated: 2006-May-28

--- METRICS ---

Files Extracted: 35
Estimated Token Budget: ~43773 tokens
Canonical Reference: https://codewiki.google/github.com/ossec/ossec-hids

Docker & Self-Hosted Deployment Verified OCI

Technical Documentation: ossec/ossec-hids

Overview (`README.md`) Copy Section

Information about OSSEC

Current Releases Copy Chapter

Development Copy Chapter

Screenshots Copy Chapter

Help / Support Copy Chapter

Credits and Thanks Copy Chapter

License Copy Chapter

Overview (`.oss-scanner/README.md`) Copy Section

Anthropic OSS Scanner support

Enrollment (separate PR) Copy Chapter

Local smoke-test of the Dockerfile Copy Chapter

Overview (`.github/workflows/README.md`) Copy Section

GitHub Actions workflows

Overview (`contrib/selinux/README.md`) Copy Section

OSSEC SELinux module Copy Chapter

Quick fix for logrotate AVCs (#1948) Copy Section

Installing this module Copy Section

Building Copy Section

Bug reports & contribution Copy Section

Overview (`src/external/cJSON/README.md`) Copy Section

cJSON

Table of contents Copy Chapter

License Copy Chapter

Usage Copy Chapter

Welcome to cJSON. Copy Section

Building Copy Section

copying the source

CMake

Makefile

Meson

Vcpkg

Including cJSON Copy Section

Data Structure Copy Section

Working with the data structure Copy Section

Basic types

Arrays

Objects

Parsing JSON Copy Section

Printing JSON Copy Section

Example Copy Section

Printing

Parsing

Caveats Copy Section

Zero Character

Character Encoding

C Standard

Floating Point Numbers

Deep Nesting Of Arrays And Objects

Thread Safety

Case Sensitivity

Duplicate Object Members

Enjoy cJSON!

Overview (`src/tests/fuzz/README.md`) Copy Section

OSSEC fuzz harnesses (OSS-Fuzz)

Local build (clang + libFuzzer + ASan) Copy Chapter

Run over the seed corpus Copy Chapter

Overview (`src/tests/regressions/README.md`) Copy Section

Analysisd / pipeline regression tests

From `src/` after building the server target:

make TARGET=server -j$(nproc)

make regression

Or directly:

make -f tests/regressions/Makefile

make -f tests/regressions/Makefile check

Shared queue/thread unit tests (automated):

make -C shared/tests check

Interactive helpers (prime/hash/merge/ip) are optional:

make -C shared/tests legacy_helpers

Covered binaries (make check):

- issue_1079_last_events_uaf frequency last_events ownership

- issue_correlation_agent_shard location EventList isolation +

if_matched_sid cross-location

- issue_1748_sid_list_crash sid_prev_matched overflow + free callback

- issue_group_list_uaf Free_Eventinfo unlinks its group_prev_matched nodes

- issue_prev_matched_race if_matched_sid list lock across match threads

- issue_1274_auth_ipv6_key auth IPv6 key handling

- issue_2065_sender_counter sender_counter isolated from agent 0 rids

Overview (`README.md`)

Current Releases

Development

Screenshots

Help / Support

Credits and Thanks

License

Overview (`.oss-scanner/README.md`)

Enrollment (separate PR)

Local smoke-test of the Dockerfile

Overview (`.github/workflows/README.md`)

Overview (`contrib/selinux/README.md`)

OSSEC SELinux module

Quick fix for logrotate AVCs (#1948)

Installing this module

Building

Bug reports & contribution

Overview (`src/external/cJSON/README.md`)

Table of contents

License

Usage

Welcome to cJSON.

Building

Including cJSON

Data Structure

Working with the data structure

Parsing JSON

Printing JSON

Example

Caveats

Overview (`src/tests/fuzz/README.md`)

Local build (clang + libFuzzer + ASan)

Run over the seed corpus

Overview (`src/tests/regressions/README.md`)

- EventList history used by Search_LastEvents is per location-hashed shard.

- Ops: events_received, events_dropped, alerts_dropped, archives_dropped*,

raw_input_queue_, fts_queue_, and *_delta fields in ossec-analysisd.state.

Doc Active Response Internal (`doc/active-response-internal.txt`)

Doc Active Response (`doc/active-response.txt`)

Doc Logs (`doc/logs.txt`)

Doc Manage Agents (`doc/manage_agents.txt`)

Doc Manager (`doc/manager.txt`)

Doc Nmap (`doc/nmap.txt`)