watchman Documentation

Technical Documentation: facebook/watchman

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`website/README.md`) Copy Section

Website

This website is built using Docusaurus 2, a modern static website generator.

Installation Copy Section

$ yarn

Local Development Copy Section

$ yarn start

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

Build Copy Section

$ yarn build

This command generates static content into the build directory and can be served using any static contents hosting service.

Overview (`build/fbcode_builder/README.md`) Copy Section

Easy builds for Facebook projects

This directory contains tools designed to simplify continuous-integration
(and other builds) of Facebook open source projects. In particular, this helps
manage builds for cross-project dependencies.

The main entry point is the getdeps.py script. This script has several
subcommands, but the most notable is the build command. This will download
and build all dependencies for a project, and then build the project itself.

getdeps.py requires Python 3.6 or newer and enforces this at startup.

Deployment Copy Chapter

This directory is copied literally into a number of different Facebook open
source repositories. Any change made to code in this directory will be
automatically be replicated by our open source tooling into all GitHub hosted
repositories that use fbcode_builder. Typically this directory is copied
into the open source repositories as build/fbcode_builder/.

Vendoring dependencies for offline builds Copy Chapter

Distributions typically require builds to run without network access and to
ship third-party sources alongside the project. getdeps.py vendor --output-dir DIR project copies the source tree of every dependency that is
not satisfied by system packages to DIR/ and records what it
vendored in DIR/getdeps-vendor.txt. A later getdeps.py --vendor-dir DIR build project then takes those trees instead of fetching, and fails if a
dependency is missing from DIR rather than downloading it. Pass the same
--allow-system-packages and --no-tests options to both commands so they
agree on the dependency set.

Project Configuration Files

The manifests subdirectory contains configuration files for many different
projects, describing how to build each project. These files also list
dependencies between projects, enabling getdeps.py to build all dependencies
for a project before building the project itself.

Manifest features

A manifest may declare named features that toggle parts of its build. Each
feature appears as a feature_<name> variable that conditional sections such
as [dependencies.feature_<name>=on] or [cmake.defines.feature_<name>=off]
can gate on. Consumers of a project pick which features they want by attaching
a feature spec to the dep entry.

A project that supports features looks like:

[features]
default = rpc
rpc =

[cmake.defines.feature_rpc=off]
THRIFT_RPC=OFF

[dependencies.feature_rpc=on]
fizz
wangle

default lists features enabled when no consumer overrides; the other entries
declare valid feature names. There is no CLI flag for selecting features —
they are configured per dependency edge inside manifests.

A consumer's dep entry uses comma-separated tokens after =:

Resolution composes feature requests by union across all edges in the dep
graph, then checks prohibitions against the union; a conflict (one consumer
requests a feature another prohibits) is a hard error. Each project produces a
single build per invocation, so its resolved feature set is global to that
invocation. The resolved set is folded into the project's cache hash, so a
serialization-only build of fbthrift caches separately from a full build.

Since this directory is copied into many Facebook open source repositories,
it is also used to help share some CMake utility files across projects. The
CMake/ subdirectory contains a number of .cmake files that are shared by
the CMake-based build systems across several different projects.

Older Build Scripts

This directory also still contains a handful of older build scripts that
pre-date the current getdeps.py build system. Most of the other .py files
in this top directory, apart from getdeps.py itself, are from this older
build system. This older system is only used by a few remaining projects, and
new projects should generally use the newer getdeps.py script, by adding a
new configuration file in the manifests/ subdirectory.

Overview (`watchman/java/README.md`) Copy Section

Watchman Java Library

This provides Java bindings to the Watchman service.

Building

Make sure that you have buck installed. In this
folder, run:

buck fetch :watchman
buck build :watchman

The resulting JAR file is found in:
`buck-out/gen/watchman.jar'

To run the tests:

buck fetch :watchman-tests
buck test :watchman-lib

Overview (`watchman/node/README.md`) Copy Section

fb-watchman

fb-watchman is a filesystem watcher that uses the
Watchman file watching service from
Facebook.

Watchman provides file change notification services using very
efficient recursive watches and also allows more advanced change matching and
filesystem tree querying operations using
a powerful expression syntax.

Install Copy Chapter

You should install Watchman to make the most of this
module.

Then simply:

$ npm install fb-watchman

Key Concepts Copy Chapter

• Watchman recursively watches directories.
• Each watched directory is called a root.
• You must initiate a watch on a root using the watch-project command prior to subscribing to changes
• Rather than separately watching many sibling directories, watch-project consolidates and re-uses existing watches relative to a project root (the location of your .watchmanconfig or source control repository root)
• change notifications are relative to the project root

How do I use it? Copy Chapter

Read the NodeJS watchman documentation

Overview (`watchman/python/README.md`) Copy Section

Watchman client for Python

This directory contains the Watchman client for Python.

Build Copy Chapter

python -m build

Overview (`watchman/node/bser/README.md`) Copy Section

BSER Binary Serialization

BSER is a binary serialization scheme that can be used as an alternative to JSON.
BSER uses a framed encoding that makes it simpler to use to stream a sequence of
encoded values.

It is intended to be used for local-IPC only and strings are represented as binary
with no specific encoding; this matches the convention employed by most operating
system filename storage.

For more details about the serialization scheme see
Watchman's docs.

API Copy Chapter

var bser = require('bser');

bser.loadFromBuffer Copy Section

The is the synchronous decoder; given an input string or buffer,
decodes a single value and returns it. Throws an error if the
input is invalid.

var obj = bser.loadFromBuffer(buf);

bser.dumpToBuffer Copy Section

Synchronously encodes a value as BSER.

var encoded = bser.dumpToBuffer(['hello']);
console.log(bser.loadFromBuffer(encoded)); // ['hello']

BunserBuf Copy Section

The asynchronous decoder API is implemented in the BunserBuf object.
You may incrementally append data to this object and it will emit the
decoded values via its value event.

var bunser = new bser.BunserBuf();

bunser.on('value', function(obj) {
  console.log(obj);
});

Then in your socket data event:

bunser.append(buf);

Example Copy Chapter

Read BSER from socket:

var bunser = new bser.BunserBuf();

bunser.on('value', function(obj) {
  console.log('data from socket', obj);
});

var socket = net.connect('/socket');

socket.on('data', function(buf) {
  bunser.append(buf);
});

Write BSER to socket:

socket.write(bser.dumpToBuffer(obj));

Overview (`watchman/ruby/ruby-watchman/README.md`) Copy Section

RubyWatchman

RubyWatchman is a gem that implements the Watchman binary
protocol. It is
implemented in C for speed, and is much faster than talking to
Watchman using the JSON protocol.

Installation Copy Chapter

Add this line to your application's Gemfile:

gem 'ruby-watchman'

And then execute:

$ bundle

Or install it yourself as:

$ gem install ruby-watchman

Usage Copy Chapter

This example shows:

1. Asking a running Watchman process for its local socket
2. Checking with Watchman whether a given path (in this case the current working
   directory) is being watched
3. Adding the path to the watch list if necessary
4. Asking Watchman for the names of all files in the watch (files which exist,
   or have existed since watching started), then printing them

require 'ruby-watchman'
require 'socket'
require 'pathname'

sockname = RubyWatchman.load(
  %x{watchman --output-encoding=bser get-sockname}
)['sockname']
raise unless $?.exitstatus.zero?

UNIXSocket.open(sockname) do |socket|
  root = Pathname.new('.').realpath.to_s
  roots = RubyWatchman.query(['watch-list'], socket)['roots']
  if !roots.include?(root)
    # this path isn't being watched yet; try to set up watch
    result = RubyWatchman.query(['watch', root], socket)

    # root_restrict_files setting may prevent Watchman from working
    raise if result.has_key?('error')
  end

  query = ['query', root, {
    'expression' => ['type', 'f'],
    'fields'     => ['name'],
  }]
  paths = RubyWatchman.query(query, socket)

  # could return error if watch is removed
  raise if paths.has_key?('error')

  p paths['files']
end

Overview (`watchman/rust/serde_bser/README.md`) Copy Section

Work in Progress!

This is work in progress on a BSER implementation that is compatible
with serde. It is not complete! If you're reading this and want
to help move it closer to completion, please don't be afraid to
work up a pull request!

Overview (`watchman/thirdparty/libart/README.md`) Copy Section

libart [](https://travis-ci.org/armon/libart)

This library provides a C99 implementation of the Adaptive Radix
Tree or ART. The ART operates similar to a traditional radix tree but
avoids the wasted space of internal nodes by changing the node size.
It makes use of 4 node sizes (4, 16, 48, 256), and can guarantee that
the overhead is no more than 52 bytes per key, though in practice it is
much lower.

As a radix tree, it provides the following:

• O(k) operations. In many cases, this can be faster than a hash table since
  the hash function is an O(k) operation, and hash tables have very poor cache locality.
• Minimum / Maximum value lookups
• Prefix compression
• Ordered iteration
• Prefix based iteration

Usage Copy Chapter

Building the test code will generate errors if libcheck is not available.
To build the test code successfully, do the following::

$ cd deps/check-0.9.8/
$ ./configure
$ make
# make install
# ldconfig (necessary on some Linux distros)
$ cd ../../
$ scons
$ ./test_runner

References Copy Chapter

Related works:

• The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases

Watchman Docs Sync (`watchman/docs/sync.md`) Copy Section

Query Synchronization

Terminology note: Like the C++ memory model, this document will use the term sequenced-before and sequenced-after to reflect an ordering relationship between operations. Not all operations have a defined order. If two separate processes write to the filesystem, even if one does complete prior to the other, it does not necessarily imply a sequenced relationship between them. If one process writes two files on the same thread, there is a sequenced relationship.

Overview Copy Chapter

Watchman provides a model of query consistency: queries sequenced-after a filesystem modification will reflect those modifications in its query results.

That is, if a file is modified, a subsequent query should observe those modifications.

The challenge is that filesystem notification APIs are asynchronous and deferred. So how can Watchman know when it has processed every notification that has happened prior?

The specifics are platform-dependent, but Watchman generally uses cookie files. It generates a file of a unique filename in the watched directory and waits to receive the corresponding notification, at which point it assumes it has observed every prior change event.

inotify Copy Chapter

In practice, cookie files work well on Linux. It's unclear whether this sequencing is guaranteed by the kernel, however. That is, is it possible for a write to file A somewhere in the directory hierarchy followed by a write to a cookie file elsewhere to be reported in the opposite order?

inotify's own man page says:

However, robust applications should allow for the fact that bugs in the monitoring logic or races of the kind described below may leave the cache inconsistent with the filesystem state. It is probably wise to do some consistency checking, and rebuild the cache when inconsistencies are detected.

In addition:

The events returned by reading from an inotify file descriptor form an ordered queue. Thus, for example, it is guaranteed that when renaming from one directory to another, events will be produced in the correct order on the inotify file descriptor.

Linux kernel 5.3 introduced a regression in inotify (likely 49246466a989 "fsnotify: move fsnotify_nameremove() hook out of d_delete()") that caused events to sometimes be reported prior to the corresponding caches being flushed after an unlink, resulting in Watchman sometimes observing stale data.

Under correct operation, inotify should carry sequencing through each step. That is, the inotify event's observation is sequenced-after its corresponding write. The regression here broke that.

FSEvents Copy Chapter

There are three mechanisms for watching for filesystem changes on macOS: kqueue, /dev/fsevents, and CoreServices FSEvents.

kqueue does not work for large repositories, as it requires a large number of file descriptors which macOS is not tolerant of.

Last time I tried I had to go into recovery mode to stop the endless reboot cycle where macOS kernel panicked due to not having any FDs available... I tried raising the max FDs to something like 128 million, but 1) this did nothing, 2) some applications were looping through all the potential FDs to try to close them.

/dev/fsevents requires root and is a private, undocumented API.

Apple encourages use of FSEvents, which has two modes: per-file notifications, and per-directory notifications. The former produces a higher volume of change events, but in theory allows Watchman to synchronize with cookies as in inotify. The latter produces fewer change events, and in turn requires Watchman stat() every child of a directory when a change notification arrives.

Here is our understanding:

┌──────────┐    ┌───────────┐    ┌───────────────┐
│ Watchman │◄──►│ fseventsd │◄──►│ Darwin kernel │
└──────────┘    └───────────┘    └───────────────┘

The kernel reports change events to fseventsd which holds them for a period of time. They may be coalesced, and coalesced events are then reported to subscribed processes like Watchman. The coalesced events may be coarse. The FSEvents documentation is not clear, but we assume it's possible for macOS to combine multiple changes in one directory into a "just scan this directory" event.

FSEvents provides a mechanism for flushing change events: FSEventStreamFlushSync and FSEventStreamFlushAsync. This only seems to ensure that events queued in fseventsd will be flushed to subscribers, but does not provide any guarantees about events pending in the kernel.

As far as we can tell, there is no reliable way to synchronize queries on macOS.

Imagine the following sequence of events:

• Watchman starts up and crawls the directory.
• A program (e.g. git checkout) writes a large number of files into that directory.
• The program finishes, and then immediately issues a Watchman query, expecting all of those changes to be correct in the result set.
• That is, the query must not begin evaluating until every notification produced prior has been observed.

Directory Notifications Copy Section

Because Watchman does not receive notifications about individual files in this mode, it will only observe the cookie by scanning a changed directory. Therefore, Watchman cannot distinguish between discovery of a cookie for an unrelated reason (e.g. processing events from before the query started) and being notified of its creation.

One option would be to try using cookie directories instead of files, and adding entries to these directories in order to produce events. This would work if

1. we were confident FSEvents provided a guarantee that changes sequenced-after each other are notified in the same order, and
2. we were confident these cookie directory events would never be coalesced into a scan.

We are not confident of either. See the next section.

File Notifications Copy Section

In production, our Watchman was instrumented with two ring buffers: one containing the last N events provided by FSEvents and the other the paths that Watchman has stat()'d. In addition, we record when Watchman noticed a cookie file and unblocked the corresponding query.

I have observed the following scenario in production:

1. src/foo.cpp changes from size 100 to size 101
2. Watchman query begins
3. Watchman writes cookie file into repository
4. FSEvents notifies Watchman of the cookie
5. Watchman notices the cookie, and starts FSEventStreamFlushSync
6. FSEventStreamFlushSync succeeds. Watchman assumes at this point it is caught up and unblocks the query, which returns size "100" for src/foo.cpp
7. Then, FSEvents notifies Watchman that src/foo.cpp has changed

Do we have any options? Copy Section

If fseventsd is inflating per-file notifications from a directory-only stream of events from the kernel, it's possible directory changes do have an ordering.

Detecting Divergence Copy Chapter

Watchman's view of the filesystem can diverge temporarily or persistently. The aforementioned Linux inotify regression was persistent. FSEvents on the other hand just causes queries to be unblocked too soon. Subsequent queries will eventually see the correct filesystem state.

Both circumstances can be discovered with watchmanctl audit, which rapidly scans the filesystem in parallel with a Watchman query and then compares the results.

Bser (`website/docs/bser.md`) Copy Section

title: BSER Binary Protocol category: Internals Copy Chapter

The basic JSON protocol in watchman allows quick and easy integration.
Applications with higher performance requirements may want to consider the
binary protocol instead.

The binary protocol is enabled by the client sending the byte sequence
"\x00x\x01".

PDU Copy Chapter

A PDU is prefixed by its length expressed as an encoded integer. This allows the
peer to determine how much storage is required to read and decode it.

Arrays Copy Chapter

Arrays are indicated by a 0x00 byte value followed by an integer value to
indicate how many items follow. Then each item is encoded one after the other.

Objects Copy Chapter

Objects are indicated by a 0x01 byte value followed by an integer value to
indicate the number of properties in the object. Then each key/value pair is
encoded one after the other.

Strings Copy Chapter

Strings are indicated by a 0x02 byte value followed by an integer value to
indicate the number of bytes in the string, followed by the bytes of the string.

Encoding Copy Section

Unlike JSON, strings are not defined as having any particular encoding; they are
transmitted as binary strings. This is because the underlying filesystem APIs
don't define any particular encoding for names.

Exception: Keys in objects that are defined by watchman commands are always
ASCII. In general, keys in objects are always UTF-8.

Rationale: Several programming languages like Python 3 expect all text to be
in a particular encoding and make it inconvenient to pass in bytestrings or
other encodings. Also, the primary purpose of not defining an encoding is that
filenames don't always have one, and filenames are unlikely to show up as keys.

Integers Copy Chapter

All integers are signed and transmitted in the host byte order of the system
running the watchman daemon.

• 0x03 indicates an int8_t. It is followed by the int8_t value.
• 0x04 indicates an int16_t. It is followed by the int16_t value.
• 0x05 indicates an int32_t. It is followed by the int32_t value.
• 0x06 indicates an int64_t. It is followed by the int64_t value.

Real Copy Chapter

A real number is indicated by a 0x07 byte followed by 8 bytes of double value.

Boolean Copy Chapter

• 0x08 indicates boolean true
• 0x09 indicates boolean false

Null Copy Chapter

0x0a indicates the null value

Array of Templated Objects Copy Chapter

0x0b indicates a compact array of objects follows. Some of the bigger
datastructures returned by watchman are tabular data expressed as an array of
objects. This serialization type factors out the repeated object keys into a
header array listing the keys, followed by an array containing all the values of
the objects.

To represent missing keys in templated arrays, the 0x0c encoding value may be
present. If encountered it is interpreted as meaning that there is no value for
the key that would have been decoded in this position. This is distinct from the
null value.

For example:

[
   {"name": "fred", "age": 20},
   {"name": "pete", "age": 30},
   {"age": 25 },
]

is represented similar to:

["name", "age"],
[
  "fred", 20,
  "pete", 30,
  0x0c,   25
]

The precise sequence is:

0b          template
00          array     -- start prop names
0302        int, 2    -- two prop names
02          string    -- first prop "name"
0304        int, 4
6e616d65    "name"
02          string    -- 2nd prop "age"
0303        int, 3
616765      "age"
0303        int, 3    -- there are 3 objects
02          string    -- object 1, prop 1 name=fred
0304        int, 4
66726564    "fred"
0314        int 0x14  -- object 1, prop 2 age=20
02          string    -- object 2, prop 1 name=pete
0304        int 4
70657465    "pete"
031e        int, 0x1e -- object 2, prop 2 age=30
0c          skip      -- object 3, prop 1, not set
0319        int, 0x19 -- object 3, prop 2 age=25

Note: to avoid hostile "decompression bombs", Watchman will reject parsing
template objects that have an empty set of keys.

Capabilities (`website/docs/capabilities.md`) Copy Section

title: Capabilities category: Compatibility Copy Chapter

Since 3.8

Capability names are used to identify modules that are either conditionally
configured or that are introduced over time.

You can use the expanded version command to query capabilities
and avoid building knowledge of version numbers in your client application(s).

You can use list-capabilities command to obtain a
list of capabilities supported by your watchman server.

Commands Copy Section

Every command is identified by the command name prefixed by the string cmd-.
For example, the watch-project command is indicated by the capability name
cmd-watch-project.

Expression Terms Copy Section

Every expression term is identified by the term name prefixed by the string
term-. For example, the match term is indicated by the capability name
term-match.

File Result Fields Copy Section

Every field is identified by the field name prefixed by the string field-. For
example, the size field is indicated by the capability name field-size.

Feature Enhancements Copy Section

Sometimes we will enhance existing functionality by adding new options to
existing commands. Since these changes won't result in adding a new command they
won't implicitly gain a capability name. In these cases we'll assign an
appropriate capability name by hand.

The following feature capabilities are possible / released:

Casefolding (`website/docs/casefolding.md`) Copy Section

title: Case-Insensitivity category: Internals Copy Chapter

Watchman is currently completely unaware of case-insensitivity in file systems,
and does not attempt to do any case-folding of file names. On a case-insensitive
file system like macOS's HFS+, this
can manifest itself in different ways:

• If a file foo.txt is renamed to FOO.txt, Watchman will report FOO.txt as
  created and foo.txt separately as changed.
• If a file foo.txt is removed and another file FOO.txt is later added,
  Watchman will report FOO.txt as added, but it might report foo.txt as
  either removed or changed.

In general, both foo.txt and FOO.txt can be reported, sometimes with
different stat data, sometimes with the same stat data.

Why doesn't Watchman support case-folding properly? Copy Chapter

One problem is that 'properly' is hard to pin down. There are at least four
levels of correctness here:

• handle ASCII case-folding only (95% solution)
• handle ASCII + accented ASCII case-folding only (98%)
• full handling of current Unicode spec using a Unicode database (99%)
• using the special folding table written to a hidden file on disk at file
  system creation time that matches Apple's interpretation of Unicode at the
  time of the OS release + their own quirks (100%)

Clients of Watchman might have their own idea of case-folding, which might or
might not be compatible with Watchman's idea of it. So far, clients have managed
to handle case-folding outside of Watchman, with some success.

Does this matter? Copy Chapter

It depends on your application.

Example 1: Your application is a build system that has a pre-baked list of
files. Your application expects files to be on disk in the correct case even on
case-insensitive file systems, and you declare that the behavior is undefined if
they aren't. You invoke Watchman by asking it what files have changed. In this
case, Watchman should work without you having to do anything special.

Example 2: Your application is a build system rule to generate CSS rules
that is run by a Watchman trigger on *.scss. You expect all files you care
about to end with the string .scss on case-insensitive file systems, and not
another variant of it like .SCSS. In this case, Watchman should work fine --
at most, it will provide you the same file multiple times with different case
variants. You might be dealing with that in your build system anyway.

Example 3: Like example 2, except you expect .SCSS and other variants to
work too. In that case the only way is to explicitly add all possible variants
to the trigger rule.

Example 4: You're a source control system that has its own ideas about
case-folding that might or might not match up with the operating system's. You
perform case-folding against an internal data structure, so that if the data
structure has foo.txt and the file system has FOO.txt you make foo.txt
take precedence. In that case, Watchman will tell you about both FOO.txt and
foo.txt, and it's up to you to perform normalization.
hgwatchman just consults the file
system in the rare case that a file changes case.

Credits Copy Chapter

The levels of correctness were proposed by Olivia Mackall [email protected].

Cli Options (`website/docs/cli-options.md`) Copy Section

title: Command Line category: Invocation sidebar_position: 1 Copy Chapter

The watchman executable contains both the client and the server components of
the watchman service.

By default, when watchman is run, it will attempt to communicate with your
existing server instance (each user has their own persistent process), and will
attempt to start it if it doesn't exist.

There are some options that affect how watchman will locate the server, some
options that affect only the client and some others that affect only the server.
Since all of the options are understood by the same executable we've broken
those out into sections of their own to make it clearer when they apply.

Quick note on default locations Copy Chapter

Watchman will prefer to resolve your user name from the $USER environmental
variable, or $LOGNAME if $USER was not set. If neither are set watchman will
look it up from the system using getpwuid(getuid()). When we refer to ``
in this documentation we mean the result of this resolution.

In some cases Watchman will need to create files in a temporary location.
Watchman will resolve this temporary location by looking at the $TMPDIR
environmental variable, or $TMP if $TMPDIR was not set. If neither are set
watchman will use /tmp. When we refer to <TMPDIR> in this documentation we
mean the result of this resolution.

Watchman tracks its persistent state in a location that we refer to as the
<STATEDIR> in this documentation.

Since 3.1.

The STATEDIR defaulted to /var/run/watchman. You can change this
default when you build watchman by using the configure option
--enable-statedir.

Earlier versions of Watchman didn't have a default statedir and would instead
use the <TMPDIR> for this state. We switched away from that because some
environments randomize the <TMPDIR> location and this made it difficult for
clients to locate the Watchman service.

Since 3.8.

The STATEDIR defaults to /var/run/watchman/-state. You can
change this default when you build watchman by using the configure option
--enable-statedir; the configure option replaces the
/var/run/watchman portion of this string. If you specify
--disable-statedir then that portion of the string will be computed from the
<TMPDIR> location.

Watchman will create the -state portion if it does not exist, and will
perform some permission and ownership checks to reduce the risk of untrusted
users placing files in this location. If those checks are not satisfied,
watchman will refuse to start.

Locating the service Copy Chapter

-U, --sockname=PATH   Specify alternate sockname

The default location for sockname will be <STATEDIR>/. Older versions of
Watchman would default to <TMPDIR>/.watchman., depending on how it was
configured.

If you are building a client to access the service programmatically, we
recommend that you invoke watchman get-sockname to
discover the path that the client and server would use. This has the side effect
of spawning the service for you if it isn't already running.

Client Options Copy Chapter

The watchman executable will attempt to start the service if there is no
response on the socket specified above. In some cases it is desirable to avoid
starting the service if it isn't running:

--no-spawn            Don't spawn service if it is not already running.
                       Will try running the command in client mode if
                       possible.
 --no-local            When no-spawn is enabled, don't use client mode

Client mode implements the watchman find command as an immediate
search.

These options control how the client talks to the server:

-p, --persistent           Persist and wait for further responses
     --server-encoding=ARG  CLI<->server encoding. json or bser.

Persistent connections have relatively limited use with the CLI, but can be
useful to connect ad-hoc to the service to receive logging information (See
log-level).

The server encoding option controls how requests and responses are formatted
when talking to the server. You generally shouldn't need to worry about this.

Input and Output Copy Section

Most simple invocations of the CLI will pass a list of arguments:

$ watchman watch /path/to/dir

This is turned into a request like this:

["watch", "/path/to/dir"]

and sent to the service using the Socket Interface.

The response is received and then sent to the stdout stream formatted based on
the selected output-encoding:

--output-encoding=ARG  CLI output encoding. json (default) or bser
     --no-pretty            Don't pretty print JSON output (more efficient
                            when being processed by another program)

Each command has its own response output but watchman will always include a
field named error if something about the request was not successful. In case
of some protocol level errors (eg: connection was terminated) instead of
printing a response on stdout, an unstructured error message will be printed
to stderr and the process will exit with a non-zero exit status.

Instead of passing the request as command line parameters, you can send a JSON
representation on the stdin stream. These invocations are all equivalent:

$ watchman watch /path/to/dir

$ watchman -j <<-EOT
["watch", "/path/to/dir"]
EOT

$ watchman -j <<< '["watch", "/path/to/dir"]'

$ echo '["watch", "/path/to/dir"]' | watchman -j

$ echo '["watch", "/path/to/dir"]' > cmd.json
$ watchman -j < cmd.json

$ watchman --json-command <<-EOT
["watch", "/path/to/dir"]
EOT

Since 3.8

The CLI now also recognizes BSER as a valid input stream when using the -j
option. This will implicitly set --server-encoding=bser and
--output-encoding=bser if those options have not been set to something else.

Exit Status Copy Chapter

The watchman binary will exit with a return code of 0 in most cases; this
indicates that the output it generated should be valid JSON. To determine if
your command was successful, you need to parse the JSON and look for the error
field as described above.

watchman will exit with a non-zero exit status in cases where something
low-level went wrong, such as protocol level errors (eg: connection was
terminated).

Server Options Copy Chapter

These options are used when starting the server. They are recognized by the
client and affect how it will start the server, but have no effect if the server
is already running. To change the effective values of these options for a
running server, you will need to restart it (you can stop it by running
watchman shutdown-server).

By default, watchman will remember all watches and associated triggers and
reinstate them if the process is restarted. This state is stored in the
statefile:

--statefile=PATH      Specify path to file to hold watch and trigger state
 -n, --no-save-state   Don't save state between invocations

The default location for statefile will be <STATEDIR>/.state. Older
versions of watchman may store the state in <TMPDIR>/.watchman..state,
depending on how they were configured.

-o, --logfile=PATH   Specify path to logfile
    --log-level      set log verbosity (0 = off, default is 1, verbose = 2)

The default location for logfile will be <STATEDIR>/.log. Older versions
of watchman may store the logs in <TMPDIR>/.watchman..log, depending on
how they were configured.

In some relatively uncommon circumstances, such as in test harnesses, you may
need to directly run the service without it putting itself into the background:

-f, --foreground      Run the service in the foreground

Since 4.6.

--inetd                Spawning from an inetd style supervisor

When this flag is specified, watchman will use stdin as the listening socket
rather than attempting to set it up for itself. This allows some other process
to maintain the socket and defer activating the watchman service until a client
is ready to connect. This is most practically beneficial when used together with
systemd.

This commit includes a sample configuration for systemd.

Clockspec (`website/docs/clockspec.md`) Copy Section

title: Clockspec category: Queries Copy Chapter

For commands that query based on time, watchman offers a couple of different
ways to measure time.

• number of seconds since the unix epoch (unix time_t style)
• a clock id of the form c:123:234
• a named cursor of the form n:whatever (but clock ids are faster!)

The first and most obvious is passing a unix timestamp. Watchman records the
observed time that files change and allows you to find files that have changed
since that time. Using a timestamp is prone to race conditions in understanding
the complete state of the file tree.

Using an abstract clock id insulates the client from these race conditions by
ticking as changes are detected rather than as time moves. Watchman returns the
current clock id when it delivers match results; you can use that value as the
clockspec in your next time relative query to get a race free assessment of
changed files.

As a convenience, watchman can maintain the last observed clock for a client by
associating it with a client defined cursor name. For example, you could
enumerate all the "C" source files on your first invocation of:

watchman since /path/to/src n:c_srcs '*.c'

and when you run it a second time, it will show you only the "C" source files
that changed since the last time that someone queried using "n:c_srcs" as the
clock spec. However, it's not possible to "roll back" a named cursor, so
advanced users desiring such functionality should use clock ids instead.

Since 4.7.

We recommend not using the n:whatever form as it requires an exclusive lock on
the view to execute; this can increase contention and result in slower queries.

Compatibility (`website/docs/compatibility.md`) Copy Section

title: Compatibility Rules category: Compatibility Copy Chapter

watchman has been used in production since a few weeks after it was first
written, and thus it has always made an effort to be backward compatible across
releases and platforms.

• Commands and options will never be removed, but new ones may be added.
• We may deprecate commands and options and remove them from documentation,
  but they will still continue to work forever.
• Whenever a command or option is deprecated, we will provide a suitable
  alternative.
• Bugfixes might cause minor behavior changes -- these changes will usually be
  documented in release notes.

watchman does not follow semantic versioning!

• Since its public APIs never make incompatible changes, MAJOR versions are
  moot.
• While in the past we've released versions with three components (x.y.z),
  starting version 3.1 the version number will only have two components that are
  meaningful (x.y), with the third component always zero.
• The version after 3.9 is expected to be 4.0. The version number string
  reported by these versions will be 3.9.0 and 4.0.0 respectively.

Since 3.8.

watchman introduces capabilities to describe new or
optional features. You can use the expanded version command
to query capabilities and avoid building knowledge of version numbers in your
client application(s).

Since May 2020

Watchman is continuously deployed inside Facebook, which means that we don't
explicitly maintain version numbers. For a while we maintained version numbers
for GitHub releases but found it to be too much overhead.

Starting in 2020 we've set up automation to cut a weekly date based on the date;
this more closely matches our internal processes than manually managing version
numbers.

You'll notice that both the
tags on GitHub and the version
reported by watchman version are date based.

Config (`website/docs/config.md`) Copy Section

title: Configuration Files category: Invocation sidebar_position: 7 Copy Chapter

Watchman looks for configuration files in two places:

• The global configuration file /etc/watchman.json
• The root specific configuration file .watchmanconfig

When watching a root, if a valid JSON file named .watchmanconfig is present in
the root directory, watchman will load it and use it as a source of
configuration information specific to that root.

The global configuration path can be changed by passing the --enable-conffile
option to configure when you build watchman. This documentation refers to it as
/etc/watchman.json throughout, just be aware that your particular installation
may locate it elsewhere. In addition, the environmental variable
$WATCHMAN_CONFIG_FILE will override the default location.

If the global configuration file does not exist, Watchman will fall back on that
path with ".default" appended (e.g. /etc/watchman.json.default). This allows the
Watchman system package to provide different configuration defaults, like
setting enforce_root_files to true.

Changes to the .watchmanconfig or /etc/watchman.json files are not picked up
automatically; you will need to remove and re-add the watch (for
.watchmanconfig) or restart watchman (for /etc/watchman.json) for those
changes to take effect.

Resolution / Scoping Copy Section

There are three configuration scopes:

• local - the option value is read from the .watchmanconfig file in the
  associated root.
• global - the option value is read from the /etc/watchman.json file
• fallback - the option value is read from the .watchmanconfig file. If
  the option was not present in the .watchmanconfig file, then read it from
  the /etc/watchman.json file.

This table shows the scoping and availability of the various options:

Configuration Options Copy Section

settle Copy Section

Specifies the settle period in milliseconds. This controls how long the
filesystem should be idle before dispatching triggers. The default value is 20
milliseconds.

root_files Copy Section

Since 3.1.

Specifies a list of files that, if present in a directory, identify that
directory as the root of a project.

If left unspecified, to aid in transitioning between versions, watchman will use
the value of the now deprecated root_restrict_files
configuration setting.

If neither root_files nor root_restrict_files is specified in the
configuration, watchman will use a default value consisting of:

• .git
• .hg
• .svn
• .watchmanconfig

Watchman will add .watchmanconfig to whatever value is specified for this
configuration value if it is not present.

This example causes only .watchmanconfig to be considered as a project root
file:

{
  "root_files": [".watchmanconfig"]
}

See the watch-project command for more information.

enforce_root_files Copy Section

Since 3.1.

This is a boolean option that defaults to false. If it is set to true then
the watch command will only succeed if the requested directory
contains one of the files listed by the root_files configuration
option, and the watch-project command will only succeed
if a valid project root is found.

If left unspecified, to aid in transitioning between versions, watchman will
check to see if the now deprecated root_restrict_files
configuration setting is present. If it is found then the effective value of
enforce_root_files is set to true.

root_restrict_files Copy Section

Deprecated starting in version 3.1; use root_files and
enforce_root_files to effect the same behavior.

Specifies a list of files, at least one of which should be present in a
directory for watchman to add it as a root. By default there are no
restrictions.

For example,

{
  "root_restrict_files": [".git", ".hg"]
}

will allow watches only in the top level of Git or Mercurial repositories.

illegal_fstypes Copy Section

Specifies a list of filesystem types that watchman is prohibited to attempt to
watch. Watchman will determine the filesystem type of the root of a watch; if
the typename is present in the illegal_fstypes list, the watch will be
prohibited. You may also specify illegal_fstypes_advice as a string with
additional advice to your user. The purpose of this configuration option is
largely to prevent the use of Watchman on network mounted filesystems. On Linux
systems, Watchman may not be able to determine the precise type name of a
mounted filesystem. If the filesystem type is not known to watchman, it will be
reported as unknown.

For example,

{
  "illegal_fstypes": ["nfs", "cifs", "smb"],
  "illegal_fstypes_advice": "use a local directory"
}

will prevent watching dirs mounted on network filesystems and provide the advice
to use a local directory. You may omit the illegal_fstypes_advice setting to
use a default suggestion to relocate the directory to local disk.

ignore_vcs Copy Section

Apply special VCS ignore logic to the set of named dirs. This option has a
default value of [".git", ".hg", ".svn", ".jj"]. Dirs that match this option
are observed and watched using special shallow logic. The shallow watch allows
watchman to mildly abuse the version control directories to store its query
cookie files and to observe VCS locking activity without having to watch the
entire set of VCS data for large trees.

ignore_dirs Copy Section

Dirs that match are completely ignored by watchman. This is useful to ignore a
directory that contains only build products and where file change notifications
are unwanted because of the sheer volume of files.

For example,

{
  "ignore_dirs": ["build"]
}

would ignore the build directory at the top level of the watched tree, and
everything below it. It will never appear in the watchman query results for the
tree.

On Linux systems, ignore_dirs is respected at the OS level; the kernel simply
will not tell watchman about changes to ignored dirs. macOS and Windows have
limited or no support for this, so watchman needs to process and ignore this
class of change.

For large trees or especially busy build dirs, it is recommended that you move
the busy build dirs out of the tree for more optimal performance.

Since version 2.9.9, if you list a dir in ignore_dirs that is also listed in
ignore_vcs, the ignore_dirs placement will take precedence. This may not
sound like a big deal, but since ignore_vcs is used as a hint to for the
placement of cookie files, having these two options overlap in
earlier versions would break watchman queries.

Since 4.6.

On macOS the first 8 items listed in ignore_dirs can be accelerated at the OS
level. This means that changes to those paths are not even communicated to the
watchman service. Entries beyond the first 8 are processed and ignored by
watchman. If your workload is prone to recrawl events you will want to
prioritize your ignore_dirs list so that the most busy ignored locations
occupy the first 8 positions in this list.

gc_age_seconds Copy Section

Deleted files (and dirs) older than this are periodically pruned from the
internal view of the filesystem. Until they are pruned, they will be visible to
queries but will have their exists field set to false. Once they are pruned,
watchman will remember the most recent clock value of the pruned nodes. Any
since queries based on a clock prior to the last prune clock will be treated as
a fresh instance query. This allows a client to detect and choose how to handle
the case where they have missed changes. See is_fresh_instance elsewhere in
this document for more information. The default for this is 43200 (12 hours).

gc_interval_seconds Copy Section

How often to check for, and prune out, deleted nodes per the gc_age_seconds
option description above. The default for this is 86400 (24 hours). Set this
to 0 to disable the periodic pruning operation.

fsevents_latency Copy Section

Controls the latency parameter that is passed to FSEventStreamCreate on macOS.
The value is measured in seconds. The fixed value of this parameter prior to
version 3.2 of watchman was 0.0001 seconds. Starting in version 3.2 of
watchman, the default is now 0.01 seconds and can be controlled on a per-root
basis.

If you observe problems with kFSEventStreamEventFlagUserDropped increasing the
latency parameter will allow the system to batch more change notifications
together and operate more efficiently.

fsevents_try_resync Copy Section

This is macOS specific.

Since 4.6.

Defaults to false. If set to true, if a watch receives a
kFSEventStreamEventFlagUserDropped event, attempt to resync from the
fsevents journal if it is available. The journal may not be available if one
or more volumes are mounted read-only, if the administrator has purged the
journal, or if the fsevents id numbers have rolled over.

This resync operation is advantageous because it effectively allows rewinding
and replaying the event stream from a known point in time and avoids the need to
recrawl the entire watch.

If this option is set to false, or if the journal is not available, the
original strategy of recrawling the watched directory tree is used instead.

Since 4.7.

The default changed to true. In addition, this resync strategy is now also
applied to kFSEventStreamEventFlagKernelDropped events.

Since December 2021.

The default changed to false. There are possible undiagnosed correctness
issues with this setting.

prefer_split_fsevents_watcher Copy Section

This is macOS specific.

Defaults to false. If set to true, Watchman will use several FSEvents
streams to watch a directory hierarchy instead of a single stream. This has been
shown to significantly reduce the number of kFSEventStreamEventFlagUserDropped
events for workflows issuing heavy writes to a top-level directory that is
listed in ignore_dirs.

idle_reap_age_seconds Copy Section

Since 3.7.

How many seconds a watch can remain idle before becoming a candidate for
reaping, measured in seconds. The default for this is 432000 (5 days). Set
this to 0 to prevent reaping.

A watch is considered to be idle when it has had no commands that operate on it
for idle_reap_age_seconds. If an idle watch has no triggers and no
subscriptions then it will be cancelled, releasing the associated operating
system resources, and removed from the state file.

hint_num_files_per_dir Copy Section

Since 3.9.

Used to pre-size hash tables used to track files per directory. This is most
impactful during the initial crawl of the filesystem. Setting this too small
will increase the chance of a hash insert having a collision and drive up the
cost of the insert and subsequent gets.

Prior to version 3.9 of watchman this value was fixed at 2. Starting in
version 3.9 the default value is 64 and can be configured via this setting in
the .watchmanconfig or the global /etc/watchman.json configuration file.

Setting this value very large increases the memory overhead per directory in the
tree; the value is rounded up to the next power of two and pre-allocated in an
array of pointers. On a 64-bit system multiply that number by 8 to arrive at the
number of bytes of overhead (halve this on a 32-bit system). The overhead is
doubled when using a case insensitive filesystem.

The ideal size from a time complexity perspective is the number of files in your
largest directory. From a space complexity perspective, the ideal size is 1; you
would pay the cost of the collisions during the initial crawl and have a more
optimal memory usage. Since watchman is primarily employed as an accelerator,
we'd recommend biasing towards using more memory and taking less time to run.

hint_num_dirs Copy Section

Since 4.6

Used to pre-size hash tables that are used to track the total set of files in
the entire watched tree. The default value for this is 131072.

The optimal size is a power-of-two larger than the number of directories in your
tree; running find . -type d | wc -l will tell you the number that you have.

Making this number too large is potentially wasteful of memory. Making this
number too small results in increased latency during crawling while the hash
tables are rebuilt.

suppress_recrawl_warnings Copy Section

Since 4.7

When set to true, watchman will not produce recrawl related warning fields in
the response PDUs of various requests. The default is false; the intent is
that someone in your organization should be aware of recrawls and be able to
manage the configuration and workload. Some sites employ an alternative
mechanism for sampling and reporting this to the right set of people and wish to
disable the warning so that it doesn't appear in front of users that are unable
to make the appropriate configuration changes for themselves.

eden_file_count_threshold_for_fresh_instance Copy Section

This is specific to the EdenFS watcher

When set to a non-zero value, Watchman will return a fresh instance to since
queries/subscriptions if the number of changed files exceeds the configured
value. In particular, during large updates of the working copies, a lot of files
may have changed forcing both Watchman and EdenFS to fetch a significant amount
of metadata to answer these queries.

This behavior is only enabled if the query specifies the
empty_on_fresh_instance option or when this config is set to 0. Default to
10000.

Contributing (`website/docs/contributing.md`) Copy Section

title: Contributing category: Internals Copy Chapter

If you're thinking of hacking on watchman we'd love to hear from you! Feel free
to use the GitHub issue tracker and pull requests discuss and submit code
changes.

We (Facebook) have to ask for a "Contributor License Agreement" from someone who
sends in a patch or code that we want to include in the codebase. This is a
legal requirement; a similar situation applies to Apache and other ASF projects.

If we ask you to fill out a CLA we'll direct you to
our online CLA page where you can complete it
easily. We use the same form as the Apache CLA so that friction is minimal.

Facebook Open Source provides a Code of Conduct statement for all projects to
follow, to promote a welcoming and safe open source community. Please
read the full text so that you can
understand what actions will and will not be tolerated.

Getting Started Copy Section

You need to be able to build watchman from source and run its test suite. You
will need:

• python
• automake
• autoconf
• cmake
• libtool (or glibtool on macOS)
• libpcre
• libfolly (only needed if building the cppclient library)
• nodejs (for fb-watchman)

The build time dependencies can be installed by running
getdeps.py --install-deps. This is run for you when you run autogen.sh in
the example below:

$ git clone https://github.com/facebook/watchman.git
$ cd watchman
$ ./autogen.sh
$ make

After making a change, run the integration tests to make sure that things are
still working well before you submit your pull request:

$ make integration

We'll probably ask you to augment the test suite to cover the functionality that
you're adding or changing.

Please keep in mind that our versioning philosophy in Watchman is to provide an
append only API. If you're changing functionality, we'll ask you to do so in
such a way that it won't break older clients of Watchman.

Don't forget the docs Copy Section

If you're changing or adding new functionality, we'll ask you to also update the
documentation. You will need ruby 2.0.0 or later to preview the documentation.

One time setup:

$ cd website
$ sudo gem install bundler
$ sudo bundler install

Then:

$ jekyll serve -w -t

This will print out a URL that you can open in your browser to preview your
documentation changes.

The source for the documentation is in the website/_docs dir in markdown
format.

Cookies (`website/docs/cookies.md`) Copy Section

title: Query Synchronization category: Internals Copy Chapter

A file system monitor needs to make sure that queries see up-to-date views.
Watchman ensures that by creating a unique cookie for each query made to it.

Background Copy Chapter

Consider a directory tree traversal to gather file status, such as the one
performed by hg status or git status. The traversal will race with any
operations happening concurrently, and this is impossible to fix. However, we do
get some weaker guarantees:

1. Every file operation that happens before the traversal is started will be
   observed.
2. File operations that happen after the traversal is started may or may not be
   observed.

For watchman, cookies enable us to provide similar guarantees. For a given
watchman query:

1. Every file operation that happens before the query is started will be
   observed.
2. File operations that happen after the query is started may or may not be
   observed.

How cookies work Copy Chapter

A cookie is a temporary file that is created inside a directory observed by
watchman. The cookie is created in a directory that is expected not to go away.
The obvious location is the root itself, but we'd like cookies not to show up in
VCS operations. So if a VCS directory (.git, .hg or .svn) is found, that's
where cookies are created instead.

The cookie is created while the root is locked, so watchman won't find the
cookie by accident while processing events from a prior run.

Once the cookie is created, the calling thread waits on a condition variable
guarded by the root's lock. This causes the lock to be released, and the root's
notify thread can now read events as usual.

When the notify thread finds that it is processing a cookie, it will signal its
respective condition variable. Importantly, this does not wake the calling
thread up immediately: since the notify thread still holds the root lock, the
calling thread will only be able to proceed once the notify thread releases the
lock.

What do cookies get us?

File monitoring systems like inotify typically provide an ordering guarantee:
notifications arrive in the order they happen. Any events happening before the
cookie is created will appear before the event for the cookie does, which means
they will be processed by the time the query is answered.

How well do cookies work?

The Mercurial test suite has proved to be a good stress test for watchman.
Before cookies were implemented, if 16 or more tests from the suite were run in
parallel, watchman would start falling behind and often produce outdated
answers. Cookies have successfully eradicated that.

Can watchman find a cookie even if not all events leading to its appearance
have been processed?

Consider this situation when cookies are created inside .hg:

1. Event A happens that would cause .hg to be read recursively
2. Event B happens that touches a file subdir/foo
3. A cookie is created inside .hg, causing event C
4. Event A is read from the OS file notification system but not events B and C
5. The cookie is found but subdir/foo is never read.

On Linux, to prevent this from happening, watchman will only consider a cookie
to be found if it is directly returned via OS notifications. The only exception
to this is during the initial crawl or a recrawl, when the cookie directory
isn't being watched yet.

On other platforms, this becomes more complicated because the respective
monitoring system only tells us that something inside a directory was created,
not what was created. This is currently an unresolved issue.

Limitation: macOS FSEvents Copy Chapter

On macOS, Watchman uses
FSEvents
to monitor filesystem changes. Watchman uses a combination of cookie files
described above and
FSEventStreamFlushSync
to attempt to catch up with all prior changes.

Unfortunately, in high-load situations like a large git checkout on a busy
host, we have observed that FSEvents from the git checkout may be received
after the cookie file notification and FSEventStreamFlushSync returning.

It turns out that FSEvents provides no guarantees here, and relying on it for
query synchronization is unsupported on any current Apple platform. Their
suggested workaround is to implement a watcher with
Endpoint Security.
Nobody has evaluated the feasibility of this yet.

In the meantime, you can set a settle_period and settle_timeout on the
query. Both are integer milliseconds, and settle_period specifies the required
quiescence before the watcher is considered caught up.

Credits Copy Chapter

The idea was originally proposed by Olivia Mackall [email protected].

Cppclient (`website/docs/cppclient.md`) Copy Section

title: C++ Client category: Invocation sidebar_position: 5 Copy Chapter

Since 4.8.

Watchman includes a C++ client library to facilitate easy access to Watchman
data from C++ applications. This library provides APIs for:

• Opening and maintaining a connection to a local Watchman server.
• Executing request-response Watchman commands.
• Subscribing to updates with in directory trees.

Installation Copy Chapter

Provided the Folly library is present when Watchman is built, the C++ client
library is automatically built and installed. For details on building Watchman
see Installation.

API Copy Chapter

The public Watchman C++ client API is entirely covered in the installed
watchman/WatchmanClient.h header file. This header contains a usage synopsis
and notes on the public API features.

For a simple example of API usage, sending simple request-response commands to
Watchman, see cppclient/CLI.cpp in the Watchman source tree. For a more
extensive example of the API including use of subscriptions, see the integration
test integration/cppclient.cpp also in the Watchman source.

The C++ client library and its API make heavy use of the Folly library and as
such familiarity with this is highly recommended. Specifically, the client
library makes extensive use of
Folly's async features
to provide high-performance asynchronous I/O, and
Folly's dynamics
to avoid needing to construct/process raw JSON in C++.

Using the C++ client in your application's build Copy Chapter

To facilitate integration into your application's build, the Watchman C++ client
library provides support for
pkg-config.

For example, if your application was contained entirely in one C++ file called
app.cpp the following would be sufficient for build on a system with GNU Make:

$ make LDFLAGS=$(pkg-config watchmanclient --libs) CPPFLAGS=$(pkg-config watchmanclient --cflags) app

If Watchman is installed in a location pkg-config does not search for packages
by default, you may need to modify the PKG_CONFIG_PATH environment variable.
For example:

$ export PKG_CONFIG_PATH=<watchman path>/lib/pkgconfig:$PKG_CONFIG_PATH

File Query (`website/docs/file-query.md`) Copy Section

title: File Queries category: Queries Copy Chapter

Watchman file queries consist of 1 or more generators that feed files through
the expression evaluator.

Generators Copy Section

Generators are analogous to the list of paths that you specify when using the
find(1) utility, but are implemented in watchman with a bit of a twist because
watchman doesn't need to crawl the filesystem in realtime and instead maintains
a couple of indexes over the tree.

A query may specify any number of generators; each generator will emit its list
of files and this may mean that you see the same file output more than once if
you specified the use of multiple generators that all produce the same file.

Watchman provides 5 generators:

• since: produces a list of files that were modified since a specific
  clockspec.
• suffix: produces a list of files that have a particular suffix.
• glob: efficiently pattern match a list of files based on their names.
• path: produces a list of files based on their path and depth.
• all: produces a list of all known files

De-duplicating results Copy Section

Since 4.7.

If your query uses multiple generators, or configures the path generator with
paths that yield multiple results, the default behavior (for backwards
compatibility reasons) is to emit those duplicate results in the query output.

You may ask Watchman to de-duplicate results for you by enabling the
dedup_results boolean in your query:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "path": ["bar", "bar"],
  "dedup_results": true
}]
EOT

You may test for this feature using an extended version command and requesting
the capability name dedup_results.

Since Generator Copy Section

The since generator produces a list of files that were modified since a
specific clockspec.

The following query will consider the set of files changed since the last query
using the named cursor mycursor and then pass them to the expression evaluator
to be filtered to just those that are files:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "since": "n:mycursor",
  "expression": ["type", "f"]
}]
EOT

If the since parameter value is blank, was produced by a different watchman
process (in other words, the watchman process was restarted between the time
that the value was obtained and the time the query was issued) or is a named
cursor that has not yet been used in a query, the since generator will
consider the state to be a fresh instance and its behavior is modified:

A fresh instance result set will only include files that currently exist and
will generate file nodes that are always considered to be new.

If the query was configured with the empty_on_fresh_instance property set to
true then the result set will be empty and the is_fresh_instance property
will be set to true in the result object.

The since generator also knows how to talk to source control;
you can read more about that here.

The since generator does not consider the targets of symlinks. In particular,
the since generator may not produce a symlink in the following cases:

• The symlink's target was a file, and the file is since modified.
• The symlink's target was a file, and the file is since deleted or replaced
  with a different file.
• An ancestor of the symlink's target was created or deleted or modified.
• The symlink's target was a directory, and a file is since added or removed
  from that directory.

Suffix Generator Copy Section

The suffix generator produces a list of files that have a particular suffix or
set of suffixes. The value can be either a string or an array of strings.

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "suffix": "js"
}]
EOT

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "suffix": ["js", "css"]
}]
EOT

If the suffix generator is given an empty array, it produces no files.

The suffix generator can produce symlinks.

The suffix generator does not follow symlinks. For example, a symlink to
/etc will not cause a "suffix": "conf" query to search within /etc and
produce /etc/resolv.conf.

Glob Generator Copy Section

Since 4.7.

The glob generator produces a list of files by matching against your input
list of patterns. It does this by building a tree from the glob expression(s)
and walking both the expression and the in-memory filesystem tree concurrently.

This query will yield a list of all of the C source and header files found
directly in the src dir:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "glob": ["src/*.c", "src/*.h"],
  "fields": ["name"]
}]

This query will yield a list of all of the C source and header files found in
any subdirectories of the root:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "glob": ["**/*.c", "**/*.h"],
  "fields": ["name"]
}]

Note that it is more efficient to use the suffix generator together with a
dirname expression term for such a broadly scoped query as it results in fewer
comparisons. This example is included as an illustration of recursive globbing.

The glob generator implicitly enables dedup_results mode.

If the glob generator is given an empty array, it produces no files.

The glob generator can produce symlinks.

The glob generator does not follow symlinks. For example, a symlink to /etc
will not cause a "glob": ["**/resolv.conf"] query to search within /etc and
produce /etc/resolv.conf.

Path Generator Copy Section

The path generator produces a list of files based on their path and depth.
Depth controls how far watchman will search down the directory tree for files.

The path generator expects an array of path specifiers. Each path specifier
can be either a string or an object and each will produce a set of files.

If it is a string then it is treated as the value for path with depth set to
infinite. If an object, the fields path (a string) and depth (an integer)
must be supplied.

Paths are relative to the root, so if watchman is watching /foo/, path bar
refers to /foo/bar.

A depth value of 0 means only files and directories which are contained in
this path. A depth value of -1 means no limit on the depth.

The following path generators are equivalent:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "path": ["bar"]
}]
EOT

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "path": [{"path": "bar", "depth": -1}]
}]
EOT

If the path generator is given an empty array, it produces no files.

The path generator can produce symlinks.

The path generator does not follow symlinks.

All Generator Copy Section

The all generator produces a list of all file nodes. It is the default
generator and is used in the case where no other generators were explicitly
specified.

$ watchman -j <<-EOT
["query", "/path/to/root", {
}]
EOT

The all generator can produce symlinks.

The all generator does not follow symlinks.

Expressions Copy Section

A watchman query expression consists of 0 or more expression terms. If no terms
are provided then each file evaluated is considered a match (equivalent to
specifying a single true expression term).

Otherwise, the expression is evaluated against the file and produces a boolean
result. If that result is true then the file is considered a match and is added
to the output set.

An expression term is canonically represented as a JSON array whose zeroth
element is a string containing the term name.

["termname", arg1, arg2]

If the term accepts no arguments you may use a short form that consists of just
the term name expressed as a string:

"true"

Expressions that match against file names may match against either the
basename or the wholename of the file. The basename is the name of the file
within its containing directory. The wholename is the name of the file relative
to the watched root.

You can find a list of all possible expression terms in the sidebar on the left
of this page.

Relative roots Copy Section

Since 3.3.

Watchman supports optionally evaluating queries with respect to a path within a
watched root. This is used with the relative_root parameter:

["query", "/path/to/watched/root", {
  "relative_root": "project1",
}]

Setting a relative root results in the following modifications to queries:

• The path generator is evaluated with respect to the relative root. In the
  above example, "path": ["dir"] will return all files inside
  /path/to/watched/root/project1/dir.
• The input expression is evaluated with respect to the relative root. In the
  above example, "expression": ["match", "dir/*.txt", "wholename"] will return
  all files inside /path/to/watched/root/project1/dir/ that match the glob
  *.txt.
• Paths inside the relative root are returned with the relative root stripped
  off. For example, a path project1/dir/file.txt would be returned as
  dir/file.txt.
• Paths outside the relative root are not returned.

Relative roots behave similarly to a separate Watchman watch on the
subdirectory, without any of the system overhead that that imposes. This is
useful for large repositories, where your script or tool is only interested in a
particular directory inside the repository.

Install (`website/docs/install.md`) Copy Section

title: Installation category: Installation Copy Chapter

System Requirements Copy Chapter

Watchman is known to compile and pass its test suite on:

• Linux systems with inotify
• macOS (uses FSEvents on 10.7+, kqueue(2) on
  earlier versions)
• Windows 10 (64-bit) and up. Windows 7 support is
  provided by community patches

Watchman used to support the following systems, but no one is actively
maintaining them. The core of the code should be OK, but they likely don't
build. We'd love it if someone would step forward to maintain them:

• BSDish systems (FreeBSD 9.1, OpenBSD 5.2) that have the kqueue(2) facility
• Illumos and Solaris style systems that have port_create(3C)

Watchman relies on the operating system facilities for file notification, which
means that you will likely have very poor results using it on any kind of remote
or distributed filesystem.

Watchman does not currently support any other operating system not covered by
the list above.

Windows Copy Chapter

Prebuilt Binaries Copy Section

1. Download and extract the windows release from the
   latest release
2. It will be named something like watchman-vYYYY.MM.DD.00-windows.zip
3. It contains a bin folder. Move that somewhere appropriate and update your
   PATH environment to reference that location.

If you encounter issues with the Windows version of watchman, please report them
via GitHub!
You can find the list of known Windows issues here.

Installing via Chocolatey Copy Section

Watchman is available via the
Chocolatey Windows package
manager. Installation is as simple as:

PS C:\> choco install watchman

The package is maintained by the community rather than by Meta, so if you
experience issues with installation or uninstallation, you should
contact the package maintainers
for assistance.

macOS Copy Chapter

Homebrew Copy Section

Homebrew's Watchman package
is community-maintained, but it works well for many.

$ brew update
$ brew install watchman

If for some reason you can't wait for the Homebrew package to update, you can
install the latest build from GitHub:

$ brew install --HEAD watchman

MacPorts Copy Section

To install the package maintained by
MacPorts:

$ sudo port install watchman

Prebuilt Binaries Copy Section

1. Download and extract the macOS release from the
   latest release
2. It will be named something like watchman-vYYYY.MM.DD.00-macos.zip

$ unzip watchman-*-macos.zip
$ cd watchman-vYYYY.MM.DD.00-macos
$ sudo mkdir -p /usr/local/{bin,lib} /usr/local/var/run/watchman
$ sudo cp bin/* /usr/local/bin
$ sudo cp lib/* /usr/local/lib
$ sudo chmod 755 /usr/local/bin/watchman
$ sudo chmod 2777 /usr/local/var/run/watchman

The Watchman binaries are not signed, so manual approval in Security & Privacy
in System Preferences may be necessary.

Linux Copy Chapter

Homebrew Copy Section

If you use Homebrew on Linux, it's a great way to get a recent Watchman build.

Follow the macOS instructions above.

Fedora (Prebuilt RPMs) Copy Section

Warning: Do not install the Fedora-supplied Watchman package. It is old and
missing security, bug, and performance fixes.

1. From the
   latest release,
   download the .rpm corresponding to your Fedora version
2. sudo dnf localinstall watchman-$VERSION.fc$FEDORA_VERSION.x86_64.rpm
3. Confirm successful installation by running watchman version

Ubuntu (Prebuilt Debs) Copy Section

Warning: Do not install the Ubuntu-supplied Watchman package. It is old and
missing security, bug, and performance fixes.

1. From the
   latest release,
   download the .deb corresponding to your Ubuntu version
2. sudo dpkg -i watchman_$UBUNTU_RELEASE_$VERSION.deb
3. You will likely see errors about unresolved dependencies. The next step will
   resolve them.
4. sudo apt-get -f install
5. Confirm successful installation by running watchman version

Building from Source Copy Section

Download a
source snapshot from the latest release
or clone from GitHub.

$ cd watchman

# Ensure Cargo is installed. Either through your OS's package manager or https://rustup.rs/
$ cargo version

# Optionally, to save time, you can ask Watchman's build process to install system dependencies
$ sudo ./install-system-packages.sh

$ ./autogen.sh

Prebuilt Binaries Copy Section

Note: Our binaries are built from the main branch only. We don't provide
binaries for v4.9.0.

Note: The Linux binaries are compiled on a GitHub Action VM (ubuntu-20.04 at
the time of this writing), and Linux binaries are
not generally compatible across distributions,
so try the prebuilt Fedora, Ubuntu, or Homebrew packages first.

Watchman is continuously deployed as it passes our internal test validation
inside Meta and doesn't use manually assigned or "approved" version numbers.

Outside Meta we have automation that cuts a tag and builds binaries on Monday of
each week and assigns a tag based on the date. That process is in a beta state;
some or all of the binaries may not be present for any given tag.

1. Download and extract the release for your system from the
   latest release
2. It will be named something like watchman-vYYYY.MM.DD.00-linux.zip

$ unzip watchman-*-linux.zip
$ cd watchman-vYYYY.MM.DD.00-linux
$ sudo mkdir -p /usr/local/{bin,lib} /usr/local/var/run/watchman
$ sudo cp bin/* /usr/local/bin
$ sudo cp lib/* /usr/local/lib
$ sudo chmod 755 /usr/local/bin/watchman
$ sudo chmod 2777 /usr/local/var/run/watchman

System Specific Preparation Copy Chapter

Linux inotify Limits Copy Section

The inotify(7) subsystem has three important tunings that impact watchman.

• /proc/sys/fs/inotify/max_user_instances impacts how many different root dirs
  you can watch.
• /proc/sys/fs/inotify/max_user_watches impacts how many dirs you can watch
  across all watched roots.
• /proc/sys/fs/inotify/max_queued_events impacts how likely it is that your
  system will experience a notification overflow.

You obviously need to ensure that max_user_instances and max_user_watches
are set so that the system is capable of keeping track of your files.

max_queued_events is important to size correctly; if it is too small, the
kernel will drop events and watchman won't be able to report on them. Making
this value bigger reduces the risk of this happening.

Watchman has two simple strategies for mitigating an overflow of
max_queued_events:

• It uses a dedicated thread to consume kernel events as quickly as possible
• When the kernel reports an overflow, watchman will assume that all the files
  have been modified and will re-crawl the directory tree as though it had just
  started watching the dir.

This means that if an overflow does occur, you won't miss a legitimate change
notification, but instead will get spurious notifications for files that haven't
actually changed.

macOS File Descriptor Limits Copy Section

Only applicable on macOS 10.6 and earlier

The default per-process descriptor limit on macOS is extremely low (256!).

Watchman will attempt to raise its descriptor limit to match
kern.maxfilesperproc when it starts up, so you shouldn't need to mess with
ulimit; just raising the sysctl should do the trick.

The following will raise the limits to allow 10 million files total, with 1
million files per process until your next reboot.

$ sudo sysctl -w kern.maxfiles=10485760
$ sudo sysctl -w kern.maxfilesperproc=1048576

Putting the following into a file named /etc/sysctl.conf on macOS will cause
these values to persist across reboots:

kern.maxfiles=10485760
kern.maxfilesperproc=1048576

Nodejs (`website/docs/nodejs.md`) Copy Section

title: NodeJS category: Invocation sidebar_position: 6 Copy Chapter

To install the nodejs client:

$ npm install fb-watchman

and to import it and create a client instance:

var watchman = require('fb-watchman');
var client = new watchman.Client();

This documentation assumes that you are using the latest available version of
the fb-watchman package published to the npm repository.

Checking for watchman availability Copy Chapter

The client can be installed without requiring that the service is installed. It
is important to handle lack of availability and also to test whether the
installed service supports the capabilities required by your
application.

The capabilityCheck method issues a version command to query
the capabilities of the server.

var watchman = require('fb-watchman');
var client = new watchman.Client();
client.capabilityCheck({optional:[], required:['relative_root']},
  function (error, resp) {
    if (error) {
      // error will be an Error object if the watchman service is not
      // installed, or if any of the names listed in the `required`
      // array are not supported by the server
      console.error(error);
    }
    // resp will be an extended version response:
    // {'version': '3.8.0', 'capabilities': {'relative_root': true}}
    console.log(resp);
  });

Initiating a watch Copy Chapter

Almost every operation in watchman revolves around watching a directory tree.
You can repeatedly ask to watch the same directory without error; watchman will
re-use an existing watch.

var watchman = require('fb-watchman');
var client = new watchman.Client();

var dir_of_interest = "/some/path";

client.capabilityCheck({optional:[], required:['relative_root']},
  function (error, resp) {
    if (error) {
      console.log(error);
      client.end();
      return;
    }

    // Initiate the watch
    client.command(['watch-project', dir_of_interest],
      function (error, resp) {
        if (error) {
          console.error('Error initiating watch:', error);
          return;
        }

        // It is considered to be best practice to show any 'warning' or
        // 'error' information to the user, as it may suggest steps
        // for remediation
        if ('warning' in resp) {
          console.log('warning: ', resp.warning);
        }

        // `watch-project` can consolidate the watch for your
        // dir_of_interest with another watch at a higher level in the
        // tree, so it is very important to record the `relative_path`
        // returned in resp

        console.log('watch established on ', resp.watch,
                    ' relative_path', resp.relative_path);
      });
  });

Subscribing to changes Copy Chapter

Most node applications are interested in subscribing to live file change
notifications. In watchman these are configured by issuing a
subscribe command. A subscription is valid for the duration
of your client connection, or until you cancel the subscription using the
unsubscribe command.

The following will generate subscription results for all files in the tree that
match the query expression and then generate subscription results as files
change:

/* Detailed source-code truncated for AI context efficiency. */

Subscribing only to changed files Copy Section

The example above will generate results for existing (and deleted!) files at the
time that the subscription is established. In some applications this can be
undesirable. The following example shows how to add a logical time constraint.

watchman tracks changes using an abstract clock. We'll determine
the current clock at the time that we initiate the watch and then add that as a
constraint in our subscription.

function make_time_constrained_subscription(client, watch, relative_path) {
  client.command(['clock', watch], function (error, resp) {
    if (error) {
      console.error('Failed to query clock:', error);
      return;
    }

    sub = {
      // Match any `.js` file in the dir_of_interest
      expression: ["allof", ["match", "*.js"]],
      // Which fields we're interested in
      fields: ["name", "size", "exists", "type"],
      // add our time constraint
      since: resp.clock
    };

    if (relative_path) {
      sub.relative_root = relative_path;
    }

    client.command(['subscribe', watch, 'mysubscription', sub],
      function (error, resp) {
        // handle the result here
      });
  });
}

NodeJS API Reference Copy Chapter

Methods Copy Chapter

client.capabilityCheck(options, done) Copy Section

The capabilityCheck method issues a version command to query
the capabilities of the server.

If the server doesn't support capabilities, capabilityCheck will emulate the
capability response for a handful of significant capabilities based on the
version reported by the server.

The options argument may contain the following properties:

• optional an array listing optional capability names
• required an array listing required capability names

The properties are passed through to the underlying version command.

The done parameter is a callback that will be passed (error, result) when the
command completes. It doesn't make sense to issue a capabilityCheck call and
not provide the done callback.

The response object will contain a capabilities object property whose keys
will be the union of the optional and required capability names and whose
values will be either true or false depending on the availability of the
capability name.

If any of the required capabilities are not supported by the server, the
error parameter in the done callback will be set and will contain a
meaningful error message.

client.capabilityCheck({optional:[], required:['relative_root']},
  function (error, resp) {
    if (error) {
      // error will be an Error object if the watchman service is not
      // installed, or if any of the names listed in the `required`
      // array are not supported by the server
      console.error(error);
    }
    // resp will be an extended version response:
    // {'version': '3.8.0', 'capabilities': {'relative_root': true}}
    console.log(resp);
  });

client.command(args [, done]) Copy Section

Sends a command to the watchman service. args is an array that specifies the
command name and any optional arguments. The command is queued and dispatched
asynchronously. You may queue multiple commands to the service; they will be
dispatched in FIFO order once the client connection is established.

The done parameter is a callback that will be passed (error, result) when the
command completes. You may omit it if you are not interested in the result of
the command.

client.command(['watch-project', process.cwd()], function(error, resp) {
  if (error) {
    console.log('watch failed: ', error);
    return;
  }
  if ('warning' in resp) {
    console.log('warning: ', resp.warning);
  }
  if ('relative_path' in resp) {
    // We will need to remember and adjust for relative_path
    console.log('watching project ', resp.watch, ' relative path to cwd is ',
      resp.relative_path);
  } else {
    console.log('watching ', resp.watch);
  }
});

If a field named warning is present in resp, the watchman service is trying
to communicate an issue that the user should see and address. For example, if
the system watch resources need adjustment, watchman will provide information
about this and how to remediate the issue. It is suggested that tools that build
on top of this library bubble the warning message up to the user.

client.end() Copy Section

Terminates the connection to the watchman service. Does not wait for any queued
commands to send.

Events Copy Chapter

The following events are emitted by the watchman client object:

Event: 'connect' Copy Section

Emitted when the client successfully connects to the watchman service

Event: 'error' Copy Section

Emitted when the socket to the watchman service encounters an error.

It may also be emitted prior to establishing a connection if we are unable to
successfully execute the watchman CLI binary to determine how to talk to the
server process.

It is passed a variable that encapsulates the error.

Event: 'end' Copy Section

Emitted when the socket to the watchman service is closed

Event: 'log' Copy Section

Emitted in response to a unilateral log PDU from the watchman service. To
enable these, you need to send a log-level command to the service:

// This is very verbose, you probably don't want to do this
client.command(['log-level', 'debug']);
client.on('log', function(info) {
  console.log(info);
});

Event: 'subscription' Copy Section

Emitted in response to a unilateral subscription PDU from the watchman
service. To enable these, you need to send a subscribe command to the service:

// Subscribe to notifications about .js files
  client.command(['subscribe', process.cwd(), 'mysubscription', {
      expression: ["match", "*.js"]
    }],
    function(error, resp) {
      if (error) {
        // Probably an error in the subscription criteria
        console.log('failed to subscribe: ', error);
        return;
      }
      console.log('subscription ' + resp.subscribe + ' established');
    }
  );

  // Subscription results are emitted via the subscription event.
  // Note that watchman will deliver a list of all current files
  // when you first subscribe, so you don't need to walk the tree
  // for yourself on startup
  client.on('subscription', function(resp) {
    console.log(resp.root, resp.subscription, resp.files);
  });

To cancel a subscription, use the unsubscribe command and pass in the name of
the subscription you want to cancel:

client.command(['unsubscribe', process.cwd(), 'mysubscription']);

Note that subscriptions names are scoped to your connection to the watchman
service; multiple different clients can use the same subscription name without
fear of colliding.

Release Notes (`website/docs/release-notes.md`) Copy Section

title: Release Notes category: Installation Copy Chapter

Watchman is continuously deployed inside Facebook, which means that we don't
explicitly maintain version numbers. We have automation that cuts a weekly tag
with a named derived from the date. You can learn more about how to reason about
supported capabilities and our backwards compatibility guidelines in the
Compatibility Rules docs.

We focus on the highlights only in these release notes. For a full history that
includes all of the gory details, please see
the commit history on GitHub.

Watchman v2020.07.13.00 Copy Section

• Added script watchman-replicate-subscription. It can replicate an existing
  watchman subscription. Integrators can use this script to validate watchman
  notifications their client is receiving.
• Added support for suffix sets in suffix expressions. You now can specify
  multiple suffixes to match against by setting the second argument to a list of
  suffixes. See suffix-set documentation for
  more details
• pywatchman: introduced new pywatchman_aio client for python
• Windows: we no longer trust environment variables to locate the state
  directory which should result in a better experience for users that mix
  cygwin, mingw, native windows and/or WSL or other environments
• Windows: we now support unix domain sockets on Windows 10. The CLI will prefer
  to use unix domain sockets when available.

We weren't great at updating the release notes since the prior release; there
was a lot of work to support our sister project EdenFS that isn't broadly
relevant to those outside FB at the time of writing.

Watchman 4.9.0 (2017-08-24) Copy Section

• New field: content.sha1hex. This field expands to the SHA1 hash of the file
  contents, expressed in hex digits (40 character hex string). Watchman
  maintains a cache of the content hashes and can compute the hash on demand and
  also heuristically as files are changed. This is useful for tooling that wants
  to perform more intelligent cache invalidation or build artifact fetching from
  content addressed storage.
• Experimental feature: Source Control Aware query mode. Currently supports only
  Mercurial (patches to add Git support are welcomed!). SCM aware query mode
  helps to keep response sizes closer to O(what-you-changed) than to
  O(all-repo-changes) when rebasing your code. Using this feature effectively
  may require some additional infrastructure to compute and associate data with
  revisions from your repo.
• Fixed an issue that resulted in the perf logging thread deadlocking when
  perf_logger_command is enabled in the global configuration
• Fixed an issue where queries larger than 1MB would likely result in a PDU
  error response.
• Reduced lock contention for subscriptions that do no use the advanced settling
  (drop, defer) options.
• Fixed since generator behavior when using unix timestamps rather than the
  preferred clock string syntax
• Improved the reporting of "new" files in watchman results
• Improved performance of handling changes on case insensitive filesystems
• Windows: promoted from alpha to beta status!
• Windows: fixed some performance and reliability issues
• Windows: now operates correctly on Windows 7
• Windows: can now see and report symlinks and junction points
• Windows: fixed potential deadlock in trigger deletion
• Windows: fixed stack trace rendering on win32
• Windows: improved IO scheduling around deletes on win32
• Windows: improved handling of case insensitive win32 driver letters
• pywatchman: the python wheel format is used for publishing watchman pypi
  package
• pywatchman: now watchman path is configurable in python client
• pywatchman: now python client can be used as a context manager
• Solaris: support for Solaris has been removed. If you'd like to commit to
  testing and maintaining Solaris support, we'd love to hear from you!

Watchman 4.8.0 (never formally released) Copy Section

Whoops, we never got around to tagging this beyond a release candidate tag!

• New command flush-subscriptions to synchronize subscriptions associated with
  the current session.
• On Windows, return / as the directory separator. Previously we used \.
  This change should be pretty neutral for clients, and makes it easier to work
  with both the internals and the integration test infrastructure.
• Enforce socket Unix groups more strongly — Watchman will now refuse to start
  if it couldn't gain the right group memberships, as can happen for sites that
  are experiencing intermittent LDAP connectivity problems.
• pywatchman now officially supports Python 3. pywatchman will return Unicode
  strings (possibly with surrogate escapes) by default, but can optionally
  return bytestrings. Note that on Python 3, pywatchman requires Watchman 4.8
  and above. The Python 2 interface and requirements remain unchanged.
• Prior to 4.8, methods on the Java WatchmanClient that returned
  ListenableFutures would swallow exceptions and hang in an unfinished state
  under situations like socket closure or thread death. This has been fixed, and
  now ListenableFutures propagate exception conditions immediately. (Note that
  this is typically unrecoverable, and users should create a new WatchmanClient
  to re-establish communication with Watchman.) See #412.
• The minimum Java version for the Watchman Java client has always been 1.7, but
  it was incorrectly described to be 1.6. The Java client's build file has been
  fixed accordingly.
• Watchman was converted from C to C++. The conversion exposed several
  concurrency bugs, all of which have now been fixed.
• Subscription queries are now executed in the context of the client thread,
  which means that subscriptions are dispatched in parallel. Previously,
  subscriptions would be serially dispatched and block the disk IO thread.
• Triggers are now dispatched in parallel and waits are managed in their own
  threads (one thread per trigger). This improves concurrency and resolves a
  couple of waitpid related issues where watchman may not reap spawned children
  in a timely fashion, or may spin on CPU until another child is spawned.
• Fixed an object lifecycle management issue that could cause a crash when aging
  out old/transient files.
• Implement an upgraded wire protocol, BSERv2, on the server and in pywatchman.
  BSERv2 can carry information about string encoding over the wire. This lets
  pywatchman convert to Unicode strings on Python 3. Clients and servers know
  how to transparently fall back to BSERv1.
• OS X: we no longer use socket activation when registering with launchd. This
  was the source of some upgrade problems for mac Homebrew users.

Watchman 4.7.0 (2016-09-10) Copy Section

• Reduced memory usage by 40%
• Queries can now run with a shared lock. It is recommended that clients move
  away from the n:FOO style server side named cursor clockspecs to take full
  advantage of this.
• Added new glob generator as a walking strategy for queries. This allows
  watchman to evaluate globs in the most efficient manner. Our friends in the
  Buck project have already integrated this into their BUCK file parsing to
  evaluate globs without touching the filesystem!
• Added "case_sensitive": true option to queries to force matches to happen in
  a case sensitive manner, even if the watched root is on a case insensitive
  filesystem. This is used to accelerate certain types of internal traversal: if
  we know that a path is case sensitive we can perform an O(1) lookup where we
  would otherwise have to perform an O(number-of-directory-entries) scan and
  compare.
• Fixed a race condition during subscription initiation that could emit
  incorrect clock values.
• Fixed spurious over-notification for parent directories of changed files on
  Mac.
• Fixed some reliability issues on Windows

Watchman 4.6.0 (2016-07-09) Copy Section

• Improved I/O scheduling when processing recursive deletes and deep directory
  rename operations.
• Improved performance of the ignore_dirs configuration option on OS X and
  Windows systems. We take advantage of an undocumented (but supported!) API to
  further accelerate this for the first 8 entries in the ignore_dirs on OS X.
  Users that depend on this configuration to avoid recrawls will want to review
  and prioritize their most active build dirs to the front of the ignore_dirs
  specified in their .watchmanconfig file.
• Added an optional recrawl recovery strategy for OS X that will attempt to
  resync from the fseventsd journal rather than performing a full filesystem
  walk. This is currently disabled by default but will likely be enabled by
  default in the next Watchman release. You can enable this by setting
  fsevents_try_resync: true in either /etc/watchman.json or your
  .watchmanconfig. This should reduce the frequency of recrawl warnings for
  some users/workloads, and also improves I/O for users with extremely large
  trees.
• Fixed accidental exponential time complexity issue with recursive deletes and
  deep directory rename operations on case-insensitive filesystems (such as OS
  X). This manifested as high CPU utilization for extended periods of time.
• Added support for allowing non-owner access to a Watchman instance. Only the
  owner is authorized to create or delete watches. Non-owners can view
  information about existing watches. Access control is based on unix domain
  socket permissions. The new but not yet documented configuration options
  sock_group and sock_access can be used to control this new behavior.
• Added support for inetd-style socket activation of the watchman service.
  this commit includes a sample configuration for systemd.
• Added the symlink_target field to the stored metadata for files. This holds
  the text of the symbolic link for symlinks. You can test whether it is
  supported by a watchman server using the capability name
  field-symlink_target.
• Fixed an issue where watchman may not reap child processes spawned by
  triggers.
• Fixed an issue where watchman may block forever during shutdown if there are
  other connected clients.
• Added hint_num_dirs configuration option.

pywatchman 1.4.0 (????-??-??) Copy Section

(These changes have not yet been released to pypi)

• Added immutable version of data results to bser. This is cheaper to build from
  a serialized bser representation than the mutable version and is better suited
  to large result sets received from watchman.
• Fixed a number of misc. portability issues
• Added Python 3.x support

Watchman 4.5.0 (2016-02-18) Copy Section

• Fixed an inotify race condition for non-atomic directory replacements that was
  introduced in Watchman 4.4.

Watchman 4.4.0 (2016-02-02) Copy Section

• Added state-enter and state-leave commands can allow subscribers to more
  intelligently settle/coalesce events around hg update or builds.
• Fixed an issue where subscriptions could double-notify for the same events.
• Fixed an issue where subscriptions that never match any files add
  O(all-observed-files) CPU cost to every subscription dispatch

Watchman 4.3.0 (2015-12-14) Copy Section

• Improved handling of case insensitive renames; halved the memory usage and
  doubled crawl speed on OS X.

Watchman 4.2.0 (2015-12-08) Copy Section

• Increased strictness of checks for symlinks; rather than just checking whether
  the leaf of a directory tree is a symlink, we now check each component down
  from the root of the watch. This improves detection and processing for
  directory-to-symlink (and vice versa) transitions.
• Increased priority of the watchman process on OS X.

pywatchman 1.3.0 (2015-10-22) Copy Section

• Added watchman-make and watchman-wait commands
• Added pure python implementation of BSER

Watchman 4.1.0 (2015-10-20) Copy Section

• Fixed an issue where symlink size was always reported as 0 on OS X using the
  new bulkstat functionality

Watchman 4.0.0 (2015-10-19) Copy Section

• Fixed an issue where a directory that was replaced by a symlink would cause a
  symlink traversal instead of correctly updating the type of the node and
  marking the children removed.
• Fixed a debugging log line that was emitted at the wrong log level on every
  directory traversal.

Watchman 3.9.0 (2015-10-12) Copy Section

• Fixed an issue where dir renames on OS X could cause us to lose track of the
  files inside the renamed dir
• Fixed an issue where dir deletes and replacements on Linux could cause us to
  lose track of the files inside the replaced dir (similar to the OS X issue
  above in manifestation, but a different root cause).
• Improved (re)crawl speed for dirs with more than a couple of entries on
  average (improvement can be up to 5x for dirs with up to 64 entries on
  average). You may now tune the hint_num_files_per_dir setting in your
  .watchmanconfig to better match your tree.
  More details
• Improved (re)crawl speed on OS X 10.10 and later by using getattrlistbulk.
  This allows us to improve the data:syscall ratio during crawling and can
  improve throughput by up to 40% for larger trees.
• Add optional sync_timeout to the clock command
• Avoid accidentally passing descriptors other than the stdio streams when we
  spawn the watchman service.
• Fixed a race condition where we could start two sets of watcher threads for
  the same dir if two clients issue a watch or watch-project at the same
  time
• Added a helpful error for a tmux + launchd issue on OS X

Watchman 3.8.0 (2015-09-14) Copy Section

• Improved latency of processing kernel notifications. It should now be far less
  likely to run into an notification queue overflow.
• Improved idle behavior. There were a couple of places where watchman would
  wake up more often than was strictly needed and these have now been fixed.
  This is mostly of interest to laptop users on battery power.
• Improved inotify move tracking. Some move operations could cause watchman to
  become confused and trigger a recrawl. This has now been resolved.
• Hardened statedir and permissions. There was a possibility of a symlink attack
  and this has now been mitigated by re-structuring the statedir layout.
• Fixed a possible deadlock in the idle watch reaper
• Fixed an issue where the watchman -p log-level debug could drop log
  notifications in the CLI
• Disabled the IO-throttling-during-crawl that we added in 3.7. It proved to be
  more harmful than beneficial.
• -j CLI option now accepts either JSON or BSER encoded command on stdin
• Added capabilities to the server, and added the
  capabilityCheck method to the python and
  node clients.

pywatchman 1.2.0 (2015-08-15) Copy Section

• Added the capabilityCheck method
• Added SocketTimeout exception to distinguish timeouts from protocol level
  exceptions

fb-watchman 1.3.0 for node (2015-08-15) Copy Section

• Added the capabilityCheck
  method.

pywatchman 1.0.0 (2015-08-06) Copy Section

• First official pypi release, thanks to @kwlzn for
  setting up the release machinery for this.

Watchman 3.7.0 (2015-08-05) Copy Section

(Watchman 3.6.0 wasn't formally released)

• Fixed bug where query match on foo*.java with wholename scope would
  incorrectly match foo/bar/baz.java.
• Added src/**/*.java recursive glob pattern support to query match.
• Added options dictionary to query's match operator.
• Added includedotfiles option to query match to include files whose names
  start with ..
• Added noescape option to query match to make \ match literal \.
• We'll now automatically age out and stop watches. See
  idle_reap_age_seconds for more information.
• watch-project will now try harder to re-use an existing watch and avoid
  creating an overlapping watch.
• Reduce I/O priority during crawling on systems that support this
• Fixed issue with the long long data type in the python BSER module

fb-watchman 1.2.0 for node (2015-07-11) Copy Section

• Updated the node client to more gracefully handle undefined values in
  objects when serializing them; we now omit keys whose values are undefined
  rather than throw an exception.

Watchman 3.5.0 (2015-06-29) Copy Section

• Fix the version number reported by watchman.

Watchman 3.4.0 (2015-06-29) Copy Section

• trigger now supports an optional relative_root argument. The trigger is
  evaluated with respect to this subdirectory. See
  trigger for more.

fb-watchman 1.1.0 for node (2015-06-25) Copy Section

• Updated the node client to handle 64-bit integer values using the
  node-int64. These are most likely
  to show up if your query fields include size and you have files larger than
  2GB in your watched root.

fb-watchman 1.0.0 for node (2015-06-23) Copy Section

• Updated the node client to support BSER encoding, fixing a
  quadratic performance issue in the JSON stream decoder that was used
  previously.

Watchman 3.3.0 (2015-06-22) Copy Section

• query and subscribe now support an optional relative_root argument.
  Inputs and outputs are evaluated with respect to this subdirectory. See
  File Queries for more.

Scm Query (`website/docs/scm-query.md`) Copy Section

title: Source Control Aware Queries category: Queries Copy Chapter

Since 2021.08.30

Git support has been added 🎊

Since 4.9.

The capability name associated with this enhanced
functionality is scm-since.

The capability name for this is scm-hg. The internal architecture allows
supporting other source control systems quite easily; it just needs someone to
implement and test them!

A common pattern for tools that consume watchman is wanting to reason about the
changes in a version controlled repository. For most repos it is fine to simply
receive information about all changed files as they are updated, even during a
rebase over several days of work by others.

For very large or very busy repositories, where a great many files can change
over a short period of time, it can be desirable to get a minimized set of
information about the changes.

For example, if your tool has the ability to load some pre-built data from some
artifact storage, rather than processing many hundreds of changed files
incrementally you may want to take the merge base of local changes and use that
to locate the pre-built data and process only the delta between that state and
the current state of the repo.

An illustration may help. Here we see that a user has a stack of two commits
based off the symbolic main commit. In this scenario, main is tracking the
tip of the repo to which the local repo is published, and the user is checked
out at the 6b38a5 commit:

| @  6b38a5  wez
| |  Add cats.cpp
| |
| o  fa2e92  wez
|/   Add cat.jpg
|
o f12345 main

Now the user synchronizes their repo with the remote, fetching the commits but
not changing their work yet. This is often combined with the step that follows,
but we are breaking it out here for the purposes of illustration. This is
equivalent to running hg pull or git fetch:

o  fabf87  coworker     main
.  Amazing new feature
.
| @  6b38a5  wez
| |  Add cats.cpp
| |
| o  fa2e92  wez
|/   Add cat.jpg
|
o

The ellipsis portion of the DAG represents uninteresting commits to wez; there
may be hundreds of files changed by those commits, but wez only cares about
the work in their local branch of the DAG.

Now wez wants to rebase their work on main. This would be done using a command
like hg rebase -d main -s fa2e92:

| @  bbbbbb  wez
| |  Add cats.cpp
| |
| o  aaaaaa  wez
|/   Add cat.jpg
|
o  fabf87  coworker     main
.  Amazing new feature
.

The crucial part of this is what happens to the working copy; assuming that we
now land on commit bbbbbb, Watchman will observe changes for all of the
hundreds of files that changed across the rebase and pass this information on to
the tools that are subscribed or are querying for this information.

If your tooling is source control aware then you can ask watchman to run since
queries in a mode where it will return you information about the merge base with
main and the minimized set of files that changed.

To enable this mode you issue a query using a new fat clock as the since
parameter for the query:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "since": {
      "scm": {
        "mergebase-with": "main"
      }
  },
  "expression": ["type", "f"],
  "fields": ["name"]
}]
EOT

This particular since value starts with an unspecified clock value and
requests that watchman run the query in source control aware mode, using the
symbolic name main to compute the merge base for the commit graph.

If we look back to the illustrations above and rewind to the first scenario, the
results of this query will look something like this:

{
   "clock": {
       "clock": "c:123:123",
       "scm": {
            "mergebase": "f12345",
            "mergebase-with": "main"
       }
    },
    "files": ["cat.jpg", "cats.cpp"]
}

This result informs the client of the merge base with main (which happens to be
main itself) and the list of changes since that merge base.

To get the next incremental change the client feeds that clock value back in to
its next query. Looking back to the second illustration above, if we were to run
this query after the running hg pull (note that this doesn't change the
working copy):

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "since": {
       "clock": "c:123:123",
       "scm": {
            "mergebase": "f12345",
            "mergebase-with": "main"
       }
  },
  "expression": ["type", "f"],
  "fields": ["name"]
}]
EOT

we'd get this result:

{
   "clock": {
       "clock": "c:123:124",
       "scm": {
            "mergebase": "f12345",
            "mergebase-with": "main"
       }
    },
    "files": []
}

Note that the files list is empty because we didn't change any files, and note
that one of the numeric portions of the clock string has changed.

Also note that the mergebase revision remains the same because we also didn't
rebase the commit yet.

This is a little white lie: the reality is that some files did change in the
version control system, and with the expression we're using we would see them,
but they are not part of the working copy so we're omitting them for the clarity
of this example.

Now if we rebase and update to the rebased revision (taking us to the last of
the illustrations from above), we'd run this query, feeding in the clock from
the last query to get the correct incremental result:

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "since": {
       "clock": "c:123:124",
       "scm": {
            "mergebase": "f12345",
            "mergebase-with": "main"
       }
  },
  "expression": ["type", "f"],
  "fields": ["name"]
}]
EOT

we'd get this result:

{
   "clock": {
       "clock": "c:123:125",
       "scm": {
            "mergebase": "fabf87",
            "mergebase-with": "main"
       }
    },
    "files": ["cat.jpg", "cats.cpp"]
}

Note that the mergebase reported in the clock has changed and note that the list
of files reported is just the two from our commit stack despite there being
hundreds of files that were physically updated on the disk.

Your client can now lookup some state based on the fabf87 revision and
download it, and can then incrementally apply the computation for cat.jpg and
cats.cpp on top of that state.

If your client doesn't know how to do this, then you shouldn't use this source
control aware query mode!

Source Control Aware Subscriptions Copy Chapter

You can also use the same source control awareness in your subscriptions. This
is basically the same procedure as making queries above, but there are some
preconditions and things to note:

• Watchman needs the cooperation of the source control system to know when it
  should defer events.
• Source control aware subscriptions implicitly enable defer_vcs and
  defer:["hg.update"]. As with the point above, this is to ensure that you
  don't get notified about files changing during the working copy update
  operation; that would defeat the point of using source control awareness.

To initiate a source control aware subscription:

["subscribe", "/path/to/root", "mysubscriptionname", {
  "fields": ["name"],
  "since": {
    "scm": {
      "mergebase-with": "main"
    }
  }
}]

You'll then receive subscription responses as files change; those responses will
contain fat clock values for the since and clock fields:

{
  "subscription": "mysubscriptionname",
  "clock": {
    "clock": "c:1234:125",
    "scm": {
      "mergebase": "fabf87",
      "mergebase-with": "main",
    }
  },
  "since": {
    "clock": "c:1234:123",
    "scm": {
      "mergebase": "f12345",
      "mergebase-with": "main",
    }
  },
  "files": ["cat.jpg", "cats.cpp"],
  "root":  "/path/to/root"
}

The clock field holds the value of the clock and the merge base as of the
subscription notification.

The since field holds the fat clock that was returned in the clock field
from the prior subscription update. It is present as a convenience for you; you
can compare the mergebase fields between the two to determine that the merge
base changed in this update. This is an important detail because more files in
the working copy have been physically changed than are reflected in the files
list; your tooling will need to so something appropriate to ensure that it
computes a consistent and correct result.

`state-enter` & `state-leave` Copy Section

Source control aware subscriptions will always include a fat clock in their
responses, however, only the regular clock is provided in state-enter and
state-leave notifications. This is because computing the source control
information is a non-trivial operation and could increase latency.

Simple Query (`website/docs/simple-query.md`) Copy Section

title: Simple Pattern Syntax category: Queries Copy Chapter

Simple patterns follow a more traditional UNIX command line approach of using
command line switches to indicate the nature of the pattern match. When simple
patterns are used, the result set unconditionally includes all core file
metadata fields. They are described in more detail below.

Simple Pattern syntax Copy Chapter

Where you see [patterns] in the command syntax for the find, since and
trigger commands, we allow filename patterns that match according the
following rules:

• We maintain an inclusion and an exclusion list. As the arguments are
  processed we'll accumulate them in one or the other. By default they are
  accumulated into the inclusion list.
• -X causes any subsequent items to be placed into the exclusion list
• -I causes any subsequent items to be placed into the inclusion list
• -- indicates the end of the set of patterns
• -p indicates that the following pattern should use pcre as the expression
  term. This is reset after generating the next term.
• -P indicates that the following pattern should use ipcre as the expression
  term and perform a case insensitive match. This is reset after generating the
  next term.
• If neither -p nor -P were used, the generated term will use match
• ! followed by a space followed by a pattern will negate the sense of the
  pattern match generating a not term.

Any elements in the inclusion list will match; they are composed together using
an "anyof" term.

The inclusion list and exclusion lists are composed using the logic
(NOT anyof exclusion) AND (anyof inclusion).

For example:

'*.c'

Generates a file expression:

["match", "*.c", "wholename"]

A list:

'*.js' '*.css'

["anyof",
  ["match", "*.js", "wholename"],
  ["match", "*.css", "wholename"]
]

An example of how the exclusion list syntax works:

-X '*.c' -I '*main*'

Generates:

["allof",
  ["not", ["match", "*.c", "wholename"]],
  ["match", "*main*", "wholename"]
]

Socket Interface (`website/docs/socket-interface.md`) Copy Section

title: Socket Interface category: Invocation sidebar_position: 8 Copy Chapter

Most simple uses of Watchman will invoke the watchman binary and process its
output. Sometimes it is desirable to avoid the overhead of an extra process and
talk directly to your watchman service.

The watchman service runs as a single long-lived process per user. The watchman
binary will take care of spawning the server process if necessary.

The server will create a unix domain socket for communication with its clients.
The location of the socket depends on compile time options and command line
flags. It is recommended that you invoke watchman get-sockname to discover the
location, or if you are being invoked via a trigger (since version 2.9.7) you
will find the location in the $WATCHMAN_SOCK environmental variable.

Watchman Protocol Copy Chapter

The unix socket implements a request-response protocol with PDUs encoded in
either JSON or BSER representation. Some watchman commands (notably subscribe
and log-level) allow the watchman service to unilaterally send any number of
PDUs to the client, and require more stateful handling.

JSON encoding Copy Section

The JSON encoding represents a request or a response as a single line of compact
JSON encoded data. The newline is used to detect the end of the PDU.

Requests from the client are always represented as a JSON array.

Responses from the server are always represented as a JSON object.

Sending the since command is simply a matter of formatting it as JSON. Note
that the JSON text must be a single line (don't send a pretty printed version of
it!) and be followed by a newline \n character:

["since", "/path/to/src", "n:c_srcs", "*.c"] <NEWLINE>

BSER encoding Copy Section

BSER is a local-only binary serialization format that can represent the same
data types as JSON, but in a more compact form and not be limited to UTF-8
representation of strings.

When you make a request using BSER, the server will respond in BSER encoding.

You can read more about BSER in the BSER specification.

Reporting Errors and Warnings Copy Section

If a Response includes a field named error it indicates that the request was
not successful. The error field, if present, is a string value that can be
presented to the user.

If a Response includes a field named warning it indicates that there is some
condition that should be reported back to the user so that the system can
operate more effectively. Tools should display this to the user as an advisory.
At the time of writing, this can be triggered due to overflowing the system
limits on file watching resources. These are important to address so that events
are not missed and thus so that correctness and system performance are
maintained.

The warning message contains a link to this documentation that provides advice
on tuning and resolving the issue.

Troubleshooting (`website/docs/troubleshooting.md`) Copy Section

title: Troubleshooting Copy Chapter

We try to give directed advice in Watchman error diagnostics, which means that
we will show a link to a section on this page with some context and advice where
we have enough information to do so. Some operating systems provide richer
diagnostic information than others, so we have to resort to more generic advice
in some cases.

The most common cause of problems is hitting system resource limits. There are
finite resources available for filesystem watching, and when they are exceeded
it can impact performance in the best case or prohibit correct operation in the
worst case.

Ensure that you are on the best available version Copy Chapter

It is generally a good idea to make sure that you are using the latest version
of the software, so that you avoid any known issues.

If you are running a pre-built binary provided by your operating system
distribution system, there is a chance that you'll need to build the latest
version from source. You can find instructions for this in
the installation section.

Recrawl Copy Chapter

A recrawl is an action that Watchman performs in order to recover from
situations where it believes that it has lost sync with the state of the
filesystem.

The most common cause for a recrawl is on Linux systems where the default
inotify limits are sized quite small. What this means is that the rate at which
your watched roots are generating changes is higher than the kernel can buffer
and relay to the watchman service. When this happens, the kernel detects the
overflow and signals IN_Q_OVERFLOW. The recovery is to recursively scan the
root to make sure that we know what is really there and re-sync with the
notification stream.

Frequent recrawls are undesirable because they result in a potentially expensive
full tree crawl, which marks all files as changed and propagates this status to
clients which will in turn perform some action on the (likely falsely) changed
state of the majority of files.

Avoiding Recrawls Copy Section

There is no simple formula for setting your system limits; bigger is better but
comes at the cost of kernel memory to maintain the buffers. You and/or your
system administrator should review the workload for your system and the
System Specific Preparation Documentation
and raise your limits accordingly.

kFSEventStreamEventFlagUserDropped Copy Section

macOS has a similar internal limit and behavior when that limit is exceeded. If
you're encountering a message like:

Recrawled this watch 1 time, most recently because:
/some/path: kFSEventStreamEventFlagUserDropped

then you are hitting the limits of your system. There is no direct control over
the limit, but starting in Watchman 3.2 you may increase the
fsevents_latency parameter in your
.watchmanconfig file.

I've changed my limits, how can I clear the warning? Copy Section

The warning will stick until you cancel the watch and reinstate it, or restart
the watchman process. The simplest resolution is to run
watchman shutdown-server and re-establish your watch on your next watchman
query.

Where are the logs? Copy Chapter

Watchman places logs in a file named <STATEDIR>/.log, where STATEDIR
is set at the time that you built watchman.

If you used the --enable-statedir=<STATEDIR> configure option, that will be
the location that holds your logs. If not, the default for STATEDIR will be
/var/run/watchman, or for older versions of watchman, the logs may be
placed in <TMPDIR>/.watchman..log.

Since 3.8.

Watchman places the logs in a file named <STATEDIR>/log, which will typically
be a location like /var/run/watchman/-state/log. If you're
running a homebrew build of watchman, `` is usually /usr/local.

The default log location may be overridden by the --logfile
Server Option.

Quick note on default locations
explains what we mean by <STATEDIR>, <TMPDIR>, `` and so on.

Poison: inotify_add_watch Copy Chapter

A non-recoverable condition has triggered.  Watchman needs your help!
The triggering condition was at timestamp=1407695600: inotify-add-watch(/my/path) -> Cannot allocate memory
All requests will continue to fail with this message until you resolve
the underlying problem.  You will find more information on fixing this at
https://facebook.github.io/watchman/docs/troubleshooting.html#poison-inotify-add-watch

If you've encountered this state it means that your kernel was unable to watch
a dir in one or more of the roots you've asked it to watch. This particular
condition is considered non-recoverable by Watchman on the basis that nothing
that the Watchman service can do can guarantee that the root cause is resolved,
and while the system is in this state, Watchman cannot guarantee that it can
respond with the correct results that its clients depend upon. We consider
ourselves poisoned and will fail all requests for all watches (not just the
watch that it triggered on) until the process is restarted.

There are two primary reasons that this can trigger:

• The user limit on the total number of inotify watches was reached or the
  kernel failed to allocate a needed resource
• Insufficient kernel memory was available

The resolution for the former is to revisit
System Specific Preparation Documentation
and raise your limits accordingly.

The latter condition implies that your workload is exceeding the available RAM
on the machine. It is difficult to give specific advice to resolve this
condition here; you may be able to tune down other system limits to free up some
resources, or you may just need to install more RAM in the system.

I've changed my limits, how can I clear the error? Copy Section

The error will stick until you restart the watchman process. The simplest
resolution is:

Since 4.6

$ watchman watch-del-all
$ watchman shutdown-server

Before 4.6

$ rm <STATEDIR>/state       # see above for what STATEDIR means
$ watchman --no-spawn --no-local shutdown-server

If you have not actually resolved the root cause you may continue to trigger and
experience this state each time the system trips over these limits.

Poison: opendir Copy Chapter

A non-recoverable condition has triggered.  Watchman needs your help!
The triggering condition was at timestamp=1407695600: opendir(/my/path) -> Too many open files in system
All requests will continue to fail with this message until you resolve
the underlying problem.  You will find more information on fixing this at
https://facebook.github.io/watchman/docs/troubleshooting.html#opendir

If you've encountered this state it means that your entire system had too many
open files, and that this prevented watchman from tracking the changes on your
system. In this case, the error isn't related to filesystem watching but to
other (likely) misbehaving processes on your system; it's usually indicative of
a runaway program or set of programs consuming resources, but in some cases it
may just be that your system workload requires that you increase your system
limits for the number of files.

How do I resolve this? Copy Section

Follow these directions

If the issue persists, consult your system administrator to identify what is
consuming these resources and remediate it, or to increase your system limits.

FSEvents Copy Chapter

FSEvents is the file watching facility on macOS. There are few diagnostics that
can help diagnose issues with FSEvents; the API itself gives little feedback on
a number of error cases and instead emits rather cryptic error messages to the
log file.

If you got here because an error message told you to read this section, it will
have also asked you to look at your log file. If you are using an older version
of watchman and encounter the error message FSEventStreamStart failed, then
you should locate your log file (see Where are the logs?
above) and look for lines that mention FSEvents and then consult the information
below.

FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21) Copy Section

Nobody outside of Apple is sure what precisely this means, but it indicates that
the fsevents service has gotten in a bad state. Possible reasons for this may
include:

• There are too many event stream clients
• One or more event stream clients has gotten in a bad state and is somehow
  impacting the fsevents service

To resolve this issue, you may wish to try the following, which are
progressively more invasive:

• Avoid establishing multiple overlapping watches within the same filesystem
  tree, especially for large trees. We recommend watching only the root of a
  project or repo and not watching sub-trees within that tree. Organizations
  with large trees may wish to deploy the
  root_restrict_files configuration option so
  that watchman will only allow watching project roots.
• Close or restart other applications that are using fsevents. Some examples
  are:
• editors such as Sublime Text and TextMate.
• Many nodejs packages and Grunt style workflows make use of fsevents. Make sure
  that you upgrade nodejs to at least version v0.11.14. If possible, configure
  your nodejs packages to use either sane
  or fb-watchman for file watching
  as this will consolidate the number of fsevents watches down to just the set
  maintained by watchman.
• Restart the fsevents service: sudo pkill -9 -x fseventsd
• Restart your computer

Triggers/Subscriptions don't fire on macOS Copy Chapter

There is a rare fsevents bug that can prevent any notifications from working in
directories where the case of the name of a directory in the kernel has an
inconsistency.

You can test whether this is happening to you by following
the instructions for the find-fsevents-bugs tool.

If it is happening to you, the resolution is to rename the directories
highlighted by the tool.

You can read more about this issue in the following resources:

• Knowledge base article for LiveReload
• issue for the Ruby fsevents module
• Open Radar bug report

ReactNative: Watcher took too long to load Copy Chapter

There was an issue that was the result of umask affecting the permissions of the
launchd plist file that Watchman uses to set up your watchman service on OS X.
This issue was resolved in Watchman version 3.1.

To update:

$ watchman shutdown-server
$ brew update
$ brew reinstall watchman

Watchman Make (`website/docs/watchman-make.md`) Copy Section

title: watchman-make category: Invocation sidebar_position: 2 Copy Chapter

watchman-make is a convenience tool to help automatically invoke a build tool
or script in response to files changing. It is useful to automate building
assets or running tests as you save files during development.

watchman-make will establish a watch on the files you specify and remain
running in the foreground, waiting for changes to occur. When a change is
triggered, your build tool or script will be run in the foreground with its
output being passed through to your terminal session (or wherever you may have
redirected it).

Events are consolidated and settled before they are dispatched to your build
tool so that it won't start executing until after the files have stopped
changing. The --settle argument controls the settle duration.

watchman-make requires pywatchman (and thus requires python) as well as
watchman.

Example Copy Section

$ watchman-make -p '**/*.c' '**/*.h' 'Makefile*' -t all -p 'tests/**/*.py' 'tests/**/*.c' -t integration
# Relative to /Users/wez/fb/watchman
# Changes to files matching **/*.c **/*.h Makefile* will execute `make all`
# Changes to files matching tests/**/*.py tests/**/*.c will execute `make integration`
# waiting for changes

Targets Copy Section

You can tell watchman-make about one or more build targets and their
dependencies, and it will then trigger the build for those targets as changes
are detected.

The example above defines two targets using the -t argument; all and
integration. These correspond to targets with the same names in the watchman
Makefile. Each target will pick up the list of patterns defined by the -p
argument that precedes it.

$ watchman-make -p '**/*.c' '**/*.h' -t all

The above defines a target named all that will be triggered whenever any
combination of files are changed that have filenames that match either of the
patterns *.c or *.h at any level in the directory tree (that's what the **
portion means). When it triggers, watchman-make will execute make all.

If you don't use make, you can use the --make option to tell watchman-make
to use your builder of choice. When a target is triggered, watchman-make will
concatenate the value of --make with the name of the target and execute that
command using the shell.

The target name has no special meaning to watchman-make, it is used only to
construct the command to invoke. There is no special logic or support that is
specific to Makefiles or make.

Multiple Targets Copy Section

There are two different ways to specify multiple targets. The first is shown in
the main example at the top of this page and is repeated here:

$ watchman-make -p '*.c' '*.h' 'Makefile*' -t all -p 'tests/**/*.py' 'tests/**/*.c' -t integration

This defines two independent targets, all and integration that each have a
list of patterns defined as their triggers. Each time you specify a target using
the -t option, the value of the -p option is cleared.

The above will cause make all to be run if you change a file that matches
.*c (at the top level of the tree), and will cause make integration to run
if you change a source file under the tests directory.

An alternative is to list multiple target names with your -t option:

$ watchman-make -p '*.c' '*.h' 'Makefile*' 'tests/**/*.py' 'tests/**/*.c' -t all integration

this will execute make all integration if you change any top level *.c file
or test source file.

Run Scripts Copy Section

Since 4.8.

As an alternative to targets, you can provide the path to a script which
watchman-make will execute when changes are detected.

$ watchman-make -p '**/*.c' '**/*.h' --run my_script.sh

The above will run the provided script whenever any combination of files are
changed that have filenames that match either of the patterns *.c or *.h at
any level in the directory tree. When it triggers, watchman-make will execute
my_script.sh.

You must run watchman-make with either --target or --run, but they cannot
be run together

Watchman Replicate Subscription (`website/docs/watchman-replicate-subscription.md`) Copy Section

title: watchman-replicate-subscription category: Invocation sidebar_position: 4 Copy Chapter

Since 5.0

watchman-replicate-subscription can replicate an existing watchman
subscription. It queries watchman for a list of subscriptions, identifies the
source subscription (the subscription to replicate) and subscribes to watchman
using the same query.

Integrators can use this client to validate the watchman notifications their
client is receiving to localize anomalous behavior.

The source subscription is identified using any combination of the 'name',
'pid', and 'client' arguments. The provided combination must uniquely identify a
subscription. Source subscription details for a watched root can be retrieved by
running the command 'watchman-replicate-subscription --list PATH'.

By default, the replicated subscription will take the source subscription name
and prepend the substring 'replicate: ' to it. The 'qname' option can be used to
specify the replicated subscription name.

The subscription can stop after a configurable number of events are observed.
The default is a single event. You may also remove the limit and allow it to
execute continuously.

watchman-replicate-subscription will print one event per line. The event
information is determined by the fields in the identified subscription, with
each field separated by a space (or your choice of --separator).

Subscription state-enter and state-leave PDUs will be interleaved with other
events. Known subscription PDUs (currently only those generated by the mercurial
fsmonitor extension) will be enclosed in square brackets. All others will be
output in JSON format.

Events are consolidated and settled by the watchman server before they are
dispatched to watchman-replicate-subscription.

watchman-replicate-subscription requires pywatchman (and thus requires python)
as well as watchman.

Source subscription Copy Section

$ watchman-replicate-subscription PATH -n NAME

The source subscription must be an existing subscription for the provided path.
Any combination of the 'name', 'pid', and 'client' arguments can be used
provided they uniquely identify a subscription. Source subscription details for
a watched root can be retrieved as follows:

$ watchman-replicate-subscriptions PATH --list

The subscription name, pid and client can then used to replicate the
subscription.

$ watchman-replicate-subscription PATH -n NAME -c CLIENT -p PID

Controlling lifetime Copy Section

There are two primary controls for how long watchman-replicate-subscription
will run:

• -t or --timeout places a time limit on execution
• -m or --max-events places a limit on the number of events to process

watchman-replicate-subscription will terminate when either the timeout is hit
or the max events limit is hit.

By default there is no time limit, but there is a default limit of a single
event.

You may specify --max-events 0 to disable the event limit.

Controlling output Copy Section

watchman-replicate-subscription will output one line per event. The following
options influence the output:

• --separator STRING - if you specified multiple fields, the separator string
  will be used when printing them. The default is --separator " " which will
  print the fields with spaces between them.

Exit Status Copy Section

The following exit status codes can be used to determine what caused
watchman-replicate-subscription to exit:

• 0 is returned after successfully waiting for event(s) or listing matching
  subscriptions
• 1 in case of a runtime error of some kind
• 2 the -t/--timeout option was used and that amount of time passed before
  an event was received
• 3 if execution was interrupted (Ctrl-C)

Watchman Wait (`website/docs/watchman-wait.md`) Copy Section

title: watchman-wait category: Invocation sidebar_position: 3 Copy Chapter

watchman-wait waits for changes to files. It uses the watchman service to
efficiently and recursively watch your specified list of paths.

It is suitable for waiting for changes to files from shell scripts. It has some
similarity to inotifywait except that it uses the watchman service to watch
files and thus can be used on any of the operating systems supported by
watchman, not just Linux.

It can stop after a configurable number of events are observed. The default is a
single event. You may also remove the limit and allow it to execute
continuously.

watchman-wait will print one event per line. The event information includes
your specified list of fields, with each field separated by a space (or your
choice of --separator).

Events are consolidated and settled by the watchman server before they are
dispatched to watchman-wait so that your script won't start executing until
after the files have stopped changing.

watchman-wait requires pywatchman (and thus requires python) as well as
watchman.

Paths and Patterns Copy Section

$ watchman-wait path [path ...]

The primary unit of watching is a path. You must specify a list of one or more
paths that you'd like to wait for. Paths can be files or directories. Each of
the paths in your list must exist at the time that you invoke watchman-wait or
an error will be reported and watchman-wait will exit.

If you'd like to wait for a file to be created you can watch the directory in
which it will be created. You may further refine your watch by limiting it to a
set of patterns.

$ watchman-wait . -p '*.so'

$ watchman-wait -p '*.so' -- .

Both of the above will wait for a shared object file to be changed in any path
under the current working directory. Since both the -p option and the list of
paths accept one or more parameters, the second form shows how to disambiguate
between the list of patterns and the list of paths using the -- separator.

Patterns are wildmatch style globs that support recursive matching via the **
placeholder.

You should always quote your pattern parameters so that they are not evaluated
by your shell.

Controlling lifetime Copy Section

There are two primary controls for how long watchman-wait will run:

• -t or --timeout places a time limit on execution
• -m or --max-events places a limit on the number of events to process

watchman-wait will terminate when either the timeout is hit or the max events
limit is hit.

By default there is no time limit, but there is a default limit of a single
event.

You may specify --max-events 0 to disable the event limit.

Controlling output Copy Section

watchman-wait will output one line per event. The following options influence
the output:

• --fields NAME,NAME - specifies the list of fields to be printed for each
  event. The default is --fields name which will print just the name of the
  file that was changed. You may use any of the available fields listed in
  available fields. The fields will be printed
  in the order you list them.
• --relative DIR - the name field will be adjusted to be relative to DIR
  before it is printed out. The default for DIR is the current working
  directory when watchman-wait is started.
• --separator STRING - if you specified multiple fields, the separator string
  will be used when printing them. The default is --separator " " which will
  print the fields with spaces between them.

Exit Status Copy Section

The following exit status codes can be used to determine what caused
watchman-wait to exit:

• 0 is returned after successfully waiting for event(s)
• 1 in case of a runtime error of some kind
• 2 the -t/--timeout option was used and that amount of time passed before
  an event was received
• 3 if execution was interrupted (Ctrl-C)

Cmd Clock (`website/docs/cmd/clock.md`) Copy Section

title: clock category: Commands Copy Chapter

Returns the current clock value for a watched root.

Since 3.9.

The capability name associated with this enhanced
functionality is clock-sync-timeout.

sync_timeout specifies the number of milliseconds that you want to wait to
observe a synchronization cookie. The synchronization cookie is created at the
start of your clock call and, once the cookie is observed, means that the clock
value returned by this command is at least as current as the time of your clock
call.

If no sync_timeout is specified, the returned clock value is the instantaneous
value of the clock associated with the watched root, and may be almost
immediately invalidated if there are any filesystem notifications that are yet
to be processed.

$ watchman clock /path/to/dir

JSON:

Note the third options argument is optional.

["clock", "/path/to/dir", {"sync_timeout": 100}]

Cmd Find (`website/docs/cmd/find.md`) Copy Section

title: find category: Commands Copy Chapter

Finds all files that match the optional list of patterns under the specified
dir. If no patterns were specified, all files are returned.

$ watchman find /path/to/dir [patterns]

Cmd Flush Subscriptions (`website/docs/cmd/flush-subscriptions.md`) Copy Section

title: flush-subscriptions category: Commands Copy Chapter

Since 4.8.

Flushes buffered updates to subscriptions associated with the current session,
guaranteeing that they are up-to-date as of the time Watchman received the
flush-subscriptions command.

Subscription updates will be interleaved between the flush-subscriptions
request and its response. Once the response has been received, subscriptions are
up-to-date.

This command is designed to be used by interactive programs that have a
background process or daemon maintaining a subscription to Watchman. The typical
pattern is for interactive commands to be forwarded to the process, which calls
flush-subscriptions and then processes any subscription updates it received.
This pattern eliminates races with files changed right before the interactive
command.

Arguments Copy Section

• sync_timeout: Required. The number of milliseconds to wait to observe a
  synchronization cookie. The synchronization cookie is created at the start of
  the flush-subscriptions call, and once the cookie is observed, means that
  the OS has sent watchman all the updates till at least the start of the
  flush-subscriptions call.
• subscriptions: Optional. Which subscriptions to flush. By default this
  flushes all subscriptions associated with this project on this session.

Examples Copy Section

Assuming subscriptions sub1, sub2 and sub3 have been established on this
session, if sub1 has updates pending, sub2 is up-to-date and sub3 is
currently dropping updates:

["flush-subscriptions", "/path/to/root", {"sync_timeout": 1000}]

In response, Watchman will first emit a unilateral subscription PDU for sub1,
then respond with

{
  "clock": "c:1446410081:18462:7:135",
  "synced": ["sub1"],
  "no_sync_needed": ["sub2"],
  "dropped": ["sub3"]
}

To flush updates for some but not all subscriptions associated with this
session:

["flush-subscriptions", "/path/to/root",
  {
    "sync_timeout": 1000,
    "subscriptions": ["sub1", "sub2"]
  }
]

Deferred and Dropped Updates Copy Section

Subscriptions will typically buffer individual updates until a settle period
has expired. flush-subscriptions will force those updates through immediately.

Subscriptions currently deferring updates because of defer or defer_vcs are
updated immediately, without waiting for the defer or defer_vcs to end.

Subscriptions currently dropping updates with a drop state will not get any
updates. Their names will be returned in the dropped field.

Notes Copy Section

• flush-subscriptions can only be used to flush subscriptions associated with
  the current session.
• A single session can be subscribed to updates from multiple projects at the
  same time. However, flush-subscriptions can only flush updates for one
  project at a time.

Cmd Get Config (`website/docs/cmd/get-config.md`) Copy Section

title: get-config category: Commands Copy Chapter

The get-config command returns the .watchmanconfig for the root. If there is
no .watchmanconfig, it returns an empty configuration field:

$ watchman get-config .
{
    "version": "2.9.9",
    "config": {}
}

$ watchman get-config /path/to/root
{
    "version": "2.9.9",
    "config": {
        "ignore_dirs": [
            "buck-out"
        ]
    }
}

Note that watchman only reads the .watchmanconfig file when the watch is
established. If changes are made after that point, the get-config response
will not reflect them.

See Configuration Options for details on
valid contents of the config field. Note that the values returned by
get-config are passed straight through from the .watchmanconfig file, and
thus may contain fields that are not strictly legal.

This command is available since watchman version 2.9.9.

Cmd Get Sockname (`website/docs/cmd/get-sockname.md`) Copy Section

title: get-sockname category: Commands Copy Chapter

If you're integrating against watchman using the unix socket and either the JSON
or BSER protocol, you may need to discover the correct socket path. Rather than
hard-coding the path or replicating the logic discussed in
Command Line, you can simply execute the CLI to determine the
path. This has the side effect of spawning the service for your user if it was
not already running--bonus!

$ watchman get-sockname
{
  "version": "2.5",
  "sockname": "/tmp/.watchman.wez"
}

Cmd List Capabilities (`website/docs/cmd/list-capabilities.md`) Copy Section

title: list-capabilities category: Commands Copy Chapter

Since 3.8.

This command returns the full list of supported capabilities
offered by the watchman server. The intention is that client applications will
use the expanded version command to check compatibility rather
than interrogating the full list.

Here's some example output. The actual capabilities list is in unspecified order
and is much longer than is reproduced here:

$ watchman list-capabilities
{
    "version": "3.8.0",
    "capabilities": [
        "field-mode",
        "term-allof",
        "cmd-trigger"
    ]
}

Cmd Log Level (`website/docs/cmd/log-level.md`) Copy Section

title: log-level category: Commands Copy Chapter

Changes the log level of your connection to the watchman service.

From the command line:

$ watchman --server-encoding=json --persistent log-level debug

JSON:

["log-level", "debug"]

This command changes the log level of your client session. Whenever watchman
writes to its log, it walks the list of client sessions and also sends a log
packet to any that have their log level set to match the current log event.

Valid log levels are:

• debug - receive all log events
• error - receive only important log events
• off - receive no log events

Note that you cannot tap into the output of triggered processes using this
mechanism.

Log events are sent unilaterally by the server as they happen, and have the
following structure:

{
  "version": "1.0",
  "log": "log this please"
}

Cmd Log (`website/docs/cmd/log.md`) Copy Section

title: log category: Commands Copy Chapter

Generates a log line in the watchman log.

$ watchman log debug "log this please"

Cmd Query (`website/docs/cmd/query.md`) Copy Section

title: query category: Commands Copy Chapter

Since 1.6.

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "suffix": "php",
  "expression": ["allof",
    ["type", "f"],
    ["not", "empty"],
    ["ipcre", "test", "basename"]
  ],
  "fields": ["name"]
}]
EOT

Executes a query against the specified root. This example uses the -j flag to
the watchman binary that tells it to read stdin and interpret it as the JSON
request object to send to the watchman service. This flag allows you to send in
a pretty JSON object (as shown above), but if you're using the socket interface
you must still format the object as a single line JSON request as documented in
the protocol spec.

The first argument to query is the path to the watched root. The second argument
holds a JSON object describing the query to be run. The query object is
processed by passing it to the query engine (see File Queries)
which will generate a set of matching files.

The query command will then consult the fields member of the query object; if
it is not present it will default to:

"fields": ["name", "exists", "new", "size", "mode"]

For each file in the result set, the query command will generate a JSON object
value populated with the requested fields. For example, the default set of
fields will return a response something like this:

{
    "version": "2.9",
    "clock": "c:80616:59",
    "is_fresh_instance": false,
    "files": [
        {
            "exists": true,
            "mode": 33188,
            "new": false,
            "name": "argv.c",
            "size": 1340,
        }
    ]
}

If a field's value cannot be computed, a field's value may be null, or may be
an object with an error key containing a descriptive message string:

{
    "version": "2019-07-22T13:50:36Z",
    "is_fresh_instance": false,
    "clock": "c:1563834049:1830370:791543813:2257494",
    "files": [
        {
            "content.sha1hex": null,
            "name": "docs"
            "symlink_target": null,
            "type": "d",
        },
        {
            "content.sha1hex": {
                "error": "eloop: file is a symlink: Invalid argument: Invalid argument"
            },
            "type": "l",
            "symlink_target": "eloop",
            "name": "eloop"
        }
    ]
}

For queries using the since generator, the is_fresh_instance member is true
if the particular clock value indicates that it was returned by a different
instance of watchman, or a named cursor hasn't been seen before. In that case,
only files that currently exist will be returned, and all files will have new
set to true. For all other queries, is_fresh_instance will always be true.
Advanced users may set the input parameter empty_on_fresh_instance to true, in
which case no files will be returned for fresh instances.

If the fields member consists of a single entry, the files result will be a
simple array of values; ~~~"fields": ["name"]~~~ produces:

{
    "version": "1.5",
    "clock": "c:80616:59",
    "files": ["argv.c", "foo.c"]
}

Available fields Copy Section

• name - string: the filename, relative to the watched root
• exists - bool: true if the file exists, false if it has been deleted
• cclock - string: the "created clock"; the clock value when we first observed
  the file, or the clock value when it last switched from !exists to exists.
• oclock - string: the "observed clock"; the clock value where we last
  observed some change in this file or its metadata.
• ctime, ctime_ms, ctime_us, ctime_ns, ctime_f - last inode change
  time measured in integer seconds, milliseconds, microseconds, nanoseconds or
  floating point seconds respectively.
• mtime, mtime_ms, mtime_us, mtime_ns, mtime_f - modified time
  measured in integer seconds, milliseconds, microseconds, nanoseconds or
  floating point seconds respectively.
• size - integer: file size in bytes
• mode - integer: file (or directory) mode expressed as a decimal integer
• uid - integer: the owning uid
• gid - integer: the owning gid
• ino - integer: the inode number
• dev - integer: the device number
• nlink - integer: number of hard links
• new - bool: whether this entry is newer than the since generator criteria

Since 3.1.

• type - string: the file type. Has the the values listed in
  the type query expression

Since 4.6.

• symlink_target - string: the target of a symbolic link if the file is a
  symbolic link

Since 4.9.

• content.sha1hex - string: the SHA-1 digest of the file's byte content,
  encoded as 40 hexidecimal digits (e.g.
  "da39a3ee5e6b4b0d3255bfef95601890afd80709" for an empty file)

Synchronization timeout (since 2.1) Copy Section

By default a query will wait for up to 60 seconds for the view of the
filesystem to become current. Watchman decides that the view is current by
creating a cookie file and waiting to observe the notification that it is
present. If the cookie is not observed within the sync_timeout period then the
query invocation will error out with a synchronization error message.

If your synchronization requirements differ from the default, you may pass in
your desired timeout when you construct your query; it must be an integer value
expressed in milliseconds:

["query", "/path/to/root", {
  "expression": ["exists"],
  "fields": ["name"],
  "sync_timeout": 60000
}]

You may specify 0 as the value if you do not wish for the query to create a
cookie and synchronize; the query will be evaluated over the present view of the
tree, which may lag behind the present state of the filesystem.

Lock timeout Copy Section

Since 4.6.

By default queries will wait for up to 60 seconds to acquire a lock to inspect
the view of the filesystem tree. In practice, this timeout should never be hit
(it is indicative of an environmental or load related issue). However, in some
situations it is important to ensure that the query attempt times out sooner
than this. You may use the lock_timeout field to control this behavior.
lock_timeout must be an integer value expressed in milliseconds:

["query", "/path/to/root", {
  "expression": ["exists"],
  "fields": ["name"],
  "lock_timeout": 60000,
  "sync_timeout": 60000
}]

Prior to version 4.6, the lock_timeout could not be configured and had an
effective value of infinity.

Case sensitivity Copy Section

Since 2.9.9.

On systems where the watched root is a case insensitive filesystem (this is the
common case for macOS and Windows), various name matching operations default to
case insensitive.

Since 4.7.

You may override the case sensitivity of the various name matching operations by
setting the case_sensitive field in your query spec. It default to the case
sensitivity of the watched root. This is useful in cases where you know that the
contents of the tree are treated case sensitively by your various tools but are
running on a case insensitive filesystem. By forcing the name matches to case
sensitive mode the matches are faster and in some cases can be accelerated by
using alternative algorithms.

$ watchman -j <<-EOT
["query", "/path/to/root", {
  "suffix": "php",
  "expression": ["match", "foo*.c", "basename"],
  "case_sensitive": true,
  "fields": ["name"]
}]
EOT

Directory Events Copy Section

Generally, Watchman should report changes to directories for all queries.

On EdenFS repositories there is an exception. When the user has changed commits
in the duration of a time query, directory changes may not be reported across
those commit changes.

the option always_include_directories exists to include events for directories
across commit transitions. This is only supported for mercurial. This can be
expensive, so clients who do not need this are recommended not to use this. This
value defaults to false.

Cmd Shutdown Server (`website/docs/cmd/shutdown-server.md`) Copy Section

title: shutdown-server category: Commands Copy Chapter

This command causes your watchman service to exit with a normal status code.

$ watchman shutdown-server

Cmd Since (`website/docs/cmd/since.md`) Copy Section

title: since category: Commands Copy Chapter

$ watchman since /path/to/dir <clockspec> [patterns]

Finds all files that were modified since the specified clockspec that match the
optional list of patterns. If no patterns are specified, all modified files are
returned.

The response includes a files array, each element of which is an object with
fields containing information about the file:

{
    "version": "2.7",
    "is_fresh_instance": true,
    "clock": "c:80616:59",
    "files": [
        {
            "cclock": "c:80616:1",
            "ctime": 1357617635,
            "dev": 16777220,
            "exists": true,
            "gid": 100,
            "ino": 20161390,
            "mode": 33188,
            "mtime": 1357617635,
            "name": "argv.c",
            "nlink": 1,
            "oclock": "c:80616:39",
            "size": 1340,
            "uid": 100
        }
    ]
}

The fields should be largely self-explanatory; they correspond to fields from
the underlying struct stat, but a couple need special mention:

• cclock - The "created" clock; the clock value representing the time that
  this file was first observed, or the clock value where this file changed from
  deleted to non-deleted state.
• oclock - The "observed" clock; the clock value representing the time that
  this file was last observed to have changed.
• exists - whether we believe that the file exists on disk or not. If this
  is false, most of the other fields will be omitted.
• new - this is only set in cases where the file results were generated as
  part of a time or clock based query, such as the since command. If the
  cclock value for the file is newer than the time you specified then the file
  entry is marked as new. This allows you to more easily determine if the file
  was newly created without having to maintain a lot of state.

Cmd State Enter (`website/docs/cmd/state-enter.md`) Copy Section

title: state-enter category: Commands Copy Chapter

Since 4.4

The state-enter command works in conjunction with
state-leave to facilitate
advanced settling in subscriptions.

state-enter causes a watch to be marked as being in a particular named state.
The state is asserted until a corresponding state-leave command is issued or
until the watchman client session that entered the state disconnects. This
automatic cleanup helps to avoid breaking subscribers if the tooling that
initiated a state terminates unexpectedly.

Subscriptions can use the defer and
drop fields to defer or drop notifications generated while
the watch is in a particular named state.

Examples Copy Section

This is the simplest example; entering a state named mystate:

["state-enter", "/path/to/root", "mystate"]

It will cause any subscribers to receive a unilateral subscription PDU from the
watchman server:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-enter":   "mystate",
  "clock":         "c:1446410081:18462:7:127"
}

The clock field in the response is the (synchronized; see below) clock at the
time that the state was entered and can be used in subsequent queries, in
combination with the corresponding state-leave subscription PDU clock, to
locate things that changed while the state was asserted.

A more complex example demonstrates passing metadata to any subscribers. The
metadata field is propagated through to the subscribers and is not interpreted
by the watchman server. It can be any JSON value.

["state-enter", "/path/to/root", {
  "name": "mystate",
  "metadata": {
    "foo": "bar"
  }
}]

This will emit the following unilateral subscription PDU to all subscribers:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-enter":   "mystate",
  "clock":         "c:1446410081:18462:7:137",
  "metadata": {
    "foo": "bar"
  }
}

Synchronization Copy Section

States are synchronized with the state of the filesystem so that it is possible
for subscribers to reason about when files changed with respect to the state.

This means that issuing a state-enter command will
perform query synchronization to ensure that
things are in sync.

The state-enter command will use a default sync_timeout of 60 seconds. If
the synchronization cookie is not observed within the configured sync_timeout,
an error will be returned and the state will not be entered.

In some cases, perhaps during the initial crawl of a very large tree, You may
specify an alternative value for the timeout; the value is expressed in
milliseconds:

["state-enter", "/path/to/root", {
  "name": "mystate",
  "sync_timeout": 10000,
  "metadata": {
    "foo": "bar"
  }
}]

You may also specify 0 for the timeout to disable synchronization for this
particular command. This may cause the state to appear to clients to have been
entered logically before it actually did in the case that there are buffered
notifications that have not yet been processed by watchman at the time that the
state was entered.

Cmd State Leave (`website/docs/cmd/state-leave.md`) Copy Section

title: state-leave category: Commands Copy Chapter

Since 4.4

The state-leave command works in conjunction with
state-enter to facilitate
advanced settling in subscriptions.

state-leave causes a watch to no longer be marked as being in a particular
named state.

Examples Copy Section

This is the simplest example, vacating a state named mystate:

["state-leave", "/path/to/root", "mystate"]

It will cause any subscribers to receive a unilateral subscription PDU from the
watchman server:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-leave":   "mystate",
  "clock":         "c:1446410081:18462:7:135"
}

The clock field in the response is the (synchronized; see below) clock at the
time that the state was entered and can be used in subsequent queries, in
combination with the corresponding state-enter subscription PDU clock, to
locate things that changed while the state was asserted.

A more complex example demonstrates passing metadata to any subscribers. The
metadata field is propagated through to the subscribers and is not interpreted
by the watchman server. It can be any JSON value.

["state-leave", "/path/to/root", {
  "name": "mystate",
  "metadata": {
    "foo": "bar"
  }
}]

This will emit the following unilateral subscription PDU to all subscribers:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-leave":   "mystate",
  "clock":         "c:1446410081:18462:7:137",
  "metadata": {
    "foo": "bar"
  }
}

Abandoned State Copy Section

A state is implicitly vacated when the watchman client session that asserted it
disconnects. This helps to avoid breaking subscribers (since the typical action
is to defer or drop notifications) if the tooling that initiated the state
terminates unexpectedly.

An abandoned state is reported to any subscribers via a unilateral subscription
PDU with the abandoned field set to true:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-leave":   "mystate",
  "clock":         "c:1446410081:18462:7:137",
  "abandoned":     true
}

This allows the subscriber to take an appropriate action.

Synchronization Copy Section

States are synchronized with the state of the filesystem so that it is possible
for subscribers to reason about when files changed with respect to the state.

This means that issuing a state-leave command will
perform query synchronization to ensure that
things are in sync.

The state-leave command will use a default sync_timeout of 60 seconds. If
the synchronization cookie is not observed within the configured sync_timeout,
an error will be returned and the state will not be entered.

In some cases, perhaps during the initial crawl of a very large tree, You may
specify an alternative value for the timeout; the value is expressed in
milliseconds:

["state-leave", "/path/to/root", {
  "name": "mystate",
  "sync_timeout": 10000,
  "metadata": {
    "foo": "bar"
  }
}]

You may also specify 0 for the timeout to disable synchronization for this
particular command. This may cause the state to appear to clients to have been
vacated logically before it actually did in the case that there are buffered
notifications that have not yet been processed by watchman at the time that the
state was vacated.

Cmd Subscribe (`website/docs/cmd/subscribe.md`) Copy Section

title: subscribe category: Commands Copy Chapter

Since 1.6

Subscribes to changes against a specified root and requests that they be sent to
the client via its connection. The updates will continue to be sent while the
connection is open. If the connection is closed, the subscription is implicitly
removed.

This makes the most sense in an application connecting via the socket interface,
but you may also subscribe via the command line tool if you're interested in
observing the changes for yourself:

$ watchman -j --server-encoding=json -p <<-EOT
["subscribe", "/path/to/root", "mysubscriptionname", {
  "expression": ["allof",
    ["type", "f"],
    ["not", "empty"],
    ["suffix", "php"]
  ],
  "fields": ["name"]
}]
EOT

The example above registers a subscription against the specified root with the
name mysubscriptionname.

The response to a subscribe command looks like this:

{
  "version":   "1.6",
  "subscribe": "mysubscriptionname"
}

When the subscription is first established, the expression term is evaluated and
if any files match, a subscription notification packet is generated and sent,
unilaterally to the client.

Then, each time a change is observed, and after the settle period has passed,
the expression is evaluated again. If any files are matched, the server will
unilaterally send the query results to the client with a packet that looks like
this:

{
  "version": "1.6",
  "clock": "c:1234:123",
  "files": ["one.php"],
  "root":  "/path/being/watched",
  "subscription": "mysubscriptionname"
}

The subscribe command object allows the client to specify a since parameter; if
present in the command, the initial set of subscription results will only
include files that changed since the specified clockspec, equivalent to using
the query command with the since generator.

["subscribe", "/path/to/root", "myname", {
  "since": "c:1234:123",
  "expression": ["not", "empty"],
  "fields": ["name"]
}]

The suggested mode of operation is for the client process to maintain its own
local copy of the last "clock" value and use that to establish the subscription
when it first connects.

Filesystem Settling Copy Chapter

Prior to watchman version 3.2, the settling behavior was to hold subscription
notifications until the kernel notification stream was complete.

Starting in watchman version 3.2, after the notification stream is complete, if
the root appears to be a version control directory, subscription notifications
will be held until an outstanding version control operation is complete (at the
time of writing, this is based on the presence of either .hg/wlock or
.git/index.lock). This behavior matches triggers and helps to avoid performing
transient work in response to files changing, for example, during a rebase
operation.

In some circumstances it is desirable for a client to observe the creation of
the control files at the start of a version control operation. You may specify
that you want this behavior by passing the defer_vcs flag to your subscription
command invocation:

$ watchman -j -p <<-EOT
["subscribe", "/path/to/root", "mysubscriptionname", {
  "expression": ["allof",
    ["type", "f"],
    ["not", "empty"],
    ["suffix", "php"]
  ],
  "defer_vcs": false,
  "fields": ["name"]
}]
EOT

Advanced Settling Copy Chapter

Since 4.4

In more complex integrations it is desirable to be able to have a watchman aware
application signal the beginning and end of some work that will generate a lot
of change notifications. For example, Mercurial or Git could communicate with
watchman before and after updating the working copy.

Some applications will want to know that the update is in progress and continue
to process notifications. Others may want to defer processing the notifications
until the update completes, and some may wish to drop any notifications produced
while the update was in progress.

Watchman subscriptions provide the mechanism for each of these use cases and
expose it via two new fields in the subscription object; defer and drop are
described below.

It can be difficult to mix defer and drop with multiple overlapping states
in the context of a given subscription stream as there is a single cursor to
track the subscription position.

If your application uses multiple overlapping states and wants to defer some
results and drop others, it is recommended that you use drop for all of the
states and then issues queries with since terms bounded by the clock fields
from the subscription state PDUs to ensure that it observes all of the results
of interest.

defer Copy Section

["subscribe", "/path/to/root", "mysubscriptionname", {
  "defer": ["mystatename"],
  "fields": ["name"]
}]

The defer field specifies a list of state names for which the subscriber
wishes to defer the notification stream. When a watchman client signals that a
state has been entered via the state-enter command, if the
state name matches any in the defer list then the subscription will emit a
unilateral subscription PDU like this:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-enter":   "mystatename",
  "clock":         "<clock>",
  "metadata":      <metadata from the state-enter command>
}

Watchman will then defer sending any subscription PDUs with files payloads
until the state is vacated either by a state-leave command
or by the client that entered the state disconnecting from the watchman service.

Once the state is vacated, watchman will emit a unilateral subscription PDU like
this:

{
  "subscription":  "mysubscriptionname",
  "root":          "/path/to/root",
  "state-leave":   "mystatename",
  "clock":         "<clock>",
  "metadata":      <metadata from the exit-state command>
}

The subscription stream will then be re-enabled and notifications received since
the corresponding state-enter will be delivered to clients.

drop Copy Section

["subscribe", "/path/to/root", "mysubscriptionname", {
  "drop": ["mystatename"],
  "fields": ["name"]
}]

The drop field specifies a list of state names for which the subscriber wishes
to discard the notification stream. It works very much like defer as described
above, but when a state is vacated, the pending notification stream is
fast-forwarded to the clock of the state-leave command, effectively
suppressing any notifications that were generated between the state-enter and
the state-leave commands.

Source Control Aware Subscriptions Copy Chapter

Since 4.9

Read more about these here

Cmd Trigger Del (`website/docs/cmd/trigger-del.md`) Copy Section

title: trigger-del category: Commands Copy Chapter

Deletes a named trigger from the list of registered triggers. This disables and
removes the trigger from both the in-memory and the saved state lists.

$ watchman trigger-del /root triggername

Cmd Trigger List (`website/docs/cmd/trigger-list.md`) Copy Section

title: trigger-list category: Commands Copy Chapter

Returns the set of registered triggers associated with a root directory.

$ watchman trigger-list /root

Note that the format of the output from trigger-list changed in Watchman
version 2.9.7. It will now output a list of trigger objects as defined by the
trigger command.

Cmd Trigger (`website/docs/cmd/trigger.md`) Copy Section

title: trigger category: Commands Copy Chapter

The trigger command will create or replace a trigger.

A trigger is a saved incremental query over a watched root. When files change
that match the query expression, Watchman will spawn a process and pass
information about the changed files to it.

Triggered processes are spawned by the Watchman server process that runs in the
background; they do not have access to your terminal and their output is
redirected (by default) to the Watchman log file.

Watchman waits for the filesystem to settle before processing any triggers,
batching the list of changed files together before invoking the registered
command. You can adjust the settle period via the .watchmanconfig file.

Note that deleted files are counted as changed files and are passed the command
in exactly the same way as changed-but-existing files.

Watchman will only run a single instance of the trigger process at a time. That
avoids fork-bomb style behavior in cases where your trigger also modifies files.
When the process terminates, watchman will re-evaluate the trigger criteria
based on the clock at the time the process was last spawned; if a file list is
generated watchman will spawn a new child with the files that changed in the
meantime.

Unless no-save-state is in use, triggers are saved and re-established across a
Watchman process restart. If you had triggeres saved prior to upgrading to
Watchman 2.9.7, those triggers will be forgotten as you upgrade past version
2.9.7; you will need to re-register them.

There are two syntaxes for registering triggers; a simple syntax that allows
very simple trigger configuration with some reasonable defaults, and a second
extended syntax which is available since Watchman version 2.9.7.

The simple syntax is implemented in terms of the extended syntax and is
preserved for backwards compatibility with older clients.

Extended syntax Copy Section

Since 2.9.7.

You may use the extended JSON trigger definition syntax detailed below. It
provides more control over how the triggered commands are invoked than was
possible in earlier versions.

JSON:

["trigger", "/path/to/dir", <triggerobj>]

Where triggerobj is a trigger configuration object with the fields defined
below.

Here's an example trigger specified via the CLI that will cause make to be run
whenever assets or sources are changed:

$ watchman -j <<-EOT
["trigger", "/path/to/root", {
  "name": "assets",
  "expression": ["pcre", "\.(js|css|c|cpp)$"],
  "command": ["make"]
}]
EOT

The possible trigger object properties are:

• name defines the name of the trigger. You may use this name to remove the
  trigger later. Registering a different trigger with the same name as an
  existing trigger will implicitly delete the old trigger and then register the
  new one, causing the trigger expression to be evaluated for the whole tree.

• command specifies the command to invoke. It must be an array of string
  values; this will form the argv array of the trigger process. When the trigger
  is spawned, the $PATH of the Watchman process will be used to locate the
  command. If you have changed your $PATH since the Watchman process was
  started, Watchman won't be able to see your new $PATH. If you are
  registering trigger that runs something from an unusual or non-default
  location, it is recommended that you specify the full path to that command. If
  you are registering a trigger script that can be found in the watched root,
  just specify the path relative to the root.

• append_files is an optional boolean parameter; if enabled, the command
  array will have the set of matching file names appended when the trigger is
  invoked. System limits such as sysconf(_SC_ARG_MAX) and/or RLIMIT_STACK
  set an upper bound on the size of the parameters and environment that are
  passed to a spawned process. Watchman will try to ensure that the command is
  runnable by keeping the number of file name arguments below the system limits.
  If the full set cannot be passed to the process, Watchman will pass as many as
  it thinks will fit and omit the rest. When this argument list truncation
  occurs, Watchman will export WATCHMAN_FILES_OVERFLOW=true into the
  environment so that the child process can determine that this has happened.
  Watchman cannot break the arguments apart and run multiple processes for each
  argument batch; for that functionality, use xargs(1) for the command and
  set the stdin property to NAME_PER_LINE.

• expression accepts a query expression. The expression is applied to the list
  of changed files to generate the set of files that are relevant to this
  trigger. If no files match, the command will not be invoked. Omitting the
  expression will match all changed files.

• stdin specifies how stdin should be configured for the command invocation.
  You may set the value of this property to one of the following:

  • the string value /dev/null - sets stdin to read from /dev/null. This is
    the default and will be used if you omit the stdin property.

  • an array value will be interpreted as a list of field names. When the
    command is invoked, Watchman will generate an array of JSON objects that
    contain those field names on stdin. For example, if stdin is set to
    ["name", "size"], stdin will be a JSON array containing the list of
    changed files, represented as objects with the name and size properties:
    [{"name": "filename.txt", "size": 123}]. The list of valid fields is the
    same as the same as that documented in the query command. Just as with the
    query command, if the field list is comprised of a single field then the
    JSON will be an array of those field values. For instance, if you set
    stdin to ["name"] the JSON will be of the form ["filename.txt"]
    instead of [{"name": "filename.txt"}].

  • the string value NAME_PER_LINE will cause Watchman to generate a list of
    file names on stdin, one name per line. No quoting will be applied to the
    names, and they may contain spaces.

• stdout and stderr control the output and error streams. If omitted, the
  corresponding stream will be inherited from the Watchman process, which
  typically means that the command output/error stream will show up in the
  Watchman log file. If specified, the value must be a string:

  • >path/to/file - causes output to redirected to the specified file. The
    path is relative to the watched root, and will be truncated prior to being
    written to, if it exists, or created if it does not exist.

  • >>path/to/file - causes output to redirected to the specified file. The
    path is relative to the watched root. If the file already exists then it
    will be appended to. The file will be created if it does not exist.

• max_files_stdin specifies a limit on the number of files reported on stdin
  when stdin is set to hold the set of matched files. If the number of files
  that matched exceeds this limit, the input will be truncated to match this
  limit and WATCHMAN_FILES_OVERFLOW=true will also be exported into the
  environment. The default, if omitted, is no limit.

• chdir can be used to specify the working directory that should be set prior
  to spawning the process. The default is to set the working directory to the
  watched root. The value of this property is a string that will be interpreted
  relative to the watched root. Note that changing the working dir does not
  cause the file names from the query result to be re-written: they will
  always be relative to the watched root. The path to the root can be found in
  the $WATCHMAN_ROOT environmental variable.

Simple syntax Copy Section

The simple syntax is easier to execute from the CLI than the JSON based extended
syntax, but doesn't allow all of the trigger options to be set. In only supports
the Simple Pattern Syntax for queries.

From the command line:

$ watchman -- trigger /path/to/dir triggername [patterns] -- [cmd]

Note that the first -- is to distinguish watchman CLI switches from the second
--, which delimits patterns from the trigger command. This is only needed when
using the CLI, not when using the JSON protocol.

JSON:

["trigger", "/path/to/dir", "triggername", <patterns>, "--", <cmd>]

For example:

$ watchman -- trigger ~/www jsfiles '*.js' -- ls -l

Note the single quotes around the *.js; if you omit them, your shell will
expand it to a list of file names and register those in the trigger. While this
would work, any *.js files that you add after registering the trigger will not
cause the trigger to run.

or in JSON:

["trigger", "/home/wez/www", "jsfiles", "*.js", "--", "ls", "-l"]

The simple syntax is interpreted as a trigger object with the following
settings:

• name is set to the triggername
• command is set to the <cmd> list
• expression is generated from the `` list using the rules laid out
  in Simple Pattern Syntax
• append_files is set to true
• stdin is set to ["name", "exists", "new", "size", "mode"]
• stdout and stderr will be set to output to the Watchman log file
• max_files_stdin will be left unset

For this simple example, if ~/www/scripts/foo.js is changed, watchman will
chdir to ~/www then invoke ls -l scripts/foo.js. Note that the output will
show up in the Watchman log file, not in your terminal.

Environment for trigger commands Copy Section

Since Watchman version 2.9.7, the following environment variables are set for
all trigger commands, even those registered using the simple trigger syntax:

• WATCHMAN_FILES_OVERFLOW is set to true if the number of files exceeds
  either the max_files_stdin limit or the system argument size limit.
• WATCHMAN_CLOCK is set to the current clock at the time of the trigger
  invocation
• WATCHMAN_SINCE is set to the clock value of the prior trigger invocation, or
  unset if this is the first trigger invocation.
• WATCHMAN_ROOT is set to the path to the watched root
• WATCHMAN_TRIGGER is set to the name of the trigger
• WATCHMAN_SOCK is set to the path to the Watchman socket, so that you can
  figure out how to connect back to Watchman.

Relative roots Copy Section

Since 3.4.

Watchman supports optionally evaluating triggers with respect to a path within a
watched root. This is used with the relative_root parameter:

["trigger", "/path/to/watched/root", {
  "name": "relative-assets",
  "expression": ["pcre", "\.(js|css|c|cpp)$"],
  "command": ["make"],
  "relative_root": "project1"
}]

Setting a relative root results in the following modifications to triggers:

• Queries are evaluated with respect to the relative root. See
  File Queries for more.
• The current directory for triggered processes is set to the relative root,
  unless it is changed with chdir. If chdir is a relative path then it will
  be evaluated with respect to the relative root. So, for the example trigger
  above, if chdir is "subdir2", the current directory for triggered make
  invocations is /path/to/watched/root/project1/subdir2.
• In the environment, WATCHMAN_ROOT is still set to the actual root.
• WATCHMAN_RELATIVE_ROOT is set to the full path of the relative root.

Relative roots behave similarly to a separate Watchman watch on the
subdirectory, without any of the system overhead that that imposes. This is
useful for large repositories, where your script or tool is only interested in a
particular directory inside the repository.

Cmd Unsubscribe (`website/docs/cmd/unsubscribe.md`) Copy Section

title: unsubscribe category: Commands Copy Chapter

Available starting in version 1.6

Cancels a named subscription against the specified root. The server side will no
longer generate subscription packets for the specified subscription.

["unsubscribe", "/path/to/root", "mysubscriptionname"]

Cmd Version (`website/docs/cmd/version.md`) Copy Section

title: version category: Commands Copy Chapter

The version command will tell you the version and build information for the
currently running watchman service:

$ watchman version
{
    "version": "2.9.6",
    "buildinfo": "git:2727d9a1e47a4a2229c65cbb2f0c7656cbd96270"
}

To get the version of the client:

$ watchman -v
2.9.8

If the server and client versions don't match up, you should probably restart
your server: watchman shutdown-server ; watchman.

Capabilities Copy Section

Since 3.8.

The version command can be used to check for named capabilities. Capabilities
make it easier to check whether the server implements functionality based on the
name of that function rather than by having the client build up knowledge about
when those functions were introduced.

You can read more about the available capability names.

To check whether the relative_root capability is supported:

$ watchman -j <<< '["version", {"optional":["relative_root"]}]'
{
    "version": "3.8.0",
    "capabilities": {
        "relative_root": true
    }
}

If the capability is not supported:

$ watchman -j <<< '["version", {"optional":["will-never-exist"]}]'
{
    "version": "3.8.0",
    "capabilities": {
        "will-never-exist": false
    }
}

To have the server generate an error response if a capability is not supported:

$ watchman -j <<< '["version", {"required":["will-never-exist"]}]'
{
    "version": "3.8.0",
    "capabilities": {
        "will-never-exist": false
    },
    "error": "client required capability `will-never-exist` is not supported by this server"
}

To require one feature and test whether some optional features are supported:

$ watchman -j <<< '["version", {"required":["term-match"],"optional":["a","b"]}]'
{
    "version": "3.8.0",
    "capabilities": {
        "a": false,
        "b": false,
        "term-match": true
    }
}

capabilityCheck Copy Section

The node and python clients provide a capabilityCheck method that will
perform the version check above, and that also provide limited support for
testing capability support against older versions of the watchman server. This
facilitates a smoother transition from version number based checks to capability
named based checks.

In python:

import pywatchman
client = pywatchman.client()
# will throw an error if any of the required names are not supported
res = client.capabilityCheck(optional=['a'], required=['term-match'])
print res
# {'version': '3.8.0', 'capabilities': {'term-match': True, 'a': False}}

In node:

var watchman = require('fb-watchman');
var client = new watchman.Client();
client.capabilityCheck({optional:['a'], required:['term-match']},
    function (error, resp) {
        if (error) {
          // error will be an Error object if any of the required named
          // are not supported
        }
        console.log(resp);
        // {'version': '3.8.0', 'capabilities': {'term-match': false, 'a': false}}
        client.end();
    });

Cmd Watch Del All (`website/docs/cmd/watch-del-all.md`) Copy Section

title: watch-del-all category: Commands Copy Chapter

Available since version 3.1.1.

Removes all watches and associated triggers.

From the command line:

$ watchman watch-del-all

JSON:

["watch-del-all"]

Analogous to the watch-del this command will remove all watches and associated
triggers from the running process, and the state file ( unless watchman service
was started with
--no-save-state server option).

Cmd Watch Del (`website/docs/cmd/watch-del.md`) Copy Section

title: watch-del category: Commands Copy Chapter

Removes a watch and any associated triggers.

From the command line:

$ watchman watch-del /path/to/dir

JSON:

["watch-del", "/path/to/dir"]

The removed watch and any associated triggers will be removed from the state
file and will not be automatically watched if/when watchman is restarted.

However, if --no-save-state was used to start the watchman service, the watch
and triggers will be deleted from the running process but no changes will be
made to the state file. If this same directory is listed in the state file, the
watch will be re-established if/when the service is restarted.

Cmd Watch List (`website/docs/cmd/watch-list.md`) Copy Section

title: watch-list category: Commands Copy Chapter

Returns a list of watched dirs.

From the command line:

$ watchman watch-list

JSON:

["watch-list"]

Result:

{
    "version": "1.9",
    "roots": [
        "/home/wez/watchman"
    ]
}

Cmd Watch Project (`website/docs/cmd/watch-project.md`) Copy Section

title: watch-project category: Commands Copy Chapter

Since 3.1.

Requests that the project containing the requested dir is watched for changes.
Watchman will track all files and dirs rooted at the project path, and respond
with the relative path difference between the project path and the requested
dir.

Rationale Copy Section

With a proliferation of tools that wish to take advantage of filesystem watching
at different locations in a filesystem tree, it is possible and likely for those
tools to establish multiple overlapping watches.

Most systems have a finite limit on the number of directories that can be
watched effectively; when that limit is exceeded the performance and reliability
of filesystem watching is degraded, sometimes to the point that it ceases to
function.

It is therefore desirable to avoid this situation and consolidate the filesystem
watches. Watchman offers the watch-project command to allow clients to opt-in
to the watch consolidation behavior described below.

What's a project path? Copy Section

A project is the logical root of a set of related files in a filesystem tree and
is a good point at which to consolidate watches. Tools such as
hgwatchman will most likely have
already established a watch at the root of a project, so any other tools that
wish to watch a sub-directory can do so for no additional cost if they re-use
that existing watch at a higher level in the filesystem tree.

The watch-project command uses a simple procedure to locate the project path
that corresponds to a given path. While simple it is rather verbose to describe
it precisely:

1. The search is begun with a list of file names; we'll refer to it as
   root_files. Any file in this list, if present in a directory, identifies
   that directory as being a valid project directory.
2. The search is begun with the candidate directory set to the argument passed
   to watch-project. The candidate directory is passed to the realpath(3)
   function and the result is set as the new value of the candidate directory.
3. The candidate directory is concatenated with each of the root_files, one by
   one, and the resultant path is tested for existence. If the path exists then
   the candidate directory is the path that will be used for watch and the
   search is halted successfully.
4. If none of the root_files can be found in the candidate directory then the
   parent of the candidate directory is used as a new candidate and the process
   is repeated at step 3 above.
5. If no viable candidates are found and the root of the filesystem is reached,
   then the search terminates unsuccessfully.

Watchman may perform the above search procedure twice. The logic is:

1. root_files will be set to list only .watchmanconfig
2. Perform the search procedure above
3. If the search terminates successfully, then the watch is established for the
   current value of the candidate directory.
4. If the search terminates unsuccessfully, root_files is set to the global
   configuration option root_files and the search
   procedure is re-run.
5. If the search terminates successfully, then the watch is established for the
   current value of the candidate directory.
6. If the global configuration option
   enforce_root_files is set to true then the
   watch attempt fails.
7. Otherwise, the watch is established for the original argument to the
   watch-project command

What this means in laymans terms is that the definitive location of the project
root is where the .watchmanconfig file is found. If it is not found then the
set of files defined by the root_files configuration is used to locate a
candidate.

If no viable candidate is found then watchman will watch the requested
directory, unless the enforce_root_files setting is set to true.

The default value for root_files will match most common version control root
directories. The default value for enforce_root_files is false.

Using watch-project Copy Section

Assuming that ~/www/.hg and ~/www/some/child/dir both exist, then the
command:

$ watchman watch-project ~/www/some/child/dir
{
  "version": "3.0.1",
  "watch": "/Users/wez/www",
  "relative_path": "some/child/dir"
}

establishes a watch on the ~/www directory because that is the directory that
contains .hg, which is one of the items listed in the default value for
root_files.

As a client using watch-project it is important to observe the relative_path
and/or watch elements of the response; they identify which directory is
actually being watched. Any triggers, subscriptions or queries that the client
issues must be relative to the watched root to operate as expected. A client
can use relative_path to more easily construct queries or adjust the results
of queries by either concatenating the string when composing paths in a query
expression or removing the string from the prefix when processing the results.

If relative_path is missing from the response it means that the requested dir
is the same as the watched dir and that the watch-project invocation turned
out to be exactly equivalent to a watch invocation for the requested
directory.

Note that, when you're using the CLI, you can specify the root as
~/www/some/child/dir because the shell will resolve ~/www/some/child/dir to
/Users/wez/www/some/child/dir, but when you use the JSON protocol, you are
responsible for supplying an absolute path.

JSON:

["watch", "/Users/wez/www/some/child/dir"]

Initiating a watch Copy Section

Once a viable candidate is found, if watchman is not already watching the
directory, then watchman will:

• Establish change notification for the directory with the kernel
• Queues up a request to crawl the directory
• As the directory contents are resolved, those are watched in a similar fashion
• All newly observed files are considered changed

Persistence Copy Section

Unless the --no-save-state server option was used to start the watchman
service, watches and their associated triggers are saved and re-established
across a process restart.

Since 3.7.

The watchman service may decide to reap watches that have been idle for an
extended period of time. A watch is considered to be idle if no watchman queries
have been issued against the watch. If a watch is idle, and has no triggers
registered or active subscriptions then it is a candidate for reaping.

The idle_reap_age_seconds configuration
parameter controls the idle timeout for a watch. The default is 5 days. A reaped
watch is cancelled and removed from the state file.

Cmd Watch (`website/docs/cmd/watch.md`) Copy Section

title: watch category: Commands Copy Chapter

Deprecated starting in version 3.1. We recommend that clients adopt the
watch-project command.

Requests that the specified dir is watched for changes. Watchman will track all
files and dirs rooted at the specified path.

From the command line:

$ watchman watch ~/www

Note that, when you're using the CLI, you can specify the root as ~/www
because the shell will resolve ~/www to /home/wez/www, but when you use the
JSON protocol, you are responsible for supplying an absolute path.

JSON:

["watch", "/home/wez/www"]

Watchman will realpath(3) the directory and start watching it if it isn't
already. A newly watched directory is processed in a couple of stages:

• Establishes change notification for the directory with the kernel
• Queues up a request to crawl the directory
• As the directory contents are resolved, those are watched in a similar fashion
• All newly observed files are considered changed

Unless the --no-save-state server option was used to start the watchman
service, watches and their associated triggers are saved and re-established
across a process restart.

Case-Insensitivity Copy Chapter

Watchman has the following level of support for case-insensitive filesystems,
starting in version 2.9.9 on macOS only:

• each watched root is queried to determine if it is case-insensitive. This is
  the common default for most Mac systems running HFS+.
• When in case-insensitive mode, Watchman will attempt to resolve the true
  canonical name of a file on the filesystem when it observes changes.
• If the case of a filename changes, Watchman will report a delete of the old
  name and a change for the new name.
• Query expressions that match names will default to case-insensitive when the
  root is on a case-insensitive filesystem.
• Watchman's case folding is ASCII case-folding. Note that the match and
  pcre query expression terms request case folding support from the containing
  library, and that their case folding behavior is not controlled by Watchman
  beyond being enabled when the root is case-insensitive.
• The path generator is always case sensitive.

Expr Allof (`website/docs/expr/allof.md`) Copy Section

title: allof category: Expression Terms Copy Chapter

The allof expression term evaluates as true if all of the grouped expressions
also evaluated as true. For example, this expression matches only files whose
name ends with .txt and that are not empty files:

["allof", ["match", "*.txt"], ["not", "empty"]]

Each array element after the term name is evaluated as an expression of its own:

["allof", expr1, expr2, ... exprN]

Evaluation of the subexpressions stops at the first one that returns false.

Expr Anyof (`website/docs/expr/anyof.md`) Copy Section

title: anyof category: Expression Terms Copy Chapter

The anyof expression term evaluates as true if any of the grouped expressions
also evaluated as true. The following expression matches files whose name ends
with either .txt or .md:

["anyof", ["match", "*.txt"], ["match", "*.md"]]

Each array element after the term name is evaluated as an expression of its own:

["anyof", expr1, expr2, ... exprN]

Evaluation of the subexpressions stops at the first one that returns true.

Expr Dirname (`website/docs/expr/dirname.md`) Copy Section

title: dirname & idirname category: Expression Terms Copy Chapter

Since version 3.1

The dirname term allows matching on the parent directory structure for a given
file.

For the examples below, given a file with a wholename (the relative path from
the project root) of foo/bar/baz, the dirname portion is foo/bar.

The following two terms will match any file whose dirname is either exactly a
match for foo/bar or is any child directory of foo/bar. The first of these
two is a shortcut for the second:

["dirname", "foo/bar"]
  ["dirname", "foo/bar", ["depth", "ge", 0]]

The second of those terms uses a relational expression based on the depth of the
file within the specified dirname. A file is considered to have depth == 0 if
it is contained directly within the specified dirname. It has depth == 1 if it
is contained in a direct child directory of the specified dirname, depth == 2
if it is contained in a grand-child directory and so on.

The relational expression accepts the same relational operators as described in
the size term.

If you wanted to match only files that were directly in the foo/bar dir:

["dirname", "foo/bar", ["depth", "eq", 0]]

If you wanted to match only files that were in a grand-child or deeper:

["dirname", "foo/bar", ["depth", "ge", 2]]

idirname is the case insensitive version of dirname. If the watched root is
detected as a case insensitive fileystem, dirname is equivalent to idirname.

Expr Empty (`website/docs/expr/empty.md`) Copy Section

title: empty category: Expression Terms Copy Chapter

Evaluates as true if the file exists, has size 0 and is a regular file or
directory.

"empty"
["empty"]

Expr Exists (`website/docs/expr/exists.md`) Copy Section

title: exists category: Expression Terms Copy Chapter

Evaluates as true if the file exists

"exists"
["exists"]

Expr False (`website/docs/expr/false.md`) Copy Section

title: "false" section: Expression Terms Copy Chapter

The false expression always evaluates as false.

"false"
["false"]

Expr Match (`website/docs/expr/match.md`) Copy Section

title: match & imatch category: Expression Terms Copy Chapter

The match expression performs a glob-style match against the basename of the
file, evaluating true if the match is successful.

["match", "*.txt"]

You may optionally provide a third argument to change the scope of the match
from the basename to the wholename of the file.

["match", "*.txt", "basename"]
["match", "dir/*.txt", "wholename"]

Case sensitivity Copy Section

match is case sensitive; for case insensitive matching use imatch instead;
it behaves identically to match except that the match is performed ignoring
case.

Since 2.9.9.

On systems where the watched root is a case insensitive filesystem (this is the
common case for macOS and Windows), match is equivalent to imatch.

Since 4.7.

You can override the case sensitivity of all name matching operations used in
the query by setting the case_sensitive field in your query.

wildmatch Copy Chapter

Since 3.7.

The match expression has been enhanced as described below. The
capability name associated with this enhanced functionality
is wildmatch.

If you want to recursively match all files under a directory, use the ** glob
operator along with the wholename scope:

["match", "src/**/*.java", "wholename"]

By default, paths whose names start with . are not included. To change this
behavior, you may optionally provide a fourth argument containing a dictionary
of flags:

["match", "*.txt", "basename", {"includedotfiles": true}]

By default, backslashes in the pattern escape the next character, so \*
matches a literal * character. To change this behavior so backslashes are
treated literally, set the noescape flag to true in the flags dictionary.
(Note that \\ is a literal \ in JSON notation):

["match", "*\\*.txt", "filename", {"noescape": true}]

matches a\b.txt.

Expr Name (`website/docs/expr/name.md`) Copy Section

title: name & iname category: Expression Terms Copy Chapter

The name expression performs exact matches against file names. By default it
is scoped to the basename of the file:

["name", "Makefile"]

You may specify multiple names to match against by setting the second argument
to an array:

["name", ["foo.txt", "Makefile"]]

This second form can be accelerated and is preferred over an anyof
construction.

You may change the scope of the match via the optional third argument:

["name", "path/to/file.txt", "wholename"]
["name", ["path/to/one", "path/to/two"], "wholename"]

Finally, you may specify case insensitive evaluation by using iname instead of
name.

Since 2.9.9.

Starting in version 2.9.9, on macOS systems where the watched root is a case
insensitive filesystem (this is the common case for macOS), name is equivalent
to iname.

Since 4.7.

You can override the case sensitivity of all name matching operations used in
the query by setting the case_sensitive field in your query.

Expr Not (`website/docs/expr/not.md`) Copy Section

title: not category: Expression Terms Copy Chapter

The not expression inverts the result of the subexpression argument:

["not", "empty"]

Expr Pcre (`website/docs/expr/pcre.md`) Copy Section

title: pcre & ipcre category: Expression Terms Copy Chapter

To use this feature, you must configure watchman --with-pcre!

The pcre expression performs a Perl Compatible Regular Expression match
against the basename of the file. This pattern matches test_plan.php but not
mytest_plan:

["pcre", "^test_"]

You may optionally provide a third argument to change the scope of the match
from the basename to the wholename of the file.

["pcre", "txt", "basename"]
["pcre", "txt", "wholename"]

pcre is case sensitive; for case insensitive matching use ipcre instead; it
behaves identically to pcre except that the match is performed ignoring case.

Since 2.9.9.

Starting in version 2.9.9, on macOS systems where the watched root is a case
insensitive filesystem (this is the common case for macOS), pcre is equivalent
to ipcre.

Since 4.7.

You can override the case sensitivity of all name matching operations used in
the query by setting the case_sensitive field in your query.

Expr Since (`website/docs/expr/since.md`) Copy Section

title: since category: Expression Terms Copy Chapter

Evaluates as true if the specified time property of the file is greater than the
since value. Note that this is not the same as the since generator; when used
as an expression term we are performing a straight clockspec comparison. When
used as a generator, candidate files are selected based on the since time
index. The end result might or might not be the same--in particular, if the
since time index is not passed in, it will be treated the same as a fresh
instance, and only files that exist will be returned. The efficiency can vary
based on the size and shape of the file tree that you are watching; it may be
cheaper to generate the candidate set of files by suffix and then check the
modification time if many files were changed since your last query.

This will yield a true value if the observed change time is more recent than the
specified clockspec (this is equivalent to specifying "oclock" as the third
parameter):

["since", "c:12345:234"]

You may specify particular fields from the filesystem metadata. In this case
your clockspec should be a unix time value:

["since", 12345668, "mtime"]
 ["since", 12345668, "ctime"]

You may explicitly request the observed clock values too; in these cases we'll
accept either a timestamp or a clock value. The oclock is the last observed
change clock value (observed clock) and the cclock is the clock value where we
first observed the file come into existence (created clock):

["since", 12345668, "oclock"]
 ["since", "c:1234:123", "oclock"]
 ["since", 12345668, "cclock"]
 ["since", "c:1234:2342", "cclock"]

Expr Size (`website/docs/expr/size.md`) Copy Section

title: size category: Expression Terms Copy Chapter

Since version 3.1

The size term allows the size of an existing file (not deleted) to be evaluated
using simple relational operators as described in the table below.

The size term must always be an array with 3 elements:

["size", "gt", 0]

The second parameter describes the relational operator and the third parameter
is the integer operand to compare against. The example above evaluates to
true if the file exists and its size is greater than zero.

Possible relational operators are:

Expr Suffix (`website/docs/expr/suffix.md`) Copy Section

title: suffix category: Expression Terms Copy Chapter

The suffix expression evaluates true if the file suffix matches the second
argument. This matches files name foo.php and foo.PHP but not foophp:

["suffix", "php"]

Suffix expression matches are case insensitive.

suffix-set Copy Chapter

Since 5.0

You may specify multiple suffixes to match against by setting the second
argument to an array:

["suffix", ["php", "css", "html"]]

This second form can be accelerated and is preferred over an anyof
construction. In the following example the two terms are functionally equivalent
but the set form has a more efficient and thus faster runtime:

["anyof", ["suffix", "php"], ["suffix", "html"]]

["suffix", ["php", "html"]]

The capability name associated with this enhanced
functionality is suffix-set.

Expr True (`website/docs/expr/true.md`) Copy Section

title: "true" section: Expression Terms Copy Chapter

The true expression always evaluates as true.

"true"
["true"]

Expr Type (`website/docs/expr/type.md`) Copy Section

title: type category: Expression Terms Copy Chapter

Evaluates as true if the type of the file matches that specified by the second
argument; this matches regular files:

["type", "f"]

Possible types are:

• b: block special file
• c: character special file
• d: directory
• f: regular file
• p: named pipe (fifo)
• l: symbolic link
• s: socket
• D: Solaris Door
• ?: An unknown file type

--- METRICS ---

• Files Extracted: 70
• Estimated Token Budget: ~54368 tokens
• Canonical Reference: https://codewiki.google/github.com/facebook/watchman