# Technical Documentation: StackStorm/st2 > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/StackStorm/st2) ### Overview (`README.md`) [](https://www.stackstorm.com) **StackStorm** is a platform for integration and automation across services and tools, taking actions in response to events. Learn more at [www.stackstorm.com](http://www.stackstorm.com/product). [](https://github.com/StackStorm/st2/actions/workflows/ci.yaml) [](https://circleci.com/gh/StackStorm/st2) [](https://codecov.io/github/StackStorm/st2?branch=master) [](https://bestpractices.coreinfrastructure.org/projects/1833) [](LICENSE) [](https://stackstorm.com/community-signup) [](https://packagecloud.io/StackStorm/) [](https://sourcegraph.com/stackstorm) [](https://github.com/StackStorm/st2/discussions) [](https://twitter.com/StackStorm/) --- ## TL;DR * Install Get yourself a clean 64-bit Linux box that fits the [system requirements](https://docs.stackstorm.com/install/system_requirements.html). Run the installer script: ```bash curl -sSL https://stackstorm.com/packages/install.sh | bash -s -- --user=st2admin --password=Ch@ngeMe ``` * Read the docs: [https://docs.stackstorm.com/index.html](https://docs.stackstorm.com/install/index.html) * Questions? Check out [forum.stackstorm.com](https://forum.stackstorm.com/) * Or join our [Slack community](https://stackstorm.com/community-signup) ## StackStorm Overview [](https://www.youtube.com/watch?v=pzZws3ftDtA) ### About StackStorm is a platform for integration and automation across services and tools. It ties together your existing infrastructure and application environment so you can more easily automate that environment -- with a particular focus on taking actions in response to events. StackStorm helps automate common operational patterns. Some examples are: * **Facilitated Troubleshooting** - triggering on system failures captured by Nagios, Sensu, New Relic and other monitoring, running a series of diagnostic checks on physical nodes, OpenStack or Amazon instances, and application components, and posting results to a shared communication context, like Slack or JIRA. * **Automated remediation** - identifying and verifying hardware failure on OpenStack compute node, properly evacuating instances and emailing VM about potential downtime, but if anything goes wrong - freezing the workflow and calling PagerDuty to wake up a human. * **Continuous deployment** - build and test with Jenkins, provision a new AWS cluster, turn on some traffic with the load balancer, and roll-forth or roll-back based on NewRelic app performance data. StackStorm helps you compose these and other operational patterns as rules and workflows or actions; and these rules and workflows - the content within the StackStorm platform - are stored *as code* which means they support the same approach to collaboration that you use today for code development and can be shared with the broader open source community via [StackStorm Exchange](https://exchange.stackstorm.org). ### Who is using StackStorm? See the list of known StackStorm [ADOPTERS.md](/ADOPTERS.md) and [Thought Leaders](https://stackstorm.com/stackstorm-thought-leaders/). ### How it works #### StackStorm architecture StackStorm plugs into the environment via an extensible set of adapters: sensors and actions. * **Sensors** are Python plugins for inbound integration that watch for events from external systems and fire a StackStorm trigger when an event happens. * **Triggers** are StackStorm representations of external events. There are generic triggers (e.g., timers, webhooks) and integration triggers (e.g., Sensu alert, JIRA issue updated). A new trigger type can be defined by writing a sensor plugin. * **Actions** are StackStorm outbound integrations. There are generic actions (SSH, HTTP request), integrations (OpenStack, Docker, Puppet), or custom actions. Actions are either Python plugins, or any scripts, consumed into StackStorm by adding a few lines of metadata. Actions can be invoked directly by user via CLI, API, or the web UI, or used and called as part of automations - rules and workflows. * **Rules** map triggers to actions (or to workflows), applying matching criterias and map trigger payload data to action inputs. * **Workflows** stitch actions together into "uber-actions", defining the order, transition conditions, and passing context data from one action to the next. Most automations are multi-step (eg: more than one action). Workflows, just like "atomic" actions, are available in the action library, and can be invoked manually or triggered by rules. * **Packs** are the units of content deployment. They simplify the management and sharing of StackStorm pluggable content by grouping integrations (triggers and actions) and automations (rules and workflows). A growing number of packs is available on the StackStorm Exchange. Users can create their own packs, share them on GitHub, or submit them to the StackStorm Exchange organization. * **Audit trail** is the historical list of action executions, manual or automated, and is recorded and stored with full details of triggering context and execution results. It is also captured in audit logs for integrating with external logging and analytical tools: LogStash, Splunk, statsd, or syslog. StackStorm is a service with modular architecture. It is comprised of loosely coupled microservice components that communicate over a message bus, and scales horizontally to deliver automation at scale. StackStorm has a full REST API, CLI client, and web UI for admins and users to operate it locally or remotely, as well as Python client bindings for developer convenience. StackStorm is an established project and remains actively developed by a broad community. ## Documentation Additional documentation, including installation procedures, action/rule/workflow authoring, and how to setup and use triggers/sensors can be found at [https://docs.stackstorm.com](https://docs.stackstorm.com). ## Hacking / Contributing To set up a development environment and run StackStorm from sources, follow [these instructions](https://docs.stackstorm.com/development/sources.html). For information on how to contribute, our style guide, coding conventions and more, please visit the [Development section](https://docs.stackstorm.com/development/index.html) in our documentation. ## Security If you believe you found a security issue or a vulnerability, please send a description of it to our private mailing list at info [at] stackstorm [dot] com. Once you've submitted an issue, you should receive an acknowledgment from one our of team members in 48 hours or less. If further action is necessary, you may receive additional follow-up emails. For more information, please refer to https://docs.stackstorm.com/latest/security.html ## Copyright, License, and Contributor Agreement Copyright 2020 The StackStorm Authors. Copyright 2019 Extreme Networks, Inc. Copyright 2014-2018 StackStorm, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the [LICENSE](LICENSE) file, or at: [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0) By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the project, present and future, pursuant to the license of the project. --- ### Overview (`contrib/README.md`) Contrib ======= 1. St2 contents that will be shipped with the product. These contents will be copied over to /opt/stackstorm on install and then auto-registered. Users are expected to be able to edit these contents post install: * `st2/contrib/core` * `st2/contrib/packs` * `st2/contrib/linux` * `st2/contrib/chatops` 2. St2 samples that may be of interest to the users. These samples will be shipped with the product but will not be auto-registered. * `st2/contrib/examples` * `st2/contrib/tests` * `st2/contrib/hello_st2` See [`st2/st2common/packaging`](../st2common/packaging) to figure where and how it is all installed. --- ### Overview (`lint-configs/README.md`) # StackStorm Lint Configs This repository contains lint configs for different programming languages and tools (flake8, pylint, etc.) used by different StackStorm repositories. Configs are grouped in sub-directories by programming language. ## Usage To use those configs, add this repository as a git subtree to the repository where you want to utilize those configs. After that is done, update make targets (or similar) to correctly pass path to the configs to the tools in question. ```bash git subtree add --prefix lint-configs https://github.com/StackStorm/lint-configs.git master --squash ``` To use it (example with pylint) ```bash pylint -E --rcfile=./lint-configs/python/.pylintrc ... ``` And once you want to pull changes / updates from the lint-configs repository: ```bash git subtree pull --prefix lint-configs https://github.com/StackStorm/lint-configs.git master --squash ``` --- ### Overview (`pants-plugins/README.md`) ## pants plugins This directory contains StackStorm-specific plugins for pantsbuild. `pants` should be the primary entry point for development related tasks. This replaces the Makefile and related scripts such that they are more discoverable. The plugins here add custom goals or other logic into pants. To see available goals, do "pants help goals" and "pants help $goal". These plugins might be useful outside of the StackStorm project: - `uses_services` These StackStorm-specific plugins might be useful in other StackStorm-related repos. - `pack_metadata` These StackStorm-specific plugins are probably only useful for the st2 repo. - `api_spec` - `macros.py` (not a plugin - see pants.toml `[GLOBAL].build_file_prelude_globs`) - `release` - `sample_conf` - `schemas` ### `api_spec` plugin This plugin wires up pants to make sure `st2common/st2common/openapi.yaml` gets regenerated if needed. Now, whenever someone runs the `fmt` goal (eg `pants fmt st2common/st2common/openapi.yaml`), the api spec will be regenerated if any of the files used to generate it has changed. Also, running the `lint` goal will fail if the schemas need to be regenerated. This plugin also wires up pants so that the `lint` goal runs additional api spec validation on `st2common/st2common/openapi.yaml` with something like `pants lint st2common/st2common/openapi.yaml`. ### `macros.py` macros [Macros](https://www.pantsbuild.org/docs/macros) are a pants feature that can reduce "boilerplate"/duplication in BUILD files. The functions defined in `macros.py` are available in all the BUILD files, and using them looks just like using the normal BUILD targets. For documentation about our macros, please refer to the function docstrings in the `macros.py` file. ### `pack_metadata` plugin This plugin adds two new targets to pants: - `pack_metadata` - `pack_metadata_in_git_submodule` These targets include all StackStorm pack metadata files in a pack. Pack metadata includes top-level files (`pack.yaml`, `.yaml.example`, `config.schema.yaml`, and `icon.png`) and metadata (`*.yaml`, `*.yml`) for actions, action-aliases, policies, rules, and sensors. This plugin also wires up the `tailor` goal, so that it will add a `pack_metadata(name="metadata")` target wherever it finds a `pack.yaml` file. One of the packs in this repo is in a git submodule to test our handling of git submodules (`st2tests/st2tests/fixtures/packs/test_content_version`). If it is not checked out, then some of the tests will fail. If it is not checked out, `pack_metadata_in_git_submodule` handles providing a helpful, instructive error message as early as possible. ### `release` plugin This plugin implements the [`SetupKwargs`](https://www.pantsbuild.org/docs/plugins-setup-py) plugin hook that pants uses when it auto-generates a `setup.py` file whenever it builds a `python_distribution()` (ie a wheel or an sdist). This makes it easy to centralize all of the common bits of metadata that need to go in all the wheels (like `author="StackStorm"` or our `project_urls`). ### `sample_conf` plugin This plugin wires up pants to make sure `conf/st2.conf.sample` gets regenerated whenever the source files change. Now, whenever someone runs the `fmt` goal (eg `pants fmt conf/st2.conf.sample`), the sample will be regenerated if any of the files used to generate it have changed. Also, running the `lint` goal will fail if the sample needs to be regenerated. ### `schemas` plugin This plugin wires up pants to make sure `contrib/schemas/*.json` gets regenerated whenever the source files change. Now, whenever someone runs the `fmt` goal (eg `pants fmt contrib/schemas::`), the schemas will be regenerated if any of the files used to generate them have changed. Also, running the `lint` goal will fail if the schemas need to be regenerated. ### `uses_services` plugin This plugin validates that services are running if required. For example, some tests need mongo, so this plugin can ensure mongo is running. If it is not running, then an error with instructions on how to run it are given to the user. `uses_services` has some StackStorm-specific assumptions in it, but it might be generalizable. There are several other StackStorm-specific plugins, but some of them are only useful in the st2 repo. --- ### Overview (`contrib/chatops/README.md`) # Chatops Integration Pack Pack that provides management/integration with Chatops. It is intended to replace Hubot pack eventually. ## Actions * `post_message` - Post raw data to stream for chatops consumers to pick up * `post_result` - Send JSON formatted action output via hubot-stackstorm adapter * `format_execution_result` - Transforming trigger into a message for chatops client to show to user ## Legal Obligatory kudos to https://icons8.com/ for the icon. --- ### Overview (`contrib/linux/README.md`) # Linux Integration Pack This pack contains actions for commonly used Linux commands and tools. ## Sensors ### FileWatchSensor This sensor monitors files that are declared by a rule (one file per rule). Once a new line is detected, a trigger is emitted. ### Example Rule: This example rule will register with the FileWatchSensor to watch the file `/tmp/st2_test`. When a new line is detected, the trigger will be emitted, and the action `core.local` will echo the trigger data. ``` --- name: sample_rule_file_watch pack: "examples" description: Sample rule custom trigger type - add a file to be watched by file_watch_sensor in linux pack. enabled: true trigger: parameters: file_path: /tmp/st2_test type: linux.file_watch.line criteria: {} action: parameters: cmd: echo "{{trigger}}" ref: core.local ``` ### linux.file_watch.line trigger Example trigger payload: ```json { "file_path": "/var/log/auth.log", "file_name": "auth.log", "line": "Jan 18 13:38:15 vagrant-ubuntu-trusty-64 sudo: vagrant : TTY=pts/3 ; PWD=/data/stanley ; USER=root ; COMMAND=/bin/ls" } ``` ## Actions * ``vmstat`` - Wrapper around the `vmstat` command. * ``rsync`` - Wrapper around the `rsync` command. * ``netstat`` - Wrapper around the `netstat` command. * ``lsof`` - Wrapper around the `lsof` command. * ``service`` - Action which allows you to perform an action (start, stop, restart, etc.) on a system service. Currently it supports the following distributions: Ubuntu / Debian (upstart, sys init), RedHat / Fedora (systemd). * ``touch`` - Action which touches a file. * ``check_loadavg`` - Action which retrieves load average from a remote host. * ``check_processes`` - Action which retrieves useful information about matching process on a remote host. * ``dig`` - Python wrapper for dig command. Requires ``bind-utils`` rpm or ``dnsutils`` deb installed. ## Troubleshooting * On CentOS7/RHEL7, dig is not installed by default. Run ``sudo yum install bind-utils`` to install. * On CentOS8/RHEL8, lsof is not installed by default. Run ``sudo yum install lsof`` to install. --- ### Overview (`contrib/schemas/README.md`) # JSON schema for pack contents The JSON files under this directory are auto-generated by st2-generate-schemas. To update the schema files, run `make schemasgen`. Please do not edit these files manually. The purpose of these schema files are used for validation of pack contents during pack development. Development tools (i.e. VSCode extension) will need to be developed to take advantage of these schema files. --- ### Overview (`scripts/ci/README.md`) # CI Scripts This directory contains various scripts used in CI/CD system. Scripts in this directory are CI system agnostic. --- ### Overview (`scripts/github/README.md`) # Github Actions Scripts This directory contains various scripts which are used to run tests and other checks on Github Actions CI. As such, files in this directory and Github Actions specified. --- ### Overview (`contrib/default/actions/README.md`) # actions The actions folder contains action scripts and metadata files. See [Actions](http://docs.stackstorm.com/actions.html) and [Workflows](http://docs.stackstorm.com/workflows.html) for specifics on writing actions. Note that the lib sub-folder is always available for access for an action script. --- ### Overview (`contrib/default/rules/README.md`) # rules The rules folder contains rules. See [Rules](http://docs.stackstorm.com/rules.html) for specifics on writing rules. --- ### Overview (`contrib/default/sensors/README.md`) # sensors The sensors folder contains sensors. See [Sensors](http://docs.stackstorm.com/sensors.html) for specifics on writing sensors and registering TriggerTypes. --- ### Overview (`contrib/default/actions/workflows/README.md`) # actions/workflows The workflows folder contains workflow files. See [Workflows](http://docs.stackstorm.com/workflows.html) for specifics on writing workflows. --- --- METRICS --- - Files Extracted: 13 - Estimated Token Budget: ~4496 tokens - Canonical Reference: https://codewiki.google/github.com/StackStorm/st2