LICENSE Documentation

1. Project Overview & Quickstart (GothenburgBitFactory/LICENSE) Copy Chapter

LICENSE

Open-source repository GothenburgBitFactory/LICENSE

Repository Details Copy Section

• Repository: GothenburgBitFactory/LICENSE
• Primary Language: Code

Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.

2. Official Technical Reference & Guides (GothenburgBitFactory/GothenburgBitFactory.github.io) Copy Chapter

File: README.md Copy Chapter

GBF Site Repository

This repository contains the content of gothenburgbitfactory.org.

Contributing Copy Chapter

To clone this repository, run

git clone https://github.com/GothenburgBitFactory/gbf.org

For minor changes, it is perfectly fine to simply change the relevant Markdown files and make a pull request.

For more complex changes, here is how to run the site locally:

1. Install Hugo.
2. Within the repository, run
   shell Copy

   hugo server -D -w

   This defaults to running a web server on http://localhost:1313.

Now you can inspect your changes locally before opening a pull request.

File: content/anomaly/_index.md Copy Chapter

title: "Anomaly"
language: en
layout: single
source: "https://github.com/GothenburgBitFactory/anomaly"
rank: 1
menu: main
aliases:
- /projects/anomaly

Anomaly

Anomaly can detect anomalous data in a numeric stream.

{{< project_facts anomaly >}}

About Copy Chapter

In order to do this, anomaly needs to see a stream of numeric data, and apply one of its detection methods.
If an anomaly is detected, a response is made, chosen from one or more built in methods.

Numeric Stream Copy Chapter

Anomaly works best in a pipe, and will read only numeric data from its input.
As a simple example, suppose you wish to monitor load average and look for unusual spikes.
The load average can be obtained from the 'uptime' command:

$ uptime
11:40  up 15 days,  4:04, 6 users, load averages: 0.38 0.32 0.32

We can extract the 5-minute load (the second of the three numbers) using this:

$ uptime | cut -f 13 -d ' '
0.29

That number can be extracted once a minute, using this:

$ while [ 1 ]; do uptime | cut -f 13 -d ' '; sleep 60; done
0.29
0.26
0.19

That is the kind of data stream that anomaly monitors.
White space (spaces, tabs, newlines) between the numbers are ignored, so we can simulate the above stream like this:

$ echo 0.29 0.26 0.19

This is a convenient way to demonstrate anomaly, shown below.

Detection - Threshold Copy Chapter

The simplest detection method is threshold, which compares the data to an absolute value.
This method can use a minimum and a maximum value for comparison.
These alternatives are all valid, and make use of --min, --max or both:

anomaly --threshold --min 1.22 --max 9.75
anomaly --threshold --min 1.22
anomaly --threshold --max 9.75

In the following example, the values '1' and '10' would be detected as anomalies:

$ echo 2 1 3 6 10 5 | anomaly --threshold --min 1.5 --max 8
Anomalous data detected.  The value 1 is below the minimum of 1.5.
Anomalous data detected.  The value 10 is above the maximum of 8.

Detection - Standard Deviation Copy Chapter

Standard deviation measures differences from the mean value of a sample of data, and is useful for detecting extraordinary values.
The sample size can be chosen such that there is enough data to determine a good mean value.
The limited sample size means that a rolling window of data is used, and therefore the mean and standard deviation is updated for the current window.
This makes the monitoring somewhat adaptive.
Here is an example:

anomaly --stddev --sample 20

This uses a sample size of the 20 most recent values, and will detect any values that are +/- 1 standard deviation from the mean.
An example:

$ echo 1 2 3 4 5 6 | anomaly --stddev --sample 5
Anomalous  data detected.  The value 6 is more than 1
sigma(s) above the mean value 3, with a sample size of 5.

With a sample size of 5, comparisons begin only after the 6th value is seen.
In the example, the mean value of [1 2 3 4 5] is 3, and the standard deviation is 1.58.
This means that the 6th value is considered an anomaly if it is outside the range (3 +/- 1.58), which is from 1.42 to 4.58.

To make this less sensitive, a coefficient is introduced, which defaults to 1.0 (as above) but can be overridden:

$ echo 1 2 3 4 5 6 | anomaly --stddev --sample 5 --coefficient 1.9
$

In this example, the 6th value is not considered an anomaly because it is within the range (3 +/- (1.9 * 1.58)), which is between -0.002 and 6.002.

Response - Message Copy Chapter

The message response is the default, and consists of a single line of printed text.
It is a description of why the data value is considered an anomaly.
Here is an example:

$ echo 1 2 3 | anomaly --threshold --max 2.5
Anomalous data detected.  The value 3 is above the maximum of 2.5.

The message can be suppressed, but another response must be specified:

$ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet ...

Response - Execute Copy Chapter

Anomaly can execute a program in response to detection.
Here an example uses the 'date' command, but any program can be used:

$ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet --execute '/bin/date +%s'
1361727327

Response - Signal Copy Chapter

Anomaly can send a USR1 signal to a program in response to detection:

$ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet --pid 12345

This sends the USR1 signal to the process with PID 12345.
The receiving program would need to respond accordingly.

File: content/anomaly/latest.md Copy Chapter

layout: plain
aliases:
- /latest/anomaly

{{< latest_version anomaly >}}

File: content/clog/docs/_index.md Copy Chapter

title: "Docs" language: en layout: single menu: main Copy Chapter

{{< lead >}}Here is the complete set of Clog documentation.{{< /lead >}}

Getting Started Copy Chapter

• What is Clog?
• How does Clog work?
• Download and build the tarball
• Clone the code repository and build the development snapshot

Usage Copy Chapter

• Getting Help
• Setup
• Rule Syntax
• Rule Sections
• Rule Patterns
• Rule Actions
• Clog Options
• Color

Example Rule Sets Copy Chapter

• Syslog
• Apache

File: content/clog/docs/actions.md Copy Chapter

title: "Clog - Rule Actions" Copy Chapter

Rule Actions

When a rules matches, an action is taken.
There are currently four supported actions:

• line
• match
• blank
• suppress

The `line` Action Copy Chapter

The line action instructs Clog to color the entire line.
This makes the whole line more visible.
An example might be to color a line in a log file if that line reports a critical error.

The `match` Action Copy Chapter

The match action instructs clog to only color the matching pattern and not the entire line.
This make individual keywords more visible.
An example might be to highlight simply an error code, or server name.

The `blank` Action Copy Chapter

The blank action causes a blank line to be written both before and after the matching line.
This is another form of emphasis.

The `suppress` Action Copy Chapter

The suppress action causes the whole line to be removed from the output.
You will not see a suppressed line at all.
This is useful for stripping out noise from a log file.

Example Copy Chapter

Here is a default rule set that includes all the different actions.
The first rule will make any line completely blue if it contains the word 'blue'.
The second rule colors the word 'red' red.
The third puts blank lines around any line containing the word 'emphasize'.
The fourth rule suppresses any line containing the word 'ignore'.

Here is a sample file that triggers every rule, so we should see the actions of all rules taken.
Can you predict what will happen?

Here is the output:

This shows rules precedence also - the second rule finds the word 'red' in the blue line.

File: content/clog/docs/clone.md Copy Chapter

title: "Clog - Clone" Copy Chapter

How To Build Clog

This is for developers.
Specifically those who know how to use tools, satisfy dependencies, and want to set up a development environment.
It is not user-friendly.

You'll need these tools:

• Git
• CMake
• Make
• C++ compiler, currently gcc 4.7+ or clang 3.3+ for full C++11 support
• Python 2.7 or later (for tests)

Cloning the Repo Copy Chapter

$ git clone --recursive https://github.com/GothenburgBitFactory/clog.git clog.git

Building the Stable Version Copy Chapter

The master branch always represents the more recently released version, and should be your preferred choice.

$ cd clog.git
$ git checkout master                # Master is the stable branch.
$ cmake -DCMAKE_BUILD_TYPE=release . # 'release' for performance.
$ make                               # Just build it.

Building the Dev Branch Copy Chapter

The dev branch is always the highest numbered branch, in this example, 1.4.0.

$ cd clog.git
$ git checkout 1.4.0                 # Dev branch
$ git submodule init && git submodule update # Fetch the submodule
$ cmake -DCMAKE_BUILD_TYPE=debug .   # debug or release.

Default: neither.

$ make VERBOSE=1                     # Shows details

Build the debug type if you want symbols in the binary for debugging.

Note: the development branch is not always stable, doesn't always pass tests, and sometimes doesn't even compile.
Choose wisely.

Running the Test Suite Copy Chapter

There are C++ and Python, build and run them:

$ cd clog.git/test
$ make VERBOSE=1     # Shows details
$ ./run_all          # Runs all tests silently > all.log
$ ./problems         # Find errors in all.log

Submitting a Patch Copy Chapter

Talk to us first - make sure you are working on something that is wanted.
Patches will not be applied simply because you did the work, or because it was easy, or because you want it.
Instead, the change has to be considered right for the project.

Remember the various forms of documentation involved, and the test suite.
Work on the dev branch, not master.
When you are ready to submit, do this:

$ git commit

Follow the standard form for commit messages, which looks like this:

Category: Short message

- Details
- Details

Here is a good example:

TW-1636: UUID with numeric-only first segment is not parsed properly

- Switched Nibbler::getUUID to Nibbler::getPartialUUID, which caused partial
  UUID matching to fail sometimes.
- Changed precedence to search for UUID before ID, which solves the numeric
  UUID problem.

Create the patch using this:

$ git format-patch HEAD^

Mail the patch to [email protected] or attach it to the appropriate ticket in the bug tracker.
If you do the latter, make sure someone knows about it, or it could go unnoticed.

Expect feedback.
It is unlikely your patch will be accepted unmodified.
Usually this is because you violated the coding style, worked in the wrong branch, or forgot about documentation and unit tests.

File: content/clog/docs/color.md Copy Chapter

title: "Clog - Color" Copy Chapter

Color

Clog uses the same color model as Taskwarrior.
A color iѕ described with one or more words.
You can specify the foreground color, the background color, and some text effects.

Color can be both simple and complex.
You can completely ignore the color models below, and just use color in one of these ways:

red
white on red
on red
bold blue
bold yellow on bright green underline

Or you can have full control using a 256-color model.
But sometimes all you need is red and green.

16-Color Model Copy Chapter

The basic color support is provided through named colors:

black, red, blue, green, magenta, cyan, yellow, white

Shown here used as foreground and background colors:

Foreground color (for text) is simply specified as one of the above colors, or not specified at all to use the default terminal text color.

Background color is specified by using the word on, and one of the above colors.
Some examples:

green
green on yellow
on yellow

These colors can be modified further, by making the foreground bold, or by making the background bright.
Some examples:

bold green
bold white on bright red
on bright cyan

Bold is not uniformly supported by all terminal emulators.
Sometimes it results in a slightly brighter color, a lightly heavier typeface, or no difference.

The order of the words is not important, so the following are equivalent:

bold green
green bold

But the on is important - colors before the on are foreground, and colors after on are background.

There is an additional underline attribute that may be used:

underline bold red on black

Much like bold, you may see different behavior among different terminal emulators.
There is also an inverse attribute that flips foreground and background colors:

inverse red

Some combinations look very nice, some look terrible.
Different terminal programs do implement slightly different versions of 'red', for example, so you may see some unexpected variation across machines.
The brightness of your display is also a factor.

256-Color Model Copy Chapter

Using 256 colors follows the same form, but the names are different, and some colors can be referenced in different ways.
First there is by color ordinal, which is like this:

color0
color1
color2
...
color255

This gives you access to all 256 colors, but doesn't help you much.
This range is a combination of 8 basic colors (color0 - color7), then 8 brighter variations (color8 - color15).
Then a block of 216 colors (color16 - color231).
Then a block of 24 gray colors (color232 - color255).

The large block of 216 colors (6x6x6 = 216) represents a color cube, which can be addressed via RGB values from 0 to 5 for each component color.
A value of 0 means none of this component color, and a value of 5 means the most intense component color.
For example, a bright red is specified as:

rgb500

And a darker red would be:

rgb300

Note that the three digits represent the three component values, so in this example the 5, 0 and 0 represent red=5, green=0, blue=0.
Combining intense red with no green and no blue yields red.
Similarly, blue and green are:

rgb005
rgb050

Another example - bright yellow - is a mix of bright red and bright green, but no blue component, so bright yellow is addressed as:

rgb550

A soft pink would be specified as:

rgb515

You may notice that the large color block is represented as 6 squares.
All colors in the first square have a red value of 0.
All colors in the 6th square have a red value of 5.
Within each square, blue ranges from 0 to 5 left to right, and within each square green ranges from 0 to 5, top to bottom.
This scheme takes some getting used to.

The block of 24 gray colors can also be accessed as gray0 - gray23, in a continuous ramp from black to white.

True Color Model Copy Chapter

This is true 24-bit color, and is not yet supported.

Mixing Color Models Copy Chapter

If you specify 16-color colors, and view on a 256-color terminal, no problem.
If you try the reverse, specifying 256-color colors and viewing on a 16-color terminal, you will be disappointed, perhaps even appalled.
The results are usually that only the 16-color model is shown, and the rest is just blank.

There is some limited color mapping - for example, if you were to specify this combination:

red on gray3

Then you are mixing a 16-color and 256-color specifications.
Taskwarrior will map red to color1, and proceed.
Note that red and color1 are not quite the same tone.

Note also that there are no bold or bright attributes when dealing with 256 colors, but there is still underline available.

Terminal Settings Copy Chapter

All of the above directly emit Xterm escape sequences to control color and have absolutely nothing to do with your $TERM setting.

File: content/clog/docs/download.md Copy Chapter

title: "Clog - Download" Copy Chapter

How To Build Clog

This is for developers.
Specifically those who know how to use tools, satisfy dependencies, and want to set up a development environment.
It is not user-friendly.

You'll need these tools:

• Git
• CMake
• Make
• C++ compiler, currently gcc 4.7+ or clang 3.3+, for full C++11 support

Downloading the Repo Copy Chapter

The next step is to obtain the code.
This means getting the released tarball.
You can download the tarball with curl, as an example of just one of many ways to download the tarball.

$ curl -L -O https://github.com/GothenburgBitFactory/clog/releases/download/v{{< latest_version clog >}}/clog-{{< latest_version clog >}}.tar.gz

Building the Tarball Copy Chapter

Expand the tarball, build Clog, and install it.
This copies files into the right place, and installs the man page.

$ tar xzf clog-{{< latest_version clog >}}.tar.gz
$ cd clog-{{< latest_version clog >}}
$ cmake -DCMAKE_BUILD_TYPE=release .
...
$ make
...
$ sudo make install

Submitting a Patch Copy Chapter

See here for details.

File: content/clog/docs/example_apache.md Copy Chapter

title: "Clog - Apache Rules" Copy Chapter

Apache Rules

This example shows color highlighting for HTTP status code.

# Apache access.log status codes

apache rule / 2[0-9][0-9] / --> green match
apache rule / 3[0-9][0-9] / --> yellow match
apache rule / 4[0-9][0-9] / --> red match
apache rule / 5[0-9][0-9] / --> bold red match

File: content/clog/docs/example_syslog.md Copy Chapter

title: "Clog - Syslog Rules" Copy Chapter

Syslog Rules

This example shows color highlighting for syslog severities.

# Standard syslog entries.

default rule /warn|debug/   --> yellow line
default rule /error|severe/ --> red line
default rule "critical"     --> bold red line
default rule "critical"     --> blank
default rule "ignore"       --> suppress