### Doc/Userguide/3rd Party Integration/Symantec Sslv
Symantec SSL Visibility (BlueCoat)
==================================
As Suricata itself cannot decrypt SSL/TLS traffic, some organizations use
a decryption product to handle this. This document will offer some advice
on using Suricata with the Symantec SSL Visibility appliance (formerly
known as BlueCoat).
Appliance Software Version
--------------------------
The appliance comes with two major software version options. The 3.x and 4.x
series. Suricata works best with the 4.x series.
TLS1.3 is only properly supported in the 4.x version of the appliance
software.
Magic Markers
-------------
The appliance has an indicator that data is decrypted. This is done using
a special magic source MAC address, or using a special VLAN header. Since
Suricata can use VLANs as part of flow tracking, it is recommended to use
the source MAC method.
In the 3.x version of the software these markers are always there, the
config just allows setting which type will be used. In the 4.x software the
markers are optional.
TCP handling
------------
In the 3.x software, a bit of care is required in TCP stream reassembly
handling in Suricata. The decrypted traffic is presented to the IDS as
TCP data packets, that are not ack'd as regularly as would be expected
in a regular TCP session. A large TCP window is used to not violate the
TCP specs. Since in IDS mode Suricata waits for ACKs for much of its
processing, this can lead to delays in detection and logging, as well
as increased resource usage due to increased data buffering.
To avoid this, enable the 'stream.inline' mode, which processed data
segments as they come in without waiting for the ACKs.
The 4.x software sends more regular ACKs and does not need any special
handling on the Suricata side.
TLS matching in Suricata
------------------------
The appliance takes care of the TLS handling and decryption, presenting
only the decrypted data to Suricata. This means that Suricata will not
see the TLS handshake. As a consequence of this, Suricata cannot inspect
the TLS handshake or otherwise process it. This means that for decrypted
TLS sessions, Suricata will not do any TLS keyword inspection (such as
fingerprint matching and ja3), TLS logging or TLS certificate extraction.
If it is important to match on and/or log such information as well, the
appliance facilities for matching and logging themselves will have to be
used.
For TLS traffic where the appliance security policy does not lead to
decryption of the traffic, the TLS handshake is presented to Suricata
for analysis and logging.
IPS
---
When using Suricata in IPS mode with the appliance, some things will
have to be considered:
* if Suricata DROPs a packet in the decrypted traffic, this will be seen
by the appliance after which it will trigger a RST session teardown.
* if a packet takes more than one second to process, it will automatically
be considered a DROP by the appliance. This should not happen in normal
traffic, but with very inefficient Lua scripts this could perhaps
happen. The appliance can also be configured to wait for 5 seconds.
* When using the Suricata 'replace' keyword to modify data, be aware
that the 3.x appliance software will not pass the modification on to
the destination so this will not have any effect. The 4.x appliance
software does support passing on modifications that were made to the
unencrypted text, by default this feature is disabled but you can
enable it if you want modifications to be passed on to the destination
in the re-encrypted stream. Due to how Suricata works, the size of
the payloads cannot be changed.
---
### Doc/Userguide/Appendix/Eve Schema
EVE JSON Schema
###############
The Suricata source distribution contains a JSON schema for the EVE
log files. This schema follows the `JSON Schema
`_: Our
own tool for verifying every Suricata pull request, validates all
EVE logs generated against the schema.
* ``./scripts/eve-parity.py``: Found inside the Suricata source code
when checked out with ``git``, is a tool to provide information on
how log fields map to keywords, or how keywords map to log entries.
* ``./scripts/evedoc.py``: Generate documentation from the schema,
such as the :doc:`eve-index` included in this documentation.
---
### Doc/Userguide/Capture Hardware/Af Packet
.. _afpacket:
AF_PACKET
#########
Introduction
************
AF_PACKET is a capture interface to the Linux Kernel.
Config Options
**************
enable-hwtimestamp
==================
Boolean option to enable hardware timestamping on an interface.
By default the hardware timestamping support is disabled.
Hardware timestamping can lead to issue of the NIC and kernel getting out of sync. See
`ticket 7585 `_.
::
af-packet:
- interface: eth0
cluster-id: 99
enable-hwtimestamp: true
cluster-type: cluster_flow
---
### Doc/Userguide/Capture Hardware/Af Xdp
AF_XDP
======
AF_XDP (eXpress Data Path) is a high speed capture framework for Linux that was
introduced in Linux v4.18. AF_XDP aims at improving capture performance by
redirecting ingress frames to user-space memory rings, thus bypassing the network
stack.
Note that during ``af_xdp`` operation the selected interface cannot be used for
regular network usage.
Further reading:
- https://www.kernel.org/doc/html/latest/networking/af_xdp.html
Compiling Suricata
------------------
Linux
~~~~~
libxdp and libpbf are required for this feature. When building from source the
development files will also be required.
Example::
dnf -y install libxdp-devel libbpf-devel
This feature is enabled provided the libraries above are installed, the user
does not need to add any additional command line options.
The command line option ``--disable-af-xdp`` can be used to disable this
feature.
Example::
./configure --disable-af-xdp
Starting Suricata
-----------------
IDS
~~~
Suricata can be started as follows to use af-xdp:
::
af-xdp:
suricata --af-xdp=
suricata --af-xdp=igb0
In the above example Suricata will start reading from the `igb0` network interface.
AF_XDP Configuration
--------------------
Each of these settings can be configured under ``af-xdp`` within the "Configure
common capture settings" section of suricata.yaml configuration file.
The number of threads created can be configured in the suricata.yaml configuration
file. It is recommended to use threads equal to NIC queues/CPU cores.
Another option is to select ``auto`` which will allow Suricata to configure the
number of threads based on the number of RSS queues available on the NIC.
With ``auto`` selected, Suricata spawns receive threads equal to the number of
configured RSS queues on the interface.
::
af-xdp:
threads:
threads: auto
threads: 8
Advanced setup
---------------
af-xdp capture source will operate using the default configuration settings.
However, these settings are available in the suricata.yaml configuration file.
Available configuration options are:
force-xdp-mode
~~~~~~~~~~~~~~
There are two operating modes employed when loading the XDP program, these are:
- XDP_DRV: Mode chosen when the driver supports AF_XDP
- XDP_SKB: Mode chosen when no AF_XDP support is unavailable
XDP_DRV mode is the preferred mode, used to ensure best performance.
::
af-xdp:
force-xdp-mode: where: value =
force-xdp-mode: drv
force-bind-mode
~~~~~~~~~~~~~~~
During binding the kernel will first attempt to use zero-copy (preferred). If
zero-copy support is unavailable it will fallback to copy mode, copying all
packets out to user space.
::
af-xdp:
force-bind-mode: where: value =
force-bind-mode: zero
For both options, the kernel will attempt the 'preferred' option first and
fallback upon failure. Therefore the default (none) means the kernel has
control of which option to apply. By configuring these options the user
is forcing said option. Note that if enabled, the bind will only attempt
this option, upon failure the bind will fail i.e. no fallback.
mem-unaligned
~~~~~~~~~~~~~~~~
AF_XDP can operate in two memory alignment modes, these are:
- Aligned chunk mode
- Unaligned chunk mode
Aligned chunk mode is the default option which ensures alignment of the
data within the UMEM.
Unaligned chunk mode uses hugepages for the UMEM.
Hugepages start at the size of 2MB but they can be as large as 1GB.
Lower count of pages (memory chunks) allows faster lookup of page entries.
The hugepages need to be allocated on the NUMA node where the NIC and CPU resides.
Otherwise, if the hugepages are allocated only on NUMA node 0 and the NIC is
connected to NUMA node 1, then the application will fail to start.
Therefore, it is recommended to first find out to which NUMA node the NIC is
connected to and only then allocate hugepages and set CPU cores affinity
to the given NUMA node.
Memory assigned per socket/thread is 16MB, so each worker thread requires at least
16MB of free space. As stated above hugepages can be of various sizes, consult the
OS to confirm with ``cat /proc/meminfo``.
Example ::
8 worker threads * 16Mb = 128Mb
hugepages = 2048 kB
so: pages required = 62.5 (63) pages
See https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt for detailed
description.
To enable unaligned chunk mode:
::
af-xdp:
mem-unaligned:
mem-unaligned: yes
Introduced from Linux v5.11 a ``SO_PREFER_BUSY_POLL`` option has been added to
AF_XDP that allows a true polling of the socket queues. This feature has
been introduced to reduce context switching and improve CPU reaction time
during traffic reception.
Enabled by default, this feature will apply the following options, unless
disabled (see below). The following options are used to configure this feature.
enable-busy-poll
~~~~~~~~~~~~~~~~
Enables or disables busy polling.
::
af-xdp:
enable-busy-poll:
enable-busy-poll: yes
busy-poll-time
~~~~~~~~~~~~~~
Sets the approximate time in microseconds to busy poll on a ``blocking receive``
when there is no data.
::
af-xdp:
busy-poll-time:
busy-poll-time: 20
busy-poll-budget
~~~~~~~~~~~~~~~~
Budget allowed for batching of ingress frames. Larger values means more
frames can be stored/read. It is recommended to test this for performance.
::
af-xdp:
busy-poll-budget:
busy-poll-budget: 64
Linux tunables
~~~~~~~~~~~~~~~
The ``SO_PREFER_BUSY_POLL`` option works in concert with the following two Linux
knobs to ensure best capture performance. These are not socket options:
- gro-flush-timeout
- napi-defer-hard-irq
The purpose of these two knobs is to defer interrupts and to allow the
NAPI context to be scheduled from a watchdog timer instead.
The ``gro-flush-timeout`` indicates the timeout period for the watchdog
timer. When no traffic is received for ``gro-flush-timeout`` the timer will
exit and softirq handling will resume.
The ``napi-defer-hard-irq`` indicates the number of queue scan attempts
before exiting to interrupt context. When enabled, the softirq NAPI context will
exit early, allowing busy polling.
::
af-xdp:
gro-flush-timeout: 2000000
napi-defer-hard-irq: 2
Hardware setup
---------------
Intel NIC setup
~~~~~~~~~~~~~~~
Intel network cards don't support symmetric hashing but it is possible to emulate
it by using a specific hashing function.
Follow these instructions closely for desired result::
ifconfig eth3 down
Enable symmetric hashing ::
ifconfig eth3 down
ethtool -L eth3 combined 16 # if you have at least 16 cores
ethtool -K eth3 rxhash on
ethtool -K eth3 ntuple on
ifconfig eth3 up
./set_irq_affinity 0-15 eth3
ethtool -X eth3 hkey 6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A equal 16
ethtool -x eth3
ethtool -n eth3
In the above setup you are free to use any recent ``set_irq_affinity`` script. It is available in any Intel x520/710 NIC sources driver download.
**NOTE:**
We use a special low entropy key for the symmetric hashing. `More info about the research for symmetric hashing set up `_
Disable any NIC offloading
~~~~~~~~~~~~~~~~~~~~~~~~~~
Suricata shall disable NIC offloading based on configuration parameter ``disable-offloading``, which is enabled by default.
See ``capture`` section of yaml file.
::
capture:
# disable NIC offloading. It's restored when Suricata exits.
# Enabled by default.
#disable-offloading: false
Balance as much as you can
~~~~~~~~~~~~~~~~~~~~~~~~~~
Try to use the network card's flow balancing as much as possible ::
for proto in tcp4 udp4 ah4 esp4 sctp4 tcp6 udp6 ah6 esp6 sctp6; do
/sbin/ethtool -N eth3 rx-flow-hash $proto sd
done
This command triggers load balancing using only source and destination IPs. This may be not optimal
in terms of load balancing fairness but this ensures all packets of a flow will reach the same thread
even in the case of IP fragmentation (where source and destination port will not be available for
some fragmented packets).
---
### Doc/Userguide/Capture Hardware/Dpdk
.. _dpdk:
DPDK
====
Introduction
-------------
The Data Plane Development Kit (DPDK) is a set of libraries and drivers that
enhance and speed up packet processing in the data plane. Its primary use is to
provide faster packet processing by bypassing the kernel network stack, which
can provide significant performance improvements. For detailed instructions on
how to setup DPDK, please refer to :doc:`../configuration/suricata-yaml` to
learn more about the basic setup for DPDK.
The following sections contain examples of how to set up DPDK and Suricata for
more obscure use-cases.
Hugepage analysis
-----------------
Suricata can analyse utilized hugepages on the system. This can be particularly
beneficial when there's a potential overallocation of hugepages.
The hugepage analysis is designed to examine the hugepages in use and
provide recommendations on an adequate number of hugepages. This then ensures
Suricata operates optimally while leaving sufficient memory for other
applications on the system. The analysis works by comparing snapshots of the
hugepages before and after Suricata is initialized. After the initialization,
no more hugepages are allocated by Suricata.
The hugepage analysis can be seen in the Perf log level and is printed out
during the Suricata start. It is only printed when Suricata detects some
disrepancies in the system related to hugepage allocation.
It's recommended to perform this analysis from a "clean" state -
that is a state when all your hugepages are free. It is especially recommended
when no other hugepage-dependent applications are running on your system.
This can be checked in one of two ways:
.. code-block::
# global check
cat /proc/meminfo
HugePages_Total: 1024
HugePages_Free: 1024
# per-numa check depends on NUMA node ID, hugepage size,
# and nr_hugepages/free_hugepages - e.g.:
cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/free_hugepages
After the termination of Suricata and other hugepage-related applications,
if the count of free hugepages is not equal with the total number of hugepages,
it indicates some hugepages were not freed completely.
This can be fixed by removing DPDK-related files from the hugepage-mounted
directory (filesystem).
It's important to exercise caution while removing hugepages, especially when
other hugepage-dependent applications are in operation, as this action will
disrupt their memory functionality.
Removing the DPDK files from the hugepage directory can often be done as:
.. code-block:: bash
sudo rm -rf /dev/hugepages/rtemap_*
# To check where hugepages are mounted:
dpdk-hugepages.py -s
# or
mount | grep huge
Bond interface
--------------
Link Bonding Poll Mode Driver (Bond PMD), is a software
mechanism provided by the Data Plane Development Kit (DPDK) for aggregating
multiple physical network interfaces into a single logical interface.
Bonding can be e.g. used to:
* deliver bidirectional flows of tapped interfaces to the same worker,
* establish redundancy by monitoring multiple links,
* improve network performance by load-balancing traffic across multiple links.
Bond PMD is essentially a virtual driver that manipulates with multiple
physical network interfaces. It can operate in multiple modes as described
in the `DPDK docs
`_
The individual bonding modes can accustom user needs.
DPDK Bond PMD has a requirement that the aggregated interfaces must be
the same device types - e.g. both physical ports run on mlx5 PMD.
Bond PMD supports multiple queues and therefore can work in workers runmode.
It should have no effect on traffic distribution of the individual ports and
flows should be distributed by physical ports according to the RSS
configuration the same way as if they would be configured independently.
As an example of Bond PMD, we can setup Suricata to monitor 2 interfaces
that receive TAP traffic from optical interfaces. This means that Suricata
receive one direction of the communication on one interface and the other
direction is received on the other interface.
::
...
dpdk:
eal-params:
proc-type: primary
vdev: 'net_bonding0,mode=0,slave=0000:04:00.0,slave=0000:04:00.1'
# DPDK capture support
# RX queues (and TX queues in IPS mode) are assigned to cores in 1:1 ratio
interfaces:
- interface: net_bonding0 # PCIe address of the NIC port
# Threading: possible values are either "auto" or number of threads
# - auto takes all cores
# in IPS mode it is required to specify the number of cores and the
# numbers on both interfaces must match
threads: 4
...
In the DPDK part of suricata.yaml we have added a new parameter to the
eal-params section for virtual devices - `vdev`.
DPDK Environment Abstraction Layer (EAL) can initialize some virtual devices
during the initialization of EAL.
In this case, EAL creates a new device of type `net_bonding`. Suffix of
`net_bonding` signifies the name of the interface (in this case the zero).
Extra arguments are passed after the device name, such as the bonding mode
(`mode=0`). This is the round-robin mode as is described in the DPDK
documentation of Bond PMD.
Members (slaves) of the `net_bonding0` interface are appended after
the bonding mode parameter.
When the device is specified within EAL parameters, it can be used within
Suricata `interfaces` list. Note that the list doesn't contain PCIe addresses
of the physical ports but instead the `net_bonding0` interface.
Threading section is also adjusted according to the items in the interfaces
list by enablign set-cpu-affinity and listing CPUs that should be used in
management and worker CPU set.
::
...
threading:
set-cpu-affinity: yes
cpu-affinity:
management-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
receive-cpu-set:
cpu: [ 0 ] # include only these CPUs in affinity settings
worker-cpu-set:
cpu: [ 2,4,6,8 ]
...
Interrupt (power-saving) mode
-----------------------------
The DPDK is traditionally recognized for its polling mode operation.
In this mode, CPU cores are continuously querying for packets from
the Network Interface Card (NIC). While this approach offers benefits like
reduced latency and improved performance, it might not be the most efficient
in scenarios with sporadic or low traffic.
The constant polling can lead to unnecessary CPU consumption.
To address this, DPDK offers an `interrupt` mode.
The obvious advantage that interrupt mode brings is power efficiency.
So far in our tests, we haven't observed a decrease in performance. Suricata's
performance has actually seen a slight improvement.
The (IPS runmode) users should be aware that interrupts can
introduce non-deterministic latency. However, the latency should never be
higher than in other (e.g. AF_PACKET/AF_XDP/...) capture methods.
Interrupt mode in DPDK can be configured on a per-interface basis.
This allows for a hybrid setup where some workers operate in polling mode,
while others utilize the interrupt mode.
The configuration for the interrupt mode can be found and modified in the
DPDK section of the suricata.yaml file.
Below is a sample configuration that demonstrates how to enable the interrupt mode for a specific interface:
::
...
dpdk:
eal-params:
proc-type: primary
interfaces:
- interface: 0000:3b:00.0
interrupt-mode: true
threads: 4
.. _dpdk-automatic-interface-configuration:
Automatic interface configuration
---------------------------------
A number of interface properties can be manually configured. However, Suricata
can automatically configure the interface properties based on the NIC
capabilities. This can be done by setting ``auto`` to ``mempool-size``,
``mempool-cache-size``, ``rx-descriptors``, and ``tx-descriptors`` interface
node properties.
This will allow Suricata to automatically set the sizes of individual properties
according to the best-effort calculation based on the NIC capabilities.
For example, receive (RX) descriptors are calculated based on the maximal
"power of 2" that is lower or equal to the number of descriptors supported
by the NIC. Number of TX descriptors depends on the configured ``copy-mode``.
IDS (none) mode uses no TX descriptors and does not create any TX queues by
default. IPS and TAP mode uses the same number of TX descriptors as RX
descriptors.
The number of mempool and its cache is then derived from the count of
descriptors.
Rx (and Tx) descriptors are set to the highest possible value to allow more
buffer room when traffic spikes occur. However, it requires more memory.
Individual properties can still be set manually if needed.
.. note:: Mellanox ConnectX-4 NICs may not support auto-configuration of
``RX /TX descriptors``. Instead it can be set to a fixed value (e.g. 16384).
.. _dpdk-link-state-change-timeout:
Link State Change timeout
-------------------------
The `linkup-timeout` YAML configuration option allows the user to set a timeout
period to wait until the interface's link is detected. This ensures that
Suricata does not start processing packets until the link is up. This option is
particularly useful for Intel E810 (Ice) NICs, which begin receiving packets
only after a few seconds have passed since the interface started. In such cases,
if this check is disabled, Suricata reports as started but only begins
processing packets after a few seconds. This issue has not been observed with
other cards.
Setting the value to 0 causes Suricata to skip the link check.
If the interface's link remains down after the timeout period, Suricata warns
the user but continues with the engine initialization.
.. _dpdk-encapsulation-stripping:
Encapsulation stripping
-----------------------
Suricata supports stripping the hardware-offloaded encapsulation stripping on
the supported NICs. Currently, VLAN encapsulation stripping is supported.
VLAN encapsulation stripping can be enabled with `vlan-strip-offload`.
---
### Doc/Userguide/Capture Hardware/Ebpf Xdp
.. _ebpf-xdp:
eBPF and XDP
============
Introduction
------------
eBPF stands for extended BPF. This is an extended version of Berkeley Packet Filter available in recent
Linux kernel versions (at least kernel version 4.13 or greater).
It provides more advanced features with eBPF programs developed in C and capability to use structured data shared
between kernel and userspace.
eBPF is used for three things in Suricata:
- eBPF filter: any BPF like filter can be developed. An example of filter accepting only packet for some VLANs is provided. A bypass implementation is also provided.
- eBPF load balancing: provide programmable load balancing. Simple ippair load balancing is provided.
- XDP programs: Suricata can load XDP programs. A bypass program is provided.
Bypass can be implemented in eBPF and XDP. The advantage of XDP is that the packets are dropped at the earliest stage
possible. So performance is better. But bypassed packets don't reach the network so you can't use this on regular
traffic but only on duplicated/sniffed traffic.
The bypass implementation relies on one of the most powerful concept of eBPF: maps. A map is a data structure
shared between user space and kernel space/hardware. It allows user space and kernel space to interact, pass
information. Maps are often implemented as arrays or hash tables that can contain arbitrary key, value pairs.
XDP
~~~
XDP provides another Linux native way of optimising Suricata's performance on sniffing high speed networks:
XDP or eXpress Data Path provides a high performance, programmable network data path in the Linux kernel as part of the IO Visor Project. XDP provides bare metal packet processing at the lowest point in the software stack which makes it ideal for speed without compromising programmability. Furthermore, new functions can be implemented dynamically with the integrated fast path without kernel modification.
More info about XDP:
- `IOVisor's XDP page `__
- `Cilium's BPF and XDP reference guide `_
Disable any NIC offloading
~~~~~~~~~~~~~~~~~~~~~~~~~~
Run the following command to disable offloading ::
for i in rx tx tso ufo gso gro lro tx nocache copy sg txvlan rxvlan; do
/sbin/ethtool -K eth3 $i off 2>&1 > /dev/null;
done
Balance as much as you can
~~~~~~~~~~~~~~~~~~~~~~~~~~
Try to use the network card's flow balancing as much as possible ::
for proto in tcp4 udp4 ah4 esp4 sctp4 tcp6 udp6 ah6 esp6 sctp6; do
/sbin/ethtool -N eth3 rx-flow-hash $proto sd
done
This command triggers load balancing using only source and destination IPs. This may be not optimal
in term of load balancing fairness but this ensures all packets of a flow will reach the same thread
even in the case of IP fragmentation (where source and destination port will not be available
for some fragmented packets).
The XDP CPU redirect case
~~~~~~~~~~~~~~~~~~~~~~~~~
If ever your hardware is not able to do a symmetric load balancing but support XDP in driver mode, you
can then use the CPU redirect map support available in the `xdp_filter.bpf` and `xdp_lb.bpf` file. In
this mode, the load balancing will be done by the XDP filter and each CPU will handle the whole packet
treatment including the creation of the skb structure in kernel.
You will need Linux 4.15 or newer to use that feature.
To do so set the `xdp-cpu-redirect` variable in af-packet interface configuration to a set of CPUs.
Then use the `cluster_cpu` as load balancing function. You will also need to set the affinity
to be certain that CPU cores that have the skb assigned are used by Suricata.
Also to avoid out of order packets, you need to set the RSS queue number to 1. So if our interface
is `eth3` ::
/sbin/ethtool -L eth3 combined 1
In case your system has more then 64 core, you need to set `CPUMAP_MAX_CPUS` to a value greater
than this number in `xdp_lb.c` and `xdp_filter.c`.
A sample configuration for pure XDP load balancing could look like ::
- interface: eth3
threads: 16
cluster-id: 97
cluster-type: cluster_cpu
xdp-mode: driver
xdp-filter-file: /usr/libexec/suricata/ebpf/xdp_lb.bpf
xdp-cpu-redirect: ["1-17"] # or ["all"] to load balance on all CPUs
ring-size: 200000
It is possible to use `xdp_monitor` to have information about the behavior of CPU redirect. This
program is available in Linux tree under the `samples/bpf` directory and will be build by the
make command. Sample output is the following ::
sudo ./xdp_monitor --stats
XDP-event CPU:to pps drop-pps extra-info
XDP_REDIRECT 11 2,880,212 0 Success
XDP_REDIRECT total 2,880,212 0 Success
XDP_REDIRECT total 0 0 Error
cpumap-enqueue 11:0 575,954 0 5.27 bulk-average
cpumap-enqueue sum:0 575,954 0 5.27 bulk-average
cpumap-kthread 0 575,990 0 56,409 sched
cpumap-kthread 1 576,090 0 54,897 sched
Start Suricata with XDP
~~~~~~~~~~~~~~~~~~~~~~~
You can now start Suricata with XDP bypass activated ::
/usr/bin/suricata -c /etc/suricata/xdp-suricata.yaml --pidfile /var/run/suricata.pid --af-packet=eth3 -vvv
Confirm you have the XDP filter engaged in the output (example)::
...
...
(runmode-af-packet.c:220) (ParseAFPConfig) -- Enabling locked memory for mmap on iface eth3
(runmode-af-packet.c:231) (ParseAFPConfig) -- Enabling tpacket v3 capture on iface eth3
(runmode-af-packet.c:326) (ParseAFPConfig) -- Using queue based cluster mode for AF_PACKET (iface eth3)
(runmode-af-packet.c:424) (ParseAFPConfig) -- af-packet will use '/usr/libexec/suricata/ebpf/xdp_filter.bpf' as XDP filter file
(runmode-af-packet.c:429) (ParseAFPConfig) -- Using bypass kernel functionality for AF_PACKET (iface eth3)
(runmode-af-packet.c:609) (ParseAFPConfig) -- eth3: enabling zero copy mode by using data release call
(util-runmodes.c:296) (RunModeSetLiveCaptureWorkersForDevice) -- Going to use 8 thread(s)
...
...
.. _ebpf-pinned-maps:
Pinned maps usage
-----------------
Pinned maps stay attached to the system if the creating process disappears and
they can also be accessed by external tools. In Suricata bypass case, this can be
used to keep bypassed flow tables active, so Suricata is not hit by previously bypassed flows when
restarting. In the socket filter case, this can be used to maintain a map from tools outside
of Suricata.
To use pinned maps, you first have to mount the `bpf` pseudo filesystem ::
sudo mount -t bpf none /sys/fs/bpf
You can also add to your `/etc/fstab` ::
bpffs /sys/fs/bpf bpf defaults 0 0
and run `sudo mount -a`.
Pinned maps will be accessible as file from the `/sys/fs/bpf` directory. Suricata
will pin them under the name `suricata-$IFACE_NAME-$MAP_NAME`.
To activate pinned maps for a interface, set `pinned-maps` to `true` in the `af-packet`
configuration of this interface ::
- interface: eth3
pinned-maps: true
XDP and pinned-maps
-------------------
This option can be used to expose the maps of a socket filter to other processes.
This allows for example, the external handling of a accept list or block list of
IP addresses. See `bpfctrl (ReceiveErfDagThreadInit) -- Attached and started stream: 0 on DAG: /dev/dag0
[5570] 10/7/2012 -- 13:52:30 - (source-erf-dag.c:288) (ReceiveErfDagThreadInit) -- Starting processing packets from stream: 0 on DAG: /dev/dag0
---
### Doc/Userguide/Capture Hardware/Index
Using Capture Hardware
======================
.. toctree::
af-packet
endace-dag
napatech
myricom
ebpf-xdp
netmap
af-xdp
dpdk
pcap-file
---
### Doc/Userguide/Capture Hardware/Myricom
Myricom
=======
From: https://blog.inliniac.net/2012/07/10/suricata-on-myricom-capture-cards/
In this guide I'll describe using the Myricom libpcap support. I'm going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure ``dmesg`` shows that the card is in sniffer mode:
::
[ 2102.860241] myri_snf INFO: eth4: Link0 is UP
[ 2101.341965] myri_snf INFO: eth5: Link0 is UP
I have installed the Myricom runtime and libraries in ``/opt/snf``
Compile Suricata against Myricom's libpcap:
::
./configure --with-libpcap-includes=/opt/snf/include/ --with-libpcap-libraries=/opt/snf/lib/ --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
sudo make install
Next, configure the amount of ringbuffers. I'm going to work with 8 here, as my quad core + hyper threading has 8 logical CPUs. *See below* for additional information about the buffer-size parameter.
::
pcap:
- interface: eth5
threads: 8
buffer-size: 512kb
checksum-checks: no
The 8 threads setting causes Suricata to create 8 reader threads for eth5. The Myricom driver makes sure each of those is attached to its own ringbuffer.
Then start Suricata as follows:
::
SNF_NUM_RINGS=8 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
If you want 16 ringbuffers, update the "threads" variable in the Suricata configuration file to `16` and start Suricata:
::
SNF_NUM_RINGS=16 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
Note that the ``pcap.buffer-size`` configuration setting shown above is currently ignored when using Myricom cards. The value is passed through to the ``pcap_set_buffer_size`` libpcap API within the Suricata source code. From Myricom support:
::
"The libpcap interface to Sniffer10G ignores the pcap_set_buffer_size() value. The call to snf_open() uses zero as the dataring_size which informs the Sniffer library to use a default value or the value from the SNF_DATARING_SIZE environment variable."
The following pull request opened by Myricom in the libpcap project indicates that a future SNF software release could provide support for setting the SNF_DATARING_SIZE via the pcap.buffer-size yaml setting:
* https://github.com/the-tcpdump-group/libpcap/pull/435
Until then, the data ring and descriptor ring values can be explicitly set using the SNF_DATARING_SIZE and SNF_DESCRING_SIZE environment variables, respectively.
The SNF_DATARING_SIZE is the total amount of memory to be used for storing incoming packet data. This size is shared across all rings.
The SNF_DESCRING_SIZE is the total amount of memory to be used for storing meta information about the packets (packet lengths, offsets, timestamps). This size is also shared across all rings.
Myricom recommends that the descriptor ring be 1/4 the size of the data ring, but the ratio can be modified based on your traffic profile.
If not set explicitly, Myricom uses the following default values: SNF_DATARING_SIZE = 256MB, and SNF_DESCRING_SIZE = 64MB
Expanding on the 16 thread example above, you can start Suricata with a 16GB Data Ring and a 4GB Descriptor Ring using the following command:
::
SNF_NUM_RINGS=16 SNF_DATARING_SIZE=17179869184 SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
Debug Info
~~~~~~~~~~
Myricom also provides a means for obtaining debug information. This can be useful for verifying your configuration and gathering additional information.
Setting SNF_DEBUG_MASK=3 enables debug information, and optionally setting the SNF_DEBUG_FILENAME allows you to specify the location of the output file.
Following through with the example:
::
SNF_NUM_RINGS=16 SNF_DATARING_SIZE=17179869184 SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 SNF_DEBUG_MASK=3 SNF_DEBUG_FILENAME="/tmp/snf.out" suricata -c suricata.yaml -i eth5 --runmode=workers
Additional Info
~~~~~~~~~~~~~~~
* http://www.40gbe.net/index_files/be59da7f2ab5bf0a299ab99ef441bb2e-28.html
* https://www.broadcom.com/support/knowledgebase/1211161394432/how-to-use-emulex-oneconnect-oce12000-d-adapters-with-faststack-
---
### Doc/Userguide/Capture Hardware/Napatech
Napatech
========
Contents
--------
* Introduction
* Package Installation
* Basic Configuration
* Advanced Multithreaded Configuration
Introduction
------------
Napatech packet capture accelerator cards can greatly improve the performance of your Suricata deployment using these
hardware based features:
* On board burst buffering (up to 12GB)
* Zero-copy kernel bypass DMA
* Non-blocking PCIe performance
* Port merging
* Load distribution to up 128 host buffers
* Precise timestamping
* Accurate time synchronization
The package uses a proprietary shell script to handle the installation process.
In either case, gcc, make and the kernel header files are required to compile the kernel module and
install the software.
Package Installation
--------------------
*Note that make, gcc, and the kernel headers are required for installation*
*Root privileges are also required*
The latest driver and tools installation package can be downloaded from: https://www.napatech.com/downloads.
*Note that you will be prompted to install the Napatech libpcap library. Answer "yes" if you would like to
use the Napatech card to capture packets in Wireshark, tcpdump, or another pcap based application.
Libpcap is not needed for Suricata as native Napatech API support is included*
Red Hat Based Distros::
$ yum install kernel-devel-$(uname -r) gcc make
$ ./package_install_3gd.sh
Debian Based Distros::
$ apt-get install linux-headers-$(uname .r) gcc make
$ ./package_install_3gd.sh
To complete installation for all distros ``ntservice``::
$ /opt/napatech3/bin/ntstart.sh -m
Suricata Installation
---------------------
After downloading and extracting the Suricata tarball, you need to run configure to enable Napatech support and
prepare for compilation::
$ ./configure --enable-napatech --with-napatech-includes=/opt/napatech3/include --with-napatech-libraries=/opt/napatech3/lib
$ make
$ make install-full
Suricata configuration
----------------------
Now edit the suricata.yaml file to configure the system. There are three ways
the system can be configured:
1. Auto-config without cpu-affinity: In this mode you specify the stream
configuration in suricata.yaml file and allow the threads to
roam freely. This is good for single processor systems where NUMA node
configuration is not a performance concern.
2. Auto-config with cpu-affinity: In this mode you use the cpu-affinity
of the worker threads to control the creation and configuration of streams.
One stream and one worker thread will be created for each cpu identified in
suricata.yaml. This is best in systems with multiple NUMA nodes (i.e.
multi-processor systems) as the NUMA node of the host buffers is matched
to the core on which the thread is running.
3. Manual-config (legacy): In this mode the underlying Napatech streams are configured
by issuing NTPL commands prior to running Suricata. Suricata then connects
to the existing streams on startup.
Example Configuration - Auto-config without cpu-affinity:
---------------------------------------------------------
If cpu-affinity is not used it is necessary to explicitly define the streams in
the Suricata configuration file. To use this option the following options should
be set in the Suricata configuration file:
1. Turn off cpu-affinity
2. Enable the Napatech "auto-config" option
3. Specify the streams that should be created on startup
4. Specify the ports that will provide traffic to Suricata
5. Specify the hashmode used to distribute traffic to the streams
Below are the options to set::
threading:
set-cpu-affinity: no
.
.
.
napatech:
auto-config: yes
streams: ["0-3"]
ports: [all]
hashmode: hash5tuplesorted
Now modify ``ntservice.ini``. You also need make sure that you have allocated enough
host buffers in ``ntservice.ini`` for the streams. It's a good idea to also set the
``TimeSyncReferencePriority``. To do this make the following changes to ntservice.ini:
HostBuffersRx = [4,16,-1] # [number of host buffers, Size(MB), NUMA node]
TimeSyncReferencePriority = OSTime # Timestamp clock synchronized to the OS
Stop and restart ``ntservice`` after making changes to ntservice::
$ /opt/napatech3/bin/ntstop.sh
$ /opt/napatech3/bin/ntstart.sh
Now you are ready to start Suricata::
$ suricata -c /usr/local/etc/suricata/suricata.yaml --napatech --runmode workers
Example Configuration - Auto-config with cpu-affinity:
------------------------------------------------------
This option will create a single worker-thread and stream for each CPU defined in the
``worker-cpu-set``. To use this option make the following changes to suricata.yaml:
1. Turn on cpu-affinity
2. Specify the worker-cpu-set
3. Enable the Napatech "auto-config" option
4. Specify the ports that will provide traffic to Suricata
5. Specify the hashmode that will be used to control the distribution of
traffic to the different streams/cpus.
When you are done it should look similar to this::
threading:
set-cpu-affinity: yes
cpu-affinity:
management-cpu-set:
cpu: [ 0 ]
receive-cpu-set:
cpu: [ 0 ]
worker-cpu-set:
cpu: [ all ]
.
.
.
napatech:
auto-config: yes
ports: [all]
hashmode: hash5tuplesorted
Prior to running Suricata in this mode you also need to configure a sufficient
number of host buffers on each NUMA node. So, for example, if you have a two
processor server with 32 total cores and you plan to use all of the cores you
will need to allocate 16 host buffers on each NUMA node. It is also desirable
to set the Napatech cards time source to the OS.
To do this make the following changes to ntservice.ini::
TimeSyncReferencePriority = OSTime # Timestamp clock synchronized to the OS
HostBuffersRx = [16,16,0],[16,16,1] # [number of host buffers, Size(MB), NUMA node]
Stop and restart ``ntservice`` after making changes to ntservice::
$ /opt/napatech3/bin/ntstop.sh -m
$ /opt/napatech3/bin/ntstart.sh -m
Now you are ready to start Suricata::
$ suricata -c /usr/local/etc/suricata/suricata.yaml --napatech --runmode workers
Example Configuration - Manual Configuration
--------------------------------------------
For Manual Configuration the Napatech streams are created by running NTPL
commands prior to running Suricata.
Note that this option is provided primarily for legacy configurations as previously
this was the only way to configure Napatech products. Newer capabilities such as
flow-awareness and inline processing cannot be configured manually.
In this example we will setup the Napatech capture accelerator to merge all physical
ports, and then distribute the merged traffic to four streams that Suricata will ingest.
The steps for this configuration are:
1. Disable the Napatech auto-config option in suricata.yaml
2. Specify the streams that Suricata is to use in suricata.yaml
3. Create a file with NTPL commands to create the underlying Napatech streams.
First suricata.yaml should be configured similar to the following::
napatech:
auto-config: no
streams: ["0-3"]
Next you need to make sure you have enough host buffers defined in ntservice.ini. As
it's also a good idea to set up the TimeSync. Here are the lines to change::
TimeSyncReferencePriority = OSTime # Timestamp clock synchronized to the OS
HostBuffersRx = [4,16,-1] # [number of host buffers, Size(MB), NUMA node]
Stop and restart ntservice after making changes to ntservice::
$ /opt/napatech3/bin/ntstop.sh
$ /opt/napatech3/bin/ntstart.sh
Now that ntservice is running we need to execute a few NTPL (Napatech Programming Language)
commands to complete the setup. Create a file will the following commands::
Delete=All # Delete any existing filters
Assign[streamid=(0..3)]= all # Assign all physical ports to stream ID 0
Next execute those command using the ``ntpl`` tool::
$ /opt/napatech3/bin/ntpl -f
Now you are ready to start Suricata::
$ suricata -c /usr/local/etc/suricata/suricata.yaml --napatech --runmode workers
It is possible to specify much more elaborate configurations using this option. Simply by
creating the appropriate NTPL file and attaching Suricata to the streams.
Bypassing Flows
---------------
On flow-aware Napatech products, traffic from individual flows can be automatically
dropped or, in the case of inline configurations, forwarded by the hardware after
an inspection of the initial packet(s) of the flow by Suricata. This will save
CPU cycles since Suricata does not process packets for a flow that has already been
adjudicated. This is enabled via the hardware-bypass option in the Napatech section
of the configuration file.
When hardware bypass is used it is important that the ports accepting upstream
and downstream traffic from the network are configured with information on
which port the two sides of the connection will arrive. This is needed for the
hardware to properly process traffic in both directions. This is indicated in the
"ports" section as a hyphen separated list of port-pairs that will be receiving
upstream and downstream traffic E.g.::
napatech:
hardware-bypass: true
ports[0-1,2-3]
Note that these "port-pairings" are also required for IDS configurations as the hardware
needs to know on which port(s) two sides of the connection will arrive.
For configurations relying on optical taps the two sides of the pairing will typically
be different ports. For SPAN port configurations where both upstream and downstream traffic
are delivered to a single port both sides of the "port-pair" will reference the same port.
For example tap configurations have a form similar to this::
ports[0-1,2-3]
Whereas SPAN port configurations it would look similar to this::
ports[0-0,1-1,2-2,3-3]
Note that SPAN and tap configurations may be combined on the same adapter.
There are multiple ways that Suricata can be configured to bypass traffic.
One way is to enable stream.bypass in the configuration file. E.g.::
stream:
bypass: true
When enabled once Suricata has evaluated the first chunk of the stream (the
size of which is also configurable) it will indicate that the rest of the
packets in the flow can be bypassed. In IDS mode this means that the subsequent
packets of the flow will be dropped and not delivered to Suricata. In inline
operation the packets will be transmitted on the output port but not delivered
to Suricata.
Another way is by specifying the "bypass" keyword in a rule. When a rule is
triggered with this keyword then the "pass" or "drop" action will be applied
to subsequent packets of the flow automatically without further analysis by
Suricata. For example given the rule::
drop tcp any 443 <> any any (msg: "SURICATA Test rule"; bypass; sid:1000001; rev:2;)
Once Suricata initially evaluates the fist packet(s) and identifies the flow,
all subsequent packets from the flow will be dropped by the hardware; thus
saving CPU cycles for more important tasks.
The timeout value for how long to wait before evicting stale flows from the
hardware flow table can be specified via the FlowTimeout attribute in ntservice.ini.
Inline Operation
----------------
Napatech flow-aware products can be configured for inline operation. This is
specified in the configuration file. When enabled, ports are specified as
port-pairs. With traffic received from one port it is transmitted out the
the peer port after inspection by Suricata. E.g. the configuration::
napatech:
inline: enabled
ports[0-1, 2-3]
Will pair ports 0 and 1; and 2 and 3 as peers. Rules can be defined to
pass traffic matching a given signature. For example, given the rule::
pass tcp any 443 <> any any (msg: "SURICATA Test rule"; bypass; sid:1000001; rev:2;)
Suricata will evaluate the initial packet(s) of the flow and program the flow
into the hardware. Subsequent packets from the flow will be automatically be
shunted from one port to it's peer.
Counters
--------
The following counters are available:
- napa_total.pkts - The total of packets received by the card.
- napa_total.byte - The total count of bytes received by the card.
- napa_total.overflow_drop_pkts - The number of packets that were dropped because
the host buffers were full. (I.e. the application is not able to process
packets quickly enough.)
- napa_total.overflow_drop_byte - The number of bytes that were dropped because
the host buffers were full. (I.e. the application is not able to process
packets quickly enough.)
On flow-aware products the following counters are also available:
- napa_dispatch_host.pkts, napa_dispatch_host.byte:
The total number of packets/bytes that were dispatched to a host buffer for
processing by Suricata. (Note: this count includes packets that may be
subsequently dropped if there is no room in the host buffer.)
- napa_dispatch_drop.pkts, napa_dispatch_drop.byte:
The total number of packets/bytes that were dropped at the hardware as
a result of a Suricata "drop" bypass rule or other adjudication by
Suricata that the flow packets should be dropped. These packets are not
delivered to the application.
- napa_dispatch_fwd.pkts, napa_dispatch_fwd.byte:
When inline operation is configured this is the total number of packets/bytes
that were forwarded as result of a Suricata "pass" bypass rule or as a result
of stream or encryption bypass being enabled in the configuration file.
These packets were not delivered to the application.
- napa_bypass.active_flows:
The number of flows actively programmed on the hardware to be forwarded or dropped.
- napa_bypass.total_flows:
The total count of flows programmed since the application started.
If enable-stream-stats is enabled in the configuration file then, for each stream
that is being processed, the following counters will be output in stats.log:
- napa.pkts: The number of packets received by the stream.
- napa.bytes: The total bytes received by the stream.
- napa.drop_pkts: The number of packets dropped from this stream due to buffer overflow conditions.
- napa.drop_byte: The number of bytes dropped from this stream due to buffer overflow conditions.
This is useful for fine-grain debugging to determine if a specific CPU core or
thread is falling behind resulting in dropped packets.
Debugging:
For debugging configurations it is useful to see what traffic is flowing as well as what streams are
created and receiving traffic. There are two tools in /opt/napatech3/bin that are useful for this:
- monitoring: this tool will, among other things, show what traffic is arriving at the port interfaces.
- profiling: this will show host-buffers, streams and traffic flow to the streams.
If Suricata terminates abnormally stream definitions, which are normally removed at shutdown, may remain in effect.
If this happens they can be cleared by issuing the "delete=all" NTPL command as follows::
# /opt/napatech3/bin/ntpl -e "delete=all"
Napatech configuration options:
-------------------------------
These are the Napatech options available in the Suricata configuration file::
napatech:
# When use_all_streams is set to "yes" the initialization code will query
# the Napatech service for all configured streams and listen on all of them.
# When set to "no" the streams config array will be used.
#
# This option necessitates running the appropriate NTPL commands to create
# the desired streams prior to running Suricata.
#use-all-streams: no
# The streams to listen on when auto-config is disabled or when threading
# cpu-affinity is disabled. This can be either:
# an individual stream (e.g. streams: [0])
# or
# a range of streams (e.g. streams: ["0-3"])
#
streams: ["0-3"]
# Stream stats can be enabled to provide fine grain packet and byte counters
# for each thread/stream that is configured.
#
enable-stream-stats: no
# When auto-config is enabled the streams will be created and assigned
# automatically to the NUMA node where the thread resides. If cpu-affinity
# is enabled in the threading section, then the streams will be created
# according to the number of worker threads specified in the worker cpu set.
# Otherwise, the streams array is used to define the streams.
#
# This option cannot be used simultaneous with "use-all-streams".
#
auto-config: yes
# Enable hardware level flow bypass.
#
hardware-bypass: yes
# Enable inline operation. When enabled traffic arriving on a given port is
# automatically forwarded out it's peer port after analysis by Suricata.
# hardware-bypass must be enabled when this is enabled.
#
inline: no
# Ports indicates which napatech ports are to be used in auto-config mode.
# these are the port ID's of the ports that will be merged prior to the
# traffic being distributed to the streams.
#
# When hardware-bypass is enabled the ports must be configured as a segment
# specify the port(s) on which upstream and downstream traffic will arrive.
# This information is necessary for the hardware to properly process flows.
#
# When using a tap configuration one of the ports will receive inbound traffic
# for the network and the other will receive outbound traffic. The two ports on a
# given segment must reside on the same network adapter.
#
# When using a SPAN-port configuration the upstream and downstream traffic
# arrives on a single port. This is configured by setting the two sides of the
# segment to reference the same port. (e.g. 0-0 to configure a SPAN port on
# port 0).
#
# port segments are specified in the form:
# ports: [0-1,2-3,4-5,6-6,7-7]
#
# For legacy systems when hardware-bypass is disabled this can be specified in any
# of the following ways:
#
# a list of individual ports (e.g. ports: [0,1,2,3])
#
# a range of ports (e.g. ports: [0-3])
#
# "all" to indicate that all ports are to be merged together
# (e.g. ports: [all])
#
# This parameter has no effect if auto-config is disabled.
#
ports: [0-1,2-3]
# When auto-config is enabled the hashmode specifies the algorithm for
# determining to which stream a given packet is to be delivered.
# This can be any valid Napatech NTPL hashmode command.
#
# The most common hashmode commands are: hash2tuple, hash2tuplesorted,
# hash5tuple, hash5tuplesorted and roundrobin.
#
# See Napatech NTPL documentation other hashmodes and details on their use.
#
# This parameter has no effect if auto-config is disabled.
#
hashmode: hash5tuplesorted
Make sure that there are enough host-buffers declared in ``ntservice.ini`` to
accommodate the number of cores/streams being used.
Support
-------
Contact a support engineer at: ntsupport@napatech.com
Napatech Documentation can be found at: https://docs.napatech.com (Click the search icon, with no search text,
to see all documents in the portal.)
---
### Doc/Userguide/Capture Hardware/Netmap
Netmap
======
Netmap is a high speed capture framework for Linux and FreeBSD. In Linux it
is available as an external module, while in FreeBSD 11+ it is available by
default.
Compiling Suricata
------------------
FreeBSD
~~~~~~~
On FreeBSD 11 and up, NETMAP is included and enabled by default in the kernel.
To build Suricata with NETMAP, add ``--enable-netmap`` to the configure line.
The location of the NETMAP includes (/usr/src/sys/net/) does not have to be
specified.
Linux
~~~~~
On Linux, NETMAP is not included by default. It can be pulled from github.
Follow the instructions on installation included in the NETMAP repository.
When NETMAP is installed, add ``--enable-netmap`` to the configure line.
If the includes are not added to a standard location, the location can
be specified when configuring Suricata.
Example::
./configure --enable-netmap --with-netmap-includes=/usr/local/include/netmap/
Starting Suricata
-----------------
When opening an interface, netmap can take various special characters as
options in the interface string.
.. warning:: the interface that netmap reads from will become unavailable
for normal network operations. You can lock yourself out of
your system.
IDS
~~~
Suricata can be started in 2 ways to use netmap:
::
suricata --netmap=
suricata --netmap=igb0
In the above example Suricata will start reading from the `igb0` network interface.
The number of threads created depends on the number of RSS queues available on the NIC.
::
suricata --netmap
In the above example Suricata will take the ``netmap`` block from the Suricata
configuration and open each of the interfaces listed.
::
netmap:
- interface: igb0
threads: 2
- interface: igb1
threads: 4
For the above configuration, both ``igb0`` and ``igb1`` would be opened. With 2
threads for ``igb0`` and 4 capture threads for ``igb1``.
.. warning:: This multi threaded setup only works correctly if the NIC
has symmetric RSS hashing. If this is not the case, consider
using the 'lb' method below.
IPS
~~~
Suricata's Netmap based IPS mode is based on the concept of creating
a layer 2 software bridge between 2 interfaces. Suricata reads packets on
one interface and transmits them on another.
Packets that are blocked by the IPS policy, are simply not transmitted.
::
netmap:
- interface: igb0
copy-mode: ips
copy-iface: igb1
- interface: igb1
copy-mode: ips
copy-iface: igb0
Advanced setups
---------------
lb (load balance)
-----------------
"lb" is a tool written by Seth Hall to allow for load balancing for single
or multiple tools. One common use case is being able to run Suricata and
Zeek together on the same traffic.
starting lb::
lb -i eth0 -p suricata:6 -p zeek:6
.. note:: On FreeBSD 11, the named prefix doesn't work.
yaml::
netmap:
- interface: netmap:suricata
threads: 6
startup::
suricata --netmap=netmap:suricata
The interface name as passed to Suricata includes a 'netmap:' prefix. This
tells Suricata that it's going to read from netmap pipes instead of a real
interface.
Then Zeek (formerly Bro) can be configured to load 6 instances. Both will
get a copy of the same traffic. The number of netmap pipes does not have
to be equal for both tools.
FreeBSD 11
~~~~~~~~~~
On FreeBSD 11 the named pipe is not available.
starting lb::
lb -i eth0 -p 6
yaml::
netmap:
- interface: netmap:eth0
threads: 6
startup::
suricata --netmap
.. note:: "lb" is bundled with netmap.
Single NIC
~~~~~~~~~~
When an interface enters NETMAP mode, it is no longer available to
the OS for other operations. This can be undesirable in certain
cases, but there is a workaround.
By running Suricata in a special inline mode, the interface will
show it's traffic to the OS.
::
netmap:
- interface: igb0
copy-mode: tap
copy-iface: igb0^
- interface: igb0^
copy-mode: tap
copy-iface: igb0
The copy-mode can be both 'tap' and 'ips', where the former never
drops packets based on the policies in use, and the latter may drop
packets.
.. warning:: Misconfiguration can lead to connectivity loss. Use
with care.
.. note:: This set up can also be used to mix NETMAP with firewall
setups like pf or ipfw.
VALE switches
~~~~~~~~~~~~~
VALE is a virtual switch that can be used to create an all virtual
network or a mix of virtual and real nics.
A simple all virtual setup::
vale-ctl -n vi0
vale-ctl -a vale0:vi0
vale-ctl -n vi1
vale-ctl -a vale0:vi1
We now have a virtual switch "vale0" with 2 ports "vi0" and "vi1".
We can start Suricata to listen on one of the ports::
suricata --netmap=vale0:vi1
Then we can
Inline IDS
----------
The inline IDS is almost the same as the IPS setup above, but it will not
enforce ``drop`` policies.
::
netmap:
- interface: igb0
copy-mode: tap
copy-iface: igb1
- interface: igb1
copy-mode: tap
copy-iface: igb0
The only difference with the IPS mode is that the ``copy-mode`` setting is
set to ``tap``.
---
### Doc/Userguide/Capture Hardware/Pcap File
.. _pcap_file:
PCAP File Reading
=================
Suricata offers a ``pcap-file`` capture method to process PCAP files and
directories of PCAP files in an offline or live-feed manner.
Configuration
-------------
.. code-block:: yaml
pcap-file:
checksum-checks: auto
# buffer-size: 128 KiB
# tenant-id: none
# Applies to file and directory. Options: false (no deletion), true (always delete),
# "non-alerts" (delete only files with no alerts)
# delete-when-done: false
# recursive: false
# continuous: false
# delay: 30
# poll-interval: 5
Buffer Size
-----------
This option specifies the size of the read buffer for the PCAP file.
The larger the buffer, the more data Suricata can read at once.
This can improve performance, especially for large files.
The size can be specified through the command line option, see
:ref:`--pcap-file-buffer-size `
Setting ``buffer-size`` to ``0`` disables ``setvbuf`` buffering. This is the
explicit opt-out for non-seekable sources such as ``/dev/stdin`` or named
pipes, where buffering the underlying file descriptor is not supported.
Directory-related options
-------------------------
The **recursive** option enables Suricata to traverse subdirectories within
the specified directory, up to a maximum depth of 255. This allows for
processing of PCAP files located in nested folders. Note that the recursive
option cannot be used together with the ``continuous`` option.
The command-line option is
:ref:`--pcap-file-recursive `.
The **continuous** option allows Suricata to monitor the specified directory
for new files, processing them as they appear.
This is useful for live environments where new PCAP files are continuously
added. The continuous option cannot be combined with the ``recursive`` option.
The command-line option is
:ref:`--pcap-file-continuous `..
The **delay** option specifies the amount of time, in seconds,
that Suricata waits before processing newly detected files.
This helps prevent the processing of incomplete files that are still
being written. The delay option is applicable with
the ``continuous`` mode.
The **poll-interval** option determines how frequently, in seconds,
Suricata checks the directory for new files. Adjusting this interval
can help balance responsiveness and resource usage.
.. note::
``continuous`` and ``recursive`` cannot be enabled simultaneously.
.. note::
Symlinks are ignored during recursive traversal.
Other options
-------------
**checksum-checks**
- **auto** (default): Suricata detects checksum offloading statistically.
- **yes**: Forces checksum validation.
- **no**: Disables checksum validation.
- The command-line option is :ref:`-k `
**tenant-id**
- Specifies the tenant for multi-tenant setups with direct select.
- The PCAP is processed by the detection engine assigned to the specified
tenant.
**delete-when-done**
Controls when PCAP files are deleted after processing. Three values are supported:
- ``false`` (default): Files are never deleted
- ``true``: Files are always deleted after processing
- ``"non-alerts"``: Files are deleted only if they didn't generate any alerts
.. note::
The command-line option :ref:`--pcap-file-delete `
overrides this configuration and forces "always delete" mode (``true``).
.. warning::
When using ``"non-alerts"`` mode, file deletion is deferred until thread
cleanup to ensure alert counts are finalized. This may delay deletion
compared to other modes.
**BPF filter**
- Suricata supports BPF filters for packet capture that is also applicable
to the ``pcap-file`` capture method.
- The BPF filter is specified in the file with the :ref:`-F `
command-line option.
---
### Doc/Userguide/Configuration/Dropping Privileges
Dropping Privileges After Startup
=================================
Currently, ``libcap-ng`` is needed for dropping privileges on Suricata
after startup. For libcap, see status of feature request number #276
-- Libcap support for dropping privileges.
Most distributions have ``libcap-ng`` in their repositories.
To download the current version of libcap-ng from upstream, see also
http://people.redhat.com/sgrubb/libcap-ng/ChangeLog
::
wget http://people.redhat.com/sgrubb/libcap-ng/libcap-ng-0.7.8.tar.gz
tar -xzvf libcap-ng-0.7.8.tar.gz
cd libcap-ng-0.7.8
./configure
make
make install
Download, configure, compile and install Suricata for your particular setup.
See :doc:`../install`. Depending on your environment, you may need to add the
--with-libpcap_ng-libraries and --with-libpcap_ng-includes options
during the configure step. e.g:
::
./configure --with-libcap_ng-libraries=/usr/local/lib \
--with-libcap_ng-includes=/usr/local/include
Now, when you run Suricata, tell it what user and/or group you want it
to run as after startup with the --user and --group options.
e.g. (this assumes a 'suri' user and group):
::
suricata -D -i eth0 --user=suri --group=suri
You will also want to make sure your user/group permissions are set so
Suricata can still write to its log files which are usually located in
/var/log/suricata.
::
mkdir -p /var/log/suricata
chown -R root:suri /var/log/suricata
chmod -R 775 /var/log/suricata
---
### Doc/Userguide/Configuration/Exception Policies
.. _exception policies:
Exception Policies
##################
Suricata has a set of configuration variables to indicate what should the engine
do when certain exception conditions, such as hitting a memcap, are reached.
They are called Exception Policies and are configurable via suricata.yaml. If
enabled, the engine will call them when it reaches exception states. Stats for
any applied exception policies can be found in counters related to the specific
configuration setting (:ref:`read more`). Some configuration is
available directly via the :ref:`stats settings`.
For developers or for researching purposes, there are also simulation options
exposed in ``qa-simulation`` mode and passed via command-line. These exist to force or
simulate failures or errors and understand Suricata behavior under such conditions.
See :any:`command-line-exception-policies` for those.
.. _master-switch:
Master Switch
*************
It is possible to set all configuration policies via what we call "master
switch". This offers a quick way to define what the engine should do in case of
traffic exceptions, while still allowing for the flexibility of indicating a
different behavior for specific exception policies your setup/environment may
have the need to.
::
# Define a common behavior for all exception policies.
# In IPS mode, the default is drop-flow. For cases when that's not possible, the
# engine will fall to drop-packet. To fallback to old behavior (setting each of
# them individually, or ignoring all), set this to ignore.
# All values available for exception policies can be used, and there is one
# extra option: auto - which means drop-flow or drop-packet (as explained above)
# in IPS mode, and ignore in IDS mode. Exception policy values are: drop-packet,
# drop-flow, reject, bypass, pass-packet, pass-flow, ignore (disable).
exception-policy: auto
This value will be overwritten by specific exception policies whose settings are
also defined in the yaml file.
Auto
====
**In IPS mode**, the default behavior for most of the exception policies is to
fail close. This means dropping the flow, or the packet, when the flow action is
not supported. The default policy for the midstream exception will be ignore if
midstream flows are accepted.
It is possible to disable this default, by setting the exception policies'
"master switch" yaml config option to ``ignore``.
**In IDS mode**, setting ``auto`` mode actually means disabling the
``master-switch``, or ignoring the exception policies.
.. note::
If no exception policy is enabled, Suricata will not log exception policy stats.
.. _eps_settings:
Specific settings
*****************
Exception policies are implemented for:
.. list-table:: Exception Policy configuration variables
:widths: 18, 18, 18, 44
:header-rows: 1
* - Config setting
- Policy variable
- Affects
- Expected behavior
* - stream.memcap
- memcap-policy
- Flow or packet
- If a stream memcap limit is reached, apply the memcap policy to the packet and/or
flow.
* - stream.midstream
- midstream-policy
- Flow
- If a session is picked up midstream, apply the midstream policy to the flow.
* - stream.reassembly.memcap
- memcap-policy
- Flow or packet
- If stream reassembly reaches memcap limit, apply memcap policy to the
packet and/or flow.
* - flow.memcap
- memcap-policy
- Packet
- Apply policy when the memcap limit for flows is reached and no flow could
be freed up. **Policy can only be applied to the packet.**
* - defrag.memcap
- memcap-policy
- Packet
- Apply policy when the memcap limit for defrag is reached and no tracker
could be picked up. **Policy can only be applied to the packet.**
* - app-layer
- error-policy
- Flow or packet
- Apply policy if a parser reaches an error state. Policy can be applied to packet and/or flow.
To change any of these, go to the specific section in the suricata.yaml file
(for more configuration details, check the :doc:`suricata.yaml's`
documentation).
The possible values for the exception policies, and the resulting behaviors,
are:
- ``drop-flow``: disable decoding and parsing for the whole flow (packets, payload,
application layer protocol), drop the packet and all future packets in the
flow.
- ``drop-packet``: drop the packet.
- ``reject``: same as ``drop-flow``, but reject the current packet as well (see
``reject`` action in Rule's :ref:`actions`).
- ``rejectboth``: same as ``reject``, except that it sends reject packets to both the
source and destination addresses.
- ``bypass``: bypass the flow. No further decoding or parsing is done. :ref:`Bypass
` may be offloaded.
- ``pass-flow``: disable payload and packet detection; stream reassembly,
app-layer parsing and logging still happen.
- ``pass-packet``: disable detection, still does stream updates and app-layer
parsing (depending on which policy triggered it).
- ``ignore``: do not apply exception policies (default behavior).
The *drop*, *pass* and *reject* are similar to the rule actions described in :ref:`rule
actions`.
Exception Policies and Midstream Pick-up Sessions
*************************************************
Suricata behavior can be difficult to track in case of midstream session
pick-ups. Consider this matrix illustrating the different interactions for
midstream pick-ups enabled or not and the various exception policy values:
.. list-table:: **Exception Policy Behaviors - IDS Mode**
:widths: auto
:header-rows: 1
:stub-columns: 1
* - Exception Policy
- Midstream pick-up sessions ENABLED (stream.midstream=true)
- Midstream pick-up sessions DISABLED (stream.midstream=false)
* - Ignore
- Session and app-layer traffic tracked and parsed, log app-layer traffic, **do** detection.
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
* - Drop-flow
- Not valid.*
- Not valid.*
* - Drop-packet
- Not valid.*
- Not valid.*
* - Reject / Rejectboth
- Not valid.*
- Session not tracked, flow REJECTED.
* - Pass-flow
- Session and app-layer traffic tracked and parsed, log app-layer traffic, **no** detection.
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
* - Pass-packet
- Not valid.*
- Not valid.*
* - Bypass
- Not valid.*
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
* - Auto
- Midstream policy applied: "ignore". Same behavior.
- Midstream policy applied: "ignore". Same behavior.
The main difference between IDS and IPS scenarios is that in IPS mode flows can
be allowed or blocked (as in with the PASS and DROP rule actions). Packet
actions are not valid, as midstream pick-up is a configuration that affects the
whole flow.
.. list-table:: **Exception Policy Behaviors - IPS Mode**
:widths: 15 42 43
:header-rows: 1
:stub-columns: 1
* - Exception Policy
- Midstream pick-up sessions ENABLED (stream.midstream=true)
- Midstream pick-up sessions DISABLED (stream.midstream=false)
* - Ignore
- Session and app-layer traffic tracked and parsed, log app-layer traffic, **do** detection.
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
* - Drop-flow
- Not valid.*
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
Flow DROPPED.
* - Drop-packet
- Not valid.*
- Not valid.*
* - Reject / Rejectboth
- Not valid.*
- Session not tracked, flow DROPPED and REJECTED.
* - Pass-flow
- Track session, parse and log app-layer traffic, **no** detection.
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
* - Pass-packet
- Not valid.*
- Not valid.*
* - Bypass
- Not valid.*
- Session not tracked. No app-layer parsing or logging. No stream reassembly. No detection.
Packets ALLOWED.
* - Auto
- Midstream policy applied: "ignore". Same behavior.
- Midstream policy applied: "drop-flow". Same behavior.
Notes:
* Not valid means that Suricata will error out and won't start.
* ``REJECT`` will make Suricata send a Reset-packet unreach error to the sender
of the matching packet.
Exception Policies Interactions
===============================
In certain scenarios, one exception policy being applied might trigger another
exception. The midstream session policy seems the most prone to be triggered,
by its definition.
If a setting can be applied to packets in **IPS mode** with a drop effect, this
can result in following packets for the affected flow(s) being considered as a
midstream session, thus triggering the related policy.
In particular, these interactions seem probable for the following policies,
**if set to drop-packet**:
* Stream Memcap
* Stream Reassembly Memcap
* Flow Memcap
* Defrag Memcap
* Application Layer Protocol Error
.. _eps_output:
Log Output
**********
.. _eps_flow_event:
Flow Event
==========
When an Exception Policy is triggered, this will be indicated in the flow log
event for the associated flow, also indicating which target triggered that, and
what policy was applied. If no exception policy is triggered, that field won't
be present in the logs.
Note that this is true even if the policy is applied only to certain packets from
a flow.
In the log sample below, the flow triggered the ``midstream policy``, leading
to Suricata applying the behavior that had been configured for such scenario:
*to pass the flow* (``pass_flow``). It also did trigger the ``app_layer_error``
exception policy, but that is set up to ``ignore``::
"flow": {
"pkts_toserver": 4,
"pkts_toclient": 5,
"bytes_toserver": 495,
"bytes_toclient": 351,
"start": "2016-07-13T22:42:07.199672+0000",
"end": "2016-07-13T22:42:07.573174+0000",
"age": 0,
"state": "new",
"reason": "shutdown",
"alerted": false,
"action": "pass",
"exception_policy": [
{
"target": "stream_midstream",
"policy": "pass_flow"
},
{
"target": "app_layer_error",
"policy": "ignore"
}
]
}
.. _eps_stats:
Available Stats
===============
There are stats counters for each supported exception policy scenario that will
be logged when exception policies are enabled:
.. list-table:: **Exception Policy Stats Counters**
:widths: 50 50
:header-rows: 1
:stub-columns: 1
* - Setting
- Counters
* - stream.memcap
- exception_policy.tcp.ssn_memcap
* - stream.reassembly.memcap
- exception_policy.tcp.reassembly.memcap
* - stream.midstream
- exception_policy.tcp.midstream
* - defrag.memcap
- exception_policy.defrag.memcap
* - flow.memcap
- exception_policy.flow.memcap
* - app-layer.error
- * exception_policy.app_layer.error
* app_layer.error.exception_policy
If a given exception policy does not apply for a setting, no related counter
is logged.
Stats for application layer errors are available in summarized form or per
application layer protocol. As the latter is extremely verbose, by default
Suricata logs only the summary. If any further investigation is needed, it
is recommended to enable per-app-proto exception policy error counters
temporarily (for more, read :ref:`stats configuration`).
.. _command-line-exception-policies:
Command-line Options for Simulating Exceptions
**********************************************
It is also possible to force specific exception scenarios, to check engine
behavior under failure or error conditions.
To enable these, you must build Suricata with ``qa-simulation`` enabled
(``--enable-qa-simulation``).
The available command-line options are:
- ``simulate-applayer-error-at-offset-ts``: force an applayer error in the to
server direction at the given offset.
- ``simulate-applayer-error-at-offset-tc``: force an applayer error in the to
client direction at the given offset.
- ``simulate-packet-loss``: simulate that the packet with the given number
(``pcap_cnt``) from the session was lost.
- ``simulate-packet-tcp-reassembly-memcap``: simulate that the TCP stream
reassembly reached memcap for the specified packet.
- ``simulate-packet-tcp-ssn-memcap``: simulate that the TCP session hit the
memcap for the specified packet.
- ``simulate-packet-flow-memcap``: force the engine to assume that flow memcap is
hit at the given packet.
- ``simulate-packet-defrag-memcap``: force Suricata to assume memcap is hit when
defragmenting specified packet.
- ``simulate-alert-queue-realloc-failure``: prevent the engine from dynamically
growing the temporary alert queue, during alerts processing.
Glossary
********
- **decoding**: traffic parsing on the packet level;
- **[app-layer] parsing**: traffic is parsed on the application layer level for
events, anomalies and logging;
- **detection**: evaluate traffic against loaded rules to generate alerts and/ or
block or allow traffic.
Common abbreviations
====================
- applayer/ app-layer: application layer protocol
- memcap: (maximum) memory capacity available
- defrag: defragmentation
---
### Doc/Userguide/Configuration/Global Thresholds
Global-Thresholds
=================
Thresholds can be configured in the rules themselves, see
:doc:`../rules/thresholding`. They are often set by rule writers based on
their intelligence for creating a rule combined with a judgement on how often
a rule will alert.
Thresholds are tracked in a hash table that is sized according to configuration, see:
:ref:`suricata-yaml-thresholds`.
Threshold Config
----------------
Next to rule thresholding more thresholding can be configured on the sensor
using the threshold.config.
threshold/event_filter
~~~~~~~~~~~~~~~~~~~~~~
Syntax:
::
threshold gen_id , sig_id , type , \
track , count , seconds
rate_filter
~~~~~~~~~~~
Rate filters allow changing of a rule action when a rule matches.
Syntax::
rate_filter gen_id , sig_id , track , \
count , seconds , new_action , timeout
Example::
rate_filter gen_id 1, sig_id 1000, track by_rule, count 100, seconds 60, \
new_action alert, timeout 30
gen_id
^^^^^^
Generator id. Normally 1, but if a rule uses the ``gid`` keyword to set
another value it has to be matched in the ``gen_id``.
sig_id
^^^^^^
Rule/signature id as set by the rule ``sid`` keyword.
track
^^^^^
Where to track the rule matches. When using by_src/by_dst the tracking is
done per IP-address. The Host table is used for storage. When using by_rule
it's done globally for the rule.
Option by_both used to track per IP pair of source and destination. Packets
going to opposite directions between same addresses tracked as the same pair.
The by_flow option tracks the rule matches in the flow.
count
^^^^^
Number of rule hits before the ``rate_filter`` is activated.
seconds
^^^^^^^
Time period within which the ``count`` needs to be reached to activate
the ``rate_filter``
new_action
^^^^^^^^^^
New action that is applied to matching traffic when the ``rate_filter``
is in place.
Values::
Note: 'sdrop' and 'log' are supported by the parser but not implemented otherwise.
timeout
^^^^^^^
Time in seconds during which the ``rate_filter`` will remain active.
Example
^^^^^^^
Let's say we want to limit incoming connections to our SSH server. The rule
``888`` below simply alerts on SYN packets to the SSH port of our SSH server.
If an IP-address triggers this more than 10 or more with a minute, the
drop ``rate_filter`` is set with a timeout of 5 minutes.
Rule::
alert tcp any any -> $MY_SSH_SERVER 22 (msg:"Connection to SSH server"; \
flow:to_server; flags:S,12; sid:888;)
Rate filter::
rate_filter gen_id 1, sig_id 888, track by_src, count 10, seconds 60, \
new_action drop, timeout 300
suppress
~~~~~~~~
Suppressions can be used to suppress alerts for a rule or a
host/network. Actions performed when a rule matches, such as setting a
flowbit, are still performed.
Syntax:
::
suppress gen_id , sig_id
suppress gen_id , sig_id , track , ip
Examples:
::
suppress gen_id 1, sig_id 2002087, track by_src, ip 209.132.180.67
This will make sure the signature 2002087 will never match for src
host 209.132.180.67.
Other possibilities/examples::
suppress gen_id 1, sig_id 2003614, track by_src, ip 217.110.97.128/25
suppress gen_id 1, sig_id 2003614, track by_src, ip [192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]
suppress gen_id 1, sig_id 2003614, track by_src, ip $HOME_NET
suppress gen_id 1, sig_id 2003614, track by_either, ip 217.110.97.128/25
In the last example above, the ``by_either`` tracking means that if either
the ``source ip`` or ``destination ip`` matches ``217.110.97.128/25`` the
rule with sid 2003614 is suppressed.
.. _global-thresholds-vs-rule-thresholds:
Global thresholds vs rule thresholds
------------------------------------
**Note: this section applies to 1.4+ In 1.3 and before mixing rule and
global thresholds is not supported.**
When a rule has a threshold/detection_filter set a rule can still be
affected by the global threshold file.
The rule below will only fire if 10 or more emails are being
delivered/sent from a host within 60 seconds.
::
alert tcp any any -> any 25 (msg:"ET POLICY Inbound Frequent Emails - Possible Spambot Inbound"; \
flow:established; content:"mail from|3a|"; nocase; \
threshold: type threshold, track by_src, count 10, seconds 60; \
reference:url,doc.emergingthreats.net/2002087; classtype:misc-activity; sid:2002087; rev:10;)
Next, we'll see how global settings affect this rule.
Suppress
~~~~~~~~
Suppressions can be combined with rules with
thresholds/detection_filters with no exceptions.
::
suppress gen_id 1, sig_id 2002087, track by_src, ip 209.132.180.67
suppress gen_id 0, sig_id 0, track by_src, ip 209.132.180.67
suppress gen_id 1, sig_id 0, track by_src, ip 209.132.180.67
Each of the rules above will make sure 2002087 doesn't alert when the
source of the emails is 209.132.180.67. It **will** alert for all other
hosts.
::
suppress gen_id 1, sig_id 2002087
This suppression will simply convert the rule to "noalert", meaning it
will never alert in any case. If the rule sets a flowbit, that will
still happen.
Threshold/event_filter
~~~~~~~~~~~~~~~~~~~~~~
When applied to a specific signature, thresholds and event_filters
(threshold from now on) will override the signature setting. This can
be useful for when the default in a signature doesn't suit your
environment.
::
threshold gen_id 1, sig_id 2002087, type both, track by_src, count 3, seconds 5
threshold gen_id 1, sig_id 2002087, type threshold, track by_src, count 10, seconds 60
threshold gen_id 1, sig_id 2002087, type limit, track by_src, count 1, seconds 15
Each of these will replace the threshold setting for 2002087 by the
new threshold setting.
**Note:** overriding all gids or sids (by using gen_id 0 or sig_id 0)
is not supported. Bug https://redmine.openinfosecfoundation.org/issues/425.
Rate_filter
~~~~~~~~~~~
see https://redmine.openinfosecfoundation.org/issues/425.
---
### Doc/Userguide/Configuration/Includes
.. _includes:
Includes
========
A Suricata configuration file (typically
``/etc/suricata/suricata.yaml``) may include other files allowing a
configuration file to be broken into multiple files. The *special*
field name ``include`` is used to include one or more files.
The contents of the *include* file are inlined at the level of the
``include`` statement. *Include* fields may also be included at any
level within a mapping.
Including a Single File
-----------------------
::
include: filename.yaml
Including Multiple Files
------------------------
::
include:
- filename1.yaml
- filename2.yaml
Include Inside a Mapping
------------------------
::
vars:
address-groups:
include: address-groups.yaml
where ``address-groups.yaml`` contains::
%YAML 1.1
---
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
is the equivalent of::
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
.. note:: Suricata versions less than 7 required multiple ``include``
statements to be specified to include more than one file. While
Suricata 7.0 still supports this it will issue a deprecation
warning. Suricata 8.0 will not allow multiple ``include``
statements at the same level as this is not allowed by YAML.
---
### Doc/Userguide/Configuration/Index
Configuration
=============
.. toctree::
suricata-yaml
global-thresholds
exception-policies
snort-to-suricata
multi-tenant
dropping-privileges
landlock
systemd-notify
includes
---
### Doc/Userguide/Configuration/Landlock
.. _landlock:
Using Landlock LSM
==================
Landlock is a Linux Security Module that has been introduced in Linux 5.13.
It allows an application to sandbox itself by selecting access right to
directories using a deny by default approach.
Given its nature, Suricata knows where it is going to read files and where
it is going to write them. So it is possible to implement an efficient
Landlock sandboxing policy.
Landlock is not active by default and needs to be activated in the
YAML configuration. Configuration should come with sane default (defined
at build time) and the command line options are used to dynamically add
some permissions.
Please note that Landlock is in blocking mode by default so careful testing
is needed in production.
To enable Landlock, edit the YAML and set ``enabled`` to ``yes``:
::
landlock:
enabled: yes
directories:
write:
- /var/log/suricata/
- /var/run/
read:
- /usr/
- /etc/
- /etc/suricata/
Following your running configuration you may have to add some directories.
There are two lists you can use, ``write`` to add directories where write is needed
and ``read`` for directories where read access is needed.
Landlock is not active in some distributions and you may need to activate it
at boot by adding ``lsm=landock`` to the Linux command line. For example,
on a Debian distribution with at least a linux 5.13, you can edit ``/etc/default/grub``
and update the ``GRUB_CMDLINE_LINUX_DEFAULT`` option:
::
GRUB_CMDLINE_LINUX_DEFAULT="quiet lsm=landlock"
Then run ``sudo update-grub`` and reboot.
You can check at boot if it is running by doing:
::
sudo dmesg | grep landlock || journalctl -kg landlock
If you are interested in reading more about Landlock, you can use https://docs.kernel.org/userspace-api/landlock.html
as entry point.
---
### Doc/Userguide/Configuration/Multi Tenant
Multi Tenancy
=============
Introduction
------------
Multi tenancy support allows different tenants to use different
rule sets with different rule variables.
Tenants are identified by their `selector`; a `selector` can be
a VLAN, interface/device, or from a pcap file ("direct").
YAML
----
Add a new section in the main ("master") Suricata configuration file -- ``suricata.yaml`` -- named ``multi-detect``.
Settings:
* `enabled`: yes/no -> is multi-tenancy support enabled
* `selector`: direct (for unix socket pcap processing, see below), VLAN or device
* `loaders`: number of `loader` threads, for parallel tenant loading at startup
* `tenants`: list of tenants
* `config-path`: path from where the tenant yamls are loaded
* id: tenant id (numeric values only)
* yaml: separate yaml file with the tenant specific settings
* `mappings`:
* VLAN id or device: The outermost VLAN is used to match.
* tenant id: tenant to associate with the VLAN id or device
::
multi-detect:
enabled: yes
#selector: direct # direct or vlan or device
selector: vlan
loaders: 3
tenants:
- id: 1
yaml: tenant-1.yaml
- id: 2
yaml: tenant-2.yaml
- id: 3
yaml: tenant-3.yaml
mappings:
- vlan-id: 1000
tenant-id: 1
- vlan-id: 2000
tenant-id: 2
- vlan-id: 1112
tenant-id: 3
The tenant-1.yaml, tenant-2.yaml, tenant-3.yaml each contain a partial
configuration:
::
# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
- rules1
# You can specify a threshold config file by setting "threshold-file"
# to the path of the threshold config file:
# threshold-file: /etc/suricata/threshold.config
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config
# Holds variables that would be used by the engine.
vars:
# Holds the address group vars that would be passed in a Signature.
# These would be retrieved during the Signature address parsing stage.
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
EXTERNAL_NET: "!$HOME_NET"
...
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
...
vlan-id
~~~~~~~
Assign tenants to VLAN ids. Suricata matches the outermost VLAN id with this value.
Multiple VLANs can have the same tenant id. VLAN id values must be between 1 and 4094.
Example of VLAN mapping::
mappings:
- vlan-id: 1000
tenant-id: 1
- vlan-id: 2000
tenant-id: 2
- vlan-id: 1112
tenant-id: 3
The mappings can also be modified over the unix socket, see below.
Note: can only be used if ``vlan.use-for-tracking`` is enabled.
device
~~~~~~
Assign tenants to devices. A single tenant can be assigned to a device.
Multiple devices can have the same tenant id.
Example of device mapping::
mappings:
- device: ens5f0
tenant-id: 1
- device: ens5f1
tenant-id: 3
The mappings are static and cannot be modified over the unix socket.
Note: Not currently supported for IPS.
Note: support depends on a capture method using the 'livedev' API. Currently
these are: pcap, AF_PACKET, PF_RING and Netmap.
Per tenant settings
-------------------
The following settings are per tenant:
* default-rule-path
* rule-files
* classification-file
* reference-config-file
* threshold-file
* address-vars
* port-vars
Unix Socket
-----------
Registration
~~~~~~~~~~~~
``register-tenant ``
Examples:
::
register-tenant 1 tenant-1.yaml
register-tenant 2 tenant-2.yaml
register-tenant 3 tenant-3.yaml
register-tenant 5 tenant-5.yaml
register-tenant 7 tenant-7.yaml
``unregister-tenant ``
::
unregister-tenant 2
unregister-tenant 1
Unix socket runmode (pcap processing)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Unix Socket ``pcap-file`` command is used to associate the tenant with
the pcap:
::
pcap-file traffic1.pcap /logs1/ 1
pcap-file traffic2.pcap /logs2/ 2
pcap-file traffic3.pcap /logs3/ 3
pcap-file traffic4.pcap /logs5/ 5
pcap-file traffic5.pcap /logs7/ 7
This runs the traffic1.pcap against tenant 1 and it logs into /logs1/,
traffic2.pcap against tenant 2 and logs to /logs2/ and so on.
Live traffic mode
~~~~~~~~~~~~~~~~~
Multi-tenancy supports both VLAN and devices with live traffic.
In the master configuration yaml file, specify ``device`` or ``vlan`` for the ``selector`` setting.
Registration
~~~~~~~~~~~~
Tenants can be mapped to vlan ids.
``register-tenant-handler vlan ``
::
register-tenant-handler 1 vlan 1000
``unregister-tenant-handler vlan ``
::
unregister-tenant-handler 4 vlan 1111
unregister-tenant-handler 1 vlan 1000
The registration of tenant and tenant handlers can be done on a
running engine.
Reloads
~~~~~~~
Reloading all tenants:
``reload-tenants``
::
reload-tenants
Reloading a single tenant:
``reload-tenant [yaml path]``
::
reload-tenant 1 tenant-1.yaml
reload-tenant 5
The ``[yaml path]`` is optional. If it isn't provided, the original path of
the tenant will be used during the reload.
Eve JSON output
---------------
When multi-tenant support is configured and the detect engine is active then
all EVE-types that report based on flows will also report the corresponding
``tenant_id`` for events matching a tenant configuration.
---
### Doc/Userguide/Configuration/Snort To Suricata
Snort.conf to Suricata.yaml
===========================
This guide is meant for those who are familiar with Snort and the
snort.conf configuration format. This guide will provide a 1:1 mapping
between Snort and Suricata configuration wherever possible.
Variables
---------
snort.conf
::
ipvar HOME_NET any
ipvar EXTERNAL_NET any
...
portvar HTTP_PORTS [80,81,311,591,593,901,1220,1414,1741,1830,2301,2381,2809,3128,3702,4343,4848,5250,7001,7145,7510,7777,7779,8000,8008,8014,8028,8080,8088,8090,8118,8123,8180,8181,8243,8280,8800,8888,8899,9000,9080,9090,9091,9443,9999,11371,55555]
portvar SHELLCODE_PORTS !80
...
suricata.yaml
::
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
EXTERNAL_NET: "!$HOME_NET"
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
Note that Suricata can automatically detect HTTP traffic regardless of
the port it uses. So the HTTP_PORTS variable is not nearly as
important as it is with Snort, **if** you use a Suricata enabled
ruleset.
Decoder alerts
--------------
snort.conf
::
# Stop generic decode events:
config disable_decode_alerts
# Stop Alerts on experimental TCP options
config disable_tcpopt_experimental_alerts
# Stop Alerts on obsolete TCP options
config disable_tcpopt_obsolete_alerts
# Stop Alerts on T/TCP alerts
config disable_tcpopt_ttcp_alerts
# Stop Alerts on all other TCPOption type events:
config disable_tcpopt_alerts
# Stop Alerts on invalid ip options
config disable_ipopt_alerts
suricata.yaml
Suricata has no specific decoder options. All decoder related alerts
are controlled by rules. See #Rules below.
Checksum handling
-----------------
snort.conf
::
config checksum_mode: all
suricata.yaml
Suricata's checksum handling works *on-demand*. The stream engine
checks TCP and IP checksum by default:
::
stream:
checksum-validation: yes # reject wrong csums
Alerting on bad checksums can be done with normal rules. See #Rules,
decoder-events.rules specifically.
Various configs
---------------
Active response
~~~~~~~~~~~~~~~
snort.conf
::
# Configure active response for non inline operation. For more information, see REAMDE.active
# config response: eth0 attempts 2
suricata.yaml
Active responses are handled automatically w/o config if rules with
the "reject" action are used.
Dropping privileges
~~~~~~~~~~~~~~~~~~~
snort.conf
::
# Configure specific UID and GID to run snort as after dropping privs. For more information see snort -h command line options
#
# config set_gid:
# config set_uid:
Suricata
To set the user and group use the --user and --group
command-line options.
Snaplen
~~~~~~~
snort.conf
::
# Configure default snaplen. Snort defaults to MTU of in use interface. For more information see README
#
# config snaplen:
#
Suricata always works at full snap length to provide full traffic visibility.
Bpf
~~~
snort.conf
::
# Configure default bpf_file to use for filtering what traffic reaches snort. For more information see snort -h command line options (-F)
#
# config bpf_file:
#
suricata.yaml
BPF filters can be set per packet acquisition method, with the "bpf-filter: " yaml option and in a file using the -F command line option.
For example:
::
pcap:
- interface: eth0
#buffer-size: 16777216
#bpf-filter: "tcp and port 25"
#checksum-checks: auto
#threads: 16
#promisc: no
#snaplen: 1518
Log directory
-------------
snort.conf
::
# Configure default log directory for snort to log to. For more information see snort -h command line options (-l)
#
# config logdir:
suricata.yaml
::
default-log-dir: /var/log/suricata/
This value is overridden by the -l command-line option.
Packet acquisition
------------------
snort.conf
::
# Configure DAQ related options for inline operation. For more information, see README.daq
#
# config daq:
# config daq_dir:
# config daq_mode:
# config daq_var:
#
# ::= pcap | afpacket | dump | nfq | ipq | ipfw
# ::= read-file | passive | inline
# ::= arbitrary = ::= path as to where to look for DAQ module so's
suricata.yaml
Suricata has all packet acquisition support built-in. It's
configuration format is very verbose.
::
pcap:
- interface: eth0
#buffer-size: 16777216
#bpf-filter: "tcp and port 25"
#checksum-checks: auto
#threads: 16
#promisc: no
#snaplen: 1518
pfring:
afpacket:
nfq:
ipfw:
Passive vs inline vs reading files is determined by how Suricata is
invoked on the command line.
Rules
-----
snort.conf:
In snort.conf a RULE_PATH variable is set, as well as variables for
shared object (SO) rules and preprocessor rules.
::
var RULE_PATH ../rules
var SO_RULE_PATH ../so_rules
var PREPROC_RULE_PATH ../preproc_rules
include $RULE_PATH/local.rules
include $RULE_PATH/emerging-activex.rules
...
suricata.yaml:
In the suricata.yaml the default rule path is set followed by a list
of rule files. Suricata does not have a concept of shared object rules
or preprocessor rules. Instead of preprocessor rules, Suricata has
several rule files for events set by the decoders, stream engine, http
parser etc.
::
default-rule-path: /etc/suricata/rules
rule-files:
- local.rules
- emerging-activex.rules
The equivalent of preprocessor rules are loaded like normal rule files:
::
rule-files:
- decoder-events.rules
- stream-events.rules
- http-events.rules
- smtp-events.rules
---
### Doc/Userguide/Configuration/Suricata Yaml
Suricata.yaml
=============
Suricata uses the Yaml format for configuration. The Suricata.yaml
file included in the source code, is the example configuration of
Suricata. This document will explain each option.
At the top of the YAML-file you will find % YAML 1.1. Suricata reads
the file and identifies the file as YAML.
.. _suricata-yaml-max-pending-packets:
Max-pending-packets
-------------------
With the max-pending-packets setting you can set the number of packets
you allow Suricata to process simultaneously. This can range from one
packet to tens of thousands/hundreds of thousands of packets. It is a
trade of higher performance and the use of more memory (RAM), or lower
performance and less use of memory. A high number of packets being
processed results in a higher performance and the use of more
memory. A low number of packets, results in lower performance and less
use of memory. Choosing a low number of packets being processed while
having many CPU's/CPU cores, can result in not making use of the whole
computer-capacity. (For instance: using one core while having three
waiting for processing packets.)
::
max-pending-packets: 1024
Runmodes
--------
By default the runmode option is disabled. With the runmodes setting
you can set the runmode you would like to use. For all runmodes
available, enter **--list-runmodes** in your command line. For more
information, see :doc:`../performance/runmodes`.
::
runmode: autofp
Default-packet-size
-------------------
For the max-pending-packets option, Suricata has to keep packets in
memory. With the default-packet-size option, you can set the size of
the packets on your network. It is possible that bigger packets have
to be processed sometimes. The engine can still process these bigger
packets, but processing it will lower the performance.
::
default-packet-size: 1514
User and group
--------------
It is possible to set the user and group to run Suricata as:
::
run-as:
user: suri
group: suri
PID File
--------
This option sets the name of the PID file when Suricata is run in
daemon mode. This file records the Suricata process ID.
::
pid-file: /var/run/suricata.pid
.. note:: This configuration file option only sets the PID file when
running in daemon mode. To force creation of a PID file when
not running in daemon mode, use the :option:`--pidfile`
command line option.
Also, if running more than one Suricata process, each
process will need to specify a different pid-file location.
.. _suricata-yaml-action-order:
Action-order
------------
All signatures have different properties. One of those is the Action
property. This one determines what will happen when a signature
matches. There are four types of Action. A summary of what will
happen when a signature matches and contains one of those Actions:
1) Pass
If a signature matches and contains pass, Suricata stops scanning the
packet and skips to the end of all rules (only for the current
packet). If the signature matches on a TCP connection, the entire
flow will be passed but details of the flow will still be logged.
2) Drop
This only concerns the IPS/inline mode. If the program finds a
signature that matches, containing drop, it stops immediately. The
packet will not be sent any further. Drawback: The receiver does not
receive a message of what is going on, resulting in a time-out
(certainly with TCP). Suricata generates an alert for this packet.
3) Reject
This is an active rejection of the packet. Both receiver and sender
receive a reject packet. There are two types of reject packets that
will be automatically selected. If the offending packet concerns TCP,
it will be a Reset-packet. For all other protocols it will be an
ICMP-error packet. Suricata also generates an alert. When in
Inline/IPS mode, the offending packet will also be dropped like with
the 'drop' action.
4) Alert
If a signature matches and contains alert, the packet will be treated
like any other non-threatening packet, except for this one an alert
will be generated by Suricata. Only the system administrator can
notice this alert.
Inline/IPS can block network traffic in two ways. One way is by drop
and the other by reject.
Rules will be loaded in the order of which they appear in files. But
they will be processed in a different order. Signatures have different
priorities. The most important signatures will be scanned first. There
is a possibility to change the order of priority. The default order
is: pass, drop, reject, alert.
::
action-order:
- pass
- drop
- reject
- alert
This means a pass rule is considered before a drop rule, a drop rule
before a reject rule and so on.
Packet alert queue settings
---------------------------
It is possible to configure the size of the alerts queue that is used to append alerts triggered by each packet.
This will influence how many alerts would be perceived to have matched against a given packet.
The default value is 15. If an invalid setting or no value is provided, the engine will fall
back to the default.
::
#Define maximum number of possible alerts that can be triggered for the same
# packet. Default is 15
packet-alert-max: 15
We recommend that you use the default value for this setting unless you are seeing a high number of discarded alerts
(``alert_queue_overflow``) - see the `Discarded and Suppressed Alerts Stats`_ section for more details.
.. _alert queue overflow impact:
Impact on engine behavior
~~~~~~~~~~~~~~~~~~~~~~~~~
Internally, the Suricata engine represents each packet with a data structure that has its own alert queue. The max size
of the queue is defined by ``packet-alert-max``. The same rule can be triggered by the same packet multiple times. As
long as there is still space in the alert queue, those are appended.
Rules that have the ``noalert`` keyword will be checked - in case their signatures have actions that must be applied to the Packet or Flow, then suppressed. They have no effect in the final alert queue.
Rules are queued by priority: higher priority rules may be kept instead of lower priority ones that may have been triggered earlier, if Suricata reaches ``packet-alert-max`` for a given packet (a.k.a. packet alert queue overflow).
Packet alert queue overflow
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Once the alert queue reaches its max size, we are potentially at packet alert queue overflow, so new alerts will only be appended in case their rules have a higher priority id (this is the internal id attributed by the engine, not the signature id).
This may happen in two different situations:
- a higher priority rule is triggered after a lower priority one: the lower priority rule is replaced in the queue;
- a lower priority rule is triggered: the rule is just discarded.
.. note ::
This behavior does not mean that triggered ``drop`` rules would have their action ignored, in IPS mode.
.. _alerts stats:
Discarded and Suppressed Alerts Stats
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Both scenarios previously described will be logged as *detect.alert_queue_overflow* in the stats logs (in stats.log and eve-log's stats event).
When ``noalert`` rules match, they appear in the stats logs as *detect.alerts_suppressed*.
::
Date: 4/6/2022 -- 17:18:08 (uptime: 0d, 00h 00m 00s)
------------------------------------------------------------------------------------
Counter | TM Name | Value
------------------------------------------------------------------------------------
detect.alert | Total | 3
detect.alert_queue_overflow | Total | 4
detect.alerts_suppressed | Total | 1
In this example from a stats.log, we read that 8 alerts were generated: 3 were kept in the packet queue while 4
were discarded due to packets having reached max size for the alert queue, and 1 was suppressed due to coming from a ``noalert``
rule.
Splitting configuration in multiple files
-----------------------------------------
Some users might have a need or a wish to split their suricata.yaml
file into separate files, this is available via the 'include' and
'!include' keyword. The first example is of taking the contents of the
outputs section and storing them in outputs.yaml.
::
# outputs.yaml
- fast
enabled: yes
filename: fast.log
append: yes
...
::
# suricata.yaml
...
outputs: !include outputs.yaml
...
The second scenario is where multiple sections are migrated to a
different YAML file.
::
# host_1.yaml
max-pending-packets: 2048
outputs:
- fast
enabled: yes
filename: fast.log
append: yes
::
# suricata.yaml
include: host_1.yaml
...
If the same section, say outputs is later redefined after the include
statement it will overwrite the included file. Therefore any include
statement at the end of the document will overwrite the already
configured sections.
Event output
------------
Default logging directory
~~~~~~~~~~~~~~~~~~~~~~~~~
In the /var/log/suricata directory, all of Suricata's output (alerts
and events) will be stored.
::
default-log-dir: /var/log/suricata
This directory can be overridden by entering the -l command line
parameter or by changing the directory directly in Yaml. To change it
with the -l command line parameter, enter the following:
::
suricata -c suricata.yaml -i eth0 -l /var/log/suricata-logs/
.. _suricata_yaml_outputs:
Stats
~~~~~
Engine statistics such as packet counters, memory use counters and others
can be logged in several ways. A separate text log 'stats.log' and an EVE
record type 'stats' are enabled by default.
The stats have a global configuration and a per logger configuration. Here
the global config is documented.
::
# global stats configuration
stats:
enabled: yes
# The interval field (in seconds) controls at what interval
# the loggers are invoked.
interval: 8
# Add decode events as stats.
#decoder-events: true
# Decoder event prefix in stats. Has been 'decoder' before, but that leads
# to missing events in the eve.stats records. See issue #2225.
#decoder-events-prefix: "decoder.event"
# Add stream events as stats.
#stream-events: false
# Exception policy stats counters options
# (Note: if exception policy: ignore, counters are not logged)
exception-policy:
#per-app-proto-errors: false # default: false. True will log errors for
# each app-proto. Warning: VERY verbose
Statistics can be `enabled` or disabled here.
Statistics are dumped on an `interval`. Setting this below 3 or 4 seconds is
not useful due to how threads are synchronized internally.
The decoder events that the decoding layer generates, can create a counter per
event type. This behaviour is enabled by default. The `decoder-events` option
can be set to `false` to disable.
In 4.1.x there was a naming clash between the regular decoder counters and
the decoder-event counters. This lead to a fair amount of decoder-event
counters not being shown in the EVE.stats records. To address this without
breaking existing setups, a config option `decoder-events-prefix` was added
to change the naming of the decoder-events from decoder.. to
decoder.event... In 5.0 this became the default.
See `issue 2225 `_.
Similar to the `decoder-events` option, the `stream-events` option controls
whether the stream-events are added as counters as well. This is disabled by
default.
If any exception policy is enabled, stats counters are logged. To control
verbosity for application layer protocol errors, leave `per-app-proto-errors`
as false.
Outputs
~~~~~~~
There are several types of output. The general structure is:
::
outputs:
- fast:
enabled: yes
filename: fast.log
append: yes/no
# Compress IPv6 addresses per RFC5952 as they are added to the fast log. The default is no.
ipv6-compress: yes/no
Enabling all of the logs, will result in a much lower performance and
the use of more disc space, so enable only the outputs you need.
Line based alerts log (fast.log)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This log contains alerts consisting of a single line. Example of the
appearance of a single fast.log-file line:
::
10/05/10-10:08:59.667372 Â [**] [1:2009187:4] ET WEB_CLIENT ACTIVEX iDefense
COMRaider ActiveX Control Arbitrary File Deletion [**] [Classification: Web
Application Attack] [Priority: 3] {TCP} xx.xx.232.144:80 -> 192.168.1.4:56068
::
-fast: #The log-name.
enabled:yes #This log is enabled. Set to 'no' to disable.
filename: fast.log #The name of the file in the default logging directory.
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
#overwritten while restarting Suricata.
# Compress IPv6 addresses per RFC5952 as they are added to the fast log. The default is no.
ipv6-compress: yes/no
.. _suricata-yaml-outputs-eve:
Eve (Extensible Event Format)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is an JSON output for alerts and events. It allows for easy
integration with 3rd party tools like logstash.
.. literalinclude:: ../partials/eve-log.yaml
For more advanced configuration options, see :ref:`Eve JSON Output `.
The format is documented in :ref:`Eve JSON Format `.
.. _suricata-yaml-outputs-tls:
TLS parameters and certificates logging (tls.log)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. attention:: tls-log is deprecated in Suricata 8.0 and will be
removed in Suricata 9.0.
The TLS handshake parameters can be logged in a line based log as well.
By default, the logfile is `tls.log` in the suricata log directory.
See :ref:`Custom TLS logging ` for details
about the configuration and customization of the log format.
Furthermore there is an output module to store TLS certificate files to
disk. This is similar to :ref:`File-store (File Extraction)
`, but for TLS certificates.
Example:
::
# output module to store certificates chain to disk
- tls-store:
enabled: yes
#certs-log-dir: certs # directory to store the certificates files
.. _suricata_yaml_pcap_log:
Packet log (pcap-log)
~~~~~~~~~~~~~~~~~~~~~
With the pcap-log option you can save all packets, that are registered
by Suricata, in a log file named _log.pcap_. This way, you can take a
look at all packets whenever you want. In the normal mode a pcap file
is created in the default-log-dir. It can also be created elsewhere if
a absolute path is set in the yaml-file.
The file that is saved in example the ``default-log-dir``
`/var/log/suricata`, can be be opened with every program which supports
the pcap file format. This can be Wireshark, TCPdump, Suricata, Snort
and many others.
The pcap-log option can be enabled and disabled.
There is a size limit for the pcap-log file that can be set. The
default limit is 32 MB. If the log-file reaches this limit, the file
will be rotated and a new one will be created.
Remember that in the 'normal' mode, the file will be saved in
default-log-dir or in the absolute path (if set).
The pcap files can be compressed before being written to disk by setting
the compression option to lz4.
Note: On Windows, this option increases disk I/O instead of
reducing it. When using lz4 compression, you can enable checksums using
the lz4-checksum option, and you can set the compression level lz4-level
to a value between 0 and 16, where higher levels result in higher
compression.
By default all packets are logged except:
- TCP streams beyond stream.reassembly.depth
- encrypted streams after the key exchange
- If a ``bpf-filter`` is set, packets that don't match the filter will
not be logged
It is possible to do conditional pcap logging by using the `conditional`
option in the pcap-log section. By default the variable is set to `all`
so all packets are logged. If the variable is set to `alerts` then only
the flow with alerts will be logged. If the variable is set to `tag`
then only packets tagged by signatures using the `tag` keyword will
be logged to the pcap file. Please note that if `alerts` or `tag` is
used, then in the case of TCP session, Suricata will use available
information from the streaming engine to log data that have triggered
the alert.
::
- pcap-log:
enabled: yes
filename: log.pcap
# Limit in MB.
limit: 32
mode: normal # "normal" or multi
conditional: alerts
# A BPF filter that will be applied to all packets being
# logged. If set, packets must match this filter otherwise they
# will not be logged.
#bpf-filter:
In ``normal`` mode a pcap file "filename" is created in the default-log-dir or as
specified by "dir". ``normal`` mode is generally not as performant as ``multi``
mode.
In multi mode, multiple pcap files are created (per thread) which performs
better than ``normal`` mode.
In multi mode the filename takes a few special variables:
- %n representing the thread number
- %i representing the thread id
- %t representing the timestamp (secs or secs.usecs based on 'ts-format')
Example: filename: pcap.%n.%t
.. note:: It is possible to use directories but the directories are not
created by Suricata. For example ``filename: pcaps/%n/log.%s`` will log into
the pre-existing ``pcaps`` directory and per thread sub directories.
.. note:: that the limit and max-files settings are enforced per thread. So the
size limit using 8 threads with 1000mb files and 2000 files is about 16TiB.
Verbose Alerts Log (alert-debug.log)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is a log type that gives supplementary information about an
alert. It is particularly convenient for people who investigate false
positives and who write signatures. However, it lowers the performance
because of the amount of information it has to store.
::
- alert-debug: #The log-name.
enabled: no #This log is not enabled. Set 'yes' to enable.
filename: alert-debug.log #The name of the file in the default logging directory.
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
# overwritten while restarting Suricata.
# Compress IPv6 addresses per RFC5952 as they are added to the debug log log. The default is no.
ipv6-compress: yes/no
Stats
~~~~~
In stats you can set the options for stats.log. When enabling
stats.log you can set the amount of time in seconds after which you
want the output-data to be written to the log file.
::
- stats:
enabled: yes #By default, the stats-option is enabled
filename: stats.log #The log-name. Combined with the default logging directory
#(default-log-dir) it will result in /var/log/suricata/stats.log.
#This directory can be overruled with a absolute path. (A
#directory starting with / ).
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
#overwritten while restarting Suricata.
The interval and several other options depend on the global stats
section as described above.
Syslog
~~~~~~
.. attention:: The syslog output is deprecated in Suricata 8.0 and
will be removed in Suricata 9.0. Please migrate to the
``eve`` output which has the ability to send to syslog.
With this option it is possible to send all alert and event output to syslog.
::
- syslog: #This is a output-module to direct log-output to several directions.
enabled: no #The use of this output-module is not enabled.
facility: local5 #In this option you can set a syslog facility.
level: Info #In this option you can set the level of output. The possible levels are:
#Emergency, Alert, Critical, Error, Warning, Notice, Info and Debug.
.. _suricata-yaml-file-store:
File-store (File Extraction)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The `file-store` output enables storing of extracted files to disk and
configures where they are stored.
The following shows the configuration options for version 2 of the
`file-store` output.
.. code-block:: yaml
- file-store:
# This configures version 2 of the file-store.
version: 2
enabled: no
# Set the directory for the filestore. If the path is not
# absolute will be be relative to the default-log-dir.
#dir: filestore
# Write out a fileinfo record for each occurrence of a
# file. Disabled by default as each occurrence is already logged
# as a fileinfo record to the main eve-log.
#write-fileinfo: yes
# Force storing of all files. Default: no.
#force-filestore: yes
# Override the global stream-depth for sessions in which we want
# to perform file extraction. Set to 0 for unlimited; otherwise,
# must be greater than the global stream-depth value to be used.
#stream-depth: 0
# Uncomment the following variable to define how many files can
# remain open for filestore by Suricata. Default value is 0 which
# means files get closed after each write
#max-open-files: 1000
# Force logging of checksums, available hash functions are md5,
# sha1 and sha256. Note that SHA256 is automatically forced by
# the use of this output module as it uses the SHA256 as the
# file naming scheme.
#force-hash: [sha1, md5]
.. _detection-engine:
Detection engine
----------------
Inspection configuration
~~~~~~~~~~~~~~~~~~~~~~~~
The detection-engine builds internal groups of signatures. Suricata
loads signatures, with which the network traffic will be compared.
The fact is, that many rules certainly will not be necessary. For
instance, if there appears a packet with the UDP-protocol, all signatures
for the TCP-protocol won't be needed. For that reason, all signatures
will be divided in groups. However, a distribution containing many
groups will make use of a lot of memory. Not every type of signature
gets its own group. There is a possibility that different signatures
with several properties in common, will be placed together in a group.
The quantity of groups will determine the balance between memory and
performance. A small number of groups will lower the performance yet
use little memory. The opposite counts for a higher amount of groups.
The engine allows you to manage the balance between memory and performance.
To manage this, (by determining the amount of groups) there are several
general options: ``high`` for good performance and more use of memory,
``low`` for low performance and little use of memory. The option ``medium``
is the balance between performance and memory usage. This is the default
setting. The option ``custom-values`` is for advanced users. This option
has values which can be managed by the user.
::
detect:
profile: medium
custom-values:
toclient-groups: 3
toserver-groups: 25
sgh-mpm-context: auto
inspection-recursion-limit: 3000
stream-tx-log-limit: 4
guess-applayer-tx: no
grouping:
tcp-priority-ports: 53, 80, 139, 443, 445, 1433, 3306, 3389, 6666, 6667, 8080
udp-priority-ports: 53, 135, 5060
flowbits:
max-per-signature: 8
At all of these options, you can add (or change) a value. Most
signatures have the adjustment to focus on one direction, meaning
focusing exclusively on the server, or exclusively on the client.
If you take a look at example 4, *the Detection-engine grouping tree*,
you see it has many branches. At the end of each branch, there is
actually a 'sig group head'. Within that sig group head there is a
container which contains a list with signatures that are significant
for that specific group/that specific end of the branch. Also within
the sig group head the settings for Multi-Pattern-Matcher (MPM) can be
found: the MPM-context.
As will be described again in :ref:`pattern-matcher-settings`,
there are several MPM-algorithms of which can be chosen from. Because
every sig group head has its own MPM-context, some algorithms use a
lot of memory. For that reason there is the option ``sgh-mpm-context``
to set whether the groups share one MPM-context, or to set that every
group has its own MPM-context.
For setting the option sgh-mpm-context, you can choose from auto, full
or single. The default setting is 'auto', meaning Suricata selects
full or single based on the algorithm you use. 'Full' means that every
group has its own MPM-context, and 'single' that all groups share one
MPM-context. The algorithm "ac" uses a single MPM-context if the
Sgh-MPM-context setting is 'auto'. The rest of the algorithms use full
in that case.
The ``inspection-recursion-limit`` option has to mitigate that possible
bugs in Suricata cause big problems. Often Suricata has to deal with
complicated issues. It could end up in an 'endless loop' due to a bug,
meaning it will repeat its actions over and over again. With the
option inspection-recursion-limit you can limit this action.
The ``stream-tx-log-limit`` defines the maximum number of times a
transaction will get logged for rules without app-layer keywords.
This is meant to avoid logging the same data an arbitrary number
of times.
The ``guess-applayer-tx`` option controls whether the engine will try to guess
and tie a transaction to a given alert if the matching signature doesn't have
app-layer keywords. If enabled, AND ONLY ONE LIVE TRANSACTION EXISTS, that
transaction's data will be added to the alert metadata. Note that this may not
be the expected data, from an analyst's perspective.
The ``grouping`` option allows user to define the most seen ports
on their network using ``tcp-priority-ports`` and ``udp-priority-ports``
settings to benefit from the internal signature groups created by Suricata.
The engine shall then try to club the rules that use the ports defined
in groups of their own and put them on top of the list of rules to be matched
against traffic on "priority".
The ``flowbits`` option carries flowbits detection specific settings. With
``max-per-signature`` setting, it is possible to define how many times flowbits
keyword can be seen in any one signature. This does not include ``flowbits:noalert;``.
Minimum value allowed is 1 and the default is 8.
*Example 4 Detection-engine grouping tree*
.. image:: suricata-yaml/grouping_tree.png
::
src Stands for source IP-address.
dst Stands for destination IP-address.
sp Stands for source port.
dp Stands for destination port.
*Example 5 Detail grouping tree*
.. image:: suricata-yaml/grouping_tree_detail.png
.. _suricata-yaml-prefilter:
Prefilter Engines
~~~~~~~~~~~~~~~~~
The concept of prefiltering is that there are far too many rules to inspect individually. The approach prefilter takes is that from each rule one condition is added to prefilter, which is then checked in one step. The most common example is MPM (also known as fast_pattern). This takes a single pattern per rule and adds it to the MPM. Only for those rules that have at least one pattern match in the MPM stage, individual inspection is performed.
Next to MPM, other types of keywords support prefiltering. ICMP itype, icode, icmp_seq and icmp_id for example. TCP window, IP TTL are other examples.
For a full list of keywords that support prefilter, see:
::
suricata --list-keywords=all
Suricata can automatically select prefilter options, or it can be set manually.
::
detect:
prefilter:
default: mpm
By default, only MPM/fast_pattern is used.
The prefilter engines for other non-MPM keywords can then be enabled in specific rules by using the 'prefilter' keyword.
E.g.
.. container:: example-rule
alert ip any any -> any any (ttl:123; prefilter; sid:1;)
To let Suricata make these decisions set default to 'auto':
::
detect:
prefilter:
default: auto
.. _suricata-yaml-thresholds:
Thresholding Settings
~~~~~~~~~~~~~~~~~~~~~
Thresholding uses a central hash table for tracking thresholds of the types: by_src, by_dst, by_both.
::
detect:
thresholds:
hash-size: 16384
memcap: 16mb
``detect.thresholds.hash-size`` controls the number of hash rows in the hash table.
``detect.thresholds.memcap`` controls how much memory can be used for the hash table and the data stored in it.
.. _pattern-matcher-settings:
Pattern matcher settings
~~~~~~~~~~~~~~~~~~~~~~~~
The multi-pattern-matcher (MPM) is a part of the detection engine
within Suricata that searches for multiple patterns at
once. Often, signatures have one or more patterns. Of each
signature, one pattern is used by the multi-pattern-matcher. That way
Suricata can exclude many signatures from being examined, because a
signature can only match when all its patterns match.
These are the proceedings:
1) A packet comes in.
2) The packed will be analyzed by the Multi-pattern-matcher in search of patterns that match.
3) All patterns that match, will be further processed by Suricata (signatures).
*Example 8 Multi-pattern-matcher*
.. image:: suricata-yaml/MPM2.png
Suricata offers various implementations of different
multi-pattern-matcher algorithm's. These can be found below.
To set the multi-pattern-matcher algorithm:
::
mpm-algo: ac
After 'mpm-algo', you can enter one of the following algorithms: ac, hs and ac-ks.
On `x86_64` hs (Hyperscan) should be used for best performance.
.. _suricata-yaml-threading:
Threading
---------
Suricata is multi-threaded. Suricata uses multiple CPUs/CPU cores so
it can process a lot of network packets simultaneously. (In a
single-core engine, the packets will be processed one at a time.)
There are four thread-modules: Packet acquisition, decode and stream
application layer, detection, and outputs.
# The packet acquisition module reads packets from the network.
# The decode module decodes the packets and the stream application
application layer has three tasks:
::
First: it performs stream-tracking, meaning it is making sure all steps will be taken to make a correct network-connection.
Second: TCP-network traffic comes in as packets. The Stream-Assembly engine reconstructs the original stream.
Finally: the application layer will be inspected. HTTP and DCERPC will be analyzed.
# The detection threads will compare signatures. There can be several detection threads so they can operate simultaneously.
# In Outputs all alerts and events will be processed.
*Example 6 Threading*
.. image:: suricata-yaml/threading.png
::
Packet acquisition: Reads packets from the network
Decode: Decodes packets.
Stream app. Layer: Performs stream-tracking and reassembly.
Detect: Compares signatures.
Outputs: Processes all events and alerts.
Most computers have multiple CPU's/ CPU cores. By default the
operating system determines which core works on which thread. When a
core is already occupied, another one will be designated to work on
the thread. So, which core works on which thread, can differ from time
to time.
There is an option within threading:
::
set-cpu-affinity: no
With this option you can cause Suricata setting fixed cores for every
thread. In that case 1, 2 and 4 are at core 0 (zero). Each core has
its own detect thread. The detect thread running on core 0 has a lower
priority than the other threads running on core 0. If these other
cores are to occupied, the detect thread on core 0 has not much
packets to process. The detect threads running on other cores will
process more packets. This is only the case after setting the option
to 'yes'.
*Example 7 Balancing workload*
.. image:: suricata-yaml/balancing_workload.png
You can set the detect-thread-ratio:
::
detect-thread-ratio: 1.5
The detect thread-ratio will determine the amount of detect
threads. By default it will be 1.5 x the amount of CPU's/CPU cores
present at your computer. This will result in having more detection
threads then CPU's/ CPU cores. Meaning you are oversubscribing the
amount of cores. This may be convenient at times when there have to be
waited for a detection thread. The remaining detection thread can
become active.
You can alter the per-thread stack-size if the default provided by
your build system is too small. The default value is provided by
your build system; we suggest setting the value to 8MB if the default
value is too small.
::
stack-size: 8MB
In the option 'cpu affinity' you can set which CPU's/cores work on which
thread. In this option there are several sets of threads. The management-,
receive-, worker- and verdict-set. These are fixed names and can not be
changed. For each set there are several options: cpu, mode, and prio. In the
option 'cpu' you can set the numbers of the CPU's/cores which will run the
threads from that set. You can set this option to 'all', use a range (0-3) or a
comma separated list (0,1). The option 'mode' can be set to 'balanced' or
'exclusive'. When set to 'balanced', the individual threads can be processed by
all cores set in the option 'cpu'. If the option 'mode' is set to 'exclusive',
there will be fixed cores for each thread. As mentioned before, threads can
have different priority's. In the option 'prio' you can set a priority for each
thread. This priority can be low, medium, high or you can set the priority to
'default'. If you do not set a priority for a CPU, than the settings in
'default' will count. By default Suricata creates one 'detect' (worker) thread
per available CPU/CPU core.
.. note:: The 'prio' settings could overwrite each other, make sure to not
include the same CPU core in different 'prio' settings.
::
threading:
set-cpu-affinity: yes
autopin: no
cpu-affinity:
management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
receive-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
worker-cpu-set:
cpu: [ "all" ]
mode: "exclusive"
# Use explicitly 3 threads and don't compute number by using
# detect-thread-ratio variable:
# threads: 3
prio:
low: [ 0 ]
medium: [ "1-2" ]
high: [ 3 ]
default: "medium"
interface-specific-cpu-set:
- interface: "enp4s0f0" # 0000:3b:00.0 # net_bonding0 # ens1f0
cpu: [ 1,3,5,7,9 ]
mode: "exclusive"
prio:
high: [ "all" ]
default: "medium"
verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high"
Relevant cpu-affinity settings for IDS mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Runmode AutoFp::
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
receive-cpu-set - used for receive and decode
worker-cpu-set - used for streamtcp,detect,output(logging),reject
Rumode Workers::
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
worker-cpu-set - used for receive,streamtcp,decode,detect,output(logging),respond/reject
Relevant cpu-affinity settings for IPS mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Runmode AutoFp::
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
receive-cpu-set - used for receive and decode
worker-cpu-set - used for streamtcp,detect,output(logging)
verdict-cpu-set - used for verdict and respond/reject
Runmode Workers::
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
worker-cpu-set - used for receive,streamtcp,decode,detect,output(logging),respond/reject, verdict
Interface-specific CPU affinity settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Using the new configuration format introduced in Suricata 8.0 it is possible
to set CPU affinity settings per interface. This can be useful
when you have multiple interfaces and you want to dedicate specific CPU cores
to specific interfaces. This can be useful, for example, when Suricata runs on
multiple NUMA nodes and reads from interfaces on each NUMA node.
Interface-specific affinity settings can be configured for the
``worker-cpu-set`` and the ``receive-cpu-set`` (only used in autofp mode).
This feature is available for capture modes which work with interfaces
(af-packet, dpdk, etc.). The value of the interface key can be the kernel
interface name (e.g. eth0 for af-packet), the PCI address of the interface
(e.g. 0000:3b:00.0 for DPDK capture mode), or the name of the virtual device
interface (e.g. net_bonding0 for DPDK capture mode).
The interface names needs to be unique and be specified in the capture mode
configuration.
The interface-specific settings will override the global settings for the
``worker-cpu-set`` and ``receive-cpu-set``. The CPUs do not need to be contained in
the parent node settings. If the interface-specific settings are not defined,
the global settings will be used.
::
threading:
set-cpu-affinity: yes
cpu-affinity:
worker-cpu-set:
interface-specific-cpu-set:
- interface: "eth0" # 0000:3b:00.0 # net_bonding0
cpu: [ 1,3,5,7,9 ]
mode: "exclusive"
prio:
high: [ "all" ]
default: "medium"
Automatic NUMA-aware CPU core pinning
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When Suricata is running on a system with multiple NUMA nodes, it is possible
to automatically use CPUs from the same NUMA node as the network capture
interface.
CPU cores on the same NUMA node as the network capture interface can have
reduced memory access latency and can increase the performance of Suricata.
This is enabled by setting the ``autopin`` option to ``yes`` in the threading
section. This option is available for worker-cpu-set and receive-cpu-set.
::
threading:
set-cpu-affinity: yes
autopin: yes
cpu-affinity:
worker-cpu-set:
cpu: [ "all" ]
mode: "exclusive"
prio:
high: [ "all" ]
Consider 2 interfaces defined in the capture mode configuration, one on each
NUMA node. The ``autopin`` option is enabled to automatically use CPUs from the
same NUMA node as the interface. The worker-cpu-set is set to use all CPUs.
When interface on the first NUMA node is used, the worker threads will be
pinned to CPUs on the first NUMA node. When interface on the second NUMA node
is used, the worker threads will be pinned to CPUs on the second NUMA node.
If the number of CPU cores on a given NUMA node is exhausted then the worker
threads will be pinned to CPUs on the other NUMA node.
The option ``threading.autopin`` can be combined with the interface-specific CPU
affinity settings.
To use the ``autopin`` option, the system must have the ``hwloc``
dependency installed and pass ``--enable-hwloc`` to the configure script.
IP Defrag
---------
Occasionally network packets appear fragmented. On some networks it
occurs more often than on others. Fragmented packets exist of many
parts. Before Suricata is able to inspect these kind of packets
accurately, the packets have to be reconstructed. This will be done by
a component of Suricata; the defragment-engine. After a fragmented
packet is reconstructed by the defragment-engine, the engine sends on
the reassembled packet to rest of Suricata.
At the moment Suricata receives a fragment of a packet, it
keeps in memory that other fragments of that packet will appear soon
to complete the packet. However, there is a possibility that one of
the fragments does not appear. To prevent Suricata for keeping waiting
for that packet (thereby using memory) there is a timespan after which
Suricata discards the fragments (timeout). This occurs by default after 60
seconds.
In IPS mode, it is possible to tell the engine what to do in case the memcap for
the defrag engine is reached: "drop-packet", "pass-packet", or "ignore" (default
behavior).
::
defrag:
memcap: 32mb
memcap-policy: ignore # in IPS mode, what to do if memcap is reached
hash-size: 65536
trackers: 65535 # number of defragmented flows to follow
max-frags: 65535 # number of fragments do keep (higher than trackers)
prealloc: yes
timeout: 60
Flow and Stream handling
------------------------
.. _suricata-yaml-flow-settings:
Flow Settings
~~~~~~~~~~~~~
Within Suricata, Flows are very important. They play a big part in the
way Suricata organizes data internally. A flow is a bit similar to a
connection, except a flow is more general. All packets having the same
Tuple (protocol, source IP, destination IP, source-port,
destination-port), belong to the same flow. Packets belonging to a
flow are connected to it internally.
*Example 9 Flow*
.. image:: suricata-yaml/flow.png
*Example 10 Tuple*
.. image:: suricata-yaml/Tuple1.png
Keeping track of all these flows, uses memory. The more flows, the
more memory it will cost.
To keep control over memory usage, there are several options:
The option memcap for setting the maximum amount of bytes the
flow-engine will use, hash-size for setting the size of the hash-table
and prealloc for the following:
For packets not yet belonging to a flow, Suricata creates a
new flow. This is a relative expensive action. The risk coming
with it, is that attackers /hackers can a attack the engine
system at this part. When they make sure a computer gets a lot
of packets with different tuples, the engine has to make a lot
of new flows. This way, an attacker could flood the system. To
mitigate the engine from being overloaded, this option
instructs Suricata to keep a number of flows ready in
memory. This way Suricata is less vulnerable to these kind of
attacks.
The flow-engine has a management thread that operates independent from
the packet processing. This thread is called the flow-manager. This
thread ensures that wherever possible and within the memcap. There
will be 10000 flows prepared.
In IPS mode, a memcap-policy exception policy can be set, telling Suricata
what to do in case memcap is hit: 'drop-packet', 'pass-packet', 'reject', or
'ignore'.
::
flow:
memcap: 33554432 #The maximum amount of bytes the flow-engine will make use of.
memcap-policy: bypass #How to handle the flow if memcap is reached (IPS mode)
hash-size: 65536 #Flows will be organized in a hash-table. With this option you can set the
#size of the hash-table.
prealloc: 10000 #The amount of flows Suricata has to keep ready in memory.
rate-tracking: #Enable tracking of flows by the following rate definition; mark them
#as elephant flows if they exceed the defined rate. Disabled by default.
bytes: 1GiB #Number of bytes to track
interval: 10 #Time interval in seconds for which tracking should be done
At the point the memcap will still be reached, despite prealloc, the
flow-engine goes into the emergency-mode. In this mode, the engine
will make use of shorter time-outs. It lets flows expire in a more
aggressive manner so there will be more space for new Flows.
``emergency-recovery`` defines the percentage of flows that the engine needs to
prune before clearing the **emergency mode**. The default ``emergency-recovery``
value is 30. This is the percentage of prealloc'd flows after which the flow
-engine will be back to normal (when 30 percent of the 10000 flows are
completed).
If during the **emergency-mode** the aggressive time-outs do not
have the desired result, this option is the final resort. It
ends some flows even if they have not reached their time-outs
yet.
::
emergency-recovery: 30 #Percentage of 10000 prealloc'd flows.
Flow Time-Outs
~~~~~~~~~~~~~~
The amount of time Suricata keeps a flow in memory is determined by
the Flow time-out.
There are different states in which a flow can be. Suricata
distinguishes three flow-states for TCP and two for UDP. For TCP,
these are: New, Established and Closed,for UDP only new and
established. For each of these states Suricata can employ different
timeouts.
The state new in a TCP-flow, means the period during the three way
handshake. The state established is the state when the three way
handshake is completed. The state closed in the TCP-flow: there a
several ways to end a flow. This is by means of Reset or the Four-way
FIN handshake.
New in a UDP-flow: the state in which packets are send from only one
direction.
Established in a UDP-flow: packets are send from both directions.
In the example configuration the are settings for each protocol. TCP,
UDP, ICMP and default (all other protocols).
::
flow-timeouts:
default:
new: 30 #Time-out in seconds after the last activity in this flow in a New state.
established: 300 #Time-out in seconds after the last activity in this flow in a Established
#state.
emergency-new: 10 #Time-out in seconds after the last activity in this flow in a New state
#during the emergency mode.
emergency-established: 100 #Time-out in seconds after the last activity in this flow in a Established
#state in the emergency mode.
tcp:
new: 60
established: 3600
closed: 120
emergency-new: 10
emergency-established: 300
emergency-closed: 20
udp:
new: 30
established: 300
emergency-new: 10
emergency-established: 100
icmp:
new: 30
established: 300
emergency-new: 10
emergency-established: 100
.. _stream-engine-yaml:
Stream-engine
~~~~~~~~~~~~~
The Stream-engine keeps track of the TCP-connections. The engine
exists of two parts: The stream tracking- and the reassembly-engine.
The stream-tracking engine monitors the state of a connection. The
reassembly-engine reconstructs the flow as it used to be, so it will
be recognized by Suricata.
The stream-engine has two memcaps that can be set. One for the
stream-tracking-engine and one for the reassembly-engine. For both cases,
in IPS mode, an exception policy (memcap-policy) can be set, telling Suricata
what to do in case memcap is hit: 'drop-flow', 'drop-packet', 'pass-flow',
'pass-packet', 'bypass', 'reject', or 'ignore'.
The stream-tracking-engine keeps information of the flow in
memory. Information about the state, TCP-sequence-numbers and the TCP
window. For keeping this information, it can make use of the capacity
the memcap allows.
TCP packets have a so-called checksum. This is an internal code which
makes it possible to see if a packet has arrived in a good state. The
stream-engine will not process packets with a wrong checksum. This
option can be set off by entering 'no' instead of 'yes'.
::
stream:
memcap: 64mb # Max memory usage (in bytes) for TCP session tracking
memcap-policy: ignore # In IPS mode, call memcap policy if memcap is reached
checksum-validation: yes # Validate packet checksum, reject packets with invalid checksums.
To mitigate Suricata from being overloaded by fast session creation,
the option prealloc-sessions instructs Suricata to keep a number of
sessions ready in memory.
A TCP-session starts with the three-way-handshake. After that, data
can be sent and received. A session can last a long time. It can happen
that Suricata will be started after a few TCP sessions have already been
started. This way, Suricata misses the original setup of those
sessions. This setup always includes a lot of information. If you want
Suricata to check the stream from that time on, you can do so by
setting the option 'midstream' to 'true'. The default setting is
'false'. In IPS mode, it is possible to define a 'midstream-policy',
indicating whether Suricata should drop-flow, drop-packet, pass-flow,
pass-packet, reject, or bypass a midstream flow. The default is ignore.
Normally Suricata is able to see all packets of a connection. Some networks
make it more complicated though. Some of the network-traffic follows a
different route than the other part, in other words: the traffic goes
asynchronous. To make sure Suricata will check the one part it does see,
instead of getting confused, the option 'async-oneside' is brought to life. By
default the option is set to 'false'.
Suricata inspects content in the normal/IDS mode in chunks. In the
inline/IPS mode it does that on the sliding window way (see example
..) In the case Suricata is set in inline mode, it has to inspect
packets immediately before sending it to the receiver. This way
Suricata is able to drop a packet directly if needed.(see example …)
It is important for Suricata to note which operating system it is
dealing with, because operating systems differ in the way they process
anomalies in streams. See :ref:`host-os-policy`.
::
prealloc-sessions: 32768 # 32k sessions prealloc'd
midstream: false # do not allow midstream session pickups
midstream-policy: drop-flow # in IPS mode, drop flows that start midstream
async-oneside: false # do not enable async stream handling
inline: no # stream inline mode
drop-invalid: yes # drop invalid packets
bypass: no
The ``drop-invalid`` option can be set to no to avoid blocking packets that are
seen invalid by the streaming engine. This can be useful to cover some weird cases
seen in some layer 2 IPS setup.
The ``bypass`` option activates 'bypass' for a flow/session when either side
of the session reaches its ``depth``.
.. warning:: ``bypass`` can lead to missing important traffic. Use with care.
**Example 11 Normal/IDS mode**
Suricata inspects traffic in chunks.
.. image:: suricata-yaml/normal_ids.png
**Example 12 Inline/IPS Sliding Window**
Suricata inspects traffic in a sliding window manner.
.. image:: suricata-yaml/inline_mode.png
**Example 13 Normal/IDS (reassembly on ACK'D data)**
.. image:: suricata-yaml/Normal_ids_ack_d.png
**Example 14 Inline/IPS (reassembly on UNACK'D data)**
.. image:: suricata-yaml/Inline_reassembly_unackd_data.png
The reassembly-engine has to keep data segments in memory in order to
be able to reconstruct a stream. To avoid resource starvation a memcap
is used to limit the memory used. In IPS mode, an exception policy
(memcap-policy) can be set, telling Suricata what to do in case memcap
is hit: 'drop-flow', 'drop-packet', 'pass-flow', 'pass-packet', 'bypass',
'reject', or 'ignore'.
Reassembling a stream is an expensive operation. With the option depth
you can control how far into a stream reassembly is done. By default
this is 1MB. This setting can be overridden per stream by the protocol
parsers that do file extraction.
Inspection of reassembled data is done in chunks. The size of these
chunks is set with ``toserver-chunk-size`` and ``toclient-chunk-size``.
To avoid making the borders predictable, the sizes can be varied by
adding in a random factor.
::
reassembly:
memcap: 256mb # Memory reserved for stream data reconstruction (in bytes)
memcap-policy: ignore # What to do when memcap for reassembly is hit
depth: 1mb # The depth of the reassembling.
toserver-chunk-size: 2560 # inspect raw stream in chunks of at least this size
toclient-chunk-size: 2560 # inspect raw stream in chunks of at least
randomize-chunk-size: yes
#randomize-chunk-range: 10
'Raw' reassembly is done for inspection by simple ``content``, ``pcre``
keywords use and other payload inspection not done on specific protocol
buffers like ``http_uri``. This type of reassembly can be turned off:
::
reassembly:
raw: no
Incoming segments are stored in a list in the stream. To avoid constant
memory allocations a per-thread pool is used.
::
reassembly:
segment-prealloc: 2048 # pre-alloc 2k segments per thread
Resending different data on the same sequence number is a way to confuse
network inspection.
::
reassembly:
check-overlap-different-data: true
*Example 15 Stream reassembly*
.. image:: suricata-yaml/reassembly1.png
.. image:: suricata-yaml/IDS_chunk_size.png
TCP Urgent Handling
^^^^^^^^^^^^^^^^^^^
TCP Urgent pointer support is a complicated topic, where it is essentially impossible
for a network device to know with certainty what the behavior of the receiving host is.
For this reason, many middleboxes strip the URG flag and reset the urgent pointer (see
for example RFC 6093, 3.4).
Several options are provided to control how to deal with the urgent pointer.
::
stream:
reassembly:
urgent:
policy: oob # drop, inline, oob (1 byte, see RFC 6093, 3.1), gap
oob-limit-policy: drop
`stream.reassembly.urgent.policy`:
- `drop`: drop URG packets before they affect the stream engine
- `inline`: ignore the urgent pointer and process all data inline
- `oob` (out of band): treat the last byte as out of band
- `gap`: skip the last byte, but do no adjust sequence offsets, leading to
gaps in the data
If the urgent policy is set to `oob`, there is an additional setting. Since OOB data does
advance the TCP sequence number, the stream engine tracks the number of bytes to make sure
no GAPs in the non-OOB data are seen by the app-layer parsers and detection engine. This
is currently limited to 64k per direction. If the number of OOB bytes exceeds that 64k, an
additional policy is triggered: `stream.reassembly.urgent.oob-limit-policy`.
`stream.reassembly.urgent.oob-limit-policy`:
- `drop`: drop URG packets before they affect the stream engine
- `inline`: ignore the urgent pointer and process all data inline
- `gap`: skip the last byte, but do no adjust sequence offsets, leading to gaps in the data
Observables
"""""""""""
Each packet with the URG flag set, will increment the `tcp.urg` counter.
When dropping the URG packets, the packets will have the drop reason
`ips.drop_reason.stream_urgent`, which is also a counter in the stats logging.
The stream event `stream-event:reassembly_urgent_oob_limit_reached` allows matching on the
packet that reaches the OOB limit. Stream rule `2210066` matches on this.
If `stats.stream-events` are enabled the counter `stream.reassembly_urgent_oob_limit_reached`
will be incremented if the OOB limit is reached.
Host Tracking
-------------
.. _suricata-yaml-host-settings:
The Host table is used for tracking per IP address. This is used for tracking
per IP thresholding, per IP tagging, storing `iprep` data and storing `hostbit`.
Settings
~~~~~~~~
The configuration allows specifying the following settings: `hash-size`, `prealloc` and `memcap`.
.. code-block:: yaml
host:
hash-size: 4096
prealloc: 1000
memcap: 32mb
* `hash-size`: size of the hash table in number of rows
* `prealloc`: number of `Host` objects preallocated for efficiency
* `memcap`: max memory use for hosts, including the hash table size
Hosts are evicted from the hash table by the Flow Manager thread when all
data in the host is expired (tag, threshold, etc). Hosts with iprep will
not expire.
Application Layer Parsers
-------------------------
The ``app-layer`` section holds application layer specific configurations.
In IPS mode, a global exception policy accessed via the ``error-policy``
setting can be defined to indicate what the engine should do in case it
encounters an app-layer error. Possible values are "drop-flow", "pass-flow",
"bypass", "drop-packet", "pass-packet", "reject" or "ignore" (which maintains
the default behavior).
Each supported protocol has a dedicated subsection under ``protocols``.
.. note:: All applayer parsers can be enabled or disabled for specific carrier
protocols. Suricata first looks for carrier protocol specific setting and
if not found, falls back to the common enabled setting. e.g. if ``sip`` is
being registered, Suricata will first look if ``app-layer.protocols.sip.tcp.enabled``
and ``app-layer.protocols.sip.udp.enabled`` are set. If not, then a search would be
made for ``app-layer.protocols.sip.enabled`` and that setting would apply to both
SIP/TCP as well as SIP/UDP.
Asn1_max_frames
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Asn1 (`Abstract Syntax One
`_) is a
standard notation to structure and describe data.
Within Asn1-max-frames there are several frames. To protect itself,
Suricata will inspect a maximum of 256. You can set this amount
differently if wanted.
Application layer protocols such as X.400 electronic mail, X.500 and
LDAP directory services, H.323 (VoIP), BACnet and SNMP, use ASN.1 to
describe the protocol data units (PDUs) they exchange. It is also
extensively used in the Access and Non-Access Strata of UMTS.
Limit for the maximum number of asn1 frames to decode (default 256):
::
asn1-max-frames: 256
.. _suricata-yaml-configure-ftp:
FTP
~~~
The FTP application layer parser is enabled by default and uses dynamic protocol
detection.
By default, FTP control channel commands and responses are limited to 4096
bytes, but this value can be changed. When a command request or response exceeds
the line length limit, the stored data will be truncated, however the parser
will continue to watch for the end of line and acquire the next command.
Commands that are truncated will be noted in the *eve* log file with the fields
``command_truncated`` or ``reply_truncated``. Please note that this affects the
control messages only, not FTP data (file transfers).
::
ftp:
enabled: yes
#memcap: 64mb
# Maximum line length for control messages before they will be truncated.
#max-line-length: 4kb
.. _suricata-yaml-configure-libhtp:
Configure HTTP (libhtp)
~~~~~~~~~~~~~~~~~~~~~~~
The library Libhtp is being used by Suricata to parse HTTP-sessions.
While processing HTTP-traffic, Suricata has to deal with different
kind of servers which each process anomalies in HTTP-traffic
differently. The most common web-server is Apache. This is an open
source web-server program.
Besides Apache, IIS (Internet Information Services/Server) a web-server
program of Microsoft is also well-known.
Like with host-os-policy, it is important for Suricata to know which
IP-address/network-address is used by which server. In Libhtp this
assigning of web-servers to IP-and network addresses is called
personality.
Currently Available Personalities:
* Minimal
* Generic
* IDS (default)
* IIS_4_0
* IIS_5_0
* IIS_5_1
* IIS_6_0
* IIS_7_0
* IIS_7_5
* Apache
* Apache_2_2
You can assign names to each block of settings. Which in this case
is -apache and -iis7. Under these names you can set IP-addresses,
network-addresses the personality and a set of features.
The version-specific personalities know exactly how web servers
behave, and emulate that. The IDS personality would try to implement
a best-effort approach that would work reasonably well in the cases
where you do not know the specifics.
The default configuration also applies to every IP-address for which
no specific setting is available.
HTTP request bodies are often big, so they take a lot of time to
process which has a significant impact on the performance. With the
option 'request-body-limit' you can set the limit (in bytes) of the
client-body that will be inspected. Setting it to 0 will inspect all
of the body.
The same goes for HTTP response bodies.
::
libhtp:
default-config:
personality: IDS
request-body-limit: 3072
response-body-limit: 3072
server-config:
- apache:
address: [192.168.1.0/24, 127.0.0.0/8, "::1"]
personality: Apache_2_2
request-body-limit: 0
response-body-limit: 0
- iis7:
address:
- 192.168.0.0/24
- 192.168.10.0/24
personality: IIS_7_0
request-body-limit: 4096
response-body-limit: 8192
Suricata makes available the whole set of libhtp customisations for its users.
You can now use these parameters in the conf to customise suricata's
use of libhtp.
::
# Configures whether backslash characters are treated as path segment
# separators. They are not on Unix systems, but are on Windows systems.
# If this setting is enabled, a path such as "/one\two/three" will be
# converted to "/one/two/three". Accepted values - yes, no.
#path-convert-backslash-separators: yes
# Configures whether input data will be converted to lowercase.
#path-convert-lowercase: yes
# Configures how the server reacts to encoded NUL bytes.
#path-nul-encoded-terminates: no
# Configures how the server reacts to raw NUL bytes.
#path-nul-raw-terminates: no
# Configures whether consecutive path segment separators will be
# compressed. When enabled, a path such as "/one//two" will be normalized
# to "/one/two". The backslash_separators and decode_separators
# parameters are used before compression takes place. For example, if
# backslash_separators and decode_separators are both enabled, the path
# "/one\\/two\/%5cthree/%2f//four" will be converted to
# "/one/two/three/four". Accepted values - yes, no.
#path-separators-compress: yes
# Configures whether encoded path segment separators will be decoded.
# Apache does not do this, but IIS does. If enabled, a path such as
# "/one%2ftwo" will be normalized to "/one/two". If the
# backslash_separators option is also enabled, encoded backslash
# characters will be converted too (and subsequently normalized to
# forward slashes). Accepted values - yes, no.
#path-separators-decode: yes
# Configures whether %u-encoded sequences in path will be decoded. Such
# sequences will be treated as invalid URL encoding if decoding is not
# desireable. Accepted values - yes, no.
#path-u-encoding-decode: yes
# Configures how server reacts to invalid encoding in path. Accepted
# values - preserve_percent, remove_percent, decode_invalid, status_400
#path-url-encoding-invalid-handling: preserve_percent
# Controls whether the data should be treated as UTF-8 and converted
# to a single-byte stream using best-fit mapping
#path-utf8-convert-bestfit:yes
# Sets the replacement character that will be used to in the lossy
# best-fit mapping from Unicode characters into single-byte streams.
# The question mark is the default replacement character.
#path-bestfit-replacement-char: ?
# Configures whether plus characters are converted to spaces
# when decoding URL-encoded strings.
#query-plusspace-decode: yes
# response-body-decompress-layer-limit:
# Limit to how many layers of compression will be
# decompressed. Defaults to 2.
# uri-include-all: Include all parts of the URI. By default the
# 'scheme', username/password, hostname and port
# are excluded.
# meta-field-limit: Hard size limit for request and response size
# limits.
# inspection limits
request-body-minimal-inspect-size: 32kb
request-body-inspect-window: 4kb
response-body-minimal-inspect-size: 40kb
response-body-inspect-window: 16kb
# auto will use http-body-inline mode in IPS mode, yes or no set it statically
http-body-inline: auto
# Decompress SWF files.
# 2 types: 'deflate', 'lzma', 'both' will decompress deflate and lzma
# compress-depth:
# Specifies the maximum amount of data to decompress,
# set 0 for unlimited.
# decompress-depth:
# Specifies the maximum amount of decompressed data to obtain,
# set 0 for unlimited.
swf-decompression:
enabled: yes
type: both
compress-depth: 0
decompress-depth: 0
# Take a random value for inspection sizes around the specified value.
# This lower the risk of some evasion technics but could lead
# detection change between runs. It is set to 'yes' by default.
#randomize-inspection-sizes: yes
# If randomize-inspection-sizes is active, the value of various
# inspection size will be chosen in the [1 - range%, 1 + range%]
# range
# Default value of randomize-inspection-range is 10.
#randomize-inspection-range: 10
# Can enable LZMA decompression
#lzma-enabled: false
# Memory limit usage for LZMA decompression dictionary
# Data is decompressed until dictionary reaches this size
#lzma-memlimit: 1 Mb
# Maximum decompressed size with a compression ratio
# above 2048 (only reachable by LZMA)
#compression-bomb-limit: 1 Mb
# Maximum time spent decompressing a single transaction in usec
#decompression-time-limit: 100000
# Maximum number of live transactions per flow
#max-tx: 512
# Maximum used number of HTTP1 headers in one request or response
#headers-limit: 1024
Other parameters are customizable from Suricata.
::
# double-decode-path: Double decode path section of the URI
# double-decode-query: Double decode query section of the URI
decompression-time-limit
^^^^^^^^^^^^^^^^^^^^^^^^
decompression-time-limit was implemented to avoid DOS by resource exhaustion
on inputs such as decompression bombs (found by fuzzing).
The lower the limit, the better the protection against DOS is, but this
may also lead to false positives.
In case the time limit is reached,
the app-layer event ``http.compression_bomb`` is set
(this event can also set from other conditions).
This can happen on slow configurations (hardware, ASAN, etc...)
Configure SMB
~~~~~~~~~~~~~
The SMB parser will parse version 1, 2 and 3 of the SMB protocol over TCP.
To enable the parser add the following to the ``app-layer`` section of the YAML.
::
smb:
enabled: yes
detection-ports:
dp: 139, 445
The parser uses pattern based protocol detection and will fallback to ``probing parsers``
if the pattern based detection fails. As usual, the pattern based detection is port
independent. The ``probing parsers`` will only run on the ``detection-ports``.
SMB is commonly used to transfer the DCERPC protocol. This traffic is also handled by
this parser.
Resource limits
^^^^^^^^^^^^^^^
Several options are available for limiting record sizes and data chunk tracking.
::
smb:
enabled: yes
max-read-size: 8mb
max-write-size: 1mb
max-read-queue-size: 16mb
max-read-queue-cnt: 16
max-write-queue-size: 16mb
max-write-queue-cnt: 16
dcerpc:
max-stub-size: 1MiB
The `max-read-size` option can be set to control the max size of accepted
READ records. Events will be raised if a READ request asks for too much data
and/or if READ responses are too big. A value of 0 disables the checks.
The `max-write-size` option can be set to control the max size of accepted
WRITE request records. Events will be raised if a WRITE request sends too much
data. A value of 0 disables the checks.
Additionally if the `max-read-size` or `max-write-size` values in the
"negotiate protocol response" exceeds this limit an event will also be raised.
To control the size of the DCERPC stub data, `dcerpc.max-stub-size` should be
used. It is by default set to 1MiB.
For file tracking, extraction and file data inspection the parser queues up
out of order data chunks for both READs and WRITEs. To avoid using too much
memory the parser allows for limiting both the size in bytes and the number
of queued chunks.
::
smb:
enabled: yes
max-read-queue-size: 16mb
max-read-queue-cnt: 16
max-write-queue-size: 16mb
max-write-queue-cnt: 16
`max-read-queue-size` controls how many bytes can be used per SMB flow for
out of order READs. `max-read-queue-cnt` controls how many READ chunks can be
queued per SMB flow. Processing of these chunks will be blocked when any of
the limits are exceeded, and an event will be raised.
`max-write-queue-size` and `max-write-queue-cnt` are as the READ variants,
but then for WRITEs.
Cache limits
^^^^^^^^^^^^
The SMB parser uses several per flow caches to track data between different records
and transactions. These caches have a size ceiling. When the size limit is reached,
new additions will automatically evict the oldest entries.
::
smb:
max-guid-cache-size: 1024
max-rec-offset-cache-size: 128
max-tree-cache-size: 512
max-dcerpc-frag-cache-size: 128
max-session-cache-size: 512
The `max-guid-cache-size` setting controls the size of the hash that maps the GUID to
filenames. These are added through CREATE commands and removed by CLOSE commands.
`max-rec-offset-cache-size` controls the size of the hash that maps the READ offset
from READ commands to the READ responses.
The `max-tree-cache-size` option contols the size of the SMB session to SMB tree hash.
`max-dcerpc-frag-cache-size` controls the size of the hash that tracks partial DCERPC
over SMB records. These are buffered in this hash to only parse the DCERPC record when
it is fully reassembled.
The `max-session-cache-size` setting controls the size of a generic hash table that maps
SMB session to filenames, GUIDs and share names.
Configure DCERPC
~~~~~~~~~~~~~~~~
DCERPC has one parameter that can be customized.
`max-stub-size` is used to control the stub data size of a DCERPC request/response. By
default, it is set to 1MiB.
Configure HTTP2
~~~~~~~~~~~~~~~
HTTP2 has 2 parameters that can be customized.
The point of these 2 parameters is to find a balance between the completeness
of analysis and the resource consumption.
`http2.max-table-size` refers to `SETTINGS_HEADER_TABLE_SIZE` from rfc 7540 section 6.5.2.
Its default value is 4096 bytes, but it can be set to any uint32 by a flow.
`http2.max-streams` refers to `SETTINGS_MAX_CONCURRENT_STREAMS` from rfc 7540 section 6.5.2.
Its default value is unlimited.
SSL/TLS
~~~~~~~
SSL/TLS parsers track encrypted SSLv2, SSLv3, TLSv1, TLSv1.1 and TLSv1.2
sessions.
Protocol detection is done using patterns and a probing parser running
on only TCP/443 by default. The pattern based protocol detection is
port independent.
::
tls:
enabled: yes
detection-ports:
dp: 443
# What to do when the encrypted communications start:
# - track-only: keep tracking TLS session, check for protocol anomalies,
# inspect tls_* keywords. Disables inspection of unmodified
# 'content' signatures.
# - bypass: stop processing this flow as much as possible. No further
# TLS parsing and inspection. Offload flow bypass to kernel
# or hardware if possible.
# - full: keep tracking and inspection as normal. Unmodified content
# keyword signatures are inspected as well.
#
# For the best performance, select 'bypass'.
#
#encryption-handling: track-only
Encrypted traffic
^^^^^^^^^^^^^^^^^
There is no decryption of encrypted traffic, so once the handshake is complete
continued tracking of the session is of limited use. The ``encryption-handling``
option in ``app-layer.protocols.tls`` and ``app-layer.protocols.ssh`` controls
the behavior after the handshake.
If the ``encryption-handling`` property of the TLS/SSH configuration nodes are set to ``track-only`` (or are not set), Suricata will continue to track the respective SSL/TLS or SSH session. Inspection will be limited, as raw ``content`` inspection will still
be disabled. There is no point in doing pattern matching on traffic known to
be encrypted. Inspection for (encrypted) Heartbleed and other protocol
anomalies still happens.
When ``encryption-handling`` is set to ``bypass``, all processing of this
session is stopped. No further parsing and inspection happens. This will also
lead to the flow being bypassed, either inside Suricata or by the capture method
if it supports it and is configured for it.
Finally, if ``encryption-handling`` is set to ``full``, Suricata will process
the flow as normal, without inspection limitations or bypass.
The option has replaced the ``no-reassemble`` option. If ``no-reassemble`` is
present, and ``encryption-handling`` is not, ``false`` is interpreted as
``encryption-handling: track-only`` and ``true`` is interpreted as
``encryption-handling: bypass``.
SSH
~~~
Besides ``encryption-handling``, ssh parser offers the ``hassh`` option
with 3 values
- yes : enables hassh logging
- auto : hassh be enabled if rules use hassh keywords
- no : disables hassh and will refuse to load rules that use hassh keywords
Modbus
~~~~~~
According to MODBUS Messaging on TCP/IP Implementation Guide V1.0b, it
is recommended to keep the TCP connection opened with a remote device
and not to open and close it for each MODBUS/TCP transaction.
In that case, it is important to set the stream-depth of the modbus as
unlimited.
::
modbus:
# Stream reassembly size for modbus, default is 0
stream-depth: 0
MQTT
~~~~
The maximum size of a MQTT message is 256MB, potentially containing a lot of
payload data (such as properties, topics, or published payloads) that would end
up parsed and logged. To acknowledge the fact that most MQTT messages, however,
will be quite small and to reduce the potential for denial of service issues,
it is possible to limit the maximum length of a message that Suricata should
parse. Any message larger than the limit will just be logged with reduced
metadata, and rules will only be evaluated against a subset of fields. The
default is 1 MB.
::
mqtt:
max-msg-length: 1mb
SMTP
~~~~~~
SMTP parsers can extract files from attachments.
It is also possible to extract raw conversations as files with the
key ``raw-extraction``. Note that in this case the whole conversation
will be stored as a file, including SMTP headers and body content. The filename
will be set to "rawmsg". Usual file-related signatures will match on the raw
content of the email.
This configuration parameter has a ``false`` default value. It is
incompatible with ``decode-mime``. If both are enabled,
``raw-extraction`` will be automatically disabled.
::
smtp:
# extract messages in raw format from SMTP
raw-extraction: true
Maximum transactions
~~~~~~~~~~~~~~~~~~~~
SMTP, MQTT, FTP, PostgreSQL, SMB, DCERPC, HTTP1, ENIP and NFS have each a `max-tx`
parameter that can be customized.
`max-tx` refers to the maximum number of live transactions for each flow.
An app-layer event `protocol.too_many_transactions` is triggered when this value is reached.
The point of this parameter is to find a balance between the completeness of analysis
and the resource consumption.
When this threshold is reached, a new transaction will not be allocated,
and the flow will be put in error state, as it would become too expensive
in terms of CPU for Suricata to continue processing it.
For HTTP2, this parameter is named `max-streams` as an HTTP2 stream will get translated
into one Suricata transaction. This configuration parameter is used whatever the
value of `SETTINGS_MAX_CONCURRENT_STREAMS` negotiated between a client and a server
in a specific flow is.
Engine Logging
--------------
The engine logging system logs information about the application such
as errors and other diagnostic information during startup, runtime and
shutdown of the Suricata engine. This does not include Suricata
generated alerts and events.
The engine logging system has the following log levels:
- ``error``
- ``warning``
- ``notice``
- ``info``
- ``perf``
- ``config``
- ``debug``
Note that debug level logging will only be emitted if Suricata was
compiled with the ``--enable-debug`` configure option.
The first option within the logging configuration is the
``default-log-level``. This option determines the severity/importance
level of information that will be displayed. Messages of lower levels
than the one set here, will not be shown. The default setting is
``Notice``. This means that ``error``, ``warning`` and ``notice`` will be shown
and messages for the other levels won't be.
Default Configuration Example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
# Logging configuration. This is not about logging IDS alerts/events, but
# output about what Suricata is doing, like startup messages, errors, etc.
logging:
# The default log level, can be overridden in an output section.
# Note that debug level logging will only be emitted if Suricata was
# compiled with the --enable-debug configure option.
#
# This value is overridden by the SC_LOG_LEVEL env var.
default-log-level: notice
# The default output format. Optional parameter, should default to
# something reasonable if not provided. Can be overridden in an
# output section. You can leave this out to get the default.
#
# This console log format value can be overridden by the SC_LOG_FORMAT env var.
#default-log-format: "%D: %S: %M"
#
# For the pre-7.0 log format use:
#default-log-format: "[%i] %t [%S] - (%f:%l) <%d> (%n) -- "
# A regex to filter output. Can be overridden in an output section.
# Defaults to empty (no filter).
#
# This value is overridden by the SC_LOG_OP_FILTER env var.
default-output-filter:
# Define your logging outputs. If none are defined, or they are all
# disabled you will get the default - console output.
outputs:
- console:
enabled: yes
# type: json
- file:
enabled: yes
level: info
filename: suricata.log
# format: "[%i - %m] %z %d: %S: %M"
# type: json
- syslog:
enabled: no
facility: local5
format: "[%i] <%d> -- "
# type: json
Default Log Level
~~~~~~~~~~~~~~~~~
Example::
logging:
default-log-level: info
This option sets the default log level. The default log level is
`notice`. This value will be used in the individual logging
configuration (console, file, syslog) if not otherwise set.
.. note:: The ``-v`` command line option can be used to quickly
increase the log level at runtime. See :ref:`the -v command
line option `.
The ``default-log-level`` set in the configuration value can be
overridden by the ``SC_LOG_LEVEL`` environment variable.
Default Log Format
~~~~~~~~~~~~~~~~~~
A logging line exists of two parts. First it displays meta information
(Log-level, Suricata module), and finally the actual log message. Example:
::
i: suricata: This is Suricata version 7.0.2 RELEASE running in USER mode
(Here the part until the second `:` is the meta info,
"This is Suricata version 7.0.2 RELEASE running in USER mode" is the actual
message.)
It is possible to determine which information will be displayed in
this line and (the manner how it will be displayed) in which format it
will be displayed. This option is the so called format string::
default-log-format: "[%i] %t - (%f:%l) <%d> (%n) -- "
The % followed by a character has a special meaning. There are thirteen
specified signs:
::
z: ISO-like formatted timestamp: YYYY-MM-DD HH:MM:SS
t: Original Suricata log timestamp: DD/MM/YYYY -- HH:MM::SS
p: Process ID. Suricata's whole processing consists of multiple threads.
i: Thread ID. ID of individual threads.
m: Thread module name. (Outputs, Detect etc.)
d: Log-level of specific log-event. (Error, info, debug etc.)
D: Compact log format (E for Error, i for info etc.)
S: Subsystem name.
T: Thread name.
M: Log message body.
f: Name of source code filename where log-event is generated.
l: Line-number within the source filename, where the log-event is generated.
n: Function-name in the source code.
The last three options, f, l and n, are mainly convenient for developers.
The log-format can be overridden in the command line by the
environment variable: ``SC_LOG_FORMAT``.
Output Filter
~~~~~~~~~~~~~
Within logging you can set an output-filter. With this output-filter
you can set which part of the event-logs should be displayed. You can
supply a regular expression (Regex). A line will be shown if the regex
matches.
::
default-output-filter: # In this option the regular expression can be entered.
This value is overridden by the environment var: ``SC_LOG_OP_FILTER``.
Logging Outputs
~~~~~~~~~~~~~~~
There are different ways of displaying output. The output can appear
directly on your screen, it can be placed in a file or via syslog. The
last mentioned is an advanced tool for log-management. The tool can be
used to direct log-output to different locations (files, other
computers etc.)
::
outputs:
- console: # Output to screen (stdout/stderr).
enabled: yes # This option is enabled.
#level: notice # Use a different level than the default.
- file: # Output stored in a file.
enabled: no # This option is not enabled.
filename: /var/log/suricata.log # Filename and location on disc.
level: info # Use a different level than the default.
- syslog: # Output using syslog.
enabled: no # The use of this program is not enabled.
facility: local5 # Syslog facility to use.
format: "[%i] <%d> -- " # Output format specific to syslog.
#level: notice # Use a different level than the default.
Packet Acquisition
------------------
.. _dpdk-capture-module:
Data Plane Development Kit (DPDK)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
`Data Plane Development Kit `_
provides a generic interface to low-level resources. It is a unique way how
DPDK libraries access NICs. EAL creates an API for an application to access NIC
resources from the userspace level. In DPDK, packets are not retrieved via
interrupt handling. Instead, the application `polls
`_ the NIC for newly
received packets.
DPDK allows the user space application to directly access memory where the NIC
stores the packets. As a result, neither DPDK nor the application copies the
packets for the inspection. The application directly processes packets via
passed packet descriptors.
.. figure:: suricata-yaml/dpdk.png
:align: center
:alt: DPDK basic architecture
:figclass: align-center
`High-level overview of DPDK application`
To use DPDK capture module, Suricata must be compiled with DPDK option enabled.
Support for DPDK can be enabled in configure step of the build process such as:
.. code-block:: bash
./configure --enable-dpdk
Suricata makes use of DPDK for packet acquisition in workers runmode.
The whole DPDK configuration resides in the ``dpdk:`` node. This node
encapsulates 2 main subnodes, and those are eal-params and interfaces.
::
dpdk:
eal-params:
proc-type: primary
allow: ["0000:3b:00.0", "0000:3b:00.1"]
interfaces:
- interface: 0000:3b:00.0
threads: auto
promisc: true
multicast: true
checksum-checks: true
checksum-checks-offload: true
vlan-strip-offload: true
linkup-timeout: 10
mtu: 1500
mempool-size: auto
mempool-cache-size: auto
rx-descriptors: auto
tx-descriptors: auto
copy-mode: none
copy-iface: none # or PCIe address of the second interface
The `DPDK arguments
`_, which
are typically provided through the command line, are contained in the node
``dpdk.eal-params``. EAL is configured and initialized using these
parameters. There are two ways to specify arguments: lengthy and short.
Dashes are omitted when describing the arguments. This setup node can be
used to set up the memory configuration, accessible NICs, and other EAL-related
parameters, among other things. The node ``dpdk.eal-params`` also supports
multiple arguments of the same type. This can be useful for EAL arguments
such as ``--vdev``, ``--allow``, or ``--block``. Values for these EAL arguments
are specified as a comma-separated list.
An example of such usage can be found in the example above where the ``allow``
argument only makes ``0000:3b:00.0`` and ``0000:3b:00.1`` accessible to
Suricata.
arguments with list node. such as --vdev, --allow, --block eal options.
The definition of lcore affinity as an EAL
parameter is a standard practice. However, lcore parameters like ``-l``, ``-c``,
and ``--lcores`` are specified within the `suricata-yaml-threading`_ section
to prevent configuration overlap.
The node ``dpdk.interfaces`` wraps a list of interface configurations. Items on
the list follow the structure that can be found in other capture interfaces.
The individual items contain the usual configuration options
such as ``threads`` / ``copy-mode`` / ``checksum-checks`` settings. Other capture
interfaces, such as AF_PACKET, rely on the user to ensure that NICs are
appropriately configured.
Configuration through the kernel does not apply to applications running under
DPDK. The application is solely responsible for the initialization of the NICs
it is using. So, before the start of Suricata, the NICs that Suricata uses,
must undergo the process of initialization.
As a result, there are extra configuration options (how NICs can be
configured) in the items (interfaces) of the ``dpdk.interfaces`` list.
At the start of the configuration process, all NIC offloads are disabled to
prevent any packet modification. According to the configuration, checksum
validation offload can be enabled to drop invalid packets. Other offloads can
not currently be enabled.
Additionally, the list items in ``dpdk.interfaces`` contain DPDK specific
settings such as ``mempool-size`` or ``rx-descriptors``. These settings adjust
individual parameters of EAL. One of the entries in ``dpdk.interfaces`` is
the ``default`` interface. When loading interface configuration and some entry
is missing, the corresponding value of the ``default`` interface is used.
The worker threads must be assigned to specific cores. The configuration
module ``threading`` must be used to set thread affinity.
Worker threads can be pinned to cores in the array configured in
``threading.cpu-affinity["worker-cpu-set"]``. Performance-oriented setups have
everything (the NIC, memory, and CPU cores interacting with the NIC) based on
one NUMA node.
It is therefore required to know the layout of the server architecture to get the
best results. The CPU core ids and NUMA locations can be determined for example
from the output of ``/proc/cpuinfo`` where ``physical id`` described the NUMA
number. The NUMA node to which the NIC is connected to can be determined from
the file ``/sys/class/net//device/numa_node``.
::
## Check ids and NUMA location of individual CPU cores
cat /proc/cpuinfo | grep 'physical id\|processor'
## Check NUMA node of the NIC
## cat /sys/class/net//device/numa_node e.g.
cat /sys/class/net/eth1/device/numa_node
Suricata operates in workers runmode. Packet distribution relies on Receive
Side Scaling (RSS), which distributes packets across the NIC queues.
Individual Suricata workers then poll packets from the NIC queues.
Internally, DPDK runmode uses a `symmetric hash (0x6d5a)
`_.
The Intel NICs have the amount of RX/TX descriptors capped at 4096.
This should be possible to change by manually compiling the DPDK while
changing the value of respective macros for the desired drivers
(e.g. IXGBE_MAX_RING_DESC/I40E_MAX_RING_DESC).
DPDK is natively supported by Mellanox and thus their NICs should work
"out of the box".
**Current DPDK support** involves Suricata running on:
* a physical machine with a physical NICs such as:
* mlx5 (ConnectX-4/ConnectX-5/ConnectX-6)
* ixgbe
* i40e
* ice
* a virtual machine with virtual interfaces such as:
* e1000
* VMXNET3
* virtio-net
Other NICs using the same driver as mentioned above should work as well.
The DPDK capture interface has not been tested neither with the virtual
interfaces nor in the virtual environments like VMs, Docker or similar.
The minimal supported DPDK is version 19.11 which should be available in most
repositories of major distributions.
Alternatively, it is also possible to use ``meson`` and ``ninja`` to build and
install DPDK from source files.
It is required to have correctly configured tool ``pkg-config`` as it is used to
load libraries and CFLAGS during the Suricata configuration and compilation.
This can be tested by querying DPDK version as:
::
pkg-config --modversion libdpdk
Pf-ring
~~~~~~~
The Pf_ring is a library that aims to improve packet capture
performance over libcap. It performs packet acquisition. There are
three options within Pf_ring: interface, cluster-id and cluster-type.
::
pfring:
interface: eth0 # In this option you can set the network-interface
# on which you want the packets of the network to be read.
Pf_ring will load balance packets based on flow. All packet
acquisition threads that will participate in the load balancing need
to have the same cluster-id. It is important to make sure this ID is
unique for this cluster of threads, so that no other engine / program
is making use of clusters with the same id.
::
cluster-id: 99
Pf_ring can load balance traffic using pf_ring-clusters. All traffic
for pf_ring can be load balanced according to the configured cluster
type value; in a round robin manner or a per flow manner that are part
of the same cluster. All traffic for pf_ring will be load balanced across
acquisition threads of the same cluster id.
The "inner" flow means that the traffic will be load balanced based on
address tuple after the outer vlan has been removed.
+----------------------------+--------------------------------------------------+
| Cluster Type | Value |
+============================+==================================================+
| cluster_flow | src ip, src_port, dst ip, dst port, proto, vlan |
+----------------------------+--------------------------------------------------+
| cluster_inner_flow | src ip, src port, dst ip, dst port, proto, vlan |
+----------------------------+--------------------------------------------------+
| cluster_inner_flow_2_tuple | src ip, dst ip |
+----------------------------+--------------------------------------------------+
| cluster_inner_flow_4_tuple | src ip, src port, dst ip, dst port |
+----------------------------+--------------------------------------------------+
| cluster_inner_flow_5_tuple | src ip, src port, dst ip, dst port, proto |
+----------------------------+--------------------------------------------------+
| cluster_round_robin | not recommended |
+----------------------------+--------------------------------------------------+
The cluster_round_robin manner is a way of distributing packets one at
a time to each thread (like distributing playing cards to fellow
players). The cluster_flow manner is a way of distributing all packets
of the same flow to the same thread. The flows itself will be
distributed to the threads in a round-robin manner.
If your deployment has VLANs, the cluster types with "inner" will use the innermost
address tuple for distribution.
The default cluster type is ``cluster_flow``; the ``cluster_round_robin`` is not recommended with Suricata.
::
cluster-type: cluster_inner_flow_5_tuple
.. _suricata-yaml-nfq:
NFQ
~~~
Using NFQUEUE in iptables rules, will send packets to Suricata. If the
mode is set to 'accept', the packet that has been send to Suricata by
a rule using NFQ, will by default not be inspected by the rest of the
iptables rules after being processed by Suricata. There are a few more
options to NFQ to change this if desired.
If the mode is set to 'repeat', the packets will be marked by Suricata
and be re-injected at the first rule of iptables. To mitigate the
packet from being going round in circles, the rule using NFQ will be
skipped because of the mark.
If the mode is set to 'route', you can make sure the packet will be
send to another tool after being processed by Suricata. It is possible
to assign this tool at the mandatory option 'route_queue'. Every
engine/tool is linked to a queue-number. This number you can add to
the NFQ rule and to the route_queue option.
Add the numbers of the options repeat_mark and route_queue to the NFQ-rule::
iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
::
nfq:
mode: accept #By default the packet will be accepted or dropped by Suricata
repeat-mark: 1 #If the mode is set to 'repeat', the packets will be marked after being
#processed by Suricata.
repeat-mask: 1
route-queue: 2 #Here you can assign the queue-number of the tool that Suricata has to
#send the packets to after processing them.
*Example 1 NFQ1*
mode: accept
.. image:: suricata-yaml/NFQ.png
*Example 2 NFQ*
mode: repeat
.. image:: suricata-yaml/NFQ1.png
*Example 3 NFQ*
mode: route
.. image:: suricata-yaml/NFQ2.png
Ipfw
~~~~
Suricata does not only support Linux, it supports the FreeBSD
operating system (this is an open source Unix operating system) and
Mac OS X as well. The in-line mode on FreeBSD uses ipfw (IP-firewall).
Certain rules in ipfw send network-traffic to Suricata. Rules have
numbers. In this option you can set the rule to which the
network-traffic will be placed back. Make sure this rule comes after
the one that sends the traffic to Suricata, otherwise it will go
around in circles.
The following tells the engine to re-inject packets back into the ipfw
firewall at rule number 5500:
::
ipfw:
ipfw-reinjection-rule-number: 5500
*Example 16 Ipfw-reinjection.*
.. image:: suricata-yaml/ipfw_reinjection.png
Rules
-----
Rule Files
~~~~~~~~~~
Suricata by default is setup for rules to be managed by Suricata-Update with
the following rule file configuration:
.. code-block:: yaml
default-rule-path: /var/lib/suricata/rules
rule-files:
- suricata.rules
A default installation of Suricata-Update will write out the rules to
/var/lib/suricata/rules/suricata.rules.
You may want to edit this section if you are not using Suricata-Update or want
to add rule files that are not managed by Suricata-Update, for example:
.. code-block:: yaml
default-rule-path: /var/lib/suricata/rules
rule-files:
- suricata.rules
- /etc/suricata/rules/custom.rules
File names can be specific with an absolute path, or just the base name. If
just the base name is provided it will be looked for in the
``default-rule-path``.
If a rule file cannot be found, Suricata will log a warning message and
continue to load, unless ``--init-errors-fatal`` has been specified on the
command line, in which case Suricata will exit with an error code.
For more information on rule management see :doc:`../rule-management/index`.
Threshold-file
~~~~~~~~~~~~~~
Within this option, you can state the directory in which the
threshold-file will be stored. The default directory is:
/etc/suricata/threshold.config
Classifications
~~~~~~~~~~~~~~~
The Classification-file is a file which makes the purpose of rules
clear.
Some rules are just for providing information. Some of them are to
warn you for serious risks like when you are being hacked etc.
In this classification-file, there is a part submitted to the rule to
make it possible for the system-administrator to distinguish events.
A rule in this file exists of three parts: the short name, a
description and the priority of the rule (in which 1 has the highest
priority and 4 the lowest).
You can notice these descriptions returning in the rule and events / alerts.
::
Example:
configuration classification: misc-activity,Misc activity,3
Rule:
alert tcp $HOME_NET 21 -> $EXTERNAL_NET any (msg:"ET POLICY FTP Login Successful (non-anonymous)";
flow:from_server,established;flowbits:isset,ET.ftp.user.login; flowbits:isnotset,ftp.user.logged_in;
flowbits:set,ftp.user.logged_in; content:"230 ";pcre:!"/^230(\s+USER)?\s+(anonymous|ftp)/smi";
classtype:misc-activity; reference:urldoc.emergingthreats.net/2003410,;
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_FTP_Login; sid:2003410; rev:7;)
Event/Alert:
10/26/10-10:13:42.904785 [**] [1:2003410:7] ET POLICY FTP Login Successful (non-anonymous) [**]
[Classification: Misc activity[Priority: 3] {TCP} 192.168.0.109:21 -> x.x.x.x:34117
You can set the direction of the classification configuration.
::
classification-file: /etc/suricata/classification.config
.. _suricata-yaml-rule-vars:
Rule-vars
~~~~~~~~~
There are variables which can be used in rules.
Within rules, there is a possibility to set for which IP-address the
rule should be checked and for which IP-address it should not.
This way, only relevant rules will be used. To prevent you from having
to set this rule by rule, there is an option in which you can set the
relevant IP-address for several rules. This option contains the
address group vars that will be passed in a rule. So, after HOME_NET
you can enter your home IP-address.
::
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]" #By using [], it is possible to set
#complicated variables.
EXTERNAL_NET: any
HTTP_SERVERS: "$HOME_NET" #The $-sign tells that what follows is
#a variable.
SMTP_SERVERS: "$HOME_NET"
SQL_SERVERS: "$HOME_NET"
DNS_SERVERS: "$HOME_NET"
TELNET_SERVERS: "$HOME_NET"
AIM_SERVERS: any
It is a convention to use upper-case characters.
There are two kinds of variables: Address groups and Port-groups. They
both have the same function: change the rule so it will be relevant to
your needs.
In a rule there is a part assigned to the address and one to the
port. Both have their variable.
All options have to be set. If it is not necessary to set a specific
address, you should enter 'any'.
::
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
ORACLE_PORTS: 1521
SSH_PORTS: 22
SIP_PORTS: "[5060, 5061]"
.. _host-os-policy:
Host-os-policy
~~~~~~~~~~~~~~
Operating systems differ in the way they process fragmented packets
and streams. Suricata performs differently with anomalies for
different operating systems. It is important to set of which operating
system your IP-address makes use of, so Suricata knows how to process
fragmented packets and streams. For example in stream-reassembly there
can be packets with overlapping payloads.
*Example 17 Overlapping payloads*
.. image:: suricata-yaml/overlap.png
In the configuration-file, the operating-systems are listed. You can
add your IP-address behind the name of the operating system you make
use of.
::
host-os-policy:
windows: [0.0.0.0/0]
bsd: []
bsd-right: []
old-linux: []
linux: [10.0.0.0/8, 192.168.1.100, "8762:2352:6241:7245:E000:0000:0000:0000"]
old-solaris: []
solaris: ["::1"]
hpux10: []
hpux11: []
irix: []
macos: []
vista: []
windows2k3: []
Suricata as a Firewall options (experimental)
---------------------------------------------
It is possible to run Suricata as a firewall.
Please read :ref:`Firewall Mode Design ` before using this.
The existing yaml configuration options are listed below. For more examples on default policy configuration by app-layer hook, check :ref:`firewall examples-default policies-http`.
If the engine is run in firewall mode, dedicated stats counters will be added to
the stats logs. To see the stats reported for the firewall mode, refer to :ref:`firewall mode stats`.
::
firewall:
# toggle to enable firewall mode
#enabled: no
# Firewall rule file are in their own path and are not managed
# by Suricata-Update.
#rule-path: /etc/suricata/firewall/
# List of files with firewall rules. Order matters, files are loaded
# in order and rules are applied in that order (per state, see docs)
#rule-files:
# - firewall.rules
# Default policies
#
# Choose a default policy for each firewall hook.
# It is also possible to specify policies by app-layer protocol.
# DNS example: Drop and alert on all DNS requests that are not allowed in firewall.rules, accept all responses.
#
#policies:
# packet-filter: ["drop:packet"]
# dns:
# request-started: ["accept:hook"]
# request-complete: ["drop:flow", "alert"]
# response-started: ["accept:tx"]
Engine analysis and profiling
-----------------------------
Suricata offers several ways of analyzing performance of rules and the
engine itself.
.. _config:engine-analysis:
Engine-analysis
~~~~~~~~~~~~~~~
The option engine-analysis provides information for signature writers
about how Suricata organizes signatures internally.
Like mentioned before, signatures have zero or more patterns on which
they can match. Only one of these patterns will be used by the multi
pattern matcher (MPM). Suricata determines which patterns will be used
unless the fast-pattern rule option is used.
The option engine-analysis creates a new log file in the default log
dir. In this file all information about signatures and patterns can be
found so signature writers are able to see which pattern is used and
change it if desired.
To create this log file, you have to run Suricata with
./src/suricata -c suricata.yaml --engine-analysis.
::
engine-analysis:
rules-fast-pattern: yes
Example:
::
[10703] 26/11/2010 -- 11:41:15 - (detect.c:560) (SigLoadSignatures)
-- Engine-Analysis for fast_pattern printed to file - /var/log/suricata/rules_fast_pattern.txt
alert tcp any any -> any any (content:"Volume Serial Number"; sid:1292;)
== Sid: 1292 ==
Fast pattern matcher: content
Fast pattern set: no
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: Volume Serial Number
Final content: Volume Serial Number
---
alert tcp any any -> any any (content:"abc"; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: no
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: defghi
Final content: defghi
---
alert tcp any any -> any any (content:"abc"; fast_pattern:only; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: yes
Fast pattern chop set: no
Content negated: no
Original content: abc
Final content: abc
---
alert tcp any any -> any any (content:"abc"; fast_pattern; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: abc
Final content: abc
---
alert tcp any any -> any any (content:"abc"; fast_pattern:1,2; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: yes
Fast pattern offset, length: 1, 2
Content negated: no
Original content: abc
Final content: bc
.. _rule-and-packet-profiling-settings:
Rule and Packet Profiling settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rule profiling is a part of Suricata to determine how expensive rules
are. Some rules are very expensive while inspecting traffic. Rule
profiling is convenient for people trying to track performance
problems and resolving them. Also for people writing signatures.
Compiling Suricata with rule-profiling will have an impact on
performance, even if the option is disabled in the configuration file.
To observe the rule-performance, there are several options.
::
profiling:
rules:
enabled: yes
This engine is not used by default. It can only be used if Suricata is
compiled with:
::
-- enable-profiling
At the end of each session, Suricata will display the profiling
statistics. The list will be displayed sorted.
This order can be changed as pleased. The choice is between ticks,
avgticks, checks, maxticks and matches. The setting of your choice
will be displayed from high to low.
The amount of time it takes to check the signatures, will be
administrated by Suricata. This will be counted in ticks. One tick is
one CPU computation. 3 GHz will be 3 billion ticks.
Beside the amount of checks, ticks and matches it will also display
the average and the maximum of a rule per session at the end of the
line.
The option Limit determines the amount of signatures of which the
statistics will be shown, based on the sorting.
::
sort: avgticks
limit: 100
Example of how the rule statistics can look like;
::
Rule Ticks % Checks Matches Max Tick Avg
Ticks
7560 107766621 0.02 138 37 105155334 780917.54
11963 1605394413 0.29 2623 1 144418923 612045.14
7040 1431034011 0.26 2500 0 106018209 572413.60
5726 1437574662 0.26 2623 1 115632900 548065.06
7037 1355312799 0.24 2562 0 116048286 529005.78
11964 1276449255 0.23 2623 1 96412347 486637.15
7042 1272562974 0.23 2623 1 96405993 485155.54
5719 1233969192 0.22 2562 0 106439661 481642.93
5720 1204053246 0.21 2562 0 125155431 469966.14
Packet Profiling
~~~~~~~~~~~~~~~~
::
packets:
# Profiling can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: yes #this option is enabled by default
filename: packet_stats.log #name of the file in which packet profiling information will be
#stored.
append: yes #If set to yes, new packet profiling information will be added to the
#information that was saved last in the file.
# per packet csv output
csv:
# Output can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: no #the sending of packet output to a csv-file is by default disabled.
filename: packet_stats.csv #name of the file in which csv packet profiling information will be
#stored
Packet profiling is enabled by default in suricata.yaml but it will
only do its job if you compiled Suricata with --enable profiling.
The filename in which packet profiling information will be stored, is
packet-stats.log. Information in this file can be added to the last
information that was saved there, or if the append option is set to
no, the existing file will be overwritten.
Per packet, you can send the output to a csv-file. This file contains
one line for each packet with all profiling information of that
packet. This option can be used only if Suricata is build
with --enable-profiling and if the packet profiling option is enabled
in yaml.
It is best to use runmode 'single' if you would like to profile the
speed of the code. When using a single thread, there is no situation
in which two threads have to wait for each other. When using two
threads, the time threads might have to wait for each other will be
taken in account when/during profiling packets. For more information
see :doc:`../performance/packet-profiling`.
Decoder
-------
Teredo
~~~~~~
The Teredo decoder can be disabled. It is enabled by default.
::
decoder:
# Teredo decoder is known to not be completely accurate
# it will sometimes detect non-teredo as teredo.
teredo:
enabled: true
# ports to look for Teredo. Max 4 ports. If no ports are given, or
# the value is set to 'any', Teredo detection runs on _all_ UDP packets.
ports: $TEREDO_PORTS # syntax: '[3544, 1234]'
Using this default configuration, Teredo detection will run on UDP port
1. If the `ports` parameter is missing, or set to `any`, all ports will be
inspected for possible presence of Teredo.
VXLAN
~~~~~
The VXLAN decoder can be configured with different reserved bits check modes.
It is enabled by default and uses UDP port 4789.
::
decoder:
# VXLAN decoder is assigned to up to 4 UDP ports. By default only the
# IANA assigned port 4789 is enabled.
vxlan:
enabled: true
ports: $VXLAN_PORTS # syntax: '[8472, 4789]' or '4789'.
# Reserved bits check mode. Possible values are:
# - strict: check all reserved bits are zero for standard VXLAN (default)
# - permissive: do not check any reserved bits (allows VXLAN extensions)
reserved-bits-check: strict
Using this default configuration, VXLAN detection will run on UDP port 4789
with strict reserved bits checking. The ``reserved-bits-check`` option controls
how strictly the decoder validates the VXLAN header:
- ``strict``: Validates all reserved bits are zero for standard VXLAN (default).
This mode follows RFC 7348 strictly and will reject VXLAN extensions like GBP.
- ``permissive``: Does not check any reserved bits. This mode accepts any
VXLAN-like traffic regardless of reserved bit values. This is mainly useful
when dealing with VXLAN extensions that may use reserved fields.
Recursion Level
~~~~~~~~~~~~~~~
Flow matching via recursion level can be disabled. It is enabled by
default.
::
decoder:
# Depending on packet pickup, incoming and outgoing tunnelled packets
# can be scanned before the kernel has stripped and encapsulated headers,
# respectively, leading to incoming and outgoing flows not being associated.
recursion-level:
use-for-tracking: true
Using this default setting, flows will be associated only if the compared packet
headers are encapsulated in the same number of headers.
Advanced Options
----------------
stacktrace
~~~~~~~~~~
Display diagnostic stacktraces when a signal unexpectedly terminates Suricata, e.g., such as
SIGSEGV or SIGABRT. Requires the ``libunwind`` library to be available. The default value is
to display the diagnostic message if a signal unexpectedly terminates Suricata -- e.g.,
``SIGABRT`` or ``SIGSEGV`` occurs while Suricata is running.
::
logging:
# Requires libunwind to be available when Suricata is configured and built.
# If a signal unexpectedly terminates Suricata, displays a brief diagnostic
# message with the offending stacktrace if enabled.
#stacktrace-on-signal: on
.. _deprecation policy: https://suricata.io/about/deprecation-policy/
.. _suricata-yaml-config-hardening:
Configuration hardening
-----------------------
The `security` section of suricata.yaml is meant to provide in-depth security configuration options.
Besides landlock, (see :ref:`landlock`), one setting is available.
`limit-noproc` is a boolean to prevent process creation by Suricata.
If you do not need Suricata to create other processes or threads
(you may need it for LUA scripts for instance or plugins), enable this to
call `setrlimit` with `RLIMIT_NPROC` argument (see `man setrlimit`).
This prevents potential exploits against Suricata to fork a new process,
even if it does not prevent the call of `exec`.
Warning! This has no effect on Linux when running as root. If you want a hardened configuration,
you probably want to set `run-as` configuration parameter so as to drop root privileges.
Beyond suricata.yaml, other ways to harden Suricata are
- compilation : enabling ASLR and other exploit mitigation techniques.
- environment : running Suricata on a device that has no direct access to Internet.
.. _suricata-yaml-lua-config:
Lua
~~~
Suricata 8.0 sandboxes Lua rules by default. The restrictions on the sandbox for Lua rules can be
modified in the ``security.lua`` section of the configuration file. This section also applies to
Lua transforms. Additionally, Lua rules can be completely disabled in the same way as for as the
Suricata 7.0 default:
::
security:
lua:
# Allow Lua rules. Enabled by default.
#allow-rules: true
# Upper bound of allocations by a Lua rule before it will fail
#max-bytes: 500000
# Upper bound of lua instructions by a Lua rule before it will fail
#max-instructions: 500000
# Allow dangerous lua operations like external packages and file io
#allow-restricted-functions: false
---