1. Project Overview & Quickstart (gravitl/netmaker)

File: docs/APIUsage.md

Most actions that can be performed via API can be performed via UI. We recommend managing your networks using the official netmaker-ui project. However, Netmaker can also be run without the UI, and all functions can be achieved via API calls. If your use case requires using Netmaker without the UI or you need to do some troubleshooting/advanced configuration, using the API directly may help.

File: docs/Authentication.md

API calls are primarily authenticated using a user authentication token. This token should be included in the header as follows:

-H "Authorization: Bearer "

To obtain YOUR_AUTH_TOKEN:
Call the api/users/adm/authenticate endpoint (see documentation below for details).

Note: While a MasterKey exists (configurable via env var or config file), it should be considered a backup option, used only when server access is lost. By default, this key is "secret key," but it's crucial to change this and keep it secure in your instance.

For more information on configuration and security best practices, refer to the Netmaker documentation.

File: docs/Pricing.md

Check out our Pricing. And Feel Free to Contact Us if you have any questions or need some clarifications.

File: k8s/server/README.md

Netmaker K8S YAML Templates - Run Netmaker on Kubernetes

This will walk you through setting up a highly-available netmaker deployment on Kubernetes. Note: do not attempt to control networking on a Kubernetes cluster where Netmaker is deployed. This can result in circular logic and unreachable networks! Typically, a cluster should be designated as a "control" cluster.

You may want a more simple Kubernetes setup. We recommend this community project. It is specific to K3S, but should be editable to work on most Kubernetes distributions.

0. Prerequisites

Your cluster must meet a few conditions to host a netmaker server. Primarily:
a) Nodes: You must have at least 3 worker nodes available for Netmaker to deploy. Netmaker nodes have anti-affinity and will not deploy on the same kubernetes node.
b) Storage: RWX and RWO storage classes must be available
c) Ingress: Ingress must be configured with certs. Nginx + LetsEncrypt configs are provided by default. Netmaker uses MQTT with Secure Websockets (WSS), and Nginx Ingress supports Websockets. If your ingress controller does not support websockets, you must configure your cluster to get traffic to MQ correctly (see below).
d) DNS: You must have a wildcard DNS entry for use with Ingress/Netmaker
e) Helm: For our Postgresql installation we rely on a helm chart, so you must have helm installed and configured.
f) MQ Broker Considerations: If your Ingress Controller does not support Websockets, we provide an alternative method for MQ using a NodePort. This can be used either with or without an external load balancer configured. If deploying without a load balancer, you must specify a node to host MQ, and this will not be HA. If using a load balancer, be aware that LB configuration could lead MQ connections to be lost. If this happens, on the client you will see a log like "unable to connect to broker, retrying ..." In either case DNS must be configured to point broker.domain either to the LB or directly to the hosting node. Finally, you can use a special TCPIngressRoute if Traefik is your Ingress provider (see this repo for an example). This is ideal, but is not a standard k8s object, so we avoid it to make installations possible across an array of k8s configurations.

Assuming you are prepared for the above, we can begin to deploy Netmaker.

1. Create Namespace

kubectl create ns netmaker
kubectl config set-context --current --namespace=netmaker

2. Deploy Database

Netmaker can use sqlite, postgres, or rqlite as a backing database. For HA, we recommend using Postgres, as Bitnami provides a reliable helm chart for deploying an HA pqsql cluster. See here for more details:

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install postgres bitnami/postgresql

Confirm the database is running:

kubectl get pods

Once completed, retrieve the password to access postgres:

kubectl get secret --namespace netmaker postgres-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d

3. Deploy MQTT

Based on the prerequisites, you will have one of the following scenarios. Configure accordingly:

text

a) **Nginx:** Uncomment the Ingress section from mosquitto.yaml and replace NETMAKER_SUBDOMAIN with your domain.

b) **External LB:** Configure an LB to load balance TLS traffic to the MQ service on port 8883. If using a port other than 443, change the value of MQ_PORT in netmaker-server.yaml. The LB must support Secure Websockets (WSS), which requires a valid TLS certificate.

c) **Traefik:** If you have Traefik as your Ingress provider, create a TCPIngressRoute from 443 to mq service on port 8883. [Example Here](https://github.com/geragcp/netmaker-k3s/blob/main/08-ingress.yaml)

Next, deploy MQ using the provided template (mosquitto.yaml). Modify the template:

text

a) **Nginx:** Remove the pod affinity section and the NodePort section. Uncomment the Ingress section at the bottom. Then, substitute in your wildcard domain:
    sed -i 's/NETMAKER_SUBDOMAIN/<your subdomain>/g' mosquitto.yaml
b or c) **Ex. LB or Traefik:** Remove the pod affinity section

Then, substitute in your RWX storage class:

sed -i 's/RWX_STORAGE_CLASS/<your storage class name>/g' mosquitto.yaml

Now, apply the file:

kubectl apply -f mosquitto.yaml

MQ should be in CrashLoopBackoff until Netmaker is deployed. If it's in pending state, check the pvc or the pod status (node selector may be incorrect).

4. Deploy Netmaker Server

Make sure Wildcard DNS is set up for a netmaker subdomain, for instance: nm.mydomain.com. If you do not wish to use wildcard, edit the YAML file directly. Note you will need entries for broker.domain, api.domain, and dashboard.domain.

If you are using Nginx as your ingress controller, uncomment the Nginx section at the bottom of the yaml file.

sed -i 's/NETMAKER_SUBDOMAIN/<your subdomain>/g' netmaker-server.yaml

Next, enter your postgres info, including the name of your postgres deployment and the password you retrieved above.

sed -i 's/DB_NAME//g' netmaker-server.yaml

sed -i 's/DB_PASS//g' netmaker-server.yaml

Next, choose a secret password for your Netmaker API:

sed -i 's/REPLACE_MASTER_KEY/<super secret password>/g' netmaker-server.yaml

Finally, you will need to create an Ingress object for your Netmaker API. An example is included in the YAML file for nginx + letsencrypt. You may use/modify this example, or create your own ingress which routes to the netmaker-rest service on port 8081. But make sure to deploy Ingress before moving on!

Now, apply the file:

kubectl apply -f netmaker-server.yaml

Most likely, MQ will require a restart at this point:

kubectl delete pod mosquitto-

When successful, all netmaker pods should display the following log:

[netmaker] 2022-00-00 00:00:00 successfully connected to mq broker

5. Deploy Netmaker UI

Much like above, you must make sure wildcard DNS is configured and make considerations for Ingress. Once again, add in your subdomain:

sed -i 's/NETMAKER_SUBDOMAIN/<your subdomain>/g' netmaker-ui.yaml

Again, Ingress is commented out. If you are using Nginx + LetsEncrypt, you can uncomment and use the yaml. Otherwise, set up Ingress manually.

kubectl apply -f netmaker-ui.yaml

At this point, you should be able to reach the server at domain.yourdomain and start setting up your networks.

Troubleshooting

Sometimes, the server has a hard time connecting to MQ using the self-generated certs on the first try. If this happens, try the following:

restart MQ: kubectl delete pod <mq pod name>
restart netmaker pods:
2.a. kubectl scale sts netmaker --replicas=0
2.b. kubectl delete pods netmaker-0 netmaker-1 netmaker-2
2.c. kubectl scale sts netmaker --replicas=3

In addition, try deleting the certs in MQ before running the above:

kubectl exec -it mosquitto- /bin/sh
rm mosquitto/certs/*
exit
kubectl delete pod mosquitto-
kubectl scale sts netmaker --replicas=0 (wait until pods are down) kubectl scale sts netmaker --replicas=3

2. Official Technical Reference & Guides (gravitl/netmaker-docs)

File: README.md

Welcome to the Netmaker documentation!

Where is the netmaker documentation hosted?

https://docs.netmaker.org

We encourage contributions! Follow the below documentation to build and make changes/additions to the documentation.

How to build the Netmaker documentation

Install dependencies: make, python3, pip, sphinx, sphinx-material theme

text

(debian instructions, varies by OS)

apt-get install make  
apt-get install python3-sphinx
apt-get install python3-pip
pip install git+https://github.com/bashtage/sphinx-material.git
pip install markupsafe

Clone the repo
git clone https://github.com/gravitl/netmaker-docs.git
Cd to root of repo and build
cd netmaker-docs && make html
Open _build/html/index.html in your browser

How to contribute to the Netmaker documentation

Be sure to follow the sphinx documentation while writing your changes: https://documentation-style-guide-sphinx.readthedocs.io/en/latest/style-guide.html

Follow the "build" instructions to clone and build the documentation
Create a branch: git checkout -b <your-doc-change>
Write your changes in the editor of your choice.
Build the documentation again to make sure it compiles: make html
View it in your browser to make sure it looks good
Once ready, commit your code and submit a PR against the develop branch, describing what changes you made.

Submitting Changes

Please label your branch using a sensible name.
Please open a Pull Request against the develop branch with your branch which clearly describes everything you've done and references any related GitHub issues.
Please respond to any feedback in a timely manner. Stale PR's will be closed periodically.

File: about.rst

========== About

.. image:: images/netmaker-simple.png
:width: 60%
:alt: Netmaker Architecture Diagram
:align: center

What is Netmaker?

Netmaker is a tool for creating and managing virtual overlay networks. If you have at least two machines with internet access that you need to connect with a secure tunnel, Netmaker is for you. If you have thousands of servers spread across multiple locations, data centers, or clouds, Netmaker is also for you. Netmaker connects machines securely, wherever they are.

.. image:: images/mesh-diagram.png
:width: 50%
:alt: WireGuard Mesh
:align: center

Netmaker takes these machines and creates a flat network so that they can all talk to each other easily and securely.
If you're familiar with AWS, it's like a VPC but made up of arbitrary computers. From the machine's perspective, all these other machines are in the same neighborhood, even if they're spread all over the world.

Beyond creating a flat network, Netmaker introduces Remote Access Gateway and Egress, which are gateways that control the flow of traffic into and out of the network. Netmaker also has ACLs, which control exactly which machines can reach each other. The end result is you can create much more complex networks than a simple mesh.

.. image:: images/netmaker-advanced.png
:width: 80%
:alt: Netmaker Architecture Diagram
:align: center

Netmaker has many similarities to Tailscale, ZeroTier, and Nebula. What makes Netmaker different is the speed and flexibility. Netmaker is faster because it uses kernel WireGuard. It is more dynamic because the server and agents are fully configurable, which lets you handle all sorts of different use cases. And, of course, you can also self-host Netmaker, which gives you complete control of your network traffic.

How Does Netmaker Work?

Netmaker relies on WireGuard to create tunnels between machines. At its core, Netmaker is managing WireGuard across machines to create sensible networks. Technically, Netmaker is two things:

the admin server, called Netmaker
the agent, called Netclient

As the network manager, you interact with the server to create and manage networks and devices. The server holds configurations for these networks and devices and pushes updates to the agents (netclient) over a message queue.

The netclient is installed on any machine you would like to add to a given network, whether that machine is a VM, Server, Kubernetes node, or IoT device. The netclient subscribes to the MQ broker, and the server tells it how to configure WireGuard. The client will let the server know when any local changes (like ip address, port) should be pushed out to the other clients. By doing this across many machines simultaneously, we create fully dynamic and configurable virtual networks.

The Netmaker server does not typically route traffic, which would be a hub-and-spoke model (much slower). Instead, Netmaker just tells the machines on the network how they can reach each other directly. This is called a full mesh network and is much faster. Even if the server goes down, as long as none of the existing machines change substantially, your network will still run just fine.

Use Cases for Netmaker

There are many use cases for Netmaker. In fact, you could probably be using it right now. Because of Netmaker's extreme speed, there is almost no cost to putting a Netmaker overlay network on top of any existing Network. The question becomes, why not use Netmaker? You've already got the machines; why not encrypt the traffic and make your network dynamic?

This list is not all-encompassing but provides a sample of how many users use Netmaker in production today. Guided setup for many of these use cases can be found in the :doc:Using Netmaker <./usage> documentation.

Automate the creation of a large WireGuard-based network
Create a flat, secure network between multiple environments (VPCs, clouds, data centers)
Secure access to a home or office network
Provide remote access to resources like an AWS VPC, data center, or K8S cluster
Create distributed, cross-cloud Kubernetes clusters or database clusters
Manage a secure mesh of IoT devices

Architecture

.. toctree::
:maxdepth: 1

architecture

File: acls.rst

.. _acls:

====== ACLs

Introduction

.. image:: images/acls-3.png
:width: 80%
:alt: ACLs
:align: center

By default, Netmaker creates a "full mesh," meaning every node in your network can talk to every other node. You don't always want this to be the case. Sometimes, only some connections should be valid. That's why Netmaker has ACLs. By using Netmaker's ACL feature, you can enable/disable any peer-to-peer connection in your network to remove its ability to communicate.

Configuring ACLs

The ACL feature can be accessed by either clicking on "ACLs" in the sidebar, or by clicking on a Node in either the Node List or in the Graph view.

ACL Screen

If you are on the ACLs tab (pictured above), you will have a complete list of all ACLs for the whole network. You can simply click on any one of these to enable or disable a pair of connections. You will note that when you "disable" the connection from any one node, it will automatically "disable" the matching connection on the other node. An update is sent to the nodes over MQ, telling them to remove the connection locally.

In the upper right-hand corner of the screen, you will also notice an "Allow All" button and a "Block All" button. "Allow All" will enable all connections, and "Block All" will block all connections. If you block all connections, no nodes will be able to talk to each other.

For more information, see the :doc:UI reference. <./ui-reference>

Node ACL Screen

Alternatively, you can reach the individual node ACLs by clicking on a Node in either the Node List or the Graph view. This will give you the ACLs for just this individual node, making it easier to enable/disable peers at an individual level.

.. image:: images/acls-2.jpg
:width: 80%
:alt: ACLs
:align: center

Default Node-Level ACLs

When you create a network, you set a "default ACL" of either "Allow" or "Deny." This means all nodes either reach each other by default, or cannot reach each other by default.

As of 0.16.0, there is another option, the default node-level ACL.

.. image:: images/acls-4.jpg
:alt: ACLs
:align: center

This allows you to set the default connectivity status of an individual node. This is useful for two scenarios.

Scenario 1: Remote Access to/from Client devices

In many cases, you have many machines which should be reachable from one machine. OR, you have one machine which should be reachable from many machines.

Example 1: A file server for employees. Employees need to access the file server from their individual devices, but should not have access to each other.

Example 2: Managing client devices. An MSP or IT Shop needs to remote into client devices, but those devices definitely should not have access to each other.

For either of these scenarios, we should do the following:

Create a network with default "DENY" policy
Create a node with default "ALLOW" policy

Once this is done, all devices in the network will have access only to the device with the default "ALLOW" policy. However, you can still override this to allow access for specific other machines.

Scenario 2: Priviledged Access to Device

In some cases, you want a machine in your network to be accessible from some, but not all, of your devices. Or maybe you have an Egress Gateway on a device, and only want some devices to use it.

In this case, we can set up a network as follows:

Create a network with default "ALLOW" policy
Create a node with default "DENY" policy

Once this is done, all devices in the network will have access to everything except the device(s) with a defult DENY policy, which can be manually overridden in the ACL management pane.

File: advanced-client-install.rst

================================ Advanced Client Installation

This document tells you how to install the netclient on machines that will be a part of your Netmaker network, as well as non-compatible systems.

These steps should be run after the Netmaker server has been created and a network has been designated within Netmaker.

Introduction to Netclient

At its heart, the netclient is a simple CLI for managing access to various WireGuard-based networks. It manages WireGuard on the host system so that you don't have to. Why is this necessary?

If you are setting up a WireGuard-based virtual network, you must configure each machine with very specific settings, so that every machine can reach it, and it can reach every other machine. Any changes to the settings of any one of these machines can break those connections. Any machine that is added, removed, or modified on the network requires reconfiguring of every peer in the network. This can be very time-consuming.

The netmaker server holds configuration details about every machine in your network and how other machines should connect to it.

The netclient agent connects to the server, pushing and pulling information when the network (or its local configuration) changes.

The netclient agent then configures WireGuard (and other network properties) locally, so that the network stays intact.

Note on MTU Settings

IPv6 requires a minimum MTU of 1280. A lot of router configurations expect a standard MTU setting. You can adjust the MTU to whatever fits your needs, but setting the MTU below the standardized 1280 may cause wireguard to have issues when setting up interfaces with some systems like "Windows" for example.

Notes on Windows

If running the netclient on Windows, you must download the netclient.exe binary and run it from Powershell as an Administrator.

Windows will by default have firewall rules that prevent inbound connections. If you wish to allow inbound connections from particular peers, use the following command:

netsh advfirewall firewall add rule name="Allow from " dir=in action=allow protocol=ANY remoteip=

If you want to allow all peers access, but do not want to configure firewall rules for all peers, you can configure access for one peer, and set it as a Relay Server(Professional Edition Feature) from Netmaker GUI. To achieve this, a netmaker pro edition is required.

Running netclient commands

If running the netclient manually ("netclient install", "netclient join", "netclient pull") it should be run from outside of the installed directory, which will be either:

C:/Program Files/netclient
C:/ProgramData/netclient

It is better to call it from a different directory.

High CPU Utilization

With some versions of WireGuard on Windows, high CPU utilization has been found with the netclient. This is typically due to interaction with the WireGuard GUI component (app). If you're experiencing high CPU utilization, close the WireGuard app. WireGuard will still be running, but the CPU usage should go back down to normal.

Changing network profile to private

By default, the netmaker network profile is added as a public network. This is the default behaviour on Windows. To change it to private, please run the Powershell command,

Set-NetConnectionProfile -InterfaceAlias 'netmaker' -NetworkCategory 'Private'

Issue after Windows sleep/hibernate

Sometimes the netclient does not work after the Windows wake up from sleep/hibernation. The root cause is not identified. Restarting the netclient service can fix the issue.

Irregular netclient restart on Windows 2016 server

There is one issue reported on the Windows 2016 server. The netclient restarted irregularly. The root cause is not identified. However, as per the feedback from the client, the issue can be fixed by disabling the ISATAP adapter and 6to4 feature.

Set-Net6to4configuration -state disabled
Set-Netisatapconfiguration -state disabled
Set-NetTeredoConfiguration -type disabled

Event id 0 in Windows Event logs

netclient service is delegated to Winsw on Windows. An issue is reported that the stop/start/restart events in Event logs show the event ID as 0 always. It does not impact any netclient functions.

Modes and System Compatibility

Note: If you would like to connect non-Linux/Unix machines to your network such as phones and Windows desktops, please see the documentation on Remote Access Clients(previously External Clients)

The netclient can be run in a few "modes". System compatibility depends on which modes you intend to use. These modes can be mixed and matched across a network, meaning all machines do not have to run with the same "mode."

CLI

In its simplest form, the netclient can be treated as just a simple, manual, CLI tool, which a user can call to configure the machine. The CLI can be compiled from source code to run on most systems and has already been compiled for x86 and ARM architectures.

As a CLI, the netclient should function on any Linux or Unix-based system that has the wireguard utility (callable with wg) installed.

Daemon

The netclient is intended to be run as a system daemon. This allows it to automatically retrieve and send updates. To do this, the netclient can install itself as a systemd service, or launchd/windows service for Mac or Windows.

If running the netclient on non-systemd Linux, it is recommended to manually configure the netclient as a daemon using whatever method is acceptable on the chosen operating system.

Private DNS Management

To manage private DNS, the netclient relies on systemd-resolved (resolvectl). Absent this, it cannot set private DNS for the machine.

A user may choose to manually set a private DNS nameserver of :53. However, beware, as netmaker sets split DNS, the system must be configured properly. Otherwise, this nameserver may break your local DNS.

Prerequisites

To obtain the netclient, go to the GitHub releases: https://github.com/gravitl/netclient/releases

For netclient cli: Linux/Unix with WireGuard installed (wg command available)

For netclient daemon: Systemd Linux + WireGuard

For Private DNS management: Resolvectl (systemd-resolved)

Please refer to the Firewall Rules for Machines Running Netclient <https://docs.netmaker.io/install.html#firewall-rules-for-machines-running-netclient>_ for more information.

Configuration

The CLI has information about all commands and variables. This section shows the "help" output for these commands as well as some additional references.

CLI Reference

sudo netclient --help

.. literalinclude:: ./examplecode/netclient-help.txt
:language: YAML

sudo netclient join --help

.. literalinclude:: ./examplecode/netclient-join.txt
:language: YAML

sudo netclient push --help

.. literalinclude:: ./examplecode/netclient-push.txt
:language: YAML

Installation

To install netclient and join a network, you need to use netclient install command and get an enrollment key for a particular network from netmaker.

An admin creates an enrollment key in the "Enrollment Keys" section of the UI. Upon creating a key, it can be viewed by clicking on the key from UI. Some details regarding the key will be visible:

Key: The enrollment key to join and authenticate to a netmaker network

Type: The Type of key determines the usage limitation of a particular key. Possible values are: Unlimited, Time Bound, Limited Number of Uses

Expires at: Shows the expiration date of the particular enrollment key

Networks: Shows which netmaker networks can be joined by using the particular enrollment key

Install Command: The CLI command to register with the server using the enrollment key, and join the networks

For first-time installations, you can run the Install Command. For additional networks, simply run netclient join -t <enrollment key>.

Managing Netclient

Connect / Disconnect

to disconnect from a network previously joined (without leaving the network):
netclient disconnect <network>

to connect with a network previously disconnected:
netclient connect <network>

Viewing Logs

to view current networks
netclient list

to tail logs
journalctl -u netclient

to get most recent log run
systemctl status netclient

Re-syncing netclient (basic troubleshooting)

If the daemon is not running correctly, try restarting the daemon, or pulling changes directly (don't do both at once)

systemctl restart netclient

sudo netclient pull

Adding/Removing Networks

netclient join -t <enrollment key>

Set any of the above flags (netclient join --help) to override settings for joining the network.

Uninstalling

netclient uninstall

File: api.rst

API Usage

Authentication

API calls must be authenticated via a header of the format -H "Authorization: Bearer <YOUR_SECRET_KEY>". There are two methods to obtain YOUR_SECRET_KEY:

#. Using the masterkey. By default, this value is "secret key," but you should change this on your instance and keep it secure. This value can be set via env var at startup or in a config file (config/environments/*.yaml). See the :ref:Server Configuration <server-configuration-reference> documentation for more details.
#. Using a JWT received for a node. This can be retrieved by calling the /api/nodes/<network>/authenticate endpoint, as documented below.

Format of Calls for ``curl``

Requests take the format of:

.. code-block::

text

curl -H "Authorization: Bearer <YOUR_SECRET_KEY>" \
  -H 'Content-Type: application/json' \
  localhost:8081/api/path/to/endpoint

API Documentation

Check the documentation for all available endpoints <https://openapi.netmaker.io>_.

File: architecture.rst

Architecture

.. image:: images/nm-diagram-3.png
:width: 100%
:alt: Netmaker Architecture Diagram
:align: center

Pictured Above: A detailed diagram of Netmaker's Architecture.

Core Concepts

Familiarity with several core concepts will help when you encounter them later in the documentation.

WireGuard

WireGuard is a relatively new but very important technology that was recently added to the Linux kernel. WireGuard creates very fast but simple encrypted tunnels between devices. From the WireGuard <https://www.wireguard.com/>_ website, "it might be regarded as the most secure, easiest to use, and simplest VPN solution in the industry."

Previous solutions like OpenVPN and IPSec are considerably more heavy and complex while being less performant. All existing VPN tunneling solutions will cause a significant increase in your network latency. WireGuard is the first to achieve near over-the-line network speeds, meaning you see no significant performance impact. With the release of WireGuard, there is little reason to use any other existing tunnel encryption technology.

Mesh Network

When we refer to a mesh network in these documents we are typically referring to a "full mesh."

.. image:: images/mesh.png
:width: 33%
:alt: Full Mesh Network Diagram
:align: center

A full mesh network <https://www.bbc.co.uk/bitesize/guides/zr3yb82/revision/2>_ exists where each machine is able to directly talk to every other machine on the network. For example, on your home network, behind your router, all the computers are likely given private addresses and can reach each other directly.

This is in contrast to a hub-and-spoke network, where each machine must first pass its traffic through a relay server before it can reach other machines.

In certain situations, you may either want or need a partial mesh network, where only some devices can reach each other directly, and other devices must route their traffic through a relay/gateway. Netmaker can use this model in some use cases where it makes sense. In the diagram at the top of this page, the setup is a partial mesh because the servers (nodes A-D) are meshed, but then Remote Access Clients come in via a gateway and are not meshed.

Mesh networks are generally faster than other topologies but are also more complicated to set up. WireGuard on its own gives you the means to create encrypted tunnels between devices, but it does not provide a method for setting up a full network. This is where Netmaker comes in.

Netmaker

Netmaker is a platform built off of WireGuard enabling users to create mesh networks between their devices. Netmaker can create both full and partial mesh networks depending on the use case.

When we refer to Netmaker in aggregate, we are typically referring to Netmaker and the netclient, as well as other supporting services such as CoreDNS, SQLite, Mosquitto (MQ Broker), and UI webserver. There is also almost always a proxy server / LB, which by default is Traefik, but could be any number of tools.

From an end user perspective, they typically interact with the Netmaker UI. The other components run in the background invisibly.

Netmaker does a lot of work to set configurations for you so that you don't have to. This includes things like WireGuard ports, endpoints, public IPs, keys, and peers. Netmaker works to abstract away as much of the network management as possible, so that you can just click to create a network, and click to add a machine to a network. That said, every machine (node) is different, and may require special configuration. That is why, while Netmaker sets practical default settings, everything within Netmaker is fully configurable.

Node

A machine in a Netmaker network, which is managed by the Netclient, is referred to as a Node, as you will see in the UI. A Node can be a VM, a bare metal server, a desktop computer, an IoT device, or any other number of internet-connected machines on which the netclient is installed. A node is simply an endpoint in the network, which can send traffic to all the other nodes, and receive traffic from all of the other nodes.

Components

Netmaker consists of several core components, which are explained in high-level technical detail below.

Netmaker Server

The Netmaker server is, at its core, a golang binary. Source code can be found on GitHub <https://github.com/gravitl/netmaker>_. The binary, by itself, can be compiled for most systems. If you need to run the Netmaker server on a particular system, it likely can be made to work. In typical deployments, it is run as a Docker container. It can also be run as a systemd service as outlined in the non-docker install guide.

The Netmaker server acts as an API to the front end and publishes messages to clients via an MQ broker. The Netmaker server also runs an embedded "netclient" for each network that is created. This is a special netclient that enabled "UDP Hole Punching" on the system. When nodes reach the server, Netmaker uses this netclient to determine a routable address for each machine, and sends this out to the network.

Most server settings are configurable via a config file, or by environment variables (which take precedence). If the server finds neither of these, it sets sensible defaults, including things like the server's reachable IP, ports, and which "modes" to run in.

The Netmaker server interacts with either SQLite (default), postgres, or rqlite, a distributed version of SQLite, as its database. This DB holds information about nodes, networks, users, and other important data. This data is configuration data.

When the netmaker server needs to send an update to nodes, it publishes a message to the broker, MQ.

The components of the server are usually proxied via Traefik or an alternative like Nginx or Caddy. The proxy handles SSL certificates to secure traffic, and routes to the UI and API.

Message Broker (Mosquitto)

The Mosquitto broker is the default MQTT broker that ships with Netmaker, though technically, any MQTT broker should work so long as the correct configuration is applied. The broker enables the establishment of a pub-sub messaging system, whereby clients subscribe to receive updates. When the server receives a change, it will publish that change to the broker that pushes out the change to the appropriate nodes.

The broker must be reachable over a public address.

Netclient

The netclient is, at its core, a golang binary. Source code can be found in the Netclient GitHub Repository <https://www.github.com/gravitl/netclient>_. The binary, by itself, can be compiled for most systems. However, this binary is designed to manage a certain number of Operating Systems.

The netclient can be installed in one of two way: using package manager for your os/DE or by downloading the binary for your operating system/architecture from the netclient github repository and running ./netclient install.

After installation netclient commands can be executed to 'join' or 'register' with the netmaker server.

The 'join' and 'register' command attempts to add the machine to the Netmaker network using sensible defaults, which can be overridden with a config file or environment variables. Assuming the netclient has a valid key (or the network allows manual node signup), it will be registered into the Netmaker network, and will be returned necessary configuration details for how to set up its local network.

The netclient automatically registers with the MQTT server running with Netmaker, which will send it periodic updates when the network changes.

The netclient then sets up the system daemon (if running in daemon mode), and configures WireGuard. At this point, it should be part of the network.

The netclient will detect local changes and send them to the server when necessary. A change to IP address or port will lead to a network update to keep everything in sync. If the node is not running with the in daemon on, it is up to the operator to keep the netclient up-to-date by running regular "pulls" (netclient pull).

The MQ pub-sub system allows Netmaker to create dynamic mesh networks. As nodes are added to, removed from, and modified on the network, other nodes are notified and make appropriate changes.

Database (SQLite, rqlite, postgres)

Netmaker uses embedded SQLite as the default database. It can also use PostgreSQL, or rqlite, a distributed (RAFT consensus) database. Netmaker interacts with the database to store and retrieve information about nodes, networks, and users.

Additional database support (besides SQLite and rqlite) is very easy to implement for special use cases. Netmaker uses simple key-value lookups to run the networks, and the database was designed to be extensible, so support for key-value stores and other SQL-based databases can be achieved by changing a single file.

Netmaker UI

The Netmaker UI is a ReactJS-based static website which can be run on top of standard webservers such as Apache and Nginx. Source code can be found here <https://github.com/gravitl/netmaker-ui-2>_. In a typical configuration, the Netmaker UI is run on Nginx as a Docker container.

Netmaker can be used in its entirety without the UI, but the UI makes things a lot easier for most users. It has a sensible flow and layout for managing Networks, Nodes, Access Keys, and DNS.

CoreDNS

As of 0.22.0, CoreDNS is an active part of the Netmaker system. We deprecated setting entries on the hosts file which was not an ideal implementation.
Netmaker server actively sets the dns entries on the CoreDNS server.
After you install the netmaker server components, you can see the corendns container running as well.
You need to make some changes manually to activate the corendns server, follow these steps on the netmaker server:

Make sure that UDP Port 53 and TCP Port 53 are allowed to pass in the network where your netmaker server lies
disable the systemd-resolved (Reason: to avoid port conflict with coredns server)

.. code-block::

sudo systemctl disable systemd-resolved.service
sudo systemctl stop systemd-resolved

Make sure the network_mode: host is set on the coredns container spec in /root/docker-compose.yml and run docker-compose up -d

And now you can point any machine in the network to use this DNS server and you can reach the other peers in the network by their domain names.

For external clients running linux, make sure 'resolvconf' is installed before setting the Wireguard configurations.

Refer to your operating system documentation for information about how to configure custom DNS network settings. Here are some general help guides on how to add custom DNS server:

Linux - https://devilbox.readthedocs.io/en/latest/howto/dns/add-custom-dns-server-on-linux.html. Configuration depends on what distribution of Linux you use.
Mac - https://devilbox.readthedocs.io/en/latest/howto/dns/add-custom-dns-server-on-mac.html
Windows - https://devilbox.readthedocs.io/en/latest/howto/dns/add-custom-dns-server-on-win.html

If your machine is virtually hosted in a cloud, you might want to refer to your VM provider's documention on how to permanently set the custom DNS resolver.

Caddy

As of 0.17.0, Caddy is the default proxy for Netmaker if you set it up via Quick Start. Caddy is a simple and docker-friendly proxy, which can be compared to Nginx, Traefik, or HAProxy.

Caddy simplifies management because the configuration file is very short, several lines compared to dozens of lines for Traefik or Nginx. In addition, it can request certificates automatically.

Traefik was previously the default and is still a functioning option, but We are moving guidance towards Caddy by default. If you are maintaining an installation that relies on Traefik, you can continue to use it with Netmaker.

Remote Access Clients

The Remote Access Clients (client external to the mesh network) is simply a configured WireGuard connection to your network, which Netmaker helps to manage.

Most machines can run WireGuard. Setting up a WireGuard connection to a single endpoint is fairly simple. It is setting up mesh networks and other topologies like site-to-site which becomes complicated.

Mac, Windows, and Linux are handled natively by the Netclient, though you can still add them as ext clients if you wish. Primarily, iPhone and Android are the main systems unsupported by the Netclient which MUST be handled via Remote Access Client.

Remote Access Clients hook into a Netmaker network via an "Remote Access Gateway (ingress)," which is configured for a given node and allows traffic to flow into the network. Remote Access Clients are also reachable via the gateway. While this is a "concentrator" and not peer-to-peer, this is often desirable.

Many users use Remote Access Clients clients as a convenient way to manage remote access for their users. Why? It works with vanilla WireGuard for one. You simply download the config and load it into WireGuard on the client device. No additional software is required. It can also be quite helpful to have a "choke point" for traffic (the gateway) rather than direct p2p connections to every machine.

Technical Process

Below is a high level, step-by-step overview of the flow of communications within Netmaker (assuming Netmaker has already been installed):

Admin creates a new network with a subnet, for instance, 10.10.10.0/24
Admin creates a registration key for signing up new nodes
Both of the above requests are routed to the server via an API call from the front end
Admin installs the netclient binary on any given node (machine) and runs netclient join or register command.
Netclient decodes key, which contains the server location
Netclient gathers and sets appropriate information to configure itself as a node: it generates key pairs, gets public and local addresses, and sets a port.
Netclient sends this information to the server, authenticating with its registration key
Netmaker server verifies information and creates the node, setting default values for any missing information, and returns a response.
Netmaker also registers the client with MQ.
Upon successful registration, Netclient pulls the latest peers list from the server and set up a WireGuard interface.
Netclient subscribes to the MQ broker.
Netclient configures itself as a daemon (if joining for the first time).
Netclient regularly retrieves local information, checking for changes in things like IP and keys. If there is a change, it pushes them to the server.
If a change occurs in any other peer or peers are added/removed, an update will be sent to the Netclient via MQ, and it will re-configure WireGuard.

Compatible Systems for Netclient

To manage a node manually, the netclient can be compiled and run for most Linux distibutions, with a prerequisite of WireGuard with kernel headers. If the netclient from the release pages does not run natively on your system, you may need to compile the netclient binary directly on the machine from the source code. This may be true for some installations of SUSE, Fedora, and some Debian-based systems. However, if the dependencies are installed on the machine, the netclient should run correctly after being compiled.

Simply clone the repo, cd to netmaker/netclient, and run "go build" (Golang must be installed).

The following systems should be operable natively with Netclient in daemon mode:
- Windows
- Mac
- FreeBSD
- OpenWRT
- Fedora
- Ubuntu
- Debian
- Mint
- SUSE
- RHEL
- Raspian
- Arch
- CentOS
- Fedora CoreOS

Systemd is a system service manager for a wide array of Linux operating systems, but not all Linux distributions have adopted systemd. If you need to run on a Linux distro without systemd, we recommend the following: Join "unmanaged" with netclient join -daemon=off on Linux systems that do not run systemd and use some other method to run the daemon like a cron job or custom script.

Limitations

Install limitations mostly include platform-specific dependencies. A failed netclient install should display information about which command is failing, or which libraries are missing. This can often be solved via machine upgrade, installing missing dependencies, or setting kernel headers on the machine for WireGuard (e.x.: Installing Kernel Headers on Debian <https://stackoverflow.com/questions/62356581/wireguard-vpn-how-to-fix-operation-not-supported-if-it-worked-before>_)

It is very helpful if an install fails to run "netclient join -t -v 4". By default, the install runs with minimal logging. The -v flags will display any encountered errors. You can set the verbosity fron 0-4.

File: conduct.rst

=============== Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment
include:

Using welcoming and inclusive language
Being respectful of differing viewpoints and experiences
Gracefully accepting constructive criticism
Focusing on what is best for the community
Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

The use of sexualized language or imagery and unwelcome sexual attention or advances
Trolling, insulting/derogatory comments, and personal or political attacks
Public or private harassment
Publishing others' private information, such as a physical or electronic address, without explicit permission
Other conduct which could reasonably be considered inappropriate in a professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [email protected]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant <https://contributor-covenant.org>, version 1.4,
available here <https://contributor-covenant.org/version/1/4>.

File: egress-gateway.rst

.. _egress:

======= egress

Introduction

.. image:: images/egress1.png
:width: 80%
:alt: Gateway
:align: center

Netmaker allows your clients to reach external networks via an Egress Gateway. The Egress Gateway is a netclient which has been deployed to a server or router with access to a given subnet.

In the Netmaker UI, that node is set as an "egress gateway." Range(s) are specified which this node has access to. Once created, all clients (and all new ext clients) in the network will be able to reach those ranges via the gateway.

Configuring an Egress Gateway

Configuring an Egress Gateway is very straight forward. As a prerequisite, you must know what you are trying to access remotely. For instance:

a VPC
a Kubernetes network
a home network
an office network
a data center

After you have determined this, you must next deploy a netclient in a compatible location where the network is accessible. For instance, a Linux server or router in the office, or a Kubernetes worker node. This machine should be stable and relatively static (not expected to change its IP frequently or shut down unexpectedly).

Once you have determined the subnet, and deployed your netclient, you can go to your Netmaker UI and set the node as a gateway. click on the network on the sidebar and navigate to the egress section.

.. image:: images/egress7.png
:width: 80%
:alt: Gateway
:align: center

At this point you will choose your selected host to use as an egress. You can choose if you would like to use NAT or not with the switch. You can put the selected CIDR for your egress range(s) in the field. click the add range button to add more egress ranges for the host. The interface is automatically chosen and will not be shown in this window. With everything filled out, click the create button.

.. image:: images/ui-6.png
:width: 80%
:alt: Gateway
:align: center

Netmaker will set either iptables or nftables rules on the node depending on which one you have installed on your client. This will then implement these rules, allowing it to route traffic from the network to the specified range(s).

Use Cases

Remote Access

A common scenario would be to combine this with a "Remote Access Gateway" to create a simple method for accessing a home or office network. Such a setup would typically have only two nodes: the remote access and egress gateways. The Remote Access Gateway should usually be globally accessible, which makes the Netmaker server itself a good candidate. This means you need only the Netmaker server as the Remote Access Gateway, and one additional machine (in the private network you wish to reach), as the Egress.

.. image:: images/egress2.png
:width: 80%
:alt: Gateway
:align: center

In some scenarios, a single node will act as both remote access gateway and egress! For instance, you can enable acess to a VPC using your Netmaker server, deployed with a public IP. Traffic comes in over the public IP (encrypted of course) and then routes to the VPC subnet via the egress gateway.

.. image:: images/egress3.png
:width: 50%
:alt: Gateway
:align: center

Advanced Use Cases

Segmenting Traffic Flow Through Egress Gateways

User will need to add these additional routing rules on the egress machine to segment traffic to multiple egress ranges as desired when on multiple networks

.. code-block::

iptables -t filter -N netmakeregress
iptables -t filter -I FORWARD -i netmaker -d egressGwRangeA,egressGwRangeB -j netmakeregress

iptables -t filter -I netmakeregress -s networkRangeA -d egressRangeA -j ACCEPT
iptables -t filter -I netmakeregress -s networkRangeB -d egressRangeB -j ACCEPT
iptables -t filter -A netmakeregress -j DROP

NAT Rules

iptables -t nat -I POSTROUTING -s networkRangeA -d egressGwRangeA -j MASQUERADE

iptables -t nat -I POSTROUTING -s egressGwRangeA -d networkRangeA -j MASQUERADE

iptables -t nat -I POSTROUTING -s networkRangeB -d egressGwRangeB -j MASQUERADE

iptables -t nat -I POSTROUTING -s egressGwRangeB -d networkRangeB -j MASQUERADE

IPv6 NAT Masquerading for Egress Gateways

Currently IPv6 Egress Gateways are not working because the default kernel builds for most common linux distributions do not support ipv6 masquerading. Custom linux kernel needs to be built with flag for enabling ipv6 masquerading to get ipv6 egress gateways working.

Some online resources about the topic:

.. code-block::

https://superuser.com/questions/1751062/ipv6-masquerading-on-linux

https://www.kernelconfig.io/config_nf_nat_masquerade_ipv6?q=&kernelversion=5.15.116&arch=x86

https://www.reddit.com/r/PFSENSE/comments/vb4r3s/ip6_masquerading

Egressing External Clients

Unmanaged external clients that are directly connected to a Remote Access Gateway can also act as egressing machines. The idea is the same as egress gateways. The only difference is that Netclient is necessary with egress gateways, whilst only Wireguard is needed with egressing external clients. This feature is provisioned for situations or scenarios where installation of Netclient is not ideal or even possible. For example most VPN routers support WireGuard, but they are available only as plugins that are tailormade or closely coupled with the router's firmware or user interface.
While there are ways to make Netclient work for some routers, the integration could get cumbersome, obsolete, or compromising. Of course this feature is also applicable for simple or ad-hoc networking purposes so long as the external client supports iptables and IP forwarding.

At the time of this writing, this feature only supports Linux-based external clients. But the remote machines can be anything, provided they are in the same local network as one of the egressing external client's network interface.

Configuring Egressing External Clients

The configuration is pretty much the same as Egress Gateways. First, make sure that iptables is installed and IP forwarding is enabled. Please refer to your distro's documentation on how to do this. For Ubuntu you might do:

.. code-block::

#update
apt-get update

#install iptables
apt-get install iptables

#enable IP forwarding
sysctl -w net.ipv4.ip_forward=1

You can then responsibly specify the applicable egress ranges on the external client's VPN configuration, specifically in the "Additional Addresses" field as shown in the image below. It goes without saying that you can specify single addresses such as 172.16.1.2/32.

.. image:: images/integration-config-additional-addresses.jpg
:width: 80%
:alt: Client additional IP addresses range
:align: center

Your Netmaker server will then pick up the egress ranges and propagate it to all the other managed devices in the netmaker network. And of course you can edit them anytime when necessary. For more information on how to create or edit client VPN configurations, please refer to these links:

In some cases you might need to add POSTROUTING rules. In Ubuntu, you might do:

.. code-block::

#get the name of the specific network interface of the egressing client machine

that is associated with the egress ranges that you have specified

ip a

#add the necessary POSTROUTING rule, say the interface name is eth1
iptables -t nat -I POSTROUTING -o eth1 -j MASQUERADE

File: external-clients.rst

.. _remote-access:

===================================== Remote Access Gateways and Clients

Introduction

.. image:: images/ingress1.png
:width: 50%
:alt: Gateway
:align: center

Netmaker allows for Remote Access Clients (RAC) to reach into a network and access services via a Remote Access Gateway (ingress). So what is a "Remote Access Clients"? A remote access clients is any machine which cannot or should not be meshed, but can be connected to a Netmaker network. They are more or less VPN clients of a remote access gateway. This can include:
- Phones
- Laptops
- Desktops
- Other devices that support WireGuard

Remote Access Client can also refer to the slim client software that runs on a machine to allow it to connect to a Netmaker network.

A remote access client is not "managed," meaning it does not automatically pull the latest network configuration or push changes to its configuration. Instead, it uses a generated WireGuard config file to access the designated Remote Access Gateway, which is a managed server (ie: a machine running netclient). This server then forwards traffic to the appropriate endpoint, acting as a middleman/relay.

By using this method, you can hook any machine that can run WireGuard into a netmaker network.

It is recommended to run the netclient where compatible, but for all other cases, a machine can be configured as a Remote Access Clients. This recommendation holds except in the situation where only remote access is required.

Remote access gateways can be configured as internet gateways. Internet gateways work like traditional VPNs: they channel all traffic of their connected clients through them, securely hiding the client's public IP address while they browse the internet.
Internet gateways can be configured via the Remote Access tab. See the below section for more information.

Configuring a Remote Access Gateway

Clients must attach to a Remote Access Gateway. By default, your network will not have a remote access gateway. To configure one, navigate to the network and go to the "Remote Access" tab.

.. image:: images/extclient1.png
:width: 80%
:alt: Gateway
:align: center

After clicking the "Create Client Config" button, a modal window will pop up asking which host you would like to use as a gateway. You can use any host in your network, but it should have a public IP address (not behind a NAT). Your Netmaker server can be a remote access gateway and makes for a good default choice if you are unsure of which node to select. You can also specify whether the gateway should route all public traffic to the internet (internet gateway). Lastly, there is an option to specify a default DNS server for all attached Clients.

.. image:: images/extclient2.png
:width: 80%
:alt: Gateway
:align: center

Adding Clients to a Gateway

Once you have figured out your gateway, You can fill in any other info you need for your client. You can give it a name (Client ID), otherwise one will be auto-generated for you. You can specify your own public key for better security. You can set a DNS for the client (if left empty, fallback would be the gateway's default client DNS), and any specific addresses you would need for the client. You can also leave these fields blank and the name, address, and public key will be automatically configured for you. Clients will be able to access other nodes in the network just as the gateway does, of course unless blocked via network ACLs.

.. image:: images/extclient3.png
:width: 80%
:alt: Edit Remote Access Client Name
:align: center

After clicking create, your Remote Access Client configuration will be ready.

.. image:: images/extclient4.png
:width: 80%
:alt: ext client pop up
:align: center

Then, you can either download the configuration file directly or scan the QR code from your phone (assuming you have the WireGuard app installed). It will accept the configuration just as it would accept a typical WireGuard configuration file.

.. image:: images/extclient5.png
:width: 80%
:alt: Gateway
:align: center

Example config file:

.. literalinclude:: ./examplecode/myclient.conf

Your client should now be able to access the network! A client can be invalidated at any time by simply deleting it from the UI.

Disabling Remote Access Clients

To (temporarily) disable an Remote Access Client's access to the Netmaker network that it belongs to, click the switch below "Enabled" on the line with the Client ID that you would like to disable. Click "Accept" when asked "Are you sure you want to disable access to this Ext. Client?"

.. image:: images/extclient-disable.png
:width: 80%
:alt: Disable an Remote Access Client
:align: center

After you click the switch and click Accept the ext client will no longer be reachable and the switch will be turned off.

.. image:: images/extclient-disabled.png
:width: 80%
:alt: Disabled Remote Access Client
:align: center

To re-enable the client click the switch again and accept. It will turn on again and the client will be re-enabled.

Internet Gateway/Traditional VPN

Most people think of a VPN as a remote server that keeps your internet traffic secure while you browse the web, or as a tool for accessing internet services in another country, using a VPN server based in that country.
This is not a typical use case for Netmaker, but can be easily enabled via the Remote Access Gateway feature. This is a Pro-only feature.

Navigate to the remote access gateway setup mentioned above. Click the Internet Gateway switch. Click "Create Gateway".

.. image:: images/internet-gateway.png
:width: 80%
:alt: Internet Gateway
:align: center

After that, your public IPv4 traffic for any connected client, will be routed through the internet gateway. Support for IPv6 is work in progress.
NOTE: You need to specify the "Default client DNS" so as to avoid DNS leaks. If you do not specify a Default client DNS, the client's local gateway DNS will be most likely used.

Netmaker Pro extends internet gateways to hosts as well as clients. View the Pro documentation _ for more information.

File: features.rst

========= Features

.. toctree::
:maxdepth: 1

user-management
egress-gateway
external-clients
acls
pro/index

User Management

:ref:user-manage

Egress Gateway

Allows clients (nodes and ext clients) to reach external networks.

:ref:egress

Remote Access Gateways & Clients

A remote access gateway enables "external" clients to connect to the network. External clients refer to clients that are not part of the mesh network, but need to connect to it. This could be a laptop, mobile device, or even a server that is not part of the network.

:ref:remote-access

Access Control Lists

ACLs control communications between nodes on a network

:ref:acls

File: getting-started.rst

================= Getting Started

.. toctree::
:maxdepth: 1

install
manual-install
netclient
advanced-client-install
server-installation

Setup

#. Create your admin user, with a username and password.
#. Login with your new user
#. Create your first network by clicking on Create Network

Create a Network

.. image:: images/create-net.jpg
:width: 80%
:alt: Create Network Screen
:align: center

This network should have a sensible name.

More importantly, it should have a non-overlapping, private address range.

If you are running a small (less than 254 machines) network, and are unsure of which CIDR's to use, you could consider:

10.11.12.0/24
10.20.30.0/24
10.99.98.0/24

Network Settings Description

The Network creation form has a few fields which may seem unfamiliar. Here is a brief description:

IPv4: Adds private IPv4 to all nodes in a network

IPv6: Adds private IPv6 to all nodes in a network

Default Access Control: Indicates the default ACL value for a node when it joins in respect to it's peers (enabled or disabled).

Once your network is created, you should see the network (wg-net here but it will be the name you chose when creating the network):

.. image:: images/network-created.jpg
:width: 80%
:alt: Node Screen
:align: center

When you click on the NetId and then the Nodes button (or go direct via the left-hand menu and then Nodes) you see that the netmaker server has added itself to the network. From here, you can move on to adding additional nodes to the network.

As of v0.18.0, netclient has been moved out of the netmaker repo and into its own repo. This means that the netmaker server will no longer create its own default node. To recreate that default node, Netclient will also have to be installed on the netmaker server and joined to the network. You can then set that node as a default node in the Hosts tab. This will make every new network have that node automatically on creation. More on hosts are mentioned in the manage Nodes/Hosts section of this page.

.. image:: images/netmaker-node.jpg
:width: 80%
:alt: Node Screen
:align: center

Create a Key

Adding nodes to the network typically requires a key. If You are on version 0.18 or newer, you will use the enrollment key to acces a netmaker server. Enrollment keys offer different ways to register with a server.

Navigate to the enrollment keys tab on the side menu. You should see a create button in the top right corner. After clicking that, you should be brought to a window like this.

.. image:: images/enrollmentkeycreate.jpg
:width: 80%
:alt: Enrollment Key Screen
:align: center

This will give you a few different options on how you want to set up your enrollment key. you can set it up with unlimited uses, limited uses, or timebound uses. You can also setup one or multiple networks to join, or you can set it to no networks and then join a network through the UI in the hosts tab. Click on newly registered host and go to the networks tab. Then click on show all networks. You can also create any tags you would like for that key

.. image:: images/networkjoinui.jpg
:width: 80%
:alt: Join network through UI
:align: center

If an enrollment key runs out of uses, or is expired, the key will show as invalid like in the image below.

.. image:: images/enrollmentkeyinvalid.jpg
:width: 80%
:alt: Enrollment Key Screen with invalid keys
:align: center

After your enrollment key is created, you can click on that key to get the registration token.

.. image:: images/enrollmentkeytoken.jpg
:width: 80%
:alt: Enrollment Key token window
:align: center

The Enrollment Key value is the secret string that will allow your node to authenticate with the Netmaker network. This can be used with existing netclient installations where additional configurations (such as setting the server IP manually) may be required. This is not typical. E.g. netclient register -k <enrollment key> -s grpc.myserver.com -p 50051
The Registration Token value is a base64 encoded string that contains the server IP and grpc port, as well as the enrollment key. This is decoded by the netclient and can be used with existing netclient installations like this: netclient register -t <registration token>. You should use this method for adding a network to a node that is already on a network. For instance, Node A is in the mynet network and now you are adding it to default.
The Register Command value is a command that can be run on Linux systems after installing the Netclient. It will register with the server directly from the command line.

Other variations (eg Docker) are covered with the remaining values.

Deploy Nodes

Prereqisite: Every machine on which you install should have WireGuard and systemd already installed.
SSH to each machine
sudo su -
Prerequisite Check: Every Linux machine on which you run the netclient must have WireGuard and systemd installed
Follow the installation instructions for your operating system here <https://docs.netmaker.io/netclient.html#installation>_

You should get output similar to the below. The netclient retrieves local settings, submits them to the server for processing, and retrieves updated settings. Then it sets the local network configuration. For more information about this process, see the :doc:client installation <./netclient> documentation. If this process failed and you do not see your node in the console (see below), then reference the :doc:troubleshooting <./troubleshoot> documentation.

.. image:: images/nc-install-output.jpg
:alt: Output from Netclient Install
:align: center

Repeat the above steps for every machine you would like to add to your network. You can re-use the same install command so long as you do not run out of uses on your access key (after which it will be invalidated and deleted).

Once installed on all nodes, you can test the connection by pinging the private address of any node from any other node.

.. image:: images/ping-node.jpg
:alt: Node Success
:align: center

Manage Nodes/Hosts

Your machines should now be visible in the control panel.

.. image:: images/nodes.jpg
:width: 80%
:alt: Node Success
:align: center

As of v0.18.0 each node has an associated host. The host is a structure that encapsulates the node and gives it the information it needs like the name and network. The host will have features like creating a proxy, setting the hosted node as a default node, joining the hosted node to new networks without the need for another access key, setting verbosity, and changing listening ports and default interfaces. Server relays will also be handled by the host. The Hosts can be found in the Hosts tab on the UI. You should be taken to a screen like this.

.. image:: images/netmakerhostpage.jpg
:width: 80%
:alt: Host page
:align: center

In here you can see the host's name, the endpoint of the server running netclient, the public key for that host, the version number, and a switch to set that host's node as the default node. When this is switched on, that node will serve as the default node when a network is created (similar to netmaker-1 in versions before v0.18.0). Clicking on a host will bring you to the host's details.

.. image:: images/hostdetails.jpg
:width: 80%
:alt: details screen of the host
:align: center

This will give you more information like the firewall in use, MTUs, and listening port. You can also see networks associated with that host and options to edit or delete the host. If you are going to delete a Host, you need to delete the associated node first.

.. image:: images/hostedit.jpg
:alt: Edit screen of a host
:align: center

In the edit screen, you can make changes to the logging verbosity, listening port and proxy listening port, local range, MTU, and name. These fields will also update in the node, as the node gets this info from the host. If you want to change the endpoint, the associated node has to be static.

You can view/modify/delete any node by selecting it in the NODES tab. For instance, you can change the name to something more sensible like "workstation" or "api server". You can also modify network settings here, such as keys or the WireGuard port. These settings will be picked up by the node on its next check-in. For more information, see Advanced Configuration in the :doc:Using Netmaker <./usage> docs.

.. image:: images/node-details.jpg
:width: 80%
:alt: Node Success
:align: center

Nodes can be added/removed/modified on the network at any time. Nodes can also be added to multiple Netmaker networks. Any changes will get picked up by any nodes on a given network and will take about ~30 seconds to take effect.

Netclient (host) proxy has been removed in v0.20.5.

Uninstalling the netclient

To remove your nodes from a network (default here), run the following on each node: sudo netclient leave default (replacing default with the actual name of the network eg wg-net)
To remove the netclient entirely from each node (after running the above step), run sudo systemctl stop netclient && sudo systemctl disable netclient && sudo systemctl daemon-reload && sudo rm -rf /etc/netclient /etc/systemd/system/netclient.service /usr/sbin/netclient

Uninstalling Netmaker

To uninstall Netmaker from the server, simply run docker-compose down or docker-compose down --volumes to remove the docker volumes for a future installation.

File: how-to-guides.rst

============== How-to-Guides

.. toctree::
:maxdepth: 1

integrating-non-native-devices
usage

Egress Gateway

Netmaker allows your clients to reach external networks via an Egress Gateway. The Egress Gateway is a netclient which has been deployed to a server or router with access to a given subnet.

:ref:egress

Remote Access Gateway/External Clients

Netmaker allows for "external clients" to reach into a network and access services via an Remote Access Gateway. So what is an "external client"? An external client is any machine which cannot or should not be meshed. This can include:
- Phones
- Laptops
- Desktops

:ref:remote-access

Access Control Lists

:ref:acls

Integrating Non-native Devices

:ref:integrating-non-native-devices

External Guides

:ref:usage

File: index.rst

.. Netmaker documentation master file, created by
sphinx-quickstart on Fri May 14 08:51:40 2021.
You can adapt this file completely to your liking, but it should at least
contain the root toctree directive.

.. image:: images/netmaker.png
:width: 100%
:alt: Netmaker WireGuard
:align: center

======================================= Welcome to the Netmaker Documentation

Netmaker is a platform for creating fast and secure virtual networks with WireGuard.

This documentation covers Netmaker's :doc:installation <./server-installation>, :doc:usage <./usage>, and :doc:troubleshooting <./support>. It also contains reference documentation for the :doc:API <./api>, :doc:UI <./ui-reference> and :doc:Netclient <./advanced-client-install> configuration. All of the source code <https://github.com/gravitl/netmaker>_ for Netmaker is on GitHub.

For Kubernetes-specific guidance, please see the Netmaker Kubernetes Documentation. <https://k8s.netmaker.org>_

.. toctree::
:maxdepth: 1

about
getting-started
features
pro/index
upgrades
how-to-guides
references
license
conduct

File: install.rst

========= Install

Choose the install method that makes sense for you.

Prerequisites

Server

All components of Netmaker can be run on a single server (Virtual Machine or Bare Metal). Here are some recommendations for setting up the server:

We highly recommend that Netmaker be deployed in a dedicated networking environment.
The machine should have a public, static IP address
The machine should have at least 1GB RAM and 1 CPU (2GB RAM preferred for production installs)
2GB+ of storage
Ubuntu 24.04 Installed

If you do not have a host for this server, here are some recommendations:

DigitalOcean (preferred) <https://www.digitalocean.com>_
Linode <https://www.linode.com>_
KeepSec <https://www.keepsec.ca>_
AWS <https://aws.amazon.com>_
Azure <https://azure.microsoft.com>_
GCP <https://cloud.google.com>_
We do not recommend Oracle Cloud. There are known issues with their network configuration.

Domain

Your server will host several services (netmaker server, UI, etc.) each of which requires a dedicated, public subdomain. Here are some recommendations:

Use a publicly owned domain (e.x. example.com, mysite.biz)
Designate a subdomain (e.g. netmaker.example.com) for netmaker's services (e.g. dashboard.netmaker.example.com)
Make sure you have permission and access to modify DNS records for your domain (e.x: Route53)
Note on Cloudflare: Many of our users use Cloudflare. Cloudflare will proxy connections, which MQ does not like. This can be disabled in the Cloudflare DNS dashboard. If setting up your Netmaker server using Cloudflare for DNS, be aware that the configuration of Cloudflare proxy may cause problems with Netmaker which must be resolved, and at this point, Netmaker is not providing guidance on this setup.

Firewall Rules for Netmaker Server

Make sure firewall settings are set for Netmaker both on the VM and with your cloud security groups (AWS, GCP, etc) or with your router/FWA.

Make sure the following ports are open both on the VM and in the cloud security groups:

443, 80 (tcp): for Caddy, which proxies the Dashboard (UI), REST API (Netmaker Server), and Broker (MQTT)
51821 (udp and tcp): for WireGuard - Install script automatically setups a netclient on the server machine with default port as 51821.
8085 (exporter Pro): If you are building a Pro server, you need this port open.
1883, 8883 8083, 18083 (if using EMQX): We use two different types of brokers. There is Mosquitto or EMQX. Mosquitto is our default offering which uses ports 8883 and 1883. If you are setting up EMQX, all four ports mentioned need to be opened for MQTT, SSL MQTT, web sockets, and the EMQX dashboard/REST API.
53 (tcp and udp): If you set the CoreDNS container, that comes with the Netmaker installation, to 'host' your domain name resolution needs

.. code-block::

sudo ufw allow proto tcp from any to any port 443
sudo ufw allow proto tcp from any to any port 80
sudo ufw allow 51821/udp (based on your netclient listen port on server machine)
sudo ufw allow 51821/tcp (based on your netclient listen port on server machine)

#optional: only when hosting DNS on the Netmaker server
sudo ufw allow 53

It is also important to make sure the server does not block forwarding traffic (it will do this by default on some providers). To ensure traffic will be forwarded:

.. code-block::

iptables --policy FORWARD ACCEPT

Again, based on your cloud provider, you may additionally need to set inbound security rules for your server (for instance, on AWS). This will be dependent on your cloud provider. Be sure to check before moving on:

allow 443/tcp from all
allow 80/tcp from all
allow 51821-51830/udp from all
allow 51821-51830/tcp from all
allow 53 from all (optional)

Firewall Rules for Machines Running Netclient

As we already know, Netclient manages WireGuard on client devices (nodes). As its name suggests, Netclient is a client in a mesh topology, thus it needs to communicate with the server and with the other clients as well. Netclient will detect local changes and send them to the server when necessary. A change in IP address or port will lead to a network update to keep everything in sync.
It goes without saying that in almost all cases a firewall must be up and running on any device that is connected to a network, especially the Internet. Firewalls are inherently restrictive for good reasons. And by default, it might not allow traffic that Netclient would use to function properly.

On Windows machines, it is possible to allow programs or applications through the firewall. Thus you might want to allow Netclient and, depending on your setup, WireGuard.

On Linux, these necessary ports are needed to be opened:

UDP and TCP ports 51821-51830 for inbound and outbound (based on your client's listen port running on the machine)
TCP port 443 for outbound
UDP ports 19302 & 3478 for STUN outbound requests

For advanced use cases, you might need to view your device's firewall logs, or in the case of Netclients behind a NAT, your Firewall-Appliance/Router's firewall logs. Look for blocked traffic coming in and out having origin/destination IPs of your devices.

For example, in UFW you may do:

.. code-block::

#set the firewall to log only the blocked traffic
ufw logging low

#clear out the current logs
cat /dev/null | sudo tee /var/log/ufw.log

#reload ufw
ufw reload

#filter the logs
cat /var/log/ufw.log | grep -e -e <other nodes' IPs>

Quick Install

We recommend using the nm-quick script for self-hosted/On-Prem setup.

.. code-block::

sudo wget -qO /root/nm-quick.sh https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick.sh && sudo chmod +x /root/nm-quick.sh && sudo /root/nm-quick.sh

IMPORTANT: Notes on Installation

Due to the high volume of installations, the auto-generated domain has been rate-limited by the certificate provider. For this reason, we strongly recommend using your domain. Using the auto-generated domain may lead to a failed installation due to rate limiting.

IMPORTANT: From v0.22.0 the install script will install PRO version of netmaker with a 14-day free trial, for you to try out full capabilities of netmaker.

Integrating OAuth

Users are also allowed to join a Netmaker server via OAuth. They can do this by clicking the "Login with SSO" button on the dashboard's login page. Check out the:doc:integrating oauth docs <./oauth>.

After the trial period ends:

text

a. if you wish to continue using PRO:-
    i. check these steps to obtain pro license `<https://docs.netmaker.io/pro/pro-setup.html>`_
    ii. Run `/root/nm-quick.sh -u`

b. if you wish to downgrade to the community version

    i. Run `/root/nm-quick.sh -d`

To get started the easiest way, visit our SaaS platform to set up a netmaker server with just a few clicks <https://app.netmaker.io>_
:doc:check out these steps for the manual installation process for on-prem, although we don't recommend this path, instead use the install script mentioned above<./manual-install>
:ref:Highly Available Installation <HAInstall>
:doc:Advanced Install Resources <./server-installation>

File: integrating-non-native-devices.rst

.. _integrating-non-native-devices:

=================================== Integrating Non-native Devices

Introduction

Netmaker manages WireGuard configurations through the Netclient and the Remote Access Client (RAC) installed on the hosts and on the external clients respectively. Basically Netmaker makes WireGuard configurations, which are inherently static, dynamic. As you setup and change your network, Netmaker propagates these changes in the configuration to the affected machines installed with either Netclient or RAC.

However in some cases, it might not be ideal or even possible to install Netclient or RAC on some of your machines/devices. In these cases, Netmaker will rely upon your intervention to install WireGuard on these machines/devices and then to manually set up or change their WireGuard configurations whenever necessary. Basically, you just need to get the current WireGuard configuration (or VPN config files) from your Netmaker Remote Access Gateway and then stick it to your device in order for it to connect to your Netmaker network.

Generating a WireGuard Configuration File on Remote Access Gateway

Netmaker allows you to generate and manage your VPN configuration files. For instructions on how to make a node a Remote Access Gateway and on how to create/generate VPN configuration files, please refer to the "Remote Access Gateway/External Clients" section under the "How-to-Guides".

You can also get the WireGuard VPN configuration by following these steps:

Navigate to your network's Remote Access tab. You should see the Gateways table to the left-hand side and then the VPN Config Files table to the right-hand side of the page
If you have multiple gateways, select the specific one by clicking on it if it hasn't been selected already
If necessary, find the VPN configuration by inputting its name in the Search box
Once you've located the configuration file, hover over or click on its 'kebab' icon to the right-hand side corner of the row. A context menu should show up similar to the screenshot below

.. image:: images/integration-get-config.jpg
:width: 80%
:alt: Get Client Config
:align: center

Now you can view and copy the configuration file by clicking on the 'View Config' option. Or you can click on the 'Download' option to get a copy of the configuration file

.. image:: images/integration-sample-config.jpg
:width: 80%
:alt: Sample Client Config
:align: center

Once you have the configuration information or the configuration file, you can now stick it to your router, IoT, or other edge devices.

Routers and Firewall Appliances (Virtual or Bare metal)

While Netclient can be installed on some routers and firewall appliances after which you can then configure as egress gateways, it is generally ideal to use these devices' built-in VPN feature for seamless integration. Since most modern VPN routers and firewalls today support WireGuard, they can connect to a Netmaker network as an external client, after which you can then responsibly expose the resources behind them by inputting specific IP address ranges in the 'Additional Addresses' field. For more information on the Egressing External Clients, please refer to this link: https://docs.netmaker.io/egress-gateway.html#egressing-external-clients.

.. image:: images/integration-config-additional-addresses.jpg
:width: 80%
:alt: Client additional IP addresses range
:align: center

The general guidelines for integrating routers and firewall appliances (FWA) to Netmaker are the following:

Before doing any further configuration, take note of your current firmware version and back up the current configuration settings
Upgrade your firmware if necessary
Install WireGuard via your router's or FWA's Package Manager. Usually this can be done from its web interface (GUI) instead of from its shell (CLI)
Input the VPN configuration information from Netmaker; or upload the configuration file if your device supports it
If necessary, create a routing entry for the WireGuard tunnel interface
Create tight and specific firewall rules for traffic going in and out between the VPN interface and your LAN [or depending on your use case, your specific device, interface/port, VLAN, DMZ, WAN, etc.]

pfSense

This guide will help you set up WireGuard on pfSense 2.7.2. We will connect to a Netmaker network via a Remote Access Gateway

A. install WireGuard using the Package Manager in System -> Package Manager -> Available Packages

.. image:: images/integration-pfsense-pkgmgr.jpg
:width: 80%
:alt: pfSense Package Manager
:align: center

B. go to VPN -> WireGuard -> Tunnels, and then create a new WireGuard tunnel using the configuration information provided by Netmaker. Click on the Generate button under the Interface Keys fields before pasting the Private Key (from the configuration file generated by Netmaker). Save or submit the form and then take note of the tunnel interface name

.. image:: images/integration-pfsense-tunnel-interface.jpg
:width: 80%
:alt: pfSense Tunnel Configuration
:align: center

C. go to VPN -> WireGuard -> Peers, and then create a peer. Input the necessary configuration infomation similar to what is shown in the image below

.. image:: images/integration-pfsense-peer.jpg
:width: 80%
:alt: pfSense Peer Configuration
:align: center

D. enable WireGuard in VPN -> WireGuard -> Settings, and then click on the Apply Changes button. Make sure that the 'handshake' icon is green under the Status tab before proceeding any further

.. image:: images/integration-pfsense-enable-wg.jpg
:width: 80%
:alt: pfSense enable WireGuard
:align: center

E. go to Interfaces -> Assignments, and then assign or add a new interface for the WireGuard tunnel you created in Step 2. Take note of the interface name (say OPT1)

.. image:: images/integration-pfsense-assign-if.jpg
:width: 80%
:alt: pfSense assign WireGuard tunnel interface
:align: center

F. go to Interfaces -> [OPT1], and then tick the 'Enable interface' checkbox, input the MTU, static IP address, and the Netmaker network prefix

.. image:: images/integration-pfsense-enable-if-init.jpg
:width: 80%
:alt: pfSense go to the WireGuard tunnel interface
:align: center

.. image:: images/integration-pfsense-enable-if.jpg
:width: 80%
:alt: pfSense enable WireGuard tunnel interface
:align: center

If you're trying to connect to a Netmaker Internet Gateway, then click on the 'Add a new gateway' button. Depending on your use case, you may tick the Default Gateway checkbox so that all internet traffic will route through the Netmaker Internet Gateway. Then go to System -> General Setup and, [again] depending on your use case, select the Netmaker Internet Gateway in the DNS Server Settings so that domain name resolution traffic will pass through it instead of the other gateways

.. image:: images/integration-pfsense-if-internet-gw.jpg
:width: 80%
:alt: pfSense create an internet gateway
:align: center

G. If you just need to connect to an Internet Gateway, you don't need to do this step. Otherwise, create a Firewall rule allowing traffic from the Netmaker network to the target resource. In this guide we are going to allow ICMP traffic to the LAN so that we can do pings. Go to Firewall -> Rules -> [OPT1] and add a rule similar to what is shown in the screenshot below

.. image:: images/integration-pfsense-fw-init.jpg
:width: 80%
:alt: pfSense add firewall rule
:align: center

.. image:: images/integration-pfsense-fw.jpg
:width: 80%
:alt: pfSense add firewall rule - form
:align: center

After saving the firewall rule, nodes from your netmaker network should now be able to ping the egress ranges you've specified, and vice versa. Edit the firewall rule above or create another one specific to your needs.

OPNsense

This guide will help you set up WireGuard on OPNsense 24.1_1. We will connect to a Netmaker network via a Remote Access Gateway:

A. WireGuard comes pre-installed on OPNsense 24.1_1. For OPNsense 23.7.12 and below, install WireGuard as a plug-in in System -> Firmware -> Plugins

B. go to VPN -> WireGuard -> Settings -> Instances, and then create a new WireGuard tunnel instance using the configuration information provided by Netmaker. Click on the Generate [gear] icon in the Public Key field before pasting the Private Key (from the configuration file generated by Netmaker). Save and then take note of the tunnel interface name

.. image:: images/integration-opnsense-tunnel-interface.jpg
:width: 80%
:alt: OPNsense Tunnel Configuration
:align: center

C. go to VPN -> WireGuard -> Settings -> Peers, and then create a WireGuard peer using the information provided by Netmaker

.. image:: images/integration-opnsense-peer.jpg
:width: 80%
:alt: OPNsense Peer Configuration
:align: center

D. enable WireGuard in VPN -> WireGuard -> Settings -> General. And then click on the Apply Changes button

.. image:: images/integration-opnsense-enable-wg.jpg
:width: 80%
:alt: OPNsense enable WireGuard
:align: center

E. go to Interfaces -> Assignments, and then assign or add a new interface for the WireGuard tunnel you created in Step 2. Take note of the interface name (say OPT1)

.. image:: images/integration-opnsense-assign-if.jpg
:width: 80%
:alt: OPNsense assign WireGuard tunnel interface
:align: center

F. go to Interfaces -> [OPT1], and then tick the 'Enable interface' and the 'Prevent interface removal' checkboxes

.. image:: images/integration-opnsense-enable-if.jpg
:width: 80%
:alt: OPNsense enable WireGuard tunnel interface
:align: center

G. Create a route to the Netmaker network by first creating a gateway. Go to System -> Gateways -> Configuration, then click on the add icon and specify the tunnel interface [OPT1] and its IP

.. image:: images/integration-opnsense-add-gw.jpg
:width: 80%
:alt: OPNsense add gateway
:align: center

H. Add the necessary routing entry. Go to System -> Routes -> Configuration, then click on the 'add' icon and specify a route to the Netmaker network via the gateway created in the previous step

.. image:: images/integration-opnsense-add-routing-entry.jpg
:width: 80%
:alt: OPNsense add routing entry
:align: center

I. Create a Firewall rule for WireGuard allowing traffic between it and the target resource. In this guide we are going to allow ICMP traffic between WireGuard tunnel interface and the LAN so that we can do pings. Go to Firewall -> Rules -> [OPT1] and add a rule similar to what is shown in the screenshot below

.. image:: images/integration-opnsense-fw.jpg
:width: 80%
:alt: OPNsense add firewall rule - form
:align: center

After saving the firewall rule, devices in your LAN should now be able to ping machines in your Netmaker network, and vice versa. Edit the firewall rule above or create one that would suit your needs.

MikroTik

This guide will help you set up WireGuard on MikroTik 7.13.3. We will connect to a Netmaker network via a Remote Access Gateway:

A. WireGuard comes pre-installed on MikroTik 7.13.3 so you don't have to do anything.

B. given a sample WireGuard configuration below, you can access MikroTik's CLI and issue the corresponding commands

.. image:: images/integration-mikrotik-wg-config.jpg
:width: 80%
:alt: Sample WireGuard configuration for MikroTik
:align: center

WireGuard interface configuration:

.. code-block::

/interface/wireguard
add name=wg-netmaker mtu=1420 private-key="iMfHqGANXMJHGMBKwuo89txiU3/9edC20TxWpFtmU2Y="

Peer configuration:

.. code-block::

/interface/wireguard/peers
add allowed-address=10.40.70.0/24 endpoint-address=188.166.235.45 endpoint-port=51821 interface=wg-netmaker public-key="GM80g/eeXgkOrk0yYtdhhU73ETHffpojG2Ewd+N4kXI=" persistent-keepalive=20 client-dns=159.159.159.159

IP and routing configuration:

.. code-block::

/ip/address
add address=10.40.70.254/32 interface=wg-netmaker
/ip/route
add dst-address=10.40.70.0/24 gateway=wg-netmaker

And that's it. Devices from your LAN should now be able to reach machines in your Netmaker network, and vice versa.

For more information, please refer to this guide from MikroTik's documentation page: https://help.mikrotik.com/docs/display/ROS/WireGuard.

Routing internet traffic to a Netmaker Internet Gateway is also possible by adding the necessary firewall NAT rules. Please refer to the MikroTik documentation for more information.

OpenWrt

This guide will help you set up WireGuard on OpenWrt 23.05.2. We will connect to a Netmaker network via a Remote Access Gateway:

A. Go to System -> Software. Click on the Update lists... button then search for WireGuard. Install WireGuard-tools and luci-proto-WireGuard (for Web GUI). Reboot.

.. image:: images/integration-openwrt-pkgmgr.jpg
:width: 80%
:alt: OpenWrt software manager
:align: center

B. go to Network -> Interfaces, and then add a new WireGuard tunnel interface

.. image:: images/integration-openwrt-assign-if.jpg
:width: 80%
:alt: OpenWrt - create tunnel interface
:align: center

C. click on the Load Configuration..., paste the WireGuard configuration and then click Import settings

.. image:: images/integration-openwrt-wg-config.jpg
:width: 80%
:alt: OpenWrt - import WireGuard configuration
:align: center

D. go to the Peers tab. Edit the generated peer, tick the Route Allowed IPs field. Save and apply the changes made.

.. image:: images/integration-openwrt-wg-peer.jpg
:width: 80%
:alt: OpenWrt - route allowed IPs
:align: center

E. go to Status -> WireGuard and make sure that a handshake has taken place. If successful, OpenWrt should be able to reach the Netmaker Remote Access Gateway but not the other way around

.. image:: images/integration-openwrt-wg-status.jpg
:width: 80%
:alt: OpenWrt - WireGuard tunnel status
:align: center

F. go to Network -> Firewall, and then add a zone allowing traffic between the WireGuard tunnel and the LAN. Please do add your own version of Firewall rule that is tight and specific to your needs. Save and apply the changes made.

.. image:: images/integration-openwrt-wg-fw.jpg
:width: 80%
:alt: OpenWrt - Firewall configuration
:align: center

And that's about it. Routing internet traffic to a Netmaker Internet Gateway is also possible by adding the necessary firewall NAT rules. Please refer to the OpenWrt documentation for more information.

Other routers

Please refer to these links for instructions on how to configure WireGuard:

A. TP-Link - https://www.tp-link.com/fr/support/faq/3772/
B. Asus - https://www.asus.com/support/faq/1048281/
C. GL.iNet - https://docs.gl-inet.com/router/en/3/tutorials/WireGuard_client/#setup-WireGuard-client
D. Teltonika - https://wiki.teltonika-networks.com/view/WireGuard_Configuration_Example
E. pcWRT - https://www.pcwrt.com/2019/12/how-to-set-up-a-WireGuard-vpn-client-connection-on-the-pcwrt-router/
F. DD-WRT - https://windscribe.com/knowledge-base/articles/WireGuard-router-setup-guide-dd-wrt

Internet of Things (IoT Devices)

Please refer to these links for instructions on how to configure WireGuard:

A. IOTstack - https://sensorsiot.github.io/IOTstack/Containers/WireGuard/
B. Embedded Linux - https://www.toradex.com/blog/embedded-linux-vpn-application
C. lwIP IP stack - https://github.com/smartalock/WireGuard-lwip

Other Devices

For other devices not covered above, please refer to your device's documentation for instructions on how to install and configure WireGuard.

Disclaimer

The information provided by us on this how-to guide is for general informational purposes only. All information on this page is provided in good faith, however we make no representation or warranty of any kind, express or implied, regarding the accuracy, adequacy, validity, reliability, availability or completeness of any information on the page.

Under no circumstance shall we have any liability to you for any loss or damage of any kind incurred as a result of the use of this how-to guide or reliance on any information provided on the page. Your use of the how-to guide and your reliance on any information on the page is solely at your own risk.

File: license.rst

======= License

Netmaker's source code and all artifacts are available in this GitHub Organization Gravitl <https://github.com/gravitl>. All versions of netmaker are published under a License, which can be found here <https://raw.githubusercontent.com/gravitl/netmaker/master/LICENSE.md>.

File: manual-install.rst

=============== Manual Install

Important Notes

WE RECOMMEND USING THE NM-QUICK SCRIPT INSTEAD OF THIS GUIDE. Our docker-compose files have changed sizeably and the install script now uses a different process to obtain certificates. The script can be found on GitHub <https://github.com/gravitl/netmaker#get-started-in-5-minutes>_ (raw script here <https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick.sh>_ ). If you would like to install netmaker manually, You can use the instructions below.
Due to the high volume of installations, the auto-generated domain has been rate-limited by the certificate provider. For this reason, we strongly recommend using your own domain. Using the auto-generated domain may lead to a failed installation due to rate limiting.
This guide is just a manual version of the steps performed by that script, and is therefore more prone to error.
You must decide if you are installing the Pro version of Netmaker or the Community version. We recommend Pro because of its substantial free limits, but it does require an account <https://app.netmaker.io>_.
If deploying to DigitalOcean, you should use the DigitalOcean 1-Click <https://marketplace.digitalocean.com/apps/netmaker>_, which uses the interactive script.
This instance will not be HA. However, it should comfortably handle 100+ concurrent clients and support the most common use cases.
For information about deploying more advanced configurations, see the :doc:Advanced Installation <./server-installation> docs.
Prerequisites

Server

All components of Netmaker can be run on a single server (Virtual Machine or Bare Metal). Here some recommendations for setting up the server:

We highly recommend that Netmaker be deployed in a dedicated networking environment.
The machine should have a public, static IP address
The machine should have at least 1GB RAM and 1 CPU (2GB RAM preferred for production installs)
2GB+ of storage
Ubuntu 21.04 Installed

If you do not have a host for this server, here are some recommendations:

DigitalOcean (preferred) <https://marketplace.digitalocean.com/apps/netmaker>_
Linode <https://www.linode.com>_
KeepSec <https://www.keepsec.ca>_
AWS <https://aws.amazon.com>_
Azure <https://azure.microsoft.com>_
GCP <https://cloud.google.com>_
We do not recommend Oracle Cloud. There are known issues with their network configuration.

Domain

Your server will host several services (netmaker server, UI, etc.) each of which requires a dedicated, public subdomain. Here are some recommendations:

Use a publicly owned domain (e.x. example.com, mysite.biz)
Designate a subdomain (e.g. netmaker.example.com) for netmaker's services (e.g. dashboard.netmaker.example.com)
Make sure you have permission and access to modify DNS records for your domain (e.x: Route53)
Note on Cloudflare: Many of our users use Cloudflare for DNS. Cloudflare has limitations on subdomains you must be aware of, which can cause issues once Netmaker is deployed. Cloudflare will also proxy connections, which MQ does not like. This can be disabled in the Cloudflare dashboard. If setting up your Netmaker server using Cloudflare for DNS, be aware that the configuration of Cloudflare may cause problems with Netmaker which must be resolved, and at this point, Netmaker is not providing guidance on this setup.

Prepare DNS

Create a wildcard A record pointing to the public IP of your VM. As an example, *.netmaker.example.com. Alternatively, create records for these specific subdomains:

dashboard.domain
api.domain
broker.domain

If deploying Pro, you will also need records for the following:IsStatic

grafana.domain
prometheus.domain
netmaker-exporter.domain

Install Dependencies

.. code-block::

ssh root@your-host
sudo apt-get update
sudo apt-get install -y docker.io docker-compose

At this point you should have all the system dependencies you need.

Open Firewall

Please refer to the Firewall Rules for Netmaker Server <https://docs.netmaker.io/install.html#firewall-rules-for-netmaker-server>_ for more information.

Prepare MQ

You must retrieve the MQ configuration file for Mosquitto and the wait script.

.. code-block::

wget -O /root/mosquitto.conf https://raw.githubusercontent.com/gravitl/netmaker/master/docker/mosquitto.conf
wget -q -O /root/wait.sh https://raw.githubusercontent.com/gravitl/netmaker/master/docker/wait.sh
chmod +x wait.sh

Install Netmaker

Prepare Docker Compose

As of 0.20.0, our docker-compose and Caddyfile now contains references to a netmaker.env file. This will cut down on repetitive entries like inserting your base domain multiple times. You only insert it once in your netmaker.env file and the backend handles placing it in the right places. The EMQX and Pro docker-composes are now extensions of the regular docker-compose file, so switching to Pro or EMQX doesn't involve recreating an entire docker-compose file.

Get the base docker-compose and Caddyfile.

.. code-block::

wget https://raw.githubusercontent.com/gravitl/netmaker/master/compose/docker-compose.yml
wget https://raw.githubusercontent.com/gravitl/netmaker/master/docker/Caddyfile

If you plan on using a Professional server (Pro), then you will need to grab the Caddyfile-pro file instead. There will be more Pro related instructions below in "Extra Steps for Pro".

.. code-block::

wget https://raw.githubusercontent.com/gravitl/netmaker/master/docker/Caddyfile-pro

You can grab the netmaker.env file here.

.. code-block::

wget https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/netmaker.default.env
cp netmaker.default.env netmaker.env

You can then use a text editor like vim or nano to go in there and fill out the fields. There is an example below to reference. You can get your ip with the command ip route get 1 | sed -n 's/^.*src $[0-9.]*$ .*$/\1/p'. You can also generate random strings for the master key and MQ passwords with the command tr -dc A-Za-z0-9 </dev/urandom | head -c 30 ; echo '' or you can enter them manually if desired. For the base domain again, we advise you use your own domain, because nip.io can hit rate limiting easily from the high volume when obtaining certificates. If you do want to use nip.io, just enter nm.<YOUR_IP_WITH_DASHES_INSTEAD_OF_DOTS>.nip.io.

.. code-block:: cfg

Email used for SSL certificates

NM_EMAIL=[email protected]

The base domain of netmaker

NM_DOMAIN=nm.123-456-789-012.nip.io

Public IP of machine

SERVER_HOST=

The admin master key for accessing the API. Change this in any production installation.

MASTER_KEY=

The username to set for MQ access

MQ_USERNAME=

The password to set for MQ access

MQ_PASSWORD=

Specify the type of server to install. Use pro for professional and ce for community edition

INSTALL_TYPE=ce

The next two are for Professional edition. You can find that info below on "Extra steps for Pro"

NETMAKER_TENANT_ID= (for Pro version)
LICENSE_KEY= (for Pro version)

The version for the netmaker and netmaker-ui servers. current version is v0.20.2.

Some versions of docker may try to include quotation marks in this reference, so don't put them in.

SERVER_IMAGE_TAG=v0.20.2
UI_IMAGE_TAG=v0.20.2

used for HA - identifies this server vs other servers

NODE_ID="netmaker-server-1"
METRICS_EXPORTER="off" (turn on for Pro)
PROMETHEUS="off" (turn on for Pro)

Enables DNS Mode, meaning all nodes will set hosts file for private dns settings

DNS_MODE="on"

Enable auto update of netclient ? ENUM:- enabled,disabled | default=enabled

NETCLIENT_AUTO_UPDATE="enabled"

The HTTP API port for Netmaker. Used for API calls / communication from front end.

If changed, need to change port of BACKEND_URL for netmaker-ui.

API_PORT="8081"
EXPORTER_API_PORT="8085"

The "allowed origin" for API requests. Change to restrict where API requests can come from with comma-separated

URLs. ex:- https://dashboard.netmaker.domain1.com,https://dashboard.netmaker.domain2.com

CORS_ALLOWED_ORIGIN="*"

Show keys permanently in UI (until deleted) as opposed to 1-time display.

DISPLAY_KEYS="on"

Database to use - sqlite, postgres, or rqlite

DATABASE="sqlite"

The address of the mq server. If running from docker compose it will be "mq". Otherwise, need to input address.

If using "host networking", it will find and detect the IP of the mq container.

SERVER_BROKER_ENDPOINT="ws://mq:1883"

The reachable port of STUN on the server

STUN_PORT="3478"

Logging verbosity level - 1, 2, or 3

VERBOSITY="1"

If ON, all new clients will enable proxy by default

If OFF, all new clients will disable proxy by default

If AUTO, stick with the existing logic for NAT detection

This setting is no longer available from v0.20.5

DEFAULT_PROXY_MODE="off"
DEBUG_MODE="off"

Enables the REST backend (API running on API_PORT at SERVER_HTTP_HOST).

Change to "off" to turn off.

REST_BACKEND="on"

If turned "on", Server will not set Host based on remote IP check.

This is already overridden if SERVER_HOST is set. Turned "off" by default.

DISABLE_REMOTE_IP_CHECK="off"

Whether or not to send telemetry data to help improve Netmaker. Switch to "off" to opt out of sending telemetry.

TELEMETRY="on"

OAuth section

""

AUTH_PROVIDER=

""

CLIENT_ID=

""

CLIENT_SECRET=

"https://dashboard."

FRONTEND_URL=

""

AZURE_TENANT=

https://oidc.yourprovider.com - URL of oidc provider

OIDC_ISSUER=

Extra Steps for Pro

Visit <https://app.netmaker.io>_ to create your account on the Netmaker SaaS platform.
Create a tenant of type On-Prem to obtain a license key. more details in :doc:Netmaker Professional setup <./pro/pro-setup>
Retrieve Tenant ID and license key from the tenant's settings tab.
Place the licence key and tenant ID in the netmaker.env file.
In the netmaker.env file, change the METRICS_EXPORTER and PROMETHEUS from off to on.
Grab the docker-compose.pro extension file from the repo and change its name to docker-compose.override.yml.

.. code-block::

wget https://raw.githubusercontent.com/gravitl/netmaker/master/compose/docker-compose.pro.yml

You will not need to make any changes to this file. It will reference the current netmaker.env file.

Then run

.. code-block::

ln -fs /root/netmaker.env /root/.env

Integrating OAuth

Start Netmaker

sudo docker-compose -f docker-compose.yml up -d --force-recreate

navigate to dashboard. to begin using Netmaker.

To troubleshoot issues, start with:

docker logs netmaker

Or check out the :doc:troubleshoooting docs <./troubleshoot>.

File: netclient.rst

.. _netclient:

========== Netclient

As of v0.18.0 Netclient is now in its own standalone repo seperate from netmaker.

Netclient GUI has been deprecated as of v0.22.1.

And the CLI has been updated.

.. image:: images/netclientcli.jpg
:alt: netclient CLI
:align: center

The Netclient manages WireGuard on client devices (nodes). This document walks through setting up the netclient on machines, including install, management, and uninstall.

The Netclient is supported on the following operating systems:

Linux (most distributions)
Windows
macOS
FreeBSD

For unsupported devices, please use the Remote Access Client config, which is just a static, vanilla, WireGuard configuration file, which can be added to any device that supports WireGuard.

Installation

Before adding the machine to a network, the netclient must be installed. A successful installation sets up the netclient executable on the machine and adds it as a system daemon. The daemon will listen for changes for any network it joins.

The client install does not add the client as a member of any network. Once the client is installed, you must run:

.. code-block::

netclient join -t

The following are install instructions for most operating systems.

Linux

Debian Distros (debian/ubuntu/mint/pop-os)

.. code-block::

curl -sL 'https://apt.netmaker.org/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/netmaker-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/netmaker-keyring.gpg] https://apt.netmaker.org stable main" | sudo tee /etc/apt/sources.list.d/netclient.list
sudo apt update
sudo apt install netclient

Red Hat Distros (fedora/redhat/centos/rocky)

.. code-block::

curl -sL 'https://rpm.netmaker.org/gpg.key' | sudo tee /tmp/gpg.key
curl -sL 'https://rpm.netmaker.org/netclient-repo' | sudo tee /etc/yum.repos.d/netclient.repo
sudo rpm --import /tmp/gpg.key
sudo dnf check-update
sudo dnf install netclient

Arch Distros (arch/manjaro/endeavouros)

.. code-block::

yay -S netclient

OpenSUSE (tumbleweed/leap)

.. code-block:: shell

sudo rpm --import https://rpm.netmaker.org/gpg.key
curl -sL 'https://rpm.netmaker.org/netclient-repo' | sudo tee /etc/zypp/repos.d/netclient.repo
sudo zypper refresh
sudo zypper install netclient

Windows

Bundled Installer

Download Link: https://fileserver.netmaker.org/releases/download/latest/netclientbundle.exe

Mac

Brew Install

.. code-block::

brew tap gravitl/netclient
(optional) brew audit netclient
brew install netclient

Installer

Download Link for Apple silicon: https://fileserver.netmaker.org/releases/download/latest/Netclient-M1.pkg

Download Link for Apple Intel: https://fileserver.netmaker.org/releases/download/latest/Netclient-Intel.pkg

Docker

You can run Netclient using Docker instead of installing it on your local machine. To ensure that the correct commands are present in order to use Docker use these steps:

.. code-block::

sudo apt-get update
sudo apt-get install -y docker.io docker-compose

After that you can proceed to join the network using the docker command from the access key for the network you wish to join. The docker command is available from the access key view in the Netmaker UI. To have the netclient docker container restart (eg after the system reboots) you'll want to use the following option when running docker run:

.. code-block::

--restart=always

If you prefer (e.g., when specifying a lot of environment variables), you can use a docker-compose.yml file such as the following instead of the docker run command:

.. code-block::

version: "3.4"

services:
netclient:
network_mode: host
privileged: true
restart: always
environment:
- TOKEN=
- PORT=
- ENDPOINT=
- MTU=
- HOST_NAME=
- IS_STATIC=<static host (true/false)>
volumes:
- '/etc/netclient:/etc/netclient'
container_name: netclient
image: 'gravitl/netclient:latest'

where is the Access Token available from the "Viewing your Access Key Details" window in the Netmaker UI.

If you are making a docker container on a server that already has a baremetal version of netclient on there, this will create a conflict. Netmaker will only make one host for that machine and not see the container as another node. Meanwhile on the client, both daemons will be running (bare metal and container), causing conflicts (both receive the peer updates) and the node becomes unreachable.

You can work around this by running the docker netclient using a seperate netclient folder and having host networking NOT enabled. Host networking must be turned off when running with multiple netclients. This means that the host will not have the private address of the container and it will be segmented.

Your compose would look more like this:

.. code-block::

version: "3.4"

services:
netclient:
privileged: true
network_mode: host
restart: always
environment:
- TOKEN=
- PORT=
- ENDPOINT=
- MTU=
- HOST_NAME=nc-docker-2
- IS_STATIC=<static host (true/false)>
- IFACE_NAME=netmaker-2
volumes:
- '/etc/netclient2:/etc/netclient'
container_name: netclient2
image: 'gravitl/netclient:latest'

By using this method, you can run many netclients on the same host and just incrementing up the volumes (netclient3, netclient4 ..... netclientN) and make sure to set the interface name, so that it won't conflict with existing netclients running on same host.

IMPORTANT:
For docker netclient to function correctly as either remote access/egress gateway, you need to additionally run the following commands on the host machine:

iptables -I DOCKER-USER -i netmaker -j ACCEPT
iptables -I DOCKER-USER -o netmaker -j ACCEPT

Joining a Network

The join command provides the following flags with short descriptions on what each one does.

.. image:: images/netclientjoincli.jpg
:width: 80%
:alt: netclient CLI
:align: center

With a token:

.. code-block::

netclient join -t

With username/password:

.. code-block::

netclient join -n -u -s api.
(example: netclient join -n mynet -u admin -s api.nm.example-domain.io)

With SSO (oauth must be configured):

.. code-block::

netclient join -n -s api.

With docker:

.. code-block::

docker run -d --network host --privileged -e TOKEN= -v /etc/netclient:/etc/netclient --name netclient gravitl/netclient:

Again, if you are making a docker container on an already existing baremetal netclient, you will have to modify the join command like this for example:

.. code-block::

docker run -d --network host --privileged -e TOKEN= -e HOST_NAME=nc-docker-2 -e IFACE_NAME="netmaker-2" -v /etc/netclient2:/etc/netclient --name netclient2 gravitl/new-netclient:

Make sure interface name you pass when running multiple netclient containers on same host doesn't conflict with each other.

These commands will be available to copy and paste in the access keys section of your netmaker UI. You can set the verbosity to a level 0-4 with the flag -v <number 0-4> in the join command if you need more info about the join.

Managing Netclient

Connecting/Disconnecting from a network

You can connect/disconnect from the network from either the netclient, or from the GUI.

From the CLI, you can use the following:

.. code-block::

netclient connect
netclient disconnect

You can also disconnect and reconnect from the UI. Click on the node you want to disconnect/reconnect and click on edit.

On the bottom, you should see a switch labeled connected like this one. toggle the switch and hit submit. That client will connect or disconnect accordingly.

.. image:: images/disconnect.jpg
:width: 80%
:alt: connect/disconnect button
:align: center

If you disconnected from the CLI, This switch should be off.

Leave a network

In the GUI, the leave network button is located in the network details. You can also leave from the CLI with the following command.

.. code-block::

netclient leave

List Networks

.. code-block::

netclient list

Multi-Server

Netclient can be registered with multiple netmaker servers. You can list the servers you are registered with using netclient server list command.
The server you are currently connected to will be marked with "active: ".
.. code-block::

netclient server list

You can switch between different netmaker servers using netclient switch command.
Warning: Switching to another server will disconnect netclient from all the networks on the current server, but you can always switch back using the netclient switch command.
.. code-block::

netclient server switch

You can completely leave a server using the netclient server leave command.
Warning: Leaving a server will remove the host from all networks on the server and delete the host from the server.
In order to reconnect to server you must join or register*

.. code-block::

netclient server leave

Use a different version

Netclient as of v0.18.0 has an option to choose which version of netclient you would like to use. This only applies to versions v0.18.0 and later.

.. code-block::

netclient use

Netclient also has an auto-update feature as of v0.18.0

Uninstalling

Leave a network:

Uninstall from CLI:

.. code-block::

netclient uninstall

Uninstall using package manager (use equivalent command for your OS):

.. code-block::

apt remove netclient

With a Mac, just go to applications in your finder and throw netclient in the trash bin, or use

.. code-block::

brew uninstall netclient

if it was installed through Homebrew

With windows go to "Add or remove programs" in your system settings and remove netclient.

File: nmctl-standalone.rst

=============================================== NMCTL - how to use netmaker from command line

a brief guide to using netmaker from the command line (without the UI)

Assumptions

using bash shell
nmctl and jq have been installed
netmaker server has been set up at example.com. This can be a SaaS (managed) tenant as well.

Setup superadmin user

Set base domain

.. code-block::

text

export NN_DOMAIN=example.com

Create SuperAdmin User

.. code-block::

text

curl --location 'https://api.$NM_DOMAIN/api/users/adm/createsuperadmin' \
            --header 'Content-Type: application/json' \
            --data '{
            "username":"superadmin",
            "password":"NetmakerIsAwe$ome"
            }'

Set Context

.. code-block::

text

nmctl context set commandline --endpoint https://api.$NM_DOMAIN --username $USER --password $PASSWORD

Create Admin User

.. code-block::

text

nmctl user create --admin --name $USER --password $PASSWORD

Create Normal User

.. code-block::

text

nmctl user create --name  --password

Normal Operations by user

assume that users have been created by superadmin

Set username/password

.. code-block::

text

export USER=
    export PASSWORD=

Set User Context

.. code-block::

text

nmctl context set commandline --endpoint https://api.$NM_DOMAIN --username $USER --password $PASSWORD
        nmctl context use commandline

Create Network

.. code-block::

text

nmctl network create --name mynetwork --ip4v_addr 10.10.10.0/24

Create Enrollment Key

create one of Unlimited/LimitedUse/Expiration

Unlimited

.. code-block::

text

export KEY=$(nmctl enrollment_key create --network mynetwork --unlimited | jq .token)

Limited Use (3)

.. code-block::

text

export KEY=$(nmctl enrollment_key create --network mynetwork --uses 3 | jq .token)

With Expiration Time (2 days)

.. code-block::

text

export EXPIRES=$(date -d "+2 days" +$s)
    export KEY=$(nmctl enrollment_key create --network mynetwork --expires $EXPIRES | jq .token)

Join network

.. code-block::

text

sudo netclient join -t $KEY

File: nmctl.rst

================================ NMCTL

NMCTL is a CLI tool for interacting with the Netmaker API.

Quick Start

Start with getting the latest nmctl binary specific to your operating system from the link below:

.. code-block::

https://github.com/gravitl/netmaker/releases/latest

Make sure the binary is executable with chmod +x nmctl and then move it into your /usr/sbin folder.

If everything is setup ok, you should be able to type nmctl and see the following:

.. code-block::

CLI for interacting with Netmaker Server

Usage:
nmctl [command]

Available Commands:
acl Manage Access Control Lists (ACLs)
completion Generate the autocompletion script for the specified shell
context Manage various netmaker server configurations
dns Manage DNS entries associated with a network
enrollment_key Manage Enrollment Keys
ext_client Manage Remote Access Clients
help Help about any command
host Manage hosts
logs Retrieve server logs
metrics Fetch metrics of nodes/networks
network Manage Netmaker Networks
network_user Manage Network Users
node Manage nodes associated with a network
server Get netmaker server information
user Manage users and permissions
usergroup Manage User Groups

Flags:
-h, --help help for nmctl

Use "nmctl [command] --help" for more information about a command.

Your CLI should be ready to go at this point.

Context

Before running any commands, a context has to be set which stores the API endpoint information. This allows the CLI to know which server to communicate with, and the user account to use.

NMCLI supports connecting to both standalone (self-hosted) and SaaS(managed) tenants. This is specified with a flag. More details below.

Connecting to standalone (self-hosted) tenants

Assuming your tenant is hosted at https://api.netmaker.example.com

You can use your username and password that you use to sign in to the dashboard UI to set the context. Then you can set the CLI to use that context.

.. code-block::

nmctl context set --endpoint=https://api.netmaker.example.com --username= --password= # create the context
nmctl context use # apply the created context

You can also authenticate via OAuth with the following:

.. code-block::

nmctl context set --endpoint=https://api.netmaker.example.com --sso # create the context for OAuth (Social Sign On)
nmctl context use # apply the created context

Connecting to SaaS (managed) tenants

You can also authenticate with a managed (SaaS) tenant with the following commands:

.. code-block::

nmctl context set --saas --tenant_id= --username= --password= # create the context
nmctl context use # apply the created context

You can also authenticate via OAuth with the following:

.. code-block::

nmctl context set --saas --sso --tenant_id= # create the context for OAuth (Social Sign On)
nmctl context use # apply the created context

List and switch between contexts

You can see a list of all your contexts that you have created with the following:

.. code-block::

nmctl context list

That list also tells you what context/tenant is currently selected.

You can switch to a different context by using the use subcommand:

.. code-block::

nmctl context use

Delete contexts

You can delete a context with the following:

.. code-block::

nmctl context delete

Network

Create a network with the name test_net and CIDR 10.11.13.0/24.

.. code-block::

nmctl network create --name="test_net" --ipv4_addr="10.11.13.0/24"

Fetch details of the created network.

.. code-block::

nmctl network list
+----------+----------------------+----------------------+---------------------------+---------------------------+
| NETID | ADDRESS RANGE (IPV4) | ADDRESS RANGE (IPV6) | NETWORK LAST MODIFIED | NODES LAST MODIFIED |
+----------+----------------------+----------------------+---------------------------+---------------------------+
| test_net | 10.11.13.0/24 | | 2022-12-14T13:08:47+05:30 | 2022-12-14T13:08:47+05:30 |
+----------+----------------------+----------------------+---------------------------+---------------------------+

Access Key

Create an access key for the created network with 100 uses. This key shall be used by nodes to join the network test_net.

.. code-block::

nmctl keys create test_net 100
{
"name": "key-818a4ac3fe85a9d0",
"value": "f0edf9ef08fa2b1a",
"accessstring": "eyJhcZljb25uc3RyaW5nIjoiYXBpLm5ldG1ha2VyLmV6ZmxvLmluOjQ0MyIsIm5ldHdvcmsiOiJ0ZXN0X25ldCIsImtleSI6ImYwZWRmOWVmMDhmYTJiMWEiLCJsb2NhbHJhbmdlIjoiIn0=",
"uses": 100,
"expiration": null
}

Nodes

Connect a node to the network using :doc:netclient <./netclient> and the access key created above. Use the accessstring as token.

.. code-block::

netclient join -t

List all nodes. This displays information about each node such as the address assigned, id, name etc

.. code-block::

nmctl node list
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+
| NAME | ADDRESSES | VERSION | NETWORK | EGRESS | REMOTE ACCESS GATEWAY | RELAY | ID |
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+
| test_node | 10.11.13.254 | v0.17.0 | test_net | no | no | no | 938d7861-55fc-40a9-970d-6d70acfc3a80 |
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+

Using nmctl, we can turn the node into egress, remote access gateway or a relay. Lets turn the node into an remote access gateway by supplying the network name and node id as parameters.

.. code-block::

nmctl node create_remote_access_gateway test_net 938d7861-55fc-40a9-970d-6d70acfc3a80

Fetching the node list once again we can see that our node has been turned into a remote access gateway.

.. code-block::

nmctl node list
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+
| NAME | ADDRESSES | VERSION | NETWORK | EGRESS | REMOTE ACCESS GATEWAY | RELAY | ID |
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+
| test_node | 10.11.13.254 | v0.17.0 | test_net | no | yes | no | 938d7861-55fc-40a9-970d-6d70acfc3a80 |
+--------------+---------------------------+---------+----------+--------+-----------------------+-------+--------------------------------------+

Remote Access Clients

Adding a :doc:Remote Access Client <./external-clients> to the network is just as easy. Requires the network name and node id as input parameters.

.. code-block::

nmctl ext_client create test_net 938d7861-55fc-40a9-970d-6d70acfc3a80
Success

List all available Remote Access Clients.

.. code-block::

The wireguard config of an Remote Access Client can also be fetched with the network name and client id.

.. code-block::

nmctl ext_client config test_net limp-chicken

[Interface]
Address = 10.11.13.2/32
PrivateKey = 4Ojhsn/uLcH6xta6zqokQ+GiRuZwesdzE2hDSa6vYWc=
MTU = 1280

[Peer]
PublicKey = h96G9R8qqHIm6OfFgIZNBlRE5uCumkSZv4Pwn2DVXEs=
AllowedIPs = 10.11.13.0/24
Endpoint = 138.209.145.214:51824
PersistentKeepalive = 20

ACLs

Access Control between hosts can be managed via the NMCTL CLI. These settings allow the network admin to specify which hosts are allowed to communicate between each other.

List

To list all access control settings for a network:

.. code-block::

nmctl acl list

Allow/Deny

To allow communication between two hosts on a network:

.. code-block::

nmctl acl allow <host 1 ID> <host 2 ID>

To deny communication between two hosts:

.. code-block::

nmctl acl deny <host 1 ID> <host 2 ID>

Host IDs can be retrieved with the nmctl node list command.

The global --output flag can be used to format how a network's ACLs are outputted.

Help

Further information about any subcommand is available using the --help flag

.. code-block::

nmctl subcommand --help

Example:-

.. code-block::

nmctl node --help
Manage nodes associated with a network

Usage:
nmctl node [command]

Available Commands:
create_egress Turn a Node into a Egress
create_remote_access_gateway Turn a Node into a Remote Access Gateway
create_relay Turn a Node into a Relay
delete Delete a Node
delete_egress Delete Egress role from a Node
delete_remote_access_gateway Delete Remote Access Gateway role from a Node
delete_relay Delete Relay role from a Node
get Get a node by ID
list List all nodes
uncordon Get a node by ID
update Update a Node

Flags:
-h, --help help for node

Use "nmctl node [command] --help" for more information about a command.

File: oauth.rst

Integrating OAuth

Introduction

As of v0.14.5, Netmaker offers integration with the following OAuth providers:

GitHub
Google
Microsoft Azure AD
Open ID Connect (OIDC)

By integrating with an OAuth provider, your Netmaker users can log in via the provider, rather than the default simple auth.

Configuring your provider

In order to use OAuth, configure your OAuth provider (GitHub, Google, Azure AD).

You must configure your provider (except for Azure AD) to use the Netmaker Dashboard URI: dashboard.<netmaker.base.domain> as the origin URL.

For example: https://dashboard.netmaker.mydomain.com

You must configure your provider to use the Netmaker API URI redirect route with the following format: api./api/oauth/callback

For example: https://api.netmaker.mydomain.com/api/oauth/callback

General provider instructions can be found with the following links:

Instructions for GitHub: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#github-auth-provider
Instructions for Google: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#google-auth-provider
Instructions for Microsoft Azure AD: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#microsoft-azure-ad-provider
Instructions for OIDC: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#openid-connect-provider

Configuring Netmaker

After you have configured your OAuth provider, take note of the CLIENT_ID and CLIENT_SECRET. If you are using Azure for oauth, you may also want to note down the Azure tenant ID you wish to use.

Next, Configure Netmaker with the following environment variables. If any are left blank, OAuth will fail.

.. code-block::

text

AUTH_PROVIDER: "<azure-ad|github|google|oidc>"
CLIENT_ID: "<client id of your oauth provider>"
CLIENT_SECRET: "<client secret of your oauth provider>"
SERVER_HTTP_HOST: "api.<netmaker base domain>"
FRONTEND_URL: "https://dashboard.<netmaker base domain>"
AZURE_TENANT: "<only for azure, you may optionally specify the tenant for the OAuth>"
OIDC_ISSUER: "<only for oidc, your issuer endpoint for OIDC> ie. http://127.0.0.1:5556/dex"

After restarting your server, the Netmaker logs will indicate if the OAuth provider was successfully initialized:

.. code-block::

sudo docker logs netmaker

Once successful, users can click on the "Login with SSO" button in the login page to sign-in with your configured OAuth provider.

.. image:: images/oauth1.jpg
:alt: Login Oauth
:align: center

Oauth Users

Users are also allowed to join a Netmaker server via OAuth. They can do this by clicking the "Login with SSO" button on the dashboard's login page.

.. image:: images/user-mgmt-pending.png
:width: 80%
:alt: Pending Users
:align: center

From v0.23.1, new accounts would be added to a pending list and would require approval from an admin before they can access any resource. This version also allows whitelisting of email domains for OAuth users.
Server admins can do that by adding a comma-separated list of domains to the ALLOWED_EMAIL_DOMAINS environment variable. eg: ALLOWED_EMAIL_DOMAINS=example.net,example.com

File: references.rst

============== References

UI

Annotated screenshots of most UI components, detailing the configuration options of each field across Nodes, Networks, DNS, Ext Clients, Users, and more.

.. toctree::
:maxdepth: 2

ui-reference

API

Netmaker can be run without the UI, and all functions can be achieved via API calls

.. toctree::
:maxdepth: 2

api

Netclient

Netclient manages WireGuard on client devices (nodes)

:ref:netclient

NMCTL

nmctl is a CLI tool for interacting with the Netmaker API

.. toctree::
:maxdepth: 2

nmctl
nmctl-standalone

FAQ

Frequently Asked Questions

.. toctree::
:maxdepth: 2

support

Troubleshooting

.. toctree::
:maxdepth: 2

troubleshoot

File: requirements.txt

sphinx-material

File: server-installation.rst

================================= Advanced Server Installation

This section outlines installing the Netmaker server, including Netmaker, Netmaker UI, rqlite, and CoreDNS.

System Compatibility

Netmaker requires elevated privileges on the host machine to perform network operations. Netmaker must be able to modify interfaces and set firewall rules using iptables.

Typically, Netmaker is run inside of containers, using Docker or Kubernetes.

Netmaker can be run without containers, but this is not recommended. You must run the Netmaker binary, CoreDNS binary, database, and a web server directly on the host.

Each of these components has its own individual requirements and the management complexity increases exponentially by running outside of containers.

For first-time installs, we recommend the quick install guide. The following documents are meant for more advanced installation environments and are not recommended for most users. However, these documents can be helpful in customizing or troubleshooting your own installation.

.. _server-configuration-reference:

Server Configuration Reference

Netmaker sets its configuration in the following order of precedence:

Environment Variables: Typically values set in the Docker Compose. This is the most common way of setting server values.
Config File: Values set in the config/environments/*.yaml file.
Defaults: Default values set on the server if no value is provided in configuration.

In most situations, if you wish to modify a server setting, set it in the netmaker.env file, then run "docker kill netmaker" and "docker-compose up -d".

Variable Description

NM_EMAIL
Default ""

text

**Description** Email used for SSL certificates

NM_DOMAIN
Default: ""

text

**Description:**  Public IP of machine

SERVER_HOST
Default: (Server detects the public IP address of machine)

text

**Description:** The public IP of the server where the machine is running.

MASTER_KEY
Default: "secretkey"

text

**Description:** The admin master key for accessing the API. Change this in any production installation.

MQ_USERNAME
Default ""

text

**Description** The username to set for MQ access

MQ_PASSWORD
Default ""

text

**Description** The password to set for MQ access

INSTALL_TYPE
Default ce

text

**Description** The installation type to run on the server. "ce" will run community edition. "pro" will run professional edition if you have an on-prem tenant.

LICENSE_KEY

text

**Default** ""

**Description** The license key from your on-prem tenent needed to validate your professional installation.

NETMAKER_TENANT_ID

text

**Default** ""

**Description** The ID of your on-prem tenant used to validate your professional installation.

SERVER_IMAGE_TAG

text

**Default**

**Description** The tag used for the server docker image. You can set it to "latest" to get the most up to date version. To stay on a certain version set the tag to the version you would like. ex: v0.24.2. if using a pro image, add "-ee" after the version. ex: v0.24.2-ee.

UI_IMAGE_TAG

text

**Default** ""

**Description** The tag used for the netmaker ui docker image. You can set it to "latest" to get the most up to date version. To stay on a certain version set the tag to the version you would like. ex: v0.24.2.

METRICS_EXPORTER

text

**Default** off

**Description** This is a pro feature that exports metrics to the netmaker server.

PROMETHEUS

text

**Default** off

**Description** This is a pro feature. Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. It is used in out metrics collection

DNS_MODE

text

**Default** on

**Description** Enables DNS Mode, meaning all nodes will set hosts file for private dns settings

NETCLIENT_AUTO_UPDATE

text

**Default** enabled

**Description** Enable/Disable auto update of netclient

API_PORT

text

**Default** 8081

**Description** The HTTP API port for Netmaker. Used for API calls / communication from front end. If changed, need to change port of BACKEND_URL for netmaker-ui.

EXPORTER_API_PORT

text

**Default** 8085

**Description** the API port to set for the metrics exporter.

CORS_ALLOWED_ORIGIN

text

**Default** "*"

**Description** The "allowed origin" for API requests. Change to restrict where API requests can come from with comma-separated URLs. ex:- https://dashboard.netmaker.domain1.com,https://dashboard.netmaker.domain2.com

DISPLAY_KEYS

text

**Default** on

**Description** Show keys permanently in UI (until deleted) as opposed to 1-time display.

DATABASE

text

**Default** sqlite

**Description** Database to use - sqlite, postgres, or rqlite

SERVER_BROKER_ENDPOINT

text

**Default** ws://mq:1883 

**Description** The address of the mq server. If running from docker compose it will be "mq". Otherwise, need to input address. If using "host networking", it will find and detect the IP of the mq container. For EMQX websockets use `SERVER_BROKER_ENDPOINT=ws://mq:8083/mqtt`

VERBOSITY

text

**Default** 1

**Description** Logging verbosity level - 1, 2, 3, or 4

REST_BACKEND

text

**Default** on

**Description** Enables the REST backend (API running on API_PORT at SERVER_HTTP_HOST).

DISABLE_REMOTE_IP_CHECK

text

**Default** off

**Description** If turned "on", Server will not set Host based on remote IP check. This is already overridden if SERVER_HOST is set.

TELEMETRY

text

**Default** on

**Description** Whether or not to send telemetry data to help improve Netmaker. Switch to "off" to opt out of sending telemetry.

ALLOWED_EMAIL_DOMAINS

text

**Default** "*"

**Description** only mentioned domains will be allowded to signup using oauth, by default all domains are allowed

AUTH_PROVIDER

text

**Default** ""

**Description** You can use azure-ad, github, google, oidc

CLIENT_ID

text

**Default** ""

**Description** The client id of your oauth provider.

CLIENT_SECRET

text

**Default** ""

**Description** The client secret of your oauth provider.

FRONTEND_URL

text

**Default** ""

**Description** https://dashboard.<netmaker base domain>

AZURE_TENANT

text

**Default** ""

**Description** only for azure, you may optionally specify the tenant for the OAuth

OIDC_ISSUER

text

**Default** ""

**Description** https://oidc.yourprovider.com - URL of oidc provider

JWT_VALIDITY_DURATION

text

**Default** 43200

**Description** Duration of JWT token validity in seconds

RAC_AUTO_DISABLE

text

**Default** false

**Description** Auto disable a user's connected clients based on JWT token expiration

CACHING_ENABLED

text

**Default** true

**Description** if turned on data will be cached on to improve performance significantly (IMPORTANT: If HA set to `false` )

ENDPOINT_DETECTION

text

**Default** true

**Description** if turned on netclient checks if peers are reachable over private/LAN address, and choose that as peer endpoint

Compose File - Annotated

All environment variables and options are enabled in this file. It is the equivalent to running the "full install" from the above section. However, all environment variables are included and are set to the default values provided by Netmaker (if the environment variable was left unset, it would not change the installation). Comments are added to each option to show how you might use it to modify your installation.

As of v0.18.0, netmaker now uses a stun server (Session Traversal Utilities for NAT). This provides a tool for communications protocols to detect and traverse NATs that are located in the path between two endpoints. By default, netmaker uses publicly available STUN servers. You are free to set up your own stun severs and use those to augment/replace the public STUN servers. Update the STUN_LIST to list the STUN servers you wish to use. Two resources for installing your own STUN server are:

https://ourcodeworld.com/articles/read/1175/how-to-create-and-configure-your-own-stun-turn-server-with-coturn-in-ubuntu-18-04

https://cloudkul.com/blog/how-to-install-turn-stun-server-on-aws-ubuntu-20-04/

There are also some environment variables that have been changed, or removed. Your updated docker-compose and .env files should look like this.

.. literalinclude:: ./examplecode/docker-compose.v0.20.3.yml
:language: YAML

.. literalinclude:: ./examplecode/netmaker.default.env
:language: YAML

Your Caddyfile should look like this.

.. code-block:: cfg

text

{
   
    email YOUR_EMAIL
}

# Dashboard
https://dashboard.NETMAKER_BASE_DOMAIN {
    # Apply basic security headers
    header {
            # Enable cross origin access to *.NETMAKER_BASE_DOMAIN
            Access-Control-Allow-Origin *.NETMAKER_BASE_DOMAIN
            # Enable HTTP Strict Transport Security (HSTS)
            Strict-Transport-Security "max-age=31536000;"
            # Enable cross-site filter (XSS) and tell browser to block detected attacks
            X-XSS-Protection "1; mode=block"
            # Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
            X-Frame-Options "SAMEORIGIN"
            # Prevent search engines from indexing
            X-Robots-Tag "none"
            # Remove the server name
            -Server
    }

    reverse_proxy http://netmaker-ui
}

# API
https://api.NETMAKER_BASE_DOMAIN {
        reverse_proxy http://netmaker:8081
}

# MQ
wss://broker.NETMAKER_BASE_DOMAIN {
        reverse_proxy ws://mq:8883
}

Available docker-compose files

The default options for docker-compose can be found here: https://github.com/gravitl/netmaker/tree/master/compose

The following is a brief description of each:

docker-compose.yml <https://github.com/gravitl/netmaker/blob/master/compose/docker-compose.yml>_ -= This maintains the most recommended setup at the moment, using the caddy proxy.
docker-compose.pro.yml <https://github.com/gravitl/netmaker/blob/master/compose/docker-compose.pro.yml>_ -= This is the compose file needed for Netmaker Professional. You will need a licence and tenant id from Netmaker's licence dashboard <https://account.netmaker.io/>_ .

.. _Emqx:

EMQX

Netmaker offers an EMQX option as a broker for your server. The main configuration changes between mosquitto and EMQX is going to take place in the docker-compose.yml, netmaker.env and the Caddyfile.

You can find the EMQX docker-compose file in the netmaker repo <https://github.com/gravitl/netmaker/blob/master/compose/docker-compose-emqx.yml>_.

You should not need to make any changes to the docker-compose-emqx.yml file. Just download this file using the command provided below on the same directory as netmaker.env file. It will grab information from the netmaker.env file.

.. code-block::

text

wget https://raw.githubusercontent.com/gravitl/netmaker/master/compose/docker-compose-emqx.yml

In your Caddyfile, the only change you need to make is in the mq block.

.. code-block:: cfg

text

# MQ
wss://broker.{$NM_DOMAIN} {
    reverse_proxy ws://mq:8083
}

basically just replace the port number on line reverse_proxy ws://mq:8883 with 8083 emqx websocket port number.

In your netmaker.env file, just replace the line SERVER_BROKER_ENDPOINT="ws://mq:1883" with SERVER_BROKER_ENDPOINT=ws://mq:8083/mqtt. Basically just change the port to 8083 and add /mqtt after that.

In your docker-compose.yml file, add /mqtt at the end of this line BROKER_ENDPOINT=wss://broker.${NM_DOMAIN} which results in BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt.

.. code-block:: yaml

text

- BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt
- BROKER_TYPE=emqx
- EMQX_REST_ENDPOINT=http://mq:18083

Then two new lines - BROKER_TYPE=emqx and - EMQX_REST_ENDPOINT=http://mq:18083 needs to be added after the line - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}/mqtt as shown above.

If you are using a professional server, you will need to make changes to your netmaker-exporter section in your docker-compose.override.yml file.

.. code-block:: yaml

text

netmaker-exporter:
    container_name: netmaker-exporter
    image: gravitl/netmaker-exporter:latest
    restart: always
    depends_on:
        - netmaker
    environment:
        SERVER_BROKER_ENDPOINT: "ws://mq:8083/mqtt"
        BROKER_ENDPOINT: "wss://broker.nm.${NM_DOMAIN}/mqtt"
        PROMETHEUS_HOST: "https://prometheus.${NM_DOMAIN}"

At this point you should be able to docker-compose down && docker-compose up -d && docker-compose -f docker-compose-emqx.yml up -d. Your docker logs mq should look something like this.

.. code-block:: cfg

text

Listener ssl:default on 0.0.0.0:8883 started.
Listener tcp:default on 0.0.0.0:1883 started.
Listener ws:default on 0.0.0.0:8083 started.
Listener wss:default on 0.0.0.0:8084 started.
Listener http:dashboard on :18083 started.
EMQX 5.0.9 is running now!

Your server is now running on an EMQX broker. you can view your EMQX dashboard with http://<serverip>:18083/. The signin credentials are the EMQX_DASHBOARD__DEFAULT_USERNAME and EMQX_DASHBOARD__DEFAULT_PASSWORD located in your netmaker.env file. This dashboard will give you all the information about your mq activity going on in your netmaker server.

Nginx Reverse Proxy Setup with https

The Swag Proxy <https://github.com/linuxserver/docker-swag>_ makes it easy to generate a valid SSL certificate for the config below. Here is the documentation <https://docs.linuxserver.io/general/swag>_ for the installation.

The following file configures Netmaker as a subdomain. This config is an adaption from the swag proxy project.

./netmaker.subdomain.conf:

.. code-block:: nginx

text

server {
    # Redirect HTTP to HTTPS.
    listen 80;
    server_name *.netmaker.example.org; # Please change to your domain
    return 301 https://$host$request_uri;
    }

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name dashboard.netmaker.example.org; # Please change to your domain
    include /config/nginx/ssl.conf;
    location / {
        proxy_pass http://<NETMAKER_IP>:8082;
        }
    }

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name api.netmaker.example.org; # Please change to your domain
    include /config/nginx/ssl.conf;

    location / {
        proxy_pass http://<NETMAKER_IP>:8081;
        proxy_set_header		Host api.netmaker.example.org; # Please change to your domain
        proxy_pass_request_headers	on;
        }
    }

Nginx Proxy Manager Setup

To use Netmaker with Nginx Proxy Manager, three proxy hosts should be added, one for each subdomain used by netmaker. Each subdomain should have SSL enable and be configured as follows:

.. code-block::

text

api.netmaker.example.com: 
Forward Hostname/IP: netmaker
Forward Port: 8081
dashboard.netmaker.example.com:
Forward Hostname/IP: netmaker-ui
Forward Port: 80
grpc.netmaker.example.com:
Forward Hostname/IP: netmaker
Forward Port: 50051
Custom Locations:
Add location /
Forward Hostname/IP: netmaker
Forward Port: 50051
Custom config (gear button): grpc_pass netmaker:50051;

The following is a cleaned up config generated by Nginx Proxy Manager to show how nginx can be configured to support Netmaker. This does not include the neccessary SSL configuration.

.. code-block:: nginx

text

# ------------------------------------------------------------
# dashboard.netmaker.example.com
# ------------------------------------------------------------
server {
  set $forward_scheme http;
  set $server         "netmaker-ui";
  set $port           80;
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name dashboard.netmaker.example.com;
  location / {
    # Proxy!
    include conf.d/include/proxy.conf;
	#above file includes:
	#add_header       X-Served-By $host;
	#proxy_set_header Host $host;
	#proxy_set_header X-Forwarded-Scheme $scheme;
	#proxy_set_header X-Forwarded-Proto  $scheme;
	#proxy_set_header X-Forwarded-For    $remote_addr;
	#proxy_set_header X-Real-IP          $remote_addr;
	#proxy_pass       $forward_scheme://$server:$port$request_uri;
  }
}

# ------------------------------------------------------------
# api.netmaker.example.com
# ------------------------------------------------------------
server {
  set $forward_scheme http;
  set $server         "netmaker";
  set $port           8081;
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
  server_name api.netmaker.example.com;
  location / {
    # Proxy!
    include conf.d/include/proxy.conf;
	#above file includes:
	#add_header       X-Served-By $host;
	#proxy_set_header Host $host;
	#proxy_set_header X-Forwarded-Scheme $scheme;
	#proxy_set_header X-Forwarded-Proto  $scheme;
	#proxy_set_header X-Forwarded-For    $remote_addr;
	#proxy_set_header X-Real-IP          $remote_addr;
	#proxy_pass       $forward_scheme://$server:$port$request_uri;
  }
}

# ------------------------------------------------------------
# grpc.netmaker.example.com
# ------------------------------------------------------------
server {
  set $forward_scheme http;
  set $server         "netmaker";
  set $port           50051;
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
  server_name grpc.netmaker.example.com;
  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Scheme $scheme;
    proxy_set_header X-Forwarded-Proto  $scheme;
    proxy_set_header X-Forwarded-For    $remote_addr;
    proxy_set_header X-Real-IP          $remote_addr;
    proxy_pass       http://netmaker:50051;
    grpc_pass netmaker:50051;
  }
}

.. _HAInstall:

Highly Available Installation (Kubernetes)

Netmaker comes with a Helm chart to deploy with High Availability on Kubernetes:

.. code-block::

text

helm repo add netmaker https://gravitl.github.io/netmaker-helm/
helm repo update

Requirements

To run HA Netmaker on Kubernetes, your cluster must have the following:

RWO and RWX Storage Classes
An Ingress Controller and valid TLS certificates

This chart can currently generate ingress for:
- Nginx Ingress + LetsEncrypt/Cert-Manager
To generate automatically, make sure one of the two is configured for your cluster
Ability to set up ingress route for Secure Web Sockets

Nginx Ingress supports Secure Web Sockets (WSS) by default. If you are not using Nginx Ingress, you must route external traffic from broker.domain to the MQTT service, and provide valid TLS certificates.

One option is to set up a Load Balancer which routes broker.domain:443 to the MQTT service on port 8883.

We do not provide guidance beyond this, and recommend using an Ingress Controller that supports websockets.

Furthermore, the chart will by default install and use a postgresql cluster as its datastore:

.. list-table::
:header-rows: 1

- Repository
- Name
- Version
- https://charts.bitnami.com/bitnami
- postgresql-ha
- 7.11.0

Example Installations:

An annotated install command:

.. code-block::

text

helm install netmaker/netmaker --generate-name \ # generate a random id for the deploy 
--set baseDomain=nm.example.com \ # the base wildcard domain to use for the netmaker api/dashboard/mq ingress 
--set server.replicas=3 \ # number of server replicas to deploy (3 by default) 
--set ingress.enabled=true \ # deploy ingress automatically (requires nginx and cert-manager + letsencrypt) 
--set ingress.kubernetes.io/ingress.class=nginx \ # ingress class to use
--set ingress.cert-manager.io/cluster-issuer=letsencrypt-prod \ # LetsEncrypt certificate issuer to use
--set postgresql-ha.postgresql.replicaCount=2 \ # number of DB replicas to deploy (default 2)

The below command will install netmaker with two server replicas, a coredns server, and ingress with routes of api.nm.example.com, grpc.nm.example.com, and dashboard.nm.example.com. CoreDNS will be reachable at 10.245.75.75, and will use NFS to share a volume with Netmaker (to configure dns entries).

.. code-block::

text

helm install netmaker/netmaker --generate-name --set baseDomain=nm.example.com \
--set replicas=2 --set ingress.enabled=true --set dns.enabled=true \
--set dns.clusterIP=10.245.75.75 --set dns.RWX.storageClassName=nfs \
--set ingress.className=nginx

The below command will install netmaker with three server replicas (the default), no coredns, and ingress with routes of api.netmaker.example.com, grpc.netmaker.example.com, and dashboard.netmaker.example.com. There will be one UI replica instead of two and one database instance instead of two. Traefik will look for a ClusterIssuer named "le-prod-2" to get valid certificates for the ingress.

.. code-block::

text

helm3 install netmaker/netmaker --generate-name \
--set baseDomain=netmaker.example.com --set postgresql-ha.postgresql.replicaCount=1 \
--set ui.replicas=1 --set ingress.enabled=true \
--set ingress.tls.issuerName=le-prod-2 --set ingress.className=traefik

Recommended Settings:

This install has some notable exceptions:

Ingress must be configured on your cluster, with cluster issuer for TLS certs
DNS will be disabled

Below, we discuss the considerations for Ingress, Kernel WireGuard, and DNS.

MQ

The MQ Broker is deployed either with Ingress (Nginx ) preconfigured, or without. If you are using an ingress controller other than Nginx, Netmaker's MQTT will not be complete. "broker.domain" must reach the MQTT service at port 8883 over WSS (Secure Web Sockets).

Ingress

To run HA Netmaker, you must have ingress installed and enabled on your cluster with valid TLS certificates (not self-signed). If you are running Nginx as your Ingress Controller and LetsEncrypt for TLS certificate management, you can run the helm install with the following settings:

--set ingress.enabled=true
--set ingress.annotations.cert-manager.io/cluster-issuer=<your LE issuer name>

If you are not using Nginx and LetsEncrypt, we recommend leaving ingress.enabled=false (default), and then manually creating the ingress objects post-install. You will need three ingress objects with TLS:

dashboard.
api.
broker.

There are some example ingress objects in the kube/example folder.

DNS

By Default, the helm chart will deploy without DNS enabled. To enable DNS, specify with:

--set dns.enabled=true

This will require specifying a RWX storage class, e.g.:

--set dns.RWX.storageClassName=nfs

This will also require specifying a service address for DNS. Choose a valid ipv4 address from the service IP CIDR for your cluster, e.g.:

--set dns.clusterIP=10.245.69.69

This address will only be reachable from hosts that have access to the cluster service CIDR. It is only designed for use cases related to k8s. If you want a more general-use Netmaker server on Kubernetes for use cases outside of k8s, you will need to do one of the following:

bind the CoreDNS service to port 53 on one of your worker nodes and set the COREDNS_ADDRESS equal to the public IP of the worker node.
Create a private Network with Netmaker and set the COREDNS_ADDRESS equal to the private address of the host running CoreDNS. For this, CoreDNS will need a node selector and will ideally run on the same host as one of the Netmaker server instances.

Values

To view all options for the chart, please visit the README in the netmaker-helm chart repo here <https://github.com/gravitl/netmaker-helm?tab=readme-ov-file#values>_ .

Security Settings

In some cases, it is useful to secure your web dashboard behind a firewall so it can only be accessed in that location. However, you may not want the API behind that firewall so the other nodes can interact with the network without the heightened security. This can be done in Your Caddyfile if you are using caddy.

For Caddy

In your /root/Caddyfile look in the Dashboard section for reverse_proxy http://netmaker-ui

Above that line add the following

.. code-block:: cfg

text

@blocked not remote_ip   
respond @blocked "Nope" 403

Replace the placeholders with your whitelist IP ranges.

For Traefik

In the labels section, add the following line:

.. code-block:: yaml

text

traefik.http.middlewares.nmui-security-1.ipwhitelist.sourcerange=YOUR_IP_CIDR

Then look for this line:

.. code-block:: yaml

text

traefik.http.routers.netmaker-ui.middlewares=nmui-security@docker

and change it to this:

.. code-block:: yaml

text

traefik.http.routers.netmaker-ui.middlewares=nmui-security-1@docker,nmui-security@docker

Replace YOUR_IP_CIDR with the whitelist IP range (can be multiple ranges).

After changes are made for your reverse proxy, docker-compose down && docker-compose up -d and you should be all set. You can now keep your dashboard secure and your API more available without having to change netmaker-ui ports.

Setup Netmaker on IPv6 only machine

This is not a guide how to add an overlay network(with IPv6) in Netmaker, it can be found in Setup <https://docs.netmaker.io/getting-started.html#setup>_ page.
This is to setup Netmaker working on an IPv6 only machine.

About the install script nm-quick.sh

At the moment which the document is written, the install script nm-quick.sh only supports IPv4. For the installation, the IPv4 needs to be enabled temporary anyway.

Add AAAA record for domain name resolution

The Netmaker client communicates with Netmaker server by domain name. AAAA record here is to resolve the domain name to IPv6 address.

By default, Netmaker works on IPv4. Because Docker works on IPv4 by default. After the installation, there are several steps to enable IPv6 for Docker and Netmaker.

Enable IPv6 support for Docker

Add/Edit the configuration file /etc/docker/daemon.json:

.. code-block:: json

text

{
  "experimental": true,
  "ip6tables": true
}

Restart the Docker daemon for your changes to take effect.

.. code-block::

text

sudo systemctl restart docker

Create a new IPv6 network, for example,

.. code-block::

text

docker network create --ipv6 --subnet 2001:0DB8::/112 ip6net

where "ip6net" is the network name, "2001:0DB8::/112" is the network range.

Enable IPv6 support for Netmaker

Edit docker-compose.yml file and add the following lines at the bottom.

.. code-block:: yaml

text

networks:
  ip6net:
    external: true

The same in docker-compose.yml file, add networks field for every service.

.. code-block:: yaml

text

networks:
  - ip6net

Run commands "docker-compose down" and "docker-compose up -d" to restart Netmaker server

File: support.rst

====== FAQ

Is Netmaker a VPN like NordVPN?

No. Netmaker makes Virtual Networks, which are technically VPNs, but different. It's more like a corporate VPN, or a VPC (if you're familiar with AWS). Netmaker is often compared to OpenVPN, Tailscale, or Nebula.

If you're looking to achieve self-hosted web browsing, with functionality similar to NordVPN, ExpressVPN, Surfshark, Tunnelbear, or Private Internet Access, this is probably not the project for you. Technically, you can accomplish this with Netmaker, but it would be a little like using an all-terrain vehicle for stock car racing.

There are many good projects out there that support general internet privacy using WireGuard. Here are just a few of them:

https://github.com/trailofbits/algo
https://github.com/pivpn/pivpn
https://github.com/subspacecloud/subspace
https://github.com/mullvad/mullvadvpn-app

Do you have an 'Exit Nodes' feature?

Please see the :doc:Egress Gateway <./egress-gateway> documentation.

Do you offer any business or professional support?

Yes, please contact [email protected] or visit https://gravitl.com/plans.

Telemetry

As of v0.10.0, Netmaker collects "opt-out" telemetry data. To opt out, simply set "TELEMETRY=off" in your netmaker.env file.

Please consider participating in telemetry, as it helps us focus on the features and bug fixes that are most useful to users. Netmaker is a broad platform, and without this data, it is difficult to know where the team should spend its limited resources.

The following is the full list of telemetry data we collect. Besides "Server Version" all data is simply an integer count:

Randomized server ID
Count of nodes
Count of "non-server" nodes
Count of Remote Access Clients
Count of networks
Count of users
Count of linux nodes
Count of freebsd nodes
Count of macos nodes
Count of windows nodes
Count of docker nodes
Count of k8s nodes
Server version

We use PostHog <https://posthog.com/>_, an open source and trusted framework for telemetry data.

To look at exactly we collect telemetry, you can view the source code under serverctl/telemetry.go: https://github.com/gravitl/netmaker/blob/master/serverctl/telemetry.go

Contact

If you need help, try the Discord or open a GitHub ticket.

Email: [email protected]

Discord: https://discord.gg/zRb9Vfhk8A

File: troubleshoot.rst

================= Troubleshooting

Common Issues

How can I connect my Android or IOS device to my Netmaker VPN?
Currently meshing one of these devices is not supported, however, it will be soon.
For now, you can connect to your VPN by making one of the nodes a Remote Access Gateway, then
create an Ext Client for each device. Finally, use the official WG app or another
WG configuration app to connect via QR or download the device's WireGuard configuration.

I've made changes to my nodes but the nodes themselves haven't updated yet, why?
Please allow your nodes to complete a check-in or two, in order to reconfigure themselves.
In some cases, it could take up to a minute or so.

Do I have to use access keys to join a network?
Although keys are the preferred way to join a network, Netmaker does allow for manual node sign-ups.
Simply turn on "allow manual signups" on your network and nodes will not connect until you manually approve each one.

Is there a community or forum to ask questions about Netmaker?
Yes, we have an active Discord <https://discord.gg/Pt4T9y9XK8>_ community and issues on our GitHub <https://github.com/gravitl/netmaker/issues>_ are answered frequently!
You can also sign-up for updates at our gravitl site <https://gravitl.com/>_!

How can I get additional support for my business?
Check out our business support subscriptions at https://gravitl.com/plans. Subscription holders can also purchase consulting credits via the site.

Server

How do I use a private address from the Netmaker Server? How do I contact nodes using their private addresses from the server?
Default nodes appear in each network with the "netmaker" name. These nodes are created by, and attached to, the server. The server is contained in docker, meaning these clients are also contained in docker. Their networking stack is also contained in docker. The "netmaker" nodes are meant to function as network utilities. They assist with UDP Hole Punching and can run Relays, Egresses and Remote Access Gateways. However, they are meant to stay contained in the server. They do not touch the host networking stack.

If you want to give the physical server / VM a private IP in the netmaker network, you must deploy an additional node using the standard netclient. The only note here is that the server consumes ports 51821-51831, so you will need to give it a port outside this range, e.x. ./netclient join <token> --port 51835. You may also need to add --udpholepunch no.

One a netclient is deployed to the underlying server/VM, you will be able to use the private address to reach other nodes from the host, or to reach the server over the private network.

I upgraded from 0.7 to 0.8 and now I don't have any data in my server!
In 0.8, SQLite becomes the default database. If you were running with rqlite, you must set the DATABASE environment variable to rqlite in order to continue using rqlite.

Can I secure/encrypt all the traffic to my server and UI?
This can be fairly simple to achieve assuming you have access to a domain and are familiar with Nginx.
Please refer to the quick-start guide to see!

Can I connect multiple nodes (mesh clients) behind a single firewall/router?
As of v0.18.0, netmaker now uses a stun server (Session Traversal Utilities for NAT). This provides a tool for communications protocols to detect and traverse NATs that are located in the path between two endpoints.

What are the minimum specs to run the server?
We recommend at least 1 CPU and 2 GB Memory.

Does this support IPv6 addressing?
Yes, Netmaker supports IPv6 addressing. When you create a network, just make sure to turn on Dual Stack.
Nodes will be given IPv6 addresses along with their IPv4 address. It does not currently support IPv6 only.

Does Netmaker support Raft Consensus?
Netmaker does not directly support it, but it uses rqlite <https://github.com/rqlite/rqlite>_ (which supports Raft) as the database.

How do I uninstall Netmaker?
There is no official uninstall script for the Netmaker server at this time. If you followed the quick-start guide, simply run sudo docker-compose -f docker-compose.quickstart.yml down --volumes
to completely wipe your server. Otherwise kill the running binary and it's up to you to remove database records/volumes.

MQ

If your client installs keep hanging or erroring out, the most common issue as of 0.13 is with MQ. There have been some architecture changes that are very important to account for in the upgrade.

Please follow this Gist if you are encountering issues with 0.13+: https://gist.github.com/mattkasun/face2a7c1f32031a2126ff7243caad12

UI

I want to make a separate network and give my friend access to only that network.
Simply navigate to the UI (as an admin account). Select users in the top left and create an account for them.
Select the network(s) to give them and they should be good to go! They are an admin of that network(s) only now.

I'm done with an access key, can I delete it?
Simply navigate to the UI (as an admin account). Select your network of interest, then select the Access Keys tab.
Then delete the rogue access key.

I can't delete my network, why?
You MUST remove all nodes in a network before you can delete it.

Can I have multiple nodes with the same name?
Yes, nodes can share names without issue. It may just be harder for you to know which is which.

Netclient

How do I connect a node to my Netmaker network with Netclient?
First get your access token (not just access key), then run sudo netclient join -t <access token>.
NOTE: netclient may be under /etc/netclient/, i.e run sudo /etc/netclient/netclient join -t <access token>

How do I disconnect a node on a Netmaker network?
In order to leave a Netmaker network, run sudo netclient leave -n <network-name>

How do I check the logs of my agent on a node?
You will need sudo/root permissions, but you can run sudo systemctl status netclient@
or you may also run sudo journalctl -u netclient@<network name>.
Note for journalctl: you should hit the end key to get to view the most recent logs quickly or use journalctl -u netclient@<network name> -f instead.

Can I check the configuration of my node on the node?
A: Yes, on the node simply run sudo cat /etc/netclient/netconfig-<network name> and you should see what your current configuration is!
You can also see the current WireGuard configuration with sudo wg show

I am done with the agent on my machine, can I uninstall it?
Yes, on the node simply run sudo /etc/netclient/netclient uninstall.

I am running SELinux and when I reboot my node I get a permission denied in my netclient logs and it doesn't connect anymore, why?
If you're running SELinux, it will interfere with systemd's ability to restart the client properly. Therefore, please run the following:
.. code-block::

text

sudo semanage fcontext -a -t bin_t '/etc/netclient/netclient' 
sudo chcon -Rv -u system_u -t bin_t '/etc/netclient/netclient' 
sudo restorecon -R -v /etc/netclient/netclient

I have a handshake with a peer but can't ping it, what gives?
This is commonly due to incorrect MTU settings. Typically, it will be because MTU is too high. Try setting MTU lower on the node. This can be done via netconfig, or by editing the node in the UI.

Note: We recommend a minimum MTU of 1280 due to most router configs having an expectation of a standard MTU setting and IPv6 requiring 1280 as a minimum. going lower than that may cause issues.

I have a hard to reach machine behind a firewall or a corporate NAT, what can I do?
In this situation, you can use the Relay Server functionality introduced in Netmaker v0.8 to designate a node as a relay to your "stuck" machine. Simply click the button to make a node into a relay and tell it to relay traffic to this hard-to-reach peer.

I am unable to run Netclient on Windows due to an error that mentions Fyne and a window creation error

Older versions of Windows and/or virtualized environments may not support the Netclient UI. To fix this, download and install Mesa 3D. One way is follow these steps:

Download Mesa 3D (e.g., from here <https://fdossena.com/?p=mesa/index.frag>_) to get OpenGL for your OS
Get 7-zip <https://www.7-zip.org/download.html>_, install it and extract the .7z Mesa 3D download (unless you download outside of Windows and extract there)
Drop the resulting .dll in the location of the .exe you're trying to run
Double click the netclient.exe
Netclient should start normally (the Fyne error should not appear)

CoreDNS

Is CoreDNS required to use Netmaker?
CoreDNS is not required. Simply start your server with DNS_MODE="off".

What is the minimum DNS entry value I can use?
Netmaker supports down to two characters for DNS names for your networks domains**

File: ui-reference.rst

This page contains annotated screenshots of most UI components, detailing the configuration options of each field across Nodes, Networks, DNS, Ext Clients, Users, and more.

Here is an arcade showing a walkthrough of our new UI. <https://app.arcade.software/share/Jdl7PnnqIbot3IkqvIaf>_

Authentication

When you start Netmaker for the first time, you will be prompted to create a superadmin account from the UI like below

.. image:: images/ui-signup.png
:width: 80%
:alt: sign up
:align: center

(1) Username: Enter a unique username for the admin user.
(2) Password: Enter a secure password for your new user.
(3) Password Confirmation: Repeat the password for verification.
(4) Signup with OAuth: Button to signup with OAuth. From v0.23.1, OAuth users will need further approval from a server admin to gain access.

.. image:: images/ui-login.png
:width: 80%
:alt: log in
:align: center

(1) Username: Enter your username.
(2) Password: Enter your password.
(3) Login: Button to login.
(4) Login with OAuth: Button to login with OAuth.

Dashboard

.. image:: images/ui-1.jpg
:width: 80%
:alt: dashboard
:align: center

Networks

Create

.. image:: images/ui-2.png
:width: 80%
:alt: create network
:align: center

.. code-block::

(1) Autofill: Provides sensible defaults for network details and makes up a name.
(2) Network Name: The name of the network. Character limited, as this translates to the interface name on hosts (nm-)
(3) Address Range: The CIDR of the network. Must be a valid IPv4 Subnet and should be a private address range.
(4) Default Access Control: Indicates the default ACL value for a node when it joins in respect to it's peers (enabled or disabled).

Hosts

In simple terms, a host is a computer or machine running the netclient software. Netmaker UI allows an admin to conviniently view and configure some host settings remotely.

Host List

.. image:: images/hosts-list.png
:width: 80%
:alt: hosts list
:align: center

(1) Host Name: Friendly name of the host. Clicking it opens a view to allow admins manage hosts.
(2) Endpoint: The public IP address of the host.
(3) Public Port: Public port of the host.
(4) Version: Indicates the version of netclient the host is running.
(5) Health Status: Indicates the connectivity of the host.
(6) Sync: Synchronise the host with the server; this triggers the host to pull latest network/server state.
(7) Actions: Quick actions that can be performed on the host.

Host Create

A host is automatically created on a server once a netclient (a machine running netclient) joins any network on the server.

Host Details

.. image:: images/host-details.png
:width: 80%
:alt: host details
:align: center

The following information is present under the host details tab:

(1) ID: Unique identifier for the host
(2) Name: Name of the host. Defaults to the machine's name.
(3) Version: Version of netclient the host is running.
(4) Operating System: Operating system (OS) the machine is running.
(5) Public Key: Public key of the host. distributed to other hosts.
(6) MTU: Maximum Transmission Unit (MTU) of the host
(7) Listen Port: The wiregaurd listen port.
(8) Proxy Listen Port: The netclient proxy listen port. this is used if Proxy Enabled is set to true. (No longer available from v0.20.5)
(9) Verbosity: Log verbosity (ranges from 1-4). Indicates level of detail the host (netclient) will output to logs.
(10) Default Interface: Default network interface used by the host.
(11) MAC Address: Media Access Control (MAC) address of the host machine.
(12) Is Default: Indicates whether the host is a default node. Hosts that are default nodes will automatically join any created network.
(13) Debug: Flag to enable additional logging on client.
(14) Proxy Enabled: Indicates whether a host is running netclient proxy. (No longer available from v0.20.5)
(15) Is Static: Indicaates whether the host's endpoint is static or not.
(16) Interfaces: Lists the available network interface for the host.

A host can be deleted from the UI. All associated nodes must be manually removed however, before deleting a host.

.. image:: images/host-nets.png
:width: 80%
:alt: host details
:align: center

Nodes

Node List

.. image:: images/nodes-1.png
:width: 80%
:alt: nodes list
:align: center

(1) Search Hosts: Look up a host by name.
(2) Host Name: Name of host. By default set to hostname of machine.
(3) Private Addresses: Private IPs of host within network.
(4) Public Address: Public IP of host.
(5) Status: Indicates how recently the host checked into the server. Displays "Warning" after 5 minutes and "Error" after 30 minutes without a check in. Does not explicitly indicate the health of the node's virtual network connections; however a healthy host will check-in regularly.
(6) Actions: Dropdown list of actions that can be performed on a host, including disconnecting and deleting the host.

A node pending deletion will be grayed out.

Create Egress

.. image:: images/ui-6.png
:width: 80%
:alt: dashboard
:align: center

(1) Egress Gateway Ranges: A comma-separated list of the subnets for which the gateway will route traffic. For instance, with Kubernetes this could be both the Service Network and Pod Network. For a standard VPN, Netmaker can use a list of the public CIDR's (see the docs). Typically, this will be something like a data center network, VPC, or home network.
(2) Interface: The interface on the machine used to access the provided egress gateway ranges. For instance, on a typical linux machine, the interface for public traffic would be "eth0". Usually you will need to check on the machine first to find the right interface. For instance, on Linux, you can find the interface by running this: ip route get

Create Relay

Check host section on hosts_. A relay can be created under host settings.

Edit Node / Node Details

.. image:: images/ui-5.jpg
:width: 80%
:alt: dashboard
:align: center

.. image:: images/ui-5-5.png
:width: 80%
:alt: dashboard
:align: center

(1) Edit Edit the node's details
(2) ACLs View the node's Access Control List (ACL)
(3) Metrics View the node's metrics
(4) Host View the node's associated host
(5) Delete Delete the node
(6) Endpoint: The (typically public) IP of the machine, which peers will use to reach it, in combination with the port. If changing this value, make sure Roaming is turned off, since otherwise, the node will check to see if there is a change in the public IP regularly and update it.
(7) Dynamic Endpoint: The endpoint may be changed automatically. Switching this off (indicating static endpoint) means the endpoint will stay the same until you change it. This can be good to set if the machine is a server sitting in a location that is not expected to change. It is also good to have this switched off for Remote Access gateway, Egress, and Relay Servers, since they should be in a reliable location.
(8) Listen Port: The port used by the node locally. This value is ignored if UDP Hole Punching is on, because port is set dynamically every time interface is created. If UDP Hole Punching is off, the port can be set to any reasonable (and available) value you'd like for the local machine.
(9) IP Address: The primary private IP address of the node. Assigned automatically by Netmaker but can be changed to whatever you want within the Network CIDR.
(10) IPv6 Address: (Only if running dual stack) the primary private IPv6 address of the node. Assigned automatically by Netmaker but can be changed to whatever you want within the Network CIDR.
(11) Local Address: The "locally reachable" address of the node. Other nodes will take note of this to see if this node is on the same network. If so, they will use this address instead of the public "Endpoint." If running a few nodes inside of a VPC, home network, or similar, make sure the local address is populated correctly for faster and more secure inter-node communication.
(12) Node Name: The name of the node within the network. Hostname by default but can be anything (within the character limits).
(13) Public Key: (Uneditable) The public key of the node, distributed to other peers in the network.
(14) Persistent Keepalive: How often packets are sent to keep connections open with other peers.
(15) Last Modified: Timestamp of the last time the node config was changed.
(16) Node Expiration Datetime: If a node should become invalid after a length of time, you can set it in this field, after which time, it will lose access to the network and will not populate to other nodes. Useful for scenarios where temporary access is granted to 3rd parties.
(17) Last Checkin: Unix timestamp of the last time the node checked in with the server. Used to determine generic health of node.
(18) MAC Address: The hardware Media Access Control (MAC) address of the machine. Used to be used as the unique ID, but is being depreciated.
(19) Egress Gateway Ranges: If Egress is enabled, the gateway ranges that this machine routes to.
(20) Local Range: If IsLocal has been enabled on the network, this is the local range in which the node will look for a private address from it's local interfaces, to use as an endpoint.
(21) Node Operating System: The OS of the machine.
(22) MTU: The MTU that the node will use on the interface. If "wg show" displays a valid handshake but pings are not working, many times the issue is MTU. Making this value lower can solve this issue. Some typical values are 1024, 1280, and 1420.
(23) Network: The network this node belongs to.
(24) Node ACL Rule The current ACL rule for this node in the network
(25) Is DNS On: DNS is solely handled by resolvectl at the moment, which is on many Linux distributions. For anything else, this value should remain off. If you wish to configure DNS for non-compatible systems, you must do so manually.
(26) Is Local: If on, will only communicate over the local address (Assumes IsLocal tuned to 'yes' on the network level.)
(27) Connected Indicates whether the node has is connected to the network

Remote Access

.. image:: images/ui-8.jpg
:width: 80%
:alt: dashboard
:align: center

(1) Gateway Name / IP Address: Information about which Node is the Remote Access Gateway.
(2) Add Client Config: Button to generate a new Remote Access Client configuration file.
(3) Client ID: The randomly-generated name of the client. Click on the ID to change the name to something sensible.
(4) IP Address: The private ip address of the ext client.
(5) QR Code: If joining form iOS or Android, open the WireGuard app and scan the QR code to join the network.
(6) Download Client Configuration: If joining from a laptop/desktop, download the config file and run "wg-quick up /path/to/config"
(7) Delete: Delete the ext client and remove its network access.

DNS

.. image:: images/ui-10.jpg
:width: 80%
:alt: dashboard
:align: center

(1) DNS Name: The private DNS entry. Must end in "." (added automatically). This avoids conflicts between networks.
(2) IP Address: The IP address of the entry. Can be anything (public addresses too!) but typically a node IP.
(3) Select Node Address: Select a node name to populate its IP address automatically.

Create / Edit Users

.. image:: images/ui-11.jpg
:width: 80%
:alt: dashboard
:align: center

(1) Username: Specify Username.
(2) Password: Specify password.
(3) Confirm Password: Confirm password.
(4) Make Admin: Make into a server admin or "super admin", which has access to all networks and server-level settings.
(5) Networks: If not made into an "admin", select the networks which this user has access to. The user will be a "network admin" of these networks, but other networks will be invisible/unaccessible.

Node Graph

.. image:: images/node-graph-1.png
:width: 80%
:alt: dashboard
:align: center

View all nodes in your network, zoom in, zoom out, and search for node names.
hover: Hover over a node to see its direct connections.

Access Control Lists

.. image:: images/acls-3.png
:width: 80%
:alt: ACLs
:align: center

(1) Reset: Reset your changes without submitting.
(2) Allow All: Enable all p2p connections
(3) Block All: Disable all p2p connections. Makes building up a Zero Trust network easier.
(4) (allowed): Click to switch a connection to "deny." Note that node names are higlighted on the side and top to track location.
(5) (blocked): Click to switch a connection to "allow."
(6) Submit Changes: Click once you are ready to submit. Will send message to update relevant nodes in network.

File: upgrades.rst

===================================== Upgrades

Introduction

These steps will help you upgrade to the latest version of Netmaker. Note that all instructions here assume you have installed using docker-compose. You may need to modify these steps depending on your setup.

As of v0.20+, the server configuration is stable, meaning you should only need to:

Change the associated docker image tags for the Netmaker server and Netmaker UI
Restart the server and UI using the new image (e.g. "docker-compose up -d")

For migrating from v0.17.1, you can use the migration steps listed below under Upgrade from v0.17.1 to Latest <https://docs.netmaker.io/upgrades.html#upgrade-from-v0-17-1-to-latest>_

For older versions of Netmaker (pre-v0.17.1), you must first manually upgrade to v0.17.1 before running the migration script.

Client Upgrades (General)

As of v0.20.5, the Netclient should automatically upgrade itself when it detects a change in the server version. For prior versions of the netclient (or if this fails), you will need to manually upgrade the client.

For Linux and Freebsd: Download the netclient for your target version, OS, and arch, from the fileserver <https://fileserver.netmaker.io/releases/download>_. Then run "./netclient install" from the downloaded executable in your terminal.

For Mac and Windows: Download and run the installer (.EXE for Windows, .PKG for Mac) for your target version, OS, and arch, from the fileserver <https://fileserver.netmaker.io/releases/download>_. Then, run the installer.

Server Upgrades (v0.20.0+)

Unless otherwise noted, for newer versions of Netmaker:

SSH to the server hosting Netmaker
Open the "netmaker.env" file using a text editor
Change UI_IMAGE_TAG and SERVER_IMAGE_TAG to the latest version
Run "docker-compose up -d"

a. For versions prior to v0.20.5, follow the Client Upgrades <https://docs.netmaker.io/upgrades.html##client-upgrades-general>_ instructions to upgrade your netclients.
b. For v0.20.5 or later, check in the UI to confirm that all hosts have successfully upgraded to the new version.

test line

Upgrade to latest from v0.17.1

These steps assume you have already upgraded your server and netclients to v0.17.1.

General Notes

The server should be upgraded before any clients.
Relays will need to be recreated after the server and all clients are upgraded. Relays are now only available on Pro.
If upgrading to Pro, a new license key and tennet id must be obtained from https://app.netmaker.io
As each netclient is updated, a new host, nodes, and gateways (if applicable) will be visible in the netmaker UI.
Extclient config files may have to be regenerated after the upgrade.

Steps

Download the nm-upgrade.sh script from https://fileserver.netmaker.io/upgrade/nm-upgrade.sh
Make the script executable and run it.
After the upgrade, you should see only one host in the Netmaker UI. It will have the same name as the hostname of your server, rather than netmaker-1.
Upgrade your netclients
do not use packages to upgrade on window/darwin, use the netclient binary to update

a. Linux/Freebsd/Darwin: On each client download the latest version <https://fileserver.netmaker.io/latest>_ of netclient and run the netclient install command
b. Windows: On each client download the latest version <https://fileserver.netmaker.io/latest>_ of netclient-windows-amd64.exe
open Powershell window as Administrator and run the following commands:
net stop netclient
c:\\Users\User\Downloads\netclient-windows-amd64.exe install

As each netclient is updated, check that a new host, nodes, and gateways (if applicable) are visible in the Netmaker UI.
If upgrading to Pro, recreate any relay gateways
Verify extcient config files are correct. Delete and regenerate if incorrect. For each peer in config file:

a. the peer's public key should be the same as the peer's public key in the netmaker UI

b. the peer's endpoint should be the same as the peer's endpoint in the netmaker UI

c. the peer's allowed ips should be the same as the peer's allowed ips in the netmaker UI

Your Netmaker server and clients should all now be running the latest version of Netmaker.

Critical Notes for 0.13.X

If upgrading from 0.12 to 0.13, refer to this gist: https://gist.github.com/afeiszli/f53f34eb4c5654d4e16da2919540d0eb

Critical Notes for 0.10.0

At the time of this writing, an upgrade process has not been defined for 0.10.0. DO NOT follow this documentation to upgrade from a prior version to 0.10.0. An upgrade process will be defined shortly. For now, if you seek to upgrade to 0.10.0, you must clear your server entirely (docker-compose down --volumes), uninstall your netclients, and re-install netmaker + netclients.

Upgrade the Server (prior to 0.10.0)

To upgrade the server, you only need to change the docker image versions:

ssh root@my-server-ip
docker-compose down
vi docker-compose.yml
Change gravitl/netmaker: and gravitl/netmaker-ui: to the new version.
Save and close the file
docker-compose up -d

Upgrade the server after v0.16.1

There have been changes to the MQ after v0.16.1. You will need to make changes to the docker-compose.yml and get the new mosquitto.conf files. We recommend upgrading your server first before any clients.

Start by shutting down your server with docker-compose down

You then need to get the updated mosquitto.conf file. You will also need to get the wait.sh file and make sure it is executable.

.. code-block::

text

wget -O /root/mosquitto.conf https://raw.githubusercontent.com/gravitl/netmaker/master/docker/mosquitto.conf
wget -q -O /root/wait.sh https://raw.githubusercontent.com/gravitl/netmaker/develop/docker/wait.sh
chmod +x wait.sh

Then make the following changes to the docker-compose.yml file.

change image tags in netmaker and netmaker-ui service sections to gravitl/netmaker:v.0.16.1.
In your netmaker service section:
a. In the volumes section, change - shared_certs:/etc/netmaker to - mosquitto_data:/etc/netmaker

b. In the environment section, add MQ_ADMIN_PASSWORD: "<CHOOSE_A_PASSWORD_YOU_WOULD_LIKE_TO_USE>"
In the mq service section:
a. Add command: ["/mosquitto/config/wait.sh"]

b. Add an environment section and add NETMAKER_SERVER_HOST: "https://api.NETMAKER_BASE_DOMAIN"

c. In the volumes, add - /root/wait.sh:/mosquitto/config/wait.sh

d. You need to make some changes to the labels. a few of them just need mqtts to be mqtt. The labels should look like this:

.. code-block::
text
```
- traefik.enable=true
 - traefik.tcp.routers.mqtt.rule=HostSNI(`broker.NETMAKER_BASE_DOMAIN`)
 - traefik.tcp.routers.mqtt.tls.certresolver=http
   - traefik.tcp.services.mqtt.loadbalancer.server.port=8883
   - traefik.tcp.routers.mqtt.entrypoints=websecure
```

Your MQ section should look like this after the changes.

.. code-block:: yaml

text

mq:
container_name: mq
image: eclipse-mosquitto:2.0.11-openssl
depends_on:
  - netmaker
restart: unless-stopped
command: ["/mosquitto/config/wait.sh"]
environment:
  NETMAKER_SERVER_HOST: "https://api.NETMAKER_BASE_DOMAIN"
volumes:
  - /root/mosquitto.conf:/mosquitto/config/mosquitto.conf
  - /root/wait.sh:/mosquitto/config/wait.sh
  - mosquitto_data:/mosquitto/data
  - mosquitto_logs:/mosquitto/log
expose:
  - "8883"
labels:
  - traefik.enable=true
  - traefik.tcp.routers.mqtt.rule=HostSNI(`broker.NETMAKER_BASE_DOMAIN`)
  - traefik.tcp.routers.mqtt.tls.certresolver=http
  - traefik.tcp.services.mqtt.loadbalancer.server.port=8883
  - traefik.tcp.routers.mqtt.entrypoints=websecure

You should be all set to docker-compose up -d

Note: Your clients will show in warning until they are also upgraded. The upgrade for clients is the regular upgrade, then do a netclient pull

Your docker logs mq should be showing logs like this:

.. code-block::

text

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Waiting for netmaker server to startup

Starting MQ...

1665067766: mosquitto version 2.0.11 starting

1665067766: Config loaded from /mosquitto/config/mosquitto.conf.

1665067766: Loading plugin: /usr/lib/mosquitto_dynamic_security.so

1665067766: Opening ipv4 listen socket on port 8883.

1665067766: Opening ipv6 listen socket on port 8883.

1665067766: Opening ipv4 listen socket on port 1883.

1665067766: Opening ipv6 listen socket on port 1883.

1665067766: mosquitto version 2.0.11 running

1665067769: New connection from 172.21.0.2:34004 on port 1883.

1665067769: New client connected from 172.21.0.2:34004 as L0vUDgN0IZFru9VaS6HoRL5 (p2, c1, k60, u'Netmaker-Admin').

1665067769: New connection from 172.21.0.2:34006 on port 1883.

1665067769: New client connected from 172.21.0.2:34006 as ydmOjmIcw9nNaT1GB1q97Se (p2, c1, k60, u'Netmaker-Server').

If you see mq logs about waiting for netmaker server to startup after longer period than usual, check if your traefik certs are generated correctly. You can try to resolve with docker restart traefik

Upgrade the server to use 0.17.0 after Upgrading for 0.16.3

Version 0.17.0 uses Caddy instead of traefik.

Open a Terminal window (shell prompt). To set up Caddy you'll need to configure the Caddyfile as follows.

If you are using the Community Edition of Netmaker use this command:

.. code-block::

text

wget -O /root/Caddyfile "https://raw.githubusercontent.com/gravitl/netmaker/master/docker/Caddyfile"

If you are using the Professional Edition of Netmaker use this command:

.. code-block::

text

wget -O /root/Caddyfile "https://raw.githubusercontent.com/gravitl/netmaker/master/docker/Caddyfile-pro"

Once you have a Caddyfile you'll need to run these two commands:

.. code-block::

sed -i "s/NETMAKER_BASE_DOMAIN/$NETMAKER_BASE_DOMAIN/g" /root/Caddyfile
sed -i "s/YOUR_EMAIL/$EMAIL/g" /root/Caddyfile

Where $NETMAKER_BASE_DOMAIN is the base domain you used for your Netmaker setup (the part after "dashboard." in your Dockerfile) and $YOUR_EMAIL is your email address.

If users still want to keep using Traefik as the reverse-proxy instead of Caddy for v0.17.0 and above, refer to this docker-compose file https://gist.github.com/alphadose/1602e5dcba500f75ab0b873d4441236b

Edit the above docker-compose file

.. code-block::

sed -i 's/NETMAKER_BASE_DOMAIN//g' docker-compose.yml
sed -i 's/SERVER_PUBLIC_IP//g' docker-compose.yml
sed -i 's/REPLACE_MASTER_KEY//g' docker-compose.yml
sed -i "s/REPLACE_MQ_ADMIN_PASSWORD//g" docker-compose.yml

After that finally start the netmaker server

.. code-block::

sudo docker-compose up -d

Upgrade the Clients (prior to 0.10.0)

To upgrade the client, you must get the new client binary and place it in /etc/netclient. Depending on the new vs. old version, there may be minor incompatibilities (discussed below).

Visit https://github.com/gravitl/netmaker/releases/
Find the appropriate binary for your machine.
Download. E.x.: wget https://github.com/gravitl/netmaker/releases/download/vX.X.X/netclient-myversion
Rename binary to netclient and move to folder. E.x.: mv netclient-myversion /etc/netclient/netclient
netclient --version (confirm it's the correct version)
netclient pull

This last step helps ensure any newly added fields are now present. You may run into a "panic" based on missing fields and your version mismatch. In such cases, you can either:

Add the missing field to /etc/netclient/config/netconfig-yournetwork and then run "netclient checkin"

Leave and rejoin the network

File: usage.rst

.. _usage:

================= External Guides

Netmaker has many use cases, from a basic virtual network to an office gateway VPN to a Kubernetes underlay. It can be a bit overwhelming to figure out where to start. If you don't find your use case here, but think Netmaker is a good fit, let us know!

Video Tutorials

Intro/Overview <https://youtu.be/PWLPT320Ybo>_: Tutorial on first-time usage, setting up a mesh network.
Site-to-Site Gateway <https://youtu.be/krCKBJhwwDk>_: Tutorial on setting up site-to-site connections, allowing peers to access external networks via gateways.
IPv6 and Private DNS <https://youtu.be/b4diaKWUcXI>_: Tutorial on dual-stack IPv6 in Netmaker and Private DNS management (separate topics).
Kubernetes Networking <https://youtu.be/z2jvlFVU3dw>_: Tutorial on setting up cross-cloud Kubernetes clusters using Netmaker.

Written Tutorials

K3s Cross-cloud cluster <https://itnext.io/how-to-deploy-a-single-kubernetes-cluster-across-multiple-clouds-using-k3s-and-wireguard-a5ae176a6e81>_: Tutorial on setting up cross-cloud K3s clusters using Netmaker.
MicroK8s Cross-cloud cluster <https://itnext.io/how-to-deploy-a-cross-cloud-kubernetes-cluster-with-built-in-disaster-recovery-bbce27fcc9d7>_: Tutorial on setting up cross-cloud MicroK8s clusters using Netmaker.
Secure access to private services <https://afeiszli.medium.com/how-to-enable-secure-access-to-your-hosted-services-using-netmaker-and-wireguard-1b3282d4b7aa>_: Tutorial on setting up secure Nextcloud with Netmaker.

File: user-management.rst

.. _user-manage:

====================== User Management

Superadmin Signup

.. include:: ui-reference.rst
:start-after: Signup
:end-before: (1

Input your username and a super memorable but strong password then click on the Sign up button. Once you've signed up, you can login to your Netmaker server with the account.

.. include:: ui-reference.rst
:start-after: Login
:end-before: (1

.. include:: oauth.rst
:end-before: Integrating OAuth

.. include:: pro/pro-users.rst
:end-before: Transferring super admin rights

.. include:: pro/rac.rst
:start-after: .. include_after_this_label

File: examplecode/dev.yaml

server:
apihost: "" # defaults to 127.0.0.1 or remote ip (SERVER_HOST) if DisableRemoteIPCheck is not set to true. SERVER_API_HOST if set
apiport: "" # defaults to 8081 or HTTP_PORT (if set)
masterkey: "" # defaults to 'secretkey' or MASTER_KEY (if set)
allowedorigin: "" # defaults to '*' or CORS_ALLOWED_ORIGIN (if set)
restbackend: "" # defaults to "on" or REST_BACKEND (if set)
clientmode: "" # defaults to "on" or CLIENT_MODE (if set)
dnsmode: "" # defaults to "on" or DNS_MODE (if set)
sqlconn: "" # defaults to "http://" or SQL_CONN (if set)
disableremoteipcheck: "" # defaults to "false" or DISABLE_REMOTE_IP_CHECK (if set)
version: "" # version of server
rce: "" # defaults to "off"
mqhost: "" # defaults to "mq"
nodeid: "" # defaults to macaddress of machine
messagequeuebackend: "" # default to "on"
database: "" # defaults to "sqlite"
verbosity: "" # defaults to 0
authprovider: "" # defaults to ""
displaykeys: "" # defaults to "on"
manageiptables: "" # defaults to "on"
portforwardservices: "" # defaults to "", options include "dns" and "ssh"
hostnetwork: "" # defaults to "off"
mqport: "" # defaults to 8883
mqserverport: "" # defaults to 1883
server: "" # defaults to "", should be broker domain

File: examplecode/docker-compose.v0.20.3.yml

version: "3.4"

services:

netmaker:
container_name: netmaker
image: gravitl/netmaker:$SERVER_IMAGE_TAG
env_file: ./netmaker.env
restart: always
volumes:
- dnsconfig:/root/config/dnsconfig
- sqldata:/root/data
environment:
# config-dependant vars
- STUN_LIST=stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302
# The domain/host IP indicating the mq broker address
- BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}
# The base domain of netmaker
- SERVER_NAME=${NM_DOMAIN}
- SERVER_API_CONN_STRING=api.${NM_DOMAIN}:443
# Address of the CoreDNS server. Defaults to SERVER_HOST
- COREDNS_ADDR=${SERVER_HOST}
# Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks.
- SERVER_HTTP_HOST=api.${NM_DOMAIN}

netmaker-ui:
container_name: netmaker-ui
image: gravitl/netmaker-ui:$UI_IMAGE_TAG
env_file: ./netmaker.env
environment:
# config-dependant vars
# URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT
BACKEND_URL: "https://api.${NM_DOMAIN}"
depends_on:
- netmaker
links:
- "netmaker:api"
restart: always

caddy:
image: caddy:2.6.2
container_name: caddy
env_file: ./netmaker.env
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- ./certs:/root/certs
- caddy_data:/data
- caddy_conf:/config
ports:
- "80:80"
- "443:443"

coredns:
container_name: coredns
image: coredns/coredns
command: -conf /root/dnsconfig/Corefile
env_file: ./netmaker.env
depends_on:
- netmaker
restart: always
volumes:
- dnsconfig:/root/dnsconfig
mq:
container_name: mq
image: eclipse-mosquitto:2.0.15-openssl
env_file: ./netmaker.env
depends_on:
- netmaker
restart: unless-stopped
command: [ "/mosquitto/config/wait.sh" ]
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf
- ./wait.sh:/mosquitto/config/wait.sh
- mosquitto_logs:/mosquitto/log
- mosquitto_data:/mosquitto/data

volumes:
caddy_data: { } # runtime data for caddy
caddy_conf: { } # configuration file for Caddy
sqldata: { }
dnsconfig: { } # storage for coredns
mosquitto_logs: { } # storage for mqtt logs
mosquitto_data: { } # storage for mqtt data

File: examplecode/netclient-help.txt

NAME:
Netclient CLI - Netmaker's netclient agent and CLI. Used to perform interactions with Netmaker server and set local WireGuard config.

USAGE:
netclient [command]

COMMANDS:
completion Generate the autocompletion script for the specified shell
connect connect to a netmaker network
daemon netclient daemon
disconnect disconnet from a network
help Help about any command
install install netclient binary and daemon
join join a network
leave leave a network
list display list of netmaker networks
pull get the latest host configuration
push push host config to server
register register to a Netmaker instance
server server commands [list, switch, leave]
uninstall uninstall netclient
use use a specific version of netclient
version Displays version information

Flags:
-h, --help help for netclient
-v, --verbosity int set logging verbosity 0-4

Use "netclient [command] --help" for more information about a command.

File: examplecode/netclient-join.txt

join a netmaker network using:

token: netclient join -t // join using token
server: netclient join -s // join a specific server via SSO if Oauth configured
net: netclient join -s -n // attempt to join specified network via auth
all-networks: netclient join -s -A // attempt to register to all allowed networks on given server via auth
user: netclient join -s -u // attempt to join/register via basic auth

NAME:
netclient join - Join a Netmaker network.

USAGE:
netclient join [flags]

OPTIONS:
-A, --all-networks attempts to join/register to all available networks to user
-e, --endpoint-ip string sets endpoint on host
-h, --help help for join
-m, --mtu string sets MTU on host
-o, --name string sets host name
-n, --net string network to attempt to join/register to
-p, --port int sets wg listen port
-s, --server string server for attempting SSO/Auth registration
-i, --static flag to set host as static
-t, --token string enrollment token for joining/registering
-u, --user string user name for attempting Basic Auth join/registration

GLOBAL OPTIONS:
-v, --verbosity int set logging verbosity 0-4

File: examplecode/netclient-push.txt

push netclient host configuration using:

NAME:
netclient push - Pushes custom host configuration.

USAGE:
netclient push [flags]

OPTIONS:
-e, --endpoint-ip string sets endpoint on host
-h, --help help for push
-m, --mtu int sets MTU on host
-o, --name string sets host name
-p, --port int sets wg listen port
-i, --static flag to set host as static

GLOBAL OPTIONS:
-v, --verbosity int set logging verbosity 0-4

File: pro/index.rst

===================== Netmaker Professional

.. toctree::
:maxdepth: 1

pro-setup
pro-users
../oauth
pro-relay-server
pro-metrics
pro-branding
rac
pro-failovers
internet-gateways

.. image:: ../images/netmaker.png
:width: 100%
:alt: Netmaker WireGuard
:align: center

Netmaker Professional is our advanced Netmaker offering for business use cases. It offers all the features of community edition plus:

Metrics: Nodes collect networking metrics such as latency, transfer, and connectivity status. These are displayed in the Netmaker UI, and also exported to Grafana via Prometheus.
Users: On community you can only create admin users, where as on PRO it gives ability to create non-admin users which you can pair with remote-access gateway to segment users on different networks.
OAuth: By integrating with an OAuth provider, Netmaker users can log in via the provider, rather than the default simple auth.
Remote Access Client: Netmaker Professional comes with a remote access client that allows you to connect to your network from anywhere. This is a great way to connect to your network from a laptop or mobile device (soon).
FailOvers: FailOvers are made to help two peers communicate where they cannot talk directly due to their firewall restrictions, in which case their connection falls back through a failover node set by the user in the network.
Relays: All traffic routing to and from in a network for a relayed machine will go through the relay machine.
Internet Gateways: These work similar to traditional VPNs, and can work with netclients (hosts in the mesh network) as well as with remote devices connected to the network via client configs.

File: pro/internet-gateways.rst

=================================== Internet Gateways

Internet gateways is a Pro-only feature that allows hosts in a Netmaker mesh network to connect to the internet via a gateway. This brings the possibility of a traditional VPN to machines with netclient installed on them. Internet gateways can be configured from the Internet gateway tab.

.. image:: images/inet-gw-feature.png
:alt: internet gateway feature
:align: center
:width: 50%

Only Linux hosts can be setup as internet gateways. Both Windows and Linux hosts can be connected to an internet gateway however. Remote clients can also be connected to internet gateways via a client config.

This feature was introduced in v0.23.0 and is in beta (for this release).

Any relay behind an internet gateway will have its traffic, as well as traffic of its relayed hosts going through the internet gateway.
This behaviour also happens with remote access gateways and their connected clients; a remote access gateway behind an internet gateway would have its traffic, as well as traffic of its clients, passing through the internet gateway.

For Community Edition (CE) Users, clients can still access internet gateways via the Remote Access tab. View here for more info <../external-clients.html#internet-gateway-traditional-vpn>_.

Usage

Create an internet gateway

.. image:: images/inet-gw-create.png
:alt: internet gateway creation
:align: center
:width: 50%

Under a network, switch to the "Internet Gateways" tab
Click on Create Gateway
Select the desired Linux host to serve as the internet gateway (VPN server, in traditional parlance)
Optionally choose the hosts you want to connect to the internet gateway
Click on create

NOTE: A host can only be connected to at most one internet gateway, regardless of the networks the host is in.

NOTE: A host already connected to an internet gateway cannot be a gateway itself. ie: internet gateway chaining is not supported currently.

Update an internet gateway

.. image:: images/inet-gw-update.png
:alt: internet gateway update
:align: center
:width: 50%

Select an internet gateway from the gateways table
Click on the update gateway button (from the table row actions dropdown or "Update connected hosts" button on top of the associated "Connected Hosts" table)
Update the gateway and save changes

Delete an internet gateway

.. image:: images/inet-gw-delete.png
:alt: internet gateway deletion
:align: center
:width: 50%

Click on the delete button from the table row actions dropdown
Confirm gateway deletion. This does not entirely remove the host from the network; it only stops the host from serving as an internet gateway.

FAQs and Known Issues

Q: Why not Remote Access Gateway as an internet gateway.

A: Although possible, that serves a similar but different use case. Internet gateways allow not only remote clients, but also and peculiarly, hosts in the Netmaker mesh network to also connect to the internet via a gateway.
This brings the possiblility of a traditional VPN to machines with netclient installed on them. This feature can be accessed from the Internet gateway tab.

File: pro/pro-branding.rst

============ UI Branding

Modifying the UI with whitelabeling.

Netmaker UI allows resellers to whitelabel and customize branding by building a custom docker image with the below environment variables set.

Open the Dockerfile.standalone file and set the following environment variables (recommended to use logos with an aspect ratio of 4:3):

(1) VITE_PRODUCT_NAME: The name of the product. This is the name that will appear in the UI.
(2) VITE_TENANT_LOGO_DARK_URL: Logo to be used in dark mode.
(3) VITE_TENANT_LOGO_LIGHT_URL: Logo to be used in light mode.
(4) VITE_TENANT_LOGO_DARK_SMALL_URL: Small varient of logo to be used in dark mode. eg: when sidenav is collapsed (optional).
(5) VITE_TENANT_LOGO_LIGHT_SMALL_URL: Small varient of logo to be used in light mode. eg: when sidenav is collapsed (optional).
(6) VITE_TENANT_LOGO_ALT_TEXT: Alternative text for logo.
(7) VITE_TENANT_FAVICON_LOGO: Favicon to use in the web browser's tile bar or tab. Defaults to light logo if not specified. Recommended to use a 32x32px .ico image
(8) VITE_TENANT_PRIMARY_COLOR: UI primary color. Replace this with your brand color. eg: red, green, "#F00", "#00FF00" (Hex values need quoting)

You could use a URL to the respective logos or put the logos in the /public directory, then build the docker image.

Reference: https://github.com/gravitl/netmaker-ui-2/blob/master/Dockerfile.standalone

For more information on how to go about whitelabelling, reach out to us at https://www.netmaker.io/contact

File: pro/pro-failovers.rst

===================================== FailOver Servers

Introduction

Sometimes nodes are in hard-to-reach places. Typically this will be due to a CGNAT, Double NAT, or a restrictive firewall. In such scenarios, a direct peer-to-peer connection with all other nodes might be impossible.

For this reason, Netmaker has a FailOver functionality. At any time you may designate a publicly reachable node (such as the Netmaker Server) as a FailOver, and netmaker will identify if any peers in the network are not able to
communicate directly then their connection will automatically fallback through the failOver node in the network

Configuring a FailOver node

To create a failover, you can use any host in your network, but it should have a public IP address (not behind a NAT).

Navigate to the network tab and click on the hosts tab

.. image:: images/failOvertab.jpg
:width: 80%
:alt: FailOver
:align: center

You can choose any public node in the network to be a failover, Once chosen any peers facing difficulties to communicate directly with each other, will have connection go over the failover node.

To reset the network connections, you can remove the failover node, and network will go back to original state from if any connections are getting failed over.

.. image:: images/failOverselect.jpg
:width: 80%
:alt: FailOver
:align: center

File: pro/pro-metrics.rst

================================= Netmaker Professional Metrics

Netmaker Pro offers metrics. With Metrics, admin users can view connectivity,
latency and data transferred between two peers or nodes on a Netmaker network. Client metrics are also available.
All of this data may be visualised in the Netmaker UI. In addition, Netmaker
has included a custom exporter for Prometheus/Grafana integration to view
the data as well.

Below are the steps in order to view metrics on your Netmaker Pro instance.

NOTE: Metrics currently relies on ICMP to be allowed between nodes.

Network metrics can be viewed under the Metrics tab within a network.

Viewing Details

To view the metrics in the Netmaker Dashboard, select a network, then click the Metrics tab.
From here, you can switch to any metric you are interested in, including metrics from clients.

NOTE: Metrics may take up to 5 minutes for nodes to report data.

.. image:: images/metrics/network-metrics.png
:width: 80%
:alt: Network Metrics
:align: center

Grafana Dashboard

If your Netmaker instance includes the Prometheus/Grafana setup and is configured with the METRICS_EXPORTER="on", you
can also view your metrics via Grafana. Simply navigate to the dashboards section of your Grafana instance.
.. code-block::

text

URL: "https://grafana.<YOUR_DOMAIN_NAME>"
Username: "admin"
Password: "admin"

There, you will be presented with two out-of-the-box Netmaker options of the following:

Netmaker Metrics Dashboard
Netmaker Network Graph

.. image:: images/metrics/metrics-grafana1.png
:width: 80%
:alt: Netmaker Grafana Dashboards
:align: center

First try viewing the Netmaker Metrics Dashboard.
This view will give you the option to select and view data on individual nodes in your Netmaker network.

.. image:: images/metrics/metrics-grafana3.png
:width: 80%
:alt: Netmaker Grafana View 1
:align: center

Then try out the Netmaker Network Graph view.
This graph gives a Network graph, in which the user may hover individual nodes to see statistics of that node.
Or hover an edge to view information about the connection between two nodes. The node edges will vary depending on connection status (green for connected or red for disconnected).

.. image:: images/metrics/metrics-grafana2.png
:width: 80%
:alt: Netmaker Grafana View 2
:align: center

Also you can view your metrics on prometheus dashboard, for the first time you will be prompted for credentials in your brower when you visit your prometheus dashboard.
.. code-block::

text

URL: "https://prometheus.<YOUR_DOMAIN_NAME>"
Username: "Netmaker-Prometheus"
Password: "<YOUR_LICENSE_KEY>"

File: pro/pro-relay-server.rst

===================================== Relay Servers

Introduction

.. image:: images/relay1.png
:width: 80%
:alt: Relay
:align: center

For this reason, Netmaker has a Relay Server functionality. At any time you may designate a publicly reachable node (such as the Netmaker Server) as a Relay, and tell it which machines it should relay. Then, all traffic routing to and from that machine will go through the relay. This allows you to circumvent the above issues and ensure connectivity when direct measures do not work.

Configuring a Relay

To create a relay, you can use any host in your network, but it should have a public IP address (not behind a NAT).

Navigate to the network name and click on the relay tab

.. image:: images/relaystatusbutton.png
:width: 80%
:alt: Relay
:align: center

When clicking on the create button, you should be prompted to select a host to use as the relaying server.

.. image:: images/selectrelaying.png
:width: 80%
:alt: Relay
:align: center

After that, select the server you would like to be relayed. Choose from the dropdown menu.

.. image:: images/selectrelayed.png
:width: 80%
:alt: Relay
:align: center

After creation, you should see your relayed host show up on a list of relayed hosts on the page.

.. image:: images/relaystatus.png
:width: 80%
:alt: Relay
:align: center

Auto Relays

When incorporating a new host into a network, you have the option to set it up as a relayed node.
To accomplish this, the initial step involves generating an enrollment key and specifying both the network and the relay to which the host will be added.

.. image:: images/pro-relay-enrollment-key.png
:width: 80%
:alt: Relay enrollment key
:align: center

When integrating a new host into a network, choose the recently generated enrollment key and proceed with the remaining setup steps.
After completing this process, you should observe your new host being successfully relayed.

.. image:: images/pro-relay-host-setup.png
:width: 80%
:alt:

File: pro/pro-setup.rst

=========================== Netmaker Professional Setup

This guide covers how to get setup with Netmaker Professional. For pricing, visit our pricing page <https://www.netmaker.io/pricing>_

Get a License

Signup here <https://account.netmaker.io/signup>_ to get a license for your self-hosted Netmaker. Never miss to input your Firstname and Lastname.

.. image:: images/profile-form-after-signup---regular-email.jpg
:width: 80%
:alt: Profile form after signup on SAAS with regular email
:align: center

.. image:: images/profile-form-after-signup---sso.jpg
:width: 80%
:alt: Profile form after signup on SAAS with SSO
:align: center

When you login to the site for the first time, normally you will be automatically directed to the tenant creation modal as shown in the first image below. But if, for some reason, you happen to have missed inputting your Firstname and Lastname after signing up, the Profile page will show up instead. Click on the [>] icon to expand the Profile form where you can input your profile info. This is required when creating tenants.

.. image:: images/pro-aftersignup.jpg
:width: 80%
:alt: screen after first login on SAAS
:align: center

.. image:: images/missed-profile-info.jpg
:width: 80%
:alt: screen after first login on SAAS - missed profile info
:align: center

For first time users Netmaker offers a free 14-day trial of the Professional plan of either a SaaS tenant or an On-Prem tenant. Click on the "Create an On-Prem tenant" option. Input the name of your new tenant then click on the Next button.

.. image:: images/tenant-name-form.jpg
:width: 80%
:alt: Tenant name form
:align: center

You can toggle between a Monthly or a Yearly subscription plan for when your trial period ends. Click on the "Tier breakdown" button to view the details of the plan. Once you're done, click on the "Add Payment Details in Stripe" button.

.. image:: images/pricing-and-limits.jpg
:width: 80%
:alt: Pricing and Limits
:align: center

Like many cloud service providers, payment information is required in order to create resources even during trial period. This can be in a form of Debit/Credit card information or a US Bank account. Fillout the form and then click on the "Add" button.

.. image:: images/payment-method-form.jpg
:width: 80%
:alt: Payment Method Form
:align: center

You should automatically be redirected back to the tenant creation modal. Review your new tenant details then click on the Create Tenant button.

.. image:: images/review-tenant-details-form.jpg
:width: 80%
:alt: Review tenant details
:align: center

Once created, your new tenant should show up in the Tenants page. Click on the Manage Account button to view its details.

.. image:: images/tenants-list-page.jpg
:width: 80%
:alt: Tenants page
:align: center

Setup your server

You will need the Licence Key and the Tenant ID which are found on the Settings tab.

.. image:: images/pro-license-key2.jpg
:width: 80%
:alt: License keys
:align: center

Once you have your license key and tenant ID, you can get the nm-quick installer and run it.

.. code-block::

text

wget https://raw.githubusercontent.com/gravitl/netmaker/master/scripts/nm-quick.sh
chmod +x nm-quick.sh
bash nm-quick.sh -u

Follow the prompts for a pro edition server. provide the License Key and Tenant ID when prompted.

Upgrade to Pro from Community Edition

You can upgrade from an existing community server to a pro server with this script. Follow the prompts to setup a pro server and the script will make the necessary changes to your netmaker.env file and grab the pro docker-compose.override.yml file.

If you prefer to upgrade manually, go to your netmaker server and add the following to your netmaker.env file:

.. code-block::

text

LICENSE_KEY=<license key>
NETMAKER_TENANT_ID=<tenant id>

Also change SERVER_IMAGE_TAG in netmaker.env to <version>-ee. For example: SERVER_IMAGE_TAG=v0.21.2-ee

Also change the INSTALL_TYPE from ce to pro.

Then you will need to get the docker-compose pro file from here

.. code-block::

text

wget -O /root/docker-compose.override.yml https://raw.githubusercontent.com/gravitl/netmaker/master/compose/docker-compose.pro.yml

No changes will need to be made to that file. It will use the configs listed in your netmaker.env file.

After that run the following command:

.. code-block::

text

docker kill netmaker netmaker-ui && docker compose up -d

When you browse to your self-hosted Netmaker via dashboard., you should see the professional UI and a new Dashboard. The top menu bar will have relays and metrics added.

.. image:: images/pro-new-dashboard.jpg
:width: 80%
:alt: new dashboard
:align: center

(Optional) Setup your server for Prometheus and Grafana

If you would like to use Netmaker's custom Prometheus exporter and Grafana dashboard, your docker-compose.override.yml file will already have those section.

In Netmaker.env, change the following:

.. code-block::

text

METRICS_EXPORTER=on
PROMETHEUS=on

File: pro/pro-users.rst

================================= Users in Netmaker Professional

Netmaker Professional offers advanced user management features. The super admin can create users with either a user or admin role. Only Admins can access the dashboard, the normal users can use remote access client to join the network through a gateway.
Admins can add users and assign them to remote access gateways, which includes managing the user's access to different remote access gateways.

Here is a breakdown of the different user types and their permissions:

User: Users do not have access to the Netmaker dashboard. They can only use the remote access client to connect to a gateway.
Admin: They can create and manage users, networks, and gateways. They cannot create/manage other admins
Super Admin: Super admins have full access to Netmaker. They can create and manage users, admins, networks, and gateways. They can also manage user and admin permissions.

Adding users

To add a user, go to the Users section and click the Add User button. Fill in the user's details, including their name, password, and role.

As a super admin, you can add users with the role of admin or user.
As an admin, you can only add users with the role of user.

.. image:: /pro/images/users/add-user.png
:width: 80%
:alt: Add User
:align: center

The credentials will need to be shared with the added user.

Attaching or removing user from a remote access gateway

To attach users to a remote access gateway or remove users from a gateway, you will need to have the gateway set up.
Once the remote access gateway is set up, you will see an option to attach or remove users from the gateway's dropdown menu on the table row.

.. image:: /pro/images/users/gateway-dropdown.png
:width: 80%
:alt: Add Remove User dropdown
:align: center

You can click the button to either attach or remove a user

.. image:: /pro/images/users/attach-remove-users.png
:width: 80%
:alt: Attach Remove User Modal
:align: center

Transferring super admin rights

Super admin rights can be transferred only to another admin. To do this, on the users page, go to the superadmin row and hover over the ellipsis.
You will see an option to transfer admin rights. On clicking it, a dialog box will open allowing you to select any admin
to transfer super admin rights to.

.. image:: /pro/images/users/transfer-super-admin-rights.png
:width: 80%
:alt: transfer super admin rights
:align: center

Integrating OAuth

File: pro/rac.rst

=================================== Netmaker Remote Access Client (RAC)

The Netmaker Remote Access Client, RAC for short is a GUI tool for easily getting access to a Netmaker network.
RAC is mostly suited for offsite machines that need access to a Netmaker network. It supports Windows, Mac, Linux and mobile (iOS and Android).

Download/Installation

For Mac:

You can download the M1 or Intel Mac installer from our fileserver link (https://fileserver.netmaker.io/releases/download/latest) run it. Then open the GUI to start using RAC!

For Windows:

You can download the remoteclientbundle.exe bundle (recommended as this install wireguard and other dependencies) or remote-access-client_86.msi installer (MSI doesnt install any dependencies) and run it to install on your windows machine. You can then open the GUI to start using RAC!
Download link: https://fileserver.netmaker.io/releases/download/latest/remoteclientbundle.exe

For mobile devices, you can download the app from the respective app stores.

Google Play Store (Android):

Scan the QR code

.. image:: /pro/images/netmaker-rac-android.png
:width: 400px
:alt: Netmaker RAC on Google Play Store
:align: left

Or use the following link:
https://play.google.com/store/apps/details?id=com.net.netmaker&pli=1&utm_source=docs

Apple App Store (iOS):

Scan the QR code

.. image:: /pro/images/netmaker-rac-apple.png
:width: 400px
:alt: Netmaker RAC on Apple App Store
:align: left

Or use the following link:
https://apps.apple.com/us/app/netmaker-rac/id6479694220?itsct=apps_box_badge&itscg=30200

Debian/Ubuntu Linux:

You can also use the following command to download the latest version:

.. code-block::

curl -sL 'https://apt.netmaker.org/remote-client/gpg.key' | sudo tee /etc/apt/trusted.gpg.d/remote-client.asc
curl -sL 'https://apt.netmaker.org/remote-client/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/remote-client.list
sudo apt update
sudo apt search remote-client # to see available versions
sudo apt install remote-client

Red Hat Distros (Fedora/RedHat/CentOSRocky):

.. code-block::

curl -sL 'https://rpm.netmaker.org/remote-client/gpg.key' | sudo tee /tmp/gpg.key
curl -sL 'https://rpm.netmaker.org/remote-client/remote-client-repo' | sudo tee /etc/yum.repos.d/remote-client.repo
sudo rpm --import /tmp/gpg.key
sudo dnf check-update
sudo dnf install remote-client

Following the above instructions, you can run RAC from your Linux desktop environment launcher or from the command line using the remote-client command.

Not finding your OS? You can download the latest version of RAC from the our file server:
https://fileserver.netmaker.io/releases/download/latest

Search for remote client and download the appropriate version for your operating system.

Quick Start

NOTE: OpenGL is a requirement for Netmaker RAC to run. If you are running RAC on a virtual machine (especially with Windows as guest OS), you may need to enable 3D acceleration in your virtual machine settings.

To use RAC, you will need to have a Netmaker server running and have a user account on that server. You will also need to have a remote access gateway set up on the server. Client devices connect to the network through the remote access gateway (ingress host).

Check :ref:this section on how to create a non-admin user.
RAC is best suited for non-admin users who want to gain remote access to the network, this also provides admins fine-grained control over users in the network by attaching/removing them from a remote access gateway. Admins can also use RAC to gain remote access to the network with a different machine.

.. include_after_this_label

Using the Remote Access Client (RAC)

Once a user has been attached to a remote access gateway, they can connect to a network using the remote access client. To do this, they will first need to log in using the credentials that were provided to them. Social login is also supported.

.. image:: /pro/images/users/remote-access-client.png
:width: 80%
:alt: Remote access login
:align: center

After successful login you will be shown all the networks and gateways you have given access to, so now you will be able to connect/disconnect/refresh your connection to a gateway. Internet gateways are depicted with a globe icon.
An internet gateway can be used to route all your traffic through the gateway, this is useful if you want to access the internet without exposing your public IP address. This behaves like a traditional VPN.

.. image:: /pro/images/users/remote-access-client-2.png
:width: 80%
:alt: Connect and disconnect
:align: center

The remote access client also has the following options:

Refresh connection: This basically disconnects the current connection to the remote access gateway and then reconnects to it.
Reload clients: This reloads the client data on the page, which can be useful if the data has changed since the page was last loaded.

.. image:: /pro/images/users/remote-access-client-3.png
:width: 80%
:alt: Reload clients
:align: center

Reset: This resets all connections to remote access gateways across all Netmaker servers and networks known to the client. This can be useful if you end up with a bad wireguard or network interface configuration or are having trouble connecting to a gateway. you should only use this option if Refresh connection does not work.

Using Netmaker like a traditional VPN

Some remote access gateways, specifically internet gateways (depicted by globe icon) can route all your traffic through the them. This can be useful if you want to access the internet without exposing your public IP address. This behaves like a traditional VPN. Internet gateways is a Pro-only feature.

Controlling RAC user sessions

On pro servers/tenants, the duration of a non-admin user's remote session can be controlled.
This can be done by setting RAC_AUTO_DISABLE (to true) and JWT_VALIDITY_DURATION (to an integer in seconds) environment variables on the server.

With RAC_AUTO_DISABLE set to true, a non-admin user's remote sessions will be disabled after the duration specified in JWT_VALIDITY_DURATION has elapsed.
The user will have to relogin to enable their remote session again.

NOTE: The JWT_VALIDITY_DURATION environment variable also configures all the JWT token validity duration for all users, regardless of whether RAC_AUTO_DISABLE is set to true or not.

(Re)Starting the service/daemon process

On very few occasion, the RAC daemon may not be running and will need to be restarted manually. There are two ways to resolve this:

Restart the computer. The daemon starts automatically on boot so restarting the computer will start the daemon on next startup.
Manual restart. The exact procedure depends on your operating system.

Manual daemon restart on Windows

Open Task Manager.
Go to the "Services" tab.
Look for the "remote-client" service.
Right-click on the service and select "Restart" or "Start".

Manual daemon restart on Linux

On Linux, RAC daemon relies on systemd to manage the service. You can restart the service using the following command:

.. code-block::

sudo systemctl restart remote-client

Manual daemon restart on Mac

On Mac, RAC daemon relies on launchd to manage the service. You can restart the service using the following command:

.. code-block::

sudo launchctl stop com.netmaker.remote-client
sudo launchctl start com.netmaker.remote-client

FAQs and Known Issues

Q: I am getting an error when trying to connect to a gateway.

A: Make sure that the gateway is running healthily and that you have access to it. Also try to "Refresh" and see if that fixes the issue. Otherwise "Reset" all connections and try again.

Q: Other WireGuard-based VPNs interfere with Netmaker RAC.

A: This is a known issue. If you have other WireGuard-based VPNs running on your machine, they may interfere with Netmaker RAC. You can try to disable them and see if that fixes the issue. Pro-tip: Netmaker Pro offers internet gateway functionality, so you can use it just as a traditional VPN. For more information, explore the Remote Access gateway feature <https://www.netmaker.io/features/remote-access-gateway>_.

File: .github/workflows/docsdeploy.yml

creates a release from master

builds docs, pushes to docker hub and updates docs server

name: Docs_deploy

on:
push:
branches:
- master
workflow_dispatch:

jobs:

release:
uses: gravitl/devops/.github/workflows/deploydocs.yml@master
secrets: inherit

File: .github/workflows/prcheck.yml

name: "Pull Request Docs Check"
on:

pull_request

jobs:
docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v1
- name: Build
uses: ammaraskar/sphinx-action@master
with:
docs-folder: "."
- name: Generate Name
id: name
run: |
echo "ARTIFACT_NAME=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
name: HTMLupload-${{ steps.name.outputs.ARTIFACT_NAME }}-${{ github.run_attempt }}
path: ./_build/html/

Docker & Self-Hosted Deployment Verified OCI

1. Project Overview & Quickstart (gravitl/netmaker) Copy Chapter

File: docs/APIUsage.md Copy Chapter

File: docs/Authentication.md Copy Chapter

File: docs/Pricing.md Copy Chapter

File: k8s/server/README.md Copy Chapter

Netmaker K8S YAML Templates - Run Netmaker on Kubernetes

0. Prerequisites Copy Section

1. Create Namespace Copy Section

2. Deploy Database Copy Section

3. Deploy MQTT Copy Section

4. Deploy Netmaker Server Copy Section

5. Deploy Netmaker UI Copy Section

Troubleshooting Copy Section

2. Official Technical Reference & Guides (gravitl/netmaker-docs) Copy Chapter

File: README.md Copy Chapter

Welcome to the Netmaker documentation!

How to build the Netmaker documentation

How to contribute to the Netmaker documentation

Submitting Changes Copy Chapter

File: about.rst Copy Chapter

========== About

What is Netmaker?

How Does Netmaker Work?

Use Cases for Netmaker

Architecture

File: acls.rst Copy Chapter

====== ACLs

Introduction

Configuring ACLs

ACL Screen Copy Chapter

Node ACL Screen Copy Chapter

Default Node-Level ACLs

Scenario 1: Remote Access to/from Client devices Copy Chapter

Scenario 2: Priviledged Access to Device Copy Chapter

File: advanced-client-install.rst Copy Chapter

================================ Advanced Client Installation

Introduction to Netclient

Note on MTU Settings

Notes on Windows

Running netclient commands Copy Chapter

High CPU Utilization Copy Chapter

Changing network profile to private Copy Chapter

Issue after Windows sleep/hibernate Copy Chapter

Irregular netclient restart on Windows 2016 server Copy Chapter

Event id 0 in Windows Event logs Copy Chapter

Modes and System Compatibility

CLI Copy Chapter

Daemon Copy Chapter

Private DNS Management Copy Chapter

Prerequisites

Configuration

CLI Reference Copy Chapter

Installation

Managing Netclient

Connect / Disconnect Copy Chapter

Viewing Logs Copy Chapter

Re-syncing netclient (basic troubleshooting) Copy Chapter

Adding/Removing Networks Copy Chapter

Uninstalling Copy Chapter

File: api.rst Copy Chapter

API Usage

Authentication

Format of Calls for ``curl``

API Documentation

File: architecture.rst Copy Chapter

Architecture

Core Concepts

WireGuard Copy Chapter

Mesh Network Copy Chapter

Netmaker Copy Chapter

Node Copy Chapter

Components

Netmaker Server Copy Chapter

Message Broker (Mosquitto) Copy Chapter

Netclient Copy Chapter

Database (SQLite, rqlite, postgres) Copy Chapter

Netmaker UI Copy Chapter

CoreDNS Copy Chapter

Caddy Copy Chapter

1. Project Overview & Quickstart (gravitl/netmaker)

File: docs/APIUsage.md

File: docs/Authentication.md

File: docs/Pricing.md

File: k8s/server/README.md

0. Prerequisites

1. Create Namespace

2. Deploy Database

3. Deploy MQTT

4. Deploy Netmaker Server

5. Deploy Netmaker UI

Troubleshooting

2. Official Technical Reference & Guides (gravitl/netmaker-docs)

File: README.md

Submitting Changes

File: about.rst

File: acls.rst

ACL Screen

Node ACL Screen

Scenario 1: Remote Access to/from Client devices

Scenario 2: Priviledged Access to Device

File: advanced-client-install.rst

Running netclient commands

High CPU Utilization

Changing network profile to private

Issue after Windows sleep/hibernate

Irregular netclient restart on Windows 2016 server

Event id 0 in Windows Event logs

CLI

Daemon

Private DNS Management

CLI Reference

Connect / Disconnect

Viewing Logs

Re-syncing netclient (basic troubleshooting)

Adding/Removing Networks

Uninstalling

File: api.rst

File: architecture.rst

WireGuard

Mesh Network

Netmaker

Node

Netmaker Server

Message Broker (Mosquitto)

Netclient

Database (SQLite, rqlite, postgres)

Netmaker UI

CoreDNS

Caddy

Remote Access Clients

File: conduct.rst

File: egress-gateway.rst

Configuring Egressing External Clients

File: external-clients.rst

File: features.rst

File: getting-started.rst

Network Settings Description

File: how-to-guides.rst

File: index.rst

File: install.rst

Server

Domain

Firewall Rules for Netmaker Server

Firewall Rules for Machines Running Netclient

File: integrating-non-native-devices.rst

File: license.rst

File: manual-install.rst