kubernetes-ingress Documentation

File: README.md Copy Chapter

NGINX Ingress Controller

This repo provides an implementation of an Ingress Controller for NGINX and NGINX Plus from the people behind NGINX.

Join The Next Community Call Copy Chapter

We value community input and would love to see you at the next community call. At these calls, we discuss PRs by community members as well as issues, discussions and feature requests.

Zoom: NGINX Ingress Controller - Community Call

Meeting ID: 985 4405 5687

Passcode: 982193

When: 16:00 GMT / Convert to your timezone, every other Monday.

You can also join the NGINX Community Forum to chat about the NGINX Ingress Controller.

NGINX Ingress Controller works with both NGINX and NGINX Plus and supports the standard Ingress features - content-based
routing and TLS/SSL termination.

Additionally, several NGINX and NGINX Plus features are available as extensions to the Ingress resource via annotations
and the ConfigMap resource. In addition to HTTP, NGINX Ingress Controller supports load balancing Websocket, gRPC, TCP
and UDP applications. See
ConfigMap and
Annotations
docs to learn more about the supported features and customization options.

As an alternative to the Ingress, NGINX Ingress Controller supports the VirtualServer and VirtualServerRoute resources.
They enable use cases not supported with the Ingress resource, such as traffic splitting and advanced content-based
routing. See VirtualServer and VirtualServerRoute resources
doc.

TCP, UDP and TLS Passthrough load balancing is also supported. See the TransportServer resource
doc.

Read this doc to learn more about NGINX Ingress
Controller with NGINX Plus.

Note

This project is different from the NGINX Ingress Controller in
kubernetes/ingress-nginx repo.

Ingress and Ingress Controller Copy Chapter

What is the Ingress? Copy Section

The Ingress is a Kubernetes resource that lets you configure an HTTP load balancer for applications running on
Kubernetes, represented by one or more Services.
Such a load balancer is necessary to deliver those applications to clients outside of the Kubernetes cluster.

The Ingress resource supports the following features:

• Content-based routing:
  • Host-based routing. For example, routing requests with the host header foo.example.com to one group of services
    and the host header bar.example.com to another group.
  • Path-based routing. For example, routing requests with the URI that starts with /serviceA to service A and
    requests with the URI that starts with /serviceB to service B.
• TLS/SSL termination for each hostname, such as foo.example.com.

See the Ingress User Guide to learn more about the
Ingress resource.

What is the Ingress Controller? Copy Section

The Ingress Controller is an application that runs in a cluster and configures an HTTP load balancer according to
Ingress resources. The load balancer can be a software load balancer running in the cluster or a hardware or cloud load
balancer running externally. Different load balancers require different Ingress Controller implementations.

In the case of NGINX, the Ingress Controller is deployed in a pod along with the load balancer.

Getting Started Copy Chapter

Note

All documentation should only be used with the latest stable release, indicated on the releases
page of the GitHub repository.

1. Install NGINX Ingress Controller using the Helm
   chart or the Kubernetes
   manifests.
2. Configure load balancing for a simple web application:
   • Use the Ingress resource. See the Cafe
     example.
   • Or the VirtualServer resource. See the Basic
     configuration
     example.
3. See additional configuration examples.
4. Learn more about all available configuration and customization in the
   docs.

NGINX Ingress Controller Releases Copy Chapter

We publish NGINX Ingress Controller releases on GitHub. See our releases
page.

The latest stable release is 5.5.4. For production
use, we recommend that you choose the latest stable release.

The edge version is useful for experimenting with new features that are not yet published in a stable release. To use
it, choose the edge version built from the latest
commit from the main branch.

To use NGINX Ingress Controller, you need to have access to:

• An NGINX Ingress Controller image.
• Installation manifests or a Helm chart.
• Documentation and examples.

It is important that the versions of those things above match.

The table below summarizes the options regarding the images, Helm chart, manifests, documentation and examples and gives
your links to the correct versions:

SBOM (Software Bill of Materials) Copy Chapter

We generate SBOMs for the binaries and the Docker images.

Binaries Copy Section

The SBOMs for the binaries are available in the releases page. The SBOMs are generated using
syft and are available in SPDX format.

Docker Images Copy Section

The SBOMs for the Docker images are available in the DockerHub, GitHub
Container, Amazon ECR Public
Gallery or Quay.io
repositories. The SBOMs are generated using syft and stored as an attestation in the
image manifest.

For example to retrieve the SBOM for linux/amd64 from Docker Hub and analyze it using
grype you can run the following command:

docker buildx imagetools inspect nginx/nginx-ingress:edge --format '{{ json (index .SBOM "linux/amd64").SPDX }}' | grype

Contacts Copy Chapter

We’d like to hear your feedback! If you have any suggestions or experience issues with our Ingress Controller, please
create an issue or send a pull request on GitHub.

Contributing Copy Chapter

If you'd like to contribute to the project, please read our Contributing guide.

Support Copy Chapter

For NGINX Plus customers NGINX Ingress Controller (when used with NGINX Plus) is covered by the support contract.

File: charts/nginx-ingress/README.md Copy Chapter

Helm Documentation

Please refer to the Installation with Helm guide in the NGINX Ingress Controller documentation site.

File: deployments/README.md Copy Chapter

Installation

This folder includes Kubernetes manifests for installing NGINX or NGINX Plus Ingress Controller. Read the installation
instructions.

File: examples/shared-examples/wildcard-tls-certificate/README.md Copy Chapter

Wildcard TLS Certificate

The wildcard TLS certificate simplifies the configuration of TLS termination if you need to use the same TLS certificate
in multiple Ingress and VirtualServer resources from various namespaces. Typically, such a certificate is for a
subdomain (for example, *.example.com), while the hosts in the Ingress and VirtualServer resources include that
subdomain (for example, foo.example.com, bar.example.com).

Example Copy Chapter

Prerequisites Copy Section

Start the Ingress Controller with the -wildcard-tls-secret command-line
argument
set to a TLS secret with a wildcard cert/key. For example:

-wildcard-tls-secret=nginx-ingress/wildcard-tls-secret

Note: the Ingress Controller supports only one wildcard TLS secret.

Configuring TLS Termination Copy Section

In the example below we configure TLS termination for an Ingress for the host foo.example.com and a VirtualServer for
the host bar.example.com:

foo Ingress from the namespace foo:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: foo
 namespace: foo
spec:
 ingressClassName: nginx
 tls:
 - hosts:
   - foo.example.com
 rules:
 - host: foo.example.com
   http:
     paths:
     - path: /
       pathType: Prefix
       backend:
         service:
           name: foo-service
           port:
             number: 80

bar VirtualServer from the namespace bar:

apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: bar
  namespace: bar
spec:
  host: bar.example.com
  tls:
    secret: ""
  upstreams:
  - name: bar
    service: bar-service
    port: 80
  routes:
  - path: /
    action:
      pass: bar

Because we don't reference any TLS secret in the resources above -- there is no secret field in the tls section of
the Ingress resource and the secret field is empty in the VirtualServer -- NGINX will use the wildcard secret
specified in the -wildcard-tls-secret command-line argument.

File: examples/shared-examples/rbac/README.md Copy Chapter

RBAC

It is possible to run the Ingress Controller in a cluster with
RBAC enabled.
Read the installation instructions.

File: examples/shared-examples/proxy-protocol/README.md Copy Chapter

PROXY Protocol

Proxies and load balancers, such as HAProxy or ELB, can pass the client's information (the IP address and the port) to
the next proxy or load balancer via the PROXY Protocol. To enable NGINX Ingress Controller to receive that information,
use the proxy-protocol ConfigMaps configuration key as well as the real-ip-header and the set-real-ip-from keys.
Once you enable the PROXY Protocol, it is enabled for every Ingress and VirtualServer resource. NOTE TransportServer
resource supports PROXY Protocol only when TLS Passthrough is enabled for the Ingress Controller.

Syntax Copy Chapter

The proxy-protocol key syntax is as follows:

proxy-protocol: "True | False"

Additionally, you must configure the following keys:

• real-ip-header: Set its value to proxy_protocol.
• set-real-ip-from: Set its value to the IP address or the subnet of the proxy or the load balancer. See
  set-real-ip-from

Example Copy Chapter

In the example below we configure the PROXY Protocol via a ConfigMaps resource. set-real-ip-from is set to
192.168.0.0/16. This is the CIDR range of the proxy that sits in front of the Ingress Controller in this example. You
can set this to 0.0.0.0/0 to trust all IPs. After we create the ConfigMaps resource, the client's IP address is
available via the $remote_addr variable in the NGINX configuration. By default, NGINX Ingress Controller logs the
value of this variable and also passes the value to the backend service in the X-Real-IP header.

The default log format for NGINX is '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"'

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
data:
  proxy-protocol: "True"
  real-ip-header: "proxy_protocol"
  set-real-ip-from: "192.168.0.0/16"

File: examples/shared-examples/otel/README.md Copy Chapter

Enable OpenTelemetry

This is the example code used in the Enable OpenTelemetry documentation.

File: examples/shared-examples/nginx-plus-secret/README.md Copy Chapter

NGINX Plus Secret

Refer to the Create License Secret docs to download and create a License Secret

File: examples/shared-examples/custom-templates/README.md Copy Chapter

Custom Templates

The Ingress Controller allows you to customize your templates through a
ConfigMap
via the following keys:

• main-template - Sets the main NGINX configuration template.
• ingress-template - Sets the Ingress NGINX configuration template for an Ingress resource.
• virtualserver-template - Sets the NGINX configuration template for an VirtualServer resource.
• transportserver-template - Sets the NGINX configuration template for a TransportServer resource.

Example Copy Chapter

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  main-template: |
    worker_processes  {{.WorkerProcesses}};
    ...
        include /etc/nginx/conf.d/*.conf;
    }
  ingress-template: |
    {{range $upstream := .Upstreams}}
    upstream {{$upstream.Name}} {
      {{if $upstream.LBMethod }}{{$upstream.LBMethod}};{{end}}
    ...
    }{{end}}
  virtualserver-template: |
    {{ range $u := .Upstreams }}
    upstream {{ $u.Name }} {
      {{ if ne $u.UpstreamZoneSize "0" }}zone {{ $u.Name }} {{ $u.UpstreamZoneSize }};{{ end }}
    ...
    }
    {{ end }}
  transportserver-template: |
    {{- range $u := .Upstreams }}
    upstream {{ $u.Name }} {
        zone {{ $u.Name }} 256k;
    ...
    }

Notes:

• The templates are truncated for the clarity of the example.
• The templates for NGINX (the main nginx.tmpl and the Ingress nginx.ingress.tmpl) and NGINX Plus (the main
  nginx-plus.tmpl and the Ingress nginx-plus.ingress.tmpl) are located at
  internal/configs/version1. The VirtualServer templates for NGINX
  (nginx.virtualserver.tmpl) and NGINX Plus (nginx-plus.virtualserver.tmpl) are located at
  internal/configs/version2.

Troubleshooting Copy Chapter

• If a custom template contained within the ConfigMap is invalid on startup, the Ingress Controller will fail to start,
  the error will be reported in the Ingress Controller logs.

  An example of an error from the logs:

  text Copy

  Error updating NGINX main template: template: nginxTemplate:98: unexpected EOF

• If a custom template contained within the ConfigMap is invalid on update, the Ingress Controller will not update the
  NGINX configuration, the error will be reported in the Ingress Controller logs and an event with the error will be
  associated with the ConfigMap.

  An example of an error from the logs:

  text Copy

  Error when updating config from ConfigMap: Invalid nginx configuration detected, not reloading

  An example of an event with an error (you can view events associated with the ConfigMap by running kubectl describe -n nginx-ingress configmap nginx-config):

  text Copy

  Events:
    Type     Reason            Age                From                      Message
    ----     ------            ----               ----                      -------
    Normal   Updated           12s (x2 over 25s)  nginx-ingress-controller  Configuration from nginx-ingress/nginx-config was updated
    Warning  UpdatedWithError  10s                nginx-ingress-controller  Configuration from nginx-ingress/nginx-config was updated, but not applied: Error when parsing the main template: template: nginxTemplate:98: unexpected EOF
    Warning  UpdatedWithError  8s                 nginx-ingress-controller  Configuration from nginx-ingress/nginx-config was updated, but not applied: Error when writing main Config

• If custom templates from before version 5.5 are used without updating to match the upstream versions, the Ingress
  Controller may fail to start. The error will be reported in the Ingress Controller logs.

  An example of an error from the logs:

  text Copy

  Error generating NGINX main config: template: nginxTemplate:142:18: executing "nginxTemplate" at <.DefaultHTTPListenerPort>: can't evaluate field DefaultHTTPListenerPort in type *version1.MainConfig

File: examples/shared-examples/custom-log-format/README.md Copy Chapter

Custom NGINX log format

This example lets you set the log-format for NGINX using the configmap resource

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  log-format:  '$remote_addr - $remote_user [$time_local] "$request" $status $grpc_status $body_bytes_sent "$http_referer"  "$http_user_agent" "$http_x_forwarded_for" "$resource_name" "$resource_type" "$resource_namespace" "$service"'

In addition to the built-in NGINX variables, you can also use the variables
that the Ingress Controller configures:

• $resource_type - The type of kubernetes resource that handled the client request.
• $resource_name - The name of the resource
• $resource_namespace - The namespace the resource exists in.
• $service - The name of the service the client request was sent to.
• $grpc_status - the gRPC status code, which is constructed either from the HTTP/2 trailer (grpc_status) returned from
  the backend for normal conditions, or from the HTTP/2 header (grpc_status) set either by the backend or by NGINX
  itself for some error conditions.

note These variables are only available for Ingress, VirtualServer and VirtualServerRoute resources.