# Technical Documentation: eksctl-io/eksctl > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/eksctl-io/eksctl) ### Overview (`README.md`) # `eksctl` - The official CLI for Amazon EKS ! eksctl [](https://goreportcard.com/report/github.com/eksctl-io/eksctl) `eksctl` is a simple CLI tool for creating clusters on EKS - Amazon's new managed Kubernetes service for EC2. It is written in Go, and uses CloudFormation. You can create a cluster in minutes with just one command – **`eksctl create cluster`**! _Need help? Join [Eksctl Slack][slackjoin]._ ## New: EKS Auto Mode Support `eksctl` now supports EKS Auto Mode! EKS Auto Mode automates routine tasks for cluster compute, storage, and networking. * Learn how to [create an EKS Auto Mode Cluster with eksctl](https://docs.aws.amazon.com/eks/latest/userguide/automode-get-started-eksctl.html). * Review the [eksctl docs](https://github.com/eksctl-io/eksctl/blob/main/userdocs/src/usage/auto-mode.md) for EKS Auto Mode. Version 0.195.0 or greater is required. ## New: EKS Hybrid Nodes Support `eksctl` now supports EKS Hybrid Nodes! Hybrid Nodes enables you to run on-premises and edge applications on customer-managed infrastructure with the same AWS EKS clusters, features, and tools you use in the AWS Cloud. * For more information, see [Amazon EKS Hybrid Nodes overview](https://docs.aws.amazon.com/eks/latest/userguide/hybrid-nodes-overview.html) in the EKS User Guide. * Review the [eksctl docs](https://github.com/eksctl-io/eksctl/blob/main/userdocs/src/usage/hybrid-nodes.md) for EKS Hybrid Nodes. Version 0.195.0 or greater is required. ## Installation `eksctl` is available to install from official releases as described below. We recommend that you install `eksctl` from only the official GitHub releases. You may opt to use a third-party installer, but please be advised that AWS does not maintain nor support these methods of installation. Use them at your own discretion. ### Prerequisite You will need to have AWS API credentials configured. What works for AWS CLI or any other tools (kops, Terraform, etc.) should be sufficient. You can use [`~/.aws/credentials` file][awsconfig] or [environment variables][awsenv]. For more information read [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html). [awsenv]: https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html [awsconfig]: https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html You will also need [AWS IAM Authenticator for Kubernetes](https://github.com/kubernetes-sigs/aws-iam-authenticator) command (either `aws-iam-authenticator` or `aws eks get-token` (available in version 1.16.156 or greater of AWS CLI) in your `PATH`. The IAM account used for EKS cluster creation should have these minimal access levels. | AWS Service | Access Level | |------------------|--------------------------------------------------------| | CloudFormation | Full Access | | EC2 | **Full:** Tagging **Limited:** List, Read, Write | | EC2 Auto Scaling | **Limited:** List, Write | | EKS | Full Access | | IAM | **Limited:** List, Read, Write, Permissions Management | | Systems Manager | **Limited:** List, Read | The inline policy json is listed in [Minimal IAM Policies](https://eksctl.io/usage/minimum-iam-policies/). ### For Unix To download the latest release, run: ```sh # for ARM systems, set ARCH to: `arm64`, `armv6` or `armv7` ARCH=$(uname -m | sed 's/x86_64/amd64/' | sed 's/aarch64/arm64/') PLATFORM=$(uname -s)_$ARCH curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz" # (Optional) Verify checksum curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz sudo install -m 0755 /tmp/eksctl /usr/local/bin && rm /tmp/eksctl ``` ### For Windows #### Direct download (latest release): [AMD64/x86_64](https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_windows_amd64.zip) - [ARMv6](https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_windows_armv6.zip) - [ARMv7](https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_windows_armv7.zip) - [ARM64](https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_windows_arm64.zip) Make sure to unzip the archive to a folder in the `PATH` variable. Optionally, verify the checksum: 1. Download the checksum file: [latest](https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt) 2. Use Command Prompt to manually compare `CertUtil`'s output to the checksum file downloaded. ```cmd REM Replace amd64 with armv6, armv7 or arm64 CertUtil -hashfile eksctl_Windows_amd64.zip SHA256 ``` 3. Using PowerShell to automate the verification using the `-eq` operator to get a `True` or `False` result: ```pwsh # Replace amd64 with armv6, armv7 or arm64 (Get-FileHash -Algorithm SHA256 .\eksctl_Windows_amd64.zip).Hash -eq ((Get-Content .\eksctl_checksums.txt) -match 'eksctl_Windows_amd64.zip' -split ' ')[0] ``` #### Using Git Bash ```sh # for ARM systems, set ARCH to: `arm64`, `armv6` or `armv7` ARCH=$(uname -m | sed 's/x86_64/amd64/' | sed 's/aarch64/arm64/') PLATFORM=windows_$ARCH curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.zip" # (Optional) Verify checksum curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check unzip eksctl_$PLATFORM.zip -d $HOME/bin rm eksctl_$PLATFORM.zip ``` The `eksctl` executable is placed in `$HOME/bin`, which is in `$PATH` from Git Bash. ### Docker For every release and RC a container image is pushed to ECR repository `public.ecr.aws/eksctl/eksctl`. Learn more about the usage on [ECR Public Gallery - eksctl](https://gallery.ecr.aws/eksctl/eksctl). For example, ```bash docker run --rm -it public.ecr.aws/eksctl/eksctl version ``` ### Third-Party Installers (Not Recommended) #### For MacOS [Homebrew](https://brew.sh) ```bash brew tap weaveworks/tap brew install weaveworks/tap/eksctl ``` [MacPorts](https://www.macports.org) ```bash port install eksctl ``` #### For Windows [chocolatey](https://chocolatey.org) ```bash choco install eksctl ``` [scoop](https://scoop.sh) ```bash scoop install eksctl ``` ## Basic usage To create a basic cluster, run: ```bash eksctl create cluster ``` A cluster will be created with default parameters - exciting auto-generated name, e.g. "fabulous-mushroom-1527688624" - 2x `m5.large` nodes (this instance type suits most common use-cases, and is good value for money) - use official AWS EKS AMI - `us-west-2` region - dedicated VPC (check your quotas) Once you have created a cluster, you will find that cluster credentials were added in `~/.kube/config`. If you have `kubectl` as well as `aws-iam-authenticator` commands in your `PATH`, you should be able to use `kubectl`. You will need to make sure to use the same AWS API credentials for this also. Check [EKS docs][ekskubectl] for instructions. [ekskubectl]: https://docs.aws.amazon.com/eks/latest/userguide/configure-kubectl.html Example output: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Install `eksctl` by following the [installation instructions](https://eksctl.io/installation). To discover the full range of capabilities that eksctl offers, visit [eksctl.io](https://eksctl.io). A great starting point is the [Getting Started](https://eksctl.io/getting-started/) guide. You can find a comprehensive list of features [here](https://eksctl.io/installation/#features). ## Contributions Code contributions are very welcome. If you are interested in helping make `eksctl` great then see our [contributing guide](CONTRIBUTING.md). We follow the [CNCF Code of Conduct](CODE_OF_CONDUCT.md). ## Releases Minor releases of `eksctl` should be expected every two weeks and patch releases will be made available as needed. One or more release candidate(s) (RC) builds will be made available prior to each minor release. RC builds are intended only for testing purposes. ## [Security Policy](SECURITY.md) If you discover a potential security issue in `eksctl` project, please follow [AWS Vulnerability Reporting process.](https://aws.amazon.com/security/vulnerability-reporting/) Do not open security related issues in the open source project. ## Get in touch [Create an issue](https://github.com/eksctl-io/eksctl/issues/new), or login to [Eksctl Community Slack (#eksctl)][slackchan] ([signup][slackjoin]). [slackjoin]: https://slack.k8s.io/ [slackchan]: https://slack.k8s.io/messages/eksctl/ > **_Logo Credits_** > > _Original Gophers drawn by [Ashley McNamara](https://twitter.com/ashleymcnamara), unique E, K, S, C, T & L Gopher identities had been produced with [Gopherize.me](https://github.com/matryer/gopherize.me/)._ --- ### Overview (`userdocs/README.md`) # Important - Content Migration to AWS Docs The eksctl docs are live on the AWS Docs platform * New site: https://docs.aws.amazon.com/eks/latest/eksctl/what-is-eksctl.html * New GitHub source: https://github.com/eksctl-io/eksctl-docs * For information about editing the new repo in AsciiDoc format, see https://docs.aws.amazon.com/eks/latest/userguide/contribute.html This directory is for the `eksctl.io` site. The `eksctl.io` site is still live, but in the future it will redirect to the AWS Docs. # Writing and publishing docs on eksctl.io As of July 2025, the `eksctl.io` domain is owned by AWS Legal. The DNS has been migrated to Route53. The site hosting has been migrated to AWS Amplify. The user docs are written in [MkDocs](https://www.mkdocs.org/) using [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/). Install `mkdocs` and then you can do the following: While writing the docs, to preview locally: ```console $ make serve-pages INFO - Building documentation... INFO - Cleaning site directory INFO - The following pages exist in the docs directory, but are not included in the "nav" configuration: - index.md [I 191218 10:16:30 server:296] Serving on http://127.0.0.1:8000 [I 191218 10:16:30 handlers:62] Start watching changes [I 191218 10:16:30 handlers:64] Start detecting changes [I 191218 10:16:33 handlers:135] Browser Connected: http://127.0.0.1:8000/introduction/ ... ``` --- ### Overview (`pkg/goformation/README.md`) # AWS GoFormation [](https://github.com/awslabs/goformation/actions) [](http://godoc.org/github.com/awslabs/goformation) `GoFormation` is a Go library for working with AWS CloudFormation / AWS Serverless Application Model (SAM) templates. - [Main features](#main-features) - [Installation](#installation) - [Usage](#usage) - [Marshalling CloudFormation/SAM described with Go structs, into YAML/JSON](#marshalling-cloudformationsam-described-with-go-structs-into-yamljson) - [Unmarshalling CloudFormation YAML/JSON into Go structs](#unmarshalling-cloudformation-yamljson-into-go-structs) - [Updating CloudFormation / SAM Resources in GoFormation](#updating-cloudformation--sam-resources-in-goformation) - [Advanced](#advanced) - [AWS CloudFormation Intrinsic Functions](#aws-cloudformation-intrinsic-functions) - [Resolving References (Ref)](#resolving-references-ref) - [Versioning](#versioning) - [Contributing](#contributing) ## Main features * Describe AWS CloudFormation and AWS SAM templates as Go objects (structs), and then turn it into JSON/YAML. * Parse JSON/YAML AWS CloudFormation and AWS SAM templates and turn them into Go structs. * Strongly typed Go structs generated for every AWS CloudFormation and AWS SAM resource. * Automatically generated, from the published AWS CloudFormation Resource Specification. ## Installation As with other Go libraries, GoFormation can be installed with `go get`. ``` $ go get github.com/awslabs/goformation/v4 ``` ## Usage ### Marshalling CloudFormation/SAM described with Go structs, into YAML/JSON Below is an example of building a CloudFormation template programmatically, then outputting the resulting JSON ```go package main import ( "fmt" "strconv" "time" "github.com/awslabs/goformation/v4/cloudformation" "github.com/awslabs/goformation/v4/cloudformation/sns" ) func main() { // Create a new CloudFormation template template := cloudformation.NewTemplate() // Create an Amazon SNS topic, with a unique name based off the current timestamp template.Resources["MyTopic"] = &sns.Topic{ TopicName: "my-topic-" + strconv.FormatInt(time.Now().Unix(), 10), } // Create a subscription, connected to our topic, that forwards notifications to an email address template.Resources["MyTopicSubscription"] = &sns.Subscription{ TopicArn: cloudformation.Ref("MyTopic"), Protocol: "email", Endpoint: "some.email@example.com", } // Let's see the JSON AWS CloudFormation template j, err := template.JSON() if err != nil { fmt.Printf("Failed to generate JSON: %s\n", err) } else { fmt.Printf("%s\n", string(j)) } // and also the YAML AWS CloudFormation template y, err := template.YAML() if err != nil { fmt.Printf("Failed to generate YAML: %s\n", err) } else { fmt.Printf("%s\n", string(y)) } } ``` Would output the following JSON template: ```javascript { "AWSTemplateFormatVersion": "2010-09-09", "Resources": { "MyTopic": { "Properties": { "TopicName": "my-topic-1536878058" }, "Type": "AWS::SNS::Topic" }, "MyTopicSubscription": { "Properties": { "Endpoint": "some.email@example.com", "Protocol": "email", "TopicArn": { "Ref": "MyTopic" } }, "Type": "AWS::SNS::Subscription" } } } ``` ...and the following YAML template: ```yaml AWSTemplateFormatVersion: 2010-09-09 Resources: MyTopic: Properties: TopicName: my-topic-1536878058 Type: AWS::SNS::Topic MyTopicSubscription: Properties: Endpoint: some.email@example.com Protocol: email TopicArn: Ref: MyTopic Type: AWS::SNS::Subscription ``` When creating templates, you can use the following convenience functions to use [AWS CloudFormation Intrinsics](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html): - `cloudformation.Ref(logicalName string)` - `cloudformation.GetAtt(logicalName string, attribute string)` - `cloudformation.ImportValue(name string)` - `cloudformation.Base64(input string)` - `cloudformation.CIDR(ipBlock, count, cidrBits string)` - `cloudformation.FindInMap(mapName, topLevelKey, secondLevelKey string)` - `cloudformation.GetAZs(region string)` - `cloudformation.Join(delimiter string, values []string)` - `cloudformation.Select(index string, list []string)` - `cloudformation.Split(delimiter, source string)` - `cloudformation.Sub(value string)` - `And(conditions []string)` - `Equals(value1, value2 string)` - `If(value, ifEqual, ifNotEqual string)` - `Not(conditions []string)` - `Or(conditions []string)` ### Unmarshalling CloudFormation YAML/JSON into Go structs GoFormation also works the other way - parsing JSON/YAML CloudFormation/SAM templates into Go structs. ```go package main import ( "log" "github.com/awslabs/goformation/v4" ) func main() { // Open a template from file (can be JSON or YAML) template, err := goformation.Open("template.yaml") if err != nil { log.Fatalf("There was an error processing the template: %s", err) } // You can extract all resources of a certain type // Each AWS CloudFormation resource is a strongly typed struct functions := template.GetAllServerlessFunctionResources() for name, function := range functions { // E.g. Found a AWS::Serverless::Function named GetHelloWorld (runtime: nodejs6.10) log.Printf("Found a %s named %s (runtime: %s)\n", function.AWSCloudFormationType(), name, function.Runtime) } // You can also search for specific resources by their logicalId search := "GetHelloWorld" function, err := template.GetServerlessFunctionWithName(search) if err != nil { log.Fatalf("Function not found") } // E.g. Found a AWS::Serverless::Function named GetHelloWorld (runtime: nodejs6.10) log.Printf("Found a %s named %s (runtime: %s)\n", function.AWSCloudFormationType(), search, function.Runtime) } ``` ## Updating CloudFormation / SAM Resources in GoFormation AWS GoFormation contains automatically generated Go structs for every CloudFormation/SAM resource, located in the [cloudformation/](cloudformation/) directory. These can be generated, from the latest [AWS CloudFormation Resource Specification](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html) published for `us-east-1` by just running `go generate`: ``` $ go generate Generated 587 AWS CloudFormation resources from specification v1.4.2 Generated 17 AWS SAM resources from specification v2016-10-31 Generated JSON Schema: schema/cloudformation.schema.json ``` The GoFormation build pipeline automatically checks for any updated AWS CloudFormation resources on a daily basis, and creates a pull request against this repository if any are found. ## Advanced ### AWS CloudFormation Intrinsic Functions The following [AWS CloudFormation Intrinsic Functions](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html) are supported in GoFormation: - [x] [Fn::Base64](intrinsics/fnbase64.go) - [x] [Fn::FindInMap](intrinsics/fnfindinmap.go) - [x] [Fn::Join](intrinsics/fnjoin.go) - [x] [Fn::Select](intrinsics/fnselect.go) - [x] [Fn::Split](intrinsics/fnsplit.go) - [x] [Fn::Sub](intrinsics/fnsub.go) - [x] [Ref](intrinsics/ref.go) - [x] [Fn::And](intrinsics/fnand.go) - [x] [Fn::Equals](intrinsics/fnequals.go) - [x] [Fn::If](intrinsics/fnif.go) - [x] [Fn::Not](intrinsics/fnnot.go) - [x] [Fn::Or](intrinsics/fnor.go) - [ ] Fn::GetAtt - [x] [Fn::GetAZs](intrinsics/fngetazs.go) - [ ] Fn::ImportValue Any unsupported intrinsic functions will return `nil`. #### Resolving References (Ref) When converting a YAML/JSON template to go, the intrinsic 'Ref' function as implemented will resolve all of the [pseudo parameters](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html) such as `AWS::AccountId` with their default value as listed on [the bottom of this page](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html). If a reference is not a pseudo parameter, GoFormation will try to resolve it within the AWS CloudFormation template. **Currently, this implementation only searches for `Parameters` with a name that matches the ref, and returns the `Default` if it has one.** ## Versioning This library is automatically versioned and tagged using [semantic-release](https://github.com/semantic-release/semantic-release). ## Contributing Contributions and feedback are welcome! Proposals and pull requests will be considered and responded to. For more information, see the [CONTRIBUTING](CONTRIBUTING.md) file. --- ### Overview (`pkg/nodebootstrap/README.md`) # Nodebootstrap Until recently, `eksctl` provided its own bootstrapping/userdata logic on disk, over-writing or ignore those which came with the AMIs. This caused numerous headaches after changes upstream and we got tired of maintaining these extra pieces. ## Implementation For Unmanaged nodes there is an interface: ```go type Bootstrapper interface { UserData() (string, error) } ``` AMI families Ubuntu, AmazonLinux2, Bottlerocket and Windows all fulfil this. As of `eksctl` version `0.45.0` unmanaged nodes of these families, as well as managed nodes (different interface), will defer to the native bootstrap script which comes built into the image. This script is found on disk at `/etc/eks/bootstrap.sh`. `UserData` will provide a wrapper script which will set custom values and delegate to the official bootstrap script. ### Ubuntu & AmazonLinux2 The bootstrapping "prep" for these 2 are fairly similar. Common setup lives in `userdata.go`. Individual scripts are prepped in `ubuntu.go` and `al2.go`. Non-dynamic assets live in `assets/`. Both bootstrappers add `assets/bootstrap.helper.sh` to the node along with either `assets/bootstrap.ubuntu.sh` or `assets/bootstrap.al2.sh`. The call to `UserData` will also dynamically add the following: - `kubelet-extra.json` - user configuration for kubelet - `docker-extra.json` - extra config for docker daemon - `kubelet.env` - env vars for kubelet The bootstrap wrapper scripts will use `jq` and `sed` to get user and our config into various files, and then call `/etc/eks/bootstrap.sh`. ### AmazonLinux2023 While AL2023 implements the `Bootstrapper` interface, the underlying userdata will be entirely different from other AMI families. Specifically, AL2023 introduces a new node initialization process nodeadm that uses a YAML configuration schema, dropping the use of `/etc/eks/bootstrap.sh` script. For self-managed nodes, and for EKS-managed nodes based on custom AMIs, eksctl will populate userdata in the fashion below: ``` MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=// --// Content-Type: application/node.eks.aws apiVersion: node.eks.aws/v1alpha1 kind: NodeConfig spec: cluster: apiServerEndpoint: https://XXXX.us-west-2.eks.amazonaws.com certificateAuthority: XXXX cidr: 10.100.0.0/16 name: my-cluster kubelet: config: clusterDNS: - 10.100.0.10 flags: - --node-labels=alpha.eksctl.io/cluster-name=my-cluster,alpha.eksctl.io/nodegroup-name=my-nodegroup - --register-with-taints=special=true:NoSchedule (only for EKS-managed nodes) --//-- ``` ## Troubleshooting ### Ubuntu ```sh sudo snap logs kubelet-eks [-n=all/20] systemctl status docker.service ``` Files: ```sh /etc/eks/bootstrap.sh /var/lib/cloud/scripts/eksctl/bootstrap.ubuntu.sh /etc/kubernetes/kubelet/kubelet-config.json /etc/docker/daemon.json ``` ### AmazonLinux2 Status: ```sh systemctl status kubelet systemctl status docker ``` Logs: ```sh journalctl -u kubelet.service ``` Files: ```sh /etc/eks/bootstrap.sh /var/lib/cloud/scripts/eksctl/bootstrap.al2.sh /etc/kubernetes/kubelet/kubelet-config.json /etc/docker/daemon.json ``` --- ### Overview (`pkg/ctl/mcp/README.md`) # eksctl MCP (Model Context Protocol) Package ## Overview The `pkg/ctl/mcp` package implements a [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol) server for eksctl. This server enables AI assistants like Amazon Q to interact with eksctl functionality directly through a standardized protocol, allowing for seamless integration of eksctl commands into AI-powered workflows. ## What is MCP? Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context and tools to Large Language Models (LLMs). It enables AI assistants to: 1. Discover available tools and their capabilities 2. Execute commands and receive structured responses 3. Provide contextual information to enhance AI interactions ## How the eksctl MCP Server Works The eksctl MCP server exposes all eksctl commands as tools that can be invoked by AI assistants. The implementation consists of several key components: ### Core Components 1. **`mcp.go`**: Entry point that defines the `mcp` command and starts the MCP server 2. **`server.go`**: Creates and configures the MCP server with all eksctl commands 3. **`tools.go`**: Handles the registration of eksctl commands as MCP tools 4. **`types.go`**: Defines types and interfaces used by the MCP implementation ### Implementation Details - The server uses the `mcp-go` library to implement the Model Context Protocol - It dynamically registers all eksctl commands (create, get, delete, etc.) as MCP tools - Each command's help text, flags, and parameters are exposed through the protocol - The server runs as a stdio server, communicating through standard input/output ### Command Registration Process 1. The server recursively traverses the eksctl command tree 2. For each command, it extracts: - Command description and usage information - Available flags and their descriptions - Required and optional parameters ## Using the eksctl MCP Server with Amazon Q Chat To use the eksctl MCP server with Amazon Q Chat, you need to register it in the Amazon Q configuration file. Create or edit the file at `$HOME/.aws/amazonq/mcp.json` with the following content: ```json { "mcpServers": { "eks-tools": { "command": "eksctl", "args": [ "mcp" ] } } } ``` This configuration tells Amazon Q Chat to: 1. Register a server named "eks-tools" 2. Use the `eksctl mcp` command to start the MCP server 3. Make all eksctl commands available as tools with the prefix `eks___` ## Benefits - **Seamless Integration**: AI assistants can execute eksctl commands directly - **Structured Responses**: Commands return structured data that can be parsed by AI models - **Discoverability**: AI assistants can discover available commands and their parameters - **Context-Aware**: Provides rich context about EKS clusters and resources ## Development When extending the MCP server functionality: 1. Add new command registrations in `server.go` if new eksctl commands are created 2. Extend type definitions in `types.go` as needed 3. Modify `tools.go` if changes to tool registration logic are required ## Example Usage in Amazon Q Chat Once configured, users can interact with eksctl through Amazon Q Chat: ``` User: What EKS clusters do I have? Amazon Q: Let me check your EKS clusters. [Amazon Q executes the eksctl get cluster command and displays the results] User: Create a new EKS cluster Amazon Q: [Guides the user through cluster creation using eksctl commands] ``` The MCP server enables these interactions by providing Amazon Q with the ability to execute eksctl commands and interpret their results. ## Quick test for MCP server To quickly test the MCP server, you can run the following command in your terminal: ```bash echo '{"jsonrpc":"2.0","method":"tools/list","id":1}' | eksctl mcp | jq ``` ## Recommendations - Monitor background operations for long-running commands. Commands such as cluster creations have a 45-second timeout for command responses but the processes continue in background. --- ### Overview (`pkg/goformation/cloudformation/README.md`) ### Important All resource files in this directory are auto-generated by running 'go generate' in the repository root directory. Do not manually edit any of generated resource files --- ### Overview (`pkg/goformation/schema/README.md`) ### Important The JSON Schema in this directory is auto-generated by running 'go generate' in the repository root directory. Do not manually edit it. --- ### Overview (`pkg/goformation/test/yaml/sam-official-samples/encryption_proxy/README.MD`) ### Encryption/Decryption Proxy This template creates the following resources: * 2 Lambda functions (Encryption, Decryption) - SAM takes care of creating 2 stages API Gateway {Stage,Prod} that sits on top * IAM Role for each function * KMS Key and KMS Decrypt/Encrypt permissions for each IAM Role separately - KMS Key ID is then passed to Encryption Lambda via Env vars #### Encrypt Action * Consumes up to 4KB of data (string, json blob, etc.) * Returns encrypted data via JSON blob as follows ```javascript { "data": "base64_encrypted_blob" } ``` **Encrypting data via cURL** ```bash encryptURL=$(aws cloudformation describe-stacks --stack-name encryption-proxy --query 'Stacks[].Outputs[?OutputKey==`EncryptURL`].OutputValue' --output text) curl -H 'Content-Type: application/json' -X POST -d 'Here it is my super secret data...' ${encryptURL} ``` #### Decrypt Action * Consumes JSON blob received out of Encryption Action * Returns decrypted data via JSON blob as follows ```javascript { "data": "decrypted_data" } ``` **Decrypting data via cURL** ```bash decryptURL=$(aws cloudformation describe-stacks --stack-name encryption-proxy --query 'Stacks[].Outputs[?OutputKey==`DecryptURL`].OutputValue' --output text) curl -H 'Content-Type: application/json' -X POST -d '{ "data": "AQECAHjgf/RDFTAO0+ief3VXaJRnnKbyaaEsVhkmYxTcbiNhcgAAAHIwcAYJKoZIhvcNAQcGoGMw\nYQIBADBcBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDBX7iv8CwG1gr2Hc4wIBEIAvS1cXraMW\n3PU96z4AACGj7Wuo007HwWjtK/quSi3FKyYFvkJ10YhDOEvzxOD7Ntw=\n" }' ${decryptURL} ``` --- ### Contributor Meetings (`docs/contributor-meetings.md`) # Contributor Meetings ## Weekly meetings - 1st/3rd Wednesday of the month: 09:00-09:30 UK time - 2nd/4th Wednesday of the month: 16:00-16:30 UK time ### Meeting notes & agenda moved To make it easier for the whole team to take notes, we moved the [meeting notes](https://docs.google.com/document/d/1wHfEw7PAUDIFKuqEr7OwkpjgUUYlIDBHFj3I323vLUA/edit). Please join our [Google group](https://groups.google.com/forum/#!forum/eksctl) to get write access to the doc and/or the meeting invite into your calendar. ### 06/02/2019 Attendees: - @errordeveloper - @mumoshu Topics: - 0.1.20 release update - testing improvements - better integration tests are needed - more complete unit tests with some kind cfn mocking - docs and website - need to separate basic usage, getting-started vs advanced config file examples ### 13/02/2019 Attendees: - @errordeveloper - @mumoshu Topics: - 0.1.20 release update - it's out - next release and `v1alpha5` - separation of `NodeGroupConfig` object, so that nodegroups can be managed with a config file ### 20/02/2019 Attendees: - @errordeveloper - @mumoshu Topics: - 0.1.22 release - update on latest contributions - progress update on `kubectl drain` refactoring - `eksctl create nodegroup` with `--config-file` ### 27/02/2019 Attendees: - @errordeveloper - @mumoshu Topics: - 0.1.23 release - update on latest contributions - progress update on `kubectl drain` refactoring - [k/k#72827](https://github.com/kubernetes/kubernetes/pull/72827) merged - `eksctl utils describe-stacks` - additon of `--trail` - call for help on improving output [#585](https://github.com/eksctl-io/eksctl/issues/585) ### 06/03/2019 Attendees: - @errordeveloper - @mumoshu Topics: - upgrades - [#608](https://github.com/eksctl-io/eksctl/issues/608) ### 13/03/2019 Attendees: - @errordeveloper - @mumoshu Topics: - upgrades - [#608](https://github.com/eksctl-io/eksctl/issues/608) - `k8s.io/client-go` & co - `RawClient` and `RawResource` - [#624](https://github.com/eksctl-io/eksctl/pull/624) - update on challenges with going to production from @mumoshu - node-local DNS and chaching - gitops with helmfile - need better way to manage `aws-auth` ### 20/03/2019 Attendees: - @errordeveloper - @mumoshu Topics: - release updates - 0.1.24 - plan towards v1alpha5 - more discussion around storing config objects [#642](https://github.com/eksctl-io/eksctl/issues/642) ### 27/03/2019 Attendees: - @errordeveloper - @mumoshu - @martina-if Topics: - update on latest features that made it to the release - v1alpha5 - nodegroup deletion [#664](https://github.com/eksctl-io/eksctl/issues/664) ## 03/04/2019 Attendees: - @errordeveloper - @mumoshu - @pawelprazak Topics: - cloudformation template export functionality - make it explicitly about importing stacks - will help with refactoring - not a replacement for `eksctl apply` - update about on-going efforts (#673, #695 etc) - more discussion of storing config in cluster, and Cluster API ## 10/04/2019 Attendees: - @errordeveloper - @mumoshu Topics: - 0.1.28 & 0.1.29 releases are out - @martina's progress on refactoring the SSH config for next release - progress on task manager refactoring, handling of creation and deletion ## 17/04/2019 Attendees: - @errordeveloper - @mumoshu - @pawelprazak Topics: - update on v1alpha5 progress - @errordeveloper will be absent from next week's meeting - progress on other ongoing work (#714, #741) ## 01/04/2019 Attendees: - @errordeveloper - @mumoshu - @martina - @pawelprazak Topics: - 0.1.31 release is out - next key feature: spot instances - @pawelprazak's PR #718 ## 15/05/2019 Attendees: - @errordeveloper - @mumoshu - @martina-if Topics: - recent meeting cancellations - no meeting for next two weeks due to KubeCon and vocations - 0.1.32 release from previous week - next feature release - spot instances - config file documentation must be our priority ## 05/06/2019 Attendees: - @errordeveloper - @mumoshu - @martina-if Topics: - update on releases: 0.1.33 and 0.1.34 - change in `--max-pods` ## 12/06/2019 Attendees: - @errordeveloper - @mumoshu - @martina-if - @gemagomez Topics: - update on releases: 0.1.35 and 0.1.36 - new commands for IAM identity mappings - progress update on website & docs ## 07/08/2019 Attendees: - @errordeveloper - @mumoshu Topics: - latest releases, new cadence - gitops features, add-ons and cluster configuration ## 14/08/2019 Attendees: - @errordeveloper Topics: - 0.4.0 update --- ### Current 001 Node Bootstrap (`docs/current-001-node_bootstrap.md`) # THIS DESIGN DEPRECATED AS OF 0.47.0 See [pkg/nodebootstrap/README.md](/pkg/nodebootstrap/README.md) for current implementation details. -------------- # Current Design #001: Node Bootstrap > NOTE: the purpose of this document is to summarize how node bootstrap process is currently implemented in `eksctl`, > so it can be referred to in any discussions. ## Guiding Principles To enable reliable node management at large-scale, the bootstrap process of a node should bear the following attributes: - stateless - immutable - deterministic - simple It **should avoid**: - non-essential runtime parameters - mutation of configuration during execution - generating complex configuration files - download of configuration files - installation of software - making unnecessary look-up calls to remote APIs - only call out to commands when absolutely essential Current implementation is build for official AL2 EKS AMIs, that includes dependencies such as AWS CLI and jq, however it's ought to be possible for one to create a stripped-down AMI and re-use the bootstrap script. Aside from all of the above concerns, it's crucial to bear in mind that the dependencies of bootstrap process should not pose requirements that affect running of the node or the entire cluster past the bootstrap stage. For example, relying on particular AWS APIs implies instance role has to be adjusted to allow for access to such APIs, which could otherwise be avoided, or any software required solely for bootstrap function has to be tracked for security vulnerabilities. ## Known Anti-patterns ### Use of Dependencies If bootstrap process requires use of a dependency, such as AWS CLI and jq, it means this dependency has to be installed on the AMI, and thereby it affects running of the node beyond the bootstrap stage. If a vulnerability is found, e.g. in one of the Python packages that AWS CLI uses, the node will have to be upgraded. ### Use of EC2 tags for parameter discovery This requires multiple remote calls, and parsing JSON. It can be done with AWS CLI, yet it's a Python dependency that is not essential otherwise. Furthermore, discovering tags requires listing tags of all instances and searching the list. It's not as simple as asking "what are the tags of this instance?", it's "what is the ID of this instance?", followed by "what are the tags of all instances? which of all instances is this instance? what are the tags of this instance?". Aside from this, this approach requires instance IAM role to have access to EC2 APIs, which is considered non-essential for any other purpose. ## Current Design Summary: - all configuration files along with [the bootstrap script](https://github.com/eksctl-io/eksctl/blob/70041a226bb8ef5c51a229d587235551a2410eda/pkg/nodebootstrap/assets/bootstrap.al2.sh) are passed to the node as cloud-init config - all configuration files are written to `/etc/eksctl` directory - the only remote API that's used is the EC2 metadata service, required to obtain instance ID, type and IP address - the behavior is fully determined by: - `ClusterConfig` set in configuration file - version of `eksctl` - single mode of execution More specifically, as part of CloudFormation template, `eksctl` will define full content of the following files: - [`/etc/systemd/system/kubelet.service.d/10-eksclt.al2.conf`](#systemd-unit-for-amazon-linux) - systemd drop-in for kubelet - [`/etc/eksctl/kubelet.yaml`](#kubelet-configuration) - fully-formed kubelet configuration file (NOTE: currently it doesn't support all the options that flags support) - [`/etc/eksctl/kubeconfig.yaml`](#kubeconfig) - client credentials for kubelet - [`/etc/eksctl/ca.crt`](#ca-file) - [`/etc/eksctl/metadata.env`](#metadata-environment-variables) - known metadata for all nodes in a nodegroup, used by the bootstrap process and for kubelet flags that are not otherwise settable via `/etc/eksctl/kubelet.yaml` - [`/etc/eksctl/kubelet.env`](#kubelet-environment-variables) - kubelet-specific metadata, used for flags that are not otherwise settable via `/etc/eksctl/kubelet.yaml` - [`/etc/eksctl/kubelet.local.env`](#kubelet-local-environment-variables) - kubelet-specific metadata that is local to the specific node - [`/etc/eksctl/max_pods.map`](#max-pods-file) - used by the bootstrap script (NOTE: based on [`eni-max-pods.txt`](https://raw.github.com/awslabs/amazon-eks-ami/master/files/eni-max-pods.txt), but format differs to ease parsing) ## Source Code The bootstrapping of un-managed nodegroups is done through CloudFormation with the files that eksctl sends as `userData` (the user data is a property of the launch template used to spawn the nodes in EC2). The main files are the [bootstrap script](#bootstrap-script-for-amazon-linux-2), the [systemd unit](#systemd-unit-for-amazon-linux), the [kubelet.yaml](#kubelet-configuration) and the [kubeconfig.yalm](#kubeconfig). ### [Bootstrap Script for Amazon Linux 2](https://github.com/eksctl-io/eksctl/blob/master/pkg/nodebootstrap/assets/bootstrap.al2.sh) This is sent as one of the `runScript`s in the `userData` and it is used to bootstrap the node. It can also be [overwritten](/usage/schema/#nodeGroups-overrideBootstrapCommand) by the user or appended with [extra shell commands](/usage/schema/#nodeGroups-preBootstrapCommands). Dependencies: - systemd - cat - bash - curl 
#!/bin/bash

set -o errexit
set -o pipefail
set -o nounset

function get_max_pods() {
  while read instance_type pods; do
    if  [[ "${instance_type}" == "${1}" ]] && [[ "${pods}" =~ ^[0-9]+$ ]] ; then
      echo ${pods}
      return
    fi
  done < /etc/eksctl/max_pods.map
}

# Use IMDSv2 to get metadata
TOKEN="$(curl --silent -X PUT -H "X-aws-ec2-metadata-token-ttl-seconds: 600" http://169.254.169.254/latest/api/token)"
function get_metadata() {
  curl --silent -H "X-aws-ec2-metadata-token: $TOKEN" "http://169.254.169.254/latest/meta-data/$1"
}

NODE_IP="$(get_metadata local-ipv4)"
INSTANCE_ID="$(get_metadata instance-id)"
INSTANCE_TYPE="$(get_metadata instance-type)"
AWS_SERVICES_DOMAIN="$(get_metadata services/domain)"


source /etc/eksctl/kubelet.env # this can override MAX_PODS

cat > /etc/eksctl/kubelet.local.env  <<EOF
NODE_IP=${NODE_IP}
INSTANCE_ID=${INSTANCE_ID}
INSTANCE_TYPE=${INSTANCE_TYPE}
AWS_SERVICES_DOMAIN=${AWS_SERVICES_DOMAIN}
MAX_PODS=${MAX_PODS:-$(get_max_pods "${INSTANCE_TYPE}")}
EOF

systemctl daemon-reload
systemctl enable kubelet
systemctl start kubelet
### [Bootstrap Script for Ubuntu](https://github.com/eksctl-io/eksctl/blob/master/pkg/nodebootstrap/assets/bootstrap.ubuntu.sh) Same as above, but Ubuntu-specific commands are used instead of `systemctl` and the drop-in unit. ### [Systemd Unit for Amazon Linux](https://github.com/eksctl-io/eksctl/blob/70041a226bb8ef5c51a229d587235551a2410eda/pkg/nodebootstrap/assets/10-eksclt.al2.conf) This is the systemd unit file used to start the kubelet service. It makes use of other configuration files sent by eksctl and it is stored in `etc/systemd/system/kubelet.service.d/10-eksctl.al2.conf`. 
# eksctl-specific systemd drop-in unit for kubelet, for Amazon Linux 2 (AL2)

[Service]
EnvironmentFile=/etc/eksctl/metadata.env
EnvironmentFile=/etc/eksctl/kubelet.env
EnvironmentFile=/etc/eksctl/kubelet.local.env

ExecStart=
ExecStart=/usr/bin/kubelet \
  --node-ip=${NODE_IP} \
  --node-labels=${NODE_LABELS},alpha.eksctl.io/instance-id=${INSTANCE_ID} \
  --max-pods=${MAX_PODS} \
  --register-node=true --register-with-taints=${NODE_TAINTS} \
  --cloud-provider=aws \
  --container-runtime=docker \
  --network-plugin=cni \
  --cni-bin-dir=/opt/cni/bin \
  --cni-conf-dir=/etc/cni/net.d \
  --pod-infra-container-image=${AWS_EKS_ECR_ACCOUNT}.dkr.ecr.${AWS_DEFAULT_REGION}.${AWS_SERVICES_DOMAIN}/eks/pause:3.1-eksbuild.1 \
  --kubeconfig=/etc/eksctl/kubeconfig.yaml \
  --config=/etc/eksctl/kubelet.yaml
### [Kubelet configuration](https://github.com/eksctl-io/eksctl/blob/70041a226bb8ef5c51a229d587235551a2410eda/pkg/nodebootstrap/assets/kubelet.yaml) The configuration for the kubelet is stored in `/etc/eksctl/kubelet.yaml`. 
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1

clusterDNS:
- 10.100.0.10

address: 0.0.0.0
clusterDomain: cluster.local
serverTLSBootstrap: true
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 2m0s
    enabled: true
  x509:
    clientCAFile: /etc/eksctl/ca.crt

authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s

cgroupDriver: cgroupfs
kubeReserved:
  cpu: 60m
  ephemeral-storage: 1Gi
  memory: 343Mi

featureGates:
  RotateKubeletServerCertificate: true
### Kubeconfig The kubeconfig is stored in `/etc/eksctl/kubeconfig.yaml`. 
kind: Config
apiVersion: v1

clusters:
- cluster:
    certificate-authority: /etc/eksctl/ca.crt
    server: https://0A3....gr7.us-west-2.eks.amazonaws.com
  name: my-cluster-1.us-west-2.eksctl.io
contexts:
- context:
    cluster: my-cluster-1.us-west-2.eksctl.io
    user: kubelet@my-cluster-1.us-west-2.eksctl.io
  name: kubelet@my-cluster-1.us-west-2.eksctl.io
current-context: kubelet@my-cluster-1.us-west-2.eksctl.io

preferences: {}

users:
- name: kubelet@my-cluster-1.us-west-2.eksctl.io
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - eks
      - get-token
      - --cluster-name
      - my-cluster-1
      - --region
      - us-west-2
      command: aws
      env:
      - name: AWS_STS_REGIONAL_ENDPOINTS
        value: regional
### Kubelet local environment variables Stored in `/etc/eksctl/kubelet.local.env`, this file contains dynamic data and is generated by eksctl when generating the CloudFormation template. ``` NODE_IP=192.168.72.51 INSTANCE_ID=i-0b704274a75a321a7 INSTANCE_TYPE=m6g.medium AWS_SERVICES_DOMAIN=amazonaws.com MAX_PODS=8 ``` ### Metadata environment variables `/etc/eksctl/metadata.env` ``` AWS_DEFAULT_REGION=us-west-2 AWS_EKS_CLUSTER_NAME=my-cluster-1 AWS_EKS_ENDPOINT=https://0A123.....gr7.us-west-2.eks.amazonaws.com AWS_EKS_ECR_ACCOUNT=602401143452 ``` ### Kubelet environment variables `/etc/eksctl/kubelet.env` ``` NODE_LABELS=alpha.eksctl.io/cluster-name=my-cluster-1,alpha.eksctl.io/nodegroup-name=ng-1 NODE_TAINTS= ``` ### CA File `etc/eksctl/ca.crt` ``` -----BEGIN CERTIFICATE----- 1231231231231231231231231231231231231231231231231231231231231233 ABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCABCA ........ -----END CERTIFICATE----- ``` ### Max pods file `/etc/eksctl/max_pods.map` ``` t2.micro 4 x1e.32xlarge 234 c1.xlarge 58 g4dn.2xlarge 29 r5.24xlarge 737 r5a.12xlarge 234 r5ad.xlarge 58 t3.2xlarge 58 c1.medium 12 m5d.16xlarge 737 ... ``` ## Comparison to EKS Amazon Linux 2 script The [EKS Amazon Linux 2 script](https://github.com/awslabs/amazon-eks-ami/blob/b85ef2fce5f46eddafa2e8881d757b31a9feed81/files/bootstrap.sh): - has 228 lines of bash code - has 9 flags representing different modes of execution - uses `sed` to mutate a YAML file (`/var/lib/kubelet/kubeconfig`) - uses `jq` to generate JSON files (`/etc/docker/daemon.json` and `/etc/kubernetes/kubelet/kubelet-config.json`) - calls `aws eks wait cluster-active` and `aws eks describe-cluster` - implements API call retry logic - uses `awk` and `grep` to parse structured data obtained from EC2 metadata API Dependencies: - systemd - cat - bash - curl - awk - sed - jq - aws ## Comparison to Ubuntu script No source for the script used by Ubuntu is publicly available. It can be obtained from the AMI, but it is not clear if copyright may permit us from sharing the script here. It is known to be very different from the AL2 script, but similar in complexity and requires access to EC2 APIs for tag-based discovery (discussed above). --- ### Proposal 000 Template (`docs/proposal-000-template.md`) # Short, descriptive title ## Authors ## Status ## Table of Contents - [Summary](#summary) - [Motivation](#motivation) - [Goals](#goals) - [Non-Goals](#non-goals) - [Linked Docs](#linked-docs) - [Proposal](#proposal) - [User Stories (Optional)](#user-stories-optional) - [Story 1](#story-1) - [Story 2](#story-2) - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) - [Risks and Mitigations](#risks-and-mitigations) - [Design Details](#design-details) - [Test Plan](#test-plan) - [Graduation Criteria (Optional)](#graduation-criteria-optional) - [Migration Strategy (Optional)](#migration-strategy-optional) - [Drawbacks (Optional)](#drawbacks-optional) - [Alternatives](#alternatives) - [Open Questions / Known Unknowns](#open-questions--known-unknowns) - [Implementation Stages (Optional)](#implementation-stages-optional) - [Success Metrics (Optional)](#success-metrics-optional) and tags, and then generate with mdtoc https://github.com/kubernetes-sigs/mdtoc. --> ## Summary ## Motivation ### Goals ### Non-Goals ### Linked Docs ## Proposal ### User Stories (Optional) #### Story 1 #### Story 2 ### Notes/Constraints/Caveats (Optional) ### Risks and Mitigations ## Design Details ### Test Plan ### Graduation Criteria (Optional) ### Migration Strategy (Optional) ## Drawbacks (Optional) ## Alternatives ## Open Questions / Known Unknowns ## Implementation Stages (Optional) ## Success Metrics (Optional) --- ### Proposal 002 Existing Vpc (`docs/proposal-002_existing-vpc.md`) # Design Proposal #002: Existing VPC > **STATUS**: This proposal is a _working draft_, it will get refined and augment as needed. > If any non-trivial changes are needed to functionality defined here, in particular the user > experience, those changes should be suggested via a PR to this proposal document. > Any other changes to the text of the proposal or technical corrections are also very welcome. Initial implementation of eksctl intentionally avoided having too many options for user to customise the VPC configuration. That was in oder to simplify how the tools works. Since then, many users asked for various features in relation to custom VPC. There a few specific use-cases: 1. co-location with kops [#50](https://github.com/eksctl-io/eksctl/issues/50) 2. set custom CIDR [#279](https://github.com/eksctl-io/eksctl/issues/279) 3. private/public subnets [#120](https://github.com/eksctl-io/eksctl/issues/120) 4. use any existing VPC [#42](https://github.com/eksctl-io/eksctl/issues/42) 5. use same subnet/AZ for a nodegroup [#232](https://github.com/eksctl-io/eksctl/issues/232) Out of the above, we already have support for 1 & 2 (as of 0.1.8) as well as 3 (as of 0.1.9). The main challenge with these customisations is user experience and how different flags interact. Ultimately, once Cluster API support is implemented (planned for 0.3.0), some of these flags will not be needed and there will be more fine-grain parameters available via Cluster API. This proposal sets out flags that should serve well for most common use-cases until 0.3.0. ## Flags defined in 0.1.9 These are fairly simple, and rather more user-friendly. These are assumed satisfactory for the purpose of this proposal. - `--vpc-from-kops-cluster=cluster-1.k8s.local`: import subnets from given kops cluster `cluster-1.k8s.local` - `--node-private-networking`: use private subnets for initial nodegroup - `--vpc-cidr=10.9.0.0/16`: use 10.9.0.0/16 as the global CIDR for the VPC ## Complete set of flags required for using existing VPC/subnets In order to use eksctl with any custom VPC, one has to provide subnet ID. VPC ID can be derived by making an API call. So the following flags will be needed: - `--vpc-private-subnets="subnet-1,subnet-2" --vpc-public-subnets="subnet-3,subnet-4"`: use given subnet IDs, detect VPC ID Also, the following combinations must be valid: - `--vpc-private-subnets="subnet-1,subnet-2"`: use only given private subnet IDs, detect VPC ID - `--vpc-public-subnets="subnet-3,subnet-4"`: use given public subnet IDs, detect VPC ID User will have to make sure IDs are correct as well as their private/public designations. For some use-cases, specifying VPC ID maybe preferred, and in that case the VPC maybe the default one (in which case it is ought to be possible to use default subnets or specific subnets). Hence the following combinations ought to be valid: - `--vpc-id=default`: use default VPC and all of its existing subnets - `--vpc-id=default --vpc-private-subnets="subnet-1,subnet-2" --vpc-public-subnets="subnet-3,subnet-4"`: use default VPC, and given subnet IDs - `--vpc-id="vpc-xxx"`: use given VPC, create dedicated subnets - `--vpc-id="vpc-xxx" --vpc-private-subnets="subnet-1,subnet-2" --vpc-public-subnets="subnet-3,subnet-4"`: use given VPC ID, and given subnets (same as if `--vpc-id` was unset, but less explicit) Needless to say that when none of `--vpc-*` flags given, eksctl will create a dedicated VPC and subnets (public as well as private). As more flags are being added, grouping them in `--help` output would be very helpful; especially because most of these flags are for advanced usage. ## Security Groups It must be noted that security groups are managed by eksctl only, as certain configuration is required to ensure cluster is is fully functional. Unlike with subnets and VPC, security groups can be update after they were created and any ingress and/or egress rules can be added or removed. If more advanced usage is commonly required, a separate design proposal will be required. It is also expected that advanced functionality will be available via Cluster API (which is planned for 0.3.0). ## Additional resources/commands It would be plausible to also provide utility for managing VPC, i.e.: - `eksctl vpc --name=`: manage VPC - `eksctl create cluster --vpc-name=`: create cluster in the given VPC This would allow users to create VPCs with recommended configurations without having to worry about the requirements, yet they will be able to do VPC resource management at a different level of access. It should be noted that similar functionality is likely to be required for IAM resources (subject to separate proposal). ## Notes on AWS CNI driver It can be safely assumed (based on simple tests) that the driver is meant to coexist very happily with anything you have in your VPC, from plain EC2 instances, through load balancers and gateways to Lambda functions. It is thereby ought to be possible to share a VPC between two or more cluster running separate instances of the driver. It is expected that driver will try and use what IP addresses are available, which is only a subject to IP address space usage, of course. Allocations and other properties of the driver can be monitored via CloudWatch and/or Prometheus. However, when very small subnets are used in a VPC, or there are too many resources using up the IP address space, one will ultimately run into issue sooner than later. If that's the case, VPC peering or some other alternative should be consider. Peering should already be possible, and custom CIDR flag is available as of 0.1.8. It must be noted, that there exists a plan to provide Weave Net as an overlay option. It maybe possible to use IPv6 also, but more experiments and research will need to be done (it not clear if EKS supports IPv6 properly yet). ## Requirements of custom VPC When user decides to use any of `--vpc-*` flags, it is up to them to ensure that: - suffiecent IP addresses are available - suffiencent number of subnets (minimum 2) - public/private subnets are configured correctly (i.e. routing and internet/NAT gateways are configured as intended) - tagging of subnets - `kubernetes.io/cluster/` tag set to either `shared` or `owned` - `kubernetes.io/role/internal-elb` tag set to `1` for private subnets - any other tags There maybe other requirements imposed by EKS or Kubernetes, and it is entirely up to the user to stay up-to-date on any requirements and/or recommendations, and find ways in which those work well with the practices at their own organisations and any requirements imposed by it. It is not our goal to keep this section of the proposal up to date. However, `eksctl create cluster` aims to always provide an up-to-date configuration with a dedicated VPC. --- ### Proposal 003 Extending Use Of Config File (`docs/proposal-003-extending-use-of-config-file.md`) # Design Proposal #003: Extending Use of `--config-file` > **STATUS**: This proposal is a _final_ state, and we expect minimal additional refinements. > If any non-trivial changes are needed to functionality defined here, in particular the user > experience, those changes should be suggested via a PR to this proposal document. > Any other changes to the text of the proposal or technical corrections are also very welcome. Support for config files had been added in eksctl 0.1.17 as an experimental alpha feature, it has become popular amongst users. Users currently can specify `ClusterConfig` object in JSON or YAML, which allows them to set more fields then they currently can with CLI flags. They can also specify several nodegroups, and create a cluster in one go. Besides many missing features in `v1alpha4` incarnation (where there is little of management functionality beyond initial cluster creation), one of commonly requested features is management of nodegroups through config files. This proposal aims to define how existing `ClusterConfig` object can be enhanced to support new type of objects that allow users to define and manage nodegroups via a separate step. Let's consider a simple config: ```YAML # cluster.yaml --- apiVersion: eksctl.io/v1alpha4 kind: ClusterConfig metadata: name: cluster-5 region: eu-north-1 nodeGroups: - name: ng1-public instanceType: m5.xlarge desiredCapacity: 4 - name: ng2-private instanceType: m5.large desiredCapacity: 10 privateNetworking: true ``` There are two nodegroups defined inline. When this file is passed to `eksctl create cluster --config-file=cluster.yaml` the cluster will be created with the nodegroups as defined. To create additional nodegroups for this cluster, we will enhance the `eksctl create nodegroup` cli to selectively, using glob patterns, pull nodegroups from this configuration file. This enables the entire cluster definition to live in a single file. For example, if we add 2 nodegroups to our original cluster config: ```YAML # cluster.yaml --- apiVersion: eksctl.io/v1alpha4 kind: ClusterConfig metadata: name: cluster-5 region: eu-north-1 nodeGroups: - name: ng1-public instanceType: m5.xlarge desiredCapacity: 4 - name: ng2-private instanceType: m5.large desiredCapacity: 10 privateNetworking: true - name: ng3-dev instanceType: m4.large desiredCapacity: 2 privateNetworking: true - name: ng2-test instanceType: m5.large desiredCapacity: 4 privateNetworking: true ``` `eksctl create nodegroup` will be updated to support glob pattern matching for nodegroup names contained within the cluster config. Therefore, a user can create these additional nodegroups using `eksctl create nodegroup --config-file=cluster.yaml --include="*dev,*test"`. eksctl will create two nodegroups in cluster named cluster-5. Here are some additional examples: ``` eksctl create ng --config-file=cluster.yaml ## all nodegroups will be created eksctl create ng --config-file=cluster.yaml --include=ng1-public ## only one nodegroup will be created eksctl create ng --config-file=cluster.yaml --include=ng1-public,ng2-private ## both nodegroups will be created ``` The `eksctl create cluster` cli will be updated to optionally ignore all nodegroups in the cluster config. https://github.com/eksctl-io/eksctl/issues/555 By using a single configuration file, users can keep their cluster definitions together and most importantly checked into source control. ## Possible future enhancements - Update `eksctl create cluster` to allow the user to create selected nodegroups defined in the cluster config > `eksctl create cluster --config-file=cluster.yaml --include='.*-private' ## only nodegroups matching glob pattern will be created` - Support storing nodegroups within their own configuration files. For example: ```YAML # extra-nodegroup.yaml --- apiVersion: eksctl.io/v1alpha4 kind: NodeGroupConfig metadata: cluster: cluster-5 region: eu-north-1 name: ng3-extra spec: instanceType: c4.xlarge minSize: 10 maxSize: 20 privateNetworking: true ``` Two ways that these config files can be used to create cluster: - `eksctl create cluster --config-file=cluster.yaml && eksctl create ng --config-file=extra-nodegroup.yaml` - `cat cluster.yaml extra-nodegroup.yaml | eksctl create cluster --config-file -` ### Questions - Is it appropriate to also allow referencing nodegroup that is defined externally within `ClusterConfig`? - Yes. That is the initial implementation we landed on. - Where in standalone `NodeGroup` config should labels live? (If we copy the same struct, we will get `spec.labels`, while it might make more sense to have them as `metadata.labels`). - We will address the definition of the actual nodegroup config file later. - How will `ClusterConfig` and `NodeGroup` compare when inspected in a running cluster? Namely, are all nodegroups supposed to be inline in `ClusterConfig` as well as represented as distinct object? (This is currently hard to say, but in the future it will become important that there is coherent representation). - For the initial version, all `nodeGroup` definitions will live in the `ClusterConfig` --- ### Proposal 004 Cost Estimation (`docs/proposal-004_cost-estimation.md`) > **STATUS**: This proposal is a _working draft_, it will get refined and augmented as needed. > If any non-trivial changes are needed to functionality defined here, in particular the user > experience, those changes should be suggested via a PR to this proposal document. > Any other changes to the text of the proposal or technical corrections are also very welcome. The main challenge of the cost estimation is to calculate the price induced by a `cluster create` or a `cluster scale` (V2) command. For the first version, we can focus on EC2 Instances type cost, EBS Volumes cost, EKS cluster cost per region. For future versions, we can introduce new features like: EC2 Data Transfer, Elastic Load balancer, etc. **API Used** Amazon provides AWS Price List API : https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/index.json. We can use it to get the current pricing of each service, in our case: EC2, EBS, EKS. ********************* **Parameters** `eksctl calculator` uses the same parameters as `eksctl create` and `eksctl scale` (V2) + specifics parameters. **Existing parameters** * __name__: ignored. No extra cost. * __tags__: ignored. No extra cost. * __zones__: ignored. No extra cost. * __version__: ignored. No extra cost. * __region__: Used to calculate the cost. The cost is per region * __nodes__: Used to calculate the cost unless nodes-min is provided * __nodes-min__: Used to calculate the cost. Minimum number of instances used * __nodes-max__: ignored. * __node-type__: Used to calculate the cost. The cost depends on the instance type * __node-volume-size__: Used to calculate the cost. The cost depends on the volume size * __node-volume-type__: Used to calculate the cost. The cost depends on the volume type * __max-pods-per-node__: ignored. No extra cost. * __node-ami__: ignored. (V2) * __node-ami-family__: ignored. (V2) * __ssh-access__: ignored * __ssh-public-key__: ignored * __node-private-networking__: ignored * __node-security-groups__: ignored. No extra cost. * __node-labels__: ignored. No extra cost. * __node-zones__: ignored. No extra cost. * __temp-node-role-policies__: ignored. No extra cost. * __temp-node-role-name__: ignored. No extra cost. * __asg-access__: ignored. No extra cost. * __external-dns-access__: ignored. No extra cost. * __full-ecr-access__: ignored. No extra cost. * __storage-class__: Used to calculate the cost. The cost depends on the storage class * __vpc-private-subnets__: ignored. (V2) * __vpc-public-subnets__: ignored. (V2) * __vpc-cidr__: ignored. (V2) * __vpc-from-kops-cluster__: ignored. (V2) **Specifics parameters** ***Usage*** Used to estimate the usage of the cluster and its nodes. By default, it's a full utilization: `--usage-type=utilization --usage-value=100`. * __usage-type__: String. Possible values: `demo` or `utilization` or `day` or `week` or `month`. * __usage-value__: Integer. Min value is `1` hour. Max value depends on `usage-type`: * `24` hours for `demo` and `day`, * `100` % for `utilization`, * `168` hours for `week`, * `732` hours for `month`. Examples: - Demo ` --usage-type=demo --usage-value=3` - % Utilized/Month ` --usage-type=utilization --usage-value=54` - Hours/Day ` --usage-type=day --usage-value=24` - Hours/Week ` --usage-type=week --usage-value=2` - Hours/Month ` --usage-type=month --usage-value=5` ***Free Tier*** * __free-tier__: yes/no (yes by default) ***Response example*** `eksctl calculator --name=cluster-1 --region=eu-west-1 --nodes=2` ``` [ℹ] using region eu-west-1 [ℹ] using "ami-0ce0ec06e682ee10e" for nodes [ℹ] 2 nodes m5.large [ℹ] EBS volumes 20go [ℹ] free tier "yes" [ℹ] usage type "utilization" [ℹ] usage value "100%" [✔] estimation (per month): - Amazon EC2 Service (EU (Ireland)): $ 158.86 -- Compute: $ 156.66 -- EBS Volumes: $ 2.20 - EKS Cluster: $ 146.4 total: $ 305.26 ``` ***Usecase (V2)*** Aims at giving an estimation of a `Common Customer Samples`: free-website, bigdata app, machine learning app, serverless, etc. It generates a yml file which contains a recommended configuration. For example: `eksctl calculator --usecase=free-websites --region=eu-west-1 --free-tier=yes` `eksctl calculator --usecase=addons --region=eu-web-1 --free-tier=yes --addons jenkins-x` --- ### Proposal 005 Upgrades (`docs/proposal-005-upgrades.md`) # Design Proposal #005: Upgrades > **STATUS**: This proposal is a _final_ state, and we expect minimal additional refinements. > If any non-trivial changes are needed to functionality defined here, in particular the user > experience, those changes should be suggested via a PR to this proposal document. > Any other changes to the text of the proposal or technical corrections are also very welcome. With `eksctl` users should be able to easily upgrade from one version of Kubernetes to another. Cluster upgrades are inherently a multi-step process, and can be fairly complex. It's important that users can drive upgrades at their own pace, if they must. There is a standard set of steps a user needs to take, most of which can be automated as a single command also. However, there may be additional manual steps with some versions, however most parts should be automated. When upgrading a cluster, one needs to call `eks.UpdateClusterVersion`. In some cases other changes to cluster stack may be required. After that they need to replace nodegroups one by one. ## Initial phase - provide command that checks cluster stack for upgradability - lets user update cluster stack to cater for any additional resources - allows to call `eks.UpdateClusterVersion` out-of-band and wait for completion - provide instruction on how to iteratively replace nodegroups - provide helper commands for upgrading add-ons ## Second phase (more automation) - use CloudFormation instead of calling `eks.UpdateClusterVersion` directly - provide automated command that replaces all nodegroups by copying configuration, or at least one command for each nodegroup ## CLI Design For initial phase: Update cluster version and append new resources to the cluster stack (if needed): ``` eksctl update cluster --name= ``` As this can have significant implications and some checks are appropriate before we try updating, `--dry-run` should be enabled by default. As control plane version can be incremented only one at a time, it would be a good idea to have `--version` flag. The default should be `--version=next`. It should be possible to jump many versions with `--version=`, and go for latest right away with `--version=latest`. Users will have to make sure that this doesn't take them too far ahead, so their nodegroup(s) don't drift too far behind. For each ``, create ``: ``` eksctl create ng --cluster= --name= eksctl delete ng --cluster= --name= ``` In this case new nodegroup will inherit cluster version automatically (`--version=auto`), however it's also possible to override that behavior. Alternatively, one can use `cluster.yaml` config file. They will need to update `metadata.version` field set in the config file, then run: ``` eksctl update cluster --config-file=cluster.yaml ``` > NOTE: to begin with this will only update cluster version and append any resources > to the stacks as needed And having added definition for new nodegroup(s), they can: ``` eksctl create ng --config-file=cluster.yaml --only= eksctl delete ng --cluster= --name= ``` In the future a flag should be added to `eksctl update cluster` that allows users to control what gets updated exactly, e.g. `--only=version,vpc,iam`. For second phase `eksctl update cluster` should update everything with (or without) the config file, including add-ons (although that may be more convenient to move into another command - TBD). ## Default Add-ons When upgrading from 1.10 to 1.11, following add-ons should be upgraded: - `kube-system:daemonset/kube-proxy` - only image tag has to change, it matches Kubernetes version - `kube-system:deployment/kube-dns` has to be replaced with `kube-system:deployment/coredns`, the manifest lives in [S3 bucket](https://amazon-eks.s3-us-west-2.amazonaws.com) - `kube-system:daemonset/aws-node` the manifest lives [in GitHub](https://github.com/aws/amazon-vpc-cni-k8s/tree/master/config) It should be possible to provide simple command under utils for each of these. ## Downgrades At the time of writing of this proposal version downgrades were not supported by EKS. It's trusted that upgrades will work as expected with EKS SLA. More specifically, an `eks.UpdateClusterVersion` call for cluster running 1.14+ trying to go back to 1.10, 1.11, 1.12 or 1.13 results in the following error: ``` An error occurred (InvalidParameterException) when calling the UpdateClusterVersion operation: unsupported Kubernetes version update from the current version, 1.12 to 1.10 ``` ## General Config Changes Eventually `eksctl update cluster` will evolve enough to support any meaningful changes to configuration, albeit within what is supported by EKS. At present it's only capable of appending new resources, such as shared SG. --- ### Proposal 006 Build Improvements (`docs/proposal-006-build-improvements.md`) # Design Proposal #006: Build Improvements ## Current State At the start of the project we've picked CircleCI as it appears to be a default choice at Weaveworks. The original solution involved: - running `docker build` to create a build image - use build image to run tests produce final image Publishing build and final images hasn't been automated, but some users requested it. When we switched [to modules](https://github.com/eksctl-io/eksctl/pull/917), we have increased the compexity of how build image worked. It's hard for a new person to understand it, e.g. for a new contributor. We currently build images in CI and run tests as part of that, we should only run tests as a primary task. Downside of current setup: - CircleCI requires delegation of access to GitHub - for releases we use a token - we don't have a way to automate workflows that require pushing to the repo - CircleCI cache - it is not very fast, it turns out pulling image with cached dependencies is as fast - it is very specific to CircleCI, not portable - we do not have a way to control it - the task of building images should not be mixed with the task of running test - images are not being automatically pushed to a registry - update of build image is manual ## Proposed Improvements ### Summary - streamline the CircleCI setup - enable GitHub Actions as an experiment Present approach: - restore cache from previous build - mount cache as docker volume - run tests and build new binary Proposed improvement: - pull docker image - run tests and build new binary So, `docker pull` for a large image that contains all cached objects takes as long (or less time) then using CircleCI cache feature. There are other benefits to this, as we can tell exactly what's being cached and can be reproduced any other CI environment. ### Details This proposal [#1200](https://github.com/eksctl-io/eksctl/pull/1200) aims to streamline the following aspects: - automate build image versioning through deterministic git object hashes (see `Makefile.docker`) - use build image as cache - initial cleanup of build scripts - separate makefiles - single static `Dockerfile` - stop using CircleCI cache and pull build image instead - switch to Docker executor using our build image in CircleCI - stop having to download Go toolchain - clear separation of concerns - creating build image vs running tests - enable GitHub Actions as an experiment, so we can evaluate it against CircleCI - based on initial tests (in GitHub Actions) - 40%-50% speed-up in on-commit execution (lint+test+build) ### Timing I have managed to bypass the need for upgrading CircleCI setup (at least for the time being). It's possible upgrading to native Docker executor will provide a speed-up, we might see shorter build queue times and it's possible that layer caching that they advertise will be beneficial, but they don't provide enough details for me to judge if it will. I'm going to evaluate the pricing and discuss it internally. Current timings on master branch (prior to this change) are between 10 & 11 minutes in total, e.g.: - restore cache - 02:00 - make image - 04:18 - save cache - 03:20 > NOTE: With current approach `make test` relies on test cache, which means tests > are a little faster, however that prevents flakes from surfacing as not all tests > run every time. _Proposed approach doesn't rely on test cache._ New timings on CircleCI are: - image pull - 01:17 - make test - 06:11 - total - 07:44 With `make build` added, here are two samples: - image pull - 01:19 / 01:20 - make test - 06:32 / 06:22 - make build - 02:08 / 02:05 - total - 10:16 / 10:04 With `make build` and `make test` combined in one step we save a couple of minutes: - image pull - 01:19 / 01:17 - make test & build - 06:30 / 06:39 - total - 08:13 / 08:12 Overall, there is a definite 2m speed-up. ## Simplifications We currently have rather complicated [CircleCI config](https://github.com/eksctl-io/eksctl/blob/950f9bc695234107725234b9e1a9c9d2ee54e51f/.circleci/config.yml#L3-L38), and [`Makefile`](https://github.com/eksctl-io/eksctl/blob/950f9bc695234107725234b9e1a9c9d2ee54e51f/Makefile#L188-L204). New configuration is much simpler, in [CircleCI config](https://github.com/eksctl-io/eksctl/blob/d3b6988562b14c9d91f4e1bf7dd2c086e06c2383/.circleci/config.yml#L3-L24) it comes down to running [`make test`](https://github.com/eksctl-io/eksctl/blob/1be26b314333467d1b67a44c77d1cd27460eaa70/Makefile#L68-L73) followed by `make build`. ## Further Improvements - [x] stop using `docker run` and `docker commit`, use another `Dockerfile` instead of `eksctl-image-builder.sh` - [x] push images to a registry (GitHub offering could be a good fit, otherwise consider ECR) - [ ] automate other workflows with bots and GitHub Actions (e.g. cherry-picking [#1284](https://github.com/eksctl-io/eksctl/issues/1284), AMI updates [#314](https://github.com/eksctl-io/eksctl/issues/314)) - [x] automate integration tests - [ ] allow running integration tests on PRs from contributors upon PR approval --- ### Proposal 007 Apply (`docs/proposal-007-apply.md`) # `eksctl apply` This proposal addresses a GitOps-compatible subcommand of `eksctl` called `apply`. **Status**: WIP * [Goals](#goals) * [Non-goals](#non-goals) * [Motivation](#motivation) * [Questions for discussion](#questions-for-discussion) * [Deletion of resources](#deletion-of-resources) * [Immutability](#immutability) * [Gradual support and config](#gradual-support-and-config) * [Proposal](#proposal) * [Experimental flag](#experimental-flag) * [Ownership](#ownership) * [eksctl-managed resources](#eksctl-managed-resources) * [Non-authoritative resources](#non-authoritative-resources) * [Non-taggable resources](#non-taggable-resources) * [eksctl owns everything](#eksctl-owns-everything) * [terraform-style state](#terraform-style-state) * [Ignore resource option](#ignore-resource-option) * [non-eksctl-managed resources](#non-eksctl-managed-resources) * [New config](#new-config) * [Goals/justification for config refactors](#goalsjustification-for-config-refactors) * [Open questions](#open-questions) ## Goals The purpose of `apply` is to subsume the various imperative commands `eksctl` provides now into one `apply` command that reconciles the actual state of an EKS cluster with the intended state as specified in the `eksctl` config file. It should be GitOpsable aka GitOps-compatible, i.e. usuable as part of a GitOps pipeline. ## Non-goals This proposal is only concerned with the `apply` subcommand. Existing behavior and commands are unaffected. Continuing to support them shouldn't complicate the implementation too much as they are more or less subsets of operations performed by `apply`. ## Motivation `eksctl` supplies users with imperative commands for clusters and cluster resources that allow them to take explicit actions, like `create` or `delete`. We also encourage the use of a config file in which the desired state of the cluster is described. Still, users are required to figure out which imperative steps to take to reconcile the cluster with the desired state. The missing capability is to give `eksctl` the ability to do this. Conceptually, `eksctl` would gather all of the information it can about the current state of the cluster in order to build a `ClusterConfig` describing the real-world cluster and then `diff` this config with the user-provided `ClusterConfig`. The `diff` is used to setup a plan of modification to the cluster and resources. ### Questions for discussion This proposal also opens the floor for discussion around non-straightforward questions around reconciliation. #### Deletion of resources When looking at a list of nodegroups in a cluster, it's not clear how to handle deletion. - Is the given list intended to be a complete list of nodegroups and no others should exist? Are they automatically deleted? - How do we handle non-eksctl managed resources? #### Immutability In order to change some properties, entire resource will have to be recreated. This should be explicit to the user. #### Gradual support and config Because it's impractical to introduce complete support for reconciliation at once, we need to instead gradually support properties and resources of a cluster. This leads to the question of whether the current `ClusterConfig` is a good structure for a config meant for reconciliation. ## Proposal Introduce a subcommand `apply` which has only a config file argument. `apply` would reconcile desired with real-world state [as described in the motivation](#motivation). The exact details per resource will potentially differ depending on the resources and won't be enumerated here. 1. `apply` is put behind the explicit "experimental" flag (cf `enable profile/repo`) 2. Take full ownership of `eksctl`-managed resources (i.e. delete missing resources by default) 3. Options for non-`eksctl`-managed resources as well as resources where ownership is unknowable are covered below 4. Changing immutable fields through recreation is the default 5. We gradually expand `apply` support to different parts of the cluster 6. As we expand `apply` support, we reevaluate and update the config structure as necessary in a new API version. Some further discussion/justification follows. ### Experimental flag We make it explicit that `apply` is experimental, very open to feedback and subject to change. ### Ownership Ownership comes into play most importantly with deletion. #### `eksctl`-managed resources Given the config: ``` nodeGroups: - name: ng-2 desiredCapacity: 1 ``` we discover an existing nodegroup `ng-1` with the `eksctl.io/owned: true` tag (for example), do we delete it? The proposed answer is yes, it should be deleted. With `apply` we assume that all previous changes initiated by `eksctl` were initiated through the same config file/reconciliation process and that this nodegroup must have been deleted from this file with the intention of deleting it from the cluster. ##### Non-authoritative resources There is precedent for adding the ability to specify resources in a non-authoritative list. For example, the [`google` terraform provider supports IAM with 3 different levels of authoritativeness]( https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam#google_storage_bucket_iam_policy#iam-policy-for-cloud-storage-bucket). Initial `apply` support will focus on authoritative resource specification. #### Non-taggable resources Some resources may not be taggable as owned by `eksctl`. If there weren't a `tags` field on addons, for example. `eksctl`-created addons would then be indistinguishable from addons created in the AWS console. There are at least 2 ways to deal with this case: ##### `eksctl` owns everything With this option, we assume that `eksctl` owns the set of such resources and delete any not listed in the config. ##### `terraform`-style state Storing some kind of state/config "somewhere" is another option and has been discussed in https://github.com/eksctl-io/eksctl/issues/642  #### Ignore resource option An `ignore` flag (cf `terraform`'s `prevent_destroy`) in the schema for all objects could be added to tell `eksctl` about such objects and prevent deletions and potentially skip reconciliation. The user would prefix their list: ``` nodeGroups: - name: ng-1 ignore: true - name: ng-2 desiredCapacity: 1 ``` #### non-`eksctl`-managed resources The behavior for resources created outside of `eksctl` could also be toggleable by settings in the config/flags on `apply`. ### New config Proposed is creating a new API version, starting with `v1alpha6`, where, as we add `apply` support for cluster/nodegroup properties, we allow for aggressive reevaluation of the existing config structure that would hold these newly supported properties/aspects of the cluster/nodegroup. If we decide, _for example_, to add support for reconciling tags, we might decide it fits better as a top level key. We would then, with `v1alpha6`, remove `tags` under `metadata` and make it a top level field. It should be possible to automatically rewrite the previous version into the current version. #### Goals/justification for config refactors - Ensure that the config makes sense in a world centered around reconciliation. - Auto upgrade makes it easier for users to use an existing config - **"Parse, don't validate"**: the code that uses the config should be insulated from how we serialize/deserialize the config. ## Open questions - What, if anything, about the VPC is owned by `eksctl` and under what circumstances? - What does it mean if the user specifies a VPC not created by `eksctl` but `eksctl` finds a discrepancy between the actual state and the config - `terraform`'s answer here is explicitness: `resource` vs `data`. --- ### Proposal 007 Instance Selector Dry Run (`docs/proposal-007-instance-selector-dry-run.md`) # Support for EC2 Instance Selector for Nodegroups and Dry Run ## Table of Contents - [Summary](#summary) - [Motivation](#motivation) - [Proposal](#proposal) - [Dry Run](#dry-run) - [Design](#design) - [eksctl create cluster](#eksctl-create-cluster) - [eksctl create nodegroup](#eksctl-create-nodegroup) - [API Changes](#api-changes) - [Caveats](#caveats) ## Summary This proposal adds support for the EC2 instance selector for both managed and self-managed nodegroups. ## Motivation eksctl supports specifying multiple instance types for managed and self-managed nodegroups, but with over 270 EC2 instance types, users have to spend time figuring out which instance types would be well suited for their nodegroup. It's even harder when using Spot instances because you need to choose a set of instances that works together well with the Cluster Autoscaler. The [EC2 Instance Selector](https://github.com/aws/amazon-ec2-instance-selector) tool addresses this problem by generating a list of instance types based on resource criteria such as vCPUs, memory, etc. This proposal aims to integrate the EC2 instance selector with eksctl to enable creating nodegroups with multiple instance types by passing the resource criteria. ## Proposal This design proposes adding a new field `instanceSelector` to both managed and self-managed nodegroups, that accepts the instance selector resource criteria, and creates nodegroups using instance types matching the criteria. ```shell $ cat cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: cluster region: us-west-2 managedNodeGroups: - name: linux instanceSelector: vCPUs: 2 memory: 4 gpus: 2 nodeGroups: - name: windows amiFamily: WindowsServer2019CoreContainer instanceSelector: vCPUs: 2 memory: 4 gpus: 2 instancesDistribution: maxPrice: 0.017 onDemandBaseCapacity: 0 onDemandPercentageAboveBaseCapacity: 50 spotInstancePools: 2 ``` In the above example, `eksctl create cluster` and `eksctl create nodegroup` will use the supplied instance selector resource criteria and create nodegroups by setting `managedNodeGroups[*].instanceTypes` and `nodeGroups[*].instancesDistribution.instanceTypes` to the instance types matching the resource criteria. The instance selector feature will also be supported as CLI flags: ```shell $ eksctl create cluster --name development --managed --instance-selector-vcpus=2 --instance-selector-memory=4 ``` ```shell $ eksctl create nodegroup --cluster development --managed --instance-selector-vcpus=1 --instance-selector-memory=2 ``` ### Dry Run While the instance selector feature helps select a set of instances matching the resource criteria, some users may want to inspect and change the instances selected by the instance selector before proceeding to creating a nodegroup, and also to document the instance types being used for their nodegroups in the ClusterConfig file. For this reason, this design also proposes adding a new general-purpose `--dry-run` option to `eksctl create cluster` and `eksctl create nodegroup` that skips cluster and nodegroup creation and instead outputs a ClusterConfig file containing the default values set by eksctl and representing the supplied CLI options. This functionality extends to the instance selector integration where eksctl will set the instance types for a nodegroup when the instance selector options are supplied. This gives users the opportunity to inspect and, if required, change the instance types before proceeding with cluster and nodegroup creation. Users can omit the `--dry-run` option if they do not care about the instance types selected by eksctl. There are certain one-off options that cannot be represented in the ClusterConfig file, e.g., `--install-vpc-controllers`. Users would expect `eksctl create cluster -- --dry-run > config.yaml` followed by `eksctl create cluster -f config.yaml` to be equivalent to running the first command without `--dry-run`. eksctl would therefore disallow passing options that cannot be represented in the config file when `--dry-run` is passed. ## Design Three new options `--instance-selector-vcpus`, `--instance-selector-memory` and `--instance-selector-gpus` will be added to `eksctl create cluster` and `eksctl create nodegroup`. These options will also be supported in the ClusterConfig file for both managed and self-managed nodegroups with the following schema: ```yaml instanceSelector: vCPUs: memory: gpus: ``` A `--dry-run` flag will also be added to generate a ClusterConfig from the supplied CLI options. At launch only the `vCPUs`, `memory` and `gpus` options will be supported, additional options may be added later based on community feedback. **Note:** Because fields are case-sensitive, `vCPUs` may be difficult to get right, especially for users unfamiliar with Go naming conventions. However, `vCPUs` with the same case is standard enough that it's widely used by all major cloud providers. `vcpus` is easier to remember but it's inconsistent with the other field names. ### eksctl create cluster When `eksctl create cluster` is called with the instance selector options and `--dry-run`, eksctl will output a ClusterConfig file containing a nodegroup representing the CLI options and the instance types set to the instances matched by the instance selector resource criteria. ```shell $ eksctl create cluster --name development --managed --instance-selector-vcpus=2 --instance-selector-memory=4 --dry-run apiVersion: eksctl.io/v1alpha5 availabilityZones: - us-west-2b - us-west-2d - us-west-2a cloudWatch: clusterLogging: {} iam: vpcResourceControllerPolicy: true withOIDC: false kind: ClusterConfig metadata: name: development region: us-west-2 version: "1.18" managedNodeGroups: - amiFamily: AmazonLinux2 desiredCapacity: 2 disableIMDSv1: false disablePodIMDS: false iam: withAddonPolicies: albIngress: false appMesh: false appMeshPreview: false autoScaler: false certManager: false cloudWatch: false ebs: false efs: false externalDNS: false fsx: false imageBuilder: false xRay: false instanceTypes: ["c5.large", "c5a.large", "c5ad.large", "c5d.large", "t2.medium", "t3.medium", "t3a.medium"] instanceSelector: vCPUs: 2 memory: 4 labels: alpha.eksctl.io/cluster-name: unique-creature-1614695544 alpha.eksctl.io/nodegroup-name: ng-38105fc5 maxSize: 2 minSize: 2 name: privateNetworking: false securityGroups: withLocal: null withShared: null ssh: allow: false enableSsm: false tags: alpha.eksctl.io/nodegroup-name: ng-38105fc5 alpha.eksctl.io/nodegroup-type: managed volumeIOPS: 3000 volumeSize: 80 volumeThroughput: 125 volumeType: gp3 ``` **Note:** For brevity, the rest of the document will omit fields not relevant to the instance selector. The `instanceSelector` field representing the CLI options will also be added to the ClusterConfig file for visibility and documentation purposes. When `--dry-run` is omitted, this field will be ignored and the `instanceTypes` field will be used, otherwise any changes to `instanceTypes` would get overridden by eksctl. For self-managed nodegroups, `instancesDistribution.instanceTypes` will be set: ```shell $ eksctl create cluster --name development --instance-selector-vcpus=2 --instance-selector-memory=4 --dry-run apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: development region: us-west-2 nodeGroups: - name: instancesDistribution: instanceTypes: ["c5.large", "c5a.large", "c5ad.large", "c5d.large", "t2.medium", "t3.medium", "t3a.medium"] instanceSelector: vCPUs: 2 memory: 4 ``` If `--dry-run` is omitted, eksctl will proceed to creating the cluster and nodegroup with the instance types matching the instance selector resource criteria. --- When a ClusterConfig file is passed with `--dry-run`, eksctl will output a ClusterConfig file containing the same set of nodegroups after expanding each nodegroup's instance selector resource criteria. ```shell $ cat cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: cluster region: us-west-2 managedNodeGroups: - name: linux instanceSelector: vCPUs: 2 memory: 4 nodeGroups: - name: windows amiFamily: WindowsServer2019CoreContainer instanceSelector: vCPUs: 2 memory: 4 instancesDistribution: maxPrice: 0.017 onDemandBaseCapacity: 0 onDemandPercentageAboveBaseCapacity: 50 spotInstancePools: 2 ``` ```shell $ eksctl create cluster -f cluster.yaml --dry-run apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: cluster region: us-west-2 managedNodeGroups: - name: linux instanceTypes: ["c3.large", "c4.large", "c5.large", "c5d.large", "c5n.large", "c5a.large"] instanceSelector: vCPUs: 2 memory: 4 nodeGroups: - name: windows amiFamily: WindowsServer2019CoreContainer instanceSelector: vCPUs: 1 memory: 2 instancesDistribution: maxPrice: 0.017 instanceTypes: ["c3.large", "c4.large", "c5.large", "c5d.large", "c5n.large", "c5a.large"] onDemandBaseCapacity: 0 onDemandPercentageAboveBaseCapacity: 50 spotInstancePools: 2 ``` **Note:** It is an error to pass both a ClusterConfig file (`-f`) and the instance selector CLI options. When using a config file, the instance selector options must be set in the `instanceSelector` field. ### eksctl create nodegroup `eksctl create nodegroup` is different from `eksctl create cluster` in that it operates on an existing cluster. When a ClusterConfig file is not passed, eksctl will not attempt to reconcile the cluster state and will instead output a ClusterConfig file containing a single nodegroup representing the CLI options. The cluster may have more than one nodegroup, so the generated ClusterConfig file will not represent the actual state of the cluster. It is the user's responsibility to copy and add the generated nodegroup config to their existing ClusterConfig file. ```shell $ eksctl create nodegroup --cluster=dev --managed --spot --instance-selector-vcpus=2 --instance-selector-memory=4 --dry-run apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: dev region: us-west-2 managedNodeGroups: - name: spot: true instanceTypes: ["c3.large", "c4.large", "c5.large", "c5d.large", "c5n.large", "c5a.large"] instanceSelector: vCPUs: 2 memory: 4 ``` By outputting a well-formed ClusterConfig file instead of just the nodegroup section, users can subsequently run `eksctl create nodegroup -f generated-config.yaml` to create a new nodegroup. **Note:** If a ClusterConfig file exists, it's recommended to add your nodegroups to the file and pass it to `eksctl create nodegroup`. --- When a ClusterConfig file is passed with `--dry-run`, eksctl will output the same set of nodegroups after expanding each nodegroup's instance selector resource criteria. ```shell $ cat cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: dev region: us-west-2 managedNodeGroups: - name: m1 instanceSelector: vCPUs: 2 memory: 4 - name: m2 desiredCapacity: 5 $ eksctl create nodegroup -f cluster.yaml --dry-run apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: dev region: us-west-2 managedNodeGroups: - name: m1 instanceTypes: ["c3.large", "c4.large", "c5.large", "c5d.large", "c5n.large", "c5a.large"] instanceSelector: vCPUs: 2 memory: 4 - name: m2 desiredCapacity: 5 ``` This consistent behaviour for both the config file version and the non-config file version ensures that subsequently running `eksctl create nodegroup -f cluster.yaml` will be identical to running the initial command without `--dry-run`. When the `--dry-run` option is omitted in either case, eksctl will proceed to nodegroup creation after applying the instance selector criteria. ### API Changes A new type `InstanceSelector` will be added to represent the instance selector options: ```go type InstanceSelector struct { VCPUs int `json:"vCPUs"` Memory int `json:"memory"` GPUs int `json:"gpus"` } ``` A new field `InstanceSelector` will be added to `NodeGroupBase`, which will make the field available for both `NodeGroup` and `ManagedNodeGroup` as it's embedded: ```go type NodeGroupBase struct { InstanceSelector *InstanceSelector `json:"instanceSelector"` } ``` ### Caveats When a ClusterConfig file is passed with `--dry-run`, the formatting and order of fields in the YAML output may not match the supplied file's formatting. --- ### Proposal 008 Flux2 (`docs/proposal-008-flux2.md`) # Flux v2 integration plan: next steps ## Authors Claudia Beresford (@Callisto13) ## Status Approved. ## Table of Contents - [Summary](#summary) - [Motivation](#motivation) - [Goals](#goals) - [Non-Goals](#non-goals) - [Linked Docs](#linked-docs) - [Details](#details) - [Removals](#removals) - [Git provider](#git-provider) - [Args](#args) - ["Dry run"](#dry-run) - [Documentation improvements](#documentation-improvements) - [Notes/Constraints/Caveats](#notesconstraintscaveats) - [Teams](#teams) - [Future work](#future-work) - [Implementation Stages](#implementation-stages) - [Remove all the things](#remove-all-the-things) - [Docs](#docs) - [Recommended Flux version](#recommended-flux-version) - ["Dry run"](#dry-run-1) - [Test coverage](#test-coverage) - [API spec](#api-spec) - [Alternatives](#alternatives) - [List of strings](#list-of-strings) - [Actual config](#actual-config) ## Summary We implemented some very basic Flux 2 integration with release `0.38.0`. This implementation is marked as experimental, and allows users to set the most common flags for bootstrapping a cluster with `github` or `gitlab`. The value in integrating Flux 2 with `eksctl` is simply to give users a way of: a) setting up their new cluster with flux immediately after it is created, as part of the same single command b) letting users save and run their flux bootstrap args via a config file, rather than a list of flags We implemented the experimental integration with eksctl config values only, and no eksctl flags, since that would defeat the purpose of value `b`: we intend to keep it this way. Exposing the rest of the options available to `flux bootstrap` is the next step. Since Flux development moves quite fast, we want to do this in a way that we don't have to continuously keep in step; we want to give users a way to provide arbitrary args. Balancing that we also don't want to lose value `b`; we want to ensure users have a nice config file UX, and a single string of flags goes against that. We have had a few issues opened since then from people asking us to expose more options (which is nice because people are using it), and we don't want to be in a position where we get an issues like that opened frequently. The suggestion in this proposal is that we: - Remove all config options exposed in the initial Flux integration - Implement a `map[string]string` through which users can pass flags to Flux For a more rigid way of doing this configuration, see [Alternatives](#alternatives) below. ## Motivation The overall motivation of this proposal is to solve 2 problems: - Users want to bootstrap their clusters with flux via eksctl's configuration - The configuration needs to be flexible enough that users do not have to request the exposure of new flags/features as Flux develops The unofficial motivations of this proposal are: - Sketch out a plan to deliver in stages (multiple PRs) so that Claudia does not open yet another 50 page diff - Have everything written down before Claudia goes on holiday for 3 weeks ### Goals - Users can bootstrap to any of the 3 supported git providers (github, gitlab, git) - Users can benefit from a standard set of configuration options - Users can provider arbitrary args as they need ### Non-Goals - Air-gapped bootstrapping - Terraform bootstrapping ### Linked Docs [Original PR](https://github.com/eksctl-io/eksctl/pull/3066). [Current eksctl docs](https://eksctl.io/usage/gitops/#experimental-installing-flux-v2-gitops-toolkit). [Current Flux api object in eksctl](https://github.com/eksctl-io/eksctl/blob/ab702daa671a87dc926b70348481ce336638f064/pkg/apis/eksctl.io/v1alpha5/types.go#L901-L937). [Expansion issue](https://github.com/eksctl-io/eksctl/issues/3238). ## Details ### Removals Pretty much everything which we exposed in the initial implementation will disappear: ```diff gitops: flux: gitProvider: github - owner: dr-who - repository: my-cluster-gitops - personal: true - path: "clusters/cluster-27" - kubeconfig: /foo/bar - branch: main - namespace: "flux-system" ``` The addition of an arbitrary list of flags and values means users can immediately use any Flux feature without waiting for us to expose it. ### Git provider By "provider" I mean which git thing you want. Flux lets you bootstrap to one of `github`, `gitlab` and `git` a generic git server. This is the one explicit bit of config we will leave, simply because it is not a flag, it is a subcommand and therefore the ordering would matter if we left it to the map. If is is an explicit config option users do not have to care about this. ```diff gitops: flux: gitProvider: github|gitlab|git ``` ### Args All other Flux bootstrap configuration will be provided via an arbitrary `map[string]string`. (I have called this key `flags` but open to whatever.) ```diff gitops: flux: gitProvider: github|gitlab|git + flags: + branch: main + namespace: default + components: "helm-controller,source-controller" + personal: true ``` ### "Dry run" One thing we lose by having these arbitrary fields is the extra validation: are users setting the correct flags for their chosen git provider? Of course Flux will complain if you get the flags wrong, and we will be directing users to the Flux docs so that they know what options are available to them, but I can see people tripping on this and it is really annoying to get to the end of a 25 min cluster create, only to have your `flux bootstrap` fail. Ideally we could "validate" the flags are at least applicable to the bootstrap command before starting the create. Flux doesn't have a dry-run bootstrap option (AFAIK), so we cannot call the command without actually running it. But we could hack it by maybe appending a garbage flag to the end of the list we pass in. Flux will fail on the first flag it does not recognise, so if it errors on anything other than our nonsense flag, then the validation is failed. I am not 100% sold on this, and we don't need it right away, but I think it may be neat to have something like it. ### Documentation improvements This should be improved in general and should redirect as much as possible to the Flux docs. We should be very clear on how much `eksctl` is doing here, and how users can pass the correct options through to Flux. The Flux 2 implementation currently shares a page with the Flux v1 eksctl.io docs, they should be split up. I don't think we are even clear about what Flux commands we run. ## Notes/Constraints/Caveats ### Teams One important caveat with the `map[string]string` approach is how Flux implements adding teams for github and gitlab providers. Their current implementation uses `StringArrayVar` which means flags which look like `--team one --team two` and therefore a yaml which looks like: ```yaml ... flags: team: one team: two ``` This obviously will not work. We intend to ask about changing that flag to a `StringSliceVar` which would support both `--team one --team two` and `--team one,two` formats (and hence a `team: "one,two"` yaml), but if this is not possible then we would have to change `flags` to be a `[]string`. Update: a PR has been opened to do this: [fluxcd/flux2#1390](https://github.com/fluxcd/flux2/pull/1390). Even if we were able to change that flag in Flux cli, we would have to hope that they didn't use `StringArrayVar` for bootstrap flags again (at least for a year, see future work [below](#future-work)). See more on alternatives [below](#alternatives). ### Future work It is worth noting that changes the future will result in whatever we do now being removed, maybe a year from now. ## Implementation Stages _These are broken down with the intention of being done in separate PRs. I will create separate tickets._ ### Remove all the things Remove all current config opts which are not `gitProvider`. Add a `flags` (or whatever) key which is a `map[string]string`. Ensure documentation is updated and that the API change is publicised. For bonus points, if they try to use the old structure (or any thing which we do not recognise), explain things have changed in logs/error and point them to the PR/docs. If Flux fails after cluster create, suggest running `eksctl enable flux` once config is resolved. Don't forget to update the example yaml with something thorough. ### Docs - Document required Env vars and Flux binary - Make sure to link to https://fluxcd.io/ everywhere possible - Document discovering all options by running `flux bootstrap --help` - Make it clear how flags can be set in eksctl config - Move Flux 2 docs away from general gitops page in eksctl.io (dedicated page) - Mark old Flux 1 docs as deprecated - Note that quickstartprofiles will not be implemented for Flux 2 ### Recommended Flux version Once [our PR](https://github.com/fluxcd/flux2/pull/1390) to Flux cli is merged and released, we should add a check in `eksctl` for that version or above. This version is not required, since it only effects those using the `team` flag for more than one team. We should log a warning that their Flux version is `foo` and in order to use `--team` they will need at least `bar`. We should not fail. (Or should we? Maybe setting a min version is just simpler :shrug:? ### "Dry run" If we think this is a good idea/worthwhile, validate that flags are applicable and whatever required vars `GITX_TOKEN` etc are set by calling Flux with a dummy command. ## Test coverage I don't think we need to get too into testing, given that we are just "passing through" and don't want to end up testing Flux itself. The current integration test can be altered pass in arbitrary args, and the units can handle testing that correct opts end up in the `exec`. This should be sufficient. ## API spec ```go // GitOps groups all configuration options related to enabling GitOps Toolkit on a // cluster and linking it to a Git repository. // Note: this will replace the older Git types type GitOps struct { // [Enable Toolkit](/usage/gitops/#experimental-installing-gitops-toolkit-flux-v2) Flux *Flux `json:"flux,omitempty"` } // Flux groups all configuration options related to a Git repository used for // GitOps Toolkit (Flux v2). type Flux struct { // The repository hosting service. Can be either Github, Gitlab or Git. Required. GitProvider string `json:"gitProvider,omitempty"` // Arbitrary map of key value pairs which will be passed to Flux bootstrap as flags Flags map[string]string `json:"namespace,omitempty"` } ``` ## Alternatives ### List of strings The main alternative for if we cannot cover all Flux flags in a map. ```diff gitops: flux: gitProvider: github + args: + - "--flag-one=value" + - "--flag-two value" + - "--bool-flag" ``` ### Actual config In this option we have a mix of 1:1 config:flag mappings and an arbitrary option for all non-explicit config. - All provider flags - Some common/global flags - Arbitrary list of string ```yaml gitops: flux: # gitlab, github and git would not be specified all at once ofc, but you get the idea gitlab: branch: main namespace: "flux-system" components: - "source-controller" - "kustomise-controller" fluxExtraArgs: - "--flag-one=value" - "--flag-two value" - "--bool-flag" interval: 1m0s path: "clusters/cluster-27" owner: dr-who repository: my-cluster-gitops personal: true hostname: "gh-enterprise.com" interval: 1m0s private: true readWriteKey: true reconcile: true teams: - "team1" - "team2" github: ... would be same as gitlag git: branch: main namespace: "flux-system" components: - "source-controller" - "kustomise-controller" fluxExtraArgs: - "--flag-one=value" - "--flag-two value" - "--bool-flag" interval: 1m0s path: "clusters/cluster-27" username: "dr-who" passwordFile: /basic/auth/pass url: https://// ``` --- ### Guidelines Design Conventions (`docs/guidelines/design-conventions.md`) # Design conventions General conventions followed when developing in eksctl: - general - never use `panic` in your code, if a library call panics - think twice - never prompt interactively, assume automation as prime use-case - never animate output - avoid foreign output message - log message style - keep it neutral, don't address the user as a human (i.e. instead of "waiting for X, please be patient" just say "waiting for X") - don't suggest how long something is going to take, it's always going to be wrong - must always begin a message with lower-case letter, it's not a sentence - if the first word is a name, it's fine to use capital letter e.g. '[i] Kubernetes welcomes you' - do not add spaces or tabs before the first character - use '%q' for names of resources - only reference flag names or API fields when absolutely certain of the use-case - use single quotes when suggesting use to run a command - only log these from top-level command handlers, not from a library - only suggest simple commands, no shell one-liners - use long form of the flags in suggestions - if referring to eksctl, try to make the command as full as possible with cluster name and region - do not use any punctuation other than column, dash or comma, and only when absolutely essential - i.e. no dots, ellipsis or semicolons - do not write long log messages, try to break up into two if extra information is needed - use `fmt.Errorf` with `%w` to wrap errors - avoid wrapping errors that are already meaningful - we use Kris Nova's logger, this may change in the future and we should probably abstract it - we use Cobra (for better or worse) - we put all shared things in `pkg/ctl/cmdutils`, familiarise yourself with it - `cmdutils.Cmd` wraps most of the things that are shared between various commands - we have customisation on top of Cobra that allow us to group flags in usage summary - this package is all about CLI use-cases, it shouldn't be consumed by libraries - don't use `cobra.MarkFlagRequired` - `fs.MarkHidden` is evil, but sometimes we have to use it - CLI - commands follows a similar convention as `kubectl`: `eksctl ` e.g.: `eksctl create nodegroup`, `eksctl enable profile` - eksctl does not use arguments except when used for the name of the resource (e.g.: `eksctl create nodegroup ng-1`). This is what we call `nameArg` in the code. The reason for this is that users get very easily confused when arguments are used and what their meaning is, and relative position with respect to flags. The name of the resource will also have the option to pass it as part of the flag called `--name` (e.g.: `eksctl create nodegroup --name ng-1`). - when the main resource is not the cluster, the cluster will be set in the `--cluster` flag (e.g.: `eksctl create nodegroup --cluster clus1 ng-1`) - for boolean flags, use `*bool` so that eksctl can know if a flag was explicitly enabled or disabled by the user - add unit tests for the flags and config file parsing. These are tedious to manual test. - keep the number of CLI flags small. Not all options should have flags, but rather have a way to use them through the config file. In general, basic use cases can be covered by flags but more advanced options should only live in the config file. --- ### Release Notes 0.1.31 (`docs/release_notes/0.1.31.md`) # Release 0.1.31 ## Improvements - upgrade `aws-iam-authenticator` to version 0.4.0 (#754): + old kubeconfig files may need updating - you may find `eksctl utils write-kubeconfig` handy + macOS: `homebrew/core/aws-iam-authenticator` is used from now on - API version v1alpha5 + Move SSH parameters under `ssh`; public keys can now be specified by local file, uploaded key in EC2 or by the key content (see `examples/07-ssh-keys.yaml` for detailed usage) (#673, #713) + rename `preBootstrapCommand` option in the config file has been renamed to `preBootstrapCommands` (#726) + `--dry-run=false` CLI argument has been renamed to `--approve` (#727) - add support to attach target groups to node groups via ARNs (#746) - add support for using config files in the drain command (#741) - add support for using config files in the command for deleting node groups (#714) - start using new `alpha.eksctl.io/` tag prefix (instead of `eksctl.cluster.k8s.io/v1alpha1/`) (#728) - ensure `ami: auto` (`--node-ami=auto`) searches official Amazon and Ubuntu AMIs by owner ID, to avoid security issue (#733) --- ### Release Notes 0.1.32 (`docs/release_notes/0.1.32.md`) # Release 0.1.32 ## Improvements - add an option to automatically allocate IPv6 blocks to all subnets (`vpc: {autoAllocateIPv6: true}`) [#776] - dynamically lookup `RootDeviceName` based on the AMI [#779] - upgrade nodegroup ASG to use Launch Template (instead of Lauch Configuration) [#743] --- ### Release Notes 0.1.33 (`docs/release_notes/0.1.33.md`) # Release 0.1.33 ## Improvements - add AWS X-Ray support through IAM policy to allow outbound write access to AWS X-Ray (#805) - add support to autoscale a nodegroup to 0 with Cluster Autoscaler (#816) - add missing Ubuntu AMIs for 1.10 and 1.11 (#823) ## Bug fixes - fix EFS IAM policy (#829) --- ### Release Notes 0.1.34 (`docs/release_notes/0.1.34.md`) # Release 0.1.34 ## Improvements - add Cloudwatch support through an IAM policy (#831) - add support for mixed nodegroups with Spot instances (#794) --- ### Release Notes 0.1.35 (`docs/release_notes/0.1.35.md`) # Release 0.1.35 ## Improvements - add commands for manipulating IAM identity mappings (#814) ## Bug fixes - fix bug in AMI selection for mixed instances nodegroups (#844 and #845) --- ### Release Notes 0.1.36 (`docs/release_notes/0.1.36.md`) # Release 0.1.36 ## Improvements - support for Kubernetes 1.13 (not default yet, use `--version=1.13`) - add Canada Central region (`ca-central-1`) - add support for encrypted root EBS volumes based on AMI (#889) - add `certManager` IAM add-on policy (#885) - add `--config-file` flag in all `iamidentitymapping` commands (#868) - make stack troubleshooter output less verbose (#865) ## Bug fixes and code improvements - fix obscure error message when cluster failed to create (#866) - code refactoring and clean up (#859, #878) --- ### Release Notes 0.1.37 (`docs/release_notes/0.1.37.md`) # Release 0.1.37 ## Bug fixes - List clusters in all regions despite regional errors (#908) - Fix an issue with integration tests (#901) --- ### Release Notes 0.1.38 (`docs/release_notes/0.1.38.md`) # Release 0.1.38 ## Improvements - Support [extra parameters in kubelet configuration](https://github.com/eksctl-io/eksctl/blob/master/userdocs/src/usage/customizing-the-kubelet.md) (#795) - Include cluster name and region in output when using a config file (#912) --- ### Release Notes 0.1.39 (`docs/release_notes/0.1.39.md`) # Release 0.1.39 ## Features - New [website](https://eksctl.io) design - Documentation reworked and restructured - Kubernetes 1.13 is now default for new clusters [#971] - New modes for NAT Gateway topology (see [`examples/09-nat-gateways.yaml`](https://github.com/eksctl-io/eksctl/blob/master/examples/09-nat-gateways.yaml)) [#694, #392] - More options for EBS root volume encryption (see [`examples/10-encrypted-volumes.yaml`](https://github.com/eksctl-io/eksctl/blob/master/examples/10-encrypted-volumes.yaml) [#952] - Use latest AMIs [#934, #982] ## Bug fixes and code quality improvements - Speed-up CI and use Go modules - Enable release candidate workflow - Use AWS API pagination consistently ## Contributions The Weaveworks team would like to thank our new contributor [**@cristian-radu**](https://github.com/cristian-radu) for the work on NAT gateway, and returning contributor [**@adamjohnson01**](https://github.com/adamjohnson01) for the work on EBS root volume encryption! --- ### Release Notes 0.1.40 (`docs/release_notes/0.1.40.md`) # Release 0.1.40 ## Bug fixes - Fix panic when using a config file without `vpc.nat` set (#1003) --- ### Release Notes 0.10.0 (`docs/release_notes/0.10.0.md`) # Release 0.10.0 ## Features - support for EKS Managed Nodegroups (#1566) ## Improvements - add NodeInstanceRoleARN to NodeGroupSummary (#1500) ## Bug Fixes - fix enabling SSH when allow is not set (#1545) --- ### Release Notes 0.10.1 (`docs/release_notes/0.10.1.md`) # Release 0.10.1 ## Bug Fixes - fix managed nodes CloudFormation template (#1573) --- ### Release Notes 0.10.2 (`docs/release_notes/0.10.2.md`) # Release 0.10.2 ## Bug Fixes - fix issue with creating LoadBalancer services (#1585) - fix `update cluster` for EKS clusters below platform version `eks.3` (#1583) --- ### Release Notes 0.100.0 (`docs/release_notes/0.100.0.md`) # Release 0.100.0 ## Features - Add p4de instance types to EFA plugin (#5330) ## Improvements - Detect kubectl version during kubeconfig generation and avoid known non-working configurations (#5288) ## Bug Fixes - Fix eksctl utils write-config breaks in kubectl 1.24 (#5287) - Fix what counts as ARM GPU instance type (#5293) ## Acknowledgments Weaveworks would like to sincerely thank: @ConnorJC3 and @torredil --- ### Release Notes 0.101.0 (`docs/release_notes/0.101.0.md`) # Release 0.101.0 ## Features - Add tolerations for taints on NVIDIA specific node groups (#5345) - Add support for managed bottlerocket GPU nodes (#5059) - Parallel drain all nodegroups (#5146) ## Improvements - Do not require overrideBootstrapCommand when ami family is Windows (#5248) ## Acknowledgments Weaveworks would like to sincerely thank: @a2ush, @hintofbasil --- ### Release Notes 0.102.0 (`docs/release_notes/0.102.0.md`) # Release 0.102.0 ## Improvements - Add additional EFA supported instances. (#5274) - Ignore IAM OIDC errors if no OIDC provider is associated with cluster (#5357) - Improved example and updated error message regarding taints (#5367) ## Bug Fixes - Fix authorization error check from Fargate (#5383) - Return an error when container runtime and overrideBootstrapCommand are defined together (#5365) ## Acknowledgments Weaveworks would like to sincerely thank: @mhuguesaws --- ### Release Notes 0.103.0 (`docs/release_notes/0.103.0.md`) # Release 0.103.0 ## Features - Add LogRetentionInDays update to update-cluster-logging command (#5404) ## Improvements - Fix documentation on updating a managed nodegroup and the error message during updating (#5392) - Add Wait for nodegroup updates (#5395) ## Bug Fixes - Do not set AZs if --dry-run is used with custom subnets (#5411) - Fix cluster crash in case of missing setting (#5410) --- ### Release Notes 0.104.0 (`docs/release_notes/0.104.0.md`) # Release 0.104.0 ## Improvements - Update aws-node addon (#5418) ## Bug Fixes - Remove aws iam authenticator (#5402) --- ### Release Notes 0.105.0 (`docs/release_notes/0.105.0.md`) # Release 0.105.0 - Fix docs for `--force` in addons (#5466) - Update taints examples to use the new syntax (#5457) ## Bug Fixes - Do not wait for coredns if there are no nodegroups (#5458) --- ### Release Notes 0.106.0 (`docs/release_notes/0.106.0.md`) # Release 0.106.0 ## Features - Lookup instance availability for all AZs (#5454) ## Acknowledgments Weaveworks would like to sincerely thank: @vflaux --- ### Release Notes 0.107.0 (`docs/release_notes/0.107.0.md`) # Release 0.107.0 ## Bug Fixes - Fix panic when stack has no physical resource ID (#5463) - Fix panic in `upgrade nodegroup` (#5517) ## Acknowledgments Weaveworks would like to sincerely thank: @vflaux --- ### Release Notes 0.108.0 (`docs/release_notes/0.108.0.md`) # Release 0.108.0 ## Features - Add capacity reservation to spec (#5462) ## Bug Fixes - Add support for ed25519 inline key (#5543) - Fix formatting in retry attempt log message for disassociate identityprovider (#5545) --- ### Release Notes 0.109.0 (`docs/release_notes/0.109.0.md`) # Release 0.109.0 ## Features - Add support for Kubernetes 1.23 (#5519) ## Improvements - Remove support for 1.19 (#5536) - Remove support for `WindowsServer20H2CoreContainer` and `WindowsServer2004CoreContainer` (#5610) - Update docs about ASG tags (#5574) - Changed albIngress to awsLoadBalancerController in usage iam-policies example (#5566) - Fix typo in error message for IAM service account (#5555) ## Bug Fixes - Fix coredns wait when there are no nodes (#5571) ## Acknowledgments Weaveworks would like to sincerely thank: @neha-viswanathan, @akhal3d96, @jinnuae40 and @matti --- ### Release Notes 0.11.0 (`docs/release_notes/0.11.0.md`) # Release 0.11.0 ## Features - add support for Fargate (#1623) - add support for `nodegroup.taints` and `nodegroup.maxPodsPerNode` for Windows nodegroups (#1604) - add `servicediscovery:ListServices` to the App Mesh node policy to allow for App Mesh controller Cloud Map integration (#1610) ## Improvements - add support for `only-missing` filter and append globs for managed nodegroups (#1593) - `eksctl` logs git commit hash if version tag is not present (#1614) - increase timeout for public/private endpoint tests (#1613) - add Makefile rule to build for multiple OSes (#1612) ## Bug Fixes - fix `eksctl create fargateprofile` when using a config file with multiple Fargate profiles (#1628) - fix broken link in [documentation](https://eksctl.io/usage/eks-managed-nodegroups/) (#1598) - depending on command, use appropriate CLI argument between `--name` and `--cluster` (#1594) - fix VPC-related tests (#1557) --- ### Release Notes 0.110.0 (`docs/release_notes/0.110.0.md`) # Release 0.110.0 ## Improvements - Karpenter v0.15.0 support (#5623) - Allow creating Windows and Ubuntu nodegroups with NVIDIA GPU instances (#5627) ## Bug Fixes - Fix ARN partition issue to support IPv6 in China regions (#5644) - Return StackName field when getting a single managed nodegroup (#5643) ## Acknowledgments Weaveworks would like to sincerely thank: @walkley --- ### Release Notes 0.111.0 (`docs/release_notes/0.111.0.md`) # Release 0.111.0 ## Improvements - Docs update for supported command (get fargate) for non-eksctl created cluster (#5659) ## Bug Fixes - Add RBAC Permissions for PVC for emr-containers (#5660) ## Acknowledgments Weaveworks would like to sincerely thank: @melodyyangaws and @surajnarwade --- ### Release Notes 0.112.0 (`docs/release_notes/0.112.0.md`) # Release v0.112.0 ## 🚀 Features - Support creating clusters and nodegroups on AWS Outposts (#5709) - Enable the use of config-file for identitymapping creation (#5464) ## Acknowledgments Weaveworks would like to sincerely thank: @adamjohnson01 --- ### Release Notes 0.113.0 (`docs/release_notes/0.113.0.md`) # Release v0.113.0 ## 🐛 Bug Fixes - Set default scaling config max size to 1 (#5701) ## Acknowledgments Weaveworks would like to sincerely thank: @Gearheads --- ### Release Notes 0.114.0 (`docs/release_notes/0.114.0.md`) # Release v0.114.0 ## 🎯 Improvements - Bump helm to 3.9.4 and pin K8s chain to 1.23.12 (#5736) - Add validation error when using private subnets with privateNetworking disabled (#5712) ## 🐛 Bug Fixes - Fix creating ARM-based GPU nodegroups for self-managed Bottlerocket (#5729) --- ### Release Notes 0.115.0 (`docs/release_notes/0.115.0.md`) # Release v0.115.0 ## 🎯 Improvements - Updates to karpenter cloudformation policies (#5735) - Make Kubernetes 1.23 the default (#5634) ## 🧰 Maintenance - Setup identity as weaveworks bot before triggering Netlify deployment (#5752) --- ### Release Notes 0.116.0 (`docs/release_notes/0.116.0.md`) # Release v0.116.0 ## 💥 Breaking Changes - Remove deprecated field disableASGTagPropagation (#5775) ## 🚀 Features - Add Support for Windows Server 2022 (#5741) ## 🎯 Improvements - Adjust the command line parameters to be consistent for shorthand option -c (#5625) ## 🐛 Bug Fixes - Fix authentication failures (#5772) ## Acknowledgments Weaveworks would like to sincerely thank: @KlwntSingh, and @ysicing --- ### Release Notes 0.117.0 (`docs/release_notes/0.117.0.md`) # Release v0.117.0 ## 🚀 Features - Add support for me-central-1 (#5837) - Add option to preserve addon config during update (#5830) ## 🎯 Improvements - Added networking.k8s.io api to allowlist ingress operations (#5834) - Add ready plugin to CoreDNS and use health.lameduck for a safer shutdown (#5695) ## 🐛 Bug Fixes - Preserve existing version during addon update when a version is not provided (#5803) - Fix max-pods for managed nodegroups (#5808) - Fix adding taints and labels as ASG tags (#5781) ## Acknowledgments Weaveworks would like to sincerely thank: @singhbhaskar, and @wind0r --- ### Release Notes 0.118.0 (`docs/release_notes/0.118.0.md`) # Release v0.118.0 ## 💥 Breaking Changes - Karpenter support with OCI registry, minimum supported version is now v0.17.0 for new cluster creation (#5832) ## 🎯 Improvements - Add trn1 instance types to the neuron device plugin (#5864) ## 🧰 Maintenance - Update python version for netlify docs (#5857) ## 📝 Documentation - Add note on how to upgrade managed nodegroups using custom AMI (#5845) ## Acknowledgments Weaveworks would like to sincerely thank: @aws-vrnatham --- ### Release Notes 0.119.0 (`docs/release_notes/0.119.0.md`) # Release v0.119.0 ## 🚀 Features - Add resolve conflicts option on addon create (#5880) ## 🎯 Improvements - Improve error message when Fargate is unsupported in an AZ (#5875) ## 🐛 Bug Fixes - Fix setting cluster DNS for managed Ubuntu (#5889) - Fix AWS auth when \`AWS\_PROFILE\` is set (#5854) - Make AMI Bottlerocket value case insensitive (#5852) ## 🧰 Maintenance - Fix publish-docs workflow (#5901) - Remove dependency on k8s.io/legacy-cloud-providers (#5899) - Upgrade vulnerable dependencies (#5890) ## 📝 Documentation - Updates to additional volumes mapping section and docs theme (#5885) ## Acknowledgments Weaveworks would like to sincerely thank: @faraazkhan --- ### Release Notes 0.12.0 (`docs/release_notes/0.12.0.md`) # Release 0.12.0 ## Features - allow restricting access to EKS Kubernetes Public API Endpoint (#1688) - `eksctl {get,delete} fargateprofile` can now read the cluster name & region from a `ClusterConfig` file (#1667) - support Managed Nodegroups and Fargate for auto-upgraded clusters (#1665) - upgrade CNI plugin to version 1.5.5 (#1554) - support using a proxy in the IAM OIDC provider configuration (#1620) - enable support for Managed Nodegroups when upgrading from EKS 1.13 to 1.14 (#1656) - `--wait` flag is no longer supported in `eksctl update cluster` ## Improvements - upgrade Go to 1.13 (#1648) - `eksctl` now leverages `cobra` to print all errors in a more standardised way (#1664) - `eksctl` now no longer print usage/help on invalid user input (#1677) - improved Fargate documentation & example `ClusterConfig` manifest (#1666) - update all AMIs (#1674) - no longer print usage help on errors (#1677) - improved Circleci build workflows (#1679) - update maxpods.go (#1659) - added `@weaveworks/team` to the pull-request template that mentions the maintainers (#1660) ## Bug fixes - fix setup script if GOPATH contains multiple paths (#1632) - fix error in processing of subnets (#1661) - fix AMI selection for Managed Nodegroups (#1649) --- ### Release Notes 0.120.0 (`docs/release_notes/0.120.0.md`) # Release v0.120.0 ## 🚀 Features - Add support for EKS 1.24 (#5807) ## 🧰 Maintenance - Update base image in the build container (#5907) --- ### Release Notes 0.121.0 (`docs/release_notes/0.121.0.md`) # Release v0.121.0 ## 🚀 Features - Support for clusters with trn1 instance types (#5906) ## 🐛 Bug Fixes - Update error handling for RegisterCluster (#5947) ## 🧰 Maintenance - Added slack notifications for publish docs workflow (#5926) - Update deprecated set-output in workflows (#5927) ## Acknowledgments Weaveworks would like to sincerely thank: @dewjam --- ### Release Notes 0.122.0 (`docs/release_notes/0.122.0.md`) # Release v0.122.0 ## 🎯 Improvements - Improve log message when calling DescribeCluster during registration fails (#5973) ## 🐛 Bug Fixes - Do not wait for aws-ebs-csi-driver if there are no nodegroups (#5929) - Fix release version in notifications (#5975) ## 🧰 Maintenance - Update Kubernetes configuration (#5970) ## Acknowledgments Weaveworks would like to sincerely thank: @radTuti --- ### Release Notes 0.123.0 (`docs/release_notes/0.123.0.md`) # Release v0.123.0 ## 🚀 Features - Support for Addon filters in utils describe-addon-versions (#5996) ## 🧰 Maintenance - Remove the pkg/apis/eksctl.io/v1alpha5/schema.go file (#5976) --- ### Release Notes 0.124.0 (`docs/release_notes/0.124.0.md`) # Release v0.124.0 ## 🚀 Features - Add support for Windows managed nodegroups (#5935) - Add support for utils describe-addon-configuration (#6056) - Add support for advanced configuration for add-ons (#6049) - Add support for new spot allocation strategies (#6038) ## 🎯 Improvements - Write kubeconfig lock file to tmp so that it gets automatically cleaned up (#6030) ## 🧰 Maintenance - Improve unit test coverage for private cluster (#6028) ## 📝 Documentation - Add logRetentionInDays in cloudwatch cluster logging docs (#6037) ## Acknowledgments Weaveworks would like to sincerely thank: @jiechen0826 and @matti --- ### Release Notes 0.125.0 (`docs/release_notes/0.125.0.md`) # Release v0.125.0 ## 🚀 Features - Add support for placement groups in Outposts (#6104) ## 🎯 Improvements - Make Kubernetes 1.24 default version in eksctl (#6094) - Remove support for heptio-authenticator-aws (#6096) - Bump sourcegraph/go-diff to a version which supports Apple diff timestamps (#6088) ## 🐛 Bug Fixes - Fix config file support for `get addons` (#6100) - Allow Nvidia, Neuron and AWS node containers to run as root (#6065) ## 🧰 Maintenance - Fix identity provider integration test (#6115) - Updated dependencies and actions (#6117) --- ### Release Notes 0.126.0 (`docs/release_notes/0.126.0.md`) # Release v0.126.0 ## 🐛 Bug Fixes - Fix include update irsa (#6031) - Add 'CreateServiceLinkedRole' to karpenter permissions (#6102) ## Acknowledgments Weaveworks would like to sincerely thank: @yuxiang-zhang --- ### Release Notes 0.127.0 (`docs/release_notes/0.127.0.md`) # Release v0.127.0 ## 🚀 Features - Support fully-private clusters for Outposts (#6172) ## 🎯 Improvements - Support for new versions of karpenter (#6169) - Remove support for EKS 1.20 (#6154) ## 🐛 Bug Fixes - Fix updating kube-proxy to a newer version (#6176) ## 🧰 Maintenance - Improve unit test coverage of create cluster - Karpenter (#6143) ## 📝 Documentation - Fixed broken site link (#6168) - Removed banner and updated outposts doc (#6178) --- ### Release Notes 0.128.0 (`docs/release_notes/0.128.0.md`) # Release v0.128.0 ## 🎯 Improvements - Port coredns lameduck fix to EKS 1.24 (#6023) ## 🐛 Bug Fixes - Check Fargate Profile existence before creating (#5722) ## 🧰 Maintenance - Improve unit test coverage of create cluster - nodegroups (#6177) - Ldflags -s -w options for make binary (#6196) ## 📝 Documentation - Fix typo in "Addons" configuration documentation (#6194) - Updated material theme, footer and landing page (#6204) ## Acknowledgments Weaveworks would like to sincerely thank: @guessi, @maximepiton and @pnmatich --- ### Release Notes 0.129.0 (`docs/release_notes/0.129.0.md`) # Release v0.129.0 ## 🚀 Features - Add support for Spain (eu-south-2) region (#6234) - Add support for Hyderabad (ap-south-2) region (#6231) - Add support for Zurich (eu-central-2) region (#6230) - Allow all EKS versions to be used for Outposts clusters (#6227) ## 🧰 Maintenance - Fix kube-proxy assertion in integration test (#6226) - Wait for coredns to be deleted before recreating in addons integration test (#6225) ## 📝 Documentation - Update availabilityZones example under managed node group (#6223) - Add user docs for Bottlerocket custom AMI support (#6228) - Add note on how pod disruption budget policies influence cluster deletion (#6209) - Updated material theme, footer and landing page (#6204) ## Acknowledgments Weaveworks would like to sincerely thank: @yuxiang-zhang and @vsoch --- ### Release Notes 0.13.0 (`docs/release_notes/0.13.0.md`) # Release 0.13.0 ## Improvements - enabled building snaps from source (#969) - added installation instructions for MacPorts on macOS (#1708) - added tests for `pkg/vpc/vpc.go` (#1712) - fixed test failing due to IPv6 (#1731) - added integration tests for backwards compatibility with older versions of eksctl (#1724) - sanitised & streamlined CI & release pipeline (#1443, #1691, #1689, #1700, #1715) - fixed typo in error message (#1694) ## Bug fixes - fixed nodegroup creation when `imageBuilder` is enabled; fixed duplicate policies (#1705) - fixed path to SSH key in integration tests (#1704) ## Acknowledgment Weaveworks would like to sincerely thank: - @clareliguori (#1694) - @dholbach (#969) - @mikeroyal (#969) - @sayboras (#1443, #1712, #1731) - @szczad (#1708) for their contributions to this release. Your work is highly appreciated and the project would not have gotten this far without people like you! --- ### Release Notes 0.130.0 (`docs/release_notes/0.130.0.md`) # Release v0.130.0 ## 🚀 Features - Pass ServiceCIDR to Windows bootstrap script (#6242) ## 🎯 Improvements - Improve UX when working with custom AMIs by having AMI Family explicitly specified (#6243) ## 🧰 Maintenance - Increase timeout and ordering for ebs-csi-driver addon test (#6267) ## 📝 Documentation - Document invalid usage of --dry-run with --profile (#6272) --- ### Release Notes 0.131.0 (`docs/release_notes/0.131.0.md`) # Release v0.131.0 ## 🎯 Improvements - Remove support for EKS 1.21 (#6237) ## 🐛 Bug Fixes - Fix `get fargateprofiles` not getting all profiles (#6279) ## 📝 Documentation - Correct docs on how scaling nodegroups is achieved (#6291) - Fix docs typos and added collapsible admonition blocks (#6296) ## Acknowledgments Weaveworks would like to sincerely thank: @rongshengfang and @yuxiang-zhang --- ### Release Notes 0.132.0 (`docs/release_notes/0.132.0.md`) # Release v0.132.0 ## 🚀 Features - Support for EKS 1.25 (#6290) - Support YAML format for Addon advanced configuration values (#6264) ## 🐛 Bug Fixes - Make Bottlerocket OS Volume Type configurable (#6294) ## Acknowledgments Weaveworks would like to sincerely thank: @yuxiang-zhang --- ### Release Notes 0.133.0 (`docs/release_notes/0.133.0.md`) # Release v0.133.0 ## 🐛 Bug Fixes - Don't specify kubernetes version when using a custom AMI (#6318) ## 🧰 Maintenance - Improve bottlerocket integration tests - custom AMI and GPU support (#6347) - Use addons published by AWS EKS for describe-addon-versions (#6341) --- ### Release Notes 0.134.0 (`docs/release_notes/0.134.0.md`) # Release v0.134.0 ## 🚀 Features - Add support for Melbourne (ap-southeast-4) (#6400) - Make 1.25 the default Kubernetes version (#6374) ## 🎯 Improvements - Improve messages during creation of iamserviceaccount (#5856) ## 🐛 Bug Fixes - Use correct image path in coredns manifest (#6392) - Add condition to restrict fargate pod execution role to source arn (#6375) ## 🧰 Maintenance - Add integration tests for addons in a cluster with no nodes (#6378) ## Acknowledgments Weaveworks would like to sincerely thank: @programmer04 --- ### Release Notes 0.135.0 (`docs/release_notes/0.135.0.md`) # Release v0.135.0 ## 🐛 Bug Fixes - [Upgrade nodegroup via EKS API] Make AMI release version customisable (#6340) - Add m7g and r7g as ARM instance types (#6436) - Handle deprecated AMIs when scaling self-managed nodegroups (#6401) ## 🧰 Maintenance - Fix typo in SharedNodeSecurityGroup validation error message (#6431) ## 📝 Documentation - Updated cloudwatch log descriptions (#6346) - Update addon example with ebs addon (#6438) ## Acknowledgments Weaveworks would like to sincerely thank: @Jennas-Lee, @tomr-jet, @tucktuck9 and @rothgar --- ### Release Notes 0.136.0 (`docs/release_notes/0.136.0.md`) # Release v0.136.0 ## 🐛 Bug Fixes - Fix panic when a nodegroup element is not set in ClusterConfig (#6443) ## 🧰 Maintenance - Log stack events on integration test failure (#6450) --- ### Release Notes 0.137.0 (`docs/release_notes/0.137.0.md`) # Release v0.137.0 ## 🚀 Features - Add wait flag to \`scale nodegroup\` (#6406) - Add support for Spot Interruption Queue within Karpenter (#6460) ## 🧰 Maintenance - Update build image manifest, tag file and workflows, dependencies (#6488) ## 📝 Documentation - Update Karpenter doc with note for logging out of ECR repository (#6465) ## Acknowledgments Weaveworks would like to sincerely thank: @danielloader and @yuxiang-zhang --- ### Release Notes 0.138.0 (`docs/release_notes/0.138.0.md`) # Release v0.138.0 ## 🚀 Features - Add support for EKS 1.26 (#6502) ## 🎯 Improvements - Add volcano podgroups permissions (#6500) ## Acknowledgments Weaveworks would like to sincerely thank: @mengfwanaws --- ### Release Notes 0.139.0 (`docs/release_notes/0.139.0.md`) # Release v0.139.0 ## 🚀 Features - Security Policy for eksctl project (#6541) ## 🐛 Bug Fixes - Fix flux version validation (#6530) ## 📝 Documentation - Fix empty info block on Default Addon Upgrades page (#6524) - Update installation instructions (#6376) - AWS Private link support for fully private cluster (#6408) ## Acknowledgments Weaveworks would like to sincerely thank: @thezanke, and @yuxiang-zhang --- ### Release Notes 0.14.0 (`docs/release_notes/0.14.0.md`) # Release 0.14.0 ## Features - Default to SSM Parameter for AMI resolution (#1814) - Upgrade CNI version to 1.6.0 (#1824) - Update Flux Helm operator to 1.0.0-rc9 (#1815) - Update to core-dns:1.6.6 (#1729) - Add NodeRole for managed group (#1747) - Update Flux packages and migrate to Helm v3 (#1761) - Add eksctl version tag in cloudformation (#1757) - Support spot instances with only one instance type (#1772) - Add tags flag in create nodegroup CLI (#1760) - Add support for permissions Boundary (#1638) - Added the ability to override the kube config cluster server (#1746) - Support public/private API endpoints settings from file (#1693) ## Improvements - Deprecate latest_release tag (#1809) - Add integration tests for Fargate (#1792) - Improve linting rules (#1786) - Add example of using labels for forcing pods to run on desired node groups (#1804) - Improve the error when resource is not known (#1776) - Update README.md (#1794) - Create Fargate role only if there is --fargate (#1749) - Fix missing tags in nodegroup after scaling (#1836) ## Bug Fixes - Fix bug in update-cluster-endpoint #1820 (#1821) - Fix duplicate listing of subnets (#1477) - Fix null pointer dereference in cluster access validation (#1852) ## Acknowledgments Weaveworks would like to sincerely thank: @ajayk, @allanchua101, @dholbach, @ota42y, @paulbes, @ryotarai, @sayboras, @stefanprodan and @treacher --- ### Release Notes 0.140.0 (`docs/release_notes/0.140.0.md`) # Release v0.140.0 ## 🐛 Bug Fixes - Only select AZs that support all instance types on nodegroup creation (#6464) - Force kubeconfig get-token output format to json (#6554) ## 📝 Documentation - Updated schema for unsupported fields for managed nodegroups (#6555) --- ### Release Notes 0.141.0 (`docs/release_notes/0.141.0.md`) # Release v0.141.0 ## 🐛 Bug Fixes - Force kubeconfig get-token output format to json (#6554) ## 📝 Documentation - Correct typo on NAT Gateway configuration value choices (#6569) ## Acknowledgments Weaveworks would like to sincerely thank: @khacminh, and @yuxiang-zhang --- ### Release Notes 0.142.0 (`docs/release_notes/0.142.0.md`) # Release v0.142.0 ## 🎯 Improvements - Add TagResource perm to EFS wellknown policy (#6611) ## 🧰 Maintenance - Update config file examples to reflect that amiFamily is required with custom AMI (#6617) - Bump dependencies and update build image (#6613) ## Acknowledgments Weaveworks would like to sincerely thank: @jimmythedog --- ### Release Notes 0.143.0 (`docs/release_notes/0.143.0.md`) # Release v0.143.0 ## 🚀 Features - Add support for EKS 1.27 (#6639) ## 🎯 Improvements - Enforce IMDSv2 by default (#6631) ## 🐛 Bug Fixes - Fix readiness probe in HTTP test (#6628) --- ### Release Notes 0.144.0 (`docs/release_notes/0.144.0.md`) # Release v0.144.0 ## 🚀 Features - Allow creating subnets with resource-based hostnames (#6651) ## 🎯 Improvements - Updated IAM policy statement for loadbalancer controller (#6655) - Port Add ready plugin to CoreDNS and use health.lameduck for a safer shutdown to 1.25, 1.26 and 1.27 (#6630) ## 🐛 Bug Fixes - Only retry on appropriate API Errors when calling `eks:DescribeUpdate` (#6645) - Clean up ALBs using `spec.ingressClassName` and ALB security groups (#6389) - Set default Windows volume name to /dev/sda1 (#6638) ## 🧰 Maintenance - Cleanup of readme and contributing docs, fixed formatting and typos (#6660) - Fix logging stack events logic for Trainium and Inferentia integration tests (#6649) - Bump schemer from v0.0.0-2021 to v0.0.0-2023 (#6646) ## 📝 Documentation - Generated ATTRIBUTIONS.md file (#6647) ## Acknowledgments Weaveworks would like to sincerely thank: @aaroniscode, @joneidaws and @wind0r --- ### Release Notes 0.145.0 (`docs/release_notes/0.145.0.md`) # Release v0.145.0 ## 🚀 Features - Make the output stream for eksctl command configurable (#6675) ## 🎯 Improvements - Updated wellKnownPolicy for Cluster Autoscaler and LoadBalancer Controller (#6667) - Add integration tests for resource-based hostnames (#6659) ## 🧰 Maintenance - Bump dependencies (#6674) ## Acknowledgments Weaveworks would like to sincerely thank: @hassaanakram and @joneidaws --- ### Release Notes 0.146.0 (`docs/release_notes/0.146.0.md`) # Release v0.146.0 ## 🎯 Improvements - Update vpc-cni to 1.12.6 (#6692) ## 🧰 Maintenance - Bump dependencies (#6690) ## 📝 Documentation - New eksctl channel (#6697) ## Acknowledgments Weaveworks would like to sincerely thank: @wind0r --- ### Release Notes 0.147.0 (`docs/release_notes/0.147.0.md`) # Release v0.147.0 ## 🎯 Improvements - Use nodeAffinity to determine if an addon supports ARM64 (#6695) ## 🐛 Bug Fixes - Fix CoreDNS probes for Kubernetes 1.26 (#6707) ## 🧰 Maintenance - Add identity action to update-generated workflow (#6735) - Bump dependencies (#6730) ## 📝 Documentation - Update docs with \`preserve\` option to create addon resolveConflicts (#6729) ## Acknowledgments Weaveworks would like to sincerely thank: @wind0r --- ### Release Notes 0.148.0 (`docs/release_notes/0.148.0.md`) # Release v0.148.0 ## 🚀 Features - Add support for `us-iso-east-1` and `us-isob-east-1` regions (#6759) ## 🎯 Improvements - Replaced weaveworksbot with eksctlbot in workflows and actions (#6758) ## 🐛 Bug Fixes - Fix index out of range error when aws-node, kube-proxy or coredns are missing (#6756) ## 🧰 Maintenance - Bump dependencies (#6754, #6742) - Migrate to sdk v2 - configuration loading (#6432) ## 📝 Documentation - Fix documentation for EKS interface endpoint (#6747) - Added new logo and favicon assets (#6746) ## Acknowledgments Weaveworks would like to sincerely thank: @wind0r --- ### Release Notes 0.149.0 (`docs/release_notes/0.149.0.md`) # Release v0.149.0 ## 🚀 Features - Update current github urls with new eksctl-io github organisation urls (#6768) ## 🐛 Bug Fixes - Fix \`AWS\_REGION\` and \`AWS\_DEFAULT\_REGION\` support for eksctl commands (#6787) - Fix generating mocks for aws-sdk-go-v2 interfaces (#6777) ## 🧰 Maintenance - Bump dependencies (#6788) - Pin K8s dependencies chain to 1.25 (#6677) --- ### Release Notes 0.15.0 (`docs/release_notes/0.15.0.md`) # Release 0.15.0 ## Features - Add support for KMS encryption provider (#1897) - Add support for China regions (#1860) - Add region Beijing (cn-north-1) (#1741) - Add region Ningxia (#1720) - Support addons for China regions, refactor setting container image for addons (#1867) - Add support for Kubernetes 1.15 (#1917 #1916) - Add support for Bottlerocket NodeGroups (#1918 #1919) ## Improvements - Remove integration tests timeout (#1857) - Store artifacts created during release and rc jobs in circleci (#1864) ## Fixes - Fix null pointer dereference in cluster access validation (#1855) - [IRSA] Set default audience based on AWS partition (#1877) - Fix #1871, add partition check(aws-cn,aws) for OpenIDConnectProviderArn (#1872) --- ### Release Notes 0.150.0 (`docs/release_notes/0.150.0.md`) # Release v0.150.0 ## 🚀 Features - Switch from Docker Hub registry to ECR registry (#6769) ## 🐛 Bug Fixes - Update measurement ID for GA (#6822) ## 🎯 Improvements - Migrate to mockery v2 (#6778) ## 🧰 Maintenance - Free up disk space in release workflow before building binaries (#6827) - Bump dependencies (#6803) ## 📝 Documentation - Add announcement for transfer to new Github Organisation (#6805) --- ### Release Notes 0.151.0 (`docs/release_notes/0.151.0.md`) # Release v0.151.0 ## 🚀 Features - Support custom AMIs for self-managed Windows nodegroups (#6804) - Support custom Ubuntu AMIs for EKS-managed nodegroups (#6850) ## 🎯 Improvements - Remove support for EKS 1.22 (#6704) ## 🐛 Bug Fixes - Fix error with tar in `Post Cache go-build and mod` step (#6840) - Fix setting link-time variables for release version (#6841) - Select one subnet for AZs where multiple are present and no VPC config provided (#6814) - Paginate instance type offerings response (#6832) ## 🧰 Maintenance - Bump dependencies (#6852, #6859) - Cleanup Flux Integration (#6836) ## Acknowledgments Weaveworks would like to sincerely thank: @watany-dev --- ### Release Notes 0.152.0 (`docs/release_notes/0.152.0.md`) # Release v0.152.0 ## 🚀 Features - Add support for Tel Aviv (il-central-1) (#6899) - Add option to print specific events when running `eksctl describe-stacks` (#6851) ## 🐛 Bug Fixes - Fix PR Rebase workflow (#6903) ## 🧰 Maintenance - Bump dependencies (#6920, #6895, #6874) - Pin actions to a full length commit SHA (#6901) - Fix `k8s.io/kubernetes` version (#6868) - Fix setting link-time variables for release version (#6841) ## 📝 Documentation - Remove `iam:ListInstanceProfiles` from the required permissions list (#6898) --- ### Release Notes 0.153.0 (`docs/release_notes/0.153.0.md`) # Release v0.153.0 ## 🚀 Features - Update Bottlerocket image version on nodegroup upgrade (#6923) ## 🎯 Improvements - Improve error message when `overrideBootstrapCommand` is required for `customAMIs` (#6897) ## 🐛 Bug Fixes - Allow setting cluster-dns-ip for Bottlerocket (#6931) ## 🧰 Maintenance - Unit test creating the EKS provider (#6875) - Remove PR rebase workflow (#6934) - Bump dependencies and Update build image tag (#6937, #6946) --- ### Release Notes 0.154.0 (`docs/release_notes/0.154.0.md`) # Release v0.154.0 ## 🐛 Bug Fixes - Fix error message for Bottlerocket validation (#6967) - Don't wait for `aws-efs-csi-driver` addon if cluster has no nodegroups (#6960) ## 🧰 Maintenance - Bring Windows integration test runtime down to less than half (#6965) ## 📝 Documentation - Add references to Enterprise support with Flux on Gitops page (#6968) --- ### Release Notes 0.155.0 (`docs/release_notes/0.155.0.md`) # Release v0.155.0 ## 🚀 Features - Add support for us-iso-west-1 region (#6997) ## 🎯 Improvements - Add validations for managed nodegroup labels (#6947) - Switch default volume type back to gp3 for ISO regions (#6969) ## 🐛 Bug Fixes - Fix merge release workflow when there is nothing to merge (#6993) ## 🧰 Maintenance - Update Ginkgo (#6996) - Replace deprecated Ginkgo CLI options (#6994) - Improve unit tests' runtime (#6992) - Bump dependencies (#6998) --- ### Release Notes 0.156.0 (`docs/release_notes/0.156.0.md`) # Release v0.156.0 ## 🚀 Features - Adopters for eksctl (#7027) ## 🐛 Bug Fixes - Cleanup subnets with invalid AZs before importing VPC from CFN stack (#6935) ## 🧰 Maintenance - Bump dependencies (#7025) --- ### Release Notes 0.157.0 (`docs/release_notes/0.157.0.md`) # Release v0.157.0 ## 🚀 Features - Material version updates and added social cards, cookie consent and feedback widgets (#7050) ## 🐛 Bug Fixes - Fix `update aws node` workflow (#7045) - Disallow passing a vpc.securityGroup with external security group rules (#7030) ## 📝 Documentation - Add note on how to use `--include/exclude` flags when creating iamserviceaccounts (#7047) --- ### Release Notes 0.158.0 (`docs/release_notes/0.158.0.md`) # Release v0.158.0 ## 🐛 Bug Fixes - Call `kubectl version` command with `--output=json` flag (#7032) ## 🧰 Maintenance - Bump `github.com/cloudflare/cfssl` to fix indirect deps DOS vulnerability (#7067) - Upgrade goproxy to fix DoS vulnerability (#7068) ## 📝 Documentation - Corrected Chocolatey Install Command (#7063) ## Acknowledgments Weaveworks would like to sincerely thank: @Nyami --- ### Release Notes 0.159.0 (`docs/release_notes/0.159.0.md`) # Release v0.159.0 ## 🧰 Maintenance - Allow managed suite specs to run in parallel (#7065) - Update to Go 1.20 (#7075) ## 📝 Documentation - Fix doc example for bottlerocket bootstrap container (#7087) ## Acknowledgments Weaveworks would like to sincerely thank: @jhart0 --- ### Release Notes 0.16.0 (`docs/release_notes/0.16.0.md`) # Release 0.16.0 # Features - Updated vpc-controllers addon deployment files to v0.2.3 (#1954) - Support tags for Fargate profiles (#1909) - Support spotAllocationStrategy on instances distribution config (#1883) - Support for North California region us-west-1 for when this becomes available (#1970) ## Bug Fixes - Correct --update-auth-configmap flag description for create nodegroup (#1890) - Fix crash when creating Windows clusters with Kubernetes 1.15 (#1957) - Add mappings as part of reconciliation (#1934) - Fix fargate profile tag input (#1953) - Fix EBS iam policy (#1905) - Fix "create nodegroup": do not add outputs for empty NATs (#1908) - Fix (plan) output when updating k8s addons (#1719) - Remove all dangling network interfaces during cluster deletion (#1880) ## Improvements - Fix unit test for go 1.14 (#1925) - Add support to bootstrap AL2 ARM based instances (#1832) - Remove --allow-privileged flag in docs (#1929) ## Acknowledgments Weaveworks would like to sincerely thank: @AtkinsChang, @abhipth, @liranp, @stevensu1977 and @sayboras --- ### Release Notes 0.160.0 (`docs/release_notes/0.160.0.md`) # Release v0.160.0 ## 🚀 Features - Add support for EKS 1.28 (#7108) ## 🧰 Maintenance - Bump dependencies (#7109) --- ### Release Notes 0.161.0 (`docs/release_notes/0.161.0.md`) # Release v0.161.0 ## 🚀 Features - Make EKS 1.27 default (#7117) - Docs Landing page (#7092) ## 🎯 Improvements - Refine log statements using `%w` for better formatting and UX (#7115) ## 🧰 Maintenance - Update gomarkdown to fix out-of-bounds read while parsing citations (#7120) - Upgrade go-restful (#7121) - Move addons related tasks to actions package (#7077) --- ### Release Notes 0.162.0 (`docs/release_notes/0.162.0.md`) # Release v0.162.0 ## 🎯 Improvements - Set AWS credential expiry window to 30 minutes (#7116) ## 🐛 Bug Fixes - Fix ELB cleanup failure due to resource name ending with a hyphen (#7029) - Fix coredns 1.28 asset (#7126) ## Acknowledgments Weaveworks would like to sincerely thank: @0xlen and @otterley --- ### Release Notes 0.163.0 (`docs/release_notes/0.163.0.md`) # Release v0.163.0 ## 🚀 Features - Add support for P5 instances (#7201) ## 🎯 Improvements - Update coredns (#7179) - Automatically update coredns assets using Github workflows (#7178) ## 🐛 Bug Fixes - Update build distros workflow to save disk space (#7173) ## 🧰 Maintenance - Fix kOps privilege escalation vulnerability, update permissions for stale workflow and dependencies (#7205) - Bump dependencies and update build image (#7170) ## Acknowledgments Weaveworks would like to sincerely thank: @vadiml --- ### Release Notes 0.164.0 (`docs/release_notes/0.164.0.md`) # Release v0.164.0 ## 🚀 Features - Allow mutating control plane subnets and security groups (#7218) ## 🎯 Improvements - Add `eksctl utils update-cluster-vpc-config` to update the cluster VPC config (#7090) ## 🐛 Bug Fixes - Don't close error channel early when wait flag is used (#7217) --- ### Release Notes 0.165.0 (`docs/release_notes/0.165.0.md`) # Release v0.165.0 ## 🚀 Features - Add support for EKS Pod Identity Associations (#7312, #7313, #7315) ## 🧰 Maintenance - Ignore google docs domains for link checker (#7316) - Bump aws-sdk dependencies (#7292) - Bump dependencies (#7267) - Improve CRUD suite runtime (#7208) ## 📝 Documentation - Fix README Links (#7230) ## Acknowledgments Weaveworks would like to sincerely thank: @KlimDos --- ### Release Notes 0.166.0 (`docs/release_notes/0.166.0.md`) # Release v0.166.0 ## 🚀 Features - Add utils command to migrate to pod identity associations (#7343) ## 🐛 Bug Fixes - Disable module caching in `actions/setup-go` (#7374) - Fix STS interface (#7345) - Fix build-integration-test for pod_identity_associations (#7336) ## 🧰 Maintenance - Update build image tag and actions (#7337) ## 📝 Documentation - Update reference for bottlerocket settings (#7375) - Documentation Update: Correct IAM Pod Identity Example Link (#7376) ## Acknowledgments Weaveworks would like to sincerely thank: @chasestech --- ### Release Notes 0.167.0 (`docs/release_notes/0.167.0.md`) # Release v0.167.0 ## 🚀 Features - Add support for EKS Access Entries (#7421, #7422, #7423) - Add support for ca-west-1 (#7419) ## 🧰 Maintenance - Bump AWS dependencies, fix test (#7420) --- ### Release Notes 0.168.0 (`docs/release_notes/0.168.0.md`) # Release v0.168.0 ## 🎯 Improvements - Safely access/mutate fargate coredns pod annotations (#7480) ## 🐛 Bug Fixes - Fix generating presigned URL for K8s authentication (#7487) - Ignore unsupported zone types when creating a zone mapping (#7204) --- ### Release Notes 0.169.0 (`docs/release_notes/0.169.0.md`) # Release v0.169.0 ## 🚀 Features - Add support for EKS 1.29 (#7498) ## 🎯 Improvements - Fix coredns pdb preventing cluster deletion in integration tests (#7496) - Update well-known policy for ebsCSIController (#7451) - Handle unordered public endpoint CIDRs from EKS in endpoint updates (#7483) ## 🧰 Maintenance - Fix coredns pdb preventing cluster deletion in integration tests (#7496) ## 📝 Documentation - Fix outdated links (#7297) ## Acknowledgments Weaveworks would like to sincerely thank: @Emberwalker, @guessi, and @teraflik --- ### Release Notes 0.17.0 (`docs/release_notes/0.17.0.md`) # Release 0.17.0 ## Improvements - New default IP mapping settings for public subnets (#2002) - Add documentation about alternative ways to delete clusters (#1770) - Improve help of update cluster command (#2009) - Make kubernetes 1.15 the default version (#1997) - Set kubeReserved defaults according to instance type (#1986) - Upgrade kube depedencies to 1.15.10 (#1964) ## Bug Fixes - Fix code generation path issues (#2007) ## Acknowledgments Weaveworks would like to sincerely thank: @rndstr, @sayboras, @sean-abbott --- ### Release Notes 0.170.0 (`docs/release_notes/0.170.0.md`) # Release v0.170.0 ## 🚀 Features - Add support for Access Entry type (#7522) ## 🎯 Improvements - Skip draining managed nodegroups when deleting cluster (#7515) - Expand Karpenter settings.aws block to settings for v0.33.0 and greater (#7503) ## 📝 Documentation - docs(fargate): eksctl update command is deprecated, use upgrade instead (#7517) ## Acknowledgments The eksctl maintainers would like to sincerely thank: @skl --- ### Release Notes 0.171.0 (`docs/release_notes/0.171.0.md`) # Release v0.171.0 ## 🚀 Features - Add support for Ubuntu 22.04 based EKS images (#7516) ## 📝 Documentation - Announce eksctl Support Status Update on eksctl.io (#7539) ## Acknowledgments The eksctl maintainers would like to sincerely thank: @toabctl --- ### Release Notes 0.172.0 (`docs/release_notes/0.172.0.md`) # Release v0.172.0 ## 🎯 Improvements - Fix checks for updated addon versions (#7471) - Check for empty region before invoking API in AWS SDK (#7523) ## 🐛 Bug Fixes - Revert removing RetryMetricsHeader in presigned requests (#7563) ## 🧰 Maintenance - Bump dependencies (#7554) - Extract common workflow steps to set up build environment (#7551) ## Acknowledgments The eksctl maintainers would like to sincerely thank: @a2ush and @mttrb --- ### Release Notes 0.173.0 (`docs/release_notes/0.173.0.md`) # Release v0.173.0 ## 🚀 Features - Make EKS 1.29 default (#7599) ## 🎯 Improvements - Add profile flag support for get and delete access entry (#7623) - Remove inline ELB and CloudWatch policies from Cluster Role (#7603) ## 🐛 Bug Fixes - Increase STS session duration to fix caching credentials with assumed role MFA session (#7626) ## 🧰 Maintenance - Automate PR with release notes draft (#7604) ## 📝 Documentation - Update docs with Ubuntu 20.04 supported in EKS \<= 1.29 (#7618) ## Acknowledgments The eksctl maintainers would like to sincerely thank @ibnjunaid, @rpocase and @vutny. --- ### Release Notes 0.174.0 (`docs/release_notes/0.174.0.md`) # Release v0.174.0 ## 🎯 Improvements - Assign EFA network interface device index 1 to additional network cards (#7643) ## 🧰 Maintenance - Refactor release workflows (#7632) ## Acknowledgments The eksctl maintainers would like to sincerely thank No contributors. --- ### Release Notes 0.175.0 (`docs/release_notes/0.175.0.md`) # Release v0.175.0 ## 🐛 Bug Fixes - Upgrade Bottlerocket node with explicit `Version` field (#7666) --- ### Release Notes 0.176.0 (`docs/release_notes/0.176.0.md`) # Release v0.176.0 ## 🚀 Features - Add support for AMIs based on AmazonLinux2023 (#7684) ## 🎯 Improvements - Display full draft release notes in PR description (#7686) ## 🐛 Bug Fixes - Enforce `authenticationMode:CONFIG\_MAP` on Outposts (#7699) - [EKSCTL create cluster command] Authorise self-managed nodes via `aws-auth configmap` when EKS access entries are disabled (#7698) - Allow GPU instance types for Windows nodes (#7681) - Include MixedInstancesPolicy LaunchTemplate for validation (#7661) - Aim for namespace uniqueness across parallel integration tests specs (#7680) ## 🧰 Maintenance - Bump mkdocs version (#7696) - Bump dependencies (#7668) ## Acknowledgments The eksctl maintainers would like to sincerely thank @qclaogui. --- ### Release Notes 0.177.0 (`docs/release_notes/0.177.0.md`) # Release v0.177.0 ## 🚀 Features - Add utils command to migrate iamidentitymappings to EKS access entries (#7710) - Add support for Ubuntu Pro 22.04 based EKS images (#7711) ## 🎯 Improvements - Handle K8s service account lifecycle on eksctl create/delete podidentityassociation commands (#7706) ## 🐛 Bug Fixes - Fix reusing instanceRoleARN for nodegroups authorized with access entry (#7707) - Allow nodegroup creation after a cluster subnet is deleted (#7714) - Fix arn build logic to support different aws partitions (#7715) ## 🧰 Maintenance - Disable IMDSv1 in unowned integration tests (#7722) ## Acknowledgments The eksctl maintainers would like to sincerely thank @punkwalker, @veekaly, @aciba90 and @timandy --- ### Release Notes 0.178.0 (`docs/release_notes/0.178.0.md`) # Release v0.178.0 ## 🐛 Bug Fixes - Fix kubeletExtraConfig support for AL2023 (#7758) - Update aws-node from 1.12.6 to 1.18.1 (#7756) ## Acknowledgments The eksctl maintainers would like to sincerely thank @consideRatio --- ### Release Notes 0.179.0 (`docs/release_notes/0.179.0.md`) # Release v0.179.0 ## 🚀 Features - Support EKS 1.30 (#7781) --- ### Release Notes 0.18.0 (`docs/release_notes/0.18.0.md`) # Release 0.18.0 ## Features - Support private networking for managed nodegroups (#1791) ## Improvements - Use only Service Linked Role for EKS clusters (#2079) - Switch to `aws eks get-token` as the authenticator for Amazon Linux 2 (#2078) - Update Flux and Helm Operator (#2017) - Add completion support for fish (#2025) ## Bug fixes - Add name argument support for create iamserviceaccount cli (#2086) - pkg/nodebootstrap: Fix Dropped Error (#2019) - Don't alter addons in plan mode (#1990) - Fix deleting security groups for ELBs still being deleted (#2012) ## Announcements - EKS added Fargate Support in Frankfurt, Oregon, Singapore, and Sydney AWS Regions. eksctl supports these new regions ## Acknowledgments Weaveworks would like to sincerely thank: @alrs, @hikhvar, @sayboras and @stefanprodan --- ### Release Notes 0.180.0 (`docs/release_notes/0.180.0.md`) # Release v0.180.0 ## 🐛 Bug Fixes - Add option to create service account for pod identities which defaults to `false` (#7784) --- ### Release Notes 0.181.0 (`docs/release_notes/0.181.0.md`) # Release v0.181.0 ## 🚀 Features - EKS-Addons support for pod identity associations (#7807) --- ### Release Notes 0.182.0 (`docs/release_notes/0.182.0.md`) # Release v0.182.0 ## 🚀 Features - Added AWS G6 GPU instance support (#7819) ## 🐛 Bug Fixes - Fix upgrading AL2 ARM64 nodegroups (#7821) - Subnets availability validation should use AZs resolved by `EC2::DescribeSubnets` call (#7816) - Fix Github workflow for updating generated files for default addons (#7766) ## 📝 Documentation - Fix typo for iam policy (#7820) - Fix formatting for notes in documentation (#7817) ## Acknowledgments The eksctl maintainers would like to sincerely thank @practicusai and @richzw. --- ### Release Notes 0.183.0 (`docs/release_notes/0.183.0.md`) # Release v0.183.0 ## 🎯 Improvements - Make EKS 1.30 the default (#7827) --- ### Release Notes 0.184.0 (`docs/release_notes/0.184.0.md`) # Release v0.184.0 ## 🚀 Features - Cluster creation flexibility for default networking addons (#7866) ## 🎯 Improvements - use string in logging instead of wrapping error (#7838) - Stop using P2 instances which will be retired (#7826) ## 🧰 Maintenance - Fix SDK paginator mocks (#7850) - Schedule pods on a nodegroup on which no concurrent actions are executed (#7834) ## Acknowledgments The eksctl maintainers would like to sincerely thank @moreandres. --- ### Release Notes 0.185.0 (`docs/release_notes/0.185.0.md`) # Release v0.185.0 ## 🎯 Improvements - Avoid creating subnets in disallowed Availability Zone IDs (#7870) - Add auto-ssm ami resolution for ubuntu (#7851) - Add/hpc7g node arm support (#6743) - fix runAsNonRoot (true) efa device plugin bug (#6302) - fixed iam permissions bug for karpenter (#7778) ## Acknowledgments The eksctl maintainers would like to sincerely thank @aciba90, @siennathesane and @vsoch. --- ### Release Notes 0.186.0 (`docs/release_notes/0.186.0.md`) # Release v0.186.0 ## 🚀 Features - Allow limiting the number of nodegroups created in parallel (#7884) ## 🎯 Improvements - Retry throttling errors, disable retry rate-limiting (#7878) --- ### Release Notes 0.187.0 (`docs/release_notes/0.187.0.md`) # Release v0.187.0 ## 🐛 Bug Fixes - Restrict `VPC.SecurityGroup` egress rules validations to self-managed nodes (#7883) --- ### Release Notes 0.188.0 (`docs/release_notes/0.188.0.md`) # Release v0.188.0 ## 🚀 Features - Add support for Kuala Lumpur region (ap-southeast-5) (#7910) ## 🎯 Improvements - Add GH workflow for automatically updating nvidia device plugin static manifest (#7898) ## 🧰 Maintenance - Update nvidia-device-plugin to v0.16.0 (#7900) --- ### Release Notes 0.189.0 (`docs/release_notes/0.189.0.md`) # Release v0.189.0 ## 🐛 Bug Fixes - Skip creating OIDC manager for Outposts clusters (#7934) - Fixes segfault when VPC CNI is disabled (#7927) - Fix SSM unit tests (#7935) ## Acknowledgments The eksctl maintainers would like to sincerely thank @EmmEff. --- ### Release Notes 0.19.0 (`docs/release_notes/0.19.0.md`) # Release 0.19.0 ## Features - Support Amazon EKS Kubernetes version 1.16 (#2116) - Support metrics collection in scaling config for unmanaged node groups (#2055) - Add support for tagging iamserviceaccount (#1756) ## Improvements - Publish RCs to the homebrew tap (#2033) - Allow private clusters to be created (#2071) ## Bug Fixes - Validate `secretsEncryption.keyARN` for nil (#2110) - Fix Windows build (#2111) --- ### Release Notes 0.190.0 (`docs/release_notes/0.190.0.md`) # Release v0.190.0 ## 🧰 Maintenance - Bump github.com/docker/docker from 26.1.4+incompatible to 26.1.5+incompatible (#7936) - Bump jinja2 from 3.1.3 to 3.1.4 in /userdocs (#7748) --- ### Release Notes 0.191.0 (`docs/release_notes/0.191.0.md`) # Release v0.191.0 ## 🚀 Features - Add support for EKS 1.31 (#7973) --- ### Release Notes 0.192.0 (`docs/release_notes/0.192.0.md`) # Release v0.192.0 ## 🚀 Features - Add support for EKS accelerated AMIs based on AL2023 (#7996) ## 🎯 Improvements - cleanup efa installer archive before install (#6870) ## 🐛 Bug Fixes - Disallow `overrideBootstrapCommand` and `preBootstrapCommands` for MNG AL2023 (#7990) ## Acknowledgments The eksctl maintainers would like to sincerely thank @vsoch. --- ### Release Notes 0.193.0 (`docs/release_notes/0.193.0.md`) # Release v0.193.0 ## 🚀 Features - Add support for M8g instance types (#8001) ## 📝 Documentation - Documentation: correct version drift limits in cluster-upgrade.md (#7994) ## Acknowledgments The eksctl maintainers would like to sincerely thank @larvacea. --- ### Release Notes 0.194.0 (`docs/release_notes/0.194.0.md`) # Release v0.194.0 ## 🚀 Features - Support EKS zonal shift config (#8005) ## 🎯 Improvements - Fix missing ELB listener attribute actions required for AWS Load Balancer Controller v2.9.0 (#7988) ## Acknowledgments The eksctl maintainers would like to sincerely thank @jonathanfoster. --- ### Release Notes 0.195.0 (`docs/release_notes/0.195.0.md`) # Release v0.195.0 ## 🚀 Features - Support EKS Auto Mode (#8058) - Add support for EKS Hybrid Nodes (#8062) ## 🧰 Maintenance - Remove deletion of aws-efs-csi-driver (#8045) - Fix addons integration test (#8042) --- ### Release Notes 0.196.0 (`docs/release_notes/0.196.0.md`) # Release v0.196.0 ## 🧰 Maintenance - Reword IRSA warning message (#8040) ## 📝 Documentation - userdocs: fix comment for custom managed nodegroup (#8036) ## Acknowledgments The eksctl maintainers would like to sincerely thank @koooosh. --- ### Release Notes 0.197.0 (`docs/release_notes/0.197.0.md`) # Release v0.197.0 ## 🐛 Bug Fixes - Add IAM capability if a custom IAM role is provided for Auto Mode (#8071) ## 📝 Documentation - Cleanup Hybrid Nodes docs (#8072) ## Acknowledgments The eksctl maintainers would like to sincerely thank @geoffcline. --- ### Release Notes 0.198.0 (`docs/release_notes/0.198.0.md`) # Release v0.198.0 ## 🚀 Features - Add support for `preBootstrapCommands` in AL2023 (#8031) --- ### Release Notes 0.199.0 (`docs/release_notes/0.199.0.md`) # Release v0.199.0 ## 🚀 Features - Support EKS managed nodegroup node repair config (#8087) - Add support for C8g and R8g instance types (#8041) - Add support for `overrideBootstrapCommand` in AL2023 (#8078) ## 🐛 Bug Fixes - Allow importing subnets with IPv6 only (#8086) - EKS Auto Mode should not require setting default addons or OIDC (#8082) ## Acknowledgments The eksctl maintainers would like to sincerely thank @cheeseandcereal and @muhammetssen. --- ### Release Notes 0.2.0 (`docs/release_notes/0.2.0.md`) # Release 0.2.0 ## Announcement This release signifies end of initial development cycle of `eksctl`. We have built many new features since first minor release (0.1.0) in August 2018, and are switching to a [new release cadence][]. Albeit we have originally planned to do a number of things in 0.2.0, the plan has evolved. From now on, please refer to [our roadmap][]. [new release cadence]: https://eksctl.io/community/roadmap/ [our roadmap]: https://github.com/eksctl-io/eksctl/projects/2 ## Bug Fixes Since 0.1.40 - remove support for Kubernetes version 1.10, as [deprecated by AWS from 22nd July 2019 onwards](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html) [#1026] - remove `eksctl utils install-coredns` command that was only relevant to Kubernetes version 1.10 - remove `--strorage-class` flag that was only relevant to Kubernetes version 1.10 - fix long-standing issue with EBS and security groups blocking VPC deletion [#103] - add support for `aws eks get-token` as fallback when `aws-iam-authenticator` is missing [#788] - add `volumeIOPS` field to enable use of EBS root volumes of type `io1` [#1006] ## New Features Since 0.1.0 ### General features - create and manage clusters using configuration files - Kubernetes versions: 1.11, 1.12 and 1.13 - cluster upgrades - support for all EKS regions ### VPC - allow re-using existing VPCs - public and private subnet topologies - NAT Gateway support (HA, single or disable) ### nodegroup - nodegroup management (create, get, scale, delete) - spot instances and mixed instance types - extra kubelet configuration parameters - custom EC2 tags and Kubernetes labels - custom bootstrap logic - single-AZ nodegroups - public or private networking options - use existing security groups - root EBS volume size, type and encryption - various options for SSH access ### IAM - use custom service role for cluster - use custom roles or policies for nodegroups - canned policies for popular add-ons: - External DNS - Cluster Autoscaler - Cert Manager - ALB Ingress - AppMesh & XRay - EBS, EFS and xFS CSI - execute CloudFormation actions using dedicated role ### utilities - describe and troubleshoot CloudFormation stacks, with CloudTrail support - update default add-ons during cluster upgrades (`coredns`, `kube-proxy` and `aws-node`) - delete stale entries in `~/.kube/config` ## Acknowledgment Weaveworks team would like to sencirely thank all of the who have contributed to the project, your work is highly appreciated and the project would get thus far without you! We have received many very useful contributions from so many of you, some of which came as code, and some as issue or participation on Slack, we thank all of you!
List of most active contributors in the last 12 month @richardcase @christopherhein @mumoshu @adamjohnson01 @archisgore @Lazyshot @kschumy @toricls @cristian-radu @tedmiston @karinnainiguez @Chabane @tiffanyfay @yutachaos @gruebel @jstrachan @PaulMaddox @ozzieba @mukaibot @jmcarp @gotjosh @philoserf @pawelprazak @nckturner @af12066 @mgalgs @Jeffwan @paulbsch @arbourd @cbluth @knorby @gchaincl @mcfedr @overdrive3000 @prageethw @patstrom @sdarwin @unguiculus @Yannig @IPyandy @bowlesns @derwasp @mikeroyal @superseb @zironycho @dcherman @silviogutierrez @sixth @dresnick-sf @jonk @danielfm @denwwer @thapakazi @lucioveloso @callmeradical @manabusakai @justincormack @austbot @pdavies011010 @mreferre @flou @polothy @Tyil @cpaika @arun-gupta @cdenneen @danielchalef @mrichman @whereisaaron @StevenACoffman @procyclinsur @braderhart @aparamon @JasonSwindle @jicowan @olipachi @trondhindenes @milkowski @dingn1 @jamesalbert @stevepe-1 @ArseniiPetrovich @jvanzyl @redborian @tkang007 @elirankon @alexclifford @arielvinas @bnutt @voxxit
--- ### Release Notes 0.2.1 (`docs/release_notes/0.2.1.md`) # Release 0.2.1 ## Bug Fixes Since 0.2.0 - more explicit deprecation message for Kubernetes version 1.10 ([#1049](https://github.com/eksctl-io/eksctl/issues/1049)) ## Bug Fixes Since 0.1.40 - remove support for Kubernetes version 1.10, as [deprecated by AWS from 22nd July 2019 onwards](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html) ([#1026](https://github.com/eksctl-io/eksctl/issues/1026)) - remove `eksctl utils install-coredns` command that was only relevant to Kubernetes version 1.10 - remove `--strorage-class` flag that was only relevant to Kubernetes version 1.10 - fix long-standing issue with EBS and security groups blocking VPC deletion ([#103](https://github.com/eksctl-io/eksctl/issues/103)) - add support for `aws eks get-token` as fallback when `aws-iam-authenticator` is missing ([#788](https://github.com/eksctl-io/eksctl/issues/788)) - add `volumeIOPS` field to enable use of EBS root volumes of type `io1` ([#1006](https://github.com/eksctl-io/eksctl/issues/1006)) ## New Features Since 0.1.0 ### General features - create and manage clusters using configuration files - Kubernetes versions: 1.11, 1.12 and 1.13 - cluster upgrades - support for all EKS regions ### VPC - allow re-using existing VPCs - public and private subnet topologies - NAT Gateway support (HA, single or disable) ### nodegroup - nodegroup management (create, get, scale, delete) - spot instances and mixed instance types - extra kubelet configuration parameters - custom EC2 tags and Kubernetes labels - custom bootstrap logic - single-AZ nodegroups - public or private networking options - use existing security groups - root EBS volume size, type and encryption - various options for SSH access ### IAM - use custom service role for cluster - use custom roles or policies for nodegroups - canned policies for popular add-ons: - External DNS - Cluster Autoscaler - Cert Manager - ALB Ingress - AppMesh & XRay - EBS, EFS and xFS CSI - execute CloudFormation actions using dedicated role ### utilities - describe and troubleshoot CloudFormation stacks, with CloudTrail support - update default add-ons during cluster upgrades (`coredns`, `kube-proxy` and `aws-node`) - delete stale entries in `~/.kube/config` ## Acknowledgment Weaveworks team would like to sencirely thank all of the who have contributed to the project, your work is highly appreciated and the project would get thus far without you! We have received many very useful contributions from so many of you, some of which came as code, and some as issue or participation on Slack, we thank all of you!
List of most active contributors in the last 12 month @richardcase @christopherhein @mumoshu @adamjohnson01 @archisgore @Lazyshot @kschumy @toricls @cristian-radu @tedmiston @karinnainiguez @Chabane @tiffanyfay @yutachaos @gruebel @jstrachan @PaulMaddox @ozzieba @mukaibot @jmcarp @gotjosh @philoserf @pawelprazak @nckturner @af12066 @mgalgs @Jeffwan @paulbsch @arbourd @cbluth @knorby @gchaincl @mcfedr @overdrive3000 @prageethw @patstrom @sdarwin @unguiculus @Yannig @IPyandy @bowlesns @derwasp @mikeroyal @superseb @zironycho @dcherman @silviogutierrez @sixth @dresnick-sf @jonk @danielfm @denwwer @thapakazi @lucioveloso @callmeradical @manabusakai @justincormack @austbot @pdavies011010 @mreferre @flou @polothy @Tyil @cpaika @arun-gupta @cdenneen @danielchalef @mrichman @whereisaaron @StevenACoffman @procyclinsur @braderhart @aparamon @JasonSwindle @jicowan @olipachi @trondhindenes @milkowski @dingn1 @jamesalbert @stevepe-1 @ArseniiPetrovich @jvanzyl @redborian @tkang007 @elirankon @alexclifford @arielvinas @bnutt @voxxit
--- ### Release Notes 0.20.0 (`docs/release_notes/0.20.0.md`) # Release 0.20.0 ## Features - Add support for GovCloud regions (#2190) - Add nodes-min and nodes-max flags in ng scale (#2022) ## Improvements - Add name argument for delete iamserviceaccount (#2165) - List supported k8s versions (#2126) ## Bug Fixes - Fix Fargate on Kubernetes 1.16 (#2183) - Fix flags issue in version cli (#2187) - Filter only kops ec2 resource with type ec2.ResourceTypeSubnet (#1752) ## Acknowledgments Weaveworks would like to sincerely thank: @agbanagba, @justinclayton, @polanfong, @rimaulana, @sayboras and @smrutiranjantripathy --- ### Release Notes 0.200.0 (`docs/release_notes/0.200.0.md`) # Release v0.200.0 ## 🚀 Features - Support for Bangkok region (`ap-southeast-7`) (#8114) --- ### Release Notes 0.201.0 (`docs/release_notes/0.201.0.md`) # Release v0.201.0 ## 🚀 Features - Ubuntu 24.04 support (#8094) - Support create cluster default addon metrics-server (#8118) - Support for Mexico region (mx-central-1) (#8117) ## 🐛 Bug Fixes - Remove EFA installation scripts; EFA is installed by default on EKS AL2 GPU, EKS Al2023 NVIDIA, and EKS AL2023 Neuron AMIs (#8113) - Repair AL2023 with IPv6 node creation (#8120) ## 🧰 Maintenance - Update nvidia-device-plugin to v0.17.0 (#7918) - Update aws-node to v1.19.2 (#7899) - Update coredns (#7901) ## 📝 Documentation - Update doc dependencies and supported Python runtime to publish latest doc updates (#8119) ## Acknowledgments The eksctl maintainers would like to sincerely thank @bryantbiggs, @gustavodiaz7722 and @toabctl. --- ### Release Notes 0.202.0 (`docs/release_notes/0.202.0.md`) # Release v0.202.0 ## 🚀 Features - Support for us-isof and eu-isoe regions (#8151) - Enable default addon creation for auto mode clusters (#8140) ## 🎯 Improvements - Use `DescribeClusterVersions` API instead of hardcoding EKS version data (#8144) ## 🐛 Bug Fixes - Wait for vpccni to become active before updating it to use IRSA (#8152) - Update import path of local goformation fork, fixing `go mod verify` (#8148) - Disable metrics-server default addon creation for unsupported regions (#8146) ## Acknowledgments The eksctl maintainers would like to sincerely thank @jdwtf --- ### Release Notes 0.203.0 (`docs/release_notes/0.203.0.md`) # Release v0.203.0 ## 🎯 Improvements - [Hybrid Nodes] Remove strict requirement of providing a VGW or TGW during cluster creation (#8168) ## 🐛 Bug Fixes - Resolve clusterDNS for IPv6 MNG AL2023 with custom AMI (#8170) - [Hybrid Nodes] Make `remoteNetworkConfig.IAM.Provider` field case insensitive (#8169) --- ### Release Notes 0.204.0 (`docs/release_notes/0.204.0.md`) # Release v0.204.0 ## 🚀 Features - Enable NVIDIA Plugin automatically for AL2023 GPU AMIs (#8199) - Added EBS IO2 volumeType support to Managed NodeGroup (#7989) - Enable Bottlerocket on Neuron Instance types (Inferentia and Trainium) (#8173) ## 🎯 Improvements - pre-Generate EC2 instance type details out of band (#8206) - New EFA plugin images need a new volume for EFA plugin to mount - `/dev/infiniband` (#8201) - Add addon name to some logging lines (#8200) ## 🧰 Maintenance - Update building with golang 1.24 (#8210) - Update aws-node to v1.19.2 (#8160) ## Acknowledgments The eksctl maintainers would like to sincerely thank @BhowmickSumit, @Juneezee, @alexandear, @dims and @vigh-m. --- ### Release Notes 0.205.0 (`docs/release_notes/0.205.0.md`) # Release v0.205.0 ## 🚀 Features - Add support for creating access entries of type `EC2` for EKS Auto Mode (#8228) - Support for setting MarketType/InstanceMarketOptions for use with capacity-block (#8207) ## 🎯 Improvements - Update eksctl api schema for neuron devices (#8255) - Fix go.sum; check go.mod and go.sum on CI (#8254) - Switch default for ExecConfig api version from v1alpha1->v1beta1 (#8252) - Allow separate explicit selection of Neuron Devices (separate from GPUs) in api (#8243) - Add support for allow/deny regexp(s) in instanceSelector (#8247) - Re-add support for env var keys from AWS SDK (#8231) ## 🐛 Bug Fixes - Allow adding remote network config to private clusters (#8261) - Re-allow use of the `cnn1-az4` availability zone (#8240) - Add missing coredns-1.32 json file (#8233) - Print the computed upgrade version correctly (#8232) - Remove support for p2 instances which require NVIDIA driver \<= 470 (#8226) - Fix issue installing the nvidia device plugin unconditionally on al2023 (#8225) - Do not create a placement group when using a reservation (#8220) ## 🧰 Maintenance - Update aws-node to v1.19.3 (#8258) ## 📝 Documentation - Update Karpenter usage documentation to reflect latest supported version(s) (#8250) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey, @alexandear, @bryantbiggs, @cenkalti, @dims, @escardoso and @vsoch. --- ### Release Notes 0.206.0 (`docs/release_notes/0.206.0.md`) # Release v0.206.0 ## 🚀 Features - Allow specification of IAM permission boundary for Auto Mode's Node Role (#8307) ## 🎯 Improvements - Default to newest k8s 1.32 version (#8300) - Support for EKS internal environments (#8294) - Update goformation from latest schema / Remove unused generated code (#8289) - Reduce blast radius of test that disables api server public access failing (#8288) - Fix failure when VPC CNI is configured to use both iam.withOIDC and useDefaultPodIdentityAssociations (#8268) - Update zz\_generated.deepcopy.go, also add new make target for the same (#8267) ## 🐛 Bug Fixes - Skip instance type support validation for custom AMI ID (#8303) - Allow core/default networking addons on auto-mode clusters with mixed node types (#8293) - Standardize on k8s 1.32.2 for go.mod (#8285) ## 🧰 Maintenance - Add support for i8g.48xlarge ec2 instance type (#8301) - Update nvidia-device-plugin to v0.17.1 (#8296) - Update ec2-info (#8297) - Update ec2-info (#8290) ## Acknowledgments The eksctl maintainers would like to sincerely thank @amdonov, and @ndbaker1. --- ### Release Notes 0.207.0 (`docs/release_notes/0.207.0.md`) # Release v0.207.0 ## 🚀 Features - Add support for new force parameter when upgrading cluster version (#8319) ## 🎯 Improvements - Enable creating IPV6 clusters with pod identities in addition to IRSA (#8322) - Add latest changes to AWS Load Balancer Controller IAM Policy (#8316) ## 🧰 Maintenance - Fully remove intermediate eksctl build image (#8341) - Remove redundant 'typecheck' linter (#8298) ## Acknowledgments The eksctl maintainers would like to sincerely thank @MartinEmrich, @alexandear, @rsumukha and @simonmarty. --- ### Release Notes 0.208.0 (`docs/release_notes/0.208.0.md`) # Release v0.208.0 ## 🚀 Features - Enable enclave options in EC2 Launch Template (#8349) - Add support for AL2023 arm64 nvidia amiType (#8351) ## 🎯 Improvements - Add AmazonLinux2 end-of-life warning message when creating clusters and nodegroups (#8376) ## 🐛 Bug Fixes - Fix nil pointer dereference when checking if auto mode enabled (#8378) - Fix for `create podidentityassociation` bug in auto-mode clusters (#8358) - Fix CoreDNS scheduling on Fargate failing when component label in selector (#8357) - Fix support for non-current-generation ec2 instance types (#8354) - Fix get iamserviceaccount returning error for unrelated CloudFormation Stack (#8350) ## 🧰 Maintenance - Update coredns default to v1.11.4-eksbuild.10 (#8363) - Update ec2-info (#8361) - Drop support for Ubuntu 18.04 images (#8344) ## 📝 Documentation - Update binary install for best practices (#8359) ## Acknowledgments The eksctl maintainers would like to sincerely thank @cloudxabide, @dheeraj-coding, @naclonts, @ndbaker1, @tarikdem and @toabctl. --- ### Release Notes 0.209.0 (`docs/release_notes/0.209.0.md`) # Release v0.209.0 ## 🎯 Improvements - Add eksctl support for new AWS region ap-east-2 (#8406) ## 🐛 Bug Fixes - Use AmazonLinux2023 as default AMI instead of AmazonLinux2 for Kubernetes version >=1.33 (#8405) - Fix missing permission setup for IPv6 cluster (#8387) ## 🧰 Maintenance - Update aws-node to v1.19.5 (#8377) - Update nvidia-device-plugin to v0.17.2 (#8381) - Update ec2-info (#8382) - Update coredns (#8385) ## 📝 Documentation - Fix missing permission setup for IPv6 cluster (#8387) ## Acknowledgments The eksctl maintainers would like to sincerely thank @guessi and @naclonts. --- ### Release Notes 0.21.0 (`docs/release_notes/0.21.0.md`) # Release 0.21.0 ## Features - Ability of changing the node instance name (#1796) - Lock kubeconfig file before accessing (#2230) ## Improvements - Use apps/v1 for Deployment for all K8s versions (#2238) - Propagate SSH_AUTH_SOCK environment variable to git executor (#2083) ## Bug Fixes - Try to delete nodegroup for any nodegroup state (#2252) - Fix bug in supporting non-default branches for gitops repos (#2246) - Don't run extra cluster tasks and nodegroup creation in parallel (#2248) - Fix update-cluster-endpoints panic when using a cluster config with missing VPC section (#2253) ## Acknowledgments Weaveworks would like to sincerely thank: @aaronjwood, @danil-smirnov, @hiddeco, @mikestef9, @rndstr, @rodrigc and @spaghettifunk --- ### Release Notes 0.210.0 (`docs/release_notes/0.210.0.md`) # Release v0.210.0 ## 🎯 Improvements - Change node role policy ReadOnly -> PullOnly (support ECR pull-through cache) (#8386) ## 🐛 Bug Fixes - Fix Clusters Fail to Create Private Endpoint and Identity Provider (#8411) ## Acknowledgments The eksctl maintainers would like to sincerely thank @amdonov and @artem-nefedov. --- ### Release Notes 0.211.0 (`docs/release_notes/0.211.0.md`) # Release v0.211.0 ## 🎯 Improvements - Migrate EBS CSI policy to AmazonEBSCSIDriverPolicy managed policy (#8422) - Add support for custom Bottlerocket AMIs for MNG (#8418) - Implementation of MCP server for eksctl (#8414) ## 🐛 Bug Fixes - Add DescribeRouteTables to load balancer controller permissions (#8447) ## 🧰 Maintenance - Update coredns (#8437) - Update aws-node to v1.20.0 (#8426) - Update ec2-info (#8427) ## 📝 Documentation - Update schema.md (#8443) - add note about docs migration (#8441) - Add announcement for docs on AWS, remove old announcements (#8442) - Fix link URL for privateNetworking (#8416) ## Acknowledgments The eksctl maintainers would like to sincerely thank @ConnorJC3, @NicholasBlaskey, @geoffcline, @koooosh and @mgalgs. --- ### Release Notes 0.212.0 (`docs/release_notes/0.212.0.md`) # Release v0.212.0 ## 🚀 Features - Add support for Cross Account Pod Identity Associations (#8425) ## Acknowledgments The eksctl maintainers would like to sincerely thank @naclonts. --- ### Release Notes 0.213.0 (`docs/release_notes/0.213.0.md`) # Release v0.213.0 ## 🚀 Features - Add namespace configuration support for EKS addons (#8476) ## 🐛 Bug Fixes - remove namespace validation for update-addon (#8479) ## Acknowledgments The eksctl maintainers would like to sincerely thank @sapphirew. --- ### Release Notes 0.214.0 (`docs/release_notes/0.214.0.md`) # Release v0.214.0 ## 🎯 Improvements - add notify homebrew update workflow (#8482) ## 🐛 Bug Fixes - Add exact match check to use specified addon versions (#8478) ## Acknowledgments The eksctl maintainers would like to sincerely thank @aoshi2000 and @sapphirew. --- ### Release Notes 0.215.0 (`docs/release_notes/0.215.0.md`) # Release v0.215.0 ## 🚀 Features - automatic security group tagging for karpenter (#8501) ## 🎯 Improvements - fix: use default security group for eks 1.33+ when EFA is enabled (#8509) - fix: karpenter tagging, addons and flaky integration tests (#8508) ## 🐛 Bug Fixes - fix: match cloudformation stack name using suffix for pod identity association (#8511) - fix: Add exact match check to use specified addon versions (#8510) - fix: Fix GPU instance filtering when --instance-selector-gpus=0 (#8506) - fix: use latest managed addon version for kube-proxy image name (#8505) ## 🧰 Maintenance - Update aws-node to v1.20.3 (#8471) - Update nvidia-device-plugin to v0.17.4 (#8453) - Update coredns (#8472) - Update ec2-info (#8454) ## 📝 Documentation - docs: Add Auto Mode behavior change notices (#8513) - Revert Add scheduled maintenance announcement for eksctl.io (#8500) - Add scheduled maintenance announcement for eksctl.io (#8498) ## Acknowledgments The eksctl maintainers would like to sincerely thank and @sapphirew. --- ### Release Notes 0.216.0 (`docs/release_notes/0.216.0.md`) # Release v0.216.0 ## 🚀 Features - Add support for ap-southeast-6 (#8544) - feat: add support for UpgradePolicy attribute in cluster creation (#8534) ## 🎯 Improvements - Fix upgrade vpc-cni test relying on assumptions about vpc versions (#8562) - Fix integration test by waiting 2 minutes for ebs update (#8561) - Fix test failure for add-on waiting for ebs csi driver (#8559) - For dns test, use an ECR busybox image (#8560) - Don't check access entry length to fix failed intg test (#8558) - Don't wait for metrics-server if its the only add-on (#8552) - Migrate completly off V1 SDK to V2 SDK (#8548) ## 🐛 Bug Fixes - Make update kube-proxy not use minimal images for >= 1.34 (#8557) - Remove kubernetes.io/cluster/\ tag from EFA security group (#8556) - Create EFA specific security group for self-managed node groups always (#8554) - Make metrics server create after node groups so it can get scheduled … (#8538) ## 🧰 Maintenance - Don't check access entry length to fix failed intg test (#8558) - Update coredns (#8537) - Migrate completly off V1 SDK to V2 SDK (#8548) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey and @guessi. --- ### Release Notes 0.217.0 (`docs/release_notes/0.217.0.md`) # Release v0.217.0 ## 🚀 Features - feat: Add enhanced node repair configuration support (#8512) ## 🐛 Bug Fixes - Fix issue with upgrade policy not being applied (#8577) ## 🧰 Maintenance - Update nvidia-device-plugin to v0.18.0 (#8549) - Update ec2-info (#8536) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey, and @sapphirew. --- ### Release Notes 0.218.0 (`docs/release_notes/0.218.0.md`) # Release v0.218.0 ## 🚀 Features - Add support for ControlPlaneScalingConfig (#8590) ## 🎯 Improvements - Enable termination protection by default for eksctl CFN stacks (#8586) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey and @abhay-krishna. --- ### Release Notes 0.219.0 (`docs/release_notes/0.219.0.md`) # Release v0.219.0 ## 💥 Breaking Changes - Make AL2023 be the default for all k8s version to handle AL2 deprecation (#8593) ## 🚀 Features - support Al2023 on outposts (#8599) ## 🎯 Improvements - Make AL2023 be the default for all k8s version to handle AL2 deprecation (#8593) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey. --- ### Release Notes 0.22.0 (`docs/release_notes/0.22.0.md`) # Release 0.22.0 ## Features - Increase default volume size to 80G (#2317) - Add support for App Mesh Preview Environment (#2310) - Rename update cluster to upgrade cluster (#2303) - Add KUBECONFIG_USE_SYSTEM_CA env variable to use the system CA instead of the EKS one (#2294) ## Improvements - Allow StackCollection.DeleteStackByName() to succeed (#2308) - Do Fargate setup before nodegroup creation (#2304) ## Acknowledgments Weaveworks would like to sincerely thank: @rndstr and @stefanprodan --- ### Release Notes 0.220.0 (`docs/release_notes/0.220.0.md`) # Release v0.220.0 ## 🚀 Features - Add support for EKS Capability (#8603) ## Acknowledgments The eksctl maintainers would like to sincerely thank @punkwalker. --- ### Release Notes 0.221.0 (`docs/release_notes/0.221.0.md`) # Release v0.221.0 ## 🎯 Improvements - added support us-isob-west-1 region (#8602) ## 🐛 Bug Fixes - Fix failures in cluster deletion in regions without capability support (#8631) - Add taints for AL2023 NodeGroups as tolerations for Nvidia device plugin daemonset (#8627) ## 🧰 Maintenance - Update ec2-info (#8578) ## Acknowledgments The eksctl maintainers would like to sincerely thank @NicholasBlaskey, @jdwtf and @knottnt. --- ### Release Notes 0.222.0 (`docs/release_notes/0.222.0.md`) # Release v0.222.0 ## 🚀 Features - Add support for EKS Windows Server 2025 in eksctl (#8648) - Allow wildcard matching of service account subject (#8629) ## 🎯 Improvements - Bump aws-sdk-go-v2 versions to support aws login (#8668) - Update default cluster version to Kubernetes v1.34 (#8645) ## 🐛 Bug Fixes - replace AmazonLinux2 amis with AmazonLinux2023 | remove override-bootstrap test in custom\_ami tests (#8666) - fix integration test with eks default version change (#8665) - only create iam role stack for capability if the roleArn is not provided (#8655) - fix ssm resolver ami test with addition of bottleRocket nvidia fips amis (#8656) ## Acknowledgments The eksctl maintainers would like to sincerely thank @KlwntSingh, @avoidik, @cdirubbio, @kprahulraj and @naclonts. --- ### Release Notes 0.223.0 (`docs/release_notes/0.223.0.md`) # Release v0.223.0 ## 🚀 Features - add support to policy for pod-identity-association create and update (#8674) ## 🐛 Bug Fixes - change ubuntu ami family to 2404 (#8670) ## Acknowledgments The eksctl maintainers would like to sincerely thank @kprahulraj. --- ### Release Notes 0.224.0 (`docs/release_notes/0.224.0.md`) # Release v0.224.0 ## 🚀 Features - Add Gateway API ELB cleanup support for eksctl cluster deletion (#8664) - Update go mod and go sum to use the latest nodeAdm api (#8677) ## 🎯 Improvements - Add Gateway API ELB cleanup support for eksctl cluster deletion (#8664) ## 🐛 Bug Fixes - add missing iam permission for enabling karpenter | fix karpenter discovery tags in subnets and security groups (#8684) ## 🧰 Maintenance - Update coredns (#8651) ## Acknowledgments The eksctl maintainers would like to sincerely thank @kprahulraj, @prasad0896 and @shraddhabang. --- ### Release Notes 0.225.0 (`docs/release_notes/0.225.0.md`) # Release v0.225.0 ## 🚀 Features - Honour permissionPolicyName for pod identity inline IAM policies (#8675) - feat: add fargate profile support for EKS internal endpoint (#8694) - feat: add permissions for ECR Public container pulls to Auto Mode NodeRole (#8698) ## 🐛 Bug Fixes - fix(goformation): Remove redundant condition in property.go (#8691) ## Acknowledgments The eksctl maintainers would like to sincerely thank @HusainZafar, @ShiriNmi1520, @avoidik and @fletcherw. --- ### Release Notes 0.226.0 (`docs/release_notes/0.226.0.md`) # Release v0.226.0 ## 🚀 Features - Adding support for upgrading eks hybrid nodes on EKS clusters (#8719) - feat(ubuntu): add support for resolute (#8683) ## 🐛 Bug Fixes - Populate role ARN from stack outputs on NoChangeError in pod identity update (#8717) - fix: only add ECR Public perms in aws partition (#8709) ## 🧰 Maintenance - Update aws-node to v1.21.1 (#8541) - Update ec2-info (#8626) - Update nvidia-device-plugin to v0.19.0 (#8647) - Update coredns (#8699) ## 📝 Documentation - fix: auto-detect system architecture instead of hardcoding (#8708) - karpenter: clarify createServiceAccount semantics in docs, schema, and runtime log (#8720) - Update Windows compatibility hint to recommend AL2023 instead of AL2 (#8714) ## Acknowledgments The eksctl maintainers would like to sincerely thank @cloudsmithy, @cs-mehta, @fletcherw, @guessi, @pokearu and @rpocase. --- ### Release Notes 0.227.0 (`docs/release_notes/0.227.0.md`) # Release v0.227.0 ## 🐛 Bug Fixes - fix: missing Karpenter permission (#8749) - fix: Regionalize all container images in UseRegionalImage() (#8744) ## Acknowledgments The eksctl maintainers would like to sincerely thank @candonov, and @sapphirew. --- ### Release Notes 0.228.0 (`docs/release_notes/0.228.0.md`) # Release v0.228.0 ## 🚀 Features - feat: add support for controlPlaneEgressMode (#8770) ## Acknowledgments The eksctl maintainers would like to sincerely thank @cxbrowne1207. --- ### Release Notes 0.229.0 (`docs/release_notes/0.229.0.md`) # Release v0.229.0 ## 🚀 Features - Add cluster version rollback support with rollbackConfig (#8780) - Add support for EKS on Outposts with EC2 instance store (#8774) ## Acknowledgments The eksctl maintainers would like to sincerely thank @DimitriPL5 and @michaelhtm. --- ### Release Notes 0.23.0 (`docs/release_notes/0.23.0.md`) # Release 0.23.0 ## Features - Add support for config file in scale nodegroup (#2257) - Mark `utils wait-nodes` as deprecated and hidden (#2344) ## Improvements - Use IMDSv2 (#2354) - Config schema doc improvements (#2347) - Sort commands/subcommands by name in help (#2365) - Remove nodegroups from authconfigmap after deletion, not before (#2343) ## Bug Fixes - Use compute-type of existing coredns on update (#2355) - Fix nodebootstrap asset generation (#2353) ## Acknowledgments Weaveworks would like to sincerely thank: @TBBle, @rndstr and @sayboras --- ### Release Notes 0.230.0 (`docs/release_notes/0.230.0.md`) # Release v0.230.0 ## 🚀 Features - Add control plane component config support (#8821) - Add support for BottlerocketFips AMI family in managed nodegroups (#8778) ## 🎯 Improvements - Make vpc-cni and kube-proxy optional when creating IPv6 clusters (#8790) ## 🐛 Bug Fixes - Grant ec2:DescribeSubnets to the IPv6 VPC CNI role (#8835) - Bump oras-go to 2.6.2 to resolve five open advisories (#8829) ## 🧰 Maintenance - Update aws-node to v1.23.0 (#8832) - Update coredns (#8833) - Update ec2-info (#8723) - Update coredns (#8724) - Update aws-node to v1.22.4 (#8747) - Update nvidia-device-plugin to v0.19.3 (#8722) ## Acknowledgments The eksctl maintainers would like to sincerely thank @41ks, @amdonov, @chithreshazad, @gustavodiaz7722, @chithresha and @michaelbeaumont. --- ### Release Notes 0.231.0 (`docs/release_notes/0.231.0.md`) # Release v0.231.0 ## 🚀 Features - Add connectionTracking support to nodegroup config (#8872) - Add TerminatedPodGcThreshold to control plane component config (#8853) - Add support for vpc.controlPlaneOnPrivateSubnets (#8793) ## 🎯 Improvements - chore: harden addon warning logging and empty drain (#8869) - ci: harden the release pipeline — isolate EKSCTLBOT\_TOKEN, sign releases, publish SBOMs (#8863) - ci: pass workflow inputs to run steps via env vars (#8860) ## 🐛 Bug Fixes - fix: resolve latest addon version before pod identity lookup (#8867) - chore: harden addon warning logging and empty drain (#8869) - fix: use interruptionQueue Helm value for Karpenter >= v0.33.0 (#8844) - Grant ec2:DescribeSecurityGroups to the IPv6 VPC CNI role (#8837) ## 🧰 Maintenance - ci: harden the release pipeline — isolate EKSCTLBOT\_TOKEN, sign releases, publish SBOMs (#8863) - ci: pass workflow inputs to run steps via env vars (#8860) - Update ec2-info (#8854) - Update coredns (#8842) - Update coredns (#8841) - Update nvidia-device-plugin to v0.20.0 (#8840) ## 📝 Documentation - docs: explain addon version differences (#8868) ## Acknowledgments The eksctl maintainers would like to sincerely thank @atharvaHJoshi, @chithreshazad, @guessi, @gustavodiaz7722, @sapphirew, @warren830, @chithresha and @michaelbeaumont. --- ### Release Notes 0.24.0 (`docs/release_notes/0.24.0.md`) # Release 0.24.0 ## Features - Add support for creating fully-private clusters (#2384) - Add support for Kubernetes 1.17 (#2407) - Upgrade AWS CNI to 1.6.3 (#2408) - Added T3 Unlimited support (#2391) - Added `kubeReserved` calculation support for mixed instance NodeGroups (#2378) ## Improvements - Added KMS resource to minimum policies documentation (#2402) - Add schema docs for git, fargate and cloudwatch (#2394) ## Bug Fixes - Fix name argument for scale nodegroup (#2417) - Fix nil pointer dereference in scale command (#2416) - Fix cluster upgrading to default version always (#2414) ## Acknowledgments Weaveworks would like to sincerely thank: @ajayk, @efim-a-efim, @elementalvoid, @mfykmn, and @sayboras --- ### Release Notes 0.25.0 (`docs/release_notes/0.25.0.md`) # Release 0.25.0 ## Features - Add disableIMDSv1 config option (#2460) - Make 1.17 default (#2444) - Add support for Milan (eu-south-1) and Cape Town (af-south-1) regions (#2495) ## Improvements - Cleanup ALBs created by AWS ALB Ingress Controller (#2465) - Don't replace service account with update-aws-node (#2459) - Updated default kubelet reserved memory calculation to match the calculation used by amazon-eks-ami (#2443) - Don't attach unnecessary policy to cluster IAM role (#2451) ## Bug Fixes - Restore "auto" version for upgrade cluster (#2464) - Allow service role some actions necessary for ELB (#2492) ## Acknowledgments Weaveworks would like to sincerely thank: @agbanagba, @brianpursley, @ismailyenigul, @lloydchang, @mfykmn, @morancj, @nckturner and @programmer04 --- ### Release Notes 0.26.0 (`docs/release_notes/0.26.0.md`) # Release 0.26.0 ## Features - Add support for launch templates, custom AMI and more features for managed nodegroups (#2544) - Add support for ARM (#2535) ## Improvements - Allow `version: auto` in the config file when creating a cluster (#2505) ## Acknowledgments Weaveworks would like to sincerely thank all our contributors! --- ### Release Notes 0.27.0 (`docs/release_notes/0.27.0.md`) # Release 0.27.0 ## Features - Add support for mapAccounts to `create iamidentitymapping` (#2549) - Added ASG SuspendProcesses support (#2487) ## Improvements - Update maxpods (#2580) - Support for managed VPC controller (#2616) ## Bug Fixes - Fix route validation for subnets with an explicit main route table association (#2577) - Use partition prefix for VPC endpoint services (#2576) - Add missing permission required for ELB (#2571) - Fix upgrading nodegroups with old template format (#2572) - Respect --ssh-access flag (#2540) ## Acknowledgments Weaveworks would like to sincerely thank: @klausenbusk and @efim-a-efim --- ### Release Notes 0.28.0 (`docs/release_notes/0.28.0.md`) # Release 0.28.0 ## Features - Add option to block IMDS from pods (#2634) - Create aws-node SA if WithOIDC (#2612) ## Improvements - Add life-cycle labels to nodes (#2615) - Add support for Windows 2004 Server AMI (#2613) ## Acknowledgments Weaveworks would like to sincerely thank: @vaibhav-random --- ### Release Notes 0.29.0 (`docs/release_notes/0.29.0.md`) # Release 0.29.0 ## Features - Add support for setting Service CIDR range (#2546) - Add placement group support for nodegroups (#2648) - Add an annotations field to the eksctl config file (#2632) ## Improvements - Update aws-node to 1.7 (#2672) - Respect the pod termination grace period when draining a nodegroup (#2660) - Update maxpods (#2670) - Go 1.15 upgrade (#2652) - Add example for kubeflow spot instance cluster config (#2640) ## Bug Fixes - Fix `delete iamserviceaccount` and withOIDC (#2671) - Don't require addons when checking for updates (#2667) ## Acknowledgments Weaveworks would like to sincerely thank: @laurovenancio, @hikhvar, @arjun921, @ajayk and @kreuzwerker-patrick --- ### Release Notes 0.29.1 (`docs/release_notes/0.29.1.md`) # Release 0.29.1 ## Bug Fixes - Do OIDC provider and SA creation before nodegroups (#2703) ## Acknowledgments Weaveworks would like to sincerely thank our contributors! --- ### Release Notes 0.29.2 (`docs/release_notes/0.29.2.md`) # Release 0.29.2 ## Bug Fixes - Fix kubernetesNetworkConfig with new nodegroups for existing clusters (#2713) ## Acknowledgments Weaveworks would like to sincerely thank our contributors! --- ### Release Notes 0.3.0 (`docs/release_notes/0.3.0.md`) # Release 0.3.0 ## Features - [EXPERIMENTAL] add `eksctl install flux` command (#1027) (#1055) - add a new command: `eksctl install flux`, which, given a Git repository and an EKS cluster, will bootstrap Flux, installing it in the cluster and initializing its manifests in the Git repository. To use this command, the environment variable `EKSCTL_EXPERIMENTAL=true` must be set - allow enabling EKS control plane logging to CloudWatch (#704) - upgrade AWS VPC CNI plugin to 1.5 (#927) - allow customizing kubelet feature gates (https://github.com/eksctl-io/eksctl/issues/784#issuecomment-513704672) - add support for Asia Pacific (Hong Kong) region (ap-east-1) ## Bug Fixes and code improvements - code refactor and declare commands in order of importance (#1061) --- ### Release Notes 0.30.0 (`docs/release_notes/0.30.0.md`) # Release 0.30.0 ## Features - Add support for Kubernetes 1.18 (#2723) - Support arm/arm64 binaries as part of release (#2725) ## Improvements - Update aws-node (#2720) - Update maxpods with p4d.24xlarge (#2715) - Handle existing instanceRoleARN with nested resource path (#2689) ## Bug fixes - Trim .git suffix when adding deploy keys (#2709) - Update nodegroup stack w/ version tag before updating release version (#2708) - Handle aws-node 1.7 initContainer properly (#2732) - Don't error if explicit aws-node given with withOIDC (#2750) ## Note We plan on removing the static node AMI resolver (`--node-ami`/`nodegroups.ami`) in `0.33.0`. Please reach out in #2650 if you have feedback. ## Acknowledgments Weaveworks would like to sincerely thank: @ajayk, @neha-viswanathan and @dmcneil --- ### Release Notes 0.31.0 (`docs/release_notes/0.31.0.md`) # Release 0.31.0 ## Features - Add Ubuntu 20.04 Image Support (#2758) - Add flag to enable Simple System Manager (SSM) for easy SSH access (#2754) - Add format option for utils describe-stacks (#2718) - Add option to not rollback failed stacks (#2744) - Add support for powershell completion (#2726) ## Improvements - Improving handling of failed nodegroups (#2762) - Return non-zero when a given nodegroup doesn't exist (#2731) - Increase cluster upgrade timeout to 50 minutes ## Bug Fixes - Ensure node drain timeout is respected (#2759) - Enforce control plane readiness before SA creation (#2760) - Handle create nodegroup and withOIDC for clusters without aws-node SA (#2778) ## Acknowledgments Weaveworks would like to sincerely thank: @aclevername, @neha-viswanathan, @sayboras, @powersj and @ajayk --- ### Release Notes 0.32.0 (`docs/release_notes/0.32.0.md`) # Release 0.32.0 ## Features - Add more flexible subnet support, specifying subnets for nodegroup (#2793) - Automatically install nvidia device plugin for GPU instances (#2807) - Support more than 4 AZs (#2804) - Support passing additional arguments to Flux and Helm Operator (#2702) - Allow creation of iamserviceaccounts on clusters not created through eksctl (#2775) ## Improvements - Update maxpods (#2821) - Upgrade flux, helm-operator and go-git-providers (#2802) - Make EKS 1.18 the default (#2857) ## Bug Fixes - Upgrade spf13/cobra to v1.1.1 to fix zsh completion issue (#2813) - Don't configure an SSH SecurityGroup when SSH.Allow is false (#2795) - Use role names instead of ARN when creating instance profiles (#2791) ## Acknowledgments Weaveworks would like to sincerely thank: @jfautley, @martina-if, @mgreensmith and @ysam12345 --- ### Release Notes 0.33.0 (`docs/release_notes/0.33.0.md`) # Release 0.33.0 ## Features - Add support for addons (#2887) - Add Spot support for Managed Nodegroups (#2889)
 ## Improvements - Handle missing iam.serviceAccounts key when creating IAMServiceAccounts (#2872) - Print empty JSON array when `get clusters -o json` returns nothing (#2873) - Do not fail Fargate profile creation if it already exists (#2871) ## Acknowledgments Weaveworks would like to sincerely thank: @hvydya, @mgreensmith --- ### Release Notes 0.34.0 (`docs/release_notes/0.34.0.md`) # Release 0.34.0 ## Features - Support enabling access for EMR containers (#2913) ## Improvements - Show status of fargate profile in get command (#2886) ## Acknowledgments Weaveworks would like to sincerely thank: @hvydya --- ### Release Notes 0.35.0 (`docs/release_notes/0.35.0.md`) # Release 0.35.0 ## Features - get/delete/drain support for non-eksctl created managed nodegroups (#2911) - Support upgrading non eksctl clusters (#2815) ## Improvements - Have kubelet use systemd cgroup driver on al2 and ubuntu (#2962) - Update maxpods (#2957) - Update aws-node (#2958) - Add AutoscalingGroupName to nodegroupsummary (#2820) - add sub condition to vpc-cni addon policy (#2948) - Deprecate v1.14 (#2941) - Bump `eks/pause` infra container image (#2932) - Bump k8s dependencies (#2907) - Update ALB policy, now AWS load balancer controller policy (#2914) - Deprecate static AMI resolver (#2661) - Update aws-node and make tests less brittle (#2920) ## Bug Fixes - encrypt bottlerocket os volume (#2952) - Don't run bootstrap scripts twice and before preBootstrap (#2924) - fix panic in VPC loading (#2933) - treat remote VPC as source of truth (#2928) - Do not set a default value for nodeGroup.ami (#2927) - Only use aws-node service account on cluster creation (#2917) ## Acknowledgments Weaveworks would like to sincerely thank: @hacker65536, @neha-viswanathan and @reegnz --- ### Release Notes 0.36.0 (`docs/release_notes/0.36.0.md`) # Release 0.36.0 ## Features - Add support for upgrading non-eksctl-created nodegroups (#2953) - Add support for deleting non-eksctl-created clusters (#2969) ## Improvements - Do not fail NodeName check when VPC DNS Hostnames is Disabled on Fargate clusters (#2999) - Fix bootstrap command in the examples (#3003) - Treat p4d as a GPU instance (#2991) - Add option to specify IAM role name (#2863) - Show progress of cluster update (#3023) ## Bug Fixes - Set cgroupdriver to systemd on GPU nodes (#3007) ## Acknowledgments Weaveworks would like to sincerely thank: @Tolsto, @damerakd, @dbluxo and @samsamoa --- ### Release Notes 0.36.1 (`docs/release_notes/0.36.1.md`) # Release 0.36.1 ## Bug Fixes - Fix bug preventing IPs be assigned to Windows pods #3067 - Make `--kubernetes-version` optional again for `upgrade nodegroup` #3104 - Fix panic in launch template parsing during `upgrade nodegroup` #3104 ## Acknowledgments Weaveworks would like to sincerely thank our contributors! --- ### Release Notes 0.36.2 (`docs/release_notes/0.36.2.md`) # Release 0.36.2 ## Improvements - Make v0.2.4 the default for VPC Resource Controller ## Acknowledgments Weaveworks would like to sincerely thank our contributors! --- ### Release Notes 0.37.0 (`docs/release_notes/0.37.0.md`) # Release 0.37.0 ## Features - Add support for scaling non-eksctl created nodegroups (#3132) - Update iamserviceaccount role policies (#3064) - Add create fargateprofile support for non-eksctl-managed clusters (#3080) - Support for Bottlerocket as a custom AMI in managed node groups (#3033) - Set gp3 as the default VolumeType (#3078) ## Improvements - Update maxpods (#3125) - Add needed parameter --cluster to eksctl utils update-* commands (#3103) - Reduce cloudformation calls in eksctl get clusters (#3073) - allow configuring node-volume-type via cli (#3090) - Use the latest neuron-device-plugin (#2954) ## Bug Fixes - Respect partition in load balancer IAM statements (#3069) ## Acknowledgments Weaveworks would like to sincerely thank: @bt909 and @dLobatog --- ### Release Notes 0.38.0 (`docs/release_notes/0.38.0.md`) # Release 0.38.0 ## Features - Support sourceSecurityGroupIds for unmanaged nodegroups (#3254) - Add RoleOnly option for iamserviceaccounts (#3122) - Flux v2 support (#3066) - Add support for well known policies with IRSA (#3045) - OIDC support (#3265) - Add support for kubernetes 1.19 (#3285) ## Improvements - Add option to pass kubeconfig path to Flux 2 installer (#3256) - Always enable log timestamps (#3251) - Add deprecation notice to enable repo command (#3206) - Detect and log unsupported region error for fully-private clusters (#3186) - Ignore ResourceNotFoundException error during cluster deletion (#3210) - Set/Unset/Get labels on unowned nodegroups (#3168) - Add x-kubernetes-group-version-kind to Schema (#3169) - Don't install nvidia for inf1 instances, reenable inf1 integration tests (#3162) ## Bug Fixes - Support secondary CIDR blocks for VPCs (#3232) - Fix creating private cluster in some regions (#3228) - Fix suspend ASG processes for nodegroups (#3218) - Support position name in get iamserviceaccount (#3205) - Support parsing YAML Cloudformation templates (#3170) - Refactor iamserviceaccounts (#3135) - Do not error for empty get nodegroups if output is yaml or json (#3178) ## Acknowledgments Weaveworks would like to sincerely thank: @Legion2, @dbluxo and @tkms0106 --- ### Release Notes 0.39.0 (`docs/release_notes/0.39.0.md`) # Release 0.39.0 ## Features - Support enabling KMS encryption for existing clusters (#3364) - add permission boundary to addon role (#3276) - support delete addon when only the addon stack exists (#3280) ## Improvements - bump EKS 1.19 coredns to 1.8.0 (#3319) - bump upgrade cluster timeout to 65 (#3310) - Make nodegroup CloudFormation stack slimmer (#3277) ## Bug Fixes - Fix specifying subnets by AZ (#3301) - Fix support for specifying subnets without ID (#3229) ## Acknowledgments Weaveworks would like to sincerely thank: @hiraken-w and @morancj --- ### Release Notes 0.4.0 (`docs/release_notes/0.4.0.md`) # Release 0.4.0 ## Features - New experimental command `eksctl generate profile` (#1132) - Added installation of Helm/Tiller as part of the `eksctl install flux` command (#1136) ## Bug Fixes and code improvements - perform defaulting and validation in all commands (#1124) - improve nodegroup validation to detect duplicate names (#1116) - tidy up command usage formatting (#1112) - various internal improvements (#1113, #1117, #1118, #1119) - remove the ability of skipping tasks (#1101) - added tests for experimental commands (#1132, #1095) - add common AWS flags to `eksctl install flux` command (#1082) - add support for kubelet config files for Ubuntu nodes (#994) - documentation improvements (#1085, #1092, #1093) --- ### Release Notes 0.4.1 (`docs/release_notes/0.4.1.md`) # Release 0.4.1 ## Bug Fixes - fix panic when using `eksctl update cluster` (#1160) --- ### Release Notes 0.4.2 (`docs/release_notes/0.4.2.md`) # Release 0.4.2 ## Bug Fixes - fix an error when using `eksctl get ng` and `eksctl create cluster --without-nodegroup` (#1186) --- ### Release Notes 0.4.3 (`docs/release_notes/0.4.3.md`) # Release 0.4.3 ## Bug Fixes - fix support for ap-east-1 region and Ubuntu1804 image family on Kubernetes 1.13 (#1206) --- ### Release Notes 0.40.0 (`docs/release_notes/0.40.0.md`) # Release 0.40.0 ## Features - Add EFA support (#2688) - Add support for Asia Pacific (Osaka) region (#3347) - Add disable-eviction flag for nodegroup drain and delete (#3330) - Allow referring to subnets by ID in nodegroups (#3331) - Create nodegroups/managed-nodegroups on unowned clusters (#3388) ## Improvements - Update back to upstream kris-nova/logger (#3359) - Update aws-node to v1.7.9 (#3353) - Create the FargatePodExecutionRole resource only when podExecutionRoleARN is unspecified (#3346) - Improve error when nodegroup is missing privateNetworking (#3342) - Also check managed groups for plugin requirements (#3337) - Ignore fields in JSON schema that are explicitly hidden - Log suggestion to delete IRSA when create stack fails (#3332) - Add sample template for new nodegroup CFN format (#3294) ## Bug Fixes - Prevent setting internal NodeGroup fields via ClusterConfig (#3339) - Fix 3393: fully-private clusters are not supported in region "" (#3394) - Set systemd cgroupdriver when HasMixedInstances (fix 3391) (#3398) ## Acknowledgments Weaveworks would like to sincerely thank: @saada --- ### Release Notes 0.41.0 (`docs/release_notes/0.41.0.md`) # Release 0.41.0 ## Features - Support tags for Addons service accounts (#3427) - Add proposal for EC2 instance selector integration and dry-run (#3107) - Add support for enabling capacity rebalancing when using Spot instances (#3383) ## Improvements - Minimize number of CFN calls for `create cluster` (#3421) - Create and delete fargate role stack as needed during create/delete fargate profile (#3321) - Wait longer before retrying throttled CloudFormation requests (#3425) - Add missing arm64 node selector during update kube-proxy (#3378) - Fix iamidentitymapping verb in help message (#3418) - Only attempt to get clusters in authorized regions (#3416) - Fix the draft 7 schema URL (#3401) ## Bug Fixes - Disallow invalid AMI values (#3412) ## Acknowledgments Weaveworks would like to sincerely thank: @frankywahl, @r-joyce and @saada --- ### Release Notes 0.42.0 (`docs/release_notes/0.42.0.md`) # Release 0.42.0 ## Features Add default security group rules to custom SGs (#3027) ## Improvements Disallow setting internal fields (#3450) ## Bug Fixes Force addons during cluster creation (#3438) ## Acknowledgments Weaveworks would like to sincerely thank: @liranp and @jalaziz --- ### Release Notes 0.43.0 (`docs/release_notes/0.43.0.md`) # Release 0.43.0 ## Improvements - Add managed-by label to eksctl created service accounts (#3463) - Deprecate WindowsServer1909CoreContainer AMI Family and improve error messages (#3455) - Added missing Bottlerocket option in docs for custom AMI (#3478) - Update maxpods (#3474) - Update aws-node (#3475) - Update EFA device plugin to 0.3.3 (#3466) - Add docs for non eksctl-created clusters (#3471) ## Bug Fixes - Attach EFA SG to network interfaces (#3467) - Fix service principal for cluster and Fargate profile role (#3473) ## Acknowledgments Weaveworks would like to sincerely thank: @sotoiwa --- ### Release Notes 0.44.0 (`docs/release_notes/0.44.0.md`) # Release 0.44.0 ## Features - Add support for EC2 instance selector (#3476) - Add support for dry-run (#3451) --- ### Release Notes 0.45.0 (`docs/release_notes/0.45.0.md`) # Release 0.45.0 ## Features - Use the AutoScalingGroup for getting & scaling nodegroups (#3543) - Make EKS 1.19 the default (#3526) - Support specifying an AMI release version for managed node groups (#3544) ## Improvements - Preserve capabilities when creating changeset (#3548) - Eksctl scale nodegroups supports only providing one of min/max/desired size (#3511) ## Bug Fixes - Support comma separated array for group flag in create iamidentitymapping (#3549) - Add capacity-optimized-prioritized (#3516) ## Acknowledgments Weaveworks would like to sincerely thank: @omnibrian and @lloydchang --- ### Release Notes 0.46.0 (`docs/release_notes/0.46.0.md`) # Release 0.46.0 ## Improvements - InstanceSelector only returns instances supported in the provided AZs (#3592) - Add well-known IAM Role Service Account policy for the EBS CSI driver (#3518) ## Bug Fixes - Removed deprecated labels for 1.19+ clusters (#3567) - Update IAMServiceaccount uses a unique changeset name to allow successive updates (#3605) - Check instance types are set in get nodegroups (#3602) ## Acknowledgments Weaveworks would like to sincerely thank: @jalaziz, and @zodiac12k --- ### Release Notes 0.47.0 (`docs/release_notes/0.47.0.md`) # Release 0.47.0 ## Improvements - Use built-in AMI bootstrapping scripts for AL2/Ubuntu unmanaged nodes (#3564) (Notice: [Upcoming breaking change](https://github.com/eksctl-io/eksctl/issues/3563)) - Load env from host into Flux exec (#3608) - Fix get nodegroups from stack that failed to delete (#3612) ## Bug Fixes - Correctly parse --install-nvidia/neuron-plugin flags (#3622) - Fix docs link to neuron tutorial (#3619) - Introduce delay in creating Fargate profiles to avoid EKS race condition ## Acknowledgments Weaveworks would like to sincerely thank: @aws-vrnatham and @hacker65536 --- ### Release Notes 0.48.0 (`docs/release_notes/0.48.0.md`) # Release 0.48.0 ## Improvements - Propagate user defined tags to instance and volumes in EKS nodes (#3639) - Addon create and update respects general timeout flag (#3643) - Addon version can be set to latest to automatically use latest version (#3577) - Update VPC controllers to v0.2.5 (#3628) - Add valid entries of `cloudWatch.clusterLogging.enableTypes` to schema (#3598) ## Bug Fixes - Fix delete iamserviceaccount; do not overwrite SA name (#3641) ## Acknowledgments Weaveworks would like to sincerely thank: @kishoregv, @mikutas, @saleem-mirza and @sstoyanovucsd --- ### Release Notes 0.49.0 (`docs/release_notes/0.49.0.md`) # Release 0.49.0 - Update managed nodegroup documentation (#3653) ## Features - Support regex version matching for addons (#3646) ## Improvements - Use public repo for neuron device plugin image (#3661) ## Bug Fixes - Parse ng taints correctly (#3659) ## Acknowledgments Weaveworks would like to sincerely thank: @aws-vrnatham, and @lboix --- ### Release Notes 0.5.0 Rc.0 (`docs/release_notes/0.5.0-rc.0.md`) # Release 0.5.0 ## Features - New experimental command `eksctl gitops apply` (#1165) ## Bug Fixes - Don't require EKS cluster to be ACTIVE before deleting the cluster (#1010) --- ### Release Notes 0.5.1 (`docs/release_notes/0.5.1.md`) # Release 0.5.1 ## Features - add support for Kubernetes 1.14 (#1281) ## Improvements - use AWS SDK v1.23.15 (#1278) - improve documentation for IAM Roles for Service Accounts (IRSA) (#1277) - add documentation for quickstart profile creators (#1269) --- ### Release Notes 0.5.2 (`docs/release_notes/0.5.2.md`) # Release 0.5.2 ## Bug Fixes - fix cluster deletion due to OIDC errors (#1289) - fix updating clusters to 1.14 (#1291) - improve log and error messaging on exclude rules (#1296) - make EBS-optimized setting optional, as enabling it by default breaks for some instance types (#1293) --- ### Release Notes 0.5.3 (`docs/release_notes/0.5.3.md`) # Release 0.5.3 ## Bug Fixes - fix IAM Service Accounts error when using inline policy (attachPolicy) (#1311) - fix a flaw in nodegroup's post-join logic (#1312) --- ### Release Notes 0.50.0 (`docs/release_notes/0.50.0.md`) # Release 0.50.0 ## Features - Support taints for managed nodegroups (#3616) --- ### Release Notes 0.51.0 (`docs/release_notes/0.51.0.md`) # Release 0.51.0 ## Features - Add support for kubernetes 1.20 (#3568) ## Improvements - update nvidia-device-plugin to v0.9.0 (#3696) ## Bug Fixes - Return StackName of managed Nodegroups created through CF (#3678) ## Acknowledgments Weaveworks would like to sincerely thank: @ajayk --- ### Release Notes 0.52.0 (`docs/release_notes/0.52.0.md`) # Release 0.52.0 ## Improvements - create addons (except vpc-cni) after nodegroups are created (#3736) - update IAM policy for aws-load-balancer-controller #3734 ## Bug Fixes - Fix taints support in unmanaged nodegroups (#3702) - Fix bug preventing nodegroups from joining fully-private clusters (#3737) - Reset cgroup drivers back to cgroupfs (#3637) ## Acknowledgments Weaveworks would like to sincerely thank: @M00nF1sh --- ### Release Notes 0.53.0 (`docs/release_notes/0.53.0.md`) # Release 0.53.0 ## Features - Support Bottlerocket as a node AMI family for managed nodegroups (#3762) - Support Ubuntu as a node AMI family for managed nodegroups (#3789) - Support constructing an IRSA with well known policies for EFS CSI driver (#3777) - Let users pass any flag to Flux bootstrap (BREAKING see issue desc) (#3758) ## Improvements - Add network-interface to TagSpecification (#3770) - Remove support for creating EKS 1.15 clusters (#3765) - Remove deprecated wait-nodes command (#3776) - Add grace period after cluster API is available (#3763) - Check for custom AMI by matching ami- prefix (#3754) ## Bug Fixes - Fix Ubuntu support in unmanaged nodegroups (#3796) - Add ASG name to nodegroup output when name flag is set (#3785) Weaveworks would like to sincerely thank: @Aym3nTN and @eze8789 --- ### Release Notes 0.54.0 (`docs/release_notes/0.54.0.md`) # Release 0.54.0 ## Features - Return k8s version when listing nodegroup(s) (#3810) - Dynamically set the year in the site footer (#3804) ## Improvements - Include all required policies in example config (#3815) - Write kubeconfig to temporary location before `enable flux` runs (#3778) ## Bug Fixes - Fix panic trying to unlock .kube/eksctl/clusters/ after locking fails (#3803) - Fix panic trying to unlock .kube/config after initial lock failed (#3802) - Handle encrypting OS volume for Bottlerocket (#3820) - Fix support for maxPodsPerNode in unmanaged nodegroups (#3826) ## Acknowledgments Weaveworks would like to sincerely thank: @wongma7 and @itniels --- ### Release Notes 0.55.0 (`docs/release_notes/0.55.0.md`) # Release 0.55.0 ## Features - eksctl info command (#3354) ## Improvements - Refactor common incompatible flags (#3895) - Remove supported MNG flags from deny-list (#3887) ## Bug Fixes - Fix cluster DNS for non-default ServiceIPv4CIDRs and Bottlerocket support in private clusters (#3867) - Check nodegroup's DesiredSize before getting k8s version of nodes (#3899) ## Acknowledgments Weaveworks would like to sincerely thank: @neha-viswanathan --- ### Release Notes 0.56.0 (`docs/release_notes/0.56.0.md`) # Release 0.56.0 ## Features - Configure parallel upgrades with UpdateConfig and MaxUnavailable (or MaxUnavailablePercentage) (#3753) - Bootstrap update nodegroup command (#3828) - Update the maxUnavailable number of nodes during parallel upgrades (#3855) - `Update` command can update multiple nodegroups (#3914) ## Improvements - Add default tags to OIDC provider resources (#3885) - Do not set cgroupdriver to systemd in legacy bootstrap path (#3908) ## Bug Fixes - Avoid cn-north-1d (#3920) - Ensure GPU AMI is used when using multiple instances (#3924) - Do not fail on valid flag combinations for managed ngs (#3930) --- ### Release Notes 0.57.0 (`docs/release_notes/0.57.0.md`) # Release 0.57.0 ## Improvements - Make EKS 1.20 the default (#3943) - Add stack status UpdateComplete as valid scenario (#3948) ## Bug Fixes - Fix panic for `get nodegroup` (#3960) - Fix --wait issue for eksctl upgrade nodegroup command (#3956) - Fix Config file schema docs page horizontal scroll (#3958) - Validate cluster and nodegroup names to avoid cloudformation error (#3942) - Use zone ID instead of zone name to avoid problematic zones (#3947) - Remove `--config-file` flag from `upgrade nodegroup` (#3952) ## Acknowledgments Weaveworks would like to sincerely thank: @vchetu --- ### Release Notes 0.58.0 (`docs/release_notes/0.58.0.md`) # Release 0.58.0 ## Features - Add support for configuring extra CIDRs for Control Plane SG (#3953) - Add support for kubernetes 1.21 (#3937) ## Improvements - Make managed nodegroups the default (#3897) ## Acknowledgments Weaveworks would like to sincerely thank: @MrBones757 --- ### Release Notes 0.59.0 (`docs/release_notes/0.59.0.md`) # Release 0.59.0 - Update documentation about how to upgrade default Add-Ons (#4008) ## Improvements - Bump docker image to Golang 1.16.6 (#4012) ## Bug Fixes - Filter out GPU instances when --instance-selector-gpus=0 (#3992) ## Acknowledgments Weaveworks would like to sincerely thank: @orsenthil --- ### Release Notes 0.6.0 (`docs/release_notes/0.6.0.md`) # Release 0.6.0 ## Improvements - rename experimental command `eksctl gitops apply` to `eksctl enable profile` (#1347) - `eksctl enable profile` now uses a temporary directory when cloning and no longer exposes `--output-path` (#1337) ## Bug fixes - fix `eksctl update cluster` for 1.14 clusters (#1304) --- ### Release Notes 0.60.0 (`docs/release_notes/0.60.0.md`) # Release 0.60.0 ## Improvements - Add / to pod eviction errors (#4005) ## Bug Fixes - Correctly parse new kubernetes version format and use `build.2` kube proxy images for 1.18+ clusters (#4021) - Fix --version flag issue for create managed nodegroup command (#3962) ## Acknowledgments Weaveworks would like to sincerely thank: @benjiro, and @vchetu --- ### Release Notes 0.61.0 (`docs/release_notes/0.61.0.md`) # Release 0.61.0 ## Features - Add ability to set a custom subnet when creating a nodegroup (#4025 #4044) - Expose nodegroup container runtime configuration (#4051) ## Bug Fixes - Fix panic when creating a nodegroup with instance selector options (#3993) - Fix panic when VPC config is missing (#4056) ## Acknowledgments Weaveworks would like to sincerely thank: @NaivePa, @peterfication and @SharpThunder --- ### Release Notes 0.62.0 (`docs/release_notes/0.62.0.md`) # Release 0.62.0 ## Improvements - Deprecate all Flux v1 commands (#4076) - Update AWS node to 1.9 (#4085) ## Bug Fixes - Revert to build.1 for kube-proxy addon versions (#4091) --- ### Release Notes 0.63.0 (`docs/release_notes/0.63.0.md`) # Release 0.63.0 ## Improvements - Fetch instance type from launch template in `eksctl get nodegroups` (#4098) - Remove update-cluster-stack cmd (#4095) - Deprecate `ssh.enableSsm` and disallow enabling it (#3989) - Update ebs-csi-driver policy (#4099) - Add support for preserving addons (#4083) ## Bug Fixes - Fix panic for getting an individual nodegroup (#4092) - Use a unique changeset name when updating addons (#4116) ## Acknowledgments Weaveworks would like to sincerely thank: @nickmancari --- ### Release Notes 0.64.0 (`docs/release_notes/0.64.0.md`) # Release 0.64.0 ## Improvements - Fail early when instance selector returns too many instances (#4139) - Bump vpc-resource-controller/webhook dep to v0.2.6 (#4104) ## Bug Fixes - Do not create nodegroup if --fargate is set and --managed is not provided (#4135) - Fix creating Windows nodegroups in fully-private clusters (#4128) - Add preBootstrapCommands to userdata when using customAMI (#4065) - Fix deprecation warning (#4148) ## Acknowledgments Weaveworks would like to sincerely thank: @EHJ-52n, @abhipth, @brsolomon-deloitte, and @kishoregv --- ### Release Notes 0.65.0 (`docs/release_notes/0.65.0.md`) # Release 0.65.0 ## Improvements - Bump dependencies (#4169) - Update to Go 1.17 (#4165) --- ### Release Notes 0.66.0 (`docs/release_notes/0.66.0.md`) # Release 0.66.0 ## Features - Add support for running external plugin (#4174) ## Acknowledgments Weaveworks would like to sincerely thank: @samir-gandhi --- ### Release Notes 0.67.0 (`docs/release_notes/0.67.0.md`) # Release 0.67.0 ## Features - Add support for registering and deregistering external clusters using EKS Connector (#4189) - Added new documentation for eksctl anywhere (#4222) - Add token cache for MFA users and credentials (#4194) - Add support for specifying wellKnownPolicies when using addons (#4161) ## Improvements - Update aws-node (#4219) - Delete IAM role if RegisterCluster fails (#4201) ## Bug Fixes - Add validation to fix gitops field panic (#4220) - import service role during upgrade cluster (#4187) --- ### Release Notes 0.68.0 (`docs/release_notes/0.68.0.md`) # Release 0.68.0 ## Features - Add support for EC2 detailed monitoring in nodeGroup (#4231) ## Improvements - Gracefully delete vpc-cni during cluster deletion (#4244) - Drain nodegroups during cluster deletion (#4205) - Remove support for 1.16 (#3944) - Fix typo in warning message for existing service accounts. (#4233) - Update documentation for EKS Connector (#4232) ## Acknowledgments Weaveworks would like to sincerely thank: @igorakkerman, and @mmasaki --- ### Release Notes 0.69.0 (`docs/release_notes/0.69.0.md`) # Release 0.69.0 ## Improvements - Skip addons task when no addon is configured (#4253) ## Bug Fixes - Enable create iamidentitymapping in AWS CN partition (#4218) ## Acknowledgments Weaveworks would like to sincerely thank: @lamricky0 and --- ### Release Notes 0.7.0 (`docs/release_notes/0.7.0.md`) # Release 0.7.0 ## Features - add support for Windows nodes (#742) - add support for modifying `mapUsers` key in `aws-auth` ConfigMap (#1328) - get eksctl ready for when São Paulo region (sa-east-1) is released (#1398) - allow passing a revision to `eksctl enable profile` (#1437) - add alias for App Mesh quick start profile (#1438) ## Improvements - update the default Kubernetes version to `1.14` (#1392) - update Flux and Helm operator dependencies to `v1.15.0` and `v1.0.0-rc2` (#1389) - add readiness probe to CoreDNS (#1346) - print `eksctl` version in the first line of any log (#1396) - add CloudMap permissions to App Mesh profile (#1413) - move go-bindata from jteeuwen->kevinburke to fix build correctness due to go.mod (#1395) - several improvements to docs (#1369, #1371, #1380, #1391, #1405, #1439) ## Bug fixes - fix building eksctl on Go 1.13 (#1388) - fix nodegroup delete failure when PDB isn't respected (#1244) --- ### Release Notes 0.70.0 (`docs/release_notes/0.70.0.md`) # Release 0.70.0 ## Features - Parse task log to be more human readable (#4290) ## Improvements - Add support for specifying latest version in cluster config (#4258) --- ### Release Notes 0.71.0 (`docs/release_notes/0.71.0.md`) # Release 0.71.0 ## Breaking Changes - [Breaking] Remove incomplete support for GPU instance types in Bottlerocket and Windows nodegroups ([#4238](https://github.com/eksctl-io/eksctl/issues/4238)) ## Features - Add iamidentitymapping delete and get commands (#4346) ## Improvements - Move from io/ioutil to io and os packages (#4333) ## Bug Fixes - Validate support for GPU instance types (#4243) - Wait for addons to be healthy only if there are nodegroups when creating a cluster (#4332) ## Acknowledgments Weaveworks would like to sincerely thank: @Juneezee --- ### Release Notes 0.72.0 (`docs/release_notes/0.72.0.md`) # Release 0.72.0 ## Features - Native Bottlerocket support in managed nodegroups (#4381) ## Bug Fixes - Add ARM support and fix AMI resolution for Ubuntu (#4367) --- ### Release Notes 0.73.0 (`docs/release_notes/0.73.0.md`) # Release 0.73.0 ## Features - Support CloudWatch log retention (#4295) - Add support for Windows Server 20H2 (#4390) ## Improvements - Make kubernetes 1.21 default (#4394) - Reduce API calls in upgrade cluster (#4366) ## Bug Fixes - Fix wait for managed nodegroups (#4401) - Scale managed nodegroup with --name flag (#4383) ## Acknowledgments Weaveworks would like to sincerely thank: @abhipth and @adammw --- ### Release Notes 0.74.0 (`docs/release_notes/0.74.0.md`) # Release 0.74.0 ## Features - Add support for Windows IPAM (#4356) ## Bug Fixes - Unpend and fix Windows integration tests (#4442) - Set volume size in data volume for Bottlerocket (#4436) ## Acknowledgments Weaveworks would like to sincerely thank: @silverskyvicto --- ### Release Notes 0.75.0 (`docs/release_notes/0.75.0.md`) # Release 0.75.0 ## Features - Add --config-file flag to get labels & nodegroups (#4469) (#4465) - Add option to skip endpoint creation for private clusters (#4457) ## Improvements - Improve kube-proxy updating to always get the latest version&build #4344 (#4440) - Document logs:PutRetentionPolicy in IAM policies doc (#4456) ## Bug Fixes - Add option to skip endpoint creation completely (#4457) - Fix update logic in iam service account (#4460) - Brew Deprecation Warning: Calling bottle :unneeded is deprecated (#4458) --- ### Release Notes 0.76.0 (`docs/release_notes/0.76.0.md`) # Release 0.76.0 ## Features - Remove support for EKS 1.17 (#4487) - Support scaling all nodegroups found in a config file (#4464) - Add config-file option to get cluster (#4468) - Add AttachPolicy to NodeGroupIAM configuration (#4331) ## Bug Fixes - Fix panic in node affinity adder (#4491) ## Improvements - Update cluster-upgrade.md docs (#4479) ## Acknowledgments Weaveworks would like to sincerely thank: @adammw and @brianrudolf-ep --- ### Release Notes 0.77.0 (`docs/release_notes/0.77.0.md`) # Release 0.77.0 ## Features - Added `--config-file` flag to `set labels` (#4502) ## Linting - Removing the usage of deprecated `golint` (#4489) ## Documentation - Added missing --region flag in the for nodegroup and iamidentitymapping pages (#4521) - Separated the VPC networking documentation into meaningful sub-sections (#4492) ## Acknowledgments Weaveworks would like to sincerely thank: - @herlon214 --- ### Release Notes 0.78.0 (`docs/release_notes/0.78.0.md`) # Release 0.78.0 ## Features - Automatically attach the correct EBS CSI Driver policies when creating the addon (#4216) ## Bug Fixes - Fix ignoring partitions on EBS controller (#4547) --- ### Release Notes 0.79.0 (`docs/release_notes/0.79.0.md`) # Release 0.79.0 ## Features - Add support for IPv6 VPC and K8s clusters (#4571) - Fail early if flux binary is missing during cluster creation (#4568) - Add g5 instance to list of GPU instance types (#4550) ## Bug Fixes - Fix `get clusters --all-regions` (#4563) - Check InstanceRoleARN exists before deleting authconfigmap entry (#4558) ## Acknowledgments Weaveworks would like to sincerely thank: @ojunkwon --- ### Release Notes 0.8.0 (`docs/release_notes/0.8.0.md`) # Release 0.8.0 ## Features - add support for resolving AMIs using SSM Parameter Store (#1393) - allow changing cluster API public/private access. See [aws docs] (https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) when enabling private access (#1149) ## Improvements - add some diagrams to help explain the gitops commands (#1455) - various improvements to the build system (#1465, #1484, #1486) - Refactor package `drain` (#1476) - Cluster Autoscaler: add note about ASG tags for scaling up from 0 (#1479) - various other improvements (#1442, #1447) ## Bug fixes - fix error with `eksctl util install-vpc-controllers` (#1452) - report error if eksctl can't write to kubeconfig file (#1402) --- ### Release Notes 0.80.0 (`docs/release_notes/0.80.0.md`) # Release 0.80.0 - Add warning to cluster access about publicAccessCIDRs (#4576) ## Improvements - Add support for eksctl utils describe-stacks with a ClusterConfig file (#4597) - Add support for eksctl utils update-legacy-subnet-settings with a ClusterConfig file (#4598) - Add support for eksctl utils write-kubeconfig with a ClusterConfig file (#4599) - Use CloudFormation for cluster endpoint access control, tagging, logging and publicAccessCIDRs (#4526) - Remove redundant calls to WaitForControlPlane (#4615) ## Bug Fixes - Make config file case insensitive for IPFamily value (#4619) - Remove NAT64/DNS64 from ipv6 clusters (#4618) - Fix the prefix of the ipv6 family example (#4605) ## Acknowledgments Weaveworks would like to sincerely thank: @SamLR --- ### Release Notes 0.81.0 (`docs/release_notes/0.81.0.md`) # Release 0.81.0 ## Features - Add node drain wait period (#4506) ## Improvements - Add ec2:DescribeInstanceTypes to autoScaler policy statements (#4664) - Add docs for troubleshooting kubectl log authorization errors (#4646) ## Bug Fixes - Apply `undo` flag when running `drain nodegroup` (#4649) - Added missing OIDC requirement (#4636) - Only wait for addons to be healthy if at least 1 node instance exists (#4629) - Added missing save after IAM mapping is created (#4623) ## Acknowledgments Weaveworks would like to sincerely thank: @PatTheSilent, @SamLR, @lbodor, and @rayterrill --- ### Release Notes 0.82.0 (`docs/release_notes/0.82.0.md`) # Release 0.82.0 ## Improvements - Improve IPv6 docs and link to docs in help text (#4689) ## Bug Fixes - Use correct kubeconfig for Karpenter install (#4704) - Fix bug causing the identityProviders to be marshalled incorrectly (#4688) - Fix EC2 instance selector (#4681) - Add ec2:DescribeVpcPeeringConnections node policy for albIngress (#4673) ## Acknowledgments Weaveworks would like to sincerely thank: @bwagner5, and @matti --- ### Release Notes 0.83.0 (`docs/release_notes/0.83.0.md`) # Release 0.83.0 ## Features - Allow additional volume mappings (#4725) - Return AMI ID in `get nodegroup` for MNGs using custom AMIs (#4724) - Adding an option for `--disable-nodegroup-eviction` when deleting a cluster (#4659) ## Bug Fixes - Make the error clearer that a public subnet is required (#4742) - Fix ASG names in `get nodegroup` (#4717) - Upgrade non-al2 managed nodegroups by updating the `Version` field in the nodegroup stack (#4666) - Fix bug preventing IPv6 clusters using an existing VPC (#4686) - Fix deletion logic in case the cluster does not exist (#4711) - Delete Karpenter stack as part of delete cluster (#4710) ## Acknowledgments Weaveworks would like to sincerely thank: @AmitBenAmi --- ### Release Notes 0.84.0 (`docs/release_notes/0.84.0.md`) # Release 0.84.0 ## Features - Add support for MaxInstanceLifetime for unmanaged nodegroups (#4781) - Show nodegroup type in get nodegroup (#4737) ## Improvements - Deprecate albIngress field and replace with awsLoadBalancerController (#4738) - Reduce the number of API calls (#4772) ## Bug Fixes - Only set 2 AZs when using us-west-1 (#4766) - Automatically detect role-name when iamserviceaccount update happens (#4744) - Add ipv6 route table entry when AllocateIPV6 is enabled (#4799) --- ### Release Notes 0.85.0 (`docs/release_notes/0.85.0.md`) # Release 0.85.0 ## Features - Dump logs to disk on failure (#4343) - Removed the cache as essentially it is not needed in most cases (#4798) - Add support for Karpenter defaultInstanceProfile setting (#4751) ## Improvements - Log region for every command and improve help message (#4791) ## Acknowledgments Weaveworks would like to sincerely thank: @niallthomson, and @yaash45 --- ### Release Notes 0.86.0 (`docs/release_notes/0.86.0.md`) # Release 0.86.0 ## Features - Add support for Bottlerocket nodgroups with NVIDIA accelerated hardware (#4809) - Add support for more Graviton instance types (#4819) - Added automatically generating ASG tags from labels and taints (#4785) ## Improvements - allow case insensitivity for AMIFamily (#4826) ## Bug Fixes - Fix aws-iam-authenticator api version in kube config file (#4834) - add support for ed25519 (#4835) - add docs on vpc-cni permission boundaries (#4831) - Fix all Karpenter issues for the new 0.6.2 version (#4805) - Allow AWS_PROFILE env var to be used write-kubeconig (#4830) - remove deprecated field that breaks `upgrade nodegroup` timeout value (#4825) ## Acknowledgments Weaveworks would like to sincerely thank: @arnaldo2792 and @otterley --- ### Release Notes 0.87.0 (`docs/release_notes/0.87.0.md`) # Release 0.87.0 ## Features - Add validation check for zones before creating cluster (#4837) ## Bug Fixes - Fix missing stackname when updating addons (#4894) - Validate AZ configuration correctly (#4896) --- ### Release Notes 0.88.0 (`docs/release_notes/0.88.0.md`) # Release 0.88.0 ## Improvements - Pin karpenter support to 0.6.* (#4903) - Don't attach the control plane security group to managed nodegroups (#4860) ## Bug Fixes - Fix setting ownership status when listing clusters (#4911) - Fix the CIDR range calculation in case of a custom CIDR block (#4904) - Fix build image URL in Dockerfile (#4893) ## Documentation - Fix broken links to 'IAM Service Accounts' in docs (#4912) - Added clarity for overrideBootstrapCommand (#4905) ## Acknowledgments Weaveworks would like to sincerely thank: @rothgar, @thinkmassive, and @tompiscitell --- ### Release Notes 0.89.0 (`docs/release_notes/0.89.0.md`) # Release 0.89.0 ## Improvements - Update go to 1.18 (#4960) ## Bug Fixes - Fix nodegroup get when no nodes are present (#4928) - Empty get addons returns empty json/yaml array instead of empty string (#4922) ## Acknowledgments Weaveworks would like to sincerely thank: @gdlx --- ### Release Notes 0.9.0 (`docs/release_notes/0.9.0.md`) # Release 0.9.0 ## Features - mark Kubernetes 1.11 as [deprecated](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html) (#1384) ## Improvements - separate responsibilities between experimental GitOps commands `eksctl enable repo` and `eksctl enable profile` (#1509) - fix some typos in the VPC documentation on the site (#1515) - refactor Nodegroup, IAM and SSH configuration (#1530) --- ### Release Notes 0.90.0 (`docs/release_notes/0.90.0.md`) # Release 0.90.0 ## Features - Automatically create ASG tags from a nodegroup's labels and taints (#4962) ## Improvements - Set up AWS SDK v2 and migrate STS service (#4926) ### Documentation - Update docs to mention what happens when you create an addon with no policy sources (#4999) ## Acknowledgments Weaveworks would like to sincerely thank: @midgleyc --- ### Release Notes 0.91.0 (`docs/release_notes/0.91.0.md`) # Release 0.91.0 ## Features - Removed legacy path for al2 and ubuntu (#4968) ## Improvements - Migrate usage of SSM to aws-sdk-go-v2 (#5009) - Migrate ELB and ELBv2 to aws-sdk-go-v2 (#4937) ## Bug Fixes - Fix missing permission boundary from Karpetner iam resource (#5024) ## Acknowledgments Weaveworks would like to sincerely thank: @michaelbeaumont --- ### Release Notes 0.92.0 (`docs/release_notes/0.92.0.md`) # Release 0.92.0 ## Features - Add support for EKS 1.22 (#4915) ## Bug Fixes - Fix bug preventing managed nodegroups being provisioned on older EKS platform versions (#5055) --- ### Release Notes 0.93.0 (`docs/release_notes/0.93.0.md`) # Release 0.93.0 ## Features - Prevent duplication, and add flag to disable shadowing of IAM Identity Mappings (#4963) ## Improvements - Remove support for 1.18 (#5073) ## Bug Fixes - Remove the restriction of defining overrideBootstrap for Windows images (#5076) - Use EKS APIs default vpc-cni addon version for IPv6 clusters (#5078) - Remove sorting in case the switch is off (#5088) - Fix iamserviceaccount `create` duplicate entry check not being behind the created flag (#5080) - Fix empty `get addons` table output (#5071) ## Documentation - Update migration document to include private nodegroups (#5084) ## Acknowledgments Weaveworks would like to sincerely thank: @DameonSmith and @gdlx --- ### Release Notes 0.94.0 (`docs/release_notes/0.94.0.md`) # Release 0.94.0 ## Improvements - Migrate usage of EC2 to aws-sdk-go-v2 (#5086) - Migrate usage of IAM to aws-sdk-go-v2 (#5035) - Add override integration test (#5094) ## Bug Fixes - Only select zones which are of type `availability-zone` (#5112) - Fix the watcher not exiting in case Kubernetes closed the channel on an internal error (#5109) --- ### Release Notes 0.95.0 (`docs/release_notes/0.95.0.md`) # Release 0.95.0 ## Features - Add containerd runtime support for Windows (#5127) ## Improvements - Handle pod eviction errors correctly (#5116) - Make Kubernetes 1.22 the default (#5136) - Add Propagation of ManagedNodeGroup Tags to their corresponding AutoScalingGroups (#5002) ## Acknowledgments Weaveworks would like to sincerely thank: @hintofbasil and @SlevinWasAlreadyTaken --- ### Release Notes 0.96.0 (`docs/release_notes/0.96.0.md`) # Release 0.96.0 ## Features - Support Jakarta (ap-southeast-3) (#5180) - Error early for unmanaged nodegroup using ARM GPU instance type (#5168) ## Improvements - Move `AssignIpv6AddressOnCreation` from task to template (#5177) --- ### Release Notes 0.97.0 (`docs/release_notes/0.97.0.md`) # Release 0.97.0 ## Features - Support Karpenter version 0.9.0 (#5201) - Support AWS Local Zones (#5187) ## Bug Fixes - Fix not checking for missing stack error (#5218) - Do not require overrideBootstrapCommand when ami family is Bottlerocket (#5185) --- ### Release Notes 0.98.0 (`docs/release_notes/0.98.0.md`) # Release 0.98.0 ## Improvements - Migrate usage of EKS to aws-sdk-go-v2 (#5228) ## Bug Fixes - Trim equal sign suffix from AWS ed25519 encoded fingerprint (#5151) --- ### Release Notes 0.99.0 (`docs/release_notes/0.99.0.md`) # Release 0.99.0 ## Improvements - Fix incorrect version info for command "eksctl info" (#5280) - Theme updates to eksctl docs (#5252) ## Bug Fixes - Fix ASG tag propagation (#5286) - Make INTEGRATION_TEST_FOCUS value work again (#5284) - Fix selecting subnet IDs from subnet names (#5261) ## Acknowledgments Weaveworks would like to sincerely thank: @guessi --- --- METRICS --- - Files Extracted: 273 - Estimated Token Budget: ~61781 tokens - Canonical Reference: https://codewiki.google/github.com/eksctl-io/eksctl