## 1. Project Overview & Quickstart (karmada-io/karmada) ## File: README.md # Karmada [](/LICENSE) [](https://github.com/karmada-io/karmada/releases/latest) [](https://slack.cncf.io) [](https://bestpractices.coreinfrastructure.org/projects/5301) [](https://securityscorecards.dev/viewer/?uri=github.com/karmada-io/karmada) [](https://goreportcard.com/report/github.com/karmada-io/karmada) [](https://codecov.io/gh/karmada-io/karmada) [](https://app.fossa.com/projects/custom%2B28176%2Fgithub.com%2Fkarmada-io%2Fkarmada?ref=badge_shield) [](https://artifacthub.io/packages/krew/krew-index/karmada) [](https://clomonitor.io/projects/cncf/karmada) ## Karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications. By speaking Kubernetes-native APIs and providing advanced scheduling capabilities, Karmada enables truly open, multi-cloud Kubernetes. Karmada aims to provide turnkey automation for multi-cluster application management in multi-cloud and hybrid cloud scenarios, with key features such as centralized multi-cloud management, high availability, failure recovery, and traffic scheduling. Karmada is an incubation project of the [Cloud Native Computing Foundation](https://cncf.io/) (CNCF). ## Why Karmada: - __K8s Native API Compatible__ - Zero change upgrade, from single-cluster to multi-cluster - Seamless integration of existing K8s tool chain - __Out of the Box__ - Built-in policy sets for scenarios, including: Active-active, Remote DR, Geo Redundant, etc. - Cross-cluster applications auto-scaling, failover and load-balancing on multi-cluster. - __Avoid Vendor Lock-in__ - Integration with mainstream cloud providers - Automatic allocation, migration across clusters - Not tied to proprietary vendor orchestration - __Centralized Management__ - Location agnostic cluster management - Support clusters in Public cloud, on-prem or edge - __Fruitful Multi-Cluster Scheduling Policies__ - Cluster Affinity, Multi Cluster Splitting/Rebalancing - Multi-Dimension HA: Region/AZ/Cluster/Provider - __Open and Neutral__ - Jointly initiated by Internet, finance, manufacturing, teleco, cloud providers, etc. - Target for open governance with CNCF **Notice: This project is developed in continuation of Kubernetes [Federation v1](https://github.com/kubernetes-retired/federation) and [v2](https://github.com/kubernetes-sigs/kubefed). Some basic concepts are inherited from these two versions.** ## Architecture The Karmada Control Plane consists of the following components: - Karmada API Server - Karmada Controller Manager - Karmada Scheduler ETCD stores the Karmada API objects, the API Server is the REST endpoint all other components talk to, and the Karmada Controller Manager performs operations based on the API objects you create through the API server. The Karmada Controller Manager runs the various controllers, the controllers watch Karmada objects and then talk to the underlying clusters' API servers to create regular Kubernetes resources. 1. Cluster Controller: attach Kubernetes clusters to Karmada for managing the lifecycle of the clusters by creating cluster objects. 2. Policy Controller: the controller watches PropagationPolicy objects. When the PropagationPolicy object is added, it selects a group of resources matching the resourceSelector and creates ResourceBinding with each single resource object. 3. Binding Controller: the controller watches ResourceBinding object and create Work object corresponding to each cluster with a single resource manifest. 4. Execution Controller: the controller watches Work objects. When Work objects are created, it will distribute the resources to member clusters. ## Concepts **Resource template**: Karmada uses Kubernetes Native API definition for federated resource template, to make it easy to integrate with existing tools that already adopt on Kubernetes **Propagation Policy**: Karmada offers a standalone Propagation(placement) Policy API to define multi-cluster scheduling and spreading requirements. - Support 1:n mapping of Policy: workload, users don't need to indicate scheduling constraints every time creating federated applications. - With default policies, users can just interact with K8s API **Override Policy**: Karmada provides standalone Override Policy API for specializing cluster relevant configuration automation. E.g.: - Override image prefix according to member cluster region - Override StorageClass according to cloud provider The following diagram shows how Karmada resources are involved when propagating resources to member clusters. ## Quick Start This guide will cover: - Install `karmada` control plane components in a Kubernetes cluster which is known as `host cluster`. - Join a member cluster to `karmada` control plane. - Propagate an application by using `karmada`. ### Prerequisites - [Go](https://golang.org/) version follows [go.mod](https://github.com/karmada-io/karmada/blob/master/go.mod#L3) - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) version v1.19+ - [kind](https://kind.sigs.k8s.io/) version v0.14.0+ ### Install the Karmada control plane #### 1. Clone this repo to your machine: ```bash git clone https://github.com/karmada-io/karmada ``` #### 2. Change to the karmada directory: ```bash cd karmada ``` #### 3. Deploy and run Karmada control plane: run the following script: ```bash hack/local-up-karmada.sh ``` This script will do the following tasks for you: - Start a Kubernetes cluster to run the Karmada control plane, aka. the `host cluster`. - Build Karmada control plane components based on a current codebase. - Deploy Karmada control plane components on the `host cluster`. - Create member clusters and join Karmada. If everything goes well, at the end of the script output, you will see similar messages as follows: ```bash Local Karmada is running. To start using your Karmada environment, run: export KUBECONFIG="$HOME/.kube/karmada.config" Please use 'kubectl config use-context karmada-host/karmada-apiserver' to switch the host and control plane cluster. To manage your member clusters, run: export KUBECONFIG="$HOME/.kube/members.config" Please use 'kubectl config use-context member1/member2/member3' to switch to the different member cluster. ``` There are two contexts in Karmada: - karmada-apiserver `kubectl config use-context karmada-apiserver` - karmada-host `kubectl config use-context karmada-host` The `karmada-apiserver` is the **main kubeconfig** to be used when interacting with the Karmada control plane, while `karmada-host` is only used for debugging Karmada installation with the host cluster. You can check all clusters at any time by running: `kubectl config view`. To switch cluster contexts, run `kubectl config use-context [CONTEXT_NAME]` ### Demo ### Propagate application In the following steps, we are going to propagate a deployment by Karmada. #### 1. Create nginx deployment in Karmada. First, create a [deployment](samples/nginx/deployment.yaml) named `nginx`: ```bash kubectl create -f samples/nginx/deployment.yaml ``` #### 2. Create PropagationPolicy that will propagate nginx to member cluster Then, we need to create a policy to propagate the deployment to our member cluster. ```bash kubectl create -f samples/nginx/propagationpolicy.yaml ``` #### 3. Check the deployment status from Karmada You can check deployment status from Karmada, don't need to access member cluster: ```bash $ kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE nginx 2/2 2 2 20s ``` ## Kubernetes compatibility Karmada is compatible with a wide range of Kubernetes versions. For detailed compatibility instructions, please refer to the [Kubernetes Compatibility](https://karmada.io/docs/administrator/compatibility/). The following table shows the compatibility test results against the latest 10 Kubernetes versions: | | Kubernetes 1.36 | Kubernetes 1.35 | Kubernetes 1.34 | Kubernetes 1.33 | Kubernetes 1.32 | Kubernetes 1.31 | Kubernetes 1.30 | Kubernetes 1.29 | Kubernetes 1.28 | Kubernetes 1.27 | Kubernetes 1.26 | Kubernetes 1.25 | |-----------------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------| | Karmada v1.16 | | | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | Karmada v1.17 | | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | Karmada v1.18 | | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | Karmada HEAD (master) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | | Key: * `✓` Karmada and the Kubernetes version are exactly compatible. * `+` Karmada has features or API objects that may not be present in the Kubernetes version. * `-` The Kubernetes version has features or API objects that Karmada can't use. ## Meeting Regular Community Meeting: * Tuesday at 14:30 UTC+8 (Chinese)(biweekly). [Convert to your timezone](https://dateful.com/convert/utc8?t=1430). * Tuesday at 15:00 UTC+0 (English)(biweekly). [Convert to your timezone](https://dateful.com/convert/coordinated-universal-time-utc?t=15). Resources: - [Meeting Notes and Agenda](https://docs.google.com/document/d/1y6YLVC-v7cmVAdbjedoyR5WL0-q45DBRXTvz5_I7bkA/edit) - [Meeting Calendar](https://calendar.google.com/calendar/embed?src=karmadaoss%40gmail.com&ctz=Asia%2FShanghai) | [Subscribe](https://calendar.google.com/calendar/u/1?cid=a2FybWFkYW9zc0BnbWFpbC5jb20) - [Meeting Link](https://zoom.com/my/karmada) ## Contact If you have questions, feel free to reach out to us in the following ways: - [mailing list](https://groups.google.com/forum/#!forum/karmada) - [slack](https://cloud-native.slack.com/archives/C02MUF8QXUN) | [Join](https://slack.cncf.io/) - [twitter](https://twitter.com/karmada_io) ## References - To learn about companies and organizations using Karmada in production, please visit [Adopters page](https://karmada.io/adopters). - For commercial support, professional services, and consulting, please visit [Partners page](https://karmada.io/partners). - For talks and presentations about Karmada, please visit [Community Talks](https://github.com/karmada-io/community/tree/main/talks). - For blogs and articles, please visit [website](https://karmada.io/blog/). ## Contributing If you're interested in being a contributor and want to get involved in developing the Karmada code, please see [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the contribution workflow. ## License Karmada is under the Apache 2.0 license. See the [LICENSE](LICENSE) file for details. --- ## File: docs/README.md Karmada documentation is all hosted on [karmada-io/website](https://github.com/karmada-io/website). ## 2. Official Technical Reference & Guides (karmada-io/website) # Karmada Docs & Website This repo contains the source code of [Karmada website](https://karmada.io/) and all of the docs for Karmada. It's built by [Docusaurus](https://docusaurus.io/), a modern static website generator. - [Karmada website](https://karmada.io/) - [Karmada docs](https://karmada.io/docs/) - [Karmada Blog](https://karmada.io/blog/) Welcome to join us and you are more than appreciated to contribute! ## Release Branch Versioning Each release branch maintains documentation for the 6 most recent versions of Karmada. For example, the `release-1.14` branch maintains documentation for Karmada versions v1.9 through v1.14. This versioning strategy ensures that users can access documentation for recent versions while keeping the repository manageable. When a new minor version is released, the documentation for the oldest version is removed from maintenance, and the branch is updated to include documentation for the new version. ## Website Preview If you only update the docs, you do not need to preview the website locally. We will create a preview link for you when you submit a pull request (PR). In an open PR, you can find the preview link in the comment of the PR. Check [Open PRs List](https://github.com/karmada-io/website/pulls). ## Run with Node.js If you have the Node.js environment, you can run the website locally. ```bash # Clone the repo, or your own fork git clone https://github.com//karmada-website.git # Install dependencies cd karmada-website npm install # Start the site yarn run start ``` ## Run with Docker If you do not want to install the Node.js environment, you can consider using Docker instead. ```bash make serve ``` With just one command, you can run it locally and then access to preview the website. ## Local Development If you want to contribute to the website, firstly, it is essential to learn some basic knowledge of [Docusaurus](https://docusaurus.io/). ### Run the development server To start a local development server, please run: ```bash yarn run start ``` This command starts a local development server and opens a browser window at . Most changes are automatically reflected live without the need to restart the server. ### Build ```bash yarn run build ``` This command generates static content into the `build` directory and can be served using any static contents hosting service. ### Deployment ```bash GIT_USER= USE_SSH=true yarn deploy ``` If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.