kruise (Agent Skills)
GitHubAutomated management of large-scale applications on Kubernetes (incubating project under CNCF)
AGENTS.md
# AGENTS.md
Guidance for AI agents working on the OpenKruise project.
## Project Overview
OpenKruise is a CNCF incubating project that extends Kubernetes with advanced workload management. It runs as a set of custom controllers and CRDs on top of Kubernetes, providing five functional domains:
- **Advanced Workloads**: CloneSet, AdvancedStatefulSet, BroadcastJob, AdvancedCronJob, DaemonSet, EphemeralJob
- **Sidecar Container Management**: SidecarSet, Container Launch Priority, SidecarTerminator
- **Multi-domain Management**: WorkloadSpread, UnitedDeployment
- **Enhanced Operations**: ContainerRecreateRequest, ImagePullJob, ImageListPullJob, ResourceDistribution, PodProbeMarker, NodeImage
- **Application Protection**: DeletionProtection, PodUnavailableBudget, PersistentPodState
| Key | Value |
|-----|------------------------------------------------------------------------------|
| Module | `github.com/openkruise/kruise` |
| Language | Go 1.23+ |
| K8s Libraries | v0.32.10 |
| controller-runtime | v0.20.2 |
| controller-tools | v0.17.3 |
| API Groups | `apps.kruise.io` (workloads and operations), `policy.kruise.io` (protection) |
| K8s Compatibility | v1.18+ (v1.28+ recommended) |
| License | Apache 2.0 |
## Project Structure
```
.
├── main.go # kruise-manager entry point
├── apis/ # API type definitions (CRD specs)
│ ├── apps/
│ │ ├── pub/ # Shared types (lifecycle, launch_priority, etc.)
│ │ ├── v1alpha1/ # v1alpha1 API types
│ │ └── v1beta1/ # v1beta1 API types (StatefulSet)
│ └── policy/v1alpha1/ # Policy API types (DeletionProtection)
├── pkg/
│ ├── controller/ # Controller implementations per workload type
│ ├── webhook/ # Admission webhook handlers (validating/mutating)
│ ├── daemon/ # Kruise-daemon (node agent, gRPC + CRI)
│ ├── control/ # Shared control logic (pubcontrol, sidecarcontrol)
│ ├── util/ # Utilities (expectations, feature gates, cache, etc.)
│ ├── client/ # Generated clientset, informer, lister [DO NOT EDIT]
│ └── features/ # Feature gate definitions
├── cmd/
│ ├── daemon/ # Kruise-daemon entry point
│ └── helm_hook/ # Helm hook binary
├── config/ # Kustomize overlays (CRD, RBAC, webhook, manager)
├── test/
│ ├── e2e/ # End-to-end tests (ginkgo/gomega)
│ └── fuzz/ # Fuzz tests
├── scripts/ # Build and code generation scripts
├── hack/ # Dev scripts (fmt-imports, boilerplate)
└── docs/ # Documentation, proposals, contributing guides
```
## Commands
### Build & Generate
| Command | Description |
|---------|-------------|
| `make build` | Build `bin/manager` (runs generate + fmt + vet + manifests first) |
| `make generate` | Regenerate DeepCopy, clientset, informer, lister, OpenAPI |
| `make manifests` | Regenerate CRD YAML and RBAC manifests |
| `make generate_helm_crds` | Regenerate CRDs for Helm charts into `bin/` |
| `make docker-build` | Build Docker image `openkruise/kruise-manager:test` |
| `make docker-multiarch` | Build multi-arch image (amd64/arm64/ppc64le) |
### Test
| Command | Description |
|---------|-------------|
| `make test` | Unit tests with race detector + coverage (envtest K8s 1.32.0) |
| `make atest` | Same as `test` but skips generate/fmt/vet/manifests |
| `make kruise-e2e-test` | Full e2e: kind cluster + build + install + test + cleanup |
| `make coverage-report` | Generate `cover.html` from `cover.out` |
### Lint & Format
| Command | Description |
|---------|-------------|
| `make lint` | golangci-lint (v1.51.2, config: `.golangci.yml`) |
| `make vet` | `go vet` |
| `make fmt` | `go fmt` |
| `make fmt-imports` | goimports with local prefix grouping |
| `typos --config typos.toml` | Spell check |
## Code Style and Conventions
### Error Handling
- **Forbidden**: `github.com/pkg/errors` (enforced by depguard linter)
- **Use instead**: `fmt.Errorf("context: %w", err)` for wrapping
### Import Ordering
Use `goimports` with local prefix `github.com/openkruise/kruise`. Groups in order:
1. Standard library (`fmt`, `os`, `context`, ...)
2. Third-party (`github.com/...`)
3. Kubernetes (`k8s.io/...`, `sigs.k8s.io/...`)
4. OpenKruise (`github.com/openkruise/kruise/...`)
Blank imports must have a comment explaining the side effect.
### Boilerplate
All `.go` files must include the Apache 2.0 license header from `hack/boilerplate.go.txt`.
### Spelling
US English locale (enforced by misspell linter).
### Generated Code (DO NOT EDIT)
- `pkg/client/` — generated clientset, informer, lister
- `apis/*/zz_generated.deepcopy.go` — generated DeepCopy methods
- `apis/*/openapi_generated.go` — generated OpenAPI specs
- `config/crd/bases/` — generated CRD YAML files
Regenerate with `make generate && make manifests`.
## Architecture Patterns
### Controller
- Each workload has its own package under `pkg/controller/<workload>/` with an `Add(mgr manager.Manager) error` entry point, registered in `pkg/controller/controllers.go`
- **Reconcile loops must be idempotent**: reprocessing the same event must produce the same result. Do not assume single execution
- Use expectation tracking (`pkg/util/expectations/`) to coordinate resource creation/deletion and avoid race conditions
- Use `Status().Update()` / `Status().Patch()` for status updates, not full resource updates
- Do not perform heavy operations (locking, blocking I/O) in event handlers; move them into the reconcile loop
- Check `deletionTimestamp` and handle finalizer cleanup before applying business logic
- Use `observedGeneration` in status to track whether the controller has processed the latest spec
### Webhook
- Each workload has handlers under `pkg/webhook/<workload>/`, registered via `pkg/webhook/add_<workload>.go`
- Webhooks should only do simple mutation and validation — move heavy operations into the controller reconcile loop
- Mutating webhooks: handle CREATE and UPDATE separately
- Validating webhooks: return clear, informative error messages on rejection
- Never panic in webhook handlers; recover and return `Allowed: false`
### Daemon (Kruise-Daemon)
- Runs as a node agent (`pkg/daemon/`), handling image pulling, container recreation, and pod probes via gRPC
- Communicates with the container runtime through CRI interface only — do not talk directly to Kubelet, Docker, or Containerd
- Should only access node-local Kubernetes resources or resources in the system namespace
- Entry point: `cmd/daemon/main.go`
### Feature Gates
- New features must be gated via `pkg/features/kruise_features.go` using `utilfeature.DefaultMutableFeatureGate`
- Feature gates must have unit tests behind the gate
## Code Generation Workflow
When modifying API types under `apis/`:
```
1. Edit types in apis/apps/v1alpha1/ or apis/apps/v1beta1/
2. make generate # DeepCopy, clientset, informer, lister, OpenAPI
3. make manifests # CRD YAML, RBAC
```
When adding a new API type or controller:
```
1. Define types in apis/apps/v1alpha1/<type>_types.go
2. Register scheme in apis/addtoscheme_apps_v1alpha1.go (or v1beta1)
3. Register controller in pkg/controller/controllers.go
4. Register webhook in pkg/webhook/add_<type>.go
5. make generate && make manifests
```
## Behavioral Rules
- Import order need matching goimports local prefix convention
- Don't perform blocking or locking operations in controller event handlers instead of the reconcile loop
- Don't Access cluster-scoped resources from daemon code
- Read related files before modifying code
- Don't edit `client/`, `proto/`, `config/crd/` — run `make generate` or `make manifests` instead
- After modifying `api/`, run `make generate manifests`
- Don't delete comments unless outdated
- New `.go` files need Apache 2.0 license header from `hack/boilerplate.go.txt`
- Use `Expectations` (`pkg/utils/expectations/`) for slow informer cache issues
- New APIs/architectural changes need proposal in `docs/proposals/`
- Ask user when unsure about business logic
- Always edit the files on your own, never use automation tools or scripts
- All comments must be in English
- Always commit with sign-off (e.g. `git commit -s`)