## File: cli/README.md # Woodpecker CLI Command line client for the Woodpecker continuous integration server. Please see the official documentation at --- ## File: docs/blog/2024-12-28-release-v3.0.0/index.md --- title: Woodpecker 3.0.0 slug: release-v300 authors: - name: pat-s title: Maintainer of Woodpecker url: https://github.com/pat-s image_url: https://github.com/pat-s.png tags: [release, major] hide_table_of_contents: false --- We are excited to announce the release of Woodpecker 3.0.0! Along with various cleanup improvements, you can now register your own agents as a user and replay pipelines directly from the server using cli exec. ## Breaking Changes To enhance the usability of Woodpecker and comply with evolving security standards, we periodically implement migrations. While we strive to minimize changes, some adjustments are essential for an improved user experience. We acknowledge that this release includes a significant number of changes, many of which require users to update their pipeline definitions. We understand that this can be a tedious task, especially when managing multiple repositories and pipelines. Rest assured that each modification was carefully considered and thoroughly discussed, with specific reasoning behind every decision. A substantial portion of these updates aims to transition away from outdated and suboptimal Drone definitions. Your patience and understanding as we implement these necessary changes are greatly appreciated. If you encounter any major issues during your migration to a new version, please don't hesitate to reach out. The Woodpecker maintainers are always eager to reassess and improve our updates based on your feedback. Security has been a primary focus in this major release. In addition to patching known vulnerabilities (which have also been backported to v2 releases), we have enhanced the secrets handling mechanism to prevent accidental leaks and simplify the process of keeping sensitive information fully encrypted. For a complete list of migration steps, please refer to the [migration guide](/migrations). ### `from_secret:` as the powerful replacement for the `secrets:` keyword Specifically, the `secrets:` keyword has been deprecated in favor of a more flexible (and secure) way to specify secrets: `from_secret:`. This new approach provides more flexibility (by using different names for the source and destination secrets) and ensures a safe internal secret parsing through a unified engine. Because secrets defined via `secrets:` were simple env vars in the end, this change also removes potential confusion about the differences between values specified in `environment:` and `secrets`. Now, both are defined in `environment:` using an expressive syntax: ```yaml steps: name: image: alpine commands: - echo "The secret is $TOKEN_ENV" environment: TOKEN_ENV: from_secret: SECRET_TOKEN ``` ## Register Your Own Agents for Users or Organizations [#3539](https://github.com/woodpecker-ci/woodpecker/pull/3539) WoodpeckerCI now lets you register custom agents scoped to individual users or organizations. This means you can bring your own agents, configured to meet the unique needs of your projects, and assign them to specific users or organizational workflows. This update provides flexibility for teams with diverse requirements, allowing them to integrate agents tailored to specific tasks or environments seamlessly into their pipelines. ## Replay Pipelines Locally Using `cli exec` [#4103](https://github.com/woodpecker-ci/woodpecker/pull/4103) Debugging pipelines no longer requires endless small adjustments and repeated pushes. With the new `woodpecker-cli exec` feature, you can download pipeline metadata directly from the server and replay it locally. This allows you to test and fix issues in a similar environment to the server, all from your machine. By locally debugging, this feature accelerates the development process and provides deeper insights into pipeline behavior without relying on server-side execution for every small change. :::info In order to use this feature, all required pipeline elements must be passed, e.g. secrets. However, secrets are not included in the pipeline metadata and must be passed manually to the local execution call. ::: ## Rootless images Woodpecker now supports running rootless images by adjusting the entrypoints and directory permissions in the containers in a way that allows non-privileged users to execute tasks. In addition, all images published by Woodpecker (Server, Agent, CLI) now use a non-privileged user (`woodpecker` with UID and GID `1000`) by default. If you have volumes attached to the containers, you may need to change the ownership of these directories from `root` to `woodpecker` by executing `chown -R 1000:1000 `. :::info The agent image must remain rootful by default to be able to mount the Docker socket when Woodpecker is used with the `docker` backend. The helm chart will start to use a non-privileged user by utilizing `securityContext`. Running a completely rootless agent with the `docker` backend may be possible by using a rootless docker daemon. However, this requires more work and is currently not supported. ::: ## Fine grained control over approvals options Woodpecker 3.0.0 introduces enhanced approval options. Beyond requiring approval for all pipeline events, you can now configure it specifically for all pull requests or only for pull requests originating from forks. By default, public repositories will now mandate approval for pull requests from forks. This helps prevent potentially malicious PRs from exposing secrets or performing unauthorized actions without the repository owner's awareness. ## UI We have fixed many UI-related bugs in this version. Many were small misalignment related to padding, margins or other edge cases related to small screen sizes. We also aimed to harmonize the icons across the UI, specifically across logical subgroups, such as status-icons or admin panel icons. UI elements are now sized in a relative way, meaning they will all scale relative when you change the font-size or zoom in/out. ## Deleting old pipeline logs Deleting a pipeline now successfully also deletes its related logs. Beforehand, there was an issue where the logs were not deleted and were kept in the DB forever. You might want to check [#4572](https://github.com/woodpecker-ci/woodpecker/pull/4572) for more details including a snippet how to delete orphaned entries of a Postgres DB. :::info There is no option yet to auto-delete old pipeline logs after a specific time or event. Please follow [#1068](https://github.com/woodpecker-ci/woodpecker/issues/1068) for future updates. ::: ## Migration to standard Linux CRON syntax CRON definitions now follow standard Linux syntax without seconds. An automatic migration will attempt to update your settings - ensure the update completes successfully. ## Known Issues The generic `pipeline definition not found` is still present and not yet understood. This error message can be triggered by various elements (which the most likely one being a (temporary) connection issue with the forge) and the error return/output must be improved first in order to take appropriate action. --- ## File: docs/blog/2024-05-27-release-v2.5.0/index.md --- title: Here is Woodpecker 2.5.0 slug: release-v250 authors: - name: Anbraten title: Maintainer of Woodpecker url: https://github.com/anbraten image_url: https://github.com/anbraten.png tags: [release, minor] hide_table_of_contents: false --- Here is the next minor release 2.5.0 of Woodpecker πŸͺΆ β˜€οΈ. As always thanks to all contributors who helped to make this release possible. It includes quite a few enhancements most users will benefit from while they are probably not that visible at first sight for most. The release also includes some preparations for new features to come in the next versions. Anyway, let's dive into some of the highlights of this release: ## Improve the way entrypoints work The implementation wasn't perfect yet so we improved the way entrypoints work: If you define [`commands`](/docs/usage/workflow-syntax#commands), the default entrypoint will be `["/bin/sh", "-c", "echo $CI_SCRIPT | base64 -d | /bin/sh -e"]`. If you define your own entrypoint, you can completely overwrite the default entrypoint. If you define `entrypoint: ["/bin/my-script", ""]` for example you can run your own binary / script. In this case the commands section will ignored, however you can still access it in your own script by using the base64 encoded string of the `CI_SCRIPT` environment variable. [#3269](https://github.com/woodpecker-ci/woodpecker/pull/3269) ## Cli output formats The cli output has been improved. The first command (mainly pipeline info, ls, create) support a `--output` flag now which allows you to change the output format. There is a new `table` format (the new default) which will look like the following and can be further customized: ```bash # use default table output ❯ woodpecker-cli pipeline ls --limit 2 2 NUMBER STATUS EVENT BRANCH COMMIT AUTHOR 43 error manual main 473761d8b26b20f7c206408563d54cf998410329 woodpecker 42 success push main 473761d8b26b20f7c206408563d54cf998410329 woodpecker # customize table output and disable header ❯ woodpecker-cli pipeline ls --limit 2 --output table=number,status,event --no-header 2 43 error manual 42 success push ``` In addition especially useful for programmatic usage there is a `go-template` output format which will output the data using the provided go template like this: ```bash ######## # go crazy and use a template layout ❯ woodpecker-cli pipeline ls --limit 2 --output go-template='{{range .}}{{printf "\x1b[33mPipeline #%d\x1b[0m\nStatus: %s\nEvent:%s\nCommit:%s\n\n" .Number .Status .Event .Commit}}{{end}}' 2 Pipeline #43 Status: error Event:manual Commit:473761d8b26b20f7c206408563d54cf998410329 Pipeline #42 Status: success Event:push Commit:473761d8b26b20f7c206408563d54cf998410329 ``` [#3660](https://github.com/woodpecker-ci/woodpecker/pull/3660) ## Deleting logs or complete pipelines If you accidentally exposed some secret to the public in your logs or you simply want to cleanup some logs you can now delete logs or complete pipelines using the api and the cli. [#3451](https://github.com/woodpecker-ci/woodpecker/pull/3451) [#3506](https://github.com/woodpecker-ci/woodpecker/pull/3506) [#3458](https://github.com/woodpecker-ci/woodpecker/pull/3458) ## Support for Github deploy tasks Woodpecker now supports Github deploy tasks. This allows you to pass the deploy task set in Github to your Woodpecker pipeline. [#3512](https://github.com/woodpecker-ci/woodpecker/pull/3512) ## Deprecations To keep things clean and simple we deprecated some pipeline options, server settings and features which will be removed in the next major release: - Deprecated `environment` filter, use `when.evaluate` - Use `WOODPECKER_EXPERT_FORGE_OAUTH_HOST` instead of `WOODPECKER_DEV_GITEA_OAUTH_URL` or `WOODPECKER_DEV_OAUTH_HOST` - Deprecated `WOODPECKER_WEBHOOK_HOST` in favor of `WOODPECKER_EXPERT_WEBHOOK_HOST` For a full list of deprecations that will be dropped in the `next` major release `3.0.0` (no eta yet), please check the [migrations](/migrations#next) section. --- ## File: docs/blog/2024-01-01-continuous-deployment/index.md --- title: '[Community] Continuous Deployment' slug: continuous-deployment authors: - name: lonix1 url: https://github.com/lonix1 image_url: https://github.com/lonix1.png hide_table_of_contents: false tags: [community, cd, deployment] --- A typical CI pipeline contains steps such as: _clone_, _build_, _test_, _package_ and _push_. The final build product may be artifacts pushed to a git repository or a docker container pushed to a container registry. When these should be deployed on an app server, the pipeline should include a _deploy_ step, which represents the "CD" in CI/CD - the automatic deployment of a pipeline's final product. There are various ways to accomplish CD with Woodpecker, depending on your project's specific needs. ## Invoking deploy script via SSH The final step in your pipeline could SSH into the app server and run a deployment script. One of the benefits would be that the deployment script's output could be included in the pipeline's log. However in general, this is a complicated option as it tightly couples the CI and app servers. An SSH step could be written by using a plugin, like [ssh](https://plugins.drone.io/plugins/ssh) or [git push](https://woodpecker-ci.org/plugins/git-push). ## Polling for asset changes This option completely decouples the CI and app servers, and there is no explicit deploy step in the pipeline. On the app server, one should create a script or cron job that polls for asset changes (every minute, say). When a new version is detected, the script redeploys the app. This option is easy to maintain, but the downside is a short delay (one minute) before new assets are detected. ## Using a configuration management tool If you are using a configuration management tool (e.g. Ansible, Chef, Puppet), then you could setup the last pipeline step to call that tool to perform the redeployment. A plugin for [Ansible](https://woodpecker-ci.org/plugins/ansible) exists and could be adapted accordingly. This option is complex and only suitable in an environment in which you're already using configuration management. ## Using webhooks (recommended) If your forge (GitHub, GitLab, Gitea, etc.) supports webhooks, then you could create a separate listening app that receives a webhook when new assets are available and redeploys your app. The listening "app" can be something as simple as a PHP script. Alternatively, there are a number of popular webhook servers that simplify this process, so you only need to write your actual deployment script. For example, [webhook](https://github.com/adnanh/webhook) and [webhookd](https://github.com/ncarlier/webhookd). This is arguably the simplest and most maintainable solution. --- ## File: docs/blog/2023-12-15-podman-sigstore/index.md --- title: '[Community] Podman image build with sigstore' slug: podman-image-build-sigstore authors: - name: handlebargh url: https://github.com/handlebargh image_url: https://github.com/handlebargh.png hide_table_of_contents: false tags: [community, image, podman, sigstore, signature] --- This example shows how to build a container image with podman while verifying the base image and signing the resulting image. The image being pulled uses a keyless signature, while the image being built will be signed by a pre-generated private key. ## Prerequisites ### Generate signing keypair You can use cosing or skopeo to generate the keypair. Using skopeo: ```bash skopeo generate-sigstore-key --output-prefix myKey ``` This command will generate a `myKey.private` and a `myKey.pub` keyfile. Store the `myKey.private` as secret in Woodpecker. In the example below, the secret is called `sigstore_private_key` ### Configure hosts pulling the resulting image See [here](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/building_running_and_managing_containers/assembly_signing-container-images_building-running-and-managing-containers#proc_verifying-sigstore-image-signatures-using-a-public-key_assembly_signing-container-images) on how to configure the hosts pulling the built and signed image. ## Repository structure Consider the `Makefile` having a `build` target that will be used in the following workflow. This target yields a Go binary with the filename `app` that will be placed in the root directory. ```bash . β”œβ”€β”€ Containerfile β”œβ”€β”€ main.go β”œβ”€β”€ go.mod β”œβ”€β”€ go.sum β”œβ”€β”€ .woodpecker.yml └── Makefile ``` ### Containerfile The Containerfile refers to the base image that will be verified when pulled. ```dockerfile FROM gcr.io/distroless/static-debian12:nonroot COPY app /app CMD ["/app"] ``` ### Woodpecker workflow ```yaml steps: build: image: docker.io/library/golang:1.21 pull: true commands: - make build publish: image: quay.io/podman/stable:latest # Caution: This image is built daily. It might fill up your image store quickly. pull: true # Fill in the trusted checkbox in Woodpecker's settings as well privileged: true commands: # Configure podman to use sigstore attachments for both, the registry you pull from and the registry you push to. - | printf "docker: registry.gitlab.com: use-sigstore-attachments: true gcr.io: use-sigstore-attachments: true" >> /etc/containers/registries.d/default.yaml # At pull, check the keyless sigstore signature of the distroless image. # This is a very strict container policy. It allows pulling from gcr.io/distroless only. Every other registry will be rejected. # See https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md for more information. # fulcio CA crt obtained from https://github.com/sigstore/sigstore/blob/main/pkg/tuf/repository/targets/fulcio_v1.crt.pem # rekor public key obtained from https://github.com/sigstore/sigstore/blob/main/pkg/tuf/repository/targets/rekor.pub # crt/key data is base64 encoded. --> echo "$CERT" | base64 - | printf '{ "default": [ { "type": "reject" } ], "transports": { "docker": { "gcr.io/distroless": [ { "type": "sigstoreSigned", "fulcio": { "caData": "LS0tLS1CRUdJTiBDR...QVRFLS0tLS0K", "oidcIssuer": "https://accounts.google.com", "subjectEmail": "keyless@distroless.iam.gserviceaccount.com" }, "rekorPublicKeyData": "LS0tLS1CRUdJTiBQVUJ...lDIEtFWS0tLS0tCg==", "signedIdentity": { "type": "matchRepository" } } ] }, "docker-daemon": { "": [ { "type": "reject" } ] } } }' > /etc/containers/policy.json # Use this key to sign the built image at push. - echo "$SIGSTORE_PRIVATE_KEY" > key.private # Login at the registry - echo $REGISTRY_LOGIN_TOKEN | podman login -u --password-stdin registry.gitlab.com # Build the container image - podman build --tag registry.gitlab.com///:latest . # Sign and push the image - podman push --sign-by-sigstore-private-key ./key.private registry.gitlab.com///:latest secrets: [sigstore_private_key, registry_login_token] ``` --- ## File: docs/blog/2023-12-13-debug-pipeline-steps/index.md --- title: '[Community] Debug pipeline steps' slug: debug-pipeline-steps authors: - name: anbraten url: https://github.com/anbraten image_url: https://github.com/anbraten.png hide_table_of_contents: true tags: [community, debug] --- Sometimes you want to debug a pipeline. Therefore I recently discovered: A simple step like should allow you to debug: ```yaml steps: - name: debug image: alpine commands: - curl -sSf https://sshx.io/get | sh && sshx # ^ # β”” This will open a remote terminal session and print the URL. It # should take under a second. ``` --- ## File: docs/blog/2023-12-12-podman-image-builds/index.md --- title: '[Community] Podman-in-Podman image builds' slug: podman-image-builds authors: - name: handlebargh url: https://github.com/handlebargh image_url: https://github.com/handlebargh.png hide_table_of_contents: true tags: [community, image, podman] --- I run Woodpecker CI with podman backend instead of docker and just figured out how to build images with buildah. Since I couldn't find this anywhere documented, I thought I might as well just share it here. It's actually pretty straight forward. Here's what my repository structure looks like: ```bash . β”œβ”€β”€ roundcube β”‚Β Β  β”œβ”€β”€ Containerfile β”‚Β Β  β”œβ”€β”€ docker-entrypoint.sh β”‚Β Β  └── php.ini └── .woodpecker └── .build_roundcube.yml ``` As you can see I'm building a roundcube mail image. This is the `.woodpecker/.build_roundcube.yaml` ```yaml when: event: [cron, manual] cron: build_roundcube steps: build-image: image: quay.io/buildah/stable:latest pull: true privileged: true commands: - echo $REGISTRY_LOGIN_TOKEN | buildah login -u --password-stdin registry.gitlab.com - cd roundcube - buildah build --tag registry.gitlab.com///roundcube:latest . - buildah push registry.gitlab.com///roundcube:latest secrets: [registry_login_token] ``` As you can see, I'm using this workflow over at gitlab.com. It should work with GitHub as well, with adjusting the registry login. You may have to adjust the `when:` to your needs. Furthermore, you must check the `trusted` checkbox in project settings. Therefore, be sure to run trusted code only in this setup. This seems to work fine so far. I wonder if anybody else made this work a different way. EDIT: Removed the additional step that would run buildah in a podman container. I didn't know it could be that easy to be honest. --- ## File: docs/blog/2023-11-09-release-v2.0.0/index.md --- title: It's time for some changes - Woodpecker 2.0.0 slug: release-v200 authors: - name: Anbraten title: Maintainer of Woodpecker url: https://github.com/anbraten image_url: https://github.com/anbraten.png - name: qwerty287 title: Maintainer of Woodpecker url: https://github.com/qwerty287 image_url: https://github.com/qwerty287.png tags: [release, major] hide_table_of_contents: false --- We are proud to present you Woodpecker v2.0.0 with more than 350 changes from our fabulous community. This release includes a lot of new features, improvements and some breaking changes which most of you probably already tested using the `next` tag or the RC version. ## How we plan to handle releases in the future In the future, there won't be backports anymore as they require quite an amount of maintenance. Instead, we'll release our current state of the `main` branch with the correct version (according to semver) every few weeks. Of course, critical bug and security fixes are released as soon as possible. To not release new major version too often, we'll try to hold back breaking changes pull-request for a longer time and release them all together in a new major version. ## Breaking changes ### Renamed some api routes We renamed some API routes to be more consistent. So we suggest admins to update all repository webhooks by clicking on the newly added `Repair all repositories` button in the admin settings. ### Dropped deprecated environment variables and CLI commands For v1.0.0, we deprecated a bunch of old environment variables like `CI_BUILD_*`. These variables were removed in this version, you therefore have to use the new ones. Also, the deprecated `build` command of the CLI was removed. Use `pipeline` instead. ### Removed SSH backend Due to various issues with the SSH backend we decided to remove it. As an alternative, you can install an agent running the local backend directly on the remote machine or you can simply execute `ssh` commands connecting to the remote server in your pipeline. ### Deprecated `platform` filter The `platform` filter has been removed. Use the more advanced labels instead ([read more](../docs/usage/workflow-syntax#filter-by-platform)). ### Update Docker to v24 We updated Docker to v24 as of some security patches. If you use an older version of Docker, you might need to upgrade it. ### Removed plugin-only option from secrets Security is pretty important to us and we want to make sure that no one can steal your secrets. Therefore, we decided to remove the plugin-only option from secrets and instead, if you define an image filter, it will be automatically only available to plugins using the defined image names. ## Migration notes There have been a few more breaking changes. [Read more about what you need to do when upgrading!](/migrations#200) ## New features But that's enough about breaking changes. Let's talk about the new features! ### Config warnings and errors in the UI You ever wondered why a secret was not working and after hours of debugging you found out that you misspelled the secret name? Or you used a wrong key in your YAML config? Woodpecker now shows errors and linter warnings directly in it's UI, notifying you about missing secrets, incorrect configuration or deprecated settings! ### Repository and organization overview for admins Admins now get an overview over all repositories and organizations registered on the server, allowing them to perform common actions like deleting directly from the admin dashboard. ### Support for user secrets It is now possible to add secrets for all repos owned by yourself, similar to organization and global secrets. ### Bitbucket cloud support for multi-workflows We enhanced support for Bitbucket, allowing you to use multiple workflows just as you probably know from all other forges already. ### Full support for Kubernetes backend Many of you already used it extensively in the past, but now we can finally call the Kubernetes backend ready for production use. Supporting all major features and even quite some Kubernetes specific options. ### Auto theme The UI now supports automatically adapting the theme to your browser config, so no more light mode in the middle of the night! ### Update notification Updates are awesome as they bring new features and bug fixes most of the time, but sometimes there are also important security fixes which should be installed as soon as possible. To not miss any of them, we added a notification to the UI for admins if there's a new update available. ## Changelog The full changelog can be viewed in our project source folder at [CHANGELOG.md](https://github.com/woodpecker-ci/woodpecker/blob/v2.0.0/CHANGELOG.md) --- ## File: docs/blog/2023-07-28-release-v1.0.0/index.md --- title: Presenting Woodpecker 1.0.0 slug: release-v100 authors: - name: 6543 title: Maintainer of Woodpecker url: https://github.com/6543 image_url: https://github.com/6543.png tags: [release, major] hide_table_of_contents: false --- We are proud to present you Woodpecker v1.0.0. It took us quite some time, but now we are sure it's ready, and you should really have a look at it. We've refactored a lot of code, so contributing to the codebase should be much easier. Furthermore, a ton of bugs where addressed and various enhancements introduced, along with some highly anticipated features. With Woodpecker v1.0.0, you can now substantially improve and streamline your code pipelines, empowering you to automate and optimize workflows like never before. ## Some picked highlights ### Add Support for Cron Jobs Automate recurring tasks with ease using Woodpecker's new cron jobs feature. Schedule pipelines to run at specified intervals or times, optimizing repetitive workflows. ### YAML Map Merge, Overrides, and Sequence Merge Support With enhanced YAML support, managing complex configurations becomes a breeze. Merge maps, apply overrides, and sequence mergingβ€”all within your YAML files. This is providing more flexibility and control over your pipelines. ### Web-UI for Admins Simplify administration tasks with Woodpecker's new Admin UI. Effortlessly manage user accounts, agents, and tasks, including adding new agents or pausing the task queue for maintenance. ### Localize Web-UI Embrace internationalization by changing your locale in the user settings. Interact with Woodpecker in the language of your choice, tailored to your preferences. If your language is not available or only partially translated, consider contributing to our [Weblate](https://translate.woodpecker-ci.org/engage/woodpecker-ci/). ### Add `evaluate` to `when` Filter Enhance pipeline flexibility with the new "when evaluate" filter, enabling or disabling steps based on custom conditions. Customize your workflows to dynamically respond to specific triggers and events. ### Global- and Organization-Secrets Save time and effort by declaring secrets for your entire instance or organization. Simplify your workflow and securely manage sensitive information across projects. ## Changelog The full changelog can be viewed in our project source folder at [CHANGELOG.md](https://github.com/woodpecker-ci/woodpecker/blob/v1.0.0/CHANGELOG.md) --- ## File: docs/blog/2023-06-11-hello-blog/index.md --- title: Welcome Woodpecker's blog slug: hello-blog authors: - name: Anbraten title: Maintainer of Woodpecker url: https://github.com/anbraten image_url: https://github.com/anbraten.png tags: [hello, woodpecker] hide_table_of_contents: false --- Welcome to this blog. This is our first post on this blog ... In the future we will post about our releases and other things like tutorials. We are currently working on the `1.0.0` release of Woodpecker. This release will include a lot of new features and improvements which most of you probably already tested using the `next` tag. If you have any suggestions or ideas for posts, feel free to open an issue in the [GitHub repository](https://github.com/woodpecker-ci/woodpecker). --- ## File: docs/docs/92-development/01-getting-started.md # Getting started ## Nix development shell If you use [Nix](https://nixos.org/), you can skip the install steps below. The repository contains a `flake.nix` that does it all for you: ```bash nix develop ``` This opens a shell with Go, make, Node.js, `pnpm`, `protoc` and all development tools. The tools are built with the same Go version as Woodpecker. Only [`pre-commit`](#install-pre-commit-optional) is not included. ## Install Go Install Golang as described by [this guide](https://go.dev/doc/install). You need at least the version that is set in the `go.mod` file. :::info An older Go version can still build Woodpecker, as it downloads the needed toolchain on its own. But tools that are installed with `go install`, like `golangci-lint`, are then built with the old version and refuse to work: ```none Error: can't load config: the Go language version (go1.26) used to build golangci-lint is lower than the targeted Go version (1.27.0) ``` In this case update Go and install the tools again. ::: ## Install make > GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files (). Install make on: - Ubuntu: `apt install make` - [Docs](https://wiki.ubuntuusers.de/Makefile/) - [Windows](https://stackoverflow.com/a/32127632/8461267) - Mac OS: `brew install make` ## Install development tools (optional) The make targets use some tools to lint, format and generate code. You can install them with: ```bash make install-tools ``` They are installed with `go install`, so make sure that the `bin` folder of your Go path (`go env GOPATH`) is part of your `PATH`. If a tool is not installed, `make lint`, `make format` and `make generate` start it with `go run` instead. This works too, but is slower. To generate the gRPC code, which is part of `make generate`, you also need [`protoc`](https://protobuf.dev/installation/). It is not installed by `make install-tools`. ## Install Node.js & `pnpm` Install [Node.js](https://nodejs.org/en/download/package-manager) if you want to build Woodpecker's UI or documentation. For dependency installation (`node_modules`) of UI and documentation of Woodpecker the package manager pnpm is used. [This guide](https://pnpm.io/installation) describes the installation of `pnpm`. ## Install `pre-commit` (optional) Woodpecker uses [`pre-commit`](https://pre-commit.com/) to allow you to easily autofix your code. To apply it during local development, take a look at [`pre-commit`s documentation](https://pre-commit.com/#usage). ## Create a `.env` file with your development configuration Similar to the environment variables you can set for your production setup of Woodpecker, you can create a `.env` file in the root of the Woodpecker project and add any needed config to it. A common config for debugging would look like this: ```ini WOODPECKER_OPEN=true WOODPECKER_ADMIN=your-username WOODPECKER_HOST=http://localhost:8000 # github (sample for a forge config - see /docs/administration/forge/overview for other forges) WOODPECKER_GITHUB=true WOODPECKER_GITHUB_CLIENT= WOODPECKER_GITHUB_SECRET= # agent WOODPECKER_SERVER=localhost:9000 WOODPECKER_AGENT_SECRET=a-long-and-secure-password-used-for-the-local-development-system WOODPECKER_MAX_WORKFLOWS=1 # enable if you want to develop the UI # WOODPECKER_DEV_WWW_PROXY=http://localhost:8010 # if you want to test webhooks with an online forge like GitHub this address needs to be set and accessible from public server WOODPECKER_EXPERT_WEBHOOK_HOST=http://your-address.com # disable health-checks while debugging (normally not needed while developing) WOODPECKER_HEALTHCHECK=false # WOODPECKER_LOG_LEVEL=debug # WOODPECKER_LOG_LEVEL=trace ``` ## Setup OAuth Create an OAuth app for your forge as described in the [forges documentation](../30-administration/10-configuration/12-forges/11-overview.md). ## Developing with VS Code You can use different methods for debugging the Woodpecker applications. One of the currently recommended ways to debug and test the Woodpecker application is using [VS-Code](https://code.visualstudio.com/) or [VS-Codium](https://vscodium.com/) (Open-Source binaries of VS-Code) as most maintainers are using it and Woodpecker already includes the needed debug configurations for it. To launch all needed services for local development, you can use "Woodpecker CI" debugging configuration that will launch UI, server and agent in debugging mode. Then open `http://localhost:8000` to access it. As a starting guide for programming Go with VS Code, you can use this video guide: [](https://www.youtube.com/watch?v=1MXIGYrMk80) ### Debugging Woodpecker The Woodpecker source code already includes launch configurations for the Woodpecker server and agent. To start debugging you can click on the debug icon in the navigation bar of VS-Code (ctrl-shift-d). On that page you will see the existing launch jobs at the top. Simply select the agent or server and click on the play button. You can set breakpoints in the source files to stop at specific points. ## Testing & linting code To test or lint parts of Woodpecker, you can run one of the following commands: ```bash title="test server code" make test-server ``` ```bash title="test agent code" make test-agent ``` ```bash title="test cli code" make test-cli ``` ```bash title="test datastore/database related code like migrations of the server" make test-server-datastore ``` ```bash title="lint go code" make lint ``` ```bash title="lint UI code" make lint-ui ``` ```bash title="test UI code" make test-ui ``` If you want to test a specific Go file, you can also use: ```bash go test -race -timeout 30s -tags test go.woodpecker-ci.org/woodpecker/v3/ ``` :::warning Always pass `-tags test`. Many test files and test helpers, like the [dummy backend](./09-testing.md#dummy-backend), are only compiled with the `test` build tag. Without it `go test` skips these tests without any hint! Make targets do it for you. ::: Or you can open the test-file inside [VS-Code](#developing-with-vs-code) and run or debug the test by clicking on the inline commands: ## Run applications from terminal If you want to run a Woodpecker applications from your terminal, you can use one of the following commands from the base of the Woodpecker project. They will execute Woodpecker in a similar way as described in [debugging Woodpecker](#debugging-woodpecker) without the ability to really debug it in your editor. ```bash title="start server" go run ./cmd/server ``` :::info The server embeds the web UI from `web/dist/` at compile time. If the UI is not built yet, compiling and running the server with `go run` will fail with: ```none web/web.go:26:12: pattern all:dist/*: no matching files found ``` Either build the UI once with `make build-ui` or create a placeholder with: ```bash mkdir -p web/dist/ && echo "test" > web/dist/index.html ``` ::: ```bash title="start agent" go run ./cmd/agent ``` ```bash title="execute cli command" go run ./cmd/cli [command] ``` --- ## File: docs/docs/92-development/02-core-ideas.md # Core ideas - A configuration (e.g. of a pipeline) should never be [turing complete](https://en.wikipedia.org/wiki/Turing_completeness) (We have agents to exec things πŸ™‚). - If possible, follow the [KISS principle](https://en.wikipedia.org/wiki/KISS_principle). - What is used most often should be default. - Keep different topics separated, so you can write plugins, port new ideas ... more easily, see [Architecture](./05-architecture.md). ## Addons and extensions If you are wondering whether your contribution will be accepted to be merged in the Woodpecker core, or whether it's better to write an [addon](../30-administration/10-configuration/100-addons.md), [extension](../20-usage/72-extensions/40-configuration-extension.md) or an [external custom backend](../30-administration/10-configuration/11-backends/50-custom.md), please check these points: - Is your change very specific to your setup and unlikely to be used by anyone else? - Does your change violate the [guidelines](#guidelines)? Both should be false when you open a pull request to get your change into the core repository. ### Guidelines #### Forges A new forge must support these features: - OAuth2 - Webhooks