## 1. Project Overview & Quickstart (dexidp/LICENSE) # LICENSE Open-source repository dexidp/LICENSE ### Repository Details - **Repository:** [dexidp/LICENSE](https://github.com/dexidp/LICENSE) - **Primary Language:** Code *Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.* ## 2. Official Technical Reference & Guides (dexidp/website) ## File: README.md # CNCF Hugo Starter This repository contains a boilerplate static site generator setup for creating CNCF documentation projects. We strongly recommend using this setup (it helps us help you and your project!), but none of the technologies in the stack are strictly required. The starter uses the following: * **[Hugo](https://gohugo.io/)** as a static site generator * **[Docsy](https://www.docsy.dev/)** as a documentation theme * **[Netlify](https://www.netlify.com/)** for building, hosting, and DNS management ## Running locally Make sure you have [npm](https://www.npmjs.com/) and [yarn](https://yarnpkg.com/) installed. Clone this repository and run the following two commands in its directory: ```shell # Run the server locally make serve ``` ## Running on Netlify Netlify is a CI/CD build tool and hosting solution for (among other things) static sites. We **strongly** recommend using Netlify unless you have a good reason not to. This repository comes with a pre-configured [`netlify.toml`](https://github.com/cncf/hugo-netlify-starter/blob/master/netlify.toml) file. To build to Netlify: 1. Go to [netlify.com](https://netlify.com) and sign up. We recommend signing up using a GitHub account. 2. Click **New Site from Git**, and give Netlify access to your GitHub account. > **Note:** For projects with lots of contributors, it can be handy to create a general/bot account instead of granting access with a personal account. 3. Install Netlify with access to your documentation site repository. 4. Leave all other settings as default and click **Deploy Site**. --- ## File: content/docs/archive/proposals/_index.md --- title: "Proposals" date: 2020-01-07T14:59:38+01:00 draft: true toc: true --- --- ## File: content/docs/archive/proposals/token-revocation.md --- title: "Proposal: Design for Revoking Refresh Tokens" linkTitle: "Design for Revoking Refresh Tokens" date: 2020-09-30 draft: true toc: true weight: 20 --- Refresh tokens are issued to the client by the authorization server and are used to request a new access token when the current access token becomes invalid or expires. It is a common usecase for the end users to revoke client access to their identity. This proposal defines the changes needed in Dex v2 to support refresh token revocation. ## Motivation 1. Currently refresh tokens are not associated with the user. Need a new "session object" for this. 2. Need an API to list refresh tokens based on the UserID. 3. We need a way for users to login to dex and revoke a client. 4. Limit the number refresh tokens for each user-client pair to 1. ## Details Currently in Dex when an end user successfully logs in via a connector and has the OfflineAccess scope set to true, a refresh token is created and stored in the backing datastore. There is no association between the end user and the refresh token. Hence if we want to support the functionality of users being able to revoke refresh tokens, the first step is to have a structure in place that allows us retrieve a list of refresh tokens depending on the authenticated user. ```go // Reference object for RefreshToken containing only metadata. type RefreshTokenRef struct { // ID of the RefreshToken ID string CreatedAt time.Time LastUsed time.Time } // Session objects pertaining to users with refresh tokens. // // Will have to handle garbage collection i.e. if no refresh token exists for a user, // this object must be cleaned up. type OfflineSession struct { // UserID of an end user who has logged into the server. UserID string // The ID of the connector used to login the user. ConnID string // List of pointers to RefreshTokens issued for SessionID Refresh []*RefreshTokenRef } // Retrieve OfflineSession obj for given userId and connID func getOfflineSession (userId string, connID string) ``` ### Changes in Dex CodeFlows 1. Client requests a refresh token: Try to retrieve the `OfflineSession` object for the User with the given `UserID + ConnID`. This leads to two possibilities: * Object exists: This means a Refresh token already exists for the user. Update the existing `OfflineSession` object with the newly received token as follows: * CreateRefresh() will create a new `RefreshToken` obj in the storage. * Update the `Refresh` list with the new `RefreshToken` pointer. * Delete the old refresh token in storage. * No object found: This implies that this will be the first refresh token for the user. * CreateRefresh() will create a new `RefreshToken` obj in the storage. * Create an OfflineSession for the user and add the new `RefreshToken` pointer to the `Refresh` list. 2. Refresh token rotation: There will be no change to this codeflow. When the client refreshes a refresh token, the `TokenID` still remains intact and only the `RefreshToken` obj gets updated with a new nonce. We do not need any additional checks in the OfflineSession objects as the `RefreshToken` pointers still remain intact. 3. User revokes a refresh token (New functionality): A user that has been authenticated externally will have the ability to revoke their refresh tokens. Please note that Dex's API does not perform the authentication, this will have to be done by an external app. Steps involved: * Get `OfflineSession` obj with given UserID + ConnID. * If a refresh token exists in `Refresh`, delete the `RefreshToken` (handle this in storage) and its pointer value in `Refresh`. Clean up the OfflineSession object. * If there is no refresh token found, handle error case. NOTE: To avoid race conditions between “requesting a refresh token” and “revoking a refresh token”, use locking mechanism when updating an `OfflineSession` object. --- ## File: content/docs/archive/proposals/upstream-refreshing.md --- title: "Proposal: Upstream Refreshing" linkTitle: "Upstream Refreshing" date: 2020-09-30 draft: true toc: true weight: 20 --- ## TL;DR Today, if a user deletes their GitHub account, dex will keep allowing clients to refresh tokens on that user's behalf because dex never checks back in with GitHub. This is a proposal to change the connector package so the dex can check back in with GitHub. ## The problem When dex is federating to an upstream identity provider (IDP), we want to ensure claims being passed onto clients remain fresh. This includes data such as Google accounts display names, LDAP group membership, account deactivations. Changes to these on an upstream IDP should always be reflected in the claims dex passes to its own clients. Refresh tokens make this complicated. When refreshing a token, unlike normal logins, dex doesn't have the opportunity to prompt for user interaction. For example, if dex is proxying to a LDAP server, it won't have the user's username and passwords. Dex can't do this today because connectors have no concept of checking back in with an upstream provider (with the sole exception of groups). They're only called during the initial login, and never consulted when dex needs to mint a new refresh token for a client. Additionally, connectors aren't actually aware of the scopes being requested by the client, so they don't know when they should setup the ability to check back in and have to treat every request identically. ## Changes to the connector package The biggest changes proposed impact the connector package and connector implementations. 1. Connectors should be consulted when dex attempts to refresh a token. 2. Connectors should be aware of the scopes requested by the client. The second bullet is important because of the first. If a client isn't requesting a refresh token, the connector shouldn't do the extra work, such as requesting additional upstream scopes. to address the first point, a top level `Scopes` object will be added to the connector package to express the scopes requested by the client. The `CallbackConnector` and `PasswordConnector` will be updated accordingly. ```go // Scopes represents additional data requested by the clients about the end user. type Scopes struct{ // The client has requested a refresh token from the server. OfflineAccess bool // The client has requested group information about the end user. Groups bool } // CallbackConnector is an interface implemented by connectors which use an OAuth // style redirect flow to determine user information. type CallbackConnector interface { // The initial URL to redirect the user to. // // OAuth2 implementations should request different scopes from the upstream // identity provider based on the scopes requested by the downstream client. // For example, if the downstream client requests a refresh token from the // server, the connector should also request a token from the provider. LoginURL(s Scopes, callbackURL, state string) (string, error) // Handle the callback to the server and return an identity. HandleCallback(s Scopes, r *http.Request) (identity Identity, state string, err error) } // PasswordConnector is an interface implemented by connectors which take a // username and password. type PasswordConnector interface { Login(s Scopes, username, password string) (identity Identity, validPassword bool, err error) } ``` The existing `GroupsConnector` plays two roles. 1. The connector only attempts to grab groups when the downstream client requests it. 2. Allow group information to be refreshed. The first issue is remedied by the added `Scopes` struct. This proposal also hopes to generalize the need of the second role by adding a more general `RefreshConnector`: ```go type Identity struct { // Existing fields... // Groups are added to the identity object, since connectors are now told // if they're being requested. // The set of groups a user is a member of. Groups []string } // RefreshConnector is a connector that can update the client claims. type RefreshConnector interface { // Refresh is called when a client attempts to claim a refresh token. The // connector should attempt to update the identity object to reflect any // changes since the token was last refreshed. Refresh(s Scopes, identity Identity) (Identity, error) // TODO(ericchiang): Should we allow connectors to indicate that the user has // been delete or an upstream token has been revoked? This would allow us to // know when we should remove the downstream refresh token, and when there was // just a server error, but might be hard to determine for certain protocols. // Might be safer to always delete the downstream token if the Refresh() // method returns an error. } ``` ## Example changes to the "passwordDB" connector The `passwordDB` connector is the internal connector maintained by the server. As an example, these are the changes to that connector if this change was accepted. ```go func (db passwordDB) Login(s connector.Scopes, username, password string) (connector.Identity, bool, error) { // No change to existing implementation. Scopes can be ignored since we'll // always have access to the password objects. } func (db passwordDB) Refresh(s connector.Scopes, identity connector.Identity) (connector.Identity, error) { // If the user has been deleted, the refresh token will be rejected. p, err := db.s.GetPassword(identity.Email) if err != nil { if err == storage.ErrNotFound { return connector.Identity{}, errors.New("user not found") } return connector.Identity{}, fmt.Errorf("get password: %v", err) } // User removed but a new user with the same email exists. if p.UserID != identity.UserID { return connector.Identity{}, errors.New("user not found") } // If a user has updated their username, that will be reflected in the // refreshed token. identity.Username = p.Username return identity, nil } ``` ## Caveats Certain providers, such as Google, will only grant a single refresh token for each client + end user pair. The second time one's requested, no refresh token is returned. This means refresh tokens must be stored by dex as objects on an upstream identity rather than part of a downstream refresh even. Right now `ConnectorData` is too general for this since it is only stored with a refresh token and can't be shared between sessions. This should be rethought in combination with the [`user-object.md`](./user-object.md) proposal to see if there are reasonable ways for us to do this. This isn't a problem for providers like GitHub because they return the same refresh token every time. We don't need to track a token per client. --- ## File: content/docs/archive/proposals/user-object.md --- title: "Proposal: User Objects for Revoking Refresh Tokens and Merging Accounts" linkTitle: "User Objects for Revoking Refresh Tokens and Merging Accounts" date: 2020-09-30 draft: true toc: true weight: 20 --- Certain operations require tracking users the have logged in through the server and storing them in the backend. Namely, allowing end users to revoke refresh tokens and merging existing accounts with upstream providers. While revoking refresh tokens is relatively easy, merging accounts is a difficult problem. What if display names or emails are different? What happens to a user with two remote identities with the same upstream service? Should this be presented differently for a user with remote identities for different upstream services? This proposal only covers a minimal merging implementation by guaranteeing that merged accounts will always be presented to clients with the same user ID. This proposal defines the following objects and methods to be added to the storage package to allow user information to be persisted. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` `UserID` fields will be added to the `AuthRequest`, `AuthCode` and `RefreshToken` structs. When a user logs in successfully through a connector [here](https://github.com/dexidp/dex/blob/95a61454b522edd6643ced36b9d4b9baa8059556/server/handlers.go#L227), the server will attempt to either get the user, or create one if none exists with the remote identity. `AuthorizedClients` serves two roles. First is makes displaying the set of clients a user is logged into easy. Second, because we don't assume multi-object transactions, we can't ensure deleting all refresh tokens a client has for a user. Between listing the set of refresh tokens and deleting a token, a client may have already redeemed the token and created a new one. When an OAuth2 client exchanges a code for a token, the following steps are taken to populate the `AuthorizedClients`: 1. Get token where the user has authorized the `offline_access` scope. 1. Update the user checking authorized clients. If client is not in the list, add it. 1. Create a refresh token and return the token. When a OAuth2 client attempts to renew a refresh token, the server ensures that the token hasn't been revoked. 1. Check authorized clients and update the `LastRefreshed` timestamp. If client isn't in list error out and delete the refresh token. 1. Continue renewing the refresh token. When the end user revokes a client, the following steps are used to. 1. Update the authorized clients by removing the client from the list. This atomic action causes any renew attempts to fail. 1. Iterate through list of refresh tokens and garbage collect any tokens issued by the user for the client. This isn't atomic, but exists so a user can re-authorize a client at a later time without authorizing old refresh tokens. This is clunky due to the lack of multi-object transactions. E.g. we can't delete all the refresh tokens at once because we don't have that guarantee. Merging accounts becomes extremely simple. Just add another remote identity to the user object. We hope to provide a web interface that a user can login to to perform these actions. Perhaps using a well known client issued exclusively for the server. The new `User` object requires adding the following methods to the storage interface, and (as a nice side effect) deleting the `ListRefreshTokens()` method. ```go type Storage interface { // ... CreateUser(u User) error DeleteUser(id string) error GetUser(id string) error GetUserByRemoteIdentity(connectorID, connectorUserID string) (User, error) // Updates are assumed to be atomic. // // When a UpdateUser is called, if clients are removed from the // AuthorizedClients list, the underlying storage SHOULD clean up refresh // tokens issued for the removed clients. This allows backends with // multi-transactional capabilities to utilize them, while key-value stores // only guarantee best effort. UpdateUser(id string, updater func(old User) (User, error)) error } ``` Importantly, this will be the first object which has a secondary index. The Kubernetes client will simply list all the users in memory then iterate over them to support this (possibly followed by a "watch" based optimization). SQL implementations will have an easier time. --- ## File: content/docs/archive/_index.md --- title: "Archive" date: 2020-01-07T14:59:38+01:00 draft: false toc: true weight: 9999 --- --- ## File: content/docs/archive/integrations.md --- title: "Integrations" date: 2020-09-30 draft: false toc: true weight: 1100 --- This document tracks the libraries and tools that are compatible with dex. [Join the community](https://github.com/dexidp/dex/), and help us keep the list up-to-date. ## Tools ## Projects with a dex dependency --- ## File: content/docs/archive/v2.md --- title: "Dex v2" linkTitle: "What's new in v2" date: 2020-09-30 draft: false toc: true weight: 1030 --- ## Streamlined deployments Many of the changes between v1 and v2 were aimed at making dex easier to deploy and manage, perhaps the biggest pain point for dex v1. Dex is now a single, scalable binary with a sole source of configuration. Many components which previously had to be set through the API, such as OAuth2 clients and IDP connectors can now be specified statically. The new architecture lacks a singleton component eliminating deployment ordering. There are no more special development modes; instructions for running dex on a workstation translate with minimal changes to a production system. All of this results in a much simpler deployment story. Write a config file, run the dex binary, and that's it. ## More storage backends Dex's internal storage interface has been improved to support multiple backing databases including Postgres, SQLite3, and the Kubernetes API through Third Party Resources. This allows dex to meet a more diverse set of use cases instead of insisting on one particular deployment pattern. For example, The Kubernetes API implementation, a [key value store][k8s-api-docs], allows dex to be run natively on top of a Kubernetes cluster with extremely little administrative overhead. Starting with support for multiple storage backends also should help ensure that the dex storage interface is actually pluggable, rather than being coupled too tightly with a single implementation. A more in depth discussion of existing storage options and how to add new ones can be found [here][storage-docs]. ## Additional improvements The rewrite came with several, miscellaneous improvements including: * More powerful connectors. For example the GitHub connector can now query for teams. * Combined the two APIs into a single [gRPC API][api-docs] with no complex authorization rules. * Expanded OAuth2 capabilities such as the implicit flow. * Simplified codebase and improved testing. ## Rethinking registration Dex v1 performed well when it could manage users. It provided features such as registration, email invites, password resets, administrative abilities, etc. However, login flows and APIs remain tightly coupled with concepts like registration and admin users even when v1 federated to an upstream identity provider (IDP) where it likely only had read only access to the actual user database. Many of v2's use cases focus on federation to other IPDs rather than managing users itself. Because of this, options associated with registration, such as SMTP credentials, have been removed. We hope to add registration and user management back into the project through orthogonal applications using the [gRPC API][api-docs], but in a way that doesn't impact other use cases. ## Removed features Dex v2 lacks certain features present in v1. For the most part _we aim to add most of these features back into v2_, but in a way that installations have to _opt in_ to a feature instead of burdening every deployment with extra configuration. Notable missing features include: * Registration flows. * Local user management. * SMTP configuration and email verification. * Several of the login connectors that have yet to be ported. ## Support for dex v1 Dex v1 will continue to live under the `github.com/dexidp/dex` repo on a branch. Bug fixes and minor changes will continue to be accepted, but development of new features by the dex team will largely cease. [k8s-api-docs]: http://kubernetes.io/docs/api/ [storage-docs]: /docs/configuration/storage [api-docs]: /docs/configuration/api --- ## File: content/docs/configuration/_index.md --- title: "Configuration" date: 2020-01-07T14:59:38+01:00 draft: false toc: true weight: 2000 --- --- ## File: content/docs/configuration/api.md --- title: "The Dex API" linkTitle: "gRPC API" date: 2020-09-30 draft: false toc: true weight: 1060 --- Dex provides a [gRPC](http://www.grpc.io/) service for programmatic modification of dex's state. The API is intended to expose hooks for management applications and is not expected to be used by most installations. This document is an overview of how to interact with the API. ## Configuration Admins that wish to expose the gRPC service must add the following entry to the dex config file. This option is off by default. ```yaml grpc: # Cannot be the same address as an HTTP(S) service. addr: 127.0.0.1:5557 # Server certs. If TLS credentials aren't provided dex will run in plaintext (HTTP) mode. tlsCert: /etc/dex/grpc.crt tlsKey: /etc/dex/grpc.key # Client auth CA. tlsClientCA: /etc/dex/client.crt # enable reflection reflection: true ``` ## Clients gRPC is a suite of tools for generating client and server bindings from a common declarative language. The canonical schema for Dex's API can be found in the source tree at [`api/v2/api.proto`](https://github.com/dexidp/dex/blob/master/api/v2/api.proto). Go bindings are generated and maintained in the same directory for both public and internal use. ### Go A Go project can import the API module directly, without having to import the entire project: ```bash go get github.com/dexidp/dex/api/v2 ``` The client then can be used as follows: ```go package main import ( "context" "fmt" "log" "github.com/dexidp/dex/api/v2" "google.golang.org/grpc" "google.golang.org/grpc/credentials" ) func newDexClient(hostAndPort, caPath string) (api.DexClient, error) { creds, err := credentials.NewClientTLSFromFile(caPath, "") if err != nil { return nil, fmt.Errorf("load dex cert: %v", err) } conn, err := grpc.Dial(hostAndPort, grpc.WithTransportCredentials(creds)) if err != nil { return nil, fmt.Errorf("dial: %v", err) } return api.NewDexClient(conn), nil } func main() { client, err := newDexClient("127.0.0.1:5557", "/etc/dex/grpc.crt") if err != nil { log.Fatalf("failed creating dex client: %v ", err) } req := &api.CreateClientReq{ Client: &api.Client{ Id: "example-app", Name: "Example App", Secret: "ZXhhbXBsZS1hcHAtc2VjcmV0", RedirectUris: []string{"http://127.0.0.1:5555/callback"}, }, } if _, err := client.CreateClient(context.TODO(), req); err != nil { log.Fatalf("failed creating oauth2 client: %v", err) } } ``` A clear working example of the Dex gRPC client for Go can be found [here](https://github.com/dexidp/dex/tree/master/examples/grpc-client/README.md). ### Other languages To generate a client for your own project install [`protoc`](https://github.com/google/protobuf/releases), install a protobuf generator for your project's language, and download the `api.proto` file. Here is an example: ```bash # Download api.proto for a given version. $ DEX_VERSION=v2.24.0 $ wget https://raw.githubusercontent.com/dexidp/dex/${DEX_VERSION}/api/v2/api.proto # Generate the client bindings. $ protoc [YOUR LANG PARAMS] api.proto ``` Client programs can then be written using the generated code. ## Authentication and access control The Dex API does not provide any authentication or authorization beyond TLS client auth. Projects that wish to add access controls on top of the existing API should build apps which perform such checks. For example to provide a "Change password" screen, a client app could use Dex's OpenID Connect flow to authenticate an end user, then call Dex's API to update that user's password. ## dexctl? Dex does not ship with a command line tool for interacting with the API. Command line tools are useful but hard to version, easy to design poorly, and expose another interface which can never be changed in the name of compatibility. While the Dex team would be open to re-implementing `dexctl` for v2 a majority of the work is writing a design document, not the actual programming effort. ## Why not REST or gRPC Gateway? Between v1 and v2, Dex switched from REST to gRPC. This largely stemmed from problems generating documentation, client bindings, and server frameworks that adequately expressed REST semantics. While [Google APIs](https://github.com/google/apis-client-generator), [Open API/Swagger](https://openapis.org/), and [gRPC Gateway](https://github.com/grpc-ecosystem/grpc-gateway) were evaluated, they often became clunky when trying to use specific HTTP error codes or complex request bodies. As a result, v2's API is entirely gRPC. Many arguments _against_ gRPC cite short term convenience rather than production use cases. Though this is a recognized shortcoming, Dex already implements many features for developer convenience. For instance, users who wish to manually edit clients during testing can use the `staticClients` config field instead of the API.