# Technical Documentation: willnorris/imageproxy > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/willnorris/imageproxy) ### Overview (`README.md`) # imageproxy [](https://pkg.go.dev/willnorris.com/go/imageproxy) [](https://github.com/willnorris/imageproxy/actions?query=workflow%3Atests) [](https://codecov.io/gh/willnorris/imageproxy) [](https://bestpractices.coreinfrastructure.org/projects/2611) imageproxy is a caching image proxy server written in go. It features: - basic image adjustments like resizing, cropping, and rotation - access control using allowed hosts list or request signing (HMAC-SHA256) - support for jpeg, png, webp (decode only), tiff, and gif image formats (including animated gifs) - caching in-memory, on disk, or with Amazon S3, Google Cloud Storage, Azure Storage, or Redis - easy deployment, since it's pure go Personally, I use it primarily to dynamically resize images hosted on my own site (read more in [this post][]). But you can also enable request signing and use it as an SSL proxy for remote images, similar to [atmos/camo][] but with additional image adjustment options. I aim to keep imageproxy compatible with the two [most recent major go releases][]. I also keep track of the minimum go version that still works (currently go1.18), but that might change at any time. You can see the go versions that are tested against in [.github/workflows/tests.yml][]. [this post]: https://willnorris.com/2014/01/a-self-hosted-alternative-to-jetpacks-photon-service [atmos/camo]: https://github.com/atmos/camo [most recent major go releases]: https://golang.org/doc/devel/release.html [.github/workflows/tests.yml]: ./.github/workflows/tests.yml ## URL Structure imageproxy URLs are of the form `http://localhost/{options}/{remote_url}`. ### Options Options are available for cropping, resizing, rotation, flipping, and digital signatures among a few others. Options for are specified as a comma delimited list of parameters, which can be supplied in any order. Duplicate parameters overwrite previous values. See the full list of available options at . ### Remote URL The URL of the original image to load is specified as the remainder of the path. It may be included in plain text without any encoding, percent-encoded (aka URL encoded), or base64 encoded (URL safe, no padding). When no encoding is used, any URL query string is treated as part of the remote URL. For example, given the proxy URL of `http://localhost/x/http://example.com/?id=1`, the remote URL is `http://example.com/?id=1`. When percent-encoding is used, the full URL must be encoded. Any query string on the proxy URL is NOT included as part of the remote URL. Percent-encoded URLs must be absolute URLs; they cannot be relative URLs used with a default base URL. For example, `http://localhost/x/http%3A%2F%2Fexample.com%2F%3Fid%3D1`. When base64 encoding is used, the full URL must be encoded. Any query string on the proxy URL is NOT included as part of the remote URL. Base64 encoded URLs may be relative URLs used with a default base URL. For example, `http://localhost/x/aHR0cDovL2V4YW1wbGUuY29tLz9pZD0x`. ### Examples The following live examples demonstrate setting different options on [this source image][small-things], which measures 1024 by 678 pixels. [small-things]: https://willnorris.com/images/imageproxy/small-things.jpg | Options | Meaning | Image | | ---------------------- | ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 200x | 200px wide, proportional height | 200x | | x0.15 | 15% original height, proportional width | x0.15 | | 100x150 | 100 by 150 pixels, cropping as needed | 100x150 | | 100 | 100px square, cropping as needed | 100 | | 150,fit | scale to fit 150px square, no cropping | 150,fit | | 100,r90 | 100px square, rotated 90 degrees | 100,r90 | | 100,fv,fh | 100px square, flipped horizontal and vertical | 100,fv,fh | | 200x,q60 | 200px wide, proportional height, 60% quality | 200x,q60 | | 200x,png | 200px wide, converted to PNG format | 200x,png | | cx175,cw400,ch300,100x | crop to 400x300px starting at (175,0), scale to 100px wide | cx175,cw400,ch300,100x | The [smart crop feature](https://pkg.go.dev/willnorris.com/go/imageproxy#hdr-Smart_Crop-ParseOptions) can best be seen by comparing crops of [this source image][judah-sheets], with and without smart crop enabled. | Options | Meaning | Image | | ---------- | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 150x300 | 150x300px, standard crop | 200x400,sc | | 150x300,sc | 150x300px, smart crop | 200x400 | [judah-sheets]: https://judahnorris.com/images/judah-sheets.jpg Transformation also works on animated gifs. Here is [this source image][material-animation] resized to 200px square and rotated 270 degrees: [material-animation]: https://willnorris.com/images/imageproxy/material-animations.gif 200,r270 ## Getting Started Install the package using: ```sh go install willnorris.com/go/imageproxy/cmd/imageproxy@latest ``` Once installed, ensure `$GOPATH/bin` is in your `$PATH`, then run the proxy using: ```sh imageproxy ``` This will start the proxy on port 8080, without any caching and with no allowed host list (meaning any remote URL can be proxied). Test this by navigating to and you should see a 500px square coder octocat. ### Cache By default, the imageproxy command does not cache responses, but caching can be enabled using the `-cache` flag. It supports the following values: - `memory` - uses an in-memory LRU cache. By default, this is limited to 100mb. To customize the size of the cache or the max age for cached items, use the format `memory:size:age` where size is measured in mb and age is a duration. For example, `memory:200:4h` will create a 200mb cache that will cache items no longer than 4 hours. - directory on local disk (e.g. `/tmp/imageproxy`) - will cache images on disk - s3 URL (e.g. `s3://region/bucket-name/optional-path-prefix`) - will cache images on Amazon S3. This requires either an IAM role and instance profile with access to your your bucket or `AWS_ACCESS_KEY_ID` and `AWS_SECRET_KEY` environmental variables be set. (Additional methods of loading credentials are documented in the [aws-sdk-go session package](https://docs.aws.amazon.com/sdk-for-go/api/aws/session/)). Additional configuration options ([further documented here][aws-options]) may be specified as URL query string parameters, which are mostly useful when working with s3-compatible services: - "endpoint" - specify an alternate API endpoint - "disableSSL" - set to "1" to disable SSL when calling the API - "s3ForcePathStyle" - set to "1" to force the request to use path-style addressing For example, when working with [minio](https://minio.io), which doesn't use regions, provide a dummy region value and custom endpoint value: ``` s3://fake-region/bucket/folder?endpoint=minio:9000&disableSSL=1&s3ForcePathStyle=1 ``` Similarly, for [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces/), provide a dummy region value and the appropriate endpoint for your space: ``` s3://fake-region/bucket/folder?endpoint=sfo2.digitaloceanspaces.com ``` [aws-options]: https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config - gcs URL (e.g. `gcs://bucket-name/optional-path-prefix`) - will cache images on Google Cloud Storage. Authentication is documented in Google's [Application Default Credentials docs](https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application). - azure URL (e.g. `azure://container-name/`) - will cache images on Azure Storage. This requires `AZURESTORAGE_ACCOUNT_NAME` and `AZURESTORAGE_ACCESS_KEY` environment variables to bet set. - redis URL (e.g. `redis://hostname/`) - will cache images on the specified redis host. The full URL syntax is defined by the [redis URI registration](https://www.iana.org/assignments/uri-schemes/prov/redis). Rather than specify password in the URI, use the `REDIS_PASSWORD` environment variable. For example, to cache files on disk in the `/tmp/imageproxy` directory: ```sh imageproxy -cache /tmp/imageproxy ``` Reload the [codercat URL][], and then inspect the contents of `/tmp/imageproxy`. Within the subdirectories, there should be two files, one for the original full-size codercat image, and one for the resized 500px version. [codercat URL]: http://localhost:8080/500/https://octodex.github.com/images/codercat.jpg Multiple caches can be specified by separating them by spaces or by repeating the `-cache` flag multiple times. The caches will be created in a [tiered fashion][]. Typically this is used to put a smaller and faster in-memory cache in front of a larger but slower on-disk cache. For example, the following will first check an in-memory cache for an image, followed by a gcs bucket: ```sh imageproxy -cache memory -cache gcs://my-bucket/ ``` [tiered fashion]: https://pkg.go.dev/github.com/die-net/lrucache/twotier #### Override Cache Directives By default, imageproxy will respect the caching directives in response headers, including the cache duration and explicit instructions **not** to cache the response, such as `no-store` and `private` cache-control directives. You can force imageproxy to cache responses, even if they explicitly say not to, by passing the `-forceCache` flag. Note that this is generally not recommended. A minimum cache duration can be set using the `-minCacheDuration` flag. This will extend the cache duration if the response header indicates a shorter value. If called without the `-forceCache` flag, this will have no effect on responses with the `no-store` or `private` directives. ```sh imageproxy -cache /tmp/imageproxy -minCacheDuration 5m ``` ### Allowed Referrer List You can limit images to only be accessible for certain hosts in the HTTP referrer header, which can help prevent others from hotlinking to images. It can be enabled by running: ```sh imageproxy -referrers example.com ``` Reload the [codercat URL][], and you should now get an error message. You can specify multiple hosts as a comma separated list, or prefix a host value with `*.` to allow all sub-domains as well. ### Allowed and Denied Hosts List You can limit the remote hosts that the proxy will fetch images from using the `allowHosts` and `denyHosts` flags. This is useful, for example, for locking the proxy down to your own hosts to prevent others from abusing it. Of course if you want to support fetching from any host, leave off these flags. Try it out by running: ```sh imageproxy -allowHosts example.com ``` Reload the [codercat URL][], and you should now get an error message. Alternately, try running: ```sh imageproxy -denyHosts octodex.github.com ``` Reloading the [codercat URL][] will still return an error message. You can specify multiple hosts as a comma separated list to either flag, or prefix a host value with `*.` to allow or deny all sub-domains. You can also specify a netblock in CIDR notation (`127.0.0.0/8`) -- this is useful for blocking reserved ranges like `127.0.0.0/8`, `192.168.0.0/16`, etc. If a host matches both an allowed and denied host, the request will be denied. ### Allowed Content-Type List You can limit what content types can be proxied by using the `contentTypes` flag. By default, this is set to `image/*`, meaning that imageproxy will process any image types. You can specify multiple content types as a comma separated list, and suffix values with `*` to perform a wildcard match. Set the flag to an empty string to proxy all requests, regardless of content type. ### Signed Requests Instead of an allowed host list, you can require that requests be signed. This is useful in preventing abuse when you don't have just a static list of hosts you want to allow. Signatures are generated using HMAC-SHA256 against the remote URL, and url-safe base64 encoding the result: ``` base64urlencode(hmac.New(sha256, ).digest()) ``` The HMAC key is specified using the `signatureKey` flag. If this flag begins with an "@", the remainder of the value is interpreted as a file on disk which contains the HMAC key. Try it out by running: ```sh imageproxy -signatureKey "secretkey" ``` Reload the [codercat URL][], and you should see an error message. Now load a [signed codercat URL][] (which contains the [signature option][]) and verify that it loads properly. [signed codercat URL]: http://localhost:8080/500,sXyMwWKIC5JPCtlYOQ2f4yMBTqpjtUsfI67Sp7huXIYY=/https://octodex.github.com/images/codercat.jpg [signature option]: https://pkg.go.dev/willnorris.com/go/imageproxy#hdr-Signature-ParseOptions Some simple code samples for generating signatures in various languages can be found in [docs/url-signing.md](/docs/url-signing.md). Multiple valid signature keys may be provided to support key rotation by repeating the `signatureKey` flag multiple times, or by providing a space-separated list of keys. To use a key with a literal space character, load the key from a file using the "@" prefix documented above. If both a whiltelist and signatureKey are specified, requests can match either. In other words, requests that match one of the allowed hosts don't necessarily need to be signed, though they can be. To limit how long a URL is valid (particularly useful for signed URLs), you can specify a "valid until" time using the `vu` option with a Unix timestamp. For example, the following signed URL would only be valid until 2020-01-01: ``` http://localhost:8080/vu1577836800,sjNcVf6LxzKEvR6Owgg3zhEMN7xbWxlpf-eyYbRfFK4A=/https://example.com/image ``` ### Default Base URL Typically, remote images to be proxied are specified as absolute URLs. However, if you commonly proxy images from a single source, you can provide a base URL and then specify remote images relative to that base. Try it out by running: ```sh imageproxy -baseURL https://octodex.github.com/ ``` Then load the codercat image, specified as a URL relative to that base: . Note that this is not an effective method to mask the true source of the images being proxied; it is trivial to discover the base URL being used. Even when a base URL is specified, you can always provide the absolute URL of the image to be proxied. ### Scaling beyond original size By default, the imageproxy won't scale images beyond their original size. However, you can use the `scaleUp` command-line flag to allow this to happen: ```sh imageproxy -scaleUp true ``` ### WebP and TIFF support Imageproxy can proxy remote webp images, but they will be served in either jpeg or png format (this is because the golang webp library only supports webp decoding) if any transformation is requested. If no format is specified, imageproxy will use jpeg by default. If no transformation is requested (for example, if you are just using imageproxy as an SSL proxy) then the original webp image will be served as-is without any format conversion. Because so few browsers support tiff images, they will be converted to jpeg by default if any transformation is requested. To force encoding as tiff, pass the "tiff" option. Like webp, tiff images will be served as-is without any format conversion if no transformation is requested. Run `imageproxy -help` for a complete list of flags the command accepts. If you want to use a different caching implementation, it's probably easiest to just make a copy of `cmd/imageproxy/main.go` and customize it to fit your needs... it's a very simple command. ### Environment Variables All configuration flags have equivalent environment variables of the form `IMAGEPROXY_$NAME`. For example, an on-disk cache could be configured by calling ```sh IMAGEPROXY_CACHE="/tmp/imageproxy" imageproxy ``` ## Deploying In most cases, you can follow the normal procedure for building a deploying any go application. For example: - `go build willnorris.com/go/imageproxy/cmd/imageproxy` - copy resulting binary to `/usr/local/bin` - copy [`etc/imageproxy.service`](etc/imageproxy.service) to `/lib/systemd/system` and enable using `systemctl`. Instructions have been contributed below for running on other platforms, but I don't have much experience with them personally. ### Heroku It's easy to vendorize the dependencies with `Godep` and deploy to Heroku. Take a look at [this GitHub repo](https://github.com/oreillymedia/prototype-imageproxy/tree/heroku) (make sure you use the `heroku` branch). ### AWS Elastic Beanstalk [O’Reilly Media](https://github.com/oreillymedia) set up [a repository](https://github.com/oreillymedia/prototype-imageproxy) with everything you need to deploy imageproxy to Elastic Beanstalk. Just follow the instructions in the [README](https://github.com/oreillymedia/prototype-imageproxy/blob/master/Readme.md). ### Docker A docker image is available at [`ghcr.io/willnorris/imageproxy`](https://github.com/willnorris/imageproxy/pkgs/container/imageproxy). You can run it by ```sh docker run -p 8080:8080 ghcr.io/willnorris/imageproxy -addr 0.0.0.0:8080 ``` Or in your Dockerfile: ```Dockerfile ENTRYPOINT ["/app/imageproxy", "-addr 0.0.0.0:8080"] ``` If running imageproxy inside docker with a bind-mounted on-disk cache, make sure the container is running as a user that has write permission to the mounted host directory. See more details in [#198](https://github.com/willnorris/imageproxy/issues/198). Note that all configuration options can be set using [environment variables](#environment-variables), which is often the preferred approach for containers. ### Caddy You can proxy requests to imageproxy in your Caddy config using the `reverse_proxy` directive: ```Caddyfile @imageproxy path /api/imageproxy/* handle @imageproxy { uri replace /api/imageproxy/ / reverse_proxy http://localhost:4593 } ``` You can also run an instance of imageproxy embedded in Caddy using the [caddy module](./caddy/). This requires a custom build of Caddy with the imageproxy module included ([example](https://github.com/willnorris/willnorris.com/blob/main/cmd/caddy/caddy.go)), and configuring it with the `imageproxy` directive in your Caddyfile: ```Caddyfile @imageproxy path /api/imageproxy/* handle @imageproxy { uri replace /api/imageproxy/ / imageproxy { cache /data/imageproxy-cache default_base_url {$IMAGEPROXY_BASEURL} allow_hosts {$IMAGEPROXY_ALLOWHOSTS} signature_key {$IMAGEPROXY_SIGNATUREKEY} } } ``` ### nginx Use the `proxy_pass` directive to send requests to your imageproxy instance. For example, to run imageproxy at the path "/api/imageproxy/", set: ```nginx location /api/imageproxy/ { proxy_pass http://localhost:4593/; } ``` Depending on other directives you may have in your nginx config, you might need to alter the precedence order by setting: ```nginx location ^~ /api/imageproxy/ { proxy_pass http://localhost:4593/; } ``` ## Clients - [Hugo partial](https://github.com/willnorris/willnorris.com/blob/main/layouts/partials/imageproxy-url.html) (I use this with an [`{{}}` shortcode](https://github.com/willnorris/willnorris.com/blob/main/layouts/shortcodes/img.html) like [this example](https://github.com/willnorris/willnorris.com/blob/b7f3451/content/about/index.md?plain=1#L7)) - [Ruby](https://github.com/azolf/imageproxy_ruby) ## License imageproxy is copyright its respective authors. All of my personal work on imageproxy through 2020 (which accounts for the majority of the code) is copyright Google, my employer at the time. It is available under the [Apache 2.0 License](./LICENSE). --- ### Overview (`third_party/envy/README.md`) envy is a copy of https://github.com/jamiealquiza/envy without the cobra support. --- ### Overview (`third_party/httpcache/README.md`) httpcache is a copy of with only the cache control header parsing logic. --- ### Changelog (`docs/changelog.md`) # Changelog This file contains all notable changes to [imageproxy](https://github.com/willnorris/imageproxy). The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] [Unreleased]: https://github.com/willnorris/imageproxy/compare/v0.9.0...HEAD ## [0.10.0] (2020-04-02) [0.10.0]: https://github.com/willnorris/imageproxy/compare/v0.9.0...v0.10.0 ### Added - add support for multiple signature keys to support key rotation ([ef09c1b](https://github.com/willnorris/imageproxy/commit/ef09c1b), [#209](https://github.com/willnorris/imageproxy/pull/209), [maurociancio](https://github.com/maurociancio)) - added option to include referer header in remote requests ([#216](https://github.com/willnorris/imageproxy/issues/216)) - added basic support for recording prometheus metrics ([#121](https://github.com/willnorris/imageproxy/pull/121) [benhaan](https://github.com/benhaan)) ### Fixed - improved content type detection for some hosts, particularly S3 ([ea95ad9](https://github.com/willnorris/imageproxy/commit/ea95ad9), [shahan312](https://github.com/shahan312)) - fix signature verification for some proxied URLs ([3589510](https://github.com/willnorris/imageproxy/commit/3589510), [#212](https://github.com/willnorris/imageproxy/issues/212), ([#215](https://github.com/willnorris/imageproxy/issues/215), thanks to [aaronpk](https://github.com/aaronpk) for helping debug and [fieldistor](https://github.com/fieldistor) for the suggested fix) ## [0.9.0] (2019-06-10) [0.9.0]: https://github.com/willnorris/imageproxy/compare/v0.8.0...v0.9.0 ### Added - allow request signatures to cover options ([#145](https://github.com/willnorris/imageproxy/issues/145)) - add simple imageproxy-sign tool for calculating signatures ([e1558d5](https://github.com/willnorris/imageproxy/commit/e1558d5)) - allow overriding the Logger used by Proxy ([#174](https://github.com/willnorris/imageproxy/pull/174), [hmhealey](https://github.com/hmhealey)) - allow using environment variables for configuration ([50e0d11](https://github.com/willnorris/imageproxy/commit/50e0d11)) - add support for BMP images ([d4ba520](https://github.com/willnorris/imageproxy/commit/d4ba520)) ### Changed - improvements to docker image: run as non-privileged user, use go1.12 compiler, and build imageproxy as a go module. - options are now sorted when converting to string. This is a breaking change for anyone relying on the option order, and will additionally invalidate most cached values, since the option string is part of the cache key. Both the original remote image, as well as any transformations on that image are cached, but only the transformed images will be impacted by this change. This will result in imageproxy having to re-perform the transformations, but should not result in re-fetching the remote image, unless it has already otherwise expired. ### Fixed - properly include Accept header on remote URL requests ([#165](https://github.com/willnorris/imageproxy/issues/165), [6aca1e0](https://github.com/willnorris/imageproxy/commit/6aca1e0)) - detect response content type if content-type header is missing ([cf54b2c](https://github.com/willnorris/imageproxy/commit/cf54b2c)) ### Removed - removed deprecated `whitelist` flag and `Proxy.Whitelist` struct field. Use `allowHosts` and `Proxy.AllowHosts` instead. ## [0.8.0] (2019-03-21) [0.8.0]: https://github.com/willnorris/imageproxy/compare/v0.7.0...v0.8.0 ### Added - added support for restricting proxied URLs [based on Content-Type headers](https://github.com/willnorris/imageproxy#allowed-content-type-list) ([#141](https://github.com/willnorris/imageproxy/pull/141), [ccbrown](https://github.com/ccbrown)) - added ability to [deny requests](https://github.com/willnorris/imageproxy#allowed-and-denied-hosts-list) for certain remote hosts ([#85](https://github.com/willnorris/imageproxy/pull/85), [geriljaSA](https://github.com/geriljaSA)) - added `userAgent` flag to specify a custom user agent when fetching images ([#83](https://github.com/willnorris/imageproxy/pull/83), [huguesalary](https://github.com/huguesalary)) - added support for [s3 compatible](https://github.com/willnorris/imageproxy#cache) storage providers ([#147](https://github.com/willnorris/imageproxy/pull/147), [ruledio](https://github.com/ruledio)) - log URL when image transform fails for easier debugging ([#149](https://github.com/willnorris/imageproxy/pull/149), [daohoangson](https://github.com/daohoangson)) - added support for building imageproxy as a [go module](https://golang.org/wiki/Modules). A future version will remove vendored dependencies, at which point building as a module will be the only supported method of building imageproxy. ### Changed - when a remote URL is denied, return a generic error message that does not specify exactly why it failed ([7e19b5c](https://github.com/willnorris/imageproxy/commit/7e19b5c)) ### Deprecated - `whitelist` flag and `Proxy.Whitelist` struct field renamed to `allowHosts` and `Proxy.AllowHosts`. Old values are still supported, but will be removed in a future release. ### Fixed - fixed tcp_mem resource leak on 304 responses ([#153](https://github.com/willnorris/imageproxy/pull/153), [Micr0mega](https://github.com/Micr0mega)) ## [0.7.0] (2018-02-06) [0.7.0]: https://github.com/willnorris/imageproxy/compare/v0.6.0...v0.7.0 ### Added - added support for arbitrary [rectangular crops](https://godoc.org/willnorris.com/go/imageproxy#hdr-Rectangle_Crop) ([#90](https://github.com/willnorris/imageproxy/pull/90), [maciejtarnowski](https://github.com/maciejtarnowski)) - added support for tiff images ([#109](https://github.com/willnorris/imageproxy/pull/109), [mikecx](https://github.com/mikecx)) - added support for additional [caching backends](https://github.com/willnorris/imageproxy#cache): - Google Cloud Storage ([#106](https://github.com/willnorris/imageproxy/pull/106), [diegomarangoni](https://github.com/diegomarangoni)) - Azure ([#79](https://github.com/willnorris/imageproxy/pull/79), [PaulARoy](https://github.com/PaulARoy)) - Redis ([#49](https://github.com/willnorris/imageproxy/issues/49) [dbfc693](https://github.com/willnorris/imageproxy/commit/dbfc693)) - Tiering multiple caches by repeating the `-cache` flag ([ec5b543](https://github.com/willnorris/imageproxy/commit/ec5b543)) - added support for EXIF orientation tags ([#63](https://github.com/willnorris/imageproxy/issues/63), [67619a6](https://github.com/willnorris/imageproxy/commit/67619a6)) - added [smart crop feature](https://godoc.org/willnorris.com/go/imageproxy#hdr-Smart_Crop) ([#55](https://github.com/willnorris/imageproxy/issues/55), [afbd254](https://github.com/willnorris/imageproxy/commit/afbd254)) ### Changed - rotate values are normalized, such that `r-90` is the same as `r270` ([07c54b4](https://github.com/willnorris/imageproxy/commit/07c54b4)) - now return `200 OK` response for requests to root `/` ([5ee7e28](https://github.com/willnorris/imageproxy/commit/5ee7e28)) - switch to using official AWS Go SDK for s3 cache storage. This is a breaking change for anyone using that cache implementation, since the URL syntax has changed. This adds support for the newer v4 auth method, as well as additional s3 regions. ([0ee5167](https://github.com/willnorris/imageproxy/commit/0ee5167)) - switched to standard go log library. Added `-verbose` flag for more logging in-memory cache backend supports limiting the max cache size ([a57047f](https://github.com/willnorris/imageproxy/commit/a57047f)) - docker image sized reduced by using scratch image and multistage build ([#113](https://github.com/willnorris/imageproxy/pull/113), [matematik7](https://github.com/matematik7)) ### Removed - removed deprecated `cacheDir` and `cacheSize` flags ### Fixed - fixed interpretation of `Last-Modified` and `If-Modified-Since` headers ([#108](https://github.com/willnorris/imageproxy/pull/108), [jamesreggio](https://github.com/jamesreggio)) - preserve original URL encoding ([#115](https://github.com/willnorris/imageproxy/issues/115)) ## [0.6.0] (2017-08-29) [0.6.0]: https://github.com/willnorris/imageproxy/compare/v0.5.1...v0.6.0 ### Added - added health check endpoint ([#54](https://github.com/willnorris/imageproxy/pull/54), [immunda](https://github.com/immunda)) - preserve Link headers from remote image ([#68](https://github.com/willnorris/imageproxy/pull/68), [xavren](https://github.com/xavren)) - added support for per-request timeout ([#75](https://github.com/willnorris/imageproxy/issues/75)) - added support for specifying output image format ([b9cc9df](https://github.com/willnorris/imageproxy/commit/b9cc9df)) - added webp support (decode only) ([3280445](https://github.com/willnorris/imageproxy/commit/3280445)) - added CORS support ([#96](https://github.com/willnorris/imageproxy/pull/96), [romdim](https://github.com/romdim)) ### Fixed - improved error messages for some authorization failures ([27d5378](https://github.com/willnorris/imageproxy/commit/27d5378)) - skip transformation when not needed ([#64](https://github.com/willnorris/imageproxy/issues/64)) - properly handled "cleaned" remote URLs ([a1af9aa](https://github.com/willnorris/imageproxy/commit/a1af9aa), [b61992e](https://github.com/willnorris/imageproxy/commit/b61992e)) ## [0.5.1] (2015-12-07) [0.5.1]: https://github.com/willnorris/imageproxy/compare/v0.5.0...v0.5.1 ### Fixed - fixed bug in gif resizing ([gifresize@104a7cd](https://github.com/willnorris/gifresize/commit/104a7cd)) ## [0.5.0] (2015-12-07) [0.5.0]: https://github.com/willnorris/imageproxy/compare/v0.4.0...v0.5.0 ## Added - added Dockerfile ([#29](https://github.com/willnorris/imageproxy/pull/29), [sevki](https://github.com/sevki)) - allow scaling image beyond its original size with `-scaleUp` flag ([#37](https://github.com/willnorris/imageproxy/pull/37), [runemadsen](https://github.com/runemadsen)) - add ability to restrict HTTP referrer ([9213c93](https://github.com/willnorris/imageproxy/commit/9213c93), [connor4312](https://github.com/connor4312)) - preserve cache-control header from remote image ([#43](https://github.com/willnorris/imageproxy/pull/43), [runemadsen](https://github.com/runemadsen)) - add support for caching images on Amazon S3 ([ec96fcb](https://github.com/willnorris/imageproxy/commit/ec96fcb) [victortrac](https://github.com/victortrac)) ## Changed - change default cache to none, and add `-cache` flag for specifying caches. This deprecates the `-cacheDir` flag. - on-disk cache now stores files in a two-level trie. For example, for a file named "c0ffee", store file as "c0/ff/c0ffee". ## Fixed - skip resizing if requested dimensions larger than original ([#46](https://github.com/willnorris/imageproxy/pull/46), [orian](https://github.com/orian)) ## [0.4.0] (2015-05-21) [0.4.0]: https://github.com/willnorris/imageproxy/compare/v0.3.0...v0.4.0 ### Added - added support for animated gifs ([#23](https://github.com/willnorris/imageproxy/issues/23)) ### Changed - non-200 responses from remote servers are proxied as-is ## [0.3.0] (2015-12-07) [0.3.0]: https://github.com/willnorris/imageproxy/compare/v0.2.3...v0.3.0 ### Added - added support for signing requests using a sha-256 HMAC. ([a9efefc](https://github.com/willnorris/imageproxy/commit/a9efefc)) - more complete logging of requests and whether response is from the cache ([#17](https://github.com/willnorris/imageproxy/issues/17)) - added support for a base URL for remote images. This allows shorter relative URLs to be specified in requests. ([#15](https://github.com/willnorris/imageproxy/issues/15)) ### Fixed - be more precise in copying over all headers from remote image response ([1bf0515](https://github.com/willnorris/imageproxy/commit/1bf0515)) ## [0.2.3] (2015-02-20) [0.2.3]: https://github.com/willnorris/imageproxy/compare/v0.2.2...v0.2.3 ### Added - added quality option ([#13](https://github.com/willnorris/imageproxy/pull/13) [cubabit](https://github.com/cubabit)) ## [0.2.2] (2014-12-08) [0.2.2]: https://github.com/willnorris/imageproxy/compare/v0.2.1...v0.2.2 ### Added - added `cacheSize` flag to command line ### Changed - improved documentation and error messages - negative width or height transformation values interpreted as 0 ## [0.2.1] (2014-08-13) [0.2.1]: https://github.com/willnorris/imageproxy/compare/v0.2.0...v0.2.1 ### Changed - restructured package so that the command line tools is now installed from `willnorris.com/go/imageproxy/cmd/imageproxy` ## [0.2.0] (2014-07-02) [0.2.0]: https://github.com/willnorris/imageproxy/compare/v0.1.0...v0.2.0 ### Added - transformed images are cached in addition to the original image ([#1](https://github.com/willnorris/imageproxy/issues/1)) - support etag and last-modified headers on incoming requests ([#3](https://github.com/willnorris/imageproxy/issues/3)) - support wildcards in list of allowed hosts ### Changed - options can be specified in any order - images cannot be resized larger than their original dimensions ## [0.1.0] (2013-12-26) [0.1.0]: https://github.com/willnorris/imageproxy/compare/5d75e8a...v0.1.0 Initial release. Supported transformation options include: - width and height - different crop modes - rotation (in 90 degree increments) - flip (horizontal or vertical) Images can be cached in-memory or on-disk. --- ### Contributing (`docs/contributing.md`) # Contributing to imageproxy ## Types of contributions Simple bug fixes for existing functionality are always welcome. In many cases, it may be helpful to include a reproducible sample case that demonstrates the bug being fixed. For new functionality, it's general best to open an issue first to discuss it. ## Reporting issues Bugs, feature requests, and development-related questions should be directed to the [GitHub issue tracker](https://github.com/willnorris/imageproxy/issues). If reporting a bug, please try and provide as much context as possible such as what version of imageproxy you're running, what configuration options, specific remote URLs that exhibit issues, and anything else that might be relevant to the bug. For feature requests, please explain what you're trying to do, and how the requested feature would help you do that. Security related bugs can either be reported in the issue tracker, or if they are more sensitive, emailed to . ## Code Style and Tests Go code should follow general best practices, such as using go fmt, go lint, and go vet (this is enforced by our continuous integration setup). Tests should always be included where possible, especially for bug fixes in order to prevent regressions. --- ### Plugin Design (`docs/plugin-design.md`) # Plugin Design Doc **Status:** idea phase, with no immediate timeline for implementation ## Objective Re-architect imageproxy to use a plugin-based system for most features like transformations, security, and caching. This should reduce build times and binary sizes in the common case, and provide a mechanism for users to easily add custom features that would not be added to core for various reasons. ## Background I created imageproxy to [scratch a personal itch](https://wjn.me/b/J_), I needed a simple way to dynamically resize images for my personal website. I published it as an open source projects because that's what I do, and I'm happy to see others finding it useful for their needs as well. But inevitably, with more users came requests for additional features because people have different use cases and requirements. Some of these requests were relatively minor, and I was happy to add them. But one of the more common requests was to support different caching backends. Personally, I still use the on-disk cache, but many people wanted to use redis or a cloud provider like AWS, Azure, or GCP. For a long time I was resistant to adding support for these, mainly out of concern for inflating build times and binary sizes. I did eventually relent, and [#49] tracked adding support for the most common backends. Unfortunately my concerns proved true, and build times are _significantly_ slower (TODO: add concrete numbers) now because of all the additional cloud SDKs that get compiled in. I don't personally care too much about binary size, since I'm not running in a constrained environment, but these build times are really wearing on me. Additionally, there are a number of outstanding pull requests for relatively obscure features that I don't really want to have to support in the main project. And quite honestly, there are a number of obscure features that did get merged in over the years that I kinda wish I could rip back out. [#49]: https://github.com/willnorris/imageproxy/issues/49 ### Plugin support in Go TODO: talk about options like - RPC () - pkg/plugin () - embedded interpreter () - custom binaries (, ) Spoiler: I'm planning on following the Caddy approach and using custom binaries. ## Design I plan to model imageproxy after Caddy, moving all key functionality into separate plugins that register themselves with the server, and which all compile to a single statically-linked binary. The core project will provide a great number of plugins to cover all of the existing functionality. I also expect I'll be much more open to adding plugins for features I may not care as much about personally. Of course, users can also write their own plugins and link them in without needing to contribute them to core if they don't want to. I anticipate providing two or three build configurations in core: - **full** - include all the plugins that are part of core (except where they may conflict) - **minimal** - some set of minimal features that only includes basic caching options, limited transformation options, etc - **my personal config** - I'll also definitely have a build that I use personally on my site. I may decide to just make that the "minimal" build and perhaps call it something different, rather than have a third configuration. Custom configurations beyond what is provided by core can be done by creating a minimal main package that imports the plugins you care about and calling some kind of bootstrap method (similar to [what Caddy now does](https://caddy.community/t/59)). ### Types of plugins (Initially in no particular order, just capturing thoughts. Lots to do here in thinking through the use cases and what kind of plugin API we really need to provide.) See also issues and PRs with [label:plugins][]. [label:plugins]: https://github.com/willnorris/imageproxy/issues?q=label:plugins #### Caching backend This is one of the most common feature requests, and is also one of the worst offender for inflating build times and binary sizes because of the size of the dependencies that are typically required. The minimal imageproxy build would probably only include the in-memory and on-disk caches. Anything that talked to an external store (redis, cloud providers, etc) would be pulled out. #### Transformation engine Today, imageproxy only performs transformations which can be done with pure Go libraries. There have been a number of requests (or at least questions) to use something like [vips] or [imagemagick], which are both C libraries. They provide more options, and (likely) better performance, at the cost of complexity and loss of portability in using cgo. These would likely replace the entire transformation engine in imageproxy, so I don't know how they would interact with other plugins that merely extend the main engine (they probably wouldn't be able to interact at all). [vips]: https://github.com/DAddYE/vips [imagemagick]: https://github.com/gographics/imagick #### Transformation options Today, imageproxy performs minimal transformations, mostly around resizing, cropping, and rotation. It doesn't support any kind of filters, brightness or contrast adjustment, etc. There are go libraries for them, they're just outside the scope of what I originally intended imageproxy for. But I'd be happy to have plugins that do that kind of thing. These plugins would need to be able to hook into the option parsing engine so that they could register their URL options. #### Image format support There have been a number of requests for image format support that require cgo libraries: - **webp encoding** - needs cgo [#114](https://github.com/willnorris/imageproxy/issues/114) - **progressive jpegs** - probably needs cgo? [#77](https://github.com/willnorris/imageproxy/issues/77) - **gif to mp4** - maybe doable in pure go, but probably belongs in a plugin [#136](https://github.com/willnorris/imageproxy/issues/136) - **HEIF** - formate used by newer iPhones ([HEIF](https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format)) #### Option parsing Today, options are specified as the first component in the URL path, but [#66] proposes optionally moving that to a query parameter (for a good reason, actually). Maybe putting that in core is okay? Maybe it belongs in a plugin, in which case we'd need to expose an API for replacing the option parsing code entirely. [#66]: https://github.com/willnorris/imageproxy/pull/66 #### Security options Some people want to add a host blacklist [#85], refusal to process non-image files [#53] [#119]. I don't think there is an issue for it, but an early fork of the project added request signing that was compatible with nginx's [secure link module](https://nginx.org/en/docs/http/ngx_http_secure_link_module.html). [#85]: https://github.com/willnorris/imageproxy/pull/85 [#53]: https://github.com/willnorris/imageproxy/issues/53 [#119]: https://github.com/willnorris/imageproxy/pull/119 ### Registering Plugins Plugins are loaded simply by importing their package. They should have an `init` func that calls `imageproxy.RegisterPlugin`: ```go type Plugin struct { } func RegisterPlugin(name string, plugin Plugin) ``` Plugins hook into various extension points of imageproxy by implementing appropriate interfaces. A single plugin can hook into multiple parts of imageproxy by implementing multiple interfaces. For example, two possible interfaces for security related plugins: ```go // A RequestAuthorizer determines if a request is authorized to be processed. // Requests are processed before the remote resource is retrieved. type RequestAuthorizer interface { // Authorize returns an error if the request should not // be processed further (for example, it doesn't have a // valid signature, is not for a whitelisted host, etc). AuthorizeRequest(req *http.Request) error } // A ResponseAuthorizer determines if a response from a remote server // is authorized to be returned. type ResponseAuthorizer interface { // AuthorizeResponse returns an error if a response should not be // returned to a client (for example, it is not for an image // resource, etc). AuthorizeResponse(res http.Response) error } ``` A hypothetical interface for plugins that transform images: ```go // An ImageTransformer transforms an image. type ImageTransformer interface { // TransformImage based on the provided options and return the result. TransformImage(m image.Image, opt Options) image.Image } ``` Plugins are additionally responsible for registering any additional command line flags they wish to expose to the user, as well as storing any global state that would previously have been stored on the Proxy struct. --- ### Url Signing (`docs/url-signing.md`) # How to generate signed requests Signing requests allows an imageproxy instance to proxy images from arbitrary remote hosts, but without opening the service up for potential abuse. When appropriately configured, the imageproxy instance will only serve requests that are for allowed hosts, or which have a valid signature. Signatures can be calculated in two ways: 1. they can be calculated solely on the remote image URL, in which case any transformations of the image can be requested without changes to the signature value. This used to be the only way to sign requests, but is no longer recommended since it still leaves the imageproxy instance open to potential abuse. 2. they can be calculated based on the combination of the remote image URL and the requested transformation options. In both cases, the signature is calculated using HMAC-SHA256 and a secret key which is provided to imageproxy on startup. The message to be signed is the remote URL, with the transformation options optionally set as the URL fragment, [as documented below](#signing-options). The signature is url-safe base64 encoded, and [provided as an option][s-option] in the imageproxy request. imageproxy will accept signatures for URLs with or without options transparently. It's up to the publisher of the signed URLs to decide which method they use to generate the URL. [s-option]: https://pkg.go.dev/willnorris.com/go/imageproxy#hdr-Signature-ParseOptions ## Signing options Transformation options for a proxied URL are [specified as a comma separated string][ParseOptions] of individual options, which can be supplied in any order. When calculating a signature, options should be put in their canonical form, sorted in lexigraphical order (omitting the signature option itself), and appended to the remote URL as the URL fragment. Currently, only [size option][] has a canonical form, which is `{width}x{height}` with the number `0` used when no value is specified. For example, a request that does not request any size option would still have a canonical size value of `0x0`, indicating that no size transformation is being performed. If only a height of 500px is requested, the canonical form would be `0x500`. For example, requesting the remote URL of `http://example.com/image.jpg`, resized to 100 pixels square, rotated 90 degrees, and converted to 75% quality might produce an imageproxy URL similar to: http://localhost:8080/100,r90,q75/http://example.com/image.jpg When calculating a signature for this request including transformation options, the signed value would be: http://example.com/image.jpg#100x100,q75,r90 The `100` size option was put in its canonical form of `100x100`, and the options are sorted, moving `q75` before `r90`. [ParseOptions]: https://pkg.go.dev/willnorris.com/go/imageproxy#ParseOptions [size option]: https://pkg.go.dev/willnorris.com/go/imageproxy#hdr-Size_and_Cropping-ParseOptions ## Signed options example Here is an example with signed options through each step. Using the github codercat, our image url is `https://octodex.github.com/images/codercat.jpg` and our options are `400x400` and `q40`. The signature key is `secretkey` The value that goes into the Digest is `https://octodex.github.com/images/codercat.jpg#400x400,q40` and our resulting signed key is `0sR2kjyfiF1RQRj4Jm2fFa3_6SDFqdAaDEmy1oD2U-4=` The final url would be `http://localhost:8080/400x400,q40,s0sR2kjyfiF1RQRj4Jm2fFa3_6SDFqdAaDEmy1oD2U-4=/https://octodex.github.com/images/codercat.jpg` ## Language Examples Here are examples of calculating signatures in a variety of languages. These demonstrate the HMAC-SHA256 bits, but not the option canonicalization. In each example, the remote URL `https://octodex.github.com/images/codercat.jpg` is signed using a signature key of `secretkey`. See also the [imageproxy-sign tool](/cmd/imageproxy-sign). ### Go main.go: ```go package main import ( "os" "fmt" "crypto/hmac" "crypto/sha256" "encoding/base64" ) func main() { key, url := os.Args[1], os.Args[2] mac := hmac.New(sha256.New, []byte(key)) mac.Write([]byte(url)) result := mac.Sum(nil) fmt.Println(base64.URLEncoding.EncodeToString(result)) } ``` ```shell $ go run sign.go "secretkey" "https://octodex.github.com/images/codercat.jpg" cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### OpenSSL ```shell $ echo -n "https://octodex.github.com/images/codercat.jpg" | openssl dgst -sha256 -hmac "secretkey" -binary|base64| tr '/+' '_-' cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### Java ```java import java.util.Base64; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; class SignUrl { public static String sign(String key, String url) throws Exception { Mac sha256_HMAC = Mac.getInstance("HmacSHA256"); SecretKeySpec secret_key = new SecretKeySpec(key.getBytes(), "HmacSHA256"); sha256_HMAC.init(secret_key); return Base64.getUrlEncoder().encodeToString(sha256_HMAC.doFinal(url.getBytes())); } public static void main(String [] args) throws Exception { System.out.println(sign(args[0], args[1])); } } ``` ```shell $ javac SignUrl.java && java SignUrl "secretkey" "https://octodex.github.com/images/codercat.jpg" cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### Ruby ```ruby require 'openssl' require 'base64' key = ARGV[0] url = ARGV[1] puts Base64.urlsafe_encode64(OpenSSL::HMAC.digest(OpenSSL::Digest.new('sha256'), key, url)).strip() ``` ```shell % ruby sign.rb "secretkey" "https://octodex.github.com/images/codercat.jpg" cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### Python ```python import base64 import hashlib import hmac import sys key = sys.argv[1] url = sys.argv[2] print base64.urlsafe_b64encode(hmac.new(key, msg=url, digestmod=hashlib.sha256).digest()) ``` ```shell $ python sign.py "secretkey" "https://octodex.github.com/images/codercat.jpg" cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### JavaScript ```javascript const crypto = require("crypto"); const URLSafeBase64 = require("urlsafe-base64"); let key = process.argv[2]; let url = process.argv[3]; console.log( URLSafeBase64.encode(crypto.createHmac("sha256", key).update(url).digest()), ); ``` ```shell $ node sign.js "secretkey" "https://octodex.github.com/images/codercat.jpg" cw34eyalj8YvpLpETxSIxv2k8QkLel2UAR5Cku2FzGM= ``` ### PHP ```php