go-elasticsearch Documentation

The official Go client for Elasticsearch

6,043 stars Go Go 1 file · ~5,876 tokens #client#elasticsearch#go#golang
RAW Doc

File: README.md

go-elasticsearch

The official Go client for Elasticsearch.

Download the latest version of Elasticsearch
or
sign-up
for a free trial of Elastic Cloud.







Documentation

Find the full reference documentation at elastic.co:

The Go package reference lives on pkg.go.dev.

Compatibility

Go

Starting from version 8.12.0, this library follows the Go language's release policy. Each major Go release is supported until there are two newer major releases. For example, Go 1.5 was supported until the Go 1.7 release, and Go 1.6 was supported until the Go 1.8 release.

Elasticsearch

Language clients are forward compatible; meaning that clients support communicating with greater or equal minor versions of Elasticsearch.
Elasticsearch language clients are only backward compatible with default distributions and without guarantees made.

When using Go modules, include the version in the import path, and specify either an explicit version or a branch:

text
require github.com/elastic/go-elasticsearch/v9 v9.x.x
require github.com/elastic/go-elasticsearch/v8 v8.x.x

It's possible to use multiple versions of the client in a single project:

text
// go.mod
github.com/elastic/go-elasticsearch/v8 v8.18.0
github.com/elastic/go-elasticsearch/v9 v9.0.0

// main.go
import (
  elasticsearch8 "github.com/elastic/go-elasticsearch/v8"
  elasticsearch9 "github.com/elastic/go-elasticsearch/v9"
)
// ...
es8, _ := elasticsearch8.New()
es9, _ := elasticsearch9.New()

The main branch of the client is compatible with the current master branch of Elasticsearch.

Installation

Refer to the Installation
page of the documentation.

Connecting

Refer to the Connecting
page of the documentation.

Operations

Helpers

The esutil package provides convenience helpers for working with the client. At the moment, it provides the esutil.JSONReader() and the esutil.BulkIndexer helpers.

Examples

The _examples folder contains a number of recipes and comprehensive examples to get you started with the client,
including configuration and customization of the client, using a custom certificate authority (CA) for security (TLS),
mocking the transport for unit tests, embedding the client in a custom type, building queries, performing requests individually and in bulk, and parsing the responses.

License

This software is licensed under the Apache 2 license. See NOTICE.

File: _benchmarks/README.md

Benchmarks

This folder contains code for running the common Elasticsearch client benchmarks.

The benchmarks package contains the general configuration for the benchmarks. The runner package provides the code for running and measuring the client actions, storing the results in an Elasticsearch cluster. The individual client interactions are defined by the actions package.

The code for setting up the benchmarks infrastructure is available at
https://github.com/elastic/elasticsearch-clients-benchmarks. It uses Terraform to create, setup and configure the "target" and "runner" system.

File: _examples/cloudfunction/README.md

Example: clusterstatus

This example demonstrates how to use the Elasticsearch Go client with
Google Cloud Functions.

Deploy the function with the gcloud command:

bash
go mod vendor
gcloud functions deploy clusterstatus \
    --entry-point Health \
    --runtime go111 \
    --trigger-http \
    --memory 128MB \
    --set-env-vars ELASTICSEARCH_URL=https://...cloud.es.io:9243

Invoke your function over HTTP:

bash
curl https://...cloudfunctions.net/clusterstatus

See the function.go file for implementation and the function_test.go file for the example of implementing a mock transport for testing.

File: _examples/bulk/benchmarks/README.md

Bulk Indexer Benchmarks

The benchmarks.go file executes end-to-end benchmarks for esutil.NewBulkIndexer. It allows to
configure indexer parameters, index settings, number of runs. See go run benchmarks.go --help for an overview of
configuration options:

text
go run benchmarks.go --help
  -count int
        Number of documents to generate (default 100000)
  -dataset string
        Dataset to use for indexing (default "small")
  -debug
        Enable logging output
  -easyjson
        Use mailru/easyjson for JSON decoding
  -fasthttp
        Use valyala/fasthttp for HTTP transport
  -flush value
        Flush threshold in bytes (default 3MB)
  -index string
        Index name (default "test-bulk-benchmarks")
  -mockserver
        Measure added, not flushed items
  -replicas int
        Number of index replicas (default 0)
  -runs int
        Number of runs (default 10)
  -shards int
        Number of index shards (default 3)
  -wait duration
        Wait duration between runs (default 1s)
  -warmup int
        Number of warmup runs (default 3)
  -workers int
        Number of indexer workers (default 4)

Before running the benchmarks, install easyjson and generate the auxiliary files:

bash
go mod download
go get -u github.com/mailru/easyjson/...
grep '~/go/bin' ~/.profile || echo 'export PATH=$PATH:~/go/bin' >> ~/.profile && source ~/.profile
go generate -v ./model

Small Document

The small dataset uses a small document (126B).

text
ELASTICSEARCH_URL=http://server:9200 go run benchmarks.go --dataset=small --count=1_000_000 --flush=2MB --shards=5 --replicas=0 --fasthttp=true --easyjson=true
small: run [10x] warmup [3x] shards [5] replicas [0] workers [8] flush [2.0 MB] wait [1s] fasthttp easyjson
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
   1) add=1M  flush=1M  fail=0  reqs=52 dur=3.58s   279,173 docs/sec
   2) add=1M  flush=1M  fail=0  reqs=52 dur=3.52s   284,090 docs/sec
   3) add=1M  flush=1M  fail=0  reqs=52 dur=3.45s   289,351 docs/sec
   4) add=1M  flush=1M  fail=0  reqs=52 dur=3.49s   286,123 docs/sec
   5) add=1M  flush=1M  fail=0  reqs=52 dur=3.47s   287,852 docs/sec
   6) add=1M  flush=1M  fail=0  reqs=52 dur=3.47s   288,184 docs/sec
   7) add=1M  flush=1M  fail=0  reqs=52 dur=3.54s   282,246 docs/sec
   8) add=1M  flush=1M  fail=0  reqs=52 dur=3.47s   288,101 docs/sec
   9) add=1M  flush=1M  fail=0  reqs=52 dur=3.54s   282,485 docs/sec
  10) add=1M  flush=1M  fail=0  reqs=52 dur=3.46s   288,350 docs/sec
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
docs/sec: min [279,173] max [289,351] mean [286,987]

HTTP Log Event

The httplog dataset uses a bigger document (2.5K), corresponding to a log event gathered
by Filebeat from Nginx.

text
ELASTICSEARCH_URL=http://server:9200 go run benchmarks.go --dataset=httplog --count=1_000_000 --flush=3MB --shards=5 --replicas=0 --fasthttp=true --easyjson=true
httplog: run [10x] warmup [3x] shards [5] replicas [0] workers [8] flush [3.0 MB] wait [1s] fasthttp easyjson
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
   1) add=1M   flush=1M fail=0   reqs=649 dur=19.93s  50,165 docs/sec
   2) add=1M   flush=1M fail=0   reqs=649 dur=18.84s  53,072 docs/sec
   3) add=1M   flush=1M fail=0   reqs=649 dur=19.13s  52,249 docs/sec
   4) add=1M   flush=1M fail=0   reqs=649 dur=19.26s  51,912 docs/sec
   5) add=1M   flush=1M fail=0   reqs=649 dur=18.98s  52,662 docs/sec
   6) add=1M   flush=1M fail=0   reqs=649 dur=19.21s  52,056 docs/sec
   7) add=1M   flush=1M fail=0   reqs=649 dur=18.91s  52,865 docs/sec
   8) add=1M   flush=1M fail=0   reqs=649 dur=19.25s  51,934 docs/sec
   9) add=1M   flush=1M fail=0   reqs=649 dur=19.44s  51,440 docs/sec
  10) add=1M   flush=1M fail=0   reqs=649 dur=19.24s  51,966 docs/sec
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
docs/sec: min [50,165] max [53,072] mean [52,011]

Mock Server

The --mockserver flag allows to run the benchmark against a "mock server", in this case Nginx, to understand a
theoretical performance of the client, without the overhead of a real Elasticsearch cluster.

text
ELASTICSEARCH_URL=http://server:8000 go run benchmarks.go --dataset=small --count=1_000_000 --flush=2MB --warmup=0 --mockserver
small: run [10x] warmup [0x] shards [3] replicas [0] workers [8] flush [2.0 MB] wait [1s]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
   1) add=1M   flush=0  fail=0   reqs=56  dur=810ms   1,222,493 docs/sec
   2) add=1M   flush=0  fail=0   reqs=56  dur=810ms   1,230,012 docs/sec
   3) add=1M   flush=0  fail=0   reqs=56  dur=790ms   1,251,564 docs/sec
   4) add=1M   flush=0  fail=0   reqs=56  dur=840ms   1,187,648 docs/sec
   5) add=1M   flush=0  fail=0   reqs=56  dur=800ms   1,237,623 docs/sec
   6) add=1M   flush=0  fail=0   reqs=56  dur=800ms   1,237,623 docs/sec
   7) add=1M   flush=0  fail=0   reqs=56  dur=800ms   1,240,694 docs/sec
   8) add=1M   flush=0  fail=0   reqs=56  dur=820ms   1,216,545 docs/sec
   9) add=1M   flush=0  fail=0   reqs=56  dur=790ms   1,253,132 docs/sec
  10) add=1M   flush=0  fail=0   reqs=56  dur=810ms   1,223,990 docs/sec
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
docs/sec: min [1,187,648] max [1,253,132] mean [1,233,818]

Environment

Please note that these results are only illustrative, and the real performance depends on many factors:
the size and structure of your data, the index settings and mappings, the cluster setup or the hardware specification.

The benchmarks have been run in the following environment:

  • OS: Ubuntu 18.04.4 LTS (5.0.0-1031-gcp)
  • Client: A n2-standard-8 GCP instance (8
    vCPUs/32GB RAM)
  • Server: A n2-standard-16 GCP instance (16
    vCPUs/64GB RAM)
  • Disk: A local SSD formatted as ext4 on NVMe interface for
    Elasticsearch data
  • A single-node Elasticsearch cluster, 7.6.0, default distribution,
    installed from a TAR, with 4GB locked for heap
  • Nginx 1.17.8 with nginx.conf

File: _examples/bulk/kafka/README.md

Example: Bulk indexing from a Kafka topic

This example demonstrates using the BulkIndexer component to ingest data consumed from a Kafka topic.

The provided docker-compose.yml file launches a realistic environment with Zookeeper, Kafka, Confluent Control Center,
Elasticsearch and Kibana, and allows to inspect data flows, indexer metrics, and see the ingested data in a dashboard.

First, launch the environment and wait until it's ready:

text
make setup

Then, launch the Kafka producers and consumers and the Elasticsearch indexer:

text
make run

Open the Kibana dashboard to see
the results and Confluent Control Center to inspect the Kafka cluster and
see details about the topic and performance of consumers.

See the producer/producer.go file for the Kafka
producer, consumer/consumer.go for the Kafka consumer, and the kafka.go file for
the main workflow. The default configuration will launch one producer, four consumers, one indexer, and will send 1,000
messages per second; see go run kafka.go --help for changing the defaults.

File: _examples/bulk/README.md

Example: Bulk Indexing

`default.go`

The default.go example demonstrates how to properly operate the Elasticsearch's
Bulk API.

The example intentionally doesn't use any abstractions or helper functions, to demonstrate the low-level mechanics of
working with the Bulk API:

  • iterating over a slice of data and preparing the meta/data pairs,
  • filling a buffer with the payload until the configured threshold for a single batch is reached,
  • sending a batch to Elasticsearch,
  • checking for a request failure or a failed response,
  • checking for individual errors in the response,
  • updating a counter of indexed and failed documents,
  • printing a report.
bash
go run default.go -count=100000 -batch=25000

# Bulk: documents [100,000] batch size [25,000]
# ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
# → Generated 100,000 articles
# → Sending batch [1/4] [2/4] [3/4] [4/4]
# ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
# Successfully indexed [100,000] documents in 3.423s (29,214 docs/sec)

`indexer.go`

The indexer.go example demonstrates how to use the esutil.BulkIndexer
helper for efficient indexing in parallel.

bash
go run indexer.go -count=100000 -flush=1000000

# BulkIndexer: documents [100,000] workers [8] flush [1.0 MB]
# ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
# → Generated 100,000 articles
# ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
# Successfully indexed [100,000] documents in 1.909s (52,383 docs/sec)

The helper allows you to Add() bulk indexer items, and flushes each batch based on the configured threshold.

golang
client, err := elasticsearch.New()
if err != nil {
	log.Fatalf("Error creating the client: %s", err)
}
defer client.Close(context.Background())

indexer, err := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{Client: client})
if err != nil {
	log.Fatalf("Error creating the indexer: %s", err)
}
defer indexer.Close(context.Background())

if err := indexer.Add(
	context.Background(),
	esutil.BulkIndexerItem{
		Action: "index",
		Body:   strings.NewReader(`{"title":"Test"}`),
	},
); err != nil {
	log.Fatalf("Unexpected error: %s", err)
}

`typed.go`

The typed.go example demonstrates how to use the typed Bulk API to build a bulk request by appending operations (for example IndexOp) and then executing it with Do().

bash
go run -tags bulk_typed typed.go -count=100000 -batch=1000

# Typed Bulk: documents [100,000] batch size [1,000]

`base64.go`

The base64.go example demonstrates indexing dense vectors using base64 encoding
with the typed Bulk API and a public Elasticsearch Rally track dataset.

bash
go run base64.go -count=20000 -batch=500

# Bulk Base64: documents [20,000] batch size [500]
# ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
# → Loaded 20,000 documents
# ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
# Successfully indexed [20,000] documents in 3.102s (6,446 docs/sec)

Please refer to the benchmarks folder for performance tests with different types of payload.

See the kafka folder for an end-to-end example of using the bulk helper for indexing data from a Kafka topic.

File: _examples/encoding/README.md

Example: encoding

Examples in this folder demonstrate how to use helper methods and third-party packages for JSON decoding and encoding.

`tidwall/gjson`

The github.com/tidwall/gjson package allows an
easy access to JSON properties, without parsing the payload into a data structure,
and is therefore convenient when accessing only selected parts of the response.

golang
var json = `{"foo":{"bar":"BAZ"}}`
fmt.Println(gjson.Get(json, "foo.bar"))
// => BAZ

The gjson.go example displays data from the
Cluster Stats API.

bash
go run gjson.go

`mailru/easyjson`

The github.com/mailru/easyjson package uses code generation to
provide fast encoding and decoding of struct types.

The easyjson.go example uses custom types for representing simpleArticle documents
and index and search responses, including error responses.

bash
make clean setup
go run easyjson.go

`esutil.JSONReader()`

The esutil.JSONReader() helper method takes a struct, a map,
or any other serializable object, and converts it to JSON wrapped in a reader, for convenient
passing to the WithBody() methods:

golang
type Document struct{ Title string }
doc := Document{Title: "Test"}
es.Search(es.Search.WithBody(esutil.NewJSONReader(&doc)))

Benchmarks

Both mailru/easyjson and tidwall/gjson provide significant performance gains in different scenarios.

You can run the included benchmarks by executing the make bench command; example output below.

text
BenchmarkEncode/Article_-_json            500000        2969 ns/op       856 B/op         8 allocs/op
BenchmarkEncode/Article_-_JSONReader      500000        2957 ns/op       920 B/op        10 allocs/op
BenchmarkEncode/Article_-_easyjson       1000000        1089 ns/op       736 B/op         5 allocs/op
BenchmarkEncode/map_-_json               1000000        2074 ns/op       720 B/op        18 allocs/op
BenchmarkEncode/map_-_JSONReader         1000000        2168 ns/op       784 B/op        20 allocs/op

BenchmarkDecode/Search_-_json              50000       38922 ns/op      8456 B/op        60 allocs/op
BenchmarkDecode/Search_-_easyjson         100000       14344 ns/op      7992 B/op        52 allocs/op
BenchmarkDecode/Cluster_-_json_-_map       20000       85742 ns/op     31972 B/op       391 allocs/op
BenchmarkDecode/Cluster_-_json_-_stc       30000       50852 ns/op     15048 B/op        17 allocs/op
BenchmarkDecode/Cluster_-_gjson          1000000        2284 ns/op       264 B/op         3 allocs/op

File: _examples/extension/README.md

Example: Extending the Client

This example demonstrates how to extend the client, in order to call custom APIs, for example added by a plugin.

The main.go example defines a custom type, which embeds the regular Elasticsearch client, and adds a Custom namespace with an Example() method.

To run the example:

bash
go run main.go

#    GET http://localhost:9209/_cat/health 200 OK 25ms
#        « 1555252476 14:34:36 go-elasticsearch green 1 1 0 0 0 0 0 0 - 100.0%
#
#    GET http://localhost:9209/_cat/example 200 OK 0s
#        « Hello from Cat Example action

File: _examples/fasthttp/README.md

Example: fasthttp

This example demonstrates how to replace the net/http transport in the client with an
HTTP client from the github.com/valyala/fasthttp package.

The fasthttp.Transport type is a drop-in replacement for elastictransport.Config.Transport, converting
request and response object between net/http and fasthttp.

To run the example:

text
go run cmd/main.go
# 1000 requests in 1.07 (933.4 req/s) | min: 749.961µs / max: 10.003318ms / mean: 1.013992ms

To run the benchmarks:

text
make bench

BenchmarkHTTPClient/Info()-4              1591      7139770 ns/op     16725 B/op       132 allocs/op
BenchmarkFastHTTPClient/Info()-4         10000      1315049 ns/op      2255 B/op        24 allocs/op

File: _examples/instrumentation/README.md

Example: Instrumentation

This example demonstrates how to instrument the Elasticsearch client.

expvar

The expvar.go example publishes client metrics via the expvar package for consumption by any compatible monitoring system.

OpenTelemetry (built-in)

The otel.go example uses the client's built-in OpenTelemetry integration via elasticsearch.NewOpenTelemetryInstrumentation(provider, captureSearchBody), passed to the constructor with elasticsearch.WithInstrumentation(...). It produces spans that follow the Elasticsearch OpenTelemetry semantic conventions for every API call, with no per-call wiring required.

bash
go run otel.go

Prefer this built-in hook for standard tracing setups. For a manual, interceptor-based alternative (useful when you need to extend the behaviour, add logging, or combine tracing with custom metrics) see ../interceptor/cmd/custom_observability.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#client Browse client catalog → #elasticsearch Browse elasticsearch catalog → #go Browse go catalog → #golang Browse golang catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →