1. Project Overview & Quickstart (buger/jsonparser)
The fastests JSON parser for Go
It does not require you to know the structure of the payload (eg. create structs), and allows accessing fields by providing the path to them. It is up to 6.5x faster than standard encoding/json package (depending on payload size and usage), allocates no memory. See benchmarks below.
Project recently celebrated 10 years!
π Formally Verified β the first Go library proven to L3 assurance by [ReqProof](https://reqproof.com)
jsonparser is the reference case study for ReqProof β a git-native requirements-engineering and formal-verification platform. Every public API is traced to a formal requirement, every requirement is tested with 100% Modified Condition/Decision Coverage (MC/DC), and the entire parser is fuzzed by a custom structure-aware JSON fuzzer (github.com/probelabs/json-fuzz) that generates grammar-valid mutations at 250,000 inputs/second.
| Metric | Value |
|---|---|
| Requirements traced | 118 (7 stakeholder + 111 system) |
| Proof audit | 0 errors, 0 warnings (L3 strict) |
| Code-level MC/DC | 100% decisions, 100% conditions |
| Requirement-side MC/DC | 377/377 witness rows covered |
| Fuzz executions | 16M+ (structure-aware + path-mutation + encoding/json differential) |
| Bugs found & fixed by the proof review | 7 (4 panics, 2 data-corruption, 1 encoding bug) |
The proof review caught bugs that years of community use, OSS-Fuzz, and standard fuzzing had missed β including a panic class across 8 unchecked-dereference sites, a silent data-loss bug in Set, and a malformed-output bug in Delete. Read the full root-cause analysis β
Rationale
Originally I made this for a project that relies on a lot of 3rd party APIs that can be unpredictable and complex.
I love simplicity and prefer to avoid external dependecies. encoding/json requires you to know exactly your data structures, or if you prefer to use map[string]interface{} instead, it will be very slow and hard to manage.
I investigated what's on the market and found that most libraries are just wrappers around encoding/json, there is few options with own parsers (ffjson, easyjson), but they still requires you to create data structures.
Goal of this project is to push JSON parser to the performance limits and not sacrifice with compliance and developer user experience.
Example
For the given JSON our goal is to extract the user's full name, number of github followers and avatar.
/* Detailed source-code truncated for AI context efficiency. */Lenient Parsing
The package-level functions remain strict RFC 8259 parsers. For inputs that use
single-quoted strings or non-standard escapes, use a Config explicitly:
data := []byte(`{'name':'Ada','role':'engineer'}`)
name, err := jsonparser.Lenient.GetString(data, "name")
// name == "Ada"Lenient enables both compatibility options. You can also enable only the
extension your input requires:
config := jsonparser.Config{AllowUnknownEscapes: true}
data := []byte("{\"path\":\"docs\\`draft\\x\"}")
path, err := config.GetString(data, "path")
// path == "docs`draftx"The config-aware Get, GetString, Set, Delete, ArrayEach, andObjectEach methods share the same signatures and behavior as their
package-level counterparts apart from the enabled parsing extensions.jsonparser.DefaultConfig is strict; jsonparser.Lenient enablesAllowSingleQuotes and AllowUnknownEscapes.
Streaming
ReaderParser provides the same path-based lookup model for JSON read from anio.Reader, so a large document does not need to be loaded into a single byte
slice:
file, err := os.Open("large.json")
if err != nil {
log.Fatal(err)
}
defer file.Close()
parser := jsonparser.NewReaderParser(file)
name, err := parser.GetString("person", "name")To process a root array incrementally, use ArrayEach. Each callback value is
valid for the duration of the callback; copy it if it must be retained:
parser := jsonparser.NewReaderParser(file)
err := parser.ArrayEach(func(value []byte, valueType jsonparser.ValueType, err error) {
if err != nil {
return
}
process(value, valueType)
})The parser reads in 64 KiB chunks and discards completed prefixes. Its default
sliding-window target is 64 MiB; customize it withjsonparser.Config{MaxBufferSize: size}. A single returned value or array
element may exceed that target because its complete bytes are supplied to the
caller. Create a new ReaderParser for each lookup or array traversal.ReaderParser also honors AllowSingleQuotes and AllowUnknownEscapes.
Reference
Library API is really simple. You just need the Get method to perform any operation. The rest is just helpers around it.
You also can view API at godoc.org
**`Get`**
func Get(data []byte, keys ...string) (value []byte, dataType jsonparser.ValueType, offset int, err error)Receives data structure, and key path to extract value from.
Returns:
value- Pointer to original data structure containing key value, or just empty slice if nothing found or errordataType- Can be:NotExist,String,Number,Object,Array,BooleanorNulloffset- Offset from provided data structure where key value ends. Used mostly internally, for example forEachArrayhelper.err- If the key is not found or any other parsing issue, it should return error. If key not found it also setsdataTypetoNotExist
Accepts multiple keys to specify path to JSON value (in case of quering nested structures).
If no keys are provided it will try to extract the closest JSON value (simple ones or object/array), useful for reading streams or arrays, see EachArray implementation.
Note that keys can be an array indexes: jsonparser.GetInt("person", "avatars", "[0]", "url"), pretty cool, yeah?
**`GetString`**
func GetString(data []byte, keys ...string) (val string, err error)Returns strings properly handing escaped and unicode characters. Note that this will cause additional memory allocations.
**`GetUnsafeString`**
If you need string in your app, and ready to sacrifice with support of escaped symbols in favor of speed. It returns string mapped to existing byte slice memory, without any allocations:
s, _, := jsonparser.GetUnsafeString(data, "person", "name", "title")
switch s {
case 'CEO':
...
case 'Engineer'
...
...
}Note that unsafe here means that your string will exist until GC will free underlying byte slice, for most of cases it means that you can use this string only in current context, and should not pass it anywhere externally: through channels or any other way.
**`GetBoolean`**, **`GetInt`** and **`GetFloat`**
func GetBoolean(data []byte, keys ...string) (val bool, err error)
func GetFloat(data []byte, keys ...string) (val float64, err error)
func GetInt(data []byte, keys ...string) (val int64, err error)If you know the key type, you can use the helpers above.
If key data type do not match, it will return error.
**`EachArray`**
func EachArray(data []byte, cb func(value []byte, dataType jsonparser.ValueType, offset int, err error), keys ...string)Needed for iterating arrays, accepts a callback function with the same return arguments as Get.ArrayEach remains available as a backward-compatible alias.
The error-returning and wildcard variants follow the same naming convention:
use EachArrayErr and EachArrayWildcard; ArrayEachErr andArrayEachWildcard remain available for backward compatibility.
**`EachObject`**
func EachObject(data []byte, callback func(key []byte, value []byte, dataType ValueType, offset int) error, keys ...string) (err error)Needed for iterating object, accepts a callback function. Example:
var handler func([]byte, []byte, jsonparser.ValueType, int) error
handler = func(key []byte, value []byte, dataType jsonparser.ValueType, offset int) error {
//do stuff here
}
jsonparser.EachObject(myJson, handler)ObjectEach remains available as a backward-compatible alias.
**`EachKey`**
func EachKey(data []byte, cb func(idx int, value []byte, dataType jsonparser.ValueType, err error), paths ...[]string)When you need to read multiple keys, and you do not afraid of low-level API EachKey is your friend. It read payload only single time, and calls callback function once path is found. For example when you call multiple times Get, it has to process payload multiple times, each time you call it. Depending on payload EachKey can be multiple times faster than Get. Path can use nested keys as well!
paths := [][]string{
[]string{"uuid"},
[]string{"tz"},
[]string{"ua"},
[]string{"st"},
}
var data SmallPayload
jsonparser.EachKey(smallFixture, func(idx int, value []byte, vt jsonparser.ValueType, err error){
switch idx {
case 0:
data.Uuid, _ = value
case 1:
v, _ := jsonparser.ParseInt(value)
data.Tz = int(v)
case 2:
data.Ua, _ = value
case 3:
v, _ := jsonparser.ParseInt(value)
data.St = int(v)
}
}, paths...)**`Set`**
func Set(data []byte, setValue []byte, keys ...string) (value []byte, err error)Receives existing data structure, key path to set, and value to set at that key. This functionality is experimental.
Returns:
value- Pointer to original data structure with updated or added key value.err- If any parsing issue, it should return error.
Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures).
Note that keys can be an array indexes: jsonparser.Set(data, []byte("http://github.com"), "person", "avatars", "[0]", "url")
**`Delete`**
func Delete(data []byte, keys ...string) value []byteReceives existing data structure, and key path to delete. This functionality is experimental.
Returns:
value- Pointer to original data structure with key path deleted if it can be found. If there is no key path, then the whole data structure is deleted.
Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures).
Note that keys can be an array indexes: jsonparser.Delete(data, "person", "avatars", "[0]", "url")
**`GetUint64`**
func GetUint64(data []byte, keys ...string) (uint64, error)Like GetInt but returns uint64. Use for large IDs, timestamps, or counters that exceed int64.
**`GetArrayLen`** and **`GetObjectLen`**
func GetArrayLen(data []byte, keys ...string) (int, error)
func GetObjectLen(data []byte, keys ...string) (int, error)Returns the number of elements in the addressed array or object without invoking a callback. Single-pass count.
**`EachKeyErr`**
func EachKeyErr(data []byte, cb func(idx int, value []byte, vt ValueType, err error) error, paths ...[]string) errorLike EachKey but the callback returns an error. Return io.EOF for graceful stop, any other error to abort iteration immediately.
**`DeleteFound`**
func DeleteFound(data []byte, keys ...string) (result []byte, found bool)Like Delete but also returns whether the key was found and removed.
**`Escape`** and **`SetString`**
func Escape(in string) []byte
func SetString(data []byte, val string, keys ...string) ([]byte, error)Escape produces a JSON-quoted, RFC 8259-escaped string literal (the inverse of Unescape).SetString is a convenience wrapper that calls Set with the value pre-escaped and quoted β so you never get invalid JSON from forgetting quotes:
// Instead of: Set(data, []byte("hello"), "key") // produces invalid JSON
data, _ = jsonparser.SetString(data, "hello", "key") // produces {"key":"hello"}**`EachKeyWildcard`**, **`EachArrayWildcard`** and **`SetWildcard`**
func EachKeyWildcard(data []byte, cb func(idx int, value []byte, vt ValueType, err error), path ...string) error
func EachArrayWildcard(data []byte, cb func(idx int, value []byte, vt ValueType, offset int, err error) error, keys ...string) (int, error)
func SetWildcard(data []byte, setValue []byte, keys ...string) ([]byte, error)Wildcard path support using [*] to fan out across all array elements:
// Set "active" on every user in the array
jsonparser.SetWildcard(data, []byte("true"), "users", "[*]", "active")**`Append`**
func Append(data []byte, value []byte, keys ...string) ([]byte, error)Appends value to the end of the JSON array addressed by keys. When keys is
empty, Append addresses the top-level value. If a keyed path does not exist,Append creates it as a single-element array using Set's auto-vivification
behavior. Returns MalformedArrayError if the addressed value is not an array.
// Append to an array without knowing its length
data, _ = jsonparser.Append(data, []byte(`"new_item"`), "items")**`ParsePath`** and **`CompilePath`**
func ParsePath(jsonPath string) ([]string, error)
func CompilePath(jsonPath string) (CompiledPath, error)Parse JSONPath-style strings into jsonparser's []string path format:
path, _ := jsonparser.ParsePath("$.person.name.fullName")
// path == []string{"person", "name", "fullName"}CompilePath pre-compiles for reuse with Get, Set, Delete, etc:
cp, _ := jsonparser.CompilePath("$.users[0].name")
name, _ := cp.Get(data) // reuse across calls
data, _ = cp.Set(data, val)- It does not rely on
encoding/json,reflectionorinterface{}, the only real package dependency isbytes. - Operates with JSON payload on byte level, providing you pointers to the original data structure: no memory allocation.
- No automatic type conversions, by default everything is a []byte, but it provides you value type, so you can convert by yourself (there is few helpers included).
- Does not parse full record, only keys you specified
Benchmarks
There are 3 benchmark types, trying to simulate real-life usage for small, medium and large JSON payloads.
For each metric, the lower value is better. Time/op is in nanoseconds. Values better than standard encoding/json marked as bold text.
Methodology: Benchmarks run with
go test -bench=. -benchmem -count=5on Apple M4 Max (ARM64, darwin), Go 1.26.3. Median of 5 runs. All comparison libraries updated to their latest versions as of 2026-07-29.
Compared libraries:
- https://golang.org/pkg/encoding/json
- https://github.com/tidwall/gjson β path-based, like jsonparser
- https://github.com/bytedance/sonic β SIMD-accelerated full deserializer
- https://github.com/Jeffail/gabs/v2
- https://github.com/pquerna/ffjson
- https://github.com/mailru/easyjson
- https://github.com/buger/jsonparser
TLDR
jsonparser is the fastest overall across all payload sizes β faster than gjson, sonic, easyjson, and encoding/json β with zero allocations on every code path.
Small payload
Each test processes 190 bytes of http log as a JSON record.
It should read multiple fields.
https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_small_payload_test.go
| Library | time/op | bytes/op | allocs/op |
|---|---|---|---|
| encoding/json struct | 1,300 | 416 | 9 |
| bytedance/sonic | 460 | 522 | 4 |
| tidwall/gjson | 402 | 64 | 3 |
| pquerna/ffjson | 632 | 520 | 10 |
| mailru/easyjson | 237 | 216 | 7 |
| buger/jsonparser (EachObject) | 205 | 64 | 2 |
| buger/jsonparser (EachKey) | 227 | 0 | 0 |
| buger/jsonparser (Get) | 339 | 0 | 0 |
Medium payload
Each test processes a 2.4kb JSON record (based on Clearbit API).
It should read multiple nested fields and 1 array.
https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_medium_payload_test.go
| Library | time/op | bytes/op | allocs/op |
|---|---|---|---|
| encoding/json struct | 9,911 | 616 | 18 |
| bytedance/sonic | 2,703 | 3,428 | 14 |
| tidwall/gjson | 2,241 | 168 | 4 |
| pquerna/ffjson | 3,731 | 736 | 15 |
| mailru/easyjson | 2,368 | 216 | 7 |
| buger/jsonparser (Get) | 3,141 | 0 | 0 |
| buger/jsonparser (EachKey) | 1,657 | 0 | 0 |
jsonparser with EachKey beats every competitor on medium payloads while remaining zero-allocation.
Large payload
Each test processes a 24kb JSON record (based on Discourse API).
It should read 2 arrays, and for each item in array get a few fields.
Basically it means processing a full JSON file.
https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_large_payload_test.go
| Library | time/op | bytes/op | allocs/op |
|---|---|---|---|
| encoding/json struct | 130,565 | 4,432 | 147 |
| bytedance/sonic | 41,053 | 31,368 | 71 |
| pquerna/ffjson | 59,063 | 4,822 | 144 |
| mailru/easyjson | 33,771 | 4,016 | 134 |
| tidwall/gjson | 22,756 | 28,672 | 2 |
| buger/jsonparser | 20,114 | 0 | 0 |
jsonparser is the fastest library overall on large payloads: 6.5x faster than encoding/json, 2x faster than sonic, 1.7x faster than easyjson, 1.1x faster than gjson β and the only zero-allocation parser.
Formal Verification
This project uses ReqProof for formal requirements verification, achieving:
- 92 formally specified requirements covering all public API behavior including edge cases, malformed input, boundary values, and error propagation
- 100% MC/DC coverage (Modified Condition/Decision Coverage) β every boolean decision in the code is independently proven exercised
- Kind2 model checking β mathematical proof that the specification is realizable and consistent
- Z3 SMT proofs β data-level properties verified for all possible inputs, not just test samples
ReqProof found 2 real bugs during the verification process (see PR #281):
Deletepanic on truncated JSON input β bounds check missing after internal sentinel valueEachArraycallback silently swallowing parse errors β the callback'serrparameter was always nil
It also identified and safely removed 7 dead code blocks that MC/DC analysis proved unreachable from any input.
The verification runs on every PR via probelabs/proof-action.
Questions and support
All bug-reports and suggestions should go though Github Issues.
Contributing
- Fork it
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am 'Added some feature')
- Push to the branch (git push origin my-new-feature)
- Create new Pull Request
Development
All my development happens using Docker, and repo include some Make tasks to simplify development.
make build- builds docker image, usually can be called only oncemake test- run testsmake fmt- run go fmtmake bench- run benchmarks (if you need to run only single benchmark modifyBENCHMARKvariable in make file)make profile- runs benchmark and generate 3 files-cpu.out,mem.mprofandbenchmark.testbinary, which can be used forgo tool pprofmake bash- enter container (i use it for runninggo tool pprofabove)
2. Official Technical Reference & Guides (buger/docs)
File: README.md
fastlane docs
This repo contains all documentation for fastlane. You can find the .md files inside the docs folder.
To preview the updated documentation locally, just clone the repo, modify the file, and run
mkdocs serveThe live version is available on docs.fastlane.tools.
Why new docs?
You can find more information here
Deploy changes
Once a PR is merged into master, the latest version will automatically be deployed to docs.fastlane.tools by one of the hard working fastlane bots. No need to do anything manually. The resulting HTML files are stored in the gh-pages branch.
Installation of [mkdocs](http://www.mkdocs.org/)
brew install python # if you don't have pip already
[sudo] pip install -r requirements.txtAdding a Python dependency
Likely, to add a Markdown extension.
- Install with
pip install <some extension> - Add
to requirements-to-freeze.txt. - Run
pip freeze > requirements.txtto update the exact requirements.
Why requirements-to-freeze.txt?
Based on A Better Pip Workflow,requirements-to-freeze.txt and requirements.txt provide a similar experience
to Gemfile and Gemfile.lock, respectively.
requirements-to-freeze.txt lets you pick out the top level packages the
project depends on, while requirements.txt exactly specifies all of the
dependencies and subdependencies for repeatable builds.
File: docs/index.md
fastlane
{: .badge }
{: .badge }
{: .badge }
fastlane is the easiest way to automate beta deployments and releases for your iOS and Android apps. π It handles all tedious tasks, like generating screenshots, dealing with code signing, and releasing your application.
You can start by creating a Fastfile file in your repository, hereβs one that defines your beta or App Store release process:
lane :beta do
increment_build_number
gym # Build your app
testflight # Upload to TestFlight
end
lane :appstore do
snapshot # Generate screenshots for the App Store
gym # Build your app
deliver # Upload the screenshots and the binary to iTunes
slack # Let your team-mates know the new version is live
endYou just defined 2 different lanes, one for beta deployment, one for App Store. To release your app in the App Store, all you have to do is
fastlane appstoreWhy fastlane?
| fastlane | |
|---|---|
| π | Save hours every time you push a new release to the store or beta testing service |
| β¨ | Integrates with all your existing tools and services (170 actions currently) |
| π | 100% open source under the MIT license |
| π© | Easy setup assistant to get started in a few minutes |
| β | Runs on your machine, it's your app and your data |
| π» | Integrates with all major CI systems |
| π₯ | Supports iOS, Mac, and Android apps |
| π§ | Extend and customise fastlane to fit your needs, you're not dependent on anyone |
| π | Never remember any commands any more, just fastlane |
| π’ | Deploy from any computer, including a CI server |
Getting Started
Make sure you have the latest version of the Xcode command line tools installed:
xcode-select --installChoose your installation method:
| Homebrew | Installer Script | Rubygems |
|---|---|---|
| macOS | macOS | macOS or Linux with Ruby 2.0.0 or above |
brew cask install fastlane |
Download the zip file. Then double click on the install script (or run it in a terminal window). |
sudo gem install fastlane -NV |
Navigate to your iOS or Android app and run
fastlane initfastlane will automatically detect your project, and ask for any missing information.
For more details about how to get up and running, check out the getting started guides:
Using the Fabric app
If you're not familiar with the terminal and prefer a graphical user interface, download the Fabric app and use the built-in fastlane button.
Questions and support
All fastlane related questions, problems and feature ideas should be posted as GitHub issue. This way, other users can search existing discussions, and everybody can jump in.
System requirements
fastlane requires macOS or Linux with Ruby 2.0.0 or above
License
This project is licensed under the terms of the MIT license. See the LICENSE file.
This project and all fastlane tools are in no way affiliated with Apple Inc or Google. This project is open source under the MIT license, which means you have full access to the source code and can modify it to fit your own needs. All fastlane tools run on your own computer or server, so your credentials or other sensitive information will never leave your own computer. You are responsible for how you use fastlane tools.