## File: README.md # Elvish [](https://github.com/elves/elvish/actions?query=workflow%3ACI) [](https://cirrus-ci.com/github/elves/elvish/master) [](https://app.codecov.io/gh/elves/elvish/tree/master) [](https://pkg.go.dev/src.elv.sh@master) [](https://repology.org/project/elvish/versions) [](https://bbs.elv.sh) [](https://twitter.com/ElvishShell) [](https://t.me/+Pv5ZYgTXD-YaKwcP) [](https://discord.gg/jrmuzRBU8D) [](https://matrix.to/#/#users:elv.sh) [](https://web.libera.chat/#elvish) [](https://gitter.im/elves/elvish) (Chat rooms are all bridged together thanks to [Matrix](https://matrix.org).) Elvish is: - A powerful scripting language. - A shell with useful interactive features built-in. - A statically linked binary for Linux, BSDs, macOS or Windows. Elvish is pre-1.0. This means that breaking changes will still happen from time to time, but it's stable enough for both scripting and interactive use. ## Documentation [](https://elv.sh) User docs are hosted on Elvish's website, [elv.sh](https://elv.sh). This includes [how to install Elvish](https://elv.sh/get/), [tutorials](https://elv.sh/learn/), [reference pages](https://elv.sh/ref/), and [news](https://elv.sh/blog/). [](./docs) Development docs are in [./docs](./docs). [](https://github.com/elves/awesome-elvish) Awesome Elvish packages and tools that support Elvish. ## License All source files use the BSD 2-clause license (see [LICENSE](LICENSE)), except for the following: - Files in [pkg/diff](pkg/diff) and [pkg/rpc](pkg/rpc) are released under the BSD 3-clause license, since they are derived from [Go's source code](https://github.com/golang/go). See [pkg/diff/LICENSE](pkg/diff/LICENSE) and [pkg/rpc/LICENSE](pkg/rpc/LICENSE). - Files in [pkg/persistent](pkg/persistent) and its subdirectories are released under EPL 1.0, since they are partially derived from [Clojure's source code](https://github.com/clojure/clojure). See [pkg/persistent/LICENSE](pkg/persistent/LICENSE). - Files in [pkg/md/spec](pkg/md/spec) are released under the Creative Commons CC-BY-SA 4.0 license, since they are derived from [the CommonMark spec](https://github.com/commonmark/commonmark-spec). See [pkg/md/spec/LICENSE](pkg/md/spec/LICENSE). --- ## File: docs/building.md # Building Elvish from source To build Elvish from source, you need - A supported OS: Linux, {Free,Net,Open}BSD, macOS, or Windows 10. Windows 10 support is experimental. - Go >= 1.21.0. To build Elvish from source, run one of the following commands: ```sh go install src.elv.sh/cmd/elvish@master # Install latest commit go install src.elv.sh/cmd/elvish@latest # Install latest released version go install src.elv.sh/cmd/elvish@v0.18.0 # Install a specific version ``` ## Controlling the installation location The [`go install`](https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies) command installs Elvish to `$GOBIN`; the binary name is `elvish`. You can control the installation location by overriding `$GOBIN`, for example by prepending `env GOBIN=...` to the `go install` command. If `$GOBIN` is not set, the installation location defaults to `$GOPATH/bin`, which in turn defaults to `~/go/bin` if `$GOPATH` is also not set. The installation directory is probably not in your OS's default `$PATH`. You should either either add it to `$PATH`, or manually copy the Elvish binary to a directory already in `$PATH`. ## Building an alternative entrypoint In additional to `src.elv.sh/cmd/elvish` (which corresponds to the [`cmd/elvish`](./cmd/elvish) directory in the repo), there are a few alternative entrypoints, all named liked `cmd/*/elvish`, with slightly different feature sets. (From the perspective of Go, these are just different `main` packages.) For example, install the `cmd/withpprof/elvish` entrypoint to get [profiling support](https://pkg.go.dev/runtime/pprof) (change the part after `@` to get different versions): ```sh go install src.elv.sh/cmd/withpprof/elvish@master ``` ## Building from a local source tree If you are modifying Elvish's source code, you will want to clone Elvish's Git repository and build Elvish from the local source tree instead. To do this, run the following from the root of the source tree: ```sh go install ./cmd/elvish ``` There is no need to specify a version like `@master`; when inside a source tree, `go install` will always use the whatever source code is present. See [contributing.md](contributing.md) for more notes for contributors. ## Building with experimental plugin support Elvish has experimental support for building and importing plugins, modules written in Go. It relies on Go's [plugin support](https://pkg.go.dev/plugin), which is only available on a few platforms. Plugin support requires building Elvish with [cgo](https://pkg.go.dev/cmd/cgo). The official [prebuilt binaries](https://elv.sh/get) are built without cgo for compatibility and reproducibility, but by default the Go toolchain builds with cgo enabled. If you have built Elvish from source on a platform with plugin support, your Elvish build probably already supports plugins. To force cgo to be used when building Elvish, you can do the following: ```sh env CGO_ENABLED=1 go install ./cmd/elvish ``` To build a plugin, see this [example](https://github.com/elves/sample-plugin). --- ## File: docs/contributing.md # Process for contributing to Elvish The only person with direct commit access is the project's founder @xiaq. If you intend to make user-visible changes to Elvish's behavior (as opposed to fixing typos and obvious bugs), it is good idea to talk to him first; this will make it easier to review your changes. He should be reachable in the user group most of the time. On the other hand, if you find it easier to express your thoughts directly in code, it is also completely fine to directly send a pull request, as long as you don't mind the risk of the PR being rejected due to lack of prior discussion. ## Licensing By contributing, you agree to license your code under the same license as existing source code of Elvish. See the [README](../README.md) at the project root for the license. --- ## File: docs/documenting.md # Documenting changes Always document user-visible changes. ## Release notes Add a brief list item to the release note of the next release, in the appropriate section. You can find the document at the root of the repo (called `$version-release-notes.md`). ## Reference docs Reference docs are written as "elvdocs", comment blocks before unindented `fn` or `var` declarations in Elvish files. A [large subset](https://pkg.go.dev/src.elv.sh/pkg/md@master) of [CommonMark](https://commonmark.org) is supported. Examples: ````elvish # Does something. # # Examples: # # ```elvish-transcript # ~> foo # some output # ``` fn foo {|a b c| } # Some variable. var bar ```` Most of Elvish's builtin modules are implemented in Go, not Elvish. For those modules, put dummy declarations in `.d.elv` files (`d` for "declaration"). For example, elvdocs for functions implemented in `builtin_fn_num.go` go in `builtin_fn_num.d.elv`. For a comment block to be considered an elvdoc, it has to be continuous, and each line should either be just `#` or start with `#` and a space. Style guides for elvdocs for functions: - The first sentence should start with a verb in 3rd person singular (i.e. ending with a "s"), as if there is an implicit subject "this function". - The end of the elvdoc should show or more `elvish-transcript` code blocks showing example usages, which are transcripts of actual REPL input and output. Transcripts must use the default prompt `~>` and default value output indicator `โ–ถ`. You can use `elvish -norc` if you have customized either in your [`rc.elv`](https://elv.sh/ref/command.html#rc-file). It is quite common for elvdocs to link to other elvdocs, and Elvish's website toolchain provides special support for that. If a link has a single code span and an empty target, it gets rewritten to a link to an elvdoc section. For example, ``[`put`]()`` will get rewritten to ``[`put`](builtin.html#put)``, or just ``[`put`](#put)`` within the documentation for the builtin module. ## Comment for unexported Go types and functions In the doc comment for exported types and functions, it's customary to use the symbol itself as the first word of the comment. For unexported types and functions, this becomes a bit awkward as their names don't start with a capital letter, so don't repeat the symbol. Examples: ```go // Foo does foo. func Foo() { } // Does foo. func foo() { } ``` --- ## File: docs/elvish-as-library.md # Using Elvish as a library Elvish's implementation is structured as a collection of Go packages with well-documented internal APIs, so it's possible to use the parts you're interested in as a Go library. - Most likely, you'll want to use Elvish's interpreter. The examples for the [`Evaler.Eval` method](https://pkg.go.dev/src.elv.sh@master/pkg/eval#Evaler.Eval) should give you a good starting point. - For a general overview of how Elvish's code is structured, read the [architecture overview](https://pkg.go.dev/src.elv.sh@master/docs/architecture). However, beware that Elvish promises no backward compatibility in its Go API. The internal API surface is large, and will change from time to time as Elvish's implementation gets refactored. For now, this is consistent with Go's semantic versioning rules as Elvish is pre-1.0. When Elvish 1.0 is eventually released, all the internal libraries will likely be moved into an `internal` directory, with a small part of the API exposed via facades in the `pkg` directory. --- ## File: docs/packaging.md # Packaging Elvish The main package of Elvish is `cmd/elvish`, and you can build it like any other Go application. ## Enhancing version information You can set some variables in the `src.elv.sh/pkg/buildinfo` package using linker flags to enhance the Elvish's version information. See the [package's API doc](https://pkg.go.dev/src.elv.sh@master/pkg/buildinfo) for details. They don't affect any other aspect of Elvish's behavior, so it's infeasible to pass those linker flags, it's fine to leave them as is. **Note**: The names and usage of these variables have changed several time in Elvish's history. If your build script has `-ldflags '-X $symbol=$value'` where `$symbol` is not documented in the linked API doc, those flags no longer do anything and should be removed. ## Running tests Some Elvish tests unfortunately rely on time thresholds. If you run tests as part of the packaging process, you may want to set the [`ELVISH_TEST_TIME_SCALE`](./testing.md#elvish_test_time_scale) environment variable to a large value like 10. --- ## File: docs/README.md ๐Ÿ’ก Tip: If you are looking for docs for Elvish users, like tutorials and reference pages, refer to Elvish's website [elv.sh](https://elv.sh) instead. This directory contains developer documentation: - ๐Ÿ—๏ธ [Building Elvish from source](building.md) - ๐Ÿ“ฆ [Packaging Elvish](packaging.md) - ๐Ÿ”‘ [Security policy](security.md) - ๐Ÿงฉ [Using Elvish as a library](elvish-as-library.md) If you'd like to contribute to Elvish: - ๐Ÿข [Architecture overview](https://pkg.go.dev/src.elv.sh@master/docs/architecture) This document is written as a godoc comment. You can also read the Go source [architecture/doc.go](architecture/doc.go). - ๐Ÿ‘‹ [Process for contributing to Elvish](contributing.md) - ๐Ÿงช [Testing changes](testing.md) - ๐Ÿ“š [Documenting changes](documenting.md) - ๐Ÿ”ง [Common development workflows](workflows.md) --- ## File: docs/security.md # Security Policy ## Supported Versions Only the HEAD and the last release is supported by the developers of Elvish. However, since some operating systems contain outdated Elvish packages, please also feel free to get in touch for security issues in unsupported versions. You can check the versions of Elvish packages on [Repology](https://repology.org/project/elvish/versions). ## Reporting a Vulnerability Please contact Qi Xiao at xiaqqaix@gmail.com. --- ## File: docs/workflows.md # Common development workflows The [`Makefile`](Makefile) encapsulates common development workflows: - Use `make fmt` to [format files](#formatting-files). - Use `make test` to [run tests](./testing.md). - Use `make all-checks` or `make most-checks` to [run checks](#running-checks). You can use the [`tools/pre-push`](../tools/pre-push) script as a Git hook, which runs all the tests and checks (`make test all-checks`), among other things. The same tests and checks are also run by Elvish's CI environments, so running them locally before pushing minimizes the chance of CI errors. (The CI environments run the tests on multiple platforms, so CI errors can still happen if you break some tests for a different platform.) ## Formatting files Use `make fmt` to format Go and Markdown files in the repo. ### Formatting Go files on save The Go plugins of most popular editors already support formatting Go files automatically on save; consult the documentation of the plugin you use. ### Formatting Markdown files on save The Markdown formatter is [`cmd/elvmdfmt`](../cmd/elvmdfmt), which lives inside this repo. Run it like this: ```sh go run src.elv.sh/cmd/elvmdfmt -width 80 -w $filename ``` To format Markdown files automatically on save, configure your editor to run the command above when saving Markdown files. You'll also want to configure this command to only run inside the Elvish repo, since `elvmdfmt` is tailored to Markdown files in this repo and may not work well for other Markdown files. If you use VS Code, install the [Run on Save](https://marketplace.visualstudio.com/items?itemName=emeraldwalk.RunOnSave) extension and add the following to the workspace (**not** user) `settings.json` file: ```json "emeraldwalk.runonsave": { "commands": [ { "match": "\\.md$", "cmd": "go run src.elv.sh/cmd/elvmdfmt -width 80 -w ${file}" } ] } ``` **Note**: Using `go run` ensures that you are always using the `elvmdfmt` implementation in the repo, but it incurs a small performance penalty since the Go toolchain does not cache binary files and has to rebuild it every time. If this is a problem (for example, if your editor runs the command synchronously), you can speed up the command by installing `src.elv.sh/cmd/elvmdfmt` and using the installed `elvmdfmt`. However, if you do this, you must re-install `elvmdfmt` whenever there is a change in its implementation that impacts the output. ## Generating code Elvish uses generated code in a few places. As is the usual case with Go projects, they are committed into the repo, and if you change the input of a generated file you should re-generate it. Use the standard command, `go generate ./...` to regenerate all files. Some of the generation rules depend on the `stringer` tool. Install with `go install golang.org/x/tools/cmd/stringer@latest`. ## Running checks There are some checks on the source code that can be run with `make all-checks` or `make most-checks`. The difference is that `all-checks` includes a check ([`tools/check-gen.sh`](../tools/check-gen.sh)) that requires the Git repo to have a clean working tree, so may not be convenient to use when you are working on the source code. The `most-checks` target excludes that, so can be always be used. The checks depend on some external programs, which can be installed as follows: ```sh go install golang.org/x/tools/cmd/goimports@latest go install honnef.co/go/tools/cmd/staticcheck@v0.6.0 pip install --user codespell==2.3.0 ``` ## Licensing By contributing, you agree to license your code under the same license as existing source code of elvish. See the LICENSE file. --- ## File: pkg/persistent/README.md # Persistent data structure in Go This is a Go clone of Clojure's persistent data structures. License is [Eclipse Public License 1.0](http://opensource.org/licenses/eclipse-1.0.php) (like Clojure). ## Implementation notes The list provided here is a singly-linked list and is very trivial to implement. The implementation of persistent vector and hash map and based on a series of [excellent](http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation) [blog](http://blog.higher-order.net/2009/09/08/understanding-clojures-persistenthashmap-deftwice) [posts](http://blog.higher-order.net/2010/08/16/assoc-and-clojures-persistenthashmap-part-ii.html) as well as the Clojure source code. Despite the hash map appearing more complicated, the vector is slightly harder to implement due to the "tail array" optimization and some tricky transformation of the tree structure, which is fully replicated here. ## Benchmarking results ### Vectors Compared to native slices, - Adding elements is anywhere from 5x to 9x as slow. - Read (sequential or random) is about 6x as slow. Benchmarked on an MacBook Air (M1, 2020), with Go 1.17.5: ``` BenchmarkConjNativeN1-8 1779234 673.3 ns/op BenchmarkConjNativeN2-8 948654 1220 ns/op BenchmarkConjNativeN3-8 61242 20138 ns/op BenchmarkConjNativeN4-8 1222 968176 ns/op BenchmarkConjPersistentN1-8 264488 4462 ns/op 6.63x BenchmarkConjPersistentN2-8 119526 9885 ns/op 8.10x BenchmarkConjPersistentN3-8 6760 173995 ns/op 8.64x BenchmarkConjPersistentN4-8 212 5576977 ns/op 5.76x BenchmarkIndexSeqNativeN4-8 32031 37344 ns/op BenchmarkIndexSeqPersistentN4-8 6145 192151 ns/op 5.15x BenchmarkIndexRandNative-8 31366 38016 ns/op BenchmarkIndexRandPersistent-8 5434 216284 ns/op 5.69x BenchmarkEqualNative-8 110090 10738 ns/op BenchmarkEqualPersistent-8 2121 557334 ns/op 51.90x ``` ### Hash map Compared to native maps, adding elements is about 3-6x slow. Difference is more pronunced when keys are sequential integers, but that workload is very rare in the real world. Benchmarked on an MacBook Air (M1, 2020), with Go 1.17.5: ``` goos: darwin goarch: arm64 pkg: src.elv.sh/pkg/persistent/hashmap BenchmarkSequentialConjNative1-8 620540 1900 ns/op BenchmarkSequentialConjNative2-8 22918 52209 ns/op BenchmarkSequentialConjNative3-8 567 2115886 ns/op BenchmarkSequentialConjPersistent1-8 169776 7026 ns/op 3.70x BenchmarkSequentialConjPersistent2-8 3374 354031 ns/op 6.78x BenchmarkSequentialConjPersistent3-8 51 23091870 ns/op 10.91x BenchmarkRandomStringsConjNative1-8 379147 3155 ns/op BenchmarkRandomStringsConjNative2-8 10000 117332 ns/op BenchmarkRandomStringsConjNative3-8 292 4034937 ns/op BenchmarkRandomStringsConjPersistent1-8 96504 12207 ns/op 3.87x BenchmarkRandomStringsConjPersistent2-8 1910 615644 ns/op 5.25x BenchmarkRandomStringsConjPersistent3-8 33 31928604 ns/op 7.91x ```