Nim Documentation

File: README.md

Nim

This repository contains the Nim compiler, Nim's stdlib, tools, and documentation.
For more information about Nim, including downloads and documentation for
the latest release, check out Nim's website or bleeding edge docs.

Community

The forum - the best place to ask questions and to discuss Nim.
#nim IRC Channel (Libera Chat) - a place to discuss Nim in real-time.
Also where most development decisions get made.
Discord - an additional place to discuss Nim in real-time. Most
channels there are bridged to IRC.
Gitter - an additional place to discuss Nim in real-time. There
is a bridge between Gitter and the IRC channel.
Matrix - the main room to discuss Nim in real-time. Matrix space contains a list of rooms, most of them are bridged to IRC.
Telegram - an additional place to discuss Nim in real-time. There
is the official Telegram channel. Not bridged to IRC.
Stack Overflow - a popular Q/A site for programming related
topics that includes posts about Nim.
GitHub Wiki - Misc user-contributed content.

Compiling

The compiler currently officially supports the following platform and
architecture combinations:

Operating System	Architectures Supported
Windows (Windows XP or greater)	x86 and x86_64
Linux (most distributions)	x86, x86_64, ppc64, and armv6l
Mac OS X (10.4 or greater)	x86, x86_64, ppc64, and Apple Silicon (ARM64)

More platforms are supported, however, they are not tested regularly and they
may not be as stable as the above-listed platforms.

Compiling the Nim compiler is quite straightforward if you follow these steps:

First, the C source of an older version of the Nim compiler is needed to
bootstrap the latest version because the Nim compiler itself is written in the
Nim programming language. Those C sources are available within the
nim-lang/csources_v3 repository.

Next, to build from source you will need:

A C compiler such as gcc 6.x/later or an alternative such as clang,
Visual C++ or Intel C++. It is recommended to use gcc 6.x or
later.
Either git or wget to download the needed source repositories.
The build-essential package when using gcc on Ubuntu (and likely
other distros as well).
On Windows MinGW 4.3.0 (GCC 8.10) is the minimum recommended compiler.
Nim hosts a known working MinGW distribution:
- MinGW32.7z
- MinGW64.7z

Windows Note: Cygwin and similar POSIX runtime environments are not supported.

Then, if you are on a *nix system or Windows, the following steps should compile
Nim from source using gcc, git, and the koch build tool.

Note: The following commands are for the development version of the compiler.
For most users, installing the latest stable version is enough. Check out
the installation instructions on the website to do so: https://nim-lang.org/install.html.

For package maintainers: see packaging guidelines.

First, get Nim from GitHub:

text

git clone https://github.com/nim-lang/Nim.git
cd Nim

Next, run the appropriate build shell script for your platform:

build_all.sh (Linux, Mac)
build_all.bat (Windows)

Finally, once you have finished the build steps (on Windows, Mac, or Linux) you
should add the bin directory to your PATH.

Koch

koch is the build tool used to build various parts of Nim and to generate
documentation and the website, among other things. The koch tool can also
be used to run the Nim test suite.

Assuming that you added Nim's bin directory to your PATH, you may execute
the tests using ./koch tests. The tests take a while to run, but you
can run a subset of tests by specifying a category (for example
./koch tests cat async).

For more information on the koch build tool please see the documentation
within the doc/koch.md file.

Nimble

nimble is Nim's package manager. To learn more about it, see the
nim-lang/nimble repository.

Contributors

This project exists thanks to all the people who contribute.

Contributing

See detailed contributing guidelines.
We welcome all contributions to Nim regardless of how small or large
they are. Everything from spelling fixes to new modules to be included in the
standard library are welcomed and appreciated. Before you start contributing,
you should familiarize yourself with the following repository structure:

bin/, build/ - these directories are empty, but are used when Nim is built.
compiler/ - the compiler source code. Also includes plugins within compiler/plugins.
nimsuggest - the nimsuggest tool that previously lived in the nim-lang/nimsuggest repository.
config/ - the configuration for the compiler and documentation generator.
doc/ - the documentation files in reStructuredText format.
lib/ - the standard library, including:
- pure/ - modules in the standard library written in pure Nim.
- impure/ - modules in the standard library written in pure Nim with
  dependencies written in other languages.
- wrappers/ - modules that wrap dependencies written in other languages.
tests/ - contains categorized tests for the compiler and standard library.
tools/ - the tools including niminst (mostly invoked via
koch).
koch.nim - the tool used to bootstrap Nim, generate C sources, build the website,
and generate the documentation.

If you are not familiar with making a pull request using GitHub and/or git, please
read this guide.

Ideally, you should make sure that all tests pass before submitting a pull request.
However, if you are short on time, you can just run the tests specific to your
changes by only running the corresponding categories of tests. CI verifies
that all tests pass before allowing the pull request to be accepted, so only
running specific tests should be harmless.
Integration tests should go in tests/untestable.

If you're looking for ways to contribute, please look at our issue tracker.
There are always plenty of issues labeled Easy; these should
be a good starting point for an initial contribution to Nim.

You can also help with the development of Nim by making donations. Donations can be
made using:

If you have any questions feel free to submit a question on the
Nim forum, or via IRC on the #nim channel.

Backers

Thank you to all our backers! [Become a backer]

License

The compiler and the standard library are licensed under the MIT license, except
for some modules which explicitly state otherwise. As a result, you may use any
compatible license (essentially any license) for your own programs developed with
Nim. You are explicitly permitted to develop commercial applications using Nim.

Please read the copying.txt file for more details.

File: Readme.md

Nim

Community

The forum - the best place to ask questions and to discuss Nim.
#nim IRC Channel (Libera Chat) - a place to discuss Nim in real-time.
Also where most development decisions get made.
Discord - an additional place to discuss Nim in real-time. Most
channels there are bridged to IRC.
Gitter - an additional place to discuss Nim in real-time. There
is a bridge between Gitter and the IRC channel.
Matrix - the main room to discuss Nim in real-time. Matrix space contains a list of rooms, most of them are bridged to IRC.
Telegram - an additional place to discuss Nim in real-time. There
is the official Telegram channel. Not bridged to IRC.
Stack Overflow - a popular Q/A site for programming related
topics that includes posts about Nim.
GitHub Wiki - Misc user-contributed content.

Compiling

The compiler currently officially supports the following platform and
architecture combinations:

Operating System	Architectures Supported
Windows (Windows XP or greater)	x86 and x86_64
Linux (most distributions)	x86, x86_64, ppc64, and armv6l
Mac OS X (10.4 or greater)	x86, x86_64, ppc64, and Apple Silicon (ARM64)

More platforms are supported, however, they are not tested regularly and they
may not be as stable as the above-listed platforms.

Compiling the Nim compiler is quite straightforward if you follow these steps:

Next, to build from source you will need:

A C compiler such as gcc 6.x/later or an alternative such as clang,
Visual C++ or Intel C++. It is recommended to use gcc 6.x or
later.
Either git or wget to download the needed source repositories.
The build-essential package when using gcc on Ubuntu (and likely
other distros as well).
On Windows MinGW 4.3.0 (GCC 8.10) is the minimum recommended compiler.
Nim hosts a known working MinGW distribution:
- MinGW32.7z
- MinGW64.7z

Windows Note: Cygwin and similar POSIX runtime environments are not supported.

Then, if you are on a *nix system or Windows, the following steps should compile
Nim from source using gcc, git, and the koch build tool.

For package maintainers: see packaging guidelines.

First, get Nim from GitHub:

text

git clone https://github.com/nim-lang/Nim.git
cd Nim

Next, run the appropriate build shell script for your platform:

build_all.sh (Linux, Mac)
build_all.bat (Windows)

Finally, once you have finished the build steps (on Windows, Mac, or Linux) you
should add the bin directory to your PATH.

Koch

koch is the build tool used to build various parts of Nim and to generate
documentation and the website, among other things. The koch tool can also
be used to run the Nim test suite.

For more information on the koch build tool please see the documentation
within the doc/koch.md file.

Nimble

nimble is Nim's package manager. To learn more about it, see the
nim-lang/nimble repository.

Contributors

This project exists thanks to all the people who contribute.

Contributing

bin/, build/ - these directories are empty, but are used when Nim is built.
compiler/ - the compiler source code. Also includes plugins within compiler/plugins.
nimsuggest - the nimsuggest tool that previously lived in the nim-lang/nimsuggest repository.
config/ - the configuration for the compiler and documentation generator.
doc/ - the documentation files in reStructuredText format.
lib/ - the standard library, including:
- pure/ - modules in the standard library written in pure Nim.
- impure/ - modules in the standard library written in pure Nim with
  dependencies written in other languages.
- wrappers/ - modules that wrap dependencies written in other languages.
tests/ - contains categorized tests for the compiler and standard library.
tools/ - the tools including niminst (mostly invoked via
koch).
koch.nim - the tool used to bootstrap Nim, generate C sources, build the website,
and generate the documentation.

If you are not familiar with making a pull request using GitHub and/or git, please
read this guide.

If you're looking for ways to contribute, please look at our issue tracker.
There are always plenty of issues labeled Easy; these should
be a good starting point for an initial contribution to Nim.

You can also help with the development of Nim by making donations. Donations can be
made using:

If you have any questions feel free to submit a question on the
Nim forum, or via IRC on the #nim channel.

Backers

Thank you to all our backers! [Become a backer]

License

Please read the copying.txt file for more details.

File: readme.md

Nim

Community

The forum - the best place to ask questions and to discuss Nim.
#nim IRC Channel (Libera Chat) - a place to discuss Nim in real-time.
Also where most development decisions get made.
Discord - an additional place to discuss Nim in real-time. Most
channels there are bridged to IRC.
Gitter - an additional place to discuss Nim in real-time. There
is a bridge between Gitter and the IRC channel.
Matrix - the main room to discuss Nim in real-time. Matrix space contains a list of rooms, most of them are bridged to IRC.
Telegram - an additional place to discuss Nim in real-time. There
is the official Telegram channel. Not bridged to IRC.
Stack Overflow - a popular Q/A site for programming related
topics that includes posts about Nim.
GitHub Wiki - Misc user-contributed content.

Compiling

The compiler currently officially supports the following platform and
architecture combinations:

Operating System	Architectures Supported
Windows (Windows XP or greater)	x86 and x86_64
Linux (most distributions)	x86, x86_64, ppc64, and armv6l
Mac OS X (10.4 or greater)	x86, x86_64, ppc64, and Apple Silicon (ARM64)

More platforms are supported, however, they are not tested regularly and they
may not be as stable as the above-listed platforms.

Compiling the Nim compiler is quite straightforward if you follow these steps:

Next, to build from source you will need:

A C compiler such as gcc 6.x/later or an alternative such as clang,
Visual C++ or Intel C++. It is recommended to use gcc 6.x or
later.
Either git or wget to download the needed source repositories.
The build-essential package when using gcc on Ubuntu (and likely
other distros as well).
On Windows MinGW 4.3.0 (GCC 8.10) is the minimum recommended compiler.
Nim hosts a known working MinGW distribution:
- MinGW32.7z
- MinGW64.7z

Windows Note: Cygwin and similar POSIX runtime environments are not supported.

Then, if you are on a *nix system or Windows, the following steps should compile
Nim from source using gcc, git, and the koch build tool.

For package maintainers: see packaging guidelines.

First, get Nim from GitHub:

text

git clone https://github.com/nim-lang/Nim.git
cd Nim

Next, run the appropriate build shell script for your platform:

build_all.sh (Linux, Mac)
build_all.bat (Windows)

Finally, once you have finished the build steps (on Windows, Mac, or Linux) you
should add the bin directory to your PATH.

Koch

koch is the build tool used to build various parts of Nim and to generate
documentation and the website, among other things. The koch tool can also
be used to run the Nim test suite.

For more information on the koch build tool please see the documentation
within the doc/koch.md file.

Nimble

nimble is Nim's package manager. To learn more about it, see the
nim-lang/nimble repository.

Contributors

This project exists thanks to all the people who contribute.

Contributing

bin/, build/ - these directories are empty, but are used when Nim is built.
compiler/ - the compiler source code. Also includes plugins within compiler/plugins.
nimsuggest - the nimsuggest tool that previously lived in the nim-lang/nimsuggest repository.
config/ - the configuration for the compiler and documentation generator.
doc/ - the documentation files in reStructuredText format.
lib/ - the standard library, including:
- pure/ - modules in the standard library written in pure Nim.
- impure/ - modules in the standard library written in pure Nim with
  dependencies written in other languages.
- wrappers/ - modules that wrap dependencies written in other languages.
tests/ - contains categorized tests for the compiler and standard library.
tools/ - the tools including niminst (mostly invoked via
koch).
koch.nim - the tool used to bootstrap Nim, generate C sources, build the website,
and generate the documentation.

If you are not familiar with making a pull request using GitHub and/or git, please
read this guide.

If you're looking for ways to contribute, please look at our issue tracker.
There are always plenty of issues labeled Easy; these should
be a good starting point for an initial contribution to Nim.

You can also help with the development of Nim by making donations. Donations can be
made using:

If you have any questions feel free to submit a question on the
Nim forum, or via IRC on the #nim channel.

Backers

Thank you to all our backers! [Become a backer]

License

Please read the copying.txt file for more details.

File: changelogs/readme.md

Update changelog

After each release, call:

text

git mv changelog.md changelogs/changelog_0_19_0.md # use correct version
cp changelogs/changelog_X_XX_X.md changelog.md
git add changelog.md

Recent changelogs are saved here (with their git history)

Older changelogs are stored in https://github.com/nim-lang/website

source files:

urls:

File: compiler/readme.md

Nim Compiler

This directory contains the Nim compiler written in Nim.
Note that this code has been translated from a bootstrapping version written in Pascal.
So the code is not a poster child of good Nim code.

See Internals of the Nim Compiler for more information.

File: doc/manual/var_t_return.md

.. default-role:: code
.. include:: ../rstcommon.rst

Memory safety for returning by var T is ensured by a simple borrowing
rule: If result does not refer to a location pointing to the heap
(that is in result = X the X involves a ptr or ref access)
then it has to be derived from the routine's first parameter:

nim

proc forward[T](x: var T): var T =
  result = x # ok, derived from the first parameter.

proc p(param: var int): var int =
  var x: int
  # we know 'forward' provides a view into the location derived from
  # its first argument 'x'.
  result = forward(x) # Error: location is derived from `x`
                      # which is not p's first parameter and lives
                      # on the stack.

In other words, the lifetime of what result points to is attached to the
lifetime of the first parameter and that is enough knowledge to verify
memory safety at the call site.

File: doc/apis.md

================= API naming design

.. default-role:: code
.. include:: rstcommon.rst

The API is designed to be easy to use and consistent. Ease of use is
measured by the number of calls to achieve a concrete high-level action.

Naming scheme

The library uses a simple naming scheme that makes use of common abbreviations
to keep the names short but meaningful. Since version 0.8.2 many symbols have
been renamed to fit this scheme. The ultimate goal is that the programmer can
guess a name.

=================== ============ ======================================
English word To use Notes
=================== ============ ======================================
initialize initT init is used to create a
value type T
new newP new is used to create a
reference type P
find find should return the position where
something was found; for a bool result
use contains
contains contains often short for find() >= 0
append add use add instead of append
compare cmp should return an int with the
< 0 == 0 or > 0 semantics;
for a bool result use sameXYZ
put put, []= consider overloading []= for put
get get, [] consider overloading [] for get;
consider to not use get as a
prefix: len instead of getLen
length len also used for number of elements
size size, len size should refer to a byte size
capacity cap
memory mem implies a low-level operation
items items default iterator over a collection
pairs pairs iterator over (key, value) pairs
delete delete, del del is supposed to be faster than
delete, because it does not keep
the order; delete keeps the order
remove delete, del inconsistent right now
remove-and-return pop Table/TableRef alias to take
include incl
exclude excl
command cmd
execute exec
environment env
variable var
value value, val val is preferred, inconsistent right
now
executable exe
directory dir
path path path is the string "/usr/bin" (for
example), dir is the content of
"/usr/bin"; inconsistent right now
extension ext
separator sep
column col, column col is preferred, inconsistent right
now
application app
configuration cfg
message msg
argument arg
object obj
parameter param
operator opr
procedure proc
function func
coordinate coord
rectangle rect
point point
symbol sym
literal lit
string str
identifier ident
indentation indent
=================== ============ ======================================

File: doc/contributing.md

============ Contributing

.. default-role:: code
.. include:: rstcommon.rst

.. contents::

Contributing happens via "Pull requests" (PR) on GitHub. Every PR needs to be
reviewed before it can be merged and the Continuous Integration should be green.
The title of a PR should contain a brief description. If it fixes an issue,
in addition to the number of the issue, the title should also contain a description
of the issue.

The PR has to be approved by two core developers or by Araq.

Writing tests

There are 4 types of tests:

runnableExamples documentation comment tests, ran by nim doc mymod.nim:cmd:
These end up in documentation and ensure documentation stays in sync with code.
separate test files, e.g.: tests/stdlib/tos.nim.
In nim repo, testament:cmd: (see below) runs all
$nim/tests/*/t*.nim test files;
for nimble packages, see https://github.com/nim-lang/nimble#tests.
(deprecated) tests in when isMainModule: block, ran by nim r mymod.nim:cmd:.
nimble test:cmd: can run those in nimble packages when specified in a
task "test".
(not preferred) .. code-block:: nim RST snippets;
these should only be used in rst sources,
in nim sources runnableExamples should now always be preferred to those for
several reasons (cleaner syntax, syntax highlights, batched testing, and
parameter rdoccmd allows customization).

Not all the tests follow the convention here, feel free to change the ones
that don't. Always leave the code cleaner than you found it.

Stdlib

Each stdlib module (anything under lib/, e.g. lib/pure/os.nim) should
preferably have a corresponding separate test file, e.g. tests/stdlib/tos.nim.
The old convention was to add a when isMainModule: block in the source file,
which only gets executed when the tester is building the file.

Each test should be in a separate block: statement, such that
each has its own scope. Use boolean conditions and doAssert for the
testing by itself, don't rely on echo statements or similar; in particular, avoid
things like echo "done". Don't use unittest.suite and unittest.test.

Sample test:

nim

block: # foo
  doAssert foo(1) == 10

block: # bug #1234
  static: doAssert 1+1 == 2

block: # bug #1235
  var seq2D = newSeqWith(4, newSeq[bool](2))
  seq2D[0][0] = true
  seq2D[1][0] = true
  seq2D[0][1] = true
  doAssert seq2D == @[@[true, true], @[true, false],
                      @[false, false], @[false, false]]
  # doAssert with `not` can now be done as follows:
  doAssert not (1 == 2)

Always refer to a GitHub issue using the following exact syntax: bug #1234 as shown
above, so that it's consistent and easier to search or for tooling. Some browser
extensions (e.g. https://github.com/sindresorhus/refined-github) will even turn those
in clickable links when it works.

Rationale for using a separate test file instead of when isMainModule: block:

allows custom compiler flags or testing options (see details below)
faster CI since they can be joined in megatest (combined into a single test)
avoids making the parser do un-necessary work when a source file is merely imported
avoids mixing source and test code when reporting line of code statistics or code coverage

Compiler

The tests for the compiler use a testing tool called testament:cmd:. They are all
located in tests/ (e.g.: tests/destructor/tdestructor3.nim).
Each test has its own file. All test files are prefixed with t. If you want
to create a file for import into another test only, use the prefix m.

At the beginning of every test is the expected behavior of the test.
Possible keys are:

cmd: A compilation command template e.g. nim $target --threads:on $options $file:cmd:
output: The expected output (stdout + stderr), most likely via echo
exitcode: Exit code of the test (via exit(number))
errormsg: The expected compiler error message
file: The file the errormsg was produced at
line: The line the errormsg was produced at

For a full spec, see here: testament/specs.nim

An example of a test:

nim

discard """
  errormsg: "type mismatch: got (PTest)"
"""

type
  PTest = ref object

proc test(x: PTest, y: int) = nil

var buf: PTest
buf.test()

Running tests

You can run the tests with

cmd

./koch tests

which will run a good subset of tests. Some tests may fail. If you
only want to see the output of failing tests, go for

cmd

./koch tests --failing all

You can also run only a single category of tests. A category is a subdirectory
in the tests/ directory. There are a couple of special categories; for a
list of these, see testament/categories.nim, at the bottom.

cmd

./koch tests c lib # compiles / runs stdlib modules, including `isMainModule` tests
./koch tests c megatest # runs a set of tests that can be combined into 1

To run a single test:

cmd

./koch test run <category>/<name>    # e.g.: tuples/ttuples_issues
./koch test run tests/stdlib/tos.nim # can also provide relative path

For reproducible tests (to reproduce an environment more similar to the one
run by Continuous Integration on GitHub actions/azure pipelines), you may want to disable your
local configuration (e.g. in ~/.config/nim/nim.cfg) which may affect some
tests; this can also be achieved by using
export XDG_CONFIG_HOME=pathtoAlternateConfig:cmd: before running ./koch:cmd:
commands.

Comparing tests

Test failures can be grepped using Failure:.

The tester can compare two test runs. First, you need to create a
reference test. You'll also need to the commit id, because that's what
the tester needs to know in order to compare the two.

cmd

git checkout devel
DEVEL_COMMIT=$(git rev-parse HEAD)
./koch tests

Then switch over to your changes and run the tester again.

cmd

git checkout your-changes
./koch tests

Then you can ask the tester to create a testresults.html which will
tell you if any new tests passed/failed.

cmd

./koch tests --print html $DEVEL_COMMIT

Deprecation

Backwards compatibility is important. When renaming types, procedures, etc. the old name
must be marked as deprecated using the deprecated pragma:

nim

# for routines (proc/template/macro/iterator) and types:
proc oldProc(a: int, b: float): bool {.deprecated:
    "deprecated since v1.2.3; use `newImpl: string -> int` instead".} = discard

# for (const/var/let/fields) the msg is not yet supported:
const Foo {.deprecated.}  = 1

# for enum types, you can deprecate the type or some elements
# (likewise with object types and their fields):
type Bar {.deprecated.} = enum bar0, bar1
type Barz  = enum baz0, baz1 {.deprecated.}, baz2

See also Deprecated
pragma in the manual.

Documentation

When contributing new procs, be sure to add documentation, especially if
the proc is public. Even private procs benefit from documentation and can be
viewed using nim doc --docInternal foo.nim:cmd:.
Documentation begins on the line
following the proc definition, and is prefixed by ## on each line.

Runnable code examples are also encouraged, to show typical behavior with a few
test cases (typically 1 to 3 assert statements, depending on complexity).
These runnableExamples are automatically run by nim doc mymodule.nim:cmd:
as well as testament:cmd: and guarantee they stay in sync.

nim

proc addBar*(a: string): string =
  ## Adds "Bar" to `a`.
  runnableExamples:
    assert "baz".addBar == "bazBar"
  result = a & "Bar"

See parentDir example.

The RestructuredText Nim uses has a special syntax for including code snippets
embedded in documentation; these are not run by nim doc:cmd: and therefore are
not guaranteed to stay in sync, so runnableExamples is almost always preferred:

nim

proc someProc*(): string =
  ## Returns "something"
  ##
  ##   ```
  ##   echo someProc() # "something"
  ##   ```
  result = "something" # single-hash comments do not produce documentation

The ``` followed by a newline and an indentation instructs the
nim doc:cmd: command to produce syntax-highlighted example code with the
documentation (``` is sufficient inside a .nim module, while from
a .md one needs to set the language explicitly as ```nim).

When forward declaration is used, the documentation should be included with the
first appearance of the proc.

nim

proc hello*(): string
  ## Put documentation here
proc nothing() = discard
proc hello*(): string =
  ## ignore this
  echo "hello"

The preferred documentation style is to begin with a capital letter and use
the third-person singular. That is, between:

nim

proc hello*(): string =
  ## Returns "hello"
  result = "hello"

nim

proc hello*(): string =
  ## say hello
  result = "hello"

the first is preferred.

When you specify an RST role (highlighting/interpretation marker) do it
in the postfix form for uniformity, that is after `text in backticks`.
For example an :idx: role for referencing a topic ("SQLite" in the
example below) from [Nim Index] can be used in doc comment this way:

nim

## A higher level `SQLite`:idx: database wrapper.

.. _Nim Index: https://nim-lang.org/docs/theindex.html

Inline monospaced text can be input using `single backticks` or
``double backticks``. The former are syntactically highlighted,
the latter are not.
To avoid accidental highlighting follow this rule in *.nim files:

Use single backticks for fragments of code in Nim and other
programming languages, including identifiers, in *.nim files.

For languages other than Nim add a role after final backtick,
e.g. for C++ inline highlighting:
text
```
`#include <stdio.h>`:cpp:
```
For a currently unsupported language add the :code: role,
like for SQL in this example:
text
```
`SELECT * FROM ;`:code:
```
Highlight shell commands by :cmd: role; for command line options use
:option: role, e.g.: `--docInternal`:option:.
Use double backticks:
- For file names: ``os.nim``
- For fragments of strings not enclosed by " and " and not
  related to code, e.g. text of compiler messages
- When code ends with a standalone \ (otherwise a combination of
  \ and a final ` would get escaped)

.. Note:: *.rst files have :literal: as their default role.
So for them the rule above is only applicable if the :nim: role
is set up manually as the default [^1]:

text

.. role:: nim(code)
     :language: nim
  .. default-role:: nim

The first 2 lines are for other RST implementations,
including Github one.

[^1]: this is fulfilled when doc/rstcommon.rst is included.

Best practices

Note: these are general guidelines, not hard rules; there are always exceptions.
Code reviews can just point to a specific section here to save time and
propagate best practices.

.. _define_needs_prefix:
New defined(foo) symbols need to be prefixed by the nimble package name, or
by nim for symbols in nim sources (e.g. compiler, standard library). This is
to avoid name conflicts across packages.

nim

# if in nim sources
when defined(allocStats): discard # bad, can cause conflicts
when defined(nimAllocStats): discard # preferred
# if in a package `cligen`:
when defined(debug): discard # bad, can cause conflicts
when defined(cligenDebug): discard # preferred

.. _noimplicitbool:
Take advantage of no implicit bool conversion

nim

doAssert isValid() == true
doAssert isValid() # preferred

.. _design_for_mcs:
Design with method call syntax chaining in mind

nim

proc foo(cond: bool, lines: seq[string]) # bad
proc foo(lines: seq[string], cond: bool) # preferred
# can be called as: `getLines().foo(false)`

.. _avoid_quit:
Use exceptions (including assert / doAssert) instead of quit
rationale: https://forum.nim-lang.org/t/4089

nim

quit() # bad in almost all cases
doAssert() # preferred

.. _tests_use_doAssert:
Use doAssert (or unittest.check, unittest.require), not assert in all
tests, so they'll be enabled even with --assertions:off:option:.

nim

block: # foo
  assert foo() # bad
  doAssert foo() # preferred

.. _runnableExamples_use_assert:
An exception to the above rule is runnableExamples and code-block rst blocks
intended to be used as runnableExamples, which for brevity use assert
instead of doAssert. Note that nim doc -d:danger main:cmd: won't pass -d:danger:option: to the
runnableExamples, but nim doc --doccmd:-d:danger main:cmd: would, and so would the
second example below:

nim

runnableExamples:
  doAssert foo() # bad
  assert foo() # preferred

runnableExamples("-d:danger"):
  doAssert foo() # `assert` would be disabled here, so `doAssert` makes more sense

.. _delegate_printing:
Delegate printing to caller: return string instead of calling echo
rationale: it's more flexible (e.g. allows the caller to call custom printing,
including prepending location info, writing to log files, etc.).

nim

proc foo() = echo "bar" # bad
proc foo(): string = "bar" # preferred (usually)

.. _use_Option:
(Ongoing debate) Consider using Option instead of return bool + var argument,
unless stack allocation is needed (e.g. for efficiency).

nim

proc foo(a: var Bar): bool
proc foo(): Option[Bar]

.. _use_doAssert_not_echo:
Tests (including in testament) should always prefer assertions over echo,
except when that's not possible. It's more precise, easier for readers and
maintainers to where expected values refer to. See for example
https://github.com/nim-lang/Nim/pull/9335 and https://forum.nim-lang.org/t/4089

nim

echo foo() # adds a line for testament in `output:` block inside `discard`.
doAssert foo() == [1, 2] # preferred, except when not possible to do so.

The `git`:cmd: stuff

General commit rules

Important, critical bugfixes that have a tiny chance of breaking
somebody's code should be backported to the latest stable release
branch (currently 1.4.x) and maybe also all the way back to the 1.0.x branch.
The commit message should contain the tag [backport] for "backport to the latest
stable release" and the tag [backport:$VERSION] for backporting back to the
given $VERSION (and all newer releases).
If you introduce changes which affect backward compatibility,
make breaking changes, or have PR which is tagged as [feature],
the changes should be mentioned in the changelog.
All changes introduced by the commit (diff lines) must be related to the
subject of the commit.

If you change something unrelated to the subject parts of the file, because
your editor reformatted automatically the code or whatever different reason,
this should be excluded from the commit.

Tip: Never commit everything as-is using git commit -a:cmd:, but review
carefully your changes with git add -p:cmd:.
Changes should not introduce any trailing whitespace.

Always check your changes for whitespace errors using git diff --check:cmd:
or add the following pre-commit hook:
cmd
```
#!/bin/sh
git diff --check --cached || exit $?
```
Describe your commit and use your common sense.
Example commit message:
text
```
Fixes #123; refs #124
```
indicates that issue #123 is completely fixed (GitHub may automatically
close it when the PR is committed), whereas issue #124 is referenced
(e.g.: partially fixed) and won't close the issue when committed.
PR body (not just PR title) should contain references to fixed/referenced GitHub
issues, e.g.: fix #123 or refs #123. This is so that you get proper
cross-referencing from linked issue to the PR (GitHub won't make those links
with just a PR title, and commit messages aren't always sufficient to ensure
that, e.g. can't be changed after a PR is merged).
Commits should be always be rebased against devel (so a fast-forward
merge can happen)

e.g.: use git pull --rebase origin devel:cmd:. This is to avoid messing up
git history.
Exceptions should be very rare: when rebase gives too many conflicts, simply
squash all commits using the script shown in
https://github.com/nim-lang/Nim/pull/9356
Do not mix pure formatting changes (e.g. whitespace changes, nimpretty) or
automated changes with other code changes: these should be in
separate commits (and the merge on GitHub should not squash these into 1).

Continuous Integration (CI)

Continuous Integration is by default run on every push in a PR; this clogs
the CI pipeline and affects other PR's; if you don't need it (e.g. for WIP or
documentation only changes), add [skip ci] to your commit message title.
This convention is supported by our GitHub actions pipelines and our azure pipeline
(using custom logic, which should complete in < 1mn) as well as our former other pipelines:
Appveyor
and Travis.
Consider enabling CI (azure, GitHub actions and builds.sr.ht) in your own Nim fork, and
waiting for CI to be green in that fork (fixing bugs as needed) before
opening your PR in the original Nim repo, to reduce CI congestion. Same
applies for updates on a PR: you can test commits on a separate private
branch before updating the main PR.

Debugging CI failures, flaky tests, etc

First check the CI logs and search for FAIL to find why CI failed; if the
failure seems related to your PR, try to fix the code instead of restarting CI.
If CI failure seems unrelated to your PR, it could be caused by a flaky test.
File a bug for it if it isn't already reported. A PR push (or opening/closing PR)
will re-trigger all CI jobs (even successful ones, which can be wasteful). Instead,
request collaboration from the Nim team. The Nim team should
follow these instructions to only restart the jobs that failed:
- Azure: if on your own fork, it's possible from inside azure console
  (e.g. dev.azure.com/username/username/_build/results?buildId=1430&view=results) via
  rerun failed jobs on top.
  If either on you own fork or in Nim repo, it's possible from inside GitHub UI
  under checks tab, see https://github.com/timotheecour/Nim/issues/211#issuecomment-629751569
- GitHub actions: under "Checks" tab, click "Re-run jobs" in the right.
- builds.sr.ht: create a SourceHut account so that you can restart a PR job as illustrated.
  builds.sr.ht also allows you to ssh to a CI machine which can help a lot for debugging
  issues, see docs in https://man.sr.ht/builds.sr.ht/build-ssh.md and
  https://drewdevault.com/2019/08/19/Introducing-shell-access-for-builds.html; see
  https://man.sr.ht/tutorials/set-up-account-and-git.md to generate and upload ssh keys.

Code reviews

Whenever possible, use GitHub's new 'Suggested change' in code reviews, which
saves time explaining the change or applying it; see also
https://forum.nim-lang.org/t/4317
When reviewing large diffs that may involve code moving around, GitHub's interface
doesn't help much, as it doesn't highlight moves. Instead, you can use something
like this, see visual results here:
cmd
```
git fetch origin pull/10431/head && git checkout FETCH_HEAD
git diff --color-moved-ws=allow-indentation-change --color-moved=blocks HEAD^
```
In addition, you can view GitHub-like diffs locally to identify what was changed
within a code block using diff-highlight:cmd: or diff-so-fancy:cmd:, e.g.:
text
```
# put this in ~/.gitconfig:
 [core]
   pager = "diff-so-fancy | less -R" # or: use: `diff-highlight`
```

.. include:: docstyle.md

Evolving the stdlib

As outlined in https://github.com/nim-lang/RFCs/issues/173 there are a couple
of guidelines about what should go into the stdlib, what should be added and
what eventually should be removed.

What the compiler itself needs must be part of the stdlib

Maybe in the future the compiler itself can depend on Nimble packages but for
the time being, we strive to have zero dependencies in the compiler as the
compiler is the root of the bootstrapping process and is also used to build
Nimble.

Vocabulary types must be part of the stdlib

These are types most packages need to agree on for better interoperability,
for example Option[T]. This rule also covers the existing collections like
Table, CountTable etc. "Sorted" containers based on a tree-like data
structure are still missing and should be added.

Time handling, especially the Time type are also covered by this rule.

Existing, battle-tested modules stay

Reason: There is no benefit in moving them around just to fulfill some design
fashion as in "Nim's core MUST BE SMALL". If you don't like an existing module,
don't import it. If a compilation target (e.g. JS) cannot support a module,
document this limitation.

This covers modules like os, osproc, strscans, strutils,
strformat, etc.

Syntactic helpers can start as experimental stdlib modules

Reason: Generally speaking as external dependencies they are not exposed
to enough users so that we can see if the shortcuts provide enough benefit
or not. Many programmers avoid external dependencies, especially for
"tiny syntactic improvements". However, this is only true for really good
syntactic improvements that have the potential to clean up other parts of
the Nim library substantially. If in doubt, new stdlib modules should start
as external, successful Nimble packages.

Other new stdlib modules do not start as stdlib modules

As we strive for higher quality everywhere, it's easier to adopt existing,
battle-tested modules eventually rather than creating modules from scratch.

Little additions are acceptable

As long as they are documented and tested well, adding little helpers
to existing modules is acceptable. For two reasons:

It makes Nim easier to learn and use in the long run.
("Why does sequtils lack a countIt?
Because version 1.0 happens to have lacked it? Silly...")
To encourage contributions. Contributors often start with PRs that
add simple things, then they stay and also fix bugs. Nim is an
open source project and lives from people's contributions and involvement.
Newly introduced issues have to be balanced against motivating new people. We know where
to find perfectly designed pieces of software that have no bugs -- these are the systems
that nobody uses.

Conventions

New stdlib modules should go under Nim/lib/std/. The rationale is to
require users to import via import std/foo instead of import foo,
which would cause potential conflicts with nimble packages.
Note that this still applies for new modules in existing logical
directories, e.g.: use lib/std/collections/foo.nim,
not lib/pure/collections/foo.nim.
New module names should prefer plural form whenever possible, e.g.:
std/sums.nim instead of std/sum.nim. In particular, this reduces
chances of conflicts between module name and the symbols it defines.
Furthermore, module names should use snake_case and not use capital
letters, which cause issues when going from an OS without case
sensitivity to an OS with it.

Breaking Changes

Introducing breaking changes, no matter how well-intentioned,
creates long-term problems for the community, in particular those looking to promote
reusable Nim code in libraries: In the Nim distribution, critical security and bugfixes,
language changes and community improvements are bundled in a single distribution - it is
difficult to make partial upgrades with only benign changes. When one library depends on
a legacy behavior, it can no longer be used together with another library that does not,
breaking all downstream applications - the standard library is unique in that it sits at
the root of all dependency trees.

There is a big difference between compile-time breaking changes and run-time breaking
changes.

Run-time breaking changes

Run-time breaking changes are to be avoided at almost all costs: Nim is used for
mission critical applications which depend on behaviours that
are not covered by the test suite. As such, it's important that changes to the
stable parts of the standard library are made avoiding changing the existing
behaviors, even when the test suite continues to pass.

Examples of run-time breaking changes:

Raising exceptions of a new type, compared to what's currently being raised.
Adding unconstrained or poorly constrained generic procs or macros
("hash now works for all ref T"): This may cause code to behave differently
depending only on which modules are imported - common examples include == and hash.
Changing behavior of existing functions like:
- "Nim's path handling procs like getXDir now consistently lack the trailing slash"
- "Nim's strformat implementation is now more consistent with Python"

Instead, write new code that explicitly announces the feature you think we announced but
didn't. For example, strformat does not say "it's compatible with Python", it
says "inspired by Python's f-strings". This new code can be submitted to the stdlib
and the old code can be deprecated or published as a Nimble package.

Sometimes, a run-time breaking change is most desirable: For example, a string
representation of a floating point number that "roundtrips" is much better than
a string representation that doesn't. These run-time breaking changes must start in the
state "opt-in" via a new -d:nimPreviewX or command line flag and then should become
the new default later, in follow-up versions. This way users can track
regressions more easily. ("git bisect" is not an acceptable alternative, that's for
Nim compiler developers, not for Nim users.)

Above all else, additive approaches that don't change existing behaviors
should be preferred.

Compile-time breaking changes

Compile-time breaking changes are usually easier to handle, but for large code bases
they can also involve a large amount of work and can hinder the adoption of a new
Nim release.
Additive approaches are to be preferred here as well.

Examples of compile-time breaking changes include (but are not limited to):

Renaming functions and modules, or moving things. Instead of a direct rename,
deprecate the old name and introduce a new one.
Renaming the parameter names: Thanks to Nim's "named parameter" calling syntax
like f(x = 0, y = 1) this is a breaking change. Instead, live with the existing
parameter names.
Adding an enum value to an existing enum. Nim's exhaustive case statements stop
compiling after such a change. Instead, consider to introduce new bool
fields/parameters. This can be impractical though, so we use good judgement
and our list of "important packages" to see if it doesn't break too much code
out there in practice.
Adding a new proc to an existing stdlib module. However, for aesthetic reasons
this is often preferred over introducing a new module with just a single proc
inside. We use good judgement and our list of "important packages" to see if
it doesn't break too much code out there in practice. The new procs need to
be annotated with a .since annotation.

Compiler/language spec bugfixes

This can even be applied to compiler "bugfixes": If the compiler should have been
"pickier" in its handling of typedesc, instead of "fixing typedesc handling bugs",
consider the following solution:

Spec out how typedesc should really work and also spec out the cases where it
should not be allowed!
Deprecate typedesc and name the new metatype something new like typeArg.
Implement the spec.

Non-breaking changes

Examples of changes that are considered non-breaking (or acceptable breaking changes) include:

Creating a new module.
Adding an overload to an already overloaded proc.
Adding new default parameters to an existing proc. It is assumed that you do not
use Nim's stdlib procs's addresses (that you don't use them as first class entities).
Changing the calling convention from nimcall to inline
(but first RFC https://github.com/nim-lang/RFCs/issues/396 needs to be implemented).
Changing the behavior from "crashing" into some other, well documented result (including
raising a Defect, but not raising an exception that does not inherit from Defect).
Adding new fields to an existing object.

Nim's introspection facilities imply that strictly speaking almost every addition can
break somebody's code. It is impractical to care about these cases, a change that only
affects introspection is not considered to be a breaking change.

File: doc/destructors.md

================================== Nim Destructors and Move Semantics

:Authors: Andreas Rumpf
:Version: |nimversion|

.. default-role:: code
.. include:: rstcommon.rst
.. contents::

About this document

This document describes the ARC/ORC Nim runtime which does
not use classical GC algorithms anymore but is based on destructors and
move semantics. The advantages are that Nim programs become
oblivious to the involved heap sizes and programs are easier to write to make
effective use of multi-core machines. As a nice bonus, files and sockets and
the like can be written not to require manual close calls anymore.

This document aims to be a precise specification about how
move semantics and destructors work in Nim.

Motivating example

With the language mechanisms described here, a custom seq could be
written as:

text

/* Detailed source-code truncated for AI context efficiency. */

Lifetime-tracking hooks

The memory management for Nim's standard string and seq types as
well as other standard collections is performed via so-called
"Lifetime-tracking hooks", which are particular type bound operators.

There are 6 different hooks for each (generic or concrete) object type T (T can also be a
distinct type) that are called implicitly by the compiler.

(Note: The word "hook" here does not imply any kind of dynamic binding
or runtime indirections, the implicit calls are statically bound and
potentially inlined.)

`=destroy` hook

A =destroy hook frees the object's associated memory and releases
other associated resources. Variables are destroyed via this hook when
they go out of scope or when the routine they were declared in is about
to return.

A =destroy hook is allowed to have a parameter of a var T or T type.
The prototype of this hook for a type T needs to be:

nim

proc `=destroy`(x: T)

The general pattern in =destroy looks like:

nim

proc `=destroy`(x: T) =
  # first check if 'x' was moved to somewhere else:
  if x.field != nil:
    freeResource(x.field)

A =destroy is implicitly annotated with .raises: []; a destructor
should not raise exceptions. For backwards compatibility the compiler
produces a warning for a =destroy that does raise.

A =destroy can explicitly list the exceptions it can raise, if any,
but this of little utility as a raising destructor is implementation defined
behavior. Later versions of the language specification might cover this case precisely.

`=wasMoved` hook

A =wasMoved hook sets the object to a state that signifies to the destructor there is nothing to destroy.

The prototype of this hook for a type T needs to be:

nim

proc `=wasMoved`(x: var T)

Usually some pointer field inside the object is set to nil:

nim

proc `=wasMoved`(x: var T) =
  x.field = nil

`=sink` hook

A =sink hook moves an object around, the resources are stolen from the source
and passed to the destination. It is ensured that the source's destructor does
not free the resources afterward by setting the object to its default value
(the value the object's state started in). Setting an object x back to its
default value is written as wasMoved(x). When not provided the compiler
is using a combination of =destroy and copyMem instead. This is efficient
hence users rarely need to implement their own =sink operator, it is enough to
provide =destroy and =copy, the compiler will take care of the rest.

The prototype of this hook for a type T needs to be:

nim

proc `=sink`(dest: var T; source: T)

The general pattern in =sink looks like:

nim

proc `=sink`(dest: var T; source: T) =
  `=destroy`(dest)
  wasMoved(dest)
  dest.field = source.field

Note: =sink does not need to check for self-assignments.
How self-assignments are handled is explained later in this document.

`=copy` hook

The ordinary assignment in Nim conceptually copies the values. The =copy hook
is called for assignments that couldn't be transformed into =sink
operations.

The prototype of this hook for a type T needs to be:

nim

proc `=copy`(dest: var T; source: T)

The general pattern in =copy looks like:

nim

proc `=copy`(dest: var T; source: T) =
  # protect against self-assignments:
  if dest.field != source.field:
    `=destroy`(dest)
    wasMoved(dest)
    dest.field = duplicateResource(source.field)

The =copy proc can be marked with the {.error.} pragma. Then any assignment
that otherwise would lead to a copy is prevented at compile-time. This looks like:

nim

proc `=copy`(dest: var T; source: T) {.error.}

but a custom error message (e.g., {.error: "custom error".}) will not be emitted
by the compiler. Notice that there is no = before the {.error.} pragma.

`=trace` hook

A custom container type can support Nim's cycle collector --mm:orc via
the =trace hook. If the container does not implement =trace, cyclic data
structures which are constructed with the help of the container might leak
memory or resources, but memory safety is not compromised.

The prototype of this hook for a type T needs to be:

nim

proc `=trace`(dest: var T; env: pointer)

env is used by ORC to keep track of its internal state, it should be passed around
to calls of the built-in =trace operation.

Usually there will only be a need for a custom =trace when a custom =destroy that deallocates
manually allocated resources is also used, and then only when there is a chance of cyclic
references from items within the manually allocated resources when it is desired that --mm:orc
is able to break and collect these cyclic referenced resources. Currently however, there is a
mutual use problem in that whichever of =destroy/=trace is used first will automatically
create a version of the other which will then conflict with the creation of the second of the
pair. The workaround for this problem is to forward declare the second of the "hooks" to
prevent the automatic creation.

The general pattern in using =destroy with =trace looks like:

nim

type
  Test[T] = object
    size: Natural
    arr: ptr UncheckedArray[T] # raw pointer field

proc makeTest[T](size: Natural): Test[T] = # custom allocation...
  Test[T](size: size, arr: cast[ptr UncheckedArray[T]](alloc0(sizeof(T) * size)))


proc `=destroy`[T](dest: Test[T]) =
  if dest.arr != nil:
    for i in 0 ..< dest.size: dest.arr[i].`=destroy`
    dealloc dest.arr

proc `=trace`[T](dest: var Test[T]; env: pointer) =
  if dest.arr != nil:
    # trace the `T`'s which may be cyclic
    for i in 0 ..< dest.size: `=trace`(dest.arr[i], env)

# following may be other custom "hooks" as required...

Note: The =trace hooks (which are only used by --mm:orc) are currently more experimental and less refined
than the other hooks.

`=dup` hook

A =dup hook duplicates an object. =dup(x) can be regarded as an optimization replacing a wasMoved(dest); =copy(dest, x) operation.

The prototype of this hook for a type T needs to be:

nim

proc `=dup`(x: T): T

The general pattern in implementing =dup looks like:

nim

type
  Ref[T] = object
    data: ptr T
    rc: ptr int

proc `=dup`[T](x: Ref[T]): Ref[T] =
  result = x
  if x.rc != nil:
    inc x.rc[]

Move semantics

A "move" can be regarded as an optimized copy operation. If the source of the
copy operation is not used afterward, the copy can be replaced by a move. This
document uses the notation lastReadOf(x) to describe that x is not
used afterward. This property is computed by a static control flow analysis
but can also be enforced by using system.move explicitly.

One can query if the analysis is able to perform a move with system.ensureMove.
move enforces a move operation and calls =wasMoved whereas ensureMove is
an annotation that implies no runtime operation. An ensureMove annotation leads to a static error
if the compiler cannot prove that a move would be safe.

For example:

nim

proc main(normalParam: string; sinkParam: sink string) =
  var x = "abc"
  # valid:
  let valid = ensureMove x
  # invalid:
  let invalid = ensureMove normalParam
  # valid:
  let alsoValid = ensureMove sinkParam

Swap

The need to check for self-assignments and also the need to destroy previous
objects inside =copy and =sink is a strong indicator to treat
system.swap as a builtin primitive of its own that simply swaps every
field in the involved objects via copyMem or a comparable mechanism.
In other words, swap(a, b) is not implemented
as let tmp = move(b); b = move(a); a = move(tmp).

This has further consequences:

Objects that contain pointers that point to the same object are not supported
by Nim's model. Otherwise swapped objects would end up in an inconsistent state.
Seqs can use realloc in the implementation.

Sink parameters

To move a variable into a collection usually sink parameters are involved.
A location that is passed to a sink parameter should not be used afterward.
This is ensured by a static analysis over a control flow graph. If it cannot be
proven to be the last usage of the location, a copy is done instead and this
copy is then passed to the sink parameter.

A sink parameter
may be consumed once in the proc's body but doesn't have to be consumed at all.
The reason for this is that signatures
like proc put(t: var Table; k: sink Key, v: sink Value) should be possible
without any further overloads and put might not take ownership of k if
k already exists in the table. Sink parameters enable an affine type system,
not a linear type system.

The employed static analysis is limited and only concerned with local variables;
however, object and tuple fields are treated as separate entities:

nim

proc consume(x: sink Obj) = discard "no implementation"

proc main =
  let tup = (Obj(), Obj())
  consume tup[0]
  # ok, only tup[0] was consumed, tup[1] is still alive:
  echo tup[1]

Sometimes it is required to explicitly move a value into its final position:

nim

proc main =
  var dest, src: array[10, string]
  # ...
  for i in 0..high(dest): dest[i] = move(src[i])

An implementation is allowed, but not required to implement even more move
optimizations (and the current implementation does not).

Sink parameter inference

The current implementation can do a limited form of sink parameter
inference. But it has to be enabled via --sinkInference:on:option:, either
on the command line or via a push pragma.

To enable it for a section of code, one can
use {.push sinkInference: on.} ... {.pop.}.

The .nosinks:idx: pragma can be used to disable this inference
for a single routine:

nim

proc addX(x: T; child: T) {.nosinks.} =
  x.s.add child

The details of the inference algorithm are currently undocumented.

Rewrite rules

Note: There are two different allowed implementation strategies:

The produced finally section can be a single section that is wrapped
around the complete routine body.
The produced finally section is wrapped around the enclosing scope.

The current implementation follows strategy (2). This means that resources are
destroyed at the scope exit.

text

var x: T; stmts
---------------             (destroy-var)
var x: T; try stmts
finally: `=destroy`(x)


g(f(...))
------------------------    (nested-function-call)
g(let tmp;
bitwiseCopy tmp, f(...);
tmp)
finally: `=destroy`(tmp)


x = f(...)
------------------------    (function-sink)
`=sink`(x, f(...))


x = lastReadOf z
------------------          (move-optimization)
`=sink`(x, z)
`=wasMoved`(z)


v = v
------------------   (self-assignment-removal)
discard "nop"


x = y
------------------          (copy)
`=copy`(x, y)


f_sink(g())
-----------------------     (call-to-sink)
f_sink(g())


f_sink(notLastReadOf y)
--------------------------     (copy-to-sink)
(let tmp = `=dup`(y);
f_sink(tmp))


f_sink(lastReadOf y)
-----------------------     (move-to-sink)
f_sink(y)
`=wasMoved`(y)

Object and array construction

Object and array construction is treated as a function call where the
function has sink parameters.

Destructor removal

=wasMoved(x) followed by a =destroy(x) operation cancel each other
out. An implementation is encouraged to exploit this in order to improve
efficiency and code sizes. The current implementation does perform this
optimization.

Self assignments

=sink in combination with =wasMoved can handle self-assignments but
it's subtle.

The simple case of x = x cannot be turned
into =sink(x, x); =wasMoved(x) because that would lose x's value.
The solution is that simple self-assignments that consist of

Symbols: x = x
Field access: x.f = x.f
Array, sequence or string access with indices known at compile-time: x[0] = x[0]

are transformed into an empty statement that does nothing.
The compiler is free to optimize further cases.

The complex case looks like a variant of x = f(x), we consider
x = select(rand() < 0.5, x, y) here:

nim

proc select(cond: bool; a, b: sink string): string =
  if cond:
    result = a # moves a into result
  else:
    result = b # moves b into result

proc main =
  var x = "abc"
  var y = "xyz"
  # possible self-assignment:
  x = select(true, x, y)

Is transformed into:

nim

proc select(cond: bool; a, b: sink string): string =
  try:
    if cond:
      `=sink`(result, a)
      `=wasMoved`(a)
    else:
      `=sink`(result, b)
      `=wasMoved`(b)
  finally:
    `=destroy`(b)
    `=destroy`(a)

proc main =
  var
    x: string
    y: string
  try:
    `=sink`(x, "abc")
    `=sink`(y, "xyz")
    `=sink`(x, select(true,
      let blitTmp = x
      `=wasMoved`(x)
      blitTmp,
      let blitTmp = y
      `=wasMoved`(y)
      blitTmp))
    echo [x]
  finally:
    `=destroy`(y)
    `=destroy`(x)

As can be manually verified, this transformation is correct for
self-assignments.

Lent type

proc p(x: sink T) means that the proc p takes ownership of x.
To eliminate even more creation/copy <-> destruction pairs, a proc's return
type can be annotated as lent T. This is useful for "getter" accessors
that seek to allow an immutable view into a container.

The sink and lent annotations allow us to remove most (if not all)
superfluous copies and destructions.

lent T is like var T a hidden pointer. It is proven by the compiler
that the pointer does not outlive its origin. No destructor call is injected
for expressions of type lent T or of type var T.

nim test

type
  Tree = object
    kids: seq[Tree]

proc construct(kids: sink seq[Tree]): Tree =
  result = Tree(kids: kids)
  # converted into:
  `=sink`(result.kids, kids); `=wasMoved`(kids)
  `=destroy`(kids)

proc `[]`*(x: Tree; i: int): lent Tree =
  result = x.kids[i]
  # borrows from 'x', this is transformed into:
  # result = addr x.kids[i]
  # This means 'lent' is like 'var T' a hidden pointer.
  # Unlike 'var' this hidden pointer cannot be used to mutate the object.

iterator children*(t: Tree): lent Tree =
  for x in t.kids: yield x

proc main =
  # everything turned into moves:
  let t = construct(@[construct(@[]), construct(@[])])
  echo t[0] # accessor does not copy the element!

The cursor pragma

Under the --mm:arc|orc:option: modes Nim's ref type is implemented
via the same runtime "hooks" and thus via reference counting.
This means that cyclic structures cannot be freed
immediately (--mm:orc:option: ships with a cycle collector).
With the cursor pragma one can break up cycles declaratively:

nim

type
  Node = ref object
    left: Node # owning ref
    right {.cursor.}: Node # non-owning ref

But please notice that this is not C++'s weak_ptr, it means the right field is not
involved in the reference counting, it is a raw pointer without runtime checks.

Automatic reference counting also has the disadvantage that it introduces overhead
when iterating over linked structures. The cursor pragma can also be used
to avoid this overhead:

nim

var it {.cursor.} = listRoot
while it != nil:
  use(it)
  it = it.next

In fact, cursor more generally prevents object construction/destruction pairs
and so can also be useful in other contexts. The alternative solution would be to
use raw pointers (ptr) instead which is more cumbersome and also more dangerous
for Nim's evolution: Later on, the compiler can try to prove cursor pragmas
to be safe, but for ptr the compiler has to remain silent about possible
problems.

Cursor inference / copy elision

The current implementation also performs cursor inference. Cursor inference is
a form of copy elision.

To see how and when we can do that, think about this question: In dest = src when
do we really have to materialize the full copy? - Only if dest or src are mutated
afterward. If dest is a local variable that is simple to analyze. And if src is a
location derived from a formal parameter, we also know it is not mutated! In other
words, we do a compile-time copy-on-write analysis.

This means that "borrowed" views can be written naturally and without explicit pointer
indirections:

nim

proc main(tab: Table[string, string]) =
  let v = tab["key"] # inferred as cursor because 'tab' is not mutated.
  # no copy into 'v', no destruction of 'v'.
  use(v)
  useItAgain(v)

Hook lifting

The hooks of a tuple type (A, B, ...) are generated by lifting the
hooks of the involved types A, B, ... to the tuple type. In
other words, a copy x = y is implemented
as x[0] = y[0]; x[1] = y[1]; ..., likewise for =sink and =destroy.

Other value-based compound types like object and array are handled
correspondingly. For object however, the compiler-generated hooks
can be overridden. This can also be important to use an alternative traversal
of the involved data structure that is more efficient or in order to avoid
deep recursions.

Hook generation

The ability to override a hook leads to a phase ordering problem:

nim

type
  Foo[T] = object

proc main =
  var f: Foo[int]
  # error: destructor for 'f' called here before
  # it was seen in this module.

proc `=destroy`[T](f: Foo[T]) =
  discard

The solution is to define proc `=destroy`[T](f: Foo[T]) before
it is used. The compiler generates implicit
hooks for all types in strategic places so that an explicitly provided
hook that comes too "late" can be detected reliably. These strategic places
have been derived from the rewrite rules and are as follows:

In the construct let/var x = ... (var/let binding)
hooks are generated for typeof(x).
In x = ... (assignment) hooks are generated for typeof(x).
In f(...) (function call) hooks are generated for typeof(f(...)).
For every sink parameter x: sink T the hooks are generated
for typeof(x).

nodestroy pragma

The experimental nodestroy:idx: pragma inhibits hook injections. This can be
used to specialize the object traversal in order to avoid deep recursions:

nim test

type Node = ref object
  x, y: int32
  left, right: Node

type Tree = object
  root: Node

proc `=destroy`(t: Tree) {.nodestroy.} =
  # use an explicit stack so that we do not get stack overflows:
  var s: seq[Node] = @[t.root]
  while s.len > 0:
    let x = s.pop
    if x.left != nil: s.add(x.left)
    if x.right != nil: s.add(x.right)
    # free the memory explicitly:
    `=dispose`(x)
  # notice how even the destructor for 's' is not called implicitly
  # anymore thanks to .nodestroy, so we have to call it on our own:
  `=destroy`(s)

As can be seen from the example, this solution is hardly sufficient and
should eventually be replaced by a better solution.

Copy on write

String literals are implemented as "copy on write".
When assigning a string literal to a variable, a copy of the literal won't be created.
Instead the variable simply points to the literal.
The literal is shared between different variables which are pointing to it.
The copy operation is deferred until the first write.

For example:

nim

var x = "abc"  # no copy
var y = x      # no copy
y[0] = 'h'     # copy

The abstraction fails for addr x because whether the address is going to be used for mutations is unknown.
prepareMutation needs to be called before the "address of" operation. For example:

nim

var x = "abc"
var y = x

prepareMutation(y)
moveMem(addr y[0], addr x[0], 3)
assert y == "abc"

File: doc/docgen.md

=================================== Nim DocGen Tools Guide

:Author: Erik O'Leary
:Version: |nimversion|

.. default-role:: code
.. include:: rstcommon.rst
.. contents::

.. importdoc:: markdown_rst.md, compiler/docgen.nim

Introduction

This document describes the documentation generation tools:idx: built into
the Nim compiler, which can generate HTML, Latex and JSON output
from input .nim files and projects.
The output documentation will include the module
dependencies (import), any top-level documentation comments (##), and
exported symbols (*), including procedures, types, and variables.

=================== ==============
command output format
=================== ==============
nim doc:cmd: .html HTML
nim doc2tex:cmd: .tex LaTeX
nim jsondoc:cmd: .json JSON
=================== ==============

Nim can generate HTML and LaTeX from input Markdown and
RST (reStructuredText) files as well, which is intended for writing
standalone documents like user's guides and technical specifications.
See [Nim-flavored Markdown and reStructuredText] document for the description
of this feature and particularly section [Command line usage] for the full
list of supported commands.

Quick start

Generate HTML documentation for a file:

cmd

nim doc <filename>.nim

Generate HTML documentation for a whole project:

cmd

# delete any htmldocs/*.idx file before starting
nim doc --project --index:on --git.url: --git.commit:<tag> --outdir:htmldocs <main_filename>.nim
# this will generate html files, a theindex.html index, css and js under `htmldocs`
# See also `--docroot` to specify a relative root.
# to get search (dochacks.js) to work locally, you need a server otherwise
# CORS will prevent opening file:// urls; this works:
python3 -m http.server 7029 --directory htmldocs
# When --outdir is omitted it defaults to $projectPath/htmldocs,
# or `$nimcache/htmldocs` with `--usenimcache` which avoids clobbering your sources;
# and likewise without `--project`.
# Adding `-r` will open in a browser directly.
# Use `--showNonExports` to show non-exported fields of an exported type.

Documentation Comments

Any comments which are preceded by a double-hash (##), are interpreted as
documentation. Comments are parsed as RST (see reference)), providing
Nim module authors the ability to easily generate richly formatted
documentation with only their well-documented code!
Basic Markdown syntax is also supported inside the doc comments.

Example:

nim

type Person* = object
  ## This type contains a description of a person
  name: string
  age: int

Outputs:

text

Person* = object
  name: string
  age: int

This type contains a description of a person

Field documentation comments can be added to fields like so:

nim

var numValues: int ## \
  ## `numValues` stores the number of values

Note that without the * following the name of the type, the documentation for
this type would not be generated. Documentation will only be generated for
exported types/procedures/etc.

It's recommended to always add exactly one space after ## for readability
of comments — this extra space will be cropped from the parsed comments and
won't influence RST formatting.

.. note:: Generally, this baseline indentation level inside a documentation
comment may not be 1: it can be any since it is determined by the offset
of the first non-whitespace character in the comment.
After that indentation must be consistent on the following lines of
the same comment.
If you still need to add an additional indentation at the very beginning
(for RST block quote syntax) use backslash \ before it:

text

```nim
 ## \
 ##
 ##    Block quote at the first line.
 ##
 ## Paragraph.
 ```

Structuring output directories

Basic directory for output is set by --outdir:OUTDIR:option: switch,
by default OUTDIR is htmldocs sub-directory in the directory of
the processed file.

There are 2 basic options as to how generated HTML output files are stored:

complex hierarchy when docgen-compiling with --project:option:,
which follows directory structure of the project itself.
So nim doc:cmd: replicates project's directory structure
inside --outdir:OUTDIR:option: directory.
--project:option: is well suited for projects that have 1 main module.
File name clashes are impossible in this case.
flattened structure, where user-provided script goes through all
needed input files and calls commands like nim doc:cmd:
with --outdir:OUTDIR:option: switch, thus putting all HTML (and
.idx) files into 1 directory.

.. Important:: Make sure that you don't have files with same base name
like x.nim and x.md in the same package, otherwise you'll
have name conflict for x.html.

.. Tip:: To structure your output directories and avoid file name
clashes you can split your project into
different packages -- parts of your repository that are
docgen-compiled with different --outdir:OUTDIR:option: options.

An example of such strategy is Nim repository itself which has:
- its stdlib .nim files from different directories and .md
  documentation from doc/ directory are all docgen-compiled
  into --outdir:web/upload/<version>/:option: directory
- its .nim files from compiler/ directory are docgen-compiled
  into --outdir:web/upload/<version>/compiler/:option: directory.
  Interestingly, it's compiled with complex hierarchy using
  --project:option: switch.
Contents of web/upload/<version> are then deployed into Nim's
Web server.

This output directory structure allows to work correctly with files like
compiler/docgen.nim (implementation) and doc/docgen.md (user
documentation) in 1 repository.

Index files

Index (.idx) files are used for 2 different purposes:

easy cross-referencing between different .nim and/or .md / .rst
files described in [Nim external referencing]
creating a whole-project index for searching of symbols and keywords,
see [Buildindex command].

Document Types

Example of Nim file input

The following examples will generate documentation for this sample
Nim module, aptly named doc/docgen_sample.nim:

nim file=docgen_sample.nim

All the below commands save their output to htmldocs directory relative to
the directory of file;
hence the output for this sample will be in doc/htmldocs.

HTML

The generation of HTML documents is done via the doc:option: command. This command
takes either a single .nim file, outputting a single .html file with the same
base filename, or multiple .nim files, outputting multiple .html files and,
optionally, an index file.

The doc:option: command:

cmd

nim doc docgen_sample.nim

Partial Output:

text

...
proc helloWorld(times: int) {.raises: [], tags: [].}
...

The full output can be seen here: docgen_sample.html.
It runs after semantic checking and includes pragmas attached implicitly by the
compiler.

LaTeX

LaTeX files are intended to be converted to PDF, especially for offline
reading or making hard copies. (LaTeX output is oftentimes better than
HTML -> PDF conversion).

The doc2tex:option: command:

cmd

nim doc2tex docgen_sample.nim
cd htmldocs
xelatex docgen_sample.tex
xelatex docgen_sample.tex
# It is usually necessary to run `xelatex` 2 times (or even 3 times for
# large documents) to get all labels generated.
# That depends on this warning in the end of `xelatex` output:
#   LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.

The output is docgen_sample.pdf.

JSON

The generation of JSON documents is done via the jsondoc:option: command.
This command takes in a .nim file and outputs a .json file with
the same base filename.
Note that this tool is built off of the doc:option: command
(previously doc2:option:), and contains the same information.

The jsondoc:option: command:

cmd

nim jsondoc docgen_sample.nim

Output:

text

{
  "orig": "docgen_sample.nim",
  "nimble": "",
  "moduleDescription": "This module is a sample",
  "entries": [
    {
      "name": "helloWorld",
      "type": "skProc",
      "line": 5,
      "col": 0,
      "description": "Takes an integer and outputs as many &quot;hello world!&quot;s",
      "code": "proc helloWorld(times: int) {.raises: [], tags: [].}"
    }
  ]
}

Similarly to the old doc:option: command, the old jsondoc:option: command has been
renamed to jsondoc0:option:.

The jsondoc0:option: command:

cmd

nim jsondoc0 docgen_sample.nim

Output:

text

[
  {
    "comment": "This module is a sample."
  },
  {
    "name": "helloWorld",
    "type": "skProc",
    "description": "Takes an integer and outputs as many &quot;hello world!&quot;s",
    "code": "proc helloWorld*(times: int)"
  }
]

Note that the jsondoc:option: command outputs its JSON without pretty-printing it,
while jsondoc0:option: outputs pretty-printed JSON.

Simple documentation links

It's possible to use normal Markdown/RST syntax to manually
reference Nim symbols using HTML anchors, however Nim has an automatic
facility that makes referencing inside .nim and .md/.rst files and
between them easy and seamless.
The point is that such links will be resolved automatically
by nim doc:cmd: (or md2html:option:, or jsondoc:option:,
or doc2tex:option:, ...). And, unlike manual links, such automatic
links check that their target exists -- a warning is emitted for
any broken link, so you avoid broken links in your project.

Nim treats both .md/.rst files and .nim modules (their doc comment
part) as documents uniformly.
Hence all directions of referencing are equally possible having the same syntax:

.md/rst -> itself (internal). See [Markup local referencing].
.md/rst -> external .md/rst. See [Markup external referencing].
To summarize, referencing in .md/.rst files was already described in
[Nim-flavored Markdown and reStructuredText]
(particularly it described usage of index files for referencing),
while in this document we focus on Nim-specific details.
.md/rst -> external .nim. See [Nim external referencing].
.nim -> itself (internal). See [Nim local referencing].
.nim -> external .md/rst. See [Markup external referencing].
.nim -> external .nim. See [Nim external referencing].

To put it shortly, local referencing always works out of the box,
external referencing requires to use .. importdoc:: <file>
directive to import file and to ensure that the corresponding
.idx file was generated.

Syntax for referencing is basically the same as for normal markup.
Recall from [Referencing] that our parser supports two equivalent syntaxes
for referencing, Markdown and RST one.
So to reference proc f one should use something like that,
depending on markup type:

text

Markdown                    RST

Ref. [proc f] or [f]        Ref. `proc f`_ or just f_ for a one-word case

Nim local referencing

You can reference Nim identifiers from Nim documentation comments
inside their .nim file (or inside a .rst file included from
a .nim).
This pertains to any exported symbol like proc, const, iterator, etc.
Link text is either one word or a group of words enclosed by delimiters
(brackets [...] for Markdown or backticks \...`_` for RST).
Link text will be displayed as is while link target will be set to
the anchor [^1] of Nim symbol that corresponds to link text.

[^1] anchors' format is described in [HTML anchor generation] section below.

If you have a constant:

nim

const pi* = 3.14

then it should be referenced in one of the 2 forms:

A. non-qualified (no symbol kind specification):

text

pi_

B. qualified (with symbol kind specification):

text

`const pi`_

For routine kinds there are more options. Consider this definition:

nim

proc foo*(a: int, b: float): string

Generally following syntax is allowed for referencing foo:

short (without parameters):

A. non-qualified:
text
```
foo_
```
B. qualified:
text
```
`proc foo`_
```
longer variants (with parameters):

A. non-qualified:
1. specifying parameters names:
  text
```
`foo(a, b)`_
```
2. specifying parameters types:
  text
```
`foo(int, float)`_
```
3. specifying both names and types:
  text
```
`foo(a: int, b: float)`_
```
4. output parameter can also be specified if you wish:
  text
```
`foo(a: int, b: float): string`_
```
B. qualified: all 4 options above are valid.
Particularly you can use the full format:
text
```
`proc foo(a: int, b: float): string`_
```

.. Tip:: Avoid cluttering your text with extraneous information by using
one of shorter forms:

text

binarySearch_
   `binarySearch(a, key, cmp)`_

Brevity is better for reading! If you use a short form and have an
ambiguity problem (see below) then just add some additional info.

Symbol kind like proc can also be specified in the postfix form:

text

`foo proc`_
`walkDir(d: string) iterator`_

.. Warning:: An ambiguity in resolving documentation links may arise because of:

clash with other RST anchors
- manually setup anchors
- automatically set up, e.g. section names
collision with other Nim symbols:
- routines with different parameters can exist e.g. for
  proc and template. In this case they are split between their
  corresponding sections in output file. Qualified references are
  useful in this case -- just disambiguate by referring to these
  sections explicitly:
  text
```
See `foo proc`_ and `foo template`_.
```
- because in Nim proc and iterator belong to different namespaces,
  so there can be a collision even if parameters are the same.
  Use \proc foo`_:literal: or `iterator foo`_`:literal: then.

Any ambiguity is always reported with Nim compiler warnings and an anchor
with higher priority is selected. Manual anchors have highest
priority, then go automatic RST anchors; then Nim-generated anchors
(while procs have higher priority than other Nim symbol kinds).

Generic parameters can also be used. All in all, this long form will be
recognized fine:

text

`proc binarySearch*[T; K](a: openArray[T], key: K, cmp: proc(T, K)): int`_

Limitations:

The parameters of a nested routine type can be specified only with types
(without parameter names, see form A.2 above).
E.g. for this signature:

nim

proc binarySearch*[T, K](a: openArray[T]; key: K;
                         cmp: proc (x: T; y: K): int {.closure.}): int
                                    ~~    ~~   ~~~~~

you cannot use names underlined by ~~ so it must be referenced with
cmp: proc(T, K). Hence these forms are valid:

text

`binarySearch(a: openArray[T], key: K, cmp: proc(T, K))`_
`binarySearch(openArray[T], K, proc(T, K))`_
`binarySearch(a, key, cmp)`_

Default values in routine parameters are not recognized, one needs to
specify the type and/or name instead. E.g. for referencing proc f(x = 7)
use one of the mentioned forms:
text
```
`f(int)`_ or `f(x)`_ or `f(x: int)`_.
```
Generic parameters must be given the same way as in the
definition of referenced symbol.
- their names should be the same
- parameters list should be given the same way, e.g. without substitutions
  between commas (,) and semicolons (;).

.. Note:: A bit special case is operators
(as their signature is also defined with ```):

text

```Nim
  func `$`(x: MyType): string
  func `[]`*[T](x: openArray[T]): T
  ```

A short form works without additional backticks:

text

`$`_
   `[]`_

However for fully-qualified reference copy-pasting backticks (`) into other
backticks will not work in our RST parser (because we use Markdown-like
inline markup rules). You need either to delete backticks or keep
them and escape with backslash \:

text

no backticks: `func $`_
   escaped:  `func \`$\``_
   no backticks: `func [][T](x: openArray[T]): T`_
   escaped:  `func \`[]\`[T](x: openArray[T]): T`_

.. Note:: Types that defined as enum, or object, or tuple can also be
referenced with those names directly (instead of type):

text

type CopyFlag = enum
     ...
   ## Ref. `CopyFlag enum`_

Nim external referencing

Just like for [Markup external referencing], which saves markup anchors,
the Nim symbols are also saved in .idx files, so one needs
to generate them beforehand, and they should be loaded by
an .. importdoc:: directive. Arguments to .. importdoc:: is a
comma-separated list of Nim modules or Markdown/RST documents.

--index:only:option: tells Nim to only generate .idx file and
do not attempt to generate HTML/LaTeX output.
For .nim modules there are 2 alternatives to work with .idx files:

using [Project switch] implies generation of .idx files,
however, if importdoc is called on upper modules as its arguments,
their .idx are not yet created. Thus one should generate all
required .idx first:
cmd
```
nim doc --project --index:only <main>.nim
nim doc --project <main>.nim
```
or run nim doc --index:only <module.nim>:cmd: command for all (used)
Nim modules in your project. Then run nim doc <module.nim> on them for
output HTML generation.

.. Warning:: A mere nim doc --index:on:cmd: may fail on an attempt to do
importdoc from another module (for which .idx was not yet
generated), that's why --index:only:option: shall be used instead.

For .md/.rst markup documents point 2 is the only option.

Then, you can freely use something like this in your_module.nim:

nim

## .. importdoc::  user_manual.md, another_module.nim

...
## Ref. [some section from User Manual].

...
## Ref. [proc f]
## (assuming you have a proc `f` in ``another_module``).

and compile it by nim doc:cmd:. Note that link text will
be automatically prefixed by the module name of symbol,
so you will see something like "Ref. another_module: proc f"
in the generated output.

It's also possible to reference a whole module by prefixing or
suffixing full canonical module name with "module":

text

Ref. [module subdir/name] or [subdir/name module].

Markup documents as a whole can be referenced just by their title
(or by their file name if the title was not set) without any prefix.

.. Tip:: During development process the stage of .idx files generation
can be done only once, after that you use already generated .idx
files while working with a document being developed (unless you do
incompatible changes to referenced documents).

.. Hint:: After changing a referenced document file one may need
to regenerate its corresponding .idx file to get correct results.
Of course, when referencing internally inside any given .nim file,
it's not needed, one can even immediately use any freshly added anchor
(a document's own .idx file is not used for resolving its internal links).

If an importdoc directive fails to find a .idx, then an error
is emitted.

In case of such compilation failures please note that:

all relative paths, given to importdoc, relate to insides of
OUTDIR, and not project's directory structure.
importdoc searches for .idx in --outdir:OUTDIR:option: directory
(htmldocs by default) and not around original modules, so:

.. Tip:: look into OUTDIR to understand what's going on.
also keep in mind that .html and .idx files should always be
output to the same directory, so check this and, if it's not true, check
that both runs with and without --index:only:option: have all
other options the same.

To summarize, for 2 basic options of [Structuring output directories]
compilation options are different:

complex hierarchy with --project:option: switch.

As the original project's directory structure is replicated in
OUTDIR, all passed paths are related to this structure also.

E.g. if a module path1/module.nim does
.. importdoc:: path2/another.nim then docgen tries to load file
OUTDIR/path1/path2/another.idx.

.. Note:: markup documents are just placed into the specified directory
OUTDIR:option: by default (i.e. they are not affected by
--project:option:), so if you have PROJECT/doc/manual.md
document and want to use complex hierarchy (with doc/),
compile it with --docroot:option::
cmd # 1st stage nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ --index:only PROJECT/doc/manual.md ... # 2nd stage nim md2html --outdir:OUTDIR --docroot:/absolute/path/to/PROJECT \ PROJECT/doc/manual.md

Then the output file will be placed as OUTDIR/doc/manual.idx.
So if you have PROJECT/path1/module.nim, then manual.md can
be referenced as ../doc/manual.md.
flattened structure.

E.g. if a module path1/module.nim does
.. importdoc:: path2/another.nim then docgen tries to load
OUTDIR/path2/another.idx, so the path path1
does not matter and providing path2 can be useful only
in the case it contains another package that was placed there
using --outdir:OUTDIR/path2:option:.

The links' text will be prefixed as another: ... in both cases.

.. Warning:: Again, the same --outdir:OUTDIR:option: option should
be provided to both doc --index:only:option: /
md2html --index:only:option: and final generation by
doc:option:/md2html:option: inside 1 package.

To temporarily disable importdoc, e.g. if you don't need
correct link resolution at the moment, use a --noImportdoc:option: switch
(only warnings about unresolved links will be generated for external references).

Project switch

cmd

nim doc --project filename.nim

This will recursively generate documentation of all Nim modules imported
into the input module that belong to the Nimble package that filename.nim
belongs to. The index files and the corresponding theindex.html will
also be generated.

Index switch

cmd

nim doc --index:on filename.nim

This will generate an index of all the exported symbols in the input Nim
module, and put it into a neighboring file with the extension of .idx. The
index file is line-oriented (newlines have to be escaped). Each line
represents a tab-separated record of several columns, the first two mandatory,
the rest optional. See the [Index (idx) file format] section for details.

.. Note:: --index:option: switch only affects creation of .idx
index files, while user-searchable Index HTML file is created by
buildIndex:option: command.

Buildindex command

Once index files have been generated for one or more modules, the Nim
compiler command nim buildIndex directory:cmd: can be run to go over all the index
files in the specified directory to generate a theindex.html
file:

cmd

nim buildIndex -o:path/to/htmldocs/theindex.html path/to/htmldocs

See source switch

cmd

nim doc --git.url: filename.nim

With the git.url:option: switch the See source hyperlink will appear below each
documented item in your source code pointing to the implementation of that
item on a GitHub repository.
You can click the link to see the implementation of the item.

The git.commit:option: switch overrides the hardcoded devel branch in
config/nimdoc.cfg.
This is useful to link to a different branch e.g. --git.commit:master:option:,
or to a tag e.g. --git.commit:1.2.3:option: or a commit.

Source URLs are generated as href="${url}/tree/${commit}/${path}#L${line}"
by default and thus compatible with GitHub but not with GitLab.

Similarly, git.devel:option: switch overrides the hardcoded devel branch
for the Edit link which is also useful if you have a different working
branch than devel e.g. --git.devel:master:option:.

Edit URLs are generated as href="${url}/tree/${devel}/${path}#L${line}"
by default.

You can edit config/nimdoc.cfg and modify the doc.item.seesrc value
with a hyperlink to your own code repository.

In the case of Nim's own documentation, the commit value is just a commit
hash to append to a formatted URL to https://github.com/nim-lang/Nim.

Substitution via environment variables

A simple substitution using environment variables is available.
A reference written as |name| is replaced during documentation generation if
a matching variable is provided. You can define it via the compiler with
--putenv. This is useful for injecting values like version strings or
build-specific text.

nim

## |foo|

cmd

nim --putenv:foo=bar doc filename.nim

The generated html will contain bar instead of foo.

Other Input Formats

The Nim compiler also has support for RST (reStructuredText) files with
the rst2html:option: and rst2tex:option: commands. Documents like this one are
initially written in a dialect of RST which adds support for Nim source code
highlighting with the .. code-block:: nim prefix. code-block also
supports highlighting of a few other languages supported by the
packages/docutils/highlite module.

See Markdown and RST markup languages for
usage of those commands.

HTML anchor generation

When you run the rst2html:option: command, all sections in the RST document will
get an anchor you can hyperlink to. Usually, you can guess the anchor lower
casing the section title and replacing spaces with dashes, and in any case, you
can get it from the table of contents. But when you run the doc:option:
command to generate API documentation, some symbol get one or two anchors at
the same time: a numerical identifier, or a plain name plus a complex name.

The numerical identifier is just a random number. The number gets assigned
according to the section and position of the symbol in the file being processed
and you should not rely on it being constant: if you add or remove a symbol the
numbers may shuffle around.

The plain name of a symbol is a simplified version of its fully exported
signature. Variables or constants have the same plain name symbol as their
complex name. The plain name for procs, templates, and other callable types
will be their unquoted value after removing parameters, return types, and
pragmas. The plain name allows short and nice linking of symbols that works
unless you have a module with collisions due to overloading.

If you hyperlink a plain name symbol and there are other matches on the same
HTML file, most browsers will go to the first one. To differentiate the rest,
you will need to use the complex name. A complex name for a callable type is
made up of several parts:

(plain symbol)(.type),(first param)?(,param type)*

The first thing to note is that all callable types have at least a comma, even
if they don't have any parameters. If there are parameters, they are
represented by their types and will be comma-separated. To the plain symbol a
suffix may be added depending on the type of the callable:

============== ==============
Callable type Suffix
============== ==============
proc, func empty string
macro .m
method .e
iterator .i
template .t
converter .c
============== ==============

The relationship of type to suffix is made by the proc complexName in the
compiler/docgen.nim file. Here are some examples of complex names for
symbols in the system module.

type SomeSignedInt = int | int8 | int16 | int32 | int64 =>
#SomeSignedInt
var globalRaiseHook: proc (e: ref E_Base): bool {.nimcall.} =>
#globalRaiseHook
const NimVersion = "0.0.0" =>
#NimVersion
proc getTotalMem(): int {.rtl, raises: [], tags: [].} =>
#getTotalMem
proc len[T](x: seq[T]): int {.magic: "LengthSeq", noSideEffect.} =>
#len,seq[T]
iterator pairs[T](a: seq[T]): tuple[key: int, val: T] {.inline.} =>
#pairs.i,seq[T]
template newException[](exceptn: typedesc; message: string; parentException: ref Exception = nil): untyped =>
#newException.t,typedesc,string,ref.Exception

Index (idx) file format

Files with the .idx extension are generated when you use the [Index
switch] along with commands to generate
documentation from source or text files. You can programmatically generate
indices with the setIndexTerm()
and writeIndexFile() <rstgen.html#writeIndexFile,RstGenerator,string>_ procs.
The purpose of idx files is to hold the interesting symbols and their HTML
references so they can be later concatenated into a big index file with
mergeIndexes(). This section documents
the file format in detail.

Index files are line-oriented and tab-separated (newline and tab characters
have to be escaped). Each line represents a record with 6 fields.
The content of these columns is:

Discriminator tag denoting type of the index entry, allowed values are:
markupTitle
: a title for .md/.rst document
nimTitle
: a title of .nim module
heading
: heading of sections, can be both in Nim and markup files
idx
: terms marked with :idx: role
nim
: a Nim symbol
nimgrp
: a Nim group for overloadable symbols like procs
Mandatory term being indexed. Terms can include quoting according to
Nim's rules (e.g. `^`).
Base filename plus anchor hyperlink (e.g. algorithm.html#*,int,SortOrder).
Optional human-readable string to display as a hyperlink. If the value is not
present or is the empty string, the hyperlink will be rendered
using the term. Prefix whitespace indicates that this entry is
not for an API symbol but for a TOC entry.
Optional title or description of the hyperlink. Browsers usually display
this as a tooltip after hovering a moment over the hyperlink.
A line number of file where the entry was defined.

The index generation tools differentiate between documentation
generated from .nim files and documentation generated from .md or
.rst files by tag nimTitle or markupTitle in the 1st line of
the .idx file.

.. TODO Normal symbols are added to the index with surrounding whitespaces removed. An
exception to this are the table of content (TOC) entries. TOC entries are added to
the index file with their third column having as much prefix spaces as their
level is in the TOC (at least 1 character). The prefix whitespace helps to
filter TOC entries from API or text symbols. This is important because the
amount of spaces is used to replicate the hierarchy for document TOCs in the
final index, and TOC entries found in .nim files are discarded.

Additional resources

Nim Compiler User Guide
already mentioned documentation for
Markdown and RST markup languages, which also
contains the list of implemented features of these markup languages.
the implementation is in [module compiler/docgen].

The output for HTML and LaTeX comes from the config/nimdoc.cfg and
config/nimdoc.tex.cfg configuration files. You can add and modify these
files to your project to change the look of the docgen output.

You can import the packages/docutils/rstgen module in your
programs if you want to reuse the compiler's documentation generation procs.

File: doc/docs.md

The documentation consists of several documents:

| Tutorial (part I)
| The Nim tutorial part one deals with the basics.
| Tutorial (part II)
| The Nim tutorial part two deals with the advanced language constructs.
| Tutorial (part III)
| The Nim tutorial part three about Nim's macro system.
| Language Manual
| The Nim manual is a draft that will evolve into a proper specification.
| Library documentation
| This document describes Nim's standard library.
| Compiler user guide
| The user guide lists command line arguments, special features of the
compiler, etc.
| Tools documentation
| Description of some tools that come with the standard distribution.
| Memory management
| Additional documentation about Nim's memory management strategies
| and how to operate them in a realtime setting.
| Source code filters
| The Nim compiler supports source code filters as a simple yet powerful
builtin templating system.
| Internal documentation
| The internal documentation describes how the compiler is implemented. Read
this if you want to hack the compiler.
| Index
| The generated index. Index + (Ctrl+F) == Joy

File: README.md Copy Chapter

Nim

Community Copy Chapter

Compiling Copy Chapter

Koch Copy Chapter

Nimble Copy Chapter

Contributors Copy Chapter

Contributing Copy Chapter

Backers Copy Chapter

Sponsors Copy Chapter

License Copy Chapter

File: Readme.md Copy Chapter

Nim

Community Copy Chapter

Compiling Copy Chapter

Koch Copy Chapter

Nimble Copy Chapter

Contributors Copy Chapter

Contributing Copy Chapter

Backers Copy Chapter

Sponsors Copy Chapter

License Copy Chapter

File: readme.md Copy Chapter

Nim

Community Copy Chapter

Compiling Copy Chapter

Koch Copy Chapter

Nimble Copy Chapter

Contributors Copy Chapter

Contributing Copy Chapter

Backers Copy Chapter

Sponsors Copy Chapter

License Copy Chapter

File: changelogs/readme.md Copy Chapter

Update changelog

Recent changelogs are saved here (with their git history) Copy Chapter

Older changelogs are stored in https://github.com/nim-lang/website Copy Chapter

source files: Copy Chapter

urls: Copy Chapter

File: compiler/readme.md Copy Chapter

Nim Compiler Copy Chapter

File: doc/manual/var_t_return.md Copy Chapter

File: doc/apis.md Copy Chapter

================= API naming design

Naming scheme

File: doc/contributing.md Copy Chapter

============ Contributing

Writing tests

Stdlib Copy Chapter

Compiler Copy Chapter

Running tests

Comparing tests

Deprecation

Documentation

Best practices

The `git`:cmd: stuff

General commit rules Copy Chapter

Continuous Integration (CI) Copy Chapter

Debugging CI failures, flaky tests, etc Copy Chapter

Code reviews Copy Chapter

Evolving the stdlib

What the compiler itself needs must be part of the stdlib Copy Chapter

Vocabulary types must be part of the stdlib Copy Chapter

Existing, battle-tested modules stay Copy Chapter

Syntactic helpers can start as experimental stdlib modules Copy Chapter

Other new stdlib modules do not start as stdlib modules Copy Chapter

Little additions are acceptable Copy Chapter

Conventions Copy Chapter

Breaking Changes

Run-time breaking changes Copy Chapter

Compile-time breaking changes Copy Chapter

Compiler/language spec bugfixes Copy Chapter

Non-breaking changes Copy Chapter

File: doc/destructors.md Copy Chapter

================================== Nim Destructors and Move Semantics

About this document

Motivating example

Lifetime-tracking hooks

`=destroy` hook Copy Chapter

`=wasMoved` hook Copy Chapter

File: README.md

Community

Compiling

Koch

Nimble

Contributors

Contributing

Backers

Sponsors

License

File: Readme.md

Community

Compiling

Koch

Nimble

Contributors

Contributing

Backers

Sponsors

License

File: readme.md

Community

Compiling

Koch

Nimble

Contributors

Contributing

Backers

Sponsors

License

File: changelogs/readme.md

Recent changelogs are saved here (with their git history)

Older changelogs are stored in https://github.com/nim-lang/website

source files:

urls:

File: compiler/readme.md

Nim Compiler

File: doc/manual/var_t_return.md

File: doc/apis.md

File: doc/contributing.md

Stdlib

Compiler

General commit rules

Continuous Integration (CI)

Debugging CI failures, flaky tests, etc

Code reviews

What the compiler itself needs must be part of the stdlib

Vocabulary types must be part of the stdlib

Existing, battle-tested modules stay

Syntactic helpers can start as experimental stdlib modules

Other new stdlib modules do not start as stdlib modules

Little additions are acceptable

Conventions

Run-time breaking changes

Compile-time breaking changes

Compiler/language spec bugfixes

Non-breaking changes

File: doc/destructors.md

`=destroy` hook

`=wasMoved` hook

`=sink` hook

`=copy` hook

`=trace` hook

`=dup` hook

File: doc/docgen.md

Quick start

Documentation Comments

Structuring output directories

Index files

Example of Nim file input

HTML

LaTeX

JSON

Nim local referencing

Nim external referencing

Project switch

Index switch

Buildindex command

See source switch

Substitution via environment variables