## 1. Project Overview & Quickstart (XAMPPRocky/tokei) ## File: README.md # Tokei ([時計](https://en.wiktionary.org/wiki/%E6%99%82%E8%A8%88)) [](https://github.com/XAMPPRocky/tokei/actions?query=workflow%3A%22Mean+Bean+CI%22) [](https://github.com/XAMPPRocky/tokei/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) [](https://docs.rs/tokei/) ) [](https://deps.rs/repo/github/XAMPPRocky/tokei) [](https://repology.org/project/tokei/versions) Tokei is a program that displays statistics about your code. Tokei will show the number of files, total lines within those files and code, comments, and blanks grouped by language. ## Example ```console ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Language Files Lines Code Comments Blanks ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BASH 4 49 30 10 9 JSON 1 1332 1332 0 0 Shell 1 49 38 1 10 TOML 2 77 64 4 9 ─────────────────────────────────────────────────────────────────────────────── Markdown 5 1355 0 1074 281 |- JSON 1 41 41 0 0 |- Rust 2 53 42 6 5 |- Shell 1 22 18 0 4 (Total) 1471 101 1080 290 ─────────────────────────────────────────────────────────────────────────────── Rust 19 3416 2840 116 460 |- Markdown 12 351 5 295 51 (Total) 3767 2845 411 511 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Total 32 6745 4410 1506 829 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` ## [API Documentation](https://docs.rs/tokei) ## Table of Contents - [Features](#features) - [Installation](#installation) - [Package Managers](#package-managers) - [Manual](#manual) - [Configuration](#configuration) - [How to use Tokei](#how-to-use-tokei) - [Options](#options) - [Supported Languages](#supported-languages) - [Changelog](CHANGELOG.md) - [Common Issues](#common-issues) - [Canonical Source](#canonical-source) - [Copyright and License](#copyright-and-license) ## Features - Tokei is **very fast**, and is able to count millions of lines of code in seconds. Check out the [11.0.0 release](https://github.com/XAMPPRocky/tokei/releases/v11.0.0) to see how Tokei's speed compares to others. - Tokei is **accurate**, Tokei correctly handles multi line comments, nested comments, and not counting comments that are in strings. Providing an accurate code statistics. - Tokei has huge range of languages, supporting over **150** languages, and their various extensions. - Tokei can output in multiple formats (**CBOR**, **JSON**, **YAML**) allowing Tokei's output to be easily stored, and reused. These can also be reused in tokei combining a previous run's statistics with another set. - Tokei is available on **Mac**, **Linux**, and **Windows**. See [installation instructions](#installation) for how to get Tokei on your platform. - Tokei is also a **library** allowing you to easily integrate it with other projects. - Tokei comes with and without color. Set the env variable NO_COLOR to 1, and it'll be black and white. ## Installation ### Package Managers #### Unix ```console # Alpine Linux (since 3.13) apk add tokei # Arch Linux pacman -S tokei # Cargo cargo install tokei # Conda conda install -c conda-forge tokei # Fedora sudo dnf install tokei # FreeBSD pkg install tokei # NetBSD pkgin install tokei # Nix/NixOS nix-env -i tokei # OpenSUSE sudo zypper install tokei # Void Linux sudo xbps-install tokei ``` #### macOS ```console # Homebrew brew install tokei # MacPorts sudo port selfupdate sudo port install tokei ``` #### Windows ```console # Winget winget install XAMPPRocky.tokei # Scoop scoop install tokei ``` ### Manual #### Downloading You can download prebuilt binaries in the [releases section](https://github.com/XAMPPRocky/tokei/releases). #### Building You can also build and install from source (requires the latest stable [Rust] compiler.) ```console cargo install --git https://github.com/XAMPPRocky/tokei.git tokei ``` [rust]: https://www.rust-lang.org ## Configuration Tokei has a [configuration] file that allows you to change default behaviour. The file can be named `tokei.toml` or `.tokeirc`. Currently tokei looks for this file in three different places. The current directory, your home directory, and your configuration directory. ## How to use Tokei #### Basic usage This is the basic way to use tokei. Which will report on the code in `./foo` and all subfolders. ```shell $ tokei ./foo ``` [configuration]: ./tokei.example.toml #### Multiple folders To have tokei report on multiple folders in the same call, simply add all the folders you'd like tokei to examine. ```shell $ tokei ./foo ./bar ./baz ``` #### Excluding folders Tokei will respect all `.gitignore` and `.ignore` files, and you can use the `--exclude` option to exclude any additional files. The `--exclude` flag has the same semantics as `.gitignore`. ```shell $ tokei ./foo --exclude *.rs ``` Paths to exclude can also be listed in a `.tokeignore` file, using the same [syntax](https://git-scm.com/docs/gitignore) as .gitignore files. #### Sorting output By default tokei sorts alphabetically by language name, however using `--sort` tokei can also sort by any of the columns. `blanks, code, comments, lines` ```shell $ tokei ./foo --sort code ``` #### Outputting file statistics By default tokei only outputs the total of the languages, and using `--files` flag tokei can also output individual file statistics. ```shell $ tokei ./foo --files ``` #### Outputting into different formats Tokei normally outputs into a nice human readable format designed for terminals. There is also using the `--output` option various other formats that are more useful for bringing the data into another program. **Note:** This version of tokei was compiled without any serialization formats, to enable serialization, reinstall tokei with the features flag. ```shell ALL: cargo install tokei --features all CBOR: cargo install tokei --features cbor YAML: cargo install tokei --features yaml ``` **Currently supported formats** - JSON `--output json` - YAML `--output yaml` - CBOR `--output cbor` ```shell $ tokei ./foo --output json ``` #### Reading in stored formats Tokei can also take in the outputted formats added in the previous results to its current run. Tokei can take either a path to a file, the format passed in as a value to the option, or from stdin. ```shell $ tokei ./foo --input ./stats.json ``` ## Options ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Dockerized version Tokei is available in a small `alpine`-based docker image, buildable through [earthly](https://github.com/earthly/earthly): ```bash earthly +docker ``` Once built, one can run the image with: ```bash docker run --rm -v /path/to/analyze:/src tokei . ``` Or, to simply analyze the current folder (linux): ```bash docker run --rm -v $(pwd):/src tokei . ``` ## Supported Languages If there is a language that you would to add to tokei feel free to make a pull request. Languages are defined in [`languages.json`](./languages.json), and you can read how to add and test your language in our [CONTRIBUTING.md](./CONTRIBUTING.md). ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Common issues ### Tokei says I have a lot of D code, but I know there is no D code! This is likely due to `gcc` generating `.d` files. Until the D people decide on a different file extension, you can always exclude `.d` files using the `-e --exclude` flag like so ``` $ tokei . -e *.d ``` ## Canonical Source The canonical source of this repo is hosted on [GitHub](https://github.com/XAMPPRocky/tokei). If you have a GitHub account, please make your issues, and pull requests there. ## Related Tools - [tokei-pie](https://github.com/laixintao/tokei-pie): Render tokei's output to interactive sunburst chart. ## Copyright and License (C) Copyright 2015 by XAMPPRocky and contributors See [the graph](https://github.com/XAMPPRocky/tokei/graphs/contributors) for a full list of contributors. Tokei is distributed under the terms of both the MIT license and the Apache License (Version 2.0). See [LICENCE-APACHE](./LICENCE-APACHE), [LICENCE-MIT](./LICENCE-MIT) for more information. --- ## File: fuzz/README.md ## Fuzzing Tokei Tokei can be fuzzed using libFuzzer, via [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz/). First install cargo-fuzz: `cargo install cargo-fuzz`. To launch a fuzzing job: `cargo +nightly fuzz run ` - it will run until you kill it with ctrl-c. To use multiple cores: `cargo +nightly fuzz run --jobs=6` To speed things up (at the expense of missing bugs that only manifest in larger files): `cargo +nightly fuzz run -- -max_len=200` Available fuzz targets: - `parse_from_slice_panic` - checks that all of the LanguageType instances' `parse_from_slice` function doesn't panic. - `parse_from_slice_total` - checks that the language stats pass a basic test of reporting no more total lines than there are new lines in the file. At the time of writing there are low-hanging bugs here. With the two `parse_from_slice` fuzz targets, it makes sense to share a common corpus directory as they have identical input formats, e.g.: `cargo +nightly fuzz run parse_from_slice_{panic,total} fuzz/corpus/common` Potential improvements: - Build the fuzz harnesses in CI, so they don't rot. - Do some coverage analysis to check if we're missing any code we would benefit from fuzzing (once it's [integrated into cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz/pull/248)) - Tighten the `parse_from_slice_total` fuzz target to check the total lines exactly matches the number of lines in the file. Only once any bugs found with the current fuzzer are fixed. - Check in a minimized corpus, and run regression over it in CI. ## 2. Official Technical Reference & Guides (XAMPPRocky/docs) ## GitHub Docs This repository contains the documentation website code and Markdown source files for [docs.github.com](https://docs.github.com). GitHub's Docs team works on pre-production content in a private repo that regularly syncs with this public repo. In this article: - [Contributing](#contributing) - [READMEs](#readmes) - [License](#license) ## Contributing ### Start contributing right now: We accept a lot of [different contributions](CONTRIBUTING.md/#types-of-contributions-memo), including some that don't require you to write a single line of code. #### Click **make a contribution** from docs As you're using the GitHub Docs, you may find something in an article that you'd like to add to, update, or change. Click on **make a contribution** to navigate directly to that article in the codebase, so that you can begin making your contribution. #### Open an issue If you've found a problem, you can open an issue using a [template](https://github.com/github/docs/issues/new/choose). #### Solve an issue If you have a solution to one of the open issues, you will need to fork the repository and submit a pull request using the [template](https://github.com/github/docs/blob/main/CONTRIBUTING.md#pull-request-template) that is visible automatically in the pull request body. For more details about this process, please check out [Getting Started with Contributing](/CONTRIBUTING.md). #### Join us in discussions We use GitHub Discussions to talk about all sorts of topics related to documentation and this site. For example: if you'd like help troubleshooting a PR, have a great new idea, or want to share something amazing you've learned in our docs, join us in [discussions](https://github.com/github/docs/discussions). #### And that's it! That's how you can get started easily as a member of the GitHub Documentation community. :sparkles: If you want to know more, or you're making a more complex contribution, check out [Getting Started with Contributing](/CONTRIBUTING.md). There are a few more things to know when you're getting started with this repo: 1. If you're having trouble with your GitHub account, contact [Support](https://support.github.com/contact). 2. We do not accept pull requests for translated content - see [CONTRIBUTING.md](/CONTRIBUTING.md) for more information. ## READMEs In addition to the README you're reading right now, this repo includes other READMEs that describe the purpose of each subdirectory in more detail: - [content/README.md](content/README.md) - [content/graphql/README.md](content/graphql/README.md) - [content/rest/README.md](content/rest/README.md) - [contributing/README.md](contributing/README.md) - [data/README.md](data/README.md) - [data/reusables/README.md](data/reusables/README.md) - [data/variables/README.md](data/variables/README.md) - [includes/liquid-tags/README.md](includes/liquid-tags/README.md) - [includes/README.md](includes/README.md) - [javascripts/README.md](javascripts/README.md) - [layouts/README.md](layouts/README.md) - [lib/liquid-tags/README.md](lib/liquid-tags/README.md) - [middleware/README.md](middleware/README.md) - [script/README.md](script/README.md) - [stylesheets/README.md](stylesheets/README.md) - [tests/README.md](tests/README.md) ## License The GitHub product documentation in the assets, content, and data folders are licensed under a [CC-BY license](LICENSE). All other code in this repository is licensed under a [MIT license](LICENSE-CODE). When using the GitHub logos, be sure to follow the [GitHub logo guidelines](https://github.com/logos).