Repository: ggml-org/ggml

Stars: 14459

README.md Copy Chapter

ggml

Manifesto

Tensor library for machine learning

*Note that this project is under active development. \
Some of the development is currently happening in the llama.cpp and whisper.cpp repos*

Features Copy Chapter

- Low-level cross-platform implementation
- Integer quantization support
- Broad hardware support
- Automatic differentiation
- ADAM and L-BFGS optimizers
- No third-party dependencies
- Zero memory allocations during runtime

Build Copy Chapter

git clone https://github.com/ggml-org/ggml
cd ggml
install python dependencies in a virtual environment

python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
build the examples

mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

GPT inference (example) Copy Chapter

run the GPT-2 small 117M model

../examples/gpt-2/download-ggml-model.sh 117M
./bin/gpt-2-backend -m models/gpt-2-117M/ggml-model.bin -p "This is an example"

For more information, checkout the corresponding programs in the examples folder.

Resources Copy Chapter

- Introduction to ggml
- The GGUF file format