# Tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

### Repository Details
- **Repository:** [osm-search/Tokenizers](https://github.com/osm-search/Tokenizers)
- **GitHub Stars:** ⭐ 10,947
- **Primary Language:** Rust

*Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.*