# Technical Documentation: hyperknot/openfreemap > ℹ️ **Provenance:** Hybrid Fusion: `hyperknot/openfreemap` (README + 10 In-Tree Chapters) · [CodeWiki Reference](https://codewiki.google/github.com/hyperknot/openfreemap) · Recency: Active (< 180 days) ## 1. Project Overview & Quickstart (hyperknot/openfreemap) [](https://openfreemap.org/) # OpenFreeMap OpenFreeMap lets you display custom maps on your website and apps for free. You can either [self-host](docs/self_hosting.md) or use our public instance. Everything is **open-source**, including the full production setup — there’s no 'open-core' model here. The map data comes from OpenStreetMap. Using our **public instance** is completely free: there are no limits on the number of map views or requests. There’s no registration, no user database, no API keys, and no cookies. We aim to cover the running costs of our public instance through donations. We also provide **weekly** full planet downloads both in Btrfs and MBTiles formats. #### Quick introduction and how to guide: [https://openfreemap.org/](https://openfreemap.org/) ## Goals of this project The goal of this project is to provide free, production-quality vector-tile hosting using existing tools. Currently these tools are: [OpenStreetMap](https://www.openstreetmap.org/copyright), [OpenMapTiles](https://github.com/openmaptiles/openmaptiles), [Planetiler](https://github.com/onthegomap/planetiler), [MapLibre](https://maplibre.org/), [Natural Earth](https://www.naturalearthdata.com/) and [Wikidata](https://www.wikidata.org/wiki/Wikidata:Main_Page). Special thanks go to [Michael Barry](https://github.com/msbarry) for developing [Planetiler](https://github.com/onthegomap/planetiler). It made it possible to generate the tiles in 5 hours instead of 5 weeks. The scope of this repo is limited (see below). Once we figure out the technical details, ideally, there should be few commits here, while everything continues to work: the map tiles are automatically generated, servers are automatically updated and load balancing takes care of any downtime. The [styles repo](https://github.com/hyperknot/openfreemap-styles), on the other hand, is continuously being developed. Contributions are more than welcome! ## Status of this project - The tile generation works - The web servers work - Weekly auto-updates work - Servers in our public instance are currently: - 1 server running tile generation - 2 servers running web hosting - Web servers are in Round-Robin DNS configuration with Let's Encrypt provided certificates. - Load-balancer script works. Currently in monitoring-only mode, as Round-Robin DNS handles downtime. - The public instance has been the production basemap service of [MapHub](https://maphub.net/) since June 2024. ## Sponsoring Please consider sponsoring our project on [GitHub Sponsors](https://github.com/sponsors/hyperknot). ## Limitations of this project The only way this project can possibly work is to be super focused about what it is and what it isn't. OpenFreeMap has the following limitations by design: 1. OpenFreeMap is not providing: - search or geocoding - route calculation, navigation or directions - static image generation - raster tile hosting - satellite image hosting - elevation lookup - custom tile or dataset hosting 2. OpenFreeMap is not something you can install locally. This repo is a deploy script specifically made to set up clean Ubuntu servers or virtual machines. It uses [Fabric](https://www.fabfile.org/) and runs commands over SSH. With a single command it can set up a production-ready server, both for tile hosting and generation. This repo is Docker-free on purpose. If someone wants to make a Docker-based version of this, I'm more than happy to link it here. 3. OpenFreeMap does not promise worry-free automatic updates for self-hosters. Only use the autoupdate version of http-host if you keep a close eye on this repo. ## Self hosting See [self hosting docs](docs/self_hosting.md). ## What is the tech stack? There is no tile server running; only Btrfs partition images with 300 million hard-linked files. This was my idea; I haven't read about anyone else doing this in production, but it works really well. There is no cloud, just dedicated servers. The web server is nginx on Ubuntu. ## Btrfs images Production-quality hosting of 300 million tiny files is hard. The average file size is just 450 byte. Dozens of tile servers have been written to tackle this problem, but they all have their limitations. The original idea of this project is to avoid using tile servers altogether. Instead, the tiles are directly served from Btrfs partition images + hard links using an optimised nginx config. I wrote [extract_mbtiles](modules/tile_gen/scripts/extract_mbtiles.py) and [shrink_btrfs](modules/tile_gen/scripts/shrink_btrfs.py) scripts for this very purpose. This replaces a running service with a pure, file-system-level implementation. Since the Linux kernel's file caching is among the highest-performing and most thoroughly tested codes ever written, it delivers serious performance. I run some [benchmarks](docs/benchmark/README.md) on a Hetzner server, the aim was to saturate a gigabit connection. At the end, it was able to serve 30 Gbit on loopback interface, on cold nginx cache. ## Code structure The project has the following parts #### deploy server - ssh_lib and init-server.py This sets up everything on a clean Ubuntu server. You run it locally and it sets up the server via SSH. #### HTTP host - modules/http_host Inside `http_host`, all work is done by `http_host.py`. It does the following: - Downloading btrfs images - Downloading assets - Mounting downloaded btrfs images - Fetches version files - Running the sync cron task (called every minute with http-host-autoupdate) You can run `./http_host.py --help` to see which options are available. #### tile generation - modules/tile_gen _note: Tile generation is 100% optional, as we are providing the processed full planet btrfs files for public download. You can download full planet images updated weekly, both in Btrfs and in MBTiles format._ The `tile_gen` script downloads a full planet OSM extract and runs it through Planetiler. The created .mbtiles file is then extracted into a Btrfs partition image using the custom [extract_mbtiles](modules/tile_gen/scripts/extract_mbtiles.py) script. The partition is shrunk using the [shrink_btrfs](modules/tile_gen/scripts/shrink_btrfs.py) script. Finally, it's uploaded to a public Cloudflare R2 bucket using rclone. #### styles - [styles repo](https://github.com/hyperknot/openfreemap-styles) The default styles. I've already put countless hours into tweaking up some nice looking styles. Still, it'll take probably the most work in the long term future. Of course, you are welcome to use custom styles. #### load balancer script - modules/loadbalancer A Round Robin DNS based load balancer script for health checking and updating records. It pushes status messages to a Telegram bot. ## FAQ ### Full planet downloads Full planet runs are uploaded weekly. You can download them both in Btrfs and in MBTiles formats. The files have the following URL patterns: https://btrfs.openfreemap.com/areas/planet/{version}/tiles.btrfs.gz (and .mbtiles) Use the [index file](https://btrfs.openfreemap.com/files.txt) to find out about versions. _Note: MBTiles files are not required for this project. We provide them for your convenience, allowing you to use the processed planet tiles with any other tool of your choice._ ### Public buckets There are two public buckets: - https://assets.openfreemap.com - contains fonts, sprites, styles, versions. index: [dirs](https://assets.openfreemap.com/dirs.txt), [files](https://assets.openfreemap.com/files.txt) - https://btrfs.openfreemap.com - full planet runs. index: [dirs](https://btrfs.openfreemap.com/dirs.txt), [files](https://btrfs.openfreemap.com/files.txt) ### Domains .org - not hosted through CloudFlare \ .com - hosted through CloudFlare - serving the public buckets ### What about PMTiles and using the Cloud? I would have loved to use PMTiles; they are a brilliant idea for serverless map hosting! Unfortunately, on Cloudflare, range requests in 90 GB files have terrible latency, and on AWS, the data transfer costs can be prohibitive. Of course, with normal usage, you might fall within cloud vendor's free tier, but the internet is full of stories about people receiving surprise bills from AWS, sometimes amounting to thousands of dollars. It only takes one bad crawling bot getting stuck in a loop on your website to trigger such a bill. In short, using cloud vendors would make it impossible for me to offer this service for free — this project simply wouldn't exist. ## Contributing Contributors welcome! Smaller tasks: - Cloudflare worker for indexing the public buckets, instead of generating index files. - [styles] Some of the POI icons are missing. Bigger tasks: - [styles] Split the styles to building blocks. For example, there should be a POI block, a label block, a road-style related block. Future: - Migrate to [Shortbread schema](https://shortbread-tiles.org/) and possibly [VersaTiles](https://versatiles.org/) #### Dev setup See [dev setup docs](docs/dev_setup.md). ## Changelog ##### v0.9 Updated Planetiler version to latest Updated OpenJDK to 24 via Temurin repo ##### v0.8 Lot of self-hosting related fixes. Generating the domain inside the style TileJSON files dynamically (using nginx sub_filter). Added SELF_SIGNED_CERTS variable for cases when the certificates are self-managed or self-signed is OK. ##### v0.7 MBTiles are now uploaded, next to the btrfs image files. ##### v0.6 Load-balancer implemented with new config format. Implemented relaxed mode for checking while deployments are happening. ##### v0.5 Using a "done" file in the R2 buckets to mark the upload as finished. All scripts are checking for this file now. Monaco is generated daily, to avoid too frequent nginx reloads, which might be bad for the in-memory cache. ##### v0.4 Auto-update works! Monaco is generated hourly. Set-latest runs every minute. Planet is generated weekly, every Wednesday. Set-latest runs every Saturday. ##### v0.3 Lot of performance related problems with Cloudflare when using Round-Robin DNS. Works much better without any Cloudflare proxying, the browsers actually do a great job of client-side failover and selecting the best host. Load-balancing script running in check mode again. ##### v0.2 Load-balancing script is running in write mode, updating records when needed. ##### v0.1 Everything works. 1 server for tile gen, 2 servers for HTTP host. Load-balancing script is running in a read-only mode. ## Attribution Attribution is required. If you are using MapLibre, they are automatically added, you have nothing to do. If you are using alternative clients, or if you are using this in printed media or video, you must add the following attribution: [OpenFreeMap](https://openfreemap.org) [© OpenMapTiles](https://www.openmaptiles.org/) Data from [OpenStreetMap](https://www.openstreetmap.org/copyright) You do not need to display the OpenFreeMap part, but it is nice if you do. ## License The license of this project is [MIT](https://www.tldrlegal.com/license/mit-license). Map data is from [OpenStreetMap](https://www.openstreetmap.org/copyright). The licenses for included projects are listed in [LICENSE.md](https://github.com/hyperknot/openfreemap/blob/main/LICENSE.md). ## 2. In-Tree Documentation Chapters (hyperknot/openfreemap) ## File: README.md [](https://openfreemap.org/) # OpenFreeMap OpenFreeMap lets you display custom maps on your website and apps for free. You can either [self-host](docs/self_hosting.md) or use our public instance. Everything is **open-source**, including the full production setup — there’s no 'open-core' model here. The map data comes from OpenStreetMap. Using our **public instance** is completely free: there are no limits on the number of map views or requests. There’s no registration, no user database, no API keys, and no cookies. We aim to cover the running costs of our public instance through donations. We also provide **weekly** full planet downloads both in Btrfs and MBTiles formats. #### Quick introduction and how to guide: [https://openfreemap.org/](https://openfreemap.org/) ## Goals of this project The goal of this project is to provide free, production-quality vector-tile hosting using existing tools. Currently these tools are: [OpenStreetMap](https://www.openstreetmap.org/copyright), [OpenMapTiles](https://github.com/openmaptiles/openmaptiles), [Planetiler](https://github.com/onthegomap/planetiler), [MapLibre](https://maplibre.org/), [Natural Earth](https://www.naturalearthdata.com/) and [Wikidata](https://www.wikidata.org/wiki/Wikidata:Main_Page). Special thanks go to [Michael Barry](https://github.com/msbarry) for developing [Planetiler](https://github.com/onthegomap/planetiler). It made it possible to generate the tiles in 5 hours instead of 5 weeks. The scope of this repo is limited (see below). Once we figure out the technical details, ideally, there should be few commits here, while everything continues to work: the map tiles are automatically generated, servers are automatically updated and load balancing takes care of any downtime. The [styles repo](https://github.com/hyperknot/openfreemap-styles), on the other hand, is continuously being developed. Contributions are more than welcome! ## Status of this project - The tile generation works - The web servers work - Weekly auto-updates work - Servers in our public instance are currently: - 1 server running tile generation - 2 servers running web hosting - Web servers are in Round-Robin DNS configuration with Let's Encrypt provided certificates. - Load-balancer script works. Currently in monitoring-only mode, as Round-Robin DNS handles downtime. - The public instance has been the production basemap service of [MapHub](https://maphub.net/) since June 2024. ## Sponsoring Please consider sponsoring our project on [GitHub Sponsors](https://github.com/sponsors/hyperknot). ## Limitations of this project The only way this project can possibly work is to be super focused about what it is and what it isn't. OpenFreeMap has the following limitations by design: 1. OpenFreeMap is not providing: - search or geocoding - route calculation, navigation or directions - static image generation - raster tile hosting - satellite image hosting - elevation lookup - custom tile or dataset hosting 2. OpenFreeMap is not something you can install locally. This repo is a deploy script specifically made to set up clean Ubuntu servers or virtual machines. It uses [Fabric](https://www.fabfile.org/) and runs commands over SSH. With a single command it can set up a production-ready server, both for tile hosting and generation. This repo is Docker-free on purpose. If someone wants to make a Docker-based version of this, I'm more than happy to link it here. 3. OpenFreeMap does not promise worry-free automatic updates for self-hosters. Only use the autoupdate version of http-host if you keep a close eye on this repo. ## Self hosting See [self hosting docs](docs/self_hosting.md). ## What is the tech stack? There is no tile server running; only Btrfs partition images with 300 million hard-linked files. This was my idea; I haven't read about anyone else doing this in production, but it works really well. There is no cloud, just dedicated servers. The web server is nginx on Ubuntu. ## Btrfs images Production-quality hosting of 300 million tiny files is hard. The average file size is just 450 byte. Dozens of tile servers have been written to tackle this problem, but they all have their limitations. The original idea of this project is to avoid using tile servers altogether. Instead, the tiles are directly served from Btrfs partition images + hard links using an optimised nginx config. I wrote [extract_mbtiles](modules/tile_gen/scripts/extract_mbtiles.py) and [shrink_btrfs](modules/tile_gen/scripts/shrink_btrfs.py) scripts for this very purpose. This replaces a running service with a pure, file-system-level implementation. Since the Linux kernel's file caching is among the highest-performing and most thoroughly tested codes ever written, it delivers serious performance. I run some [benchmarks](docs/benchmark/README.md) on a Hetzner server, the aim was to saturate a gigabit connection. At the end, it was able to serve 30 Gbit on loopback interface, on cold nginx cache. ## Code structure The project has the following parts #### deploy server - ssh_lib and init-server.py This sets up everything on a clean Ubuntu server. You run it locally and it sets up the server via SSH. #### HTTP host - modules/http_host Inside `http_host`, all work is done by `http_host.py`. It does the following: - Downloading btrfs images - Downloading assets - Mounting downloaded btrfs images - Fetches version files - Running the sync cron task (called every minute with http-host-autoupdate) You can run `./http_host.py --help` to see which options are available. #### tile generation - modules/tile_gen _note: Tile generation is 100% optional, as we are providing the processed full planet btrfs files for public download. You can download full planet images updated weekly, both in Btrfs and in MBTiles format._ The `tile_gen` script downloads a full planet OSM extract and runs it through Planetiler. The created .mbtiles file is then extracted into a Btrfs partition image using the custom [extract_mbtiles](modules/tile_gen/scripts/extract_mbtiles.py) script. The partition is shrunk using the [shrink_btrfs](modules/tile_gen/scripts/shrink_btrfs.py) script. Finally, it's uploaded to a public Cloudflare R2 bucket using rclone. #### styles - [styles repo](https://github.com/hyperknot/openfreemap-styles) The default styles. I've already put countless hours into tweaking up some nice looking styles. Still, it'll take probably the most work in the long term future. Of course, you are welcome to use custom styles. #### load balancer script - modules/loadbalancer A Round Robin DNS based load balancer script for health checking and updating records. It pushes status messages to a Telegram bot. ## FAQ ### Full planet downloads Full planet runs are uploaded weekly. You can download them both in Btrfs and in MBTiles formats. The files have the following URL patterns: https://btrfs.openfreemap.com/areas/planet/{version}/tiles.btrfs.gz (and .mbtiles) Use the [index file](https://btrfs.openfreemap.com/files.txt) to find out about versions. _Note: MBTiles files are not required for this project. We provide them for your convenience, allowing you to use the processed planet tiles with any other tool of your choice._ ### Public buckets There are two public buckets: - https://assets.openfreemap.com - contains fonts, sprites, styles, versions. index: [dirs](https://assets.openfreemap.com/dirs.txt), [files](https://assets.openfreemap.com/files.txt) - https://btrfs.openfreemap.com - full planet runs. index: [dirs](https://btrfs.openfreemap.com/dirs.txt), [files](https://btrfs.openfreemap.com/files.txt) ### Domains .org - not hosted through CloudFlare \ .com - hosted through CloudFlare - serving the public buckets ### What about PMTiles and using the Cloud? I would have loved to use PMTiles; they are a brilliant idea for serverless map hosting! Unfortunately, on Cloudflare, range requests in 90 GB files have terrible latency, and on AWS, the data transfer costs can be prohibitive. Of course, with normal usage, you might fall within cloud vendor's free tier, but the internet is full of stories about people receiving surprise bills from AWS, sometimes amounting to thousands of dollars. It only takes one bad crawling bot getting stuck in a loop on your website to trigger such a bill. In short, using cloud vendors would make it impossible for me to offer this service for free — this project simply wouldn't exist. ## Contributing Contributors welcome! Smaller tasks: - Cloudflare worker for indexing the public buckets, instead of generating index files. - [styles] Some of the POI icons are missing. Bigger tasks: - [styles] Split the styles to building blocks. For example, there should be a POI block, a label block, a road-style related block. Future: - Migrate to [Shortbread schema](https://shortbread-tiles.org/) and possibly [VersaTiles](https://versatiles.org/) #### Dev setup See [dev setup docs](docs/dev_setup.md). ## Changelog ##### v0.9 Updated Planetiler version to latest Updated OpenJDK to 24 via Temurin repo ##### v0.8 Lot of self-hosting related fixes. Generating the domain inside the style TileJSON files dynamically (using nginx sub_filter). Added SELF_SIGNED_CERTS variable for cases when the certificates are self-managed or self-signed is OK. ##### v0.7 MBTiles are now uploaded, next to the btrfs image files. ##### v0.6 Load-balancer implemented with new config format. Implemented relaxed mode for checking while deployments are happening. ##### v0.5 Using a "done" file in the R2 buckets to mark the upload as finished. All scripts are checking for this file now. Monaco is generated daily, to avoid too frequent nginx reloads, which might be bad for the in-memory cache. ##### v0.4 Auto-update works! Monaco is generated hourly. Set-latest runs every minute. Planet is generated weekly, every Wednesday. Set-latest runs every Saturday. ##### v0.3 Lot of performance related problems with Cloudflare when using Round-Robin DNS. Works much better without any Cloudflare proxying, the browsers actually do a great job of client-side failover and selecting the best host. Load-balancing script running in check mode again. ##### v0.2 Load-balancing script is running in write mode, updating records when needed. ##### v0.1 Everything works. 1 server for tile gen, 2 servers for HTTP host. Load-balancing script is running in a read-only mode. ## Attribution Attribution is required. If you are using MapLibre, they are automatically added, you have nothing to do. If you are using alternative clients, or if you are using this in printed media or video, you must add the following attribution: [OpenFreeMap](https://openfreemap.org) [© OpenMapTiles](https://www.openmaptiles.org/) Data from [OpenStreetMap](https://www.openstreetmap.org/copyright) You do not need to display the OpenFreeMap part, but it is nice if you do. ## License The license of this project is [MIT](https://www.tldrlegal.com/license/mit-license). Map data is from [OpenStreetMap](https://www.openstreetmap.org/copyright). The licenses for included projects are listed in [LICENSE.md](https://github.com/hyperknot/openfreemap/blob/main/LICENSE.md). --- ## File: docs/benchmark/README.md # HTTP Hosts Benchmarking This repository contains tools and scripts for benchmarking HTTP hosts performance. ## Prerequisites Before running the benchmarks, you need to create a path list (`path_list_500k.txt`). You have two options: 1. Generate from real-world server logs using `nginx_to_path_list.py` 2. Generate randomly (Note: real-world usage patterns are typically non-random, e.g., ocean tiles are rarely accessed) ## Important Notes - Run the benchmarks on `localhost`, and not over the internet! Otherwise you'd be just testing your internet speed. - The benchmark uses [wrk](https://github.com/wg/wrk) HTTP benchmarking tool ## Usage Basic command: ```bash wrk -c10 -t4 -d10s -s /data/ofm/benchmark/wrk_custom_list.lua http://localhost ``` ### Parameters Explained - `-c10`: Number of connections to keep open - `-t4`: Number of threads to use - `-d10s`: Duration of the test (10 seconds) - `-s`: Script file to use ### Thread Count Considerations - `-t1`: More accurate results as the URL list is loaded exactly in sequence - `-t4`: Better reflects real-world usage patterns ## Results Benchmark results can be found in [results.md](results.md) ## Contributing Feel free to submit your results including which hosts were used. --- ## File: docs/benchmark/results.md # wrk benchmarks Real world usage, 500k requests replayed from server log. ### Hetnzer dedicated server with NVME ssd #### localhost clean cache after nginx restart. ``` service nginx restart wrk -c10 -t4 -d60s -s /data/ofm/benchmark/wrk_custom_list.lua http://localhost Running 1m test @ http://localhost 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 2.02ms 7.04ms 50.43ms 93.23% Req/Sec 8.42k 2.01k 18.52k 69.79% 2871265 requests in 1.00m, 230.65GB read Requests/sec: 47811.00 Transfer/sec: 3.84GB ``` Super much overkill, we'd only need 125 MB/s for Gigabit connection and this is 3840 MB/s. Also max request time is super nice + no errors. #### over network ``` wrk -c10 -t4 -d60s -s /data/ofm/benchmark/wrk_custom_list.lua http://x.x.x.x Running 1m te st @ http://144.76.168.195 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 7.57ms 6.61ms 45.34ms 84.32% Req/Sec 293.85 141.33 1.18k 73.07% 71628 requests in 1.00m, 6.05GB read Requests/sec: 1191.88 Transfer/sec: 103.01MB ``` Realistically this is the max over Gigabit connection. --- ### BuyVM KVM machine with 1 TB BuyVM Block Storage Slab Advertisement: 40Gbit+ InfiniBand RDMA Storage Fabric giving near local storage performance. Reality: ``` wrk -c10 -t4 -d60s -s /data/ofm/benchmark/wrk_custom_list.lua http://localhost Running 1m test @ http://localhost 4 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 226.10ms 343.52ms 1.99s 87.75% Req/Sec 29.77 38.06 272.00 89.72% 3655 requests in 1.00m, 232.76MB read Socket errors: connect 0, read 0, write 0, timeout 8 Requests/sec: 60.87 Transfer/sec: 3.88MB ``` Wow, this is 60 request per second compared to Hetzner's 47000, just wow! Repeated tests with hot cache resulted in a bit better performance, but still not Gigabit. ``` Requests/sec: 266.99 Transfer/sec: 23.07MB ``` Abandoned the idea of using BuyVM, even though their unlimited bandwidth is quite unique in this price range in USA. --- ## File: docs/_not_used/fs_stats/ext4.md ## deduplicated on ext4 ```df -h mnt_rw df -h mnt_rw Filesystem Size Used Avail Use% Mounted on /dev/loop0 1.4T 187G 1.2T 14% ``` ``` df -i mnt_rw Filesystem Inodes IUsed IFree IUse% Mounted on /dev/loop0 393216000 39614466 353601534 11% ``` --- ## File: docs/_not_used/fs_stats/naive_mbutil_ext4.md ## native mapbox/mbutil // pip install git+https://github.com/mapbox/mbutil.git@544c76e ``` Filesystem 1K-blocks Used Available Use% Mounted on /dev/loop0 1,474,386,100 1,119,622,516 354,763,584 76% ``` ``` Filesystem Inodes IUsed IFree IUse% Mounted on /dev/loop0 393,216,000 269,252,174 123,963,826 69% ``` --- ## File: docs/_not_used/fs_stats/README.md # Comparing filesystem stats after extraction Run: planet/20231208_091355_pt dedupl-fix.log contains the files which were created for the btrfs 64k limit workaround. --- ## File: docs/debugging_names.md # Debugging international names If there is an issue about international names not being displayed correctly, first, we need to find **one specific example** and check at which stage does the problem appear. OpenFreeMap map data is created by the following stack: **OpenStreetMap data ➔ OpenMapTiles specification ➔ Planetiler** 1. To debug OpenStreetMap data, go to OpenStreetMap.org and search for the query string. For example "Iwate Prefecture" gives these results: [nominatim](https://nominatim.openstreetmap.org/ui/details.html?osmtype=R&osmid=3792412&class=boundary) and [openstreetmap](https://www.openstreetmap.org/relation/3792412) 2. Then we need to check what the data is in the generated vector tiles. The best way to do this is to go to [Maputnik editor](https://maputnik.github.io/editor?style=https://tiles.openfreemap.org/styles/bright) and select View / Inspect. 3. Then you can search for the little red dot matching your label and make a screenshot. Now we can compare where the naming problem is coming from. In conclusion: for the **one specific example**, please link the OSM pages and add the inspector screenshot, then we can start with the debugging. ## Next steps It'd be nice to compare with other OpenMapTiles implementations like [tilemaker](https://github.com/systemed/tilemaker) or the [OpenMapTiles reference](https://github.com/openmaptiles/openmaptiles). I don't have full planet datasets from these implementations currently, so if someone is willing to run one it'd be a great contribution. --- ## File: docs/dev_setup.md # dev setup ### macOS On macOS, I recommend [OrbStack](https://orbstack.dev/). I saved this function into my bash_profile. It sets up a clean x64-based Ubuntu 22 VM in a few seconds. ``` orb_reset() { orbctl delete -f ubuntu-test orbctl create -a amd64 ubuntu:jammy ubuntu-test } ``` I saved the following in `.ssh/config`: ``` Host orb_my Hostname 127.0.0.1 Port 32222 IdentityFile ~/.orbstack/ssh/id_ed25519 ``` Then I run commands like the following: ``` ./init-server.py http-host-static orb_my ./init-server.py debug orb_my ``` --- ## File: docs/self_hosting.md # Self-hosting Howto You can either self-host or use our public instance. Everything is **open-source**, including the full production setup — there’s no 'open-core' model here. When self-hosting, there are two modules you can set up on a server (see details in the repo README). - **http-host** - **tile-gen** There is a 99.9% chance you only need **http-host**. Tile-gen is slow, needs a huge machine and is totally pointless, since we upload the processed files every week. ### System requirements **http-host**: 300 GB disk space for hosting a single run. SSD is recommended, but not required. **tile-gen**: 500 GB SDD and at least 64 GB ram **Ubuntu 22** or newer ### Provider recommendation One amazing deal, which is tested and known to work well for http-host is the €4.5 / month [Contabo Storage VPS](https://contabo.com/en/storage-vps/) --- ### Warning This project is made to run on **clean servers** or virtual machines dedicated for this project. The scripts need sudo permissions as they mount/unmount disk images. Do not run this on your dev machine without using virtual machines. If you do, please make sure you understand exactly what each script is doing. If you run it on a non-clean server, please understand that this will modify your nginx config! --- ## Instructions I recommend running things quickly first, with `SKIP_PLANET=true` and then once it works, running it with `SKIP_PLANET=false`. #### 1. DNS setup Set up a server with at least 300 GB SSD space and configure the DNS for the subdomain of your choice. For example, make an A record for "maps.example.com" -> 185.199.110.153 #### 2. Clone and prepare `config` folder ``` git clone https://github.com/hyperknot/openfreemap ``` In the config folder, copy `.env.sample` to `.env` and set the values. `DOMAIN_DIRECT` - Your subdomain \ `LETSENCRYPT_EMAIL` - Your email for Let's Encrypt Set `SKIP_PLANET=true` first. #### 3. Set up Python if you don't have it yet On Ubuntu you can get it by `sudo apt install python3-pip` On macOS you can do `brew install python` #### 4. Prepare the Python environment You run the deploy script locally, and it deploys to a remote server over SSH. You can use a virtualenv if you are used to working with them, but it's not necessary. ``` cd openfreemap pip install -e . ``` #### 5. Deploy quick version with `SKIP_PLANET=true` Run the actual deploy command and wait a few minutes ``` ./init-server.py http-host-static HOSTNAME ``` #### 5. Check If everything is OK, you'll have some curl lines printed. Run the first one locally and make sure it's showing HTTP/2 200. For example this is an OK response. ```locally to test them. curl -sI https://test.openfreemap.org/monaco | sort HTTP/2 200 access-control-allow-origin: * cache-control: max-age=86400 cache-control: public content-length: 5776 content-type: application/json date: Fri, 11 Oct 2024 21:01:23 GMT etag: "670991d1-1690" expires: Sat, 12 Oct 2024 21:01:23 GMT last-modified: Fri, 11 Oct 2024 21:00:01 GMT server: nginx x-ofm-debug: latest JSON monaco ``` #### 6. Deploy and check with `SKIP_PLANET=false` Update your `.env` file and re-run the same `./init-server.py http-host-static HOSTNAME` as before. Go for a walk and by the time you come back it should be up and running with the latest planet tiles deployed. Don't worry about the "Download aborted" lines in the meanwhile, it's a bug in CloudFlare. If your server doesn't have an SSD, the download + uncompressing process can take hours. --- #### Deploy tile-gen server (optional) If you have a really beefy machine (see above) and you really want to generate tiles yourself, you can run `./init-server.py tile-gen HOSTNAME`. Trigger a run manually, by running ``` sudo /data/ofm/venv/bin/python -u /data/ofm/tile_gen/bin/tile_gen.py make-tiles planet ``` It's recommended to use tmux or similar, as it can take days to complete. --- ## File: modules/tile_gen/scripts/README.md These are self contained Python scripts, they can be run outside of this project's environment. --- METRICS --- - Files Extracted: 11 - Estimated Token Budget: ~8374 tokens - Recency Window: Active (< 180 days) - Canonical Reference: https://codewiki.google/github.com/hyperknot/openfreemap