Technical Documentation: six2dez/reconftw

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (README.md) Copy Section

reconFTW

reconFTW is a powerful automated reconnaissance tool designed for security researchers and penetration testers. It streamlines the process of gathering intelligence on a target by performing subdomain enumeration, vulnerability scanning, OSINT and more. With a modular design, extensive configuration options, and support for distributed scanning via AX Framework, reconFTW is built to deliver comprehensive results efficiently.

reconFTW leverages a wide range of techniques, including passive and active subdomain discovery, web vulnerability checks (e.g., XSS, SSRF, SQLi), OSINT, directory fuzzing, port scanning and screenshotting. It integrates with cutting-edge tools and APIs to maximize coverage and accuracy, ensuring you stay ahead in your reconnaissance efforts.

Key Features:

• Comprehensive subdomain enumeration (passive, bruteforce, permutations, certificate transparency, etc.)
• Vulnerability scanning for XSS, SSRF, SQLi, LFI, SSTI, and more
• OSINT for emails, metadata, API leaks, and third-party misconfigurations
• Distributed scanning with AX Framework for faster execution
• Customizable workflows with a detailed configuration file
• Integration with Faraday for reporting and visualization
• Support for Docker, Terraform and Ansible deployments

Disclaimer: Usage of reconFTW for attacking targets without prior consent is illegal. It is the user's responsibility to obey all applicable laws. The developers assume no liability for misuse or damage caused by this tool. Use responsibly.

📔 Table of Contents Copy Chapter

• 📔 Table of Contents
• ✨ Features
  • OSINT
  • Subdomains
  • Hosts
  • Web Analysis
  • Vulnerability Checks
  • Extras
• 🏗️ Architecture
• 💿 Installation
  • Local Installation (PC/VPS/VM)
  • Docker
  • Terraform + Ansible
• Trying the v2 beta (Go rewrite)
• ⚙️ Configuration
• 🚀 Usage
  • Target Options
  • Mode Options
  • General Options
  • Example Usage
• ☁️ Ax Framework Support (previously Axiom)
• 💻 Faraday Support
• 🧠 AI Integration
• 🗂️ Data Management
  • Makefile
  • Manual
• 🧪 Testing
• Mindmap/Workflow
• Sample video
• 🤝 How to Contribute
• 🔒 Security
• ❓ Need Help?
• 💖 Support This Project
• 🙏 Thanks
• 📝 Changelog
• 🛠️ Development
• 📜 License
• ⭐ Star History

✨ Features Copy Chapter

reconFTW is packed with features to make reconnaissance thorough and efficient. Below is a detailed breakdown of its capabilities, updated to reflect the latest functionality in the script and configuration.

OSINT Copy Section

• Domain Information: WHOIS lookup for domain registration details (whois).
• Email and Password Leaks: Searches for leaked emails and credentials (emailfinder and LeakSearch).
• Microsoft 365/Azure Mapping: Identifies Microsoft 365 and Azure tenants (msftrecon).
• Metadata Extraction: Extracts metadata from indexed office documents (metagoofil).
• API Leaks: Detects exposed APIs in public sources (porch-pirate, SwaggerSpy and postleaksNg).
• Google Dorking: Automated Google dork queries for sensitive information (dorks_hunter and xnldorker).
• GitHub Analysis: Scans GitHub organizations for repositories and secrets with selectable engines (enumerepo, trufflehog, gitleaks, titus, noseyparker).
• GitHub Actions Audit (Optional): Audits workflow artifacts and CI/CD exposure with gato.
• Third-Party Misconfigurations: Identifies misconfigured third-party services (misconfig-mapper).
• Mail Hygiene: Reviews SPF/DMARC configuration to flag spoofing or deliverability issues.
• Cloud Storage Enumeration: Surveys buckets across major providers for exposure (cloud_enum).
• Spoofable Domains: Checks for domains vulnerable to spoofing (spoofcheck).

Subdomains Copy Section

• Passive Enumeration: Uses APIs and public sources for subdomain discovery (subfinder and github-subdomains).
• Certificate Transparency: Queries certificate transparency logs (crt).
• NOERROR Discovery: Identifies subdomains with DNS NOERROR responses (dnsx, more info here).
• Bruteforce: Performs DNS bruteforcing with customizable wordlists (puredns and custom wordlists).
• Permutations: Generates subdomain permutations using AI, regex and tools (Gotator as the single permutation engine, plus regulator and subwiz).
• Web Scraping: Extracts subdomains from passive URL sources and live web metadata (urlfinder, waymore, httpx, csprecon).
• DNS Records: Resolves DNS records for subdomains (dnsx).
• Google Analytics: Identifies subdomains via Analytics IDs (AnalyticsRelationships).
• TLS Handshake: Discovers subdomains via TLS ports (tlsx).
• Recursive Search: Performs recursive passive or bruteforce enumeration combined (dsieve).
• Subdomain Takeover: Detects vulnerable subdomains (nuclei and dnstake).
• DNS Zone Transfer: Checks for misconfigured DNS zone transfers (dig).
• Cloud Buckets: Identifies misconfigured cloud buckets and exposed storage assets (S3Scanner and cloud_enum).
• Cloud Coverage Note: Cloud bucket checks no longer include Alibaba OSS coverage after replacing CloudHunter with cloud_enum.
• Cloud Output Migration: Legacy cloudhunter_* bucket artifacts were removed; use subdomains/cloud_enum_buckets_trufflehog.txt instead.
• Reverse IP Lookup: Discovers subdomains via IP ranges (hakip2host).

Hosts Copy Section

• IP Information: Retrieves geolocation and WHOIS data (ipinfo).
• CDN Detection: Identifies IPs behind CDNs (cdncheck).
• WAF Detection: Detects Web Application Firewalls (wafw00f).
• Port Scanning: Active scanning with nmap (optionally preceded by naabu) and passive scanning with smap.
• Service Fingerprinting: Fingerprints exposed services on discovered host:port pairs with nerva.
• Service Vulnerabilities (Optional): Deep portscan profile can enrich results with CVE matching via vulners.
• Password Spraying: Attempts password spraying on identified services with engine selection (brutespray or brutus).
• Geolocation: Maps IP addresses to geographic locations (ipinfo).
• IPv6 Discovery: Optionally enumerates and scans discovered IPv6 targets when IPV6_SCAN is enabled.

Web Analysis Copy Section

• Web Probing: Detects live web servers on standard and uncommon ports ((httpx)).
• Screenshots: Captures screenshots of web pages (nuclei).
• Virtual Host Fuzzing: Identifies virtual hosts by fuzzing HTTP headers (VhostFinder).
• CMS Detection: Identifies content management systems (CMSeeK).
• URL Extraction: Collects URLs passively and actively (urlfinder, waymore, katana, github-endpoints and JSA).
• URL Pattern Analysis: Classifies URLs using patterns (urless, gf and gf-patterns).
• Favicon Tech Recon: Identifies technologies from favicon hashes (favirecon).
• JavaScript Analysis: Extracts secrets and endpoints from JS files (subjs, JSA, xnLinkFinder, getjswords, mantra, jsluice).
• Source Map Extraction: Retrieves sensitive data from JavaScript source maps (sourcemapper).
• GraphQL Detection: Discovers GraphQL endpoints with nuclei and optionally performs in-depth introspection (GQLSpection).
• Parameter Discovery: Bruteforces hidden parameters on endpoints (arjun).
• WebSocket Auditing: Validates upgrade handshakes and origin handling on ws:// and wss:// endpoints.
• gRPC Reflection: Probes common gRPC ports for exposed service reflection (grpcurl).
• LLM Service Fingerprinting (Optional): Probes discovered web/API endpoints for exposed LLM services with julius.
• Fuzzing: Performs directory and parameter fuzzing (ffuf).
• File Extension Sorting: Organizes URLs by file extensions.
• Wordlist Generation: Creates custom wordlists for fuzzing.
• Password Dictionary: Generates password dictionaries from live content (cewler).
• IIS Shortname Scanning: Detects IIS shortname vulnerabilities (shortscan).

Vulnerability Checks Copy Section

• CVEs: Checks for CVE and common vulnerabilites nuclei
• Nuclei DAST: Runs nuclei -dast templates over collected URLs and GF candidates for additional DAST coverage.
• XSS: Tests for cross-site scripting vulnerabilities (dalfox).
• SSL/TLS: Checks for SSL/TLS misconfigurations (testssl).
• SSRF: Tests for server-side request forgery (interactsh, parameter values with ffuf, and optional alternate protocol payloads).
• CRLF: Checks for CRLF injection vulnerabilities (crlfuzz).
• LFI: Tests for local file inclusion via fuzzing (ffuf).
• SSTI: Detects server-side template injection (TInjA).
• SQLi: Tests for SQL injection (SQLMap and ghauri).
• Broken Links: Identifies broken links and external references likely to be takeover-prone (second-order).
• Command Injection: Tests for command injection vulnerabilities (commix).
• HTTP Request Smuggling: Checks for request smuggling vulnerabilities (smugglex).
• Web Cache: Identifies web cache vulnerabilities (Web-Cache-Vulnerability-Scanner and toxicache).
• 4XX Bypassing: Attempts to bypass 4XX responses (nomore403).
• Parameter Fuzzing: Fuzzes URL parameters for vulnerabilities (nuclei).

Extras Copy Section

• Multithreading: Optimizes performance (Interlace).
• Custom Resolvers: Generates DNS resolvers (dnsvalidator).
• Docker Support: Official Docker image on DockerHub.
• AWS Deployment: Deploys via Terraform and Ansible.
• IP/CIDR Support: Scans IP ranges and CIDR blocks.
• Scan Resumption: Resumes scans from the last completed step.
• Custom Output: Saves results to a user-defined directory.
• Diff Mode: Highlights new findings in subsequent scans.
• Scope Filtering: Supports in-scope and out-of-scope lists (inscope).
• Notifications: Sends alerts via Slack, Discord, or Telegram (notify).
• Result Zipping: Compresses and sends results.
• Faraday Integration: Exports results to Faraday for reporting .
• AI Report Generation: Generates reports using local AI models (reconftw_ai).
• Quick Rescan Mode: Skips heavy stages automatically when no new assets are discovered (--quick-rescan / QUICK_RESCAN).
• Hotlist Builder: Scores and highlights the riskiest assets (hotlist.txt) based on new findings.
• Command Tracing: Toggle SHOW_COMMANDS to log every executed command into target logs for debugging.
• Asset Store: Appends findings to assets.jsonl for downstream automation when ASSET_STORE is enabled.
• Consolidated Report: Auto-generates report/report.json and report/index.html at end of scan.
• ARM Support: Compatible with Raspberry Pi and ARM architectures (including MacOS MX).
• Health Check: Built-in system health check via --health-check (also used by Docker HEALTHCHECK).
• Incremental Mode: Only scan new findings since last run (--incremental).
• Adaptive Rate Limiting: Automatically back off on 429/503 errors (--adaptive-rate).
• Structured Logging: Optional JSON log output for advanced analysis (STRUCTURED_LOGGING).
• Input Sanitization: All user input is sanitized to prevent command injection.
• Dry-Run Mode: Preview what would be executed without running commands (--dry-run).
• Parallel Mode: Run independent functions in parallel for faster scans (--parallel, disable with --no-parallel).
• Modular Architecture: Codebase split into 8 focused modules for maintainability.
• Secrets Management: Environment variables, secrets.cfg, and Docker runtime secrets (see SECURITY.md).
• Circuit Breaker: Automatically skips tools after repeated failures to avoid scan hangs.
• Checkpoint System: Resume interrupted scans from the last successful phase.
• macOS Native Support: Full compatibility with macOS (BSD coreutils, Homebrew Bash 4+).

🏗️ Architecture Copy Chapter

reconFTW uses a modular architecture. The main entry point (reconftw.sh) handles argument parsing and sources 8 specialized modules from the modules/ directory.

Directory Layout Copy Section

reconftw/
├── reconftw.sh          # Entry point — arg parsing, module loading, dispatch
├── reconftw.cfg         # Default configuration
├── install.sh           # Installer
├── Makefile             # Data management, lint, fmt, test targets
├── modules/
│   ├── core.sh          # Lifecycle, logging, notifications, cleanup (1024 lines)
│   ├── modes.sh         # Scan modes, argument parsing, help (902 lines)
│   ├── subdomains.sh    # Subdomain enumeration (1938 lines)
│   ├── web.sh           # Web analysis, fuzzing, JS checks (1712 lines)
│   ├── vulns.sh         # Vulnerability scanning (926 lines)
│   ├── osint.sh         # OSINT functions (500 lines)
│   ├── axiom.sh         # Ax/Axiom fleet helpers (143 lines)
│   └── utils.sh         # Utilities, sanitization, validation (508 lines)
├── tests/
│   ├── run_tests.sh     # Test runner
│   ├── unit/            # bats-core unit tests
│   ├── integration/     # Integration tests
│   └── fixtures/        # Test data
├── Docker/
│   └── Dockerfile       # Official Docker image
└── Terraform/           # AWS deployment

Module Reference Copy Section

The --source-only flag allows sourcing reconftw.sh without executing the main logic, enabling unit testing of individual functions.

💿 Installation Copy Chapter

reconFTW supports multiple installation methods to suit different environments. Ensure you have sufficient disk space (at least 10 GB recommended) and a stable internet connection.

Quickstart Copy Section

1. Clone and install

git clone https://github.com/six2dez/reconftw
cd reconftw
./install.sh --verbose

2. Run a scan (full + resume)

./reconftw.sh -d example.com -r

3. Minimal run (passive-only footprint)

./reconftw.sh -d example.com -p

Tip: re-run ./install.sh --tools later to refresh the toolchain without reinstalling system packages.

Local Installation (PC/VPS/VM) Copy Section

1. Prerequisites:

   • Golang: Latest version (install_golang enabled by default in reconftw.cfg).
   • System Permissions: If not running as root, configure sudo to avoid prompts:
     bash Copy

     sudo echo "${USERNAME} ALL=(ALL:ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers.d/reconFTW

2. Steps:

   bash Copy

   git clone https://github.com/six2dez/reconftw
   cd reconftw
   ./install.sh
   ./reconftw.sh -d target.com -r

3. Notes:

• The install.sh script installs dependencies, tools, and configures paths (GOROOT, GOPATH, PATH).
• Set install_golang=false in reconftw.cfg if Golang is already configured.
• For existing setups, run ./install.sh --tools to refresh Go binaries, pipx packages, and repositories without touching system packages.
• Check the Installation Guide for detailed instructions.

Docker Copy Section

1. Pull the Image:

   bash Copy

   docker pull six2dez/reconftw:main

2. Run the Container:

   bash Copy

   docker run -it --rm \
     -v "${PWD}/OutputFolder/:/reconftw/Recon/" \
     six2dez/reconftw:main -d example.com -r

   For a list of targets, bind the list file into the container and reference the in-container path:

   bash Copy

   docker run -it --rm \
     -v "${PWD}/domains.txt:/reconftw/domains.txt:ro" \
     -v "${PWD}/OutputFolder/:/reconftw/Recon/" \
     six2dez/reconftw:main -l /reconftw/domains.txt -r

3. View Results:

   • Results are saved in the OutputFolder directory on the host (not inside the container).

4. Customization:

   • Modify the Docker image or build your own; see the Docker Guide.
   • To skip Ax tooling in custom builds, pass --build-arg INSTALL_AXIOM=false.
   • Mount your notify config at ~/.config/notify/provider-config.yaml inside the container if you use notifications.

5. Secrets at Runtime:

   Pass API keys and secrets via environment variables — never bake them into the image:

   bash Copy

   docker run -it --rm \
     -e SHODAN_API_KEY="your-key" \
     -e PDCP_API_KEY="your-projectdiscovery-key" \
     -e COLLAB_SERVER="your-server" \
     -e XSS_SERVER="your-server" \
     -v "${PWD}/OutputFolder/:/reconftw/Recon/" \
     six2dez/reconftw:main -d example.com -r

   See SECURITY.md for full secrets management guidance.

6. Health Check:

   The Docker image includes a built-in HEALTHCHECK that runs ./reconftw.sh --health-check every 60 seconds. You can also run it manually:

   bash Copy

   docker exec <container-id> ./reconftw.sh --health-check

Terraform + Ansible Copy Section

• Deploy reconFTW on AWS using Terraform and Ansible.
• Follow the guide in Terraform/README.md for setup instructions.

Trying the v2 beta (Go rewrite) Copy Chapter

reconFTW is being rewritten in Go. The rewrite ships as an opt-in pre-release: it is
listed on the releases page, and GitHub
never points releases/latest at a pre-release — so if you do nothing, you keep getting
the bash release. That is deliberate.

The Go binary is reconftw; the bash entry point is reconftw.sh. They do not overwrite
each other, so you can keep both and fall back at any time.

The current beta is v5.0.0-beta.1. It is not served by releases/latest, so you
name the tag explicitly:

curl -sSL "https://github.com/six2dez/reconftw/releases/download/v5.0.0-beta.1/reconftw_Linux_x86_64.tar.gz" | tar xz
sudo install -m 755 reconftw /usr/local/bin/reconftw
reconftw version

Pick the asset matching your platform from the
release page — Darwin
and arm64 builds, a static musl build, and .deb/.rpm packages are all published.

• What the beta is, and what it is not — including three
  things that are explicitly not finished yet.
• Report something
  — the v2 beta feedback template. Bugs in the bash release still go to the normal Bug report.

🛠️ Troubleshooting Copy Chapter

• Bash 4+ on macOS: The scripts auto-relaunch under Homebrew Bash. If you see a message about Bash < 4, run brew install bash, open a new terminal, and re-run ./install.sh.
• timeout on macOS: macOS provides gtimeout via brew install coreutils. The scripts now detect and use it automatically.
• Network hiccups: Installers hide most command output. If something fails, re-run with upgrade_tools=true in reconftw.cfg, execute ./install.sh --tools, or install the missing tool manually (the error will name it).
• GOPATH binaries: Binaries are copied to /usr/local/bin. If you prefer not to, ensure ~/go/bin is in your PATH.
• Nuclei templates: If templates weren’t cloned, remove ~/nuclei-templates and re-run ./install.sh.

🔑 API Checklist (Optional) Copy Chapter

• subfinder: ~/.config/subfinder/provider-config.yaml
• GitHub tokens: ~/Tools/.github_tokens (one per line)
• GitLab tokens: ~/Tools/.gitlab_tokens (one per line)
• WHOISXML: set WHOISXML_API in reconftw.cfg or env var
• ASN enumeration (asnmap): set PDCP_API_KEY in env/config (ASN_ENUM skips if unset)
• Slack/Discord/Telegram: configure notify in ~/.config/notify/provider-config.yaml
• SSRF server: set COLLAB_SERVER env/cfg if used
• Blind XSS server: set XSS_SERVER env/cfg if used

💾 Requirements Copy Chapter

• Disk: 10–20 GB free recommended (toolchain + data)
• Network: stable connection during installation and updates
• OS: Linux/macOS with Bash ≥ 4
• Extras: shellcheck and shfmt (optional) for make lint/make fmt

⚙️ Configuration Copy Chapter

The reconftw.cfg file controls the entire execution of reconFTW. It allows fine-grained customization of:

• Tool Paths: Set paths for tools, resolvers, and wordlists (tools, resolvers, fuzz_wordlist).
• API Keys: Configure keys for Shodan, WHOISXML, etc. via environment variables or secrets.cfg (see SECURITY.md).
• Scanning Modes: Enable/disable modules (e.g., OSINT, SUBDOMAINS_GENERAL, VULNS_GENERAL).
• Performance: Adjust threads, rate limits, and timeouts (e.g., FFUF_THREADS, HTTPX_RATELIMIT).
• Adaptive Rate Limiting: Automatically back off on 429/503 errors (ADAPTIVE_RATE_LIMIT, MIN_RATE_LIMIT, MAX_RATE_LIMIT).
• Incremental Scanning: Only scan new findings since last run (INCREMENTAL_MODE).
• Notifications: Set up Slack, Discord, or Telegram notifications (NOTIFY_CONFIG).
• Ax (formerly Axiom): Configure distributed scanning and resolver paths (AXIOM_FLEET_NAME, AXIOM_FLEET_COUNT, AXIOM_RESOLVERS_PATH).
• AI Reporting: Configure model/profile/format and context controls (AI_MODEL, AI_REPORT_PROFILE, AI_REPORT_TYPE, AI_MAX_CHARS_PER_FILE).
• Advanced Web Checks: Toggle GraphQL introspection, parameter discovery, WebSocket testing, gRPC probing, and IPv6 scanning.
• Automation & Data: Control quick rescan heuristics, asset logging, chunk sizes, hotlists, and debug tracing (QUICK_RESCAN, ASSET_STORE, CHUNK_LIMIT, HOTLIST_TOP, SHOW_COMMANDS).
• Disk & Logging: Pre-flight disk check (MIN_DISK_SPACE_GB), log rotation (MAX_LOG_FILES, MAX_LOG_AGE_DAYS), structured JSON logging (STRUCTURED_LOGGING).
• Caching: Configure cache expiry for wordlists and resolvers (CACHE_MAX_AGE_DAYS).
• DNS Resolver Safety: Missing resolver files fail fast, resolver downloads use configurable retry/timeout knobs (RESOLVER_DOWNLOAD_*), and DNS brute/resolve timeout defaults to disabled (DNS_*_TIMEOUT=0) with heartbeat progress.
• Secrets: Use secrets.cfg for local overrides or environment variables for CI/Docker (see SECURITY.md).

Example Configuration:

/* Detailed source-code truncated for AI context efficiency. */

DNS resolver guardrails:

• Missing/empty resolver files now fail fast before DNS brute/resolve starts.
• Resolver downloads are configurable with RESOLVER_DOWNLOAD_CONNECT_TIMEOUT, RESOLVER_DOWNLOAD_MAX_TIME, RESOLVER_DOWNLOAD_RETRY, and RESOLVER_DOWNLOAD_RETRY_DELAY.
• DNS_BRUTE_TIMEOUT=0 and DNS_RESOLVE_TIMEOUT=0 disable hard-timeout by default (recommended for very large target sets). Heartbeat progress still prints every DNS_HEARTBEAT_INTERVAL_SECONDS.

DNS_BRUTE_TIMEOUT=4h
DNS_RESOLVE_TIMEOUT=6h
DNS_HEARTBEAT_INTERVAL_SECONDS=20

Full Details: See the Configuration Guide.

🚀 Usage Copy Chapter

reconFTW supports multiple modes and options for flexible reconnaissance. Use the -h flag to view the help menu.

Target Options Copy Section

Mode Options Copy Section

General Options Copy Section

Example Usage Copy Section

1. Full Recon on a Single Target:

   bash Copy

   ./reconftw.sh -d target.com -r

2. Recon on Multiple Targets:

   bash Copy

   ./reconftw.sh -l targets.txt -r -o /path/to/output/

3. Deep Recon (VPS Recommended):

   bash Copy

   ./reconftw.sh -d target.com -r --deep

4. Parallel Mode (Faster, requires more RAM):

   bash Copy

   ./reconftw.sh -d target.com -r --parallel

5. Force Sequential Mode:

   bash Copy

   ./reconftw.sh -d target.com -r --no-parallel

6. Multi-Domain Recon:

   bash Copy

   ./reconftw.sh -m company -l domains.txt -r

7. Ax Integration:

   bash Copy

   ./reconftw.sh -d target.com -r -v

8. Ax Integration with fleet override:

   bash Copy

   ./reconftw.sh -d target.com -r -v 30

9. Full Recon with Attacks (YOLO Mode):

   bash Copy

   ./reconftw.sh -d target.com -a

10. Show Help:

./reconftw.sh -h

11. Force cache refresh:

./reconftw.sh -d target.com -r --refresh-cache

12. Export all report artifacts:

./reconftw.sh -d target.com -r --export all

13. Continuous monitoring (every 30m, 48 cycles):

./reconftw.sh -d target.com -r --monitor --monitor-interval 30 --monitor-cycles 48

14. Rebuild reports only (no scan):

./reconftw.sh -d target.com --report-only --export all

Full Guide: See the Usage Guide.

☁️ Ax Framework Support (previously Axiom) Copy Chapter

reconFTW integrates with Ax for distributed scanning, reducing execution time by distributing tasks across multiple cloud instances.

• Setup: Select reconftw as the provisioner during Ax configuration (axiom-configure).
• Fleet Management: Automatically create and destroy fleets (AXIOM_FLEET_LAUNCH, AXIOM_FLEET_SHUTDOWN) or use an existing fleet.
• Configuration: Set fleet size, region, and name in reconftw.cfg (AXIOM_FLEET_COUNT, AXIOM_FLEET_REGIONS, AXIOM_FLEET_NAME).

Example:

./reconftw.sh -d target.com -r -v

Details: See the official Ax Docs and the attacksurge/ax repo.

💻 Faraday Support Copy Chapter

reconFTW integrates with Faraday for web-based reporting and vulnerability management.

• Setup: Install Faraday, authenticate via faraday-cli, and configure the workspace in reconftw.cfg (FARADAY_WORKSPACE).
• Usage: Enable with FARADAY=true in reconftw.cfg.

🧠 AI Integration Copy Chapter

reconFTW uses AI to generate detailed reports from scan results with the tool reconftw_ai.

• Model: Configurable AI model (e.g., llama3:8b via AI_MODEL).
• Report Types: Markdown or plain text (AI_REPORT_TYPE).
• Profiles: Executive, brief, or bug hunter (AI_REPORT_PROFILE).
• Structured Output: reconftw stores a machine-readable report in ai_result/reconftw_analysis.json.
• Context Controls: Bound input size using AI_MAX_CHARS_PER_FILE and AI_MAX_FILES_PER_CATEGORY.
• Safety Controls: Toggle redaction and strict mode with AI_REDACT and AI_STRICT.

Example:

AI_EXECUTABLE="python3"
AI_MODEL="llama3:8b"
AI_REPORT_TYPE="md"
AI_REPORT_PROFILE="bughunter"
AI_MAX_CHARS_PER_FILE=50000
AI_MAX_FILES_PER_CATEGORY=200
AI_REDACT=true
AI_ALLOW_MODEL_PULL=false
AI_STRICT=false

🗂️ Data Management Copy Chapter

Manage scan data and API keys securely using a private repository.

When ASSET_STORE=true, reconFTW aggregates key findings into assets.jsonl during each run, making it easy to sync only actionable deltas to your private repo.

Makefile Copy Section

Use the provided Makefile for easy repository management (requires GitHub CLI).

1. Bootstrap:

   bash Copy

   export PRIV_REPO="$HOME/reconftw-data"
   make bootstrap

2. Sync with Upstream:

   bash Copy

   make sync

3. Upload Data:

   bash Copy

   make upload

4. Lint / Format Scripts:

   bash Copy

   make lint   # shellcheck for reconftw.sh, modules/*.sh & install.sh
   make fmt    # shfmt with project defaults

5. Run Tests:

   bash Copy

   make test       # unit tests (bats-core)
   make test-all   # unit + integration tests

Manual Copy Section

1. Create a private repository on GitHub/GitLab.

2. Clone and configure:

   bash Copy

   git clone https://github.com/yourusername/reconftw-data
   cd reconftw-data
   git commit --allow-empty -m "Initial commit"
   git remote add upstream https://github.com/six2dez/reconftw
   git fetch upstream
   git rebase upstream/main master

3. Upload Changes:

   bash Copy

   git add .
   git commit -m "Data upload"
   git push origin master

4. Update Tool:

   bash Copy

   git fetch upstream
   git rebase upstream/main master

🧪 Testing Copy Chapter

reconFTW uses bats-core for automated testing.

Install bats-core Copy Section

# macOS
brew install bats-core

# Debian/Ubuntu
apt install bats

# From source
git clone https://github.com/bats-core/bats-core.git /tmp/bats
sudo /tmp/bats/install.sh /usr/local

Running Tests Copy Section

# Unit tests only
make test

# Unit + integration tests
make test-all

# Via the runner script
./tests/run_tests.sh         # unit only
./tests/run_tests.sh --all   # unit + integration

Test Directory Structure Copy Section

tests/
├── run_tests.sh        # Test runner script
├── unit/               # Unit tests (fast, no network)
│   ├── test_sanitize.bats
│   ├── test_utils.bats
│   └── test_validation.bats
├── integration/        # Integration tests (require installed tools)
│   └── test_smoke.bats
├── security/           # Security tests (injection, etc.)
│   └── test_injection.bats
├── mocks/              # Mock tools for offline testing
└── fixtures/           # Shared test data files

Running Security Tests Copy Section

# Test command injection prevention
make test-security

# Or directly
bats tests/security/

Writing Tests Copy Section

Tests use the --source-only pattern to load functions without executing the main script:

#!/usr/bin/env bats

setup() {
    source ./reconftw.sh --source-only
}

@test "sanitize_domain strips invalid chars" {
    result="$(sanitize_domain 'exam;ple.com')"
    [ "$result" = "example.com" ]
}

CI Pipeline Copy Section

The GitHub Actions workflow (.github/workflows/tests.yml) runs on every push and pull request:

1. ShellCheck — lints reconftw.sh, modules/*.sh, and install.sh
2. Unit Tests — runs all tests/unit/*.bats files
3. Integration Tests — installs reconFTW and validates tool availability

Mindmap/Workflow Copy Chapter

Sample video Copy Chapter

🤝 How to Contribute Copy Chapter

See CONTRIBUTING.md for the full contributing guide, including development setup, code style, testing, and PR process.

Quick links:

• Report a Bug
• Submit a Pull Request (target the dev branch)
• Code of Conduct

🔒 Security Copy Chapter

For security policy, secrets management, and vulnerability reporting, see SECURITY.md.

❓ Need Help? Copy Chapter

• Wiki: Explore the reconFTW Wiki.
• FAQ: Check the FAQ.
• Community: Join the Discord server or Telegram group.

💖 Support This Project Copy Chapter

Support reconFTW’s development through:

• Buy Me a Coffee: buymeacoffee.com/six2dez

• DigitalOcean Referral: Referral Link

• GitHub Sponsorship: github.com/sponsors/six2dez

🙏 Thanks Copy Chapter

Special thanks to the following services for supporting reconFTW:

• C99
• CIRCL
• NetworksDB
• ipinfo
• hackertarget
• Censys
• Fofa
• intelx
• Whoxy

📝 Changelog Copy Chapter

See CHANGELOG.md for a detailed list of changes in each release.

🛠️ Development Copy Chapter

Project Structure Copy Section

reconftw/
├── reconftw.sh          # Main entry point (~500 lines)
├── reconftw.cfg         # Configuration file
├── modules/             # Phase modules
│   ├── utils.sh         # Utilities, sanitization, caching, circuit breaker
│   ├── core.sh          # Framework core, logging, lifecycle, health check
│   ├── modes.sh         # Scan modes, argument parsing
│   ├── subdomains.sh    # Subdomain enumeration
│   ├── web.sh           # Web analysis, nuclei scans
│   ├── vulns.sh         # Vulnerability scanning
│   ├── osint.sh         # OSINT functions
│   └── axiom.sh         # Ax/Axiom fleet helpers
├── lib/                 # Pure utility libraries
│   └── validation.sh    # Input validation functions
├── tests/               # Test suite (100+ tests)
│   ├── unit/            # Unit tests (bats)
│   ├── integration/     # Integration/smoke tests
│   └── security/        # Injection prevention tests
├── docs/                # Documentation
│   └── ARCHITECTURE.md  # Detailed architecture guide
└── secrets.cfg.example  # Template for API keys

Running Tests Copy Section

make test          # Unit tests
make test-security # Security tests
make test-all      # All tests
make lint          # Shellcheck
make lint-fix      # Auto-fix with shfmt

Development Workflow Copy Section

# 1. Source without executing (for testing)
source ./reconftw.sh --source-only

# 2. Test individual functions
sanitize_domain "test;domain.com"

# 3. Run health check
./reconftw.sh --health-check

# 4. Dry run to preview
./reconftw.sh -d example.com -r --dry-run

Contributing Copy Section

See CONTRIBUTING.md for development guidelines and docs/ARCHITECTURE.md for technical details.

📜 License Copy Chapter

reconFTW is licensed under the MIT License.

⭐ Star History Copy Chapter

ARCHITECTURE (`docs/ARCHITECTURE.md`) Copy Section

reconFTW Architecture

Overview Copy Chapter

reconFTW is a modular reconnaissance framework. The codebase is organized as follows:

reconftw/
├── reconftw.sh          # Main entry point, argument parsing, orchestration
├── reconftw.cfg         # Main configuration file (all settings)
├── modules/             # Phase-specific modules (what to do)
│   ├── utils.sh         # Utility functions (cleanup, sanitization, caching)
│   ├── core.sh          # Framework core (logging, tools check, lifecycle)
│   ├── osint.sh         # OSINT gathering functions
│   ├── subdomains.sh    # Subdomain enumeration
│   ├── web.sh           # Web analysis and crawling
│   ├── vulns.sh         # Vulnerability scanning
│   ├── axiom.sh         # Ax (formerly Axiom) distributed scanning
│   └── modes.sh         # Scan mode orchestration
├── lib/                 # Pure utility libraries (how to do it)
│   ├── config.sh        # [FUTURE] Modular config loader
│   └── validation.sh    # Input validation functions
└── tests/               # Test suite
    ├── unit/            # Unit tests (bats)
    ├── security/        # Security tests
    └── mocks/           # Tool mocks for testing

Module Load Order Copy Chapter

Modules are sourced in this specific order (defined in reconftw.sh):

1. modules/utils.sh - Base utilities (must be first)
2. modules/core.sh - Framework infrastructure
3. modules/osint.sh - OSINT functions
4. modules/subdomains.sh - Subdomain enumeration
5. modules/web.sh - Web analysis
6. modules/vulns.sh - Vulnerability scanning
7. modules/axiom.sh - Distributed scanning
8. modules/modes.sh - Mode orchestration (must be last)

Data Flow Copy Chapter

Target Input → Sanitization → Phase Execution → Output Files
     ↓              ↓               ↓               ↓
  --domain     sanitize_domain   start_func     Recon/$domain/
  --list       sanitize_ip       [tool calls]   subdomains/
  --cidr                         end_func       webs/vulns/

Key Functions Copy Chapter

Lifecycle Functions (core.sh) Copy Section

• start_func(name, description) - Begin a function, check if already done
• end_func(message, name) - Complete a function, mark as done

Utility Functions (utils.sh) Copy Section

• sanitize_domain(domain) - Sanitize domain input (lowercase, safe chars)
• sanitize_ip(ip) - Sanitize IP/CIDR input
• should_run_deep(count, limit) - Check if DEEP mode should run
• deleteOutScoped(scopefile, targetfile) - Remove out-of-scope entries
• run_command(cmd, args...) - Execute with logging (respects DRY_RUN)

DNS Resolution Functions (utils.sh) Copy Section

All DNS resolution goes through two global wrappers that auto-select between puredns and dnsx based on the network environment:

• _get_local_ip() - Get the primary local IP address (cross-platform). Uses route -n get default on macOS and ip -4 route get 1.1.1.1 on Linux to find the correct interface regardless of naming.
• _ip_is_public_ipv4(ip) - Conservative check for publicly routable IPv4. Treats RFC1918, CGNAT (100.64.0.0/10), loopback, link-local, multicast/reserved as non-public.
• _is_behind_nat() - Detect if running behind NAT (or unknown network) by checking the primary local IP. Defaults to NAT-safe behavior if detection fails.
• init_dns_resolver() - Evaluates resolver selection once per run and caches it in DNS_RESOLVER_SELECTED when DNS_RESOLVER=auto.
• _select_dns_resolver() - Select DNS resolver based on DNS_RESOLVER config. puredns/dnsx force the resolver; auto uses the cached selection from init_dns_resolver() (with a safe fallback if unset).
• _resolve_domains(input_file, output_file) - Resolve a list of domains. Uses puredns (massdns + wildcard filter) on VPS or dnsx (Go net resolver + basic wildcard threshold) behind NAT.
• _bruteforce_domains(wordlist, target_domain, output_file) - Bruteforce subdomains. Equivalent to puredns bruteforce wordlist domain or dnsx -d domain -w wordlist.

Why two resolvers: puredns uses massdns which sends raw UDP packets to thousands of resolvers simultaneously. This floods the NAT table on consumer routers and can kill connectivity. dnsx uses Go's standard net resolver through the OS networking stack, which is NAT-friendly but lacks puredns's advanced wildcard detection.

Permutation Functions (subdomains.sh) Copy Section

• _select_permutations_wordlist(source_file) - Choose wordlist based on PERMUTATIONS_WORDLIST_MODE and subdomain count. In auto mode: full list (849 words) for <=100 subs or DEEP mode, short list (162 words) for >100 subs.
• _run_permutation_engine(source_file) - Run gotator with the auto-selected wordlist.
• _generate_permutation_candidates(source_file, output_file) - Generate permutation candidates with byte-size limit.

Validation Functions (lib/validation.sh) Copy Section

• validate_domain(domain) - Validate domain format
• validate_ipv4(ip) - Validate IPv4 address
• validate_boolean(value) - Validate boolean string

Global Variables Copy Chapter

Required (set before module load) Copy Section

• $SCRIPTPATH - Path to reconftw installation
• $domain - Target domain
• $dir - Output directory for current scan

Configuration (from reconftw.cfg) Copy Section

• $DEEP - Enable deep/thorough scanning
• $DEEP_LIMIT - Threshold for auto-deep mode
• $AXIOM - Enable distributed scanning
• $*_THREADS - Thread counts per tool
• $DNS_RESOLVER - DNS resolver selection (auto|puredns|dnsx)
• $DNS_RESOLVER_SELECTED - Cached resolver selection for this run (set by init_dns_resolver() when DNS_RESOLVER=auto)
• $PERMUTATIONS_WORDLIST_MODE - Wordlist selection (auto|full|short)
• $PERMUTATIONS_SHORT_THRESHOLD - Subdomain count threshold for short wordlist (default: 100)

Error Codes Copy Chapter

Defined in lib/validation.sh:

• E_SUCCESS=0 - Success
• E_GENERAL=1 - General error
• E_MISSING_DEP=2 - Missing dependency
• E_INVALID_INPUT=3 - Invalid user input
• E_CONFIG=8 - Configuration error

Testing Copy Chapter

make test          # Unit tests only
make test-security # Security tests only
make test-all      # All tests
make lint          # Shellcheck errors
make lint-fix      # Shellcheck warnings with context

V2 BETA ANNOUNCEMENT (`docs/V2-BETA-ANNOUNCEMENT.md`) Copy Section

reconFTW v2 beta — `v5.0.0-beta.1`

reconFTW has been rewritten from bash to Go. This is the first public build of
that rewrite, and it ships as an opt-in pre-release.

This document is the canonical announcement for v5.0.0-beta.1. It leads with
what the beta is, and then — at more length, because it matters more — with what
it is not.

What this is Copy Chapter

A single statically-linked binary that replaces the sourced-bash engine. Same
recon philosophy: one command, a complete passive/active/vulnerability picture of
a target. Different machinery underneath:

• Resumable by construction. Every task records a checkpoint keyed by a hash
  of its inputs — config slice, target, wordlist contents. Re-running skips
  completed work.
• Structured output. Artefacts are JSONL with a schema, written atomically,
  scope-checked at the write boundary.
• Bounded concurrency. One scheduler with a global ceiling and per-tool rate
  limits, instead of unbounded background jobs sharing global state.
• Portable. No bash 4.3+ requirement, no GNU-coreutils-on-macOS dance. Linux
  and macOS, amd64 and arm64, plus a fully-static musl build for Alpine.
• TOML config, with a migrator that reads your existing reconftw.cfg and
  tells you loudly about every key it could not map.

The ~100 external tools are still the ~100 external tools. The rewrite changed
the orchestrator, not the toolbox.

What this is NOT Copy Chapter

Three things are open. None of them is a surprise we are hoping you will not
notice; they are the reason the beta exists at all.

1. Parity is measured against ONE target, not three to five Copy Section

Bug-for-bug parity against bash v1 has been measured on one canonical target,
run side-by-side with the outputs diffed. The acceptance criterion, CUT-11,
asks for three to five canonical targets.

That gap matters in a specific way: differences on any other target are
unmeasured, not verified-absent. If v2 returns something bash v1 did not — or
misses something bash v1 found — on a target we have not run, no test currently
in the tree would have caught it. Please report those. A side-by-side diff on a
target we have never run is the single most valuable thing a beta tester can
send us.

2. Throughput is unmeasured Copy Section

No claim is made that v2 is faster than bash v1, or slower. Nobody has
benchmarked the two end to end against the same targets under the same rate
limits, so any number you might expect to see here would be invented.

The architecture changed in ways that plausibly cut either way — a bounded
scheduler replaces unbounded background jobs, which can trade peak speed for
predictability. Which way it actually lands is an open empirical question. If
you run both, the timings are worth reporting; treat any assertion about v2's
speed, including a favourable one, as unsupported until then.

3. `Recon//` is not created — **CUT-08** is open Copy Section

This is the one most likely to break something of yours.

The beta does not create the top-level Recon/<domain>/ tree. The
bash-shape output tree is written under the workspace root as _compat/
instead. Filenames inside it follow the v1 shape; the parent path does not.

So a pipeline that reads Recon/<domain>/subdomains/subdomains.txt finds
nothing under the beta, and a pipeline pointed at _compat/ finds what it
expects.

Whether v2 should also create the top-level Recon/<domain>/ path is an
open decision, tracked as CUT-08 — not an oversight, and not something the
beta is quietly hoping to skip. It is open because the answer depends on how
many people actually script against that path and how, which is exactly what a
beta period is for. If this breaks a script of yours, say so. That report is
the input the decision is waiting on.

Why the tag is `v5.0.0-beta.1` and not `v2.0.0-beta` Copy Chapter

"v2" is the internal milestone name for the Go rewrite. It is not the public
version number, and it never will be.

Public reconFTW tags are already at v4.1. A v2.0.0-beta tag would sort
below the current public release: every tool that orders versions — package
managers, update checkers, humans skimming a releases page — would read it as a
downgrade, and some would refuse to install it over an existing v4.x. So the
rewrite's first public tag continues the public sequence instead of restarting
it: v5.0.0-beta.1.

Expect v5.0.0-beta.2, -beta.3 and so on during the beta period, and
v5.0.0 when it is no longer a beta.

You are not affected unless you opt in Copy Chapter

releases/latest continues to resolve to bash v4.1.

That is not a promise we are keeping by hand — it is how GitHub works. GitHub
never points releases/latest at a pre-release, and the release pipeline marks
this build as a pre-release. The install command documented at the top of the
README resolves through releases/latest, so it keeps serving the bash release
for the entire beta period.

If you do nothing, nothing changes for you. To get the beta you have to name
the tag explicitly — see the opt-in section in the README. The Go binary is
reconftw and the bash entry point is reconftw.sh; they do not overwrite each
other, so you can install the beta alongside your existing setup and fall back at
any time by just running the other one.

Migration details, and every breaking change with a before/after example, are in
MIGRATION.md — start at §0.

Where feedback goes Copy Chapter

Use the v2 beta feedback
issue template (.github/ISSUE_TEMPLATE/v2-beta-feedback.md). It applies the
v2-beta and feedback labels automatically, which is what makes the triage
below possible.

Bugs in the bash release still go to the normal Bug report template. The beta
template is for the Go rewrite only.

Nothing in the template is required. A one-line "this broke" with the output of
reconftw version is already useful.

Triage commitment (CUT-10): a triage pass over open v2-beta issues once a
month, with the summary posted publicly as a repository issue tagged v2-beta —
what came in, what was fixed, what was deferred and why. The first pass happens
the month after the beta ships.

No triage has happened yet, because the beta has not shipped yet. This
section describes a commitment, not a track record; judge it once there is
something to judge.

Known issues already tracked Copy Chapter

Please skip these — they are open on purpose, and reporting them adds noise
rather than information:

• Recon/<domain>/ is not created (CUT-08, above). Do tell us if it breaks a
  script — that specific report is wanted.
• dnscewl and dnstake do not resolve on macOS in this beta.
• dalfox may exit non-zero on --no-spinner if you have dalfox v3.x
  installed; the pinned argument vector targets v2.9.2.

--- METRICS ---

• Files Extracted: 3
• Estimated Token Budget: ~14975 tokens
• Canonical Reference: https://codewiki.google/github.com/six2dez/reconftw