Site-Crawler
Fast multi-threaded web crawler for site mapping and technical audits
Specifications
- Pricing
- open-source
- Price range
- 0
- Billing
- one-time
- License
- MIT
- Platform
- cli, api
- Languages
- en
- Target user
- intermediate
- Company size
- small-team
- API
- Yes
- Data export
- Yes
- Last update
- 2025-07-22
Tags
Related tools
- 🔎 Sitechecker
Real-time website monitoring and technical audit platform for ongoing SEO health tracking
- 🏗️ Netpeak Spider
Personalized crawler for day-to-day SEO audits and comprehensive technical site analysis
- 🔧 SEOnaut
Comprehensive open-source SEO audit tool for deep site scans
- 📉 Broken-Link-Checker
Find broken links on websites with this highly configurable CLI tool
- 🔗 Link Checker
Verify internal and external links for broken connections and redirect loops
- 🌑 Blacklight-CLI
Identify ad trackers and privacy issues that could impact SEO rankings