Netpeak Spider - A desktop tool for fast and comprehensive technical audit of the entire website.
Scrapy - Scrapy | A Fast and Powerful Scraping and Web Crawling Framework
Screaming Frog SEO Spider - The Screaming Frog SEO Spider is a small desktop program (PC or Mac) which crawls websites’ links...
Apify - Apify is a web scraping and automation platform that can turn any website into an API.
Pulno - SEO Audit and Website Analysis. Pull up your SEO.
Portia - An open-source visual scraping tool that lets you scrape the web without coding, built by Scrapy...