Screaming Frog - Screaming Frog is search engine marketing agency offering search engine optimisation and pay-per-click advertising services.
SEO Crawler - A web-based crawler with real-time crawl feedback.
SEMRush - All-in-one Marketing Toolkit for digital marketing professionals.
DYNO Mapper - Create Sitemaps with the DYNO Mapper Visual Sitemap Generator.
SEOmator - SEO website analysis & audit tool
Scrapy - Scrapy | A Fast and Powerful Scraping and Web Crawling Framework