Scrapy - Scrapy | A Fast and Powerful Scraping and Web Crawling Framework
import.io - Import. io helps its users find the internet data they need, organize and store it, and transform it into a format that provides them with the context they need.
StormCrawler - StormCrawler is an open source SDK for building distributed web crawlers with Apache Storm.
Octoparse - Octoparse provides easy web scraping for anyone. Our advanced web crawler, allows users to turn web pages into structured spreadsheets within clicks.
Apache Nutch - Apache Nutch is a highly extensible and scalable open source web crawler software project.
Apify - Apify is a web scraping and automation platform that can turn any website into an API.