ScrapeStorm
Agenty
Octoparse
Screen Scraper
Diggernaut
Crawlbase
artoo.js
eScraper
Heritrix
Scrapy
StormCrawler
Apache Nutch
Apache Solr
Mixnode
CommonCrawl
ElasticSearch
ScrapeStorm is an AI-Powered visual web scraping tool，which can be used to extract data from almost any websites without writing any code. It is powerful and very easy to use. You only need to enter the URLs, it can intelligently identify the content and next page button, no complicated configuration, one-click scraping. ScrapeStorm is a desktop app available for Windows, Mac, and Linux users. You can download the results in various formats including Excel, HTML, Txt and CSV. Moreover, you can export data to databases and websites.
ScrapeStorm HeritrixThe software can collect price information on various e-commerce websites, and can help me stably grasp the price changes of competing products in marketing, so that I can respond in a timely manner. It can also collect advertising flow data to further analyze subsequent advertising arrangements.
This is a general-purpose web page collection software, which can adapt to the collection of most websites. The matching degree is relatively high, which is indeed a great advantage compared to other software. And it also has intelligent recognition, which is more friendly to novices. But I still prefer to use the flowchart mode, and the upper limit of operation is relatively high.
It is rare to find a software that supports audio downloading and collection. It allows me to download songs on the webpage in batches without clicking one by one. super convenient!
Agenty - Machine Intelligence, Web scraping tool
Scrapy - Scrapy | A Fast and Powerful Scraping and Web Crawling Framework
Octoparse - Octoparse provides easy web scraping for anyone. Our advanced web crawler, allows users to turn web pages into structured spreadsheets within clicks.
StormCrawler - StormCrawler is an open source SDK for building distributed web crawlers with Apache Storm.
Screen Scraper - Screen Scraper is a web data extraction tool that allows users to extract data from any website according to their requirement and save it online or download it.
Apache Nutch - Apache Nutch is a highly extensible and scalable open source web crawler software project.
