CommonCrawl
Common Crawl
CommonCrawl Alternatives
The best CommonCrawl alternatives based on verified products, community votes, reviews and other factors.
Latest update:
-
Scrapy | A Fast and Powerful Scraping and Web Crawling Framework
-
StormCrawler is an open source SDK for building distributed web crawlers with Apache Storm.
-
Get proxy servers featuring IPv4, HTTP/HTTPs, and SOCKS4/5 protocols. Choose from static and rotating IP addresses. ProxyCompass is here to support your business around the clock.
-
Apache Nutch is a highly extensible and scalable open source web crawler software project.
-
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web...
-
YaCy is a free search engine that anyone can use to build a search portal for their intranet or to...
-
Solr is an open source enterprise search server based on Lucene search library, with XML/HTTP and...
-
ACHE is a web crawler for domain-specific search.
-
Self-hosted BitTorrent DHT search engine suite designed for end-users.
-
Turn the web into a database!
-
Search thousands of sites directly from DuckDuckGo
-
Apify is a web scraping and automation platform that can turn any website into an API.
-
🐎 On average 2x faster than Lucene 🔎 Full-text search ⚙️ Configurable tokenizer (stemming available for 17 languages) 🚀 Tiny startup time (<10ms) ⌨️ Natural and Phrase Queries ䷴ Range Queries 🛠 Incremental Indexing 💨 Multi-threaded Indexing 🔩 JSON F…
-
Elasticsearch is an open source, distributed, RESTful search engine.
