Swiftype is a customizable website search and content marketing embedded platform that creates real-time search results based on suggestions and search preference. Swiftype lets you manage results via the user login dashboard and assess search trends in order to identify profitable terms and promote products and services.
Swiftype lets you create a free account and view a demo version before payment is required and also provides online support for sales and technical enquiries. The Swiftype software is completely optimizable for mobile devices which gives your website users complete functionality.
Algolia - Algolia's Search API makes it easy to deliver a great search experience in your apps & websites. Algolia Search provides hosted full-text, numerical, faceted and geolocalized search.
Scrapy - Scrapy | A Fast and Powerful Scraping and Web Crawling Framework
ElasticSearch - Elasticsearch is an open source, distributed, RESTful search engine.
StormCrawler - StormCrawler is an open source SDK for building distributed web crawlers with Apache Storm.
Apache Solr - Solr is an open source enterprise search server based on Lucene search library, with XML/HTTP and...
Apache Nutch - Apache Nutch is a highly extensible and scalable open source web crawler software project.