Need CLI tool that can handle large txt files and another tool that can conduct a lightning fast search of disk files.

1

Xapian

For fast searching, it usually requires indexing the files in question. There are a number of text-file indexing solutions, many of which use xapian, sphinx, or lucene/solr under the hood. Based on conditions (watching files/directories, cron jobs, new-mail triggers, etc), they'll add/remove files to the index, and you can then use a corresponding command to compose queries across that data. If it's indexed, it can be fast; if you have to scour the drive and open every file each time that you want to do a search, it will be a lot slower. I know the Gnome project had Tracker which provided such functionality. There was also recoll.

#Full-text Search #Custom Search Engine #Custom Search 7 social mentions

2

Apache Solr

For fast searching, it usually requires indexing the files in question. There are a number of text-file indexing solutions, many of which use xapian, sphinx, or lucene/solr under the hood. Based on conditions (watching files/directories, cron jobs, new-mail triggers, etc), they'll add/remove files to the index, and you can then use a corresponding command to compose queries across that data. If it's indexed, it can be fast; if you have to scour the drive and open every file each time that you want to do a search, it will be a lot slower. I know the Gnome project had Tracker which provided such functionality. There was also recoll.

#Custom Search Engine #Custom Search #Search Engine 17 social mentions

3

Apache Lucene

For fast searching, it usually requires indexing the files in question. There are a number of text-file indexing solutions, many of which use xapian, sphinx, or lucene/solr under the hood. Based on conditions (watching files/directories, cron jobs, new-mail triggers, etc), they'll add/remove files to the index, and you can then use a corresponding command to compose queries across that data. If it's indexed, it can be fast; if you have to scour the drive and open every file each time that you want to do a search, it will be a lot slower. I know the Gnome project had Tracker which provided such functionality. There was also recoll.

#Custom Search Engine #Search Engine #Custom Search 7 social mentions