Have a look at ElasticSearch. I was taken aback by how fast it was performing complicated full text queries in a dataset of 300 million web pages. It was essen

Have a look at ElasticSearch. I was taken aback by how fast it was performing complicated full text queries in a dataset of 300 million web pages. It was essentially a matter of scrape - clean (remove HTML) - index (add to the DB) - search. Easy as that. If you have prior experience with JS and scraping it should be a day’s work for the basics. The documentation is really well done and easy to follow and the API is intuitive.