You have 2008 script tags on that page. I know why but you could have at least had them remove themselves from the DOM after they were done adding their content

You have 2008 script tags on that page. I know why but you could have at least had them remove themselves from the DOM after they were done adding their content. > Sadly one cant push bits to files but for a new article bits are to be appended to each data file eventually. You can it is just that each of the files has to be padded out to a multiple of full bytes. Just read the last byte and OR in the new bit at its offset. The method you are using is called q-gram filtration. The main optimization you could do is to use the specificities of the grams to prioritize the ANDing so that the most specific grams are chosen first. This can allow early stopping for misses. You can also use multiple threads to read and AND in parallel after a AND tree order has been decided from the specificities. I would also remark that it is possible to implement approximate matching or error tolerant search with q-grams. That is to rank results according to edit distance or Levenshtein distance from the query. It is also possible to do regular expression search over them as Russ Cox explains how he implement the Google Code Search: Regular Expression Matching with a Trigram Index or How Google Code Search Worked https://swtch.com/~rsc/regexp/regexp4.html https://swtch.com/~rsc/regexp/regexp4.html This is not specific to 3-grams. It can be made to work with any gram size or even indexes consisting of variable size grams.