Sharding a search system by documents gives you several advantages. You can scale horizontally by adding additional indexes for new documents. You can tweak a g

Sharding a search system by documents gives you several advantages. You can scale horizontally by adding additional indexes for new documents. You can tweak a group of documents more easily. E.g. rank wikipedia higher. Assign better hardware (if necessary), higher priority, etc. Performance is also more uniform. Easier to index content at different frequencies. It's also easier for re-indexing content after tweaking algorithms. If you shard a search index by term instead, you will end up with duplicate documents stored in each index that contain the same term. And for a large index, you need to scale up your hardware to handle the term or shard by document within that term anyway.