We can evaluate a query based on three parameters:
- Relevance rate vs noise
- Completeness
- Amount of results
Relevancy
100% interesting results without any noise is actually bad news, because it means you are certainly losing some interesting articles.
You should instead go for 20% (1 interesting item VS 4 not relevant ones) or 30 (+-1 on 3)
Completeness
This parameter is hard to define. How many items are you losing because they are not present in your results ?
The more you are looking for completeness, the more results you should process and less relevancy you will get. The opposite is also true.
Number of results
How many items can you read per week ?
When you run your query in a theme, just below the search bar you should get the number of results during the last year period.
Divide this number by 365 and you should have your average daily amount.
Do you have the resources & time to check all these ?
Where to start then ?
Simply start defining how much results per week you can handle with your team. This will be your goal.
Then try to have the best relevancy matching this number of results.
Post your comment on this topic.