3 ms·
Thank you for giving us helpful feedback, and taking the time to try out Andi and give it a good workout! Your example questions are a great chance to explore h
by jedwhite 5y ago
Thank you for giving us helpful feedback, and taking the time to try out Andi and give it a good workout! Your example questions are a great chance to explore how you can get better results with language models and semantic search by asking questions with more detail and specifics. With a search engine like Andi, better questions produce much higher quality results, and it's worth looking at some examples. I've broken the response into two parts: 1. visuals and 2. answers.
1. Site visuals:
We know we have a lot of work to do on the UX. We've been getting a ton of suggestions and ideas for improvements and we're working on this.
As a small team with one developer, we're covering a lot of ground, and I'm a back-end hacker rather than a front-end guy. So it's fair to say we've focused on the search results and conversational side first. But we're getting a ton of great feedback and suggestions to improve the UX, and we're working to act on that and iterate the UI to improve it as quickly as we can.
It's fascinating how polarized views on the chat interface are. It's almost like we have two totally different products that we're receiving feedback about. People are split pretty evenly down the middle about using a chat interface vs search box with a list of blue links.
It reminds me a little of when gmail first came out, and there was a similar split among people I knew between "it should look like Outlook" vs "finally something new in email".
Our users who like it tend to mention the visual results and views, and reader view. Others dislike it, and that feedback tends to focus on the visual aesthetic - colors, styling, layout. We know we have a lot to do there.
2. Question answer and asking great questions to get great results:
Thank you for giving us feedback on the answers too. We don't log searches or answers, so we rely on feedback like this to improve them.
Let me step through them in a little detail, because the examples you gave are great pointers for how to get better results with Andi, and when using searches that use natural language interpretation rather than keyword search more generally.
With the covid cases question, this would have been a knowledge graph intent, and the answer there was sourced from the Wolfram|Alpha API. Technically, the answer is correct because as of the latest data, there have been 16.5m covid cases in the UK so far.
But it wasn't the answer you were looking for. With NLP/language models and semantic searches, a more specific question can help you to get the right answer. This is Andi's answer with a more specific question.
Q: How many confirmed cases of covid-19 have their been in the UK in the last 7 days?
A: I found this deep answer on data.gov.uk: 599,244 Total number of people tested positive reported in the last 7 days (19 March 2022 - 25 March 2022) There has been an increase of 47,046 (8.5%) compared to the previous 7 days.
The NBA one question is another good example to look at also.
With natural language questions, more information produces better results, because the confidence score is used to determine whether to give an answer or web results.
In this case, the first place for the conference hasn't been decided so confidence was likely low to offer an answer, hence the snippet from the last round. Lots of work to do there.
Given that the Eastern Conference hasn't been decided yet, Andi will do better if you give it more specific information to work with in plain language.
So asking:
Q: Which team is currently the favorite to win first place for the NBA's Eastern Conference seed position?
A: I found this deep answer on cbssports.com: It is difficult to say which team is currently the favorite to win first place for the NBA's Eastern Conference seed position as six different teams have held control of the No. 1 seed at some point this season.
So asking a question with more detail and specifics can produce really great answers even on difficult topics.
Us humans are pretty good at figuring out extra implied information from context to clarify questions. If you'd asked me this question, I would have asked for more information to be able to research a good specific answer with high confidence. With Andi, more specific queries do well. Without specific information, it will tend to provide web results and a snippet.
With NLP, language models and semantic search, asking more detailed questions with specifics (as you might a researcher) produces much higher quality answers.
Thank you for the examples, and giving me the chance to explain this in some more depth.