News

Search optimisation improvements

Related products: Search

Over the past week, we worked on making small improvements to make the sorting of the results more relevant.

We ran an AB test to tweak the search algorithm. The idea was to see if small changes in how we match queries with search results could positively impact search success metrics.

 

The changes we made:

 

  1. The matches inside topic titles,  tags, topic and category titles now all have the same weight. Previously matches at the beginning of the topic titles scored higher in the search result.

 

  1. Proximity of words in matches now have more importance in the result ranking than the order of attributes where the match was found (title, opening post post, best answer, replies ...)

For example:

A end user makes the search query "frontend developer" and the two below topics exist:

 

Topic A

Title "Backend developer"

Opening post "Backend engineering is more fun than frontend engineering"

 

Topic B

Title "Looking for work"

Opening post "I'm a frontend developer looking for work"

 

Our new algorithm will prioritise topic B over topic A in the search results because the two words "frontend" and "developer" are close together in the content. The algorithm will see the fact that the two words are close together as more important the fact the mach is in the opening post rather than the title.

 

The results we got:

 

We run an AB test with the two above tweaks over 8 communities for a month and got the following results on average:

  • Click Through Rate 

The Click Through Rate is the percent of searches where at least one result was clicked on by the end user.

CTR increased from 21.8% to 25.3%
  • The Conversion rate 

We consider the search to be successful (aka to lead to a conversion) when the user spends at least 30 seconds on the clicked topic.

Conversion rate increased from 14.6% to 17.2%

 

Those two changes are now implemented in our new algorithm,  We hope to make further improvements like this along the way to our search algorithm.  

Please share any feedback or ideas you make have related to this :)

@Marion Frecaut this is awesome. We’ve received feedback from some of our community members that search performance within our community can be spotty, and I’ve often noticed the same. I’m excited to monitor these improvements in the coming weeks to see if I can tell a noticeable difference.

 

I’m curious, is there any way we could perform a similar test/comparison via Google Analytics for our own platform? I imagine there might be a way to build a report in GA that could show us the amount of time that users spend on pages where the entrance to the page was the search results page?

 

CC: @erin.brisson 


Hi @Marion Frecaut thanks to the team for the improvements it’s great!

Are there any other criteria that affect the search results (tags for instance?) that we should be aware of?


@victorlacombe  we made no other changes that the one we mention in the article :)


Hello Marion Frecaut,

The changes or an improvement you made really work. Thanks a lot.