Google Publishes a Post on How They Evaluate and Improve the SERP

Google talks about how they go through an evaluation process for implementing features in search so that people around the world continue to find Google useful for whatever they’re looking for.

Their research team: whose job it is to talk to people all around the world to understand how Search can be more useful. They invite people to give them feedback on different iterations of their projects and they also do field research to understand how people in different communities access information online.

Search quality raters: They  have a group of more than 10,000 people all over the world they call “search quality raters.” Raters help them measure how people are likely to experience their results. They provide ratings based on their guidelines and represent real users and their likely information needs, using their best judgment to represent their locale.

They also talk about how raters work and their ratings are not used directly for search ranking, etc. Last year, they did more than 383,605 search quality tests and 62,937 side-by-side experiments with their search quality raters to measure the quality of their results and help them make more than 3,600 improvements to their search algorithms.

In-product (live) experiments: They test how people interact with new features through live experiments. . To test a change, they will launch a feature to a small percentage of all queries they get, and then they look at a number of different metrics to measure the impact. In 2019, they ran more than 17,000 live traffic experiments to test out new features and improvements to Search.