Patent: On How Google May Rank Images using Machine Learning Models

Bill Slawski breaks down a newly granted Google patent Ranking Image Search Results Using Machine Learning Models that sheds some light on how Google might rank images based on a combination of the query used, features of the images and features of the landing page using various Machine Learning models.

As always I highly recommend you go check out Bill’s breakdown for the full details, this is just a short summary:

Features Extracted from the Image Search Query

• Language of the search query
• Some or all of the terms in the search query
• Time that the search query was submitted
• Location from which the search query was submitted
• Data characterizing the user device from which the query was received
• So on

Features Extracted From the Image:

• These features can include vectors that represent the content of the image.

Features Extracted From the Landing Page: 

• The date the page was first crawled or updated
• Data characterizing the author of the landing page
• The language of the landing page
• Features of the domain that the landing page belong to
• Keywords representing the content of the landing page
• Features of the links to the image and landing page such as the anchor text or source page for the links
• Features that describe the context of the image in the landing page
• So on

Features Extracted From The Landing Page That Describes The Context of the Image in the Landing Page

• Data characterizing the location of the image within the landing page
• Prominence of the image on the landing page
• Textual descriptions of the image on the landing page
• Etc.

Check out Bill’s post on more info on how it all comes together and how the initial ranking scores are adjusted.