Google Patent About Assistant Listening for Semantic Entities

The patent called Systems and Methods for Identifying and Providing Information about Semantic Entities in Audio Signals is about how Google will use an approach that uses machine learning to identify semantic entities in audio signals that are heard by a user, to display those semantic entities to the user, and to provide additional information to the user about those semantic entities. These entities are more than just the people talking in audio signals.

The patent starts out by telling us that is in an “always-on operating mode” to enable it to “identify semantic entities that a user hears from a variety of audio signals”, such as:

• Media files playing on the computing device
• In-person conversations or other audio the user overhears in his/her environment
• Telephone conversations
• Etc.

In addition to identifying semantic entities (e.g., people, places, locations, etc.) in an audio signal heard by the user, a device could also display those semantic entities to the user, with a user interface displayed on a computing device. That user could also then select that particular semantic entity, using the user interface to access supplemental information about the selected semantic entity.