In the course of a strategy/IT consulting project, we developed a PoC for a face recognition AI/IoT device.

Toward objective measurement unaffected by how cooperative panelists are

TV ratings are often measured by distributing survey devices that respond to the infrared signals of remote controls to the households being measured (hereinafter “panelists”). However, the accuracy of this method depends heavily on how cooperative the panelists are. For example, with a method in which panelists tell the survey device who is watching TV by pressing the colored buttons on the remote control, “careless” panelists simply press the colored buttons at random. To avoid such cases, it was necessary to install a camera device on the TV and use face recognition and pose estimation to objectively measure when, by whom, and how the TV was being watched, regardless of how cooperative the panelists were.

A PoC for an AI/IoT camera device that accurately measures viewing

Fast individual identification and pose estimation using state-of-the-art deep learning models

Using high-accuracy individual identification and head pose estimation algorithms, we developed a device that accurately measures who was facing the TV, when, and to what extent. To enable second-by-second measurement, we incorporated techniques throughout that make fast estimation possible.

A fast and reliable gaze determination algorithm

We developed an algorithm that determines, from head pose estimation, whether or not a person was watching the TV (a gaze determination algorithm). Balancing accuracy and speed, we eliminated needless complexity and devised a simple, practical algorithm.

A face registration app that ensures the accuracy and speed of face recognition

Registering appropriate face photos is essential to ensuring the accuracy of face recognition. To have a wide variety of panelists carry out this registration reliably, the face registration app required a range of design considerations. We conducted multiple tests in-house and developed an approach that strikes a fine balance between the burden on panelists and the maintenance of accuracy.