This week, the US Patent & Trademark Office released a patent application from Apple covering a possible future generation of Group FaceTime conferences for iPhone, iPad, Macs and their Mixed Reality Headset. This future release will integrate the can technology into their hardware camera systems in a way that will uniquely engage participants directly.
Nowadays, when a participant speaks in a FaceTime call, his or her image is magnified on the screen. But the person they may be addressing continues to stare at the screen statically like all the other participants. In this future version of FaceTime, the speaker and/or presenter can focus on one person in a conversation and the target participant, captured via gaze, turns their head towards the speaker. In some examples, such as wearing a mixed reality headset, this is done virtually.
Lyrically, this is an extremely complex patent application to keep your head up and so we skip straight to patent on FIG. 6A which brings us to the core of the invention. Here we see ‘User A’ on his MacBook looking at and speaking to ‘User B.’ This version of FaceTime supports ‘Gaze Controls’.” Once the MacBook’s camera captures the gaze direction and which FaceTime participant “User A” is targeting, the target “User B” image will be seen as if they were turning their head to look at and address ‘User A’.
Apple’s patent FIG. 6G below gives an overview of how this could play out on other devices.
Apple’s patent FIG. 10a below presents an unwritten ‘computer system’ (which could be an iMac or a much larger system for teachers and teachers in virtual classrooms) with 8 cameras around the screen. This is designed to increase the accuracy of the user’s gaze to a conference call participant.
Depending on which row the speaker is targeting, a different camera is activated to better capture the gaze accuracy within that row. The software then positions the head of the specific user that is being targeted so that it appears as if they are turning their head towards the speaker who is specifically addressing them.