2017 iPhone 3D face scanner for ID unlock fast enough for video depth capture

Monday, 21 August 2017

In reports coming from Asia it is rumoured that one of the iPhones Apple plans to announce later this year will have a face scanner that will allow users to unlock their phones without using a TouchID fingerprint scanner:

The Korea Herald yesterday:

The new facial recognition scanner with 3-D sensors can deeply sense a user’s face in the millionths of a second. Also, 3-D sensors are said to be adopted for the front and rear of the device to realize AR applications, which integrate 3-D virtual images with user’s environment in real time.

This is interesting news for those who want to work with footage that includes depth information. The kind of camera required to quickly distinguish between different faces probably needs to sample the depth in a very short space of time to counteract phone and face movement.

Apple’s ARkit and Metal 2 are based around a 90fps refresh rate, so the sampling rate for both cameras is more than enough for VR and AR experiences.

Another tid-bit is that the new Apple phone is said to be able to recognise a person’s face even when the phone is lying on a table. That says to me that the volume that the phone's depth sensor will be able to capture will be much wider than the light sensor in the phone’s camera. The ‘angle of view’ required for a sensor in a phone lying on a table to read a nearby face would have to be at least 120º.

Now we need applications that can help people use this depth information in creative ways. At the very least, it will mean there will be no need to use green and blue screens to separate people from backgrounds. All objects further than a specific distance from the camera can be defined as transparent for production purposes.