VR, Final Cut Pro X and Apple WWDC17
This week at their Worldwide Developer Conference in San Jose, Apple are making announcements about future products and services. They are also giving presentations for developers about macOS, iOS, tvOS and watchOS.
Over the week I’m hoping to see updates relevant to VR and Final Cut Pro X at WWDC17.
VR
To mix reality (make it seem that graphics is interacting with the real world) it is useful to be able to record the distance of objects from the camera. This means graphics can be seen to be hidden behind objects caught on camera. So I’m hoping for depth maps recording by Apple’s device cameras ( Apple’s patent on Depth mapping based on pattern matching and stereoscopic information ) – as well as the iPhone, this would be useful for iPad, Macs and Apple TVs.
If Apple want to help applications deal with mixed reality, it would be very useful if there was at lease one new flavour of ProRes, one that records depth maps too. ProRes 44444 anyone? While we are on the subject, context sensitive video overlays are likely to become more popular in coming years, so it would be very useful if more PreRes flavours included alpha channel information: ProRes 4224 and 4224 Proxy would be a start!
Siri Speaker room sensor. If Apple to release a high-quality speaker that customises the sound it produces based on position relative to a screen and depending on the shape of the space it is in, those sensors could also be used for room-scale VR and MR. The speaker could help detect where VR/MR headsets being used are in 3D space.
The problem with phone screen resolution and VR headset resolution is that VR resolution on an iPhone is overkill, but iPhone resolution is a little low for high-quality VR. For when Mixed Reality becomes popular, you won’t be able to see what is in the real world if an iPhone is in the way. However, the GPU in iPhones is very powerful. If it could be kept in a pocket (to provide relative position in 3D space) while providing GPU power to an external device via Lightning (or Thunderbolt 3), then that lightweight device can be that much lighter and need less battery. So look for iPhone GPU power for external devices via lightning or thunderbolt.
Final Cut Pro X
As regards Final Cut Pro X, look out for updates to Core Media and AV Foundation in macOS (and iOS if Final Cut Pro X/iMovie is to move to the iPad). Although some say that the ProApps team don’t use developer frameworks, you can see how the OS team and the ProApps team are thinking about media in modern OSes with the Core Media and AV Foundation updates. 2015 AV Foundation presentation, 2016 AV Foundation presentation.
For those interested in cloud-based Apple services for pros, also look out for updates on HLS – HTTP Live Streaming and iCloud. 2016 HLS presentation.
The good news is that Apple will publish videos and transcripts of all the WWDC 2017 presentations. I’ll update this post with links to relevant videos as they become available.