Justin Bieber’s new song is number in the UK. The New York Times has made an 8-minute video about how “Where Are Ü Now” was made.

It was conceived from the start as a video that works in more than one aspect ratio. Human interface experts NiemanLab have written a ‘making of’ abut this ‘making of’:

Unsurprisingly, the combination of Bieber and The Grey Lady turned some heads. But the Times’ video is interesting for another reason — it was designed from the beginning to be as compelling viewed vertically as horizontally. In a world where young people are watching more video on smartphones than on TV screens, making a video work in both aspect ratios can help it reach a broader audience.

This was an aesthetic as well as technical problem – how to combine filmed footage with motion graphics overlays that look good both on a TV and a vertically-held phone.

It is worth reading, but perhaps post-production people should consider whether timelines should have a fixed aspect ratio. They already don’t have a fixed resolution.

Not the ‘where’ of a video element – the ‘what’

I suggest that elements for future videos may be exported as layered movies. It will be up to the playback device or software to how to show the elements so they work in the aspect ratio needed for each viewing.

This already happens for audio in Final Cut Pro X. Instead of defining the speaker through which audio should be heard, all audio is given a ‘role.’ This metadata can then be used by broadcasters and distributors to determine which audio should be played back – depending on context.

The standard audio expected for UK TV production expects programmes to include a stereo mix, a surround mix, a stereo audio description (in which a voiceover during gaps in dialogue describes what happens on screen), music and effects only and alternate languages.

Imagine if programmes also had layers marked as Base video,’ ‘Signs and information in English,’ ‘Behind the scenes information,’ ‘Purchasing information,’ and ‘Signs and information in an alternate language.’ In the case of signs and text, this is how Pixar generates its movies.

In the case of the New York Times video, the motion graphics elements would be included in a separate layer which would be composited in different positions or even angles depending on the orientation of the playback device.

The answer to the problem of vertical video is to make sure videos look good when viewed at any aspect ratio.

That means editing applications will be able to playback the same content at multiple aspect ratios – much like page layout applications eventually added features which allowed designers to work with multiple aspect ratios for magazines and adverts.

To support multiple aspect ratios video makers will need tools that let them define what a video element is – the playback device can then determine the best place to play it back. Even if that is a second screen…