Ex-chief architect of Apple’s video & photo applications to be interviewed live on stage

Wednesday, 17 June 2015

Those who paid attention to 'About this software' dialog boxes in the 90s who used Adobe Premiere will recognise the name Randy Ubillos.

He was the lead developer for both Adobe Premiere in the early 90s and Apple’s Final Cut Pro in the late 90s.

By the time Final Cut Pro X was launched in 2011, he was chief architect for Apple’s photo & video applications. Apple included him in many important keynotes. His presentations included demos of new versions of iPhoto and iMovie for iOS as well as iMovie ’09:

Randy retired from Apple in April this year, but he is already making public appearances. Next he’ll be at the Bay Area SuperMeetUp in San Jose on June 26th. The SuperMeetUp is one of a series of events for those who use Macs and PCs for TV and film making.

I'm happy to say that part of his appearance will be an on-stage interview where I’ll ask him about storytelling and what has driven him over the years to make tools that have changed millions of people’s lives. As well as talking about developing applications that went on to be used by professionals to make TV shows and feature films all over the world, he’ll discuss the value of creating tools for everyone else to tell their stories.

Apple opening up on the same day

That same day FCPX Creative Summit delegates will be attending a presentation at Apple's offices about the latest version of Final Cut Pro X

FCPX Creative Summit attendees have the unique opportunity to visit the Apple Campus in Cupertino and hear directly from FCPX product managers! You’ll get a unique perspective on how this video editing software has changed the industry and how it continues to innovate today.

Get an update from Apple Product Managers on the current release of Final Cut Pro X, exciting customer stories, and the thriving ecosystem of third-party software and hardware.

Representatives of Apple's ProApps team have appeared at professional events over the years, but this event marks the first time a large group of post production professionals have been invited to visit Apple.

These days we expect all live presentations to be filmed and made available on the internet within hours. This makes attending live much less essential. Despite Apple opening up more recently, they still ask that Final Cut Pro X team public presentations aren't recorded and put online. Most assume that this is part of Apple’s culture of secrecy. In practice it might be due to the ProApps team wanting to use footage they are not cleared to show online. Footage such as rushes and alternate takes from Warner Bros. recent Will Smith and Margot Robbie feature film which was edited in Final Cut Pro X.

That week is the 4th anniversary of the radical reinvention of Final Cut Pro X. Some Final Cut users hope that Apple’s invitation shows that they will introduce exciting new features as part of a birthday celebration. Although that is possible, even if Final Cut remains unchanged, it is worth visiting the mother ship to learn from those who make the software.  

‘3D Bumpy pixels’ – Oculus Touch haptic feedback

Thursday, 11 June 2015

A few weeks ago I wrote about an unheralded feature of the trackpad in Apple's 2015 MacBook computers: a force-feedback effect that could help users ‘feel’ application UIs - 'Bumpy' pixels: iMovie Apple Force Touch trackpad haptic feedback

Today, in the last five minutes of their pre-E3 press conference [1:23:59 in], Oculus announced ‘Oculus Touch’ – wireless handsets for interacting with virtual reality experiences. One of the features was described as the ability to feel objects in virtual 3D space using haptic feedback:

Touch also includes haptics that developers can use to deliver feedback when you're interacting with objects in the virtual world so that you can actually feel the things you are touching.

Following on from my ‘Bumpy Pixels’ name, maybe Oculus Touch will mean ‘Bumpy 3D pixels.’

‘Heavy’ video clips

When it comes to Mac, PC and TV applications that might mean giving different textures to different UI elements or user content.

For example, if I have a long list of video clips, it would useful to use one hand to quickly scroll them past the other, which would be able to ‘feel’ information about the clips going by. Most clips would feel smooth under my finger tips. The more highly rated a clip was, the more ‘sticky’ it could be made to feel.

Force feedback could also cue you as to the length of a clip - longer files could be made to feel ‘heavier’ as I move them around.

What different textures do your fingertips feel in daily life? What associations would you like to give them in the applications you use?

  • Categories UI

AV Foundation in OS X El Capitan adds movie editing code for developers

Thursday, 11 June 2015

The Apple WWDC 15 session video on AV Foundation shows there are new features for developers who want to manipulate QuickTime movies on the Mac.

Some notes from the video:

[0:38]
New version of AV Foundation provides new classes for applications to edit QuickTime movie files.

AV-Foundation-Session
Click screen shot to go to Apple Developer site

[1:19]
Open QuickTime movie files and perform range-based editing on movies in tracks.

You select a segment of a movie and copy it into some other movie.

Add and remove tracks (tracks in QuickTime can refer to any time-based information, such as subtitles, GPS info, camera metadata)

Associate one track with another - such as saying that this track is the chapter break information for that track.

Add or modify movie and track metadata.

Create movie files and URL sample reference movie files.

[4:06]
'QuickTime movie' means data in a file that conforms to the QuickTime movie file format or ISO base media file formats that were based on QuickTime such as MPEG-4.

[7:15]
Sample data (audio and video content) can be in files separate from the QuickTime movie.

[8:05]
Movies that reference external media are 'fragile' - if the media is deleted or moved, the movie cannot play.

[13:07]
AV Foundation can now update an existing movie file without worrying about the sample data. That means edits, tracks and metadata can all be changed if the samples stay the same - "In place editing" (URLs in the context of AV Foundation are usually describe the location of files in connected storage)

AV-Foundation-Session-2

[19:52]
A example project that shows how an application can combine many gigabytes of footage with metadata.

What does this mean for non-developers?

Good news for post production people who need developers to make applications that support complex workflows, and for those that hope existing tools will get useful new features.

Digitial Rebellion are the makers of Pro Media Tools for Final Cut Pro, Avid and Adobe software.

Before AV Foundation QuickTime libraries in older versions of OS X were able to manipulate QuickTime reference movies. These were small files that were able to represent complex edits of multiple external media files. Reference movies are much simpler to work with that gigabytes of video and audio footage.

Maybe it’s time to do a quick course in Swift so you can make your own post production OS X applications!

OS X for now

Note that the screenshot shows that these new features are OS X Capitan only (the OS X logo in the top right of the screen). Once they're available on iOS, tools for iPhones and iPads will be able to do much more with movie files.

Autodesk likes Final Cut Pro X XML as an interchange format

Thursday, 11 June 2015

Autodesk Smoke 2016 includes improved support for Final Cut Pro X XML export. 

Although Smoke can use other formats for export, the new help file says out that Final Cut's format is the one to use for collaboration:

Use FCP X XML Export when you want to share a sequence with third party applications.

The XML Export generates a simplified sequence that can be used in third party applications for creative editorial, color correction, media management, etc.

There are new Sequence Publish presets available, which output FCP X XML sequences.

  • XML for DaVinci Resolve for Source Grading (ProRes 422 and 24-bit WAVE)
  • XML for DaVinci Resolve for Source Grading (Sequence-only)
  • XML for DaVinci Resolve (ProRes 422 and 24-bit WAVE)

Smoke 2016 can also now conform Final Cut Pro X timelines that include MXF format clips.

Shout out to @finalcutproes for the link!

Brian Mulligan pointed out:

Looks like Autodesk also likes Apple's ProRes 4444 XQ.

The Foundry on high-end post production applications for iOS

Wednesday, 10 June 2015

Don't believe Final Cut Pro X or Adobe Premiere will run on iOS one day? Apple’s Metal for iOS might be the key.

Jack Greasely, Head Of New Technology at The Foundry (makers of feature film post tools such as NUKE and MODO) talking to RGB HQ:

As Metal originated in iOS does this mean that there is the potential to run 'serious' applications, such as MODO, NUKE or even MARI on an iPad one day?

Anything is possible. Having a common graphics API between the two is certainly a start. What is maybe more interesting is a WYSIWYG workflow between IOS and OSX. You could use your Mac to design assets in MARI / MODO / NUKE and then have them display / rendering live on a mobile device looking exactly the same.

Using the iPad's accelerometer, Foundry tools might be able to render graphics as AR overlays.

On stage at Apple's WWDC 15

Jack also appeared on stage at the Apple WWDC conference this week - 10:58 into the video at developer.apple.com. He showed how much The Foundry team were able to achieve in four weeks of adding Metal to MODO, their 3D modelling and animation application.

Airlines to squeeze hand baggage dimensions

Wednesday, 10 June 2015

If you don't want to transport computers and cameras in aeroplane hold luggage, you may need to buy new carry-on bags.

Airlines are changing the rules because too many passengers bring the the maximum size luggage. New worldwide rules:

The recommendation by the International Air Transport Association suggests an “optimal” carry-on size at 21.5 inches (55cm) tall by 13.5 inches (34cm) wide by 7.5 inches (19cm) deep. That’s smaller than the current maximum size allowed by many airlines.

The new recommendation is 56% of the current volume of the British Airways allowance of 22 inches (56cm) tall by 18 inches (45cm) wide by 10 inches (25cm) deep.

Apple's official new mission: “Leave the world a better place”

Wednesday, 10 June 2015

Apple has changed the corporate definition that they include in their press releases:

Apple revolutionized personal technology with the introduction of the Macintosh in 1984. Today, Apple leads the world in innovation with iPhone, iPad, the Mac and Apple Watch. Apple’s three software platforms — iOS, OS X and watchOS — provide seamless experiences across all Apple devices and empower people with breakthrough services including the App Store, Apple Music, Apple Pay and iCloud. Apple’s 100,000 employees are dedicated to making the best products on earth, and to leaving the world better than we found it.

Pity the grammar is a little off. Shouldn't it be "leaving the world better than they found it"?

Compare this new definition with the way Apple described itself last week - which had remained unchanged for over three years - since January 2012: 

Apple designs Macs, the best personal computers in the world, along with OS X, iLife, iWork and professional software. Apple leads the digital music revolution with its iPods and iTunes online store. Apple has reinvented the mobile phone with its revolutionary iPhone and App Store, and is defining the future of mobile media and computing devices with iPad.

Apple's mission was 1995:

Apple Computer, Inc., a recognized pioneer and innovator in the information industry, creates powerful solutions based on easy to use personal computers, servers peripherals, software, online services and personal digital assistants. Headquartered in Cupertino, California, Apple Computer, Inc. (NASDAQ: AAPL) develops, manufactures, licenses, and markets products, technologies, and services for the business, education, consumer, scientific & engineering and government markets in over 140 countries.

A corporate definition that could apply to almost any tech company back then - apart from the mention of PDAs.

I've written previously about how this definition changed between 1995 and 2012

Credit to noticing Monday's change goes to UK-based Mac journalist Lucy Hattersley.

Apple WWDC 2015 and post production

Tuesday, 09 June 2015

Here's my take on the announcements at the Apple Worldwide Developer's Conference 2015.

OS X El Capitan

First came details on the next version of OS X, named El Capitan. El Capitan is one of the mountains in the Yosemite National Park in California. This naming is smiliar to how OS X Mountain Lion came after OS X Lion and OS X Snow Leopard came after OS X Leopard. It signals that this update isn't as big from users point of view. OS X Mountain Lion, Snow Leopard and El Capitan have less big new features that most Mac users will get excited about.

These updates give developers that chance to catch up on new Apple technologies and Apple the chance to introduce innovations that developers can use to do new things. An example could be that if Apple have added more features to AV Foundation, the code that lets application developers (inside and outside of Apple) to do more with movies and audio. Apple Marketing won't tell the public about AV Foundation updates this year, but will hope that new OS X (and iOS watchOS) abilities will mean better AV applications in the coming months.

Spotlight improvements

El Capitan adds natural language searches "The pictures I took last year in London." This should also work for searches based media metadata, and eventually metadata added within applications such as Final Cut Pro X: "Select favourites from the second half of the concert in Manchester featuring the drummer and the bass guitarist shot on a GoPro." Apple haven't yet announced links between media metadata and Spotlight searches, but natural langauge searches in iMovie and Final Cut Pro X would be useful.

In iOS apps can make their content available to Spotlight so that an iPhone- or iPad-wide search will find content in a specific part of the app.

Metal

[20:30 into main keynote] Last year Apple introduced Metal - a way for iOS gaming applications to better access the power of iPhone, iPad and iPad CPUs and GPUs. Last year the emphasis was on how this would make iOS games better. This year Apple had a demo of how well a game worked with Metal on OS X.

Metal has also evolved to speed up more of OS X. In El Capitan Metal improves the speed of Core Animation and Core Graphics. Compared with when these libaries executing OpenGL commands, they now render 'up to' 50% faster on the same hardware.

[21:20] Interestingly for post production people, Apple also said how Metal would speed up 'high performance apps.' It does this by replacing OpenGL graphics code and OpenCL distributed processing code (for sharing work between CPUs).

[21:45] The first developer story of the WWDC keynote was from Adobe. They've been able to speed up After Effects CC rendering by 8x using Metal for OS X. Animations can be rendered in real time. Instead of waiting for Illustrator CC to rerender complex graphics when zooming, now rendering happens in real time. This brings the power of interactive graphic changes - no waiting for rendering in Illustrator.

“We are committed to adopting Metal on our OS X apps. With performance increases of up to 8x, we are excited about what Metal can do for our Creative Cloud users.” - David Wadhwani, Sr. VP & GM, Digital Media, Adobe [22:08] 

David McGavran of Adobe Systems demoed the speed improvements in After Effects CC and Illustrator CC during the ‘Platforms State of the Nation’ session [1:32:15 into this video]. He said that Adobe apps like Premiere Clip already benefit from Metal on iOS.

“Pro app makers are seeing the benefits of Metal like The Foundry and Autodesk. I think were going to see pro users, gamers and all of us benefiting from the performance advantages of Metal” Craig Federighi, Apple [26:55]

AV Foundation

AV Foundation is the part of OS X (and iOS) that applications use to manipulate video and audio. The Editing Movies in AV Foundation developer session has the following description:

Learn how to use the new AVMutableMovie class to modify media files and simplify your editing workflows. See how to support segment-based editing and discover the power of sample reference movies.

The developer documentation for the version of AV Foundation in El Capitan hasn't yet been updated to include AVMutableMovie.

Speed improvement

According to Pedro Santamaría on Twitter, the current version Final Cut Pro X runs faster on his 2012 MacBook Air - as tested using my BruceX benchmark:

This is impressive given that operating system betas aren't tuned for speed. I'll add any update he gives on how much faster the Mac Pro is running El Capitan.

Also…

In each keynote Apple likes to show slides that list ‘too many features to go into right now.’ Some that are relevant to post production are:

File copy resume - could mean that the Finder (or other applications) will resume copying files after a crash or other interruption

Photos editing extensions - could be possible to make changes to photos within video and motion graphics applications. No ‘Movie editing extentions’ yet

Airplay Video - OS X users can already play videos on Apple TVs on the same network, perhaps this mention means that other Macs will be able to play back video.

Should I install OS X El Capitan?

No. Not today.

Unless you are developing Mac software. Although Apple hope it won't cause any problems on your Mac, it wouldn't suprise exprienced developers if a fault wipes all hard drives. At the moment there are reports ranging from "No problems" to "Final Cut Pro X crashes constantly." If you must try it, I suggest you wait for the version Apple releases as part of its public Beta programme.

As regards compatibility, if a Mac can run OS X Yosemite today, it will be able to run the release version of OS El Capitan tomorrow.

Apple ecosystems

Other keynote announcements show that Apple want to maintain and create new ecosystems. As well as supporting big players, their ecosystems include support for small companies and individuals to do well. This makes sense to iOS and OS X developers selling through Apple's App Stores.

Apple Pay

As well as adding more banks and the UK to Apple Pay, Apple mentioned that Square will soon introduce a terminal that will allow anyone to accept ApplePay payments.

News

The News iOS app is a place for syndicated content from news and media organisations. Apple is also considering content from smaller sites and individuals. For now they need to set up an RSS feed of their stories and apply to Apple stating which kinds of content they create:

News brings together high-quality news, magazine, and blog sources in a single beautiful content experience. Whether you’re a major news organization or an individual blogger, you can sign up to deliver your content to millions of iOS users.

...

Topics are created and assigned by Apple’s expert editors and sophisticated algorithms.

News Publishing Guide - Apple

This means that if you can demonstrate that you provide relevant content on a subject of interest to just a few thousand people, Apple's News app might be able to help you connect with the iOS users amongst them. 

As well as being able to monetise your content with 100% from any advertising you include, you optionally get 70% of income from Apple's iAd system.

Apple Music

During the launch of Apple Music, Apple made a point of including unknown musicians. As well as being able to have their music included in Apple Music, Connect helps them maintain their community of fans by adding text, audio, pictures and video to their Apple Music page. Apple Music also takes into account how individuals within families have different music preferences by offering a good value family plan.

At the moment Apple’s Beats 1 worldwide radio station seems aimed at a limited demographic - those interested enough in current and new music to want to hear well chosen music. Those willing to pay for a music subscription. Hopefully Apple will be able to create more advanced radio experiences in future.

Future media ecosystems

This prompts the question of where video, TV and film fits into Apple's plan. If Apple is consistent with what they are about to do with News and Music, people, small groups and large content creators will be able to share their video content in the same way.

If Apple Movies was built in the same way, there would be an iOS application which would provide a single place to consume and discover video content. It would combine human curation with algorithms that would learn your preferences. If you are a producer, Apple would provide simple tools to make your content available (News) and build audiences (Apple Music Connect).

A similar ecosystem could be built around podcasts - perhaps supported by a worldwide Apple radio station that features presenters and excerpts from podcasts, audio books and radio drama.

The Apple Music family plan prompts me to point out that some media - music, TV, movies - is fun to share with others. Perhaps Apple should find a way for software to create combined streams that would entertain groups of people: A family playlist for everyone until 10pm, then content for the parents. "Stick with this 15 minute short that only your brother likes, something you really like will be on next." This could work for any group of people - including groups not gathered in one place: hanging out across the internet.

If your media has to fit in a shared customised stream, the methods you use to tell stories might change.

If Google and Apple will eventually meet in a battle of software on hardware vs. software in the cloud, Apple might need to change the field of battle. If hardware devices become so ambient as not needing to be associated with an individual - apart from an earpiece running Siri - Apple's hardware integration edge will become irrelevant. What survives will be Apple's ability to maintain and support media ecosystems. 

Exploring 360° video with Final Cut Pro X

Monday, 08 June 2015

I saw an interesting music video today from Bjork - another 360° ‘VR’ video, which prompted me to find out how to create 360° motion graphics using Final Cut Pro X.

If you view this video with the Chrome browser on YouTube, you can drag within the video to look around - left and right, up and down:

Use the cog settings control to increase the resolution to 2160p-4K.

I made this video by scaling a still equirectangular panorama down to 4320x2160 and importing it into a new 25p Final Cut Pro X project.

I then overlaid text on top, animating some of it.

Here is the 'flat' video - scaled down to HD from 4K:

Where I wanted text to appear 'behind' the initial position - where the left and right edges of the panorama meet, I created two copies of the same title, so it wouldn't be cut off by the edge.

I exported the video as an H.264 encoded mp4 scaled to 3840x2160 with a data rate of 30 Mbps (more on YouTube's video upload specs).

For YouTube to recognise that this 4K video was designed for 360° video, I opened the Final Cut output file with Google's 360 Video Metadata application. The simple UI has a single button:

Screenshot of Google's 360 Video Metadata application UI

I clicked 'Inject and save' and saved a new file which I uploaded to YouTube.

Looks like I made my graphics too large, but if you avoid moving too far up or down on your background, overlaid graphics should work OK.

 

Visit the Apple Campus for a Final Cut Pro X presentation on June 26

Thursday, 04 June 2015

It seems that after years of very little access, Apple is opening up a little more. On June 26 members of the public will be visiting Apple's offices to get an update on Final Cut Pro X. The kind of access that usually granted only to a favoured few is available to attendees of Future Media Concepts' FCPX Creative Summit:

FCPX Creative Summit attendees have the unique opportunity to visit the Apple Campus in Cupertino and hear directly from FCPX product managers! You’ll get a unique perspective on how this video editing software has changed the industry and how it continues to innovate today.

Get an update from Apple Product Managers on the current release of Final Cut Pro X, exciting customer stories, and the thriving ecosystem of third-party software and hardware.

Representatives of Apple's ProApps team have appeared at professional events over the years, but this event marks the first time a large group of professionals have been invited to visit Apple.

Future Media Concepts is a company that runs training courses in media production in the USA, Canada and online. They also organise post production events such as the Editors Retreat, After Effects World and the Creative Cloud Masters conference.

Livinia Smith, Future Media Concepts' event marketing manager for the FCPX Creative Summit says that after running events for Adobe and Avid users for many years, recent improvements in Final Cut prompted them to turn to Apple's software. The weekend of June 26-28 is just over four years since Final Cut Pro X was launched. Did that factor into the timing? "Future Media Concepts approached Apple about hosting an event dedicated to this platform. We both decided the date for the conference" says Smith.

Smith went on "Regarding the visit to the Apple Campus, when we pitched the idea to Apple, they saw value in directly interacting with this community of FCP users and they agreed to host a talk with the conference attendees in a lecture room at Apple."

Peeking out over the parapet of a besieged castle

Although Final Cut Pro X and its companion applications Compressor and Motion have been very successful over the years, Apple hasn't seen the need to publically involve itself with the user community. Compare their activities with those of Adobe and Avid - companies whose video editing applications were the traditional competitors of Final Cut Pro.

As well as constantly updating their websites with Premiere Pro and Media Composer case studies, their online activities include blog posts, tweets and Facebook updates with named staff members. They run support forums that feature contributions from software engineers. If a small user group somewhere in the USA gets in touch with Adobe to say they're organising a meeting about Premiere Pro, there's a good chance product manager Al Mooney will appear to give an entertaining presentation on his baby.

In recent years parts of Apple have been interacting a little more with the wider world. For example last year's launch of Swift, a new programming language for developing OS X, iPhone and now Watch apps was a big surprise. Apple going on to launch a programming blog on Swift is even more of a surprise. 

Anyone who visits the online forums discussing Adobe Premiere Pro CC, Avid Media Composer and Final Cut Pro X know that the harshest critics of most applications are those who use them every day for their livelihood. The combination of a long-established culture of Apple not sharing much information and the rabid nature of online power user debate means that it will be hard for the Final Cut Pro X team to change how they interact with the wider Final Cut community.

On the way towards a professional application community

Hopefully the ProApps team will be able to more directly support a Final Cut Pro X community. Online support would include

  • A buyers guide for third-party hardware and software
  • A consultants network
  • Continually updated training materials
  • A job board for employers and job seekers
  • Forums and discussion groups where the developers of the application itself can take part
  • Regular conferences so people can learn from each other and network

The majority of Final Cut users are individuals don't need to set up complex workflows and never need to call on consultants. However, knowing that there is a robust community standing by makes trying a new complex application that bit less daunting.

Although this kind of community might seem at odds with the way Apple works, they have a model of their own they can look to: FileMaker. FileMaker is Apple's professional database system. The FileMaker website has all the features I listed above.

It is interesting that Apple refers to FileMaker as a platform - as it is made up of an authoring tool, a server product and software that runs on Macs, PCs, iOS devices and in web browsers.

Perhaps the ProApps applications might end up as a platform/ecosystem too. I hope June's FCPX Creative Summit is a step on the way.

Disclosure: I'm happy to say I'm presenting a couple of sessions on Apple Motion at the Summit.