Articles tagged with: UI

‘3D Bumpy pixels’ – Oculus Touch haptic feedback

Thursday, 11 June 2015

A few weeks ago I wrote about an unheralded feature of the trackpad in Apple's 2015 MacBook computers: a force-feedback effect that could help users ‘feel’ application UIs - 'Bumpy' pixels: iMovie Apple Force Touch trackpad haptic feedback

Today, in the last five minutes of their pre-E3 press conference [1:23:59 in], Oculus announced ‘Oculus Touch’ – wireless handsets for interacting with virtual reality experiences. One of the features was described as the ability to feel objects in virtual 3D space using haptic feedback:

Touch also includes haptics that developers can use to deliver feedback when you're interacting with objects in the virtual world so that you can actually feel the things you are touching.

Following on from my ‘Bumpy Pixels’ name, maybe Oculus Touch will mean ‘Bumpy 3D pixels.’

‘Heavy’ video clips

When it comes to Mac, PC and TV applications that might mean giving different textures to different UI elements or user content.

For example, if I have a long list of video clips, it would useful to use one hand to quickly scroll them past the other, which would be able to ‘feel’ information about the clips going by. Most clips would feel smooth under my finger tips. The more highly rated a clip was, the more ‘sticky’ it could be made to feel.

Force feedback could also cue you as to the length of a clip - longer files could be made to feel ‘heavier’ as I move them around.

What different textures do your fingertips feel in daily life? What associations would you like to give them in the applications you use?

  • Categories UI

'Bumpy' pixels: iMovie Apple Force Touch trackpad haptic feedback

Monday, 16 March 2015

Apple has updated iMovie 10.0.7 to provide context-specific haptic feedback for those using a Force Touch trackpad.

As part of their March 9, 2015 event Apple announced a new kind of trackpad for their MacBook computers. Instead of registering clicks using a switch, the new trackpad is able to recognise a range of pressures. The Force Touch trackpad can detect a light touch for when the user wants to move the cursor without clicking and dragging, a heavier touch for when the user wants to click or drag, and a heavier touch - a 'force click' - which is used for shortcuts.

As this new trackpad has no click switch, it is hard for users to know how hard they are pressing without physical feedback. They need to be able to feel the difference between moving the cursor, clicking a UI object and force touching a part of an application. The Force Touch trackpad includes a 'Taptic Engine' - tiny magnets that move the trackpad in such a way that they feel as if the trackpad has flexed downwards.

An Apple support document lists some examples of shortcuts accessible by force clicking:

  • Link previews: Force click a link in Safari or Mail to see an inline preview of the webpage.
  • File icons: Force click a file icon to see a Quick Look preview of it.
  • File names: Force click a file name in the Finder or on your desktop to let you edit the file name.
  • iMovie: When your iMovie project has an animated Map or Globe, you can Force click the map in the Timeline to access a Style menu. This lets you choose from four different styles.

As well as being able to simulate old physical trackpad features, the Taptic Engine can also provide physical feedback based on context:

  • iMovie: When dragging a video clip to its maximum length, you’ll get feedback letting you know you’ve hit the end of the clip. Add a title and you’ll get feedback as the title snaps into position at the beginning or end of a clip. Subtle feedback is also provided with the alignment guides that appear in the Viewer when cropping clips.

Final Cut Pro X is my video editing application of choice. iMovie is a full version of Final Cut Pro X running an additional consumer UI. As Final Cut Pro X hasn't been updated since December, iMovie's use of the Force Touch trackpad is a preview of features I hope to see in the next version of Final Cut.

I visited an Apple Store in London to see how iMovie 'felt' on the new version of the 13" MacBook Pro with Retina. 

'Feeling' the user interface

I tried two out of the three features mentioned in the support document. I couldn't feel any 'snapping' as I moved a title to the start or finish of a clip.

When I dragged the clip to its maximum length I did feel a little bump. Without looking at the timeline and looking at the viewer, I could 'feel' the end of the clip.

This feature presages the ability for UI pixels to be 'bumpy' - for user to feel the texture of application UIs without having to look at where the cursor is. This means that seemingly textured software keyboards and control layouts will be able to be implemented on future trackpads, iPhones and iPads.

Perhaps we'll look back and realise that the iOS 7 update removed borders from button because one day Apple user interfaces will be able to be felt as much as seen, and button text labels will feel more distinctive than button borders under our fingertips.

Non-visual manipulation

Film and video editing is an interesting UI problem: You need to look at the footage you are editing while you manipulate the clips that represent the footage in a timeline. That is why keyboard shortcuts are especially popular amongst video editors. No need to look at your mouse pointer the timeline as you manipulate clips - just press the keys that change the edit. 

Once a complex timeline can be represented by a touch only UI, editing will go full-screen. The screen will show footage only while the editor will be able to feel the edits as the story plays out.