Design proposal for the top level UI

The most important part of AaltoWindow is its content, which is dynamic, scalable, and accessible to multiple users at the same time. Here’s a proposal that allows users to share the interaction surface with multiple co-running application instances.

The first idea is that the minimized-state applications are represented as app discs, which are interactive widgets blessed with a virtual physical mass – like hockey pucks or tiny curling stones on ice. The discs can be flicked into motion, there’s (slight) friction to slow them down, collision detection, and other physics-inspired properties to make the model feel more natural. The figure below shows a freeze-framed snapshot of the proposed model:

But how do those discs end up gliding on the surface in the first place? Here’s the second concept: in the beginning the surface is empty, just showing an animated circular arc with a text string, prompting the user to draw a circle on the surface (this idea was borrowed from the multi-user desktop of PyMT). But instead of opening a modal rectangular popup menu as PyMT does, the AaltoWindow model continues in the circular domain:

The user first gestures a rough circle, which the system recognizes as a generic “launch app” command (1). An empty app disc fades in, listing the available applications at the outer ring of the disc (the list can be hierarchical if needed). The user then rotates the outer ring to view a screenshot, title and author info of the application currently located at the 12 o’clock position. The desired application can be launched either by tapping in the middle of the disc (to launch the 12 o’clock app), or by double-tapping an icon in the outer ring (2). After this, the chrome fades out and the app is shown in its minimized state, i.e., as a branded app disc.

The launched app may show some useful info already in its minimized state, but since the disc is going to be small, most apps need to switch into ‘windowed’ mode for the actual interaction tasks. To do this, the user stretches the app disc, and after it reaches a specific threshold size, the application opens into the windowed state (3):

The animation between the minimized and windowed states is two-fold: if the windowed state has a distinct geometrical shape (such as a round rect), the circle morphs into the rect using Gielis’ superformula. However, if the windowed state is not a single shape (e.g., if it is a spring graph or a collection of 3D shapes – the content form depends entirely on the app), the app disc just fades out while the app content fades in. The air-space problem
will restrict the transparency options between the apps, though. In any case, co-centric alpha-blended ripples are emitted from the vanishing app disc to indicate that it is about to evaporate into an open app.

The user then interacts with the application, and when done, closes the application by going from the windowed state back to the minimized state. This basically reverses the opening UI sequence (4a), although some apps may employ creative means in changing their states. For example, an app with a spring graph content, as displayed in the middle of the figure below, minimizes when the nodes are pulled together into a dense cluster:

It is also possible to promptly peek into the app disc by performing a stretch – wait – pinch sequence without raising the fingers from the surface, i.e., using a single gesture for the peek. This might be useful when there are many different applications available on the surface, and one wants to know whether there is anything interesting inside the disc.

Because some apps might behave extremely creatively when defining the windowed-minimized transition sequence, and as some apps might not support zooming at all, it should also be possible to minimize/maximize the app using the chrome (4b). Following the design principles – and having seen the Windows 8 preview videos – I would like to keep the chrome hidden and display it only when requested by the user’s side-swipe gesture. But how does the user know that the chrome is made visible using such a gesture? In future, Windows 8 slates might well make this gesture de facto, and it might turn out to be as ubiquitous as pinching or flicking are these days. But there has to be a way to indicate the option to the user somehow, need to think about that.

Finally, to remove the app disc from the surface, the user just needs to shrink the disc, and after it becomes smaller than a certain threshold, the disc fades out and the app is unloaded (5):

In summary, the design principles were based on (borrowing some jargon from the MIX’11 panel):

  • direct manipulation
  • scaffolding, or learning by using
  • natural albeit super-realistic environment
  • semantic zooming
  • morphing the UI to accommodate the content, minimal chrome
  • ease of use is not a strict requirement
  • magic

The magic bit is actually still on the wish list, as it materializes only with the implementation. But we will soon see how that goes, I’m having terrific time working with the code (WPF, BodyBehavior, $1 gesture recognizer etc.).