Development
The AR Library — Exploring Multimodal Interactions
The design exploration phase brought the initial research findings to life. This page outlines the creation of intuitive VR interactions, from prototyping gestures to creating three-dimensional menu placements.
The AR Library and the Interaction Matrix are deeply interconnected, with each layer complementing the other. While the Matrix focuses on the theoretical possibilities and combinations of interactions, the AR Library provides a virtual pre-application of these concepts, allowing users to preview and experiment with interactions in a tangible way.
How it works
Each card in the matrix represents a unique combination of variables, displayed as a distinct visual pattern. This pattern not only defines the interaction, but also acts as an Augmented Reality (AR) tracker. When the card is scanned, the AR library overlays an animated visualisation on top of the card's information layer. This makes abstract concepts easier to understand by showing what the interaction looks like and how it works in practice.
The AR Library helps users see how variables such as function, navigation and interaction medium come together to create virtual interactions. These animations provide a clear and simple preview of how the interaction could be applied in a VR environment.
The library also includes small differences in patterns and animations to show variations in similar interactions. For example, two interactions might have the same result, but look or move differently depending on the context. This flexibility makes the AR Library a powerful tool for exploring and improving interaction design.
Examples of AR Animated INTERACTION Cards
Tap to Open (01.003.NL)
This interaction is inspired by the common "tap to open" feature found in touch-based devices, such as smartphones. The animation depicts a container holding three sub-containers. By tapping on the container, it opens and adjusts its form based on the shape of the sub-content.
Learning Modalities Impacted: The visual design activates the visual learning process, as the unique shapes of the containers help users quickly identify and differentiate content. Adding a text label to the container further supports the read/write modality, enhancing the user’s ability to understand the content structure.
Purpose: This interaction simplifies the learning curve by using a familiar action, helping users feel more comfortable with the interface and reducing the risk of alienation during first contact. The use of distinct container shapes enhances the user's ability to process information visually, making the interaction more intuitive and engaging.
Teletransport (05.017.VK)
This interaction demonstrates a "teleportation" function, allowing users to move within the virtual environment. The animation shows the spatial output of the interaction, such as a destination on a map or a visible floor surface for short-distance teleportation. Unlike traditional VR teleportation, this interaction avoids using a laser pointer, instead relying on a more intuitive approach.
Learning Modalities Impacted: The changing landscapes and spatial visualizations engage kinesthetic and visual learners, providing a more immersive and accessible navigation experience.
Purpose: This interaction enhances immersion by providing a seamless way to navigate large or small distances within the virtual environment. By showing spatial relationships and destinations clearly, it helps users orient themselves and understand the virtual space more effectively.
The Ideal Scenario
Designers and developers can use the AR Library as a blueprint for structuring user interactions in VR. The modular design of the cards allows experimentation with different combinations, allowing the creation of intuitive interfaces and interactions. By using the information side of each card, users are given clear guidance on the purpose and function of each interaction, while the augmented reality animated visualisations of the matrix further facilitate understanding and refinement.
Ideally, the cards would also contain a code package that could be read by VR software. This functionality would allow users to scan combinations of cards and automatically generate interactions in virtual applications. Such a feature would streamline the development process, connect the conceptual design and implementation workflows, and make VR interaction design accessible to non-technical users.
