Development
The Bhelo Matrix — A Library of Multimodal Learning Interactions
The Bhelo Interaction Matrix is the conceptual core of the project, designed as an exploration to organise the wide range of interaction possibilities in VR environments. It serves as a blueprint for creating intuitive, inclusive and multimodal experiences that address diverse learning preferences.
Development
Initially, the matrix began as a raw compilation of over 1,000 (thousand) potential connections between categories such as content, representation, action/movement, and function. These categories were derived from the structure of two-dimensional interface design, which served as a foundational reference point for exploring interactions in the new medium of VR.
A system of hexagonal cards was developed to visually and functionally represent the interactions. The hexagon was specifically chosen due to ergonomics research conducted in the MedicVR project, which demonstrated its adaptability and ease of use in interactive design systems. The cards are designed to encourage experimentation and creativity. By arranging the cards next to one another, users can create an experience map, balancing and combining different types of interactions to design tailored VR applications.
Structure and Information Segmentation
The Bhelo Interaction Matrix is organised into five essential categories, each representing a critical component of interaction design within the VR. These categories provide a structured framework for mapping user interactions, system responses and the overall interaction flow, ensuring both functionality and adaptability.
Navigation
The navigation category includes the elements that guide the user through the virtual environment. These include interactive visual components such as buttons, menus or text boxes that help users orient themselves and move seamlessly through the experience. Navigation elements are similar to those found in traditional 2D interfaces, but adapted to the immersive nature of VR.
Interactive Content / Geometry
This category refers to dynamic objects within the virtual environment that can change their appearance or behaviour depending on their function. For example, interactive containers can act like folders that visually adapt to indicate their state or purpose. These elements provide users with engaging and responsive ways to interact with content.
Medium of Interaction
The medium of interaction focuses on the specific body parts or devices used to perform actions. In Bhelo, the primary focus is on hand-based interactions using movements of fingers, hands or arms. However, the matrix is designed to be scalable so that other media of interaction, such as controllers or tracking devices, can be included in future iterations.
Action/Interaction
The Action category captures the physical movements or gestures that users make to interact with the system. These actions - such as tapping, swiping or pointing - are essential for triggering functions and navigating the virtual environment.
Function
The Function category defines the intended outcome of an interaction. It represents the result or purpose behind the user's actions, whether it's navigating to a new area, starting an animation, or completing a task. Some interactions may involve a single function, while others may involve multiple outcomes, reflecting the complexity of certain scenarios.
Interaction and Information as the BUILDING Bricks
Far from being static representations, these maps are dynamic design tools. customized for specific learning goals. By arranging them side by side, users can construct an experience map, balancing and combining different interactions to design VR applications. Inspired by the modular structure of geodesic domes (as in the MedicVR project), their hexagonal shape ensures the adaptability of VR interfaces. Each card is composed of two distinct sides, each serving a specific purpose:
Side A: The Matrix Side
This side visually represents the combination of variables that define a particular interaction. It maps out the connection between categories such as Function, Navigation, and Action, providing a clear and structured overview of the interaction’s design.
Side B: The Informational Side
This side provides detailed descriptions of the interaction, including its purpose, potential applications, and the learning modalities (VARK) it may impact. It also explains how the interaction can be implemented and its relevance to the overall user experience.
Additionally, the cards double as AR trackers, enabling users to preview interactions in augmented reality. By scanning the cards, users can trigger AR animations that depict the interaction combinations outlined on the matrix side of the card. This feature bridges the gap between conceptual design and hands-on testing, enhancing the overall creative process.
