Case Study
Magic Leap
Building an XR spatial presentation tool for digital assets
Extended Reality, Digital Design
2019-2020
Roles
Product vision
UX strategy
Concept visuals
Experience design
UX/UI
Prototyping & Testing
Product guidelines
Softwares
Unity Engine
Rhino 3D
Maya
Keyshot
Photoshop
After Effects
Platforms
Magic Leap One
Desktop / PC
Background
I joined Magic Leap as a product experience designer in 2019 to solve some of the most challenging problems in the world of mixed reality, and envision the future of spatial computing. Magic Leap's technology has the potential to completely revolutionize how we interact with every aspect of our lives from gaming and entertainment to manufacturing and healthcare.
I worked collaboratively with multiple teams including design, engineering, product, prototyping, marketing, and business units to explore the potential of extended reality in empowering individuals and potential enterprise clients to create human-centric experiences that enhance our capacity to connect, imagine, and create. One such initiative was Spatial Present - an XR tool to design, build, and view a presentation of digital assets in your physical environment.
Overview
We all have built a presentation at least once in our lives, for various reasons. Our current tools give us the ability to design stellar presentations to convey our ideas & thoughts, and make them awe-inspiring & appealing to our audience. But the fact remains that these assets which we build & present are mostly trapped in 2D screens, even thought the world we experience on a daily basis is 3-dimensional and spatial in nature.
Traditional presentations are 2-dimensional and flat, and they convey incomplete information.
Extended reality gives us the opportunity to experience and interact with 3-dimensional elements, immerse and gain a sense of presence with virtual content, and opens up possibilities to create experiences that extend human capabilities. We aspired to address the following challenge -
How to build presentations that are spatial in nature?
It started off with this simple idea but soon made quite an impact for our internal teams and Magic Leap's business in general. Spatial Present helped us land our first paying customer.
Image Credit: Magic Leap
Designing a tool for presentations of various shapes, needs, and formats
A presentation in it's basic form is just a method of communication in a formal way. Everything from a zoom call to a conference room setup, from a TED talk to a trade show, and from a stage performance to a museum walk-through, can be qualified as a presentation.
We build and design presentations for various purposes. There are some functional needs to present like - to convey information, to inspire others, persuade your audience, or to simply motivate someone. Similarly, there are emotional needs and human-centric values for why we present, like - spreading awareness, for recognition & acceptance, or simply for our freedom of expression.
We began mapping down these attributes and breaking them down to their fundamental components like - ways of authoring, various methods of presenting, relationship with your audience, types of assets used, and the context.
The relationship between these components determine a presentation's impact and success.
Our goal was not to simply create an XR power-point, but to design a tool that would cater to all forms of presentations regardless of their size, format, or need.
Image Credit: BrainLab
Leveraging spatial anchors & MR scanning to build digital twins
Before placing digital assets into the physical world, we need to understand the layout of the presentation space. 'Space' being a place in the physical world where an extended reality presentation can live and persist. We achieved this by building Digital Twins of the physical space by leveraging Magic Leap One's 3D scanning capability to create a low-poly mesh of the location, and then tethering that mesh to the respective physical location using Spatial Anchors, which the Magic Leap technology was capable of at that time.
To understand this experience from a user standpoint, let's look at a scenario where User A - Bryan, walks into a room, let's call this Room A, where he wants to build a spatial presentation. Bryan wears his Magic Leap headset and 3D scans the entire Room A. This creates a digital twin of room A, which he can then directly upload to the Magic Leap server.
Location: Room A
Mapping Room A using ML Headset
Bryan then uses a desktop PC based Authoring tool to build his spatial presentation. This tool allows him to download the Digital Twin mesh of the room A that he had scanned & uploaded to the server, and use it as the base of his presentation to place digital assets in space. Once he is done building the it, the tool allows Bryan to 'publish' his work to the server, which can then be accessed and opened on the 'Spatial Present' companion app installed on his Magic Leap headset.
The Authoring tool is a custom Unity game engine based platform built using the Magic Leap software development kit. It allows you to build a spatial presentation using any traditional 2D or 3D assets you'd normally use to build a presentation deck. Due to NDAs signed, I'm unable to share any further details about this tool.
Viewing the spatial presentation in Room A
Co-present experience: Collaborative viewing of the presentation in real-time
Conducting co-present, collaborative, spatial presentations with the ability to edit in real-time.
After publishing his work, Bryan can then open the presentation on his Spatial Present app on his Magic Leap headset and walkthrough his work in the physical space that he had scanned. He is able to view all the digital assets he had places in various scenes throughout the space and can navigate the presentation using the device controller.
He can also invite other collaborators into the presentation session and walk them through the experience in real-time, thus having a co-present experience of viewing & interacting with digital assets in physical space. Users can also join the presentation from any geolocation, thus allowing remote virtual collaborations possible, as seen below.
Remote collaborative viewing of the presentation and live-editing features
Migrating spatial presentations to enable multiple synchronous real-time experiences
Now imagine if Bryan wants to migrate this already designed spatial presentation to a new physical location, away from it's originally anchored space. He can do so by scanning a new location, let's say Room B, and relocating assets from his presentation into the new Digital Twin of Room B. This allows Bryan to conduct a single presentation synchronously in multiple locations in at the same time. This ultimately allows users to scale a spatial presentation to multiple physical spaces and have a synchronous experience in real-time.
Location: Room B
Relocating all digital assets and scenes into Room B
Multiple synchronous real-time spatial presentations in different locations
Impact of Spatial Present
The Spatial Present tool started off with this simple idea of making traditional presentation decks more spatial in nature, but soon made quite an impact for our internal teams and Magic Leap's business in general.
We built many iterations of the tool to various degrees of finishes, and authored at least a few dozen presentations inside our office workspaces. We built and organized white-glove demos for clients & potential investors of Magic Leap in order to showcase how the ML technology can elevate their current business workflows & processes. Spatial Present soon became the go-to design review tool for our internal UX & industrial design teams to see their designs virtually in space before pushing it to productization. We also landed our first paying customer who invested in Spatial Present.
The following videos are few examples of the demo presentations we built inside our office workspace using our tool.
A spatial presentation of the Magic Leap One designs, inspirations, and capabilities.
A spatial presentation built for a client demo which showcases automotive workflows & designs
A spatial presentation built for a client demo which showcases MR use cases for sports & entertainment.
A spatial presentation built for a client demo which showcases MR use cases for art galleries & experiences
A spatial presentation built for a client demo which showcases MR use cases for the healthcare industry
Learnings & Reflections on a successful prototype and the failure to productize
I had the privilege of working as part of a rapid-prototyping team within Magic Leap which mainly consisted of developers, UX prototypers, design, and a 3D artist. Our main responsibility was to conduct R&D and prototype new workflows & unique use cases that explores the depths of Magic Leap's spatial computing technology.
Being a solo designer working under a creative manager and a VP of product, I learnt to trust my decisions and fight for my ideas. Working in Magic Leap's fast-paced startup culture also thought me to back my ideas with proper research and to always over-communicate any design decisions, especially while working in a multi-disciplinary team. I learnt to be as visual as possible in my approach and break down every concept, feature, and narrative to its simplest elements that define it's nature or creates value for the customer.
The hardest and most valuable lesson I learnt was that - a product can fail even after countless successful validations and attaining product market fit (PMF).
We developed multiple versions of the Spatial Present tool within 10 months and user tested the solution with numerous internal and external stakeholders. And despite the tremendous positive feedback and customer interests for it, the solution remains as a hacky prototype till date due to Magic Leap's infamous pivot from consumer to enterprise market following the mass layoffs in 2020 during Covid. However, I still believe it to be an extremely successful initiative which opened countless discussions and allowed us to explore & experiment various concepts & ideas with ease!
Quick early story boarding that evolved into the Spatial Present tool