Microsoft is again at the centre of XR news this week following its recent partnership with Magic Leap and Qualcomm. The alliance sees the firms take equal stakes in scaling Microsoft’s Mixed Reality Toolkit 3 (MRTK3) for developers and enterprise partners.
The partnership also sees the MRTK3 framework transition to its own independent organization within the Microsoft-owned firm GitHub. It opens MRTK3 to further investment by a broader range of users and hardware developers.
Additionally, Microsoft announced that it expects to debut the general availability of MRTK3, ready for developers, in the second week of September 2023, Q3.
Robin Seiler, the Corporate Vice President and Chief Operating Officer of the Windows + Devices organization, Microsoft, explained:
We believe that to be a true multi-platform toolkit that enables greater third-party collaboration, MRTK needs to be recognized as an independent organization that expands beyond Microsoft’s sole jurisdiction. That’s why we have moved MRTK3 Open-Source Software (OSS) outside of Microsoft into its own independent organization within GitHub.
From MRTK’s inception, Seiler noted that Microsoft built it to “be cross-platform and open-source to benefit the entire ecosystem, not just HoloLens 2.”
Microsoft, Magic Leap, and Qualcomm Partner for the Next Stage of MRTK3
MRTK3 now operates under Microsoft, Magic Leap, and Qualcomm via a steering committee, ensuring operational success for the MR framework while improving accessibility and interoperability.
Magic Leap, which Robin Seiler describes as a “critical player” in AR devices, is joining the MRTK3 steering committee following a storied use of the service.
By having Magic Leap as a committee member and equal stakeholder for MRTK3, Microsoft hopes to “enrich” its services and foster its developer community toward creating more immersive applications “that span different platforms,” adds Seiler.
In a press release, Magic Leap also noted how the MRTK3 partnership enables the firms to deliver an open ecosystem that scales developer opportunities. The firm explained that the move helps enterprise end-users gain “complete control” of their workplace AR solutions, with the firm also adding that “walled gardens” hold back the potential of XR.
Magic Leap hopes to drive the scalability and flexibility of MRTK3. Moreover, the move helps Magic Leap support porting MRTK3 from other OpenXR devices to its brand of AR hardware.
Moreover, leading XR chipset manufacturer Qualcomm is joining the steering committee. The move also follows a long working relationship between Microsoft and Qualcomm on AR initiatives such as the System-on-Chip (SoC) platform used in HoloLens and the inclusion of MRTK3 on the Snapdragon Spaces XR Developer Platform.
Robin Seiler also added:
With Qualcomm Technologies joining the steering committee, two of the original architects of MRTK are returning to the project. Given their expertise and contributions, we are grateful to solidify their support and investments in the MRTK3 program and look forward to continued collaboration.
Microsoft Drives Forward with HoloLens, Towards 2024 Industrial Metaverse Plans
At its Inspire 2023 showcase event covering its latest innovations, Microsoft quietly teased information regarding an industrial metaverse initiative – set to kick off in 2024.
The firm is moving forward with its MR technologies to include deep cross-pollination with its genAI investments to produce innovative workplace digital solutions.
The firm launched the start of its industrial metaverse roadmap as part of its AI Cloud Partner Program, which brings significant partners together to fuel growth and innovation towards enterprise-grade AI – with this plan also supporting efforts to grow industrial metaverse solutions and related technologies.
The partnership looks to boost enterprise use cases of AI, Cloud, and Metaverse solutions to improve productivity, resiliency, and sustainability outcomes.
Microsoft’s industrial Metaverse roadmap will also offer a “curated learning path” to provide end-users with resources to introduce workplace XR technology, with considerations for integrated first-party technologies, such as Microsoft Co-Pilot, Azure OpenAI, AI, ML, Azure Databricks, Synapse Analytics (2024), Microsoft Fabric (2024), Microsoft Purview, Power Platform, D365 Field Service, D365 Guides, D365 Remote Assist, HoloLens 2, Azure Maps, Azure HPC product, and Azure Digital Twins.
Moreover, news of a Microsoft MR headset comes following the firm filling a patent request for an MR device, drawing similarities to the legacy HoloLens line-up.
According to the request, the conceptual device will be lighter and contain a modular framework – providing enterprise end-users with improved usability and form factors – a head-mounted device with a “rigid frame and front and back visor lens” with integrated sensors, waveguide optics, display/projection devices, dimmable displays, and additional equipment.
However, this is only a patent request. The device and its features may never see the light of day. However, some of the improvements, such as a lightweight design, modular frame, and dimmable display, fit alongside its IVAS military-grade XR device – perhaps a glimpse into the future of the HoloLens 3.
The IVAS headset is undergoing a crucial testing face, which sees military professionals use the new and improved version of the device in the field.
Following previous testing periods, the most recent IVAS iteration leverages MR visualizations to assist soldiers on the field by displaying tactical heads-up displays (HUDs), thermal vision, night vision, and passive targeting. Moreover, the recent iteration contains minor improvements – which may suggest features in a further HoloLens product – such as device visibility at night, weight, and form factor.
Mixed Reality is the “Eyes and Ears of AI”
Lili Cheng, Corporate Vice President of Business Applications and Platforms at Microsoft, recently wrote on how MR solutions can assist frontline workers in keeping up with modern workplace challenges such as training, upskilling, downtime, and productivity.
Cheng explained how Microsoft is investing in the emerging generative AI-powered service CoPilot as a tool that works alongside its MR dynamics guide service for XR headsets.
Microsoft exec also noted the importance and roles of integrated technologies such as digital twins. She explained how RT3D visualization and data allow frontline workers to “receive a better understanding of the machines and processes at hand.”
Cheng also added that access to related materials, such as digital twins, can be “further enhanced by AI,” saying:
Mixed reality is the eyes and ears of AI. Delving into operation nuances for personalized, in-depth learning becomes easier. When integrated, mixed reality and AI accelerate worker training, shortening steps and supplying users with the working knowledge they need for the task at hand.
Cheng also said that MR “facilitates knowledge retention and transfer across an organization through the creation of immersive experiences and simulations, regardless of location or device.”
Microsoft’s MR service claims to improve worker accuracy, productivity, and efficiency by democratizing immersive services across the workforce through worker access to its Microsoft Dynamics 365 Field Service immersive mobile application.
Additionally, Cheng noted the “versatility” of frontline MR applications makes “up-skilling faster and more accessible,” going on to say how Microsoft’s Dynamics 365 Guides gives industrial workers the tools to document “unique situational processes and procedures not included in asset operating manuals.”
