Itinai.com it company office background blured chaos 50 v 9b8ecd9e 98cd 4a82 a026 ad27aa55c6b9 1
Itinai.com it company office background blured chaos 50 v 9b8ecd9e 98cd 4a82 a026 ad27aa55c6b9 1

A New Microsoft AI Research Proposes HMD-NeMo: A New Approach that Addresses Plausible and Accurate Full Body Motion Generation Even When the Hands may be Only Partially Visible

Researchers from Microsoft Mixed Reality & AI Lab have introduced a groundbreaking approach called HMD-NeMo (HMD Neural Motion Model) that generates accurate full-body motion in immersive mixed-reality scenarios, even when hands are only partially visible. HMD-NeMo uses a spatiotemporal encoder with novel mask tokens to encourage plausible motion, and it operates in real-time and online. The approach demonstrates superior accuracy and smoothness compared to existing methods, making it a pioneering solution in the field.

 A New Microsoft AI Research Proposes HMD-NeMo: A New Approach that Addresses Plausible and Accurate Full Body Motion Generation Even When the Hands may be Only Partially Visible

A New Approach to Full Body Motion Generation in Mixed-Reality Scenarios

In the world of immersive experiences in mixed-reality scenarios, generating accurate and realistic full-body avatar motion has been a challenge. Existing solutions rely on limited input signals, such as head and hand movements. However, these solutions assume full-hand visibility, which is not always the case in mixed reality experiences.

Researchers from Microsoft Mixed Reality & AI Lab have introduced a groundbreaking approach called HMD-NeMo (HMD Neural Motion Model). This unified neural network generates plausible and accurate full-body motion even when hands are only partially visible. HMD-NeMo operates in real-time and online, making it suitable for dynamic mixed-reality scenarios.

Key Features of HMD-NeMo:

  • Utilizes a spatiotemporal encoder with novel temporally adaptable mask tokens (TAMT) to encourage plausible motion in the absence of hand observations.
  • Incorporates recurrent neural networks and a transformer to efficiently capture temporal information and model complex relations between input signal components.
  • Handles both motion controller and hand tracking scenarios within a unified framework.
  • Maintains temporal coherence even when hands are partially or entirely out of the field of view.

The proposed method is trained using a loss function that considers data accuracy, smoothness, and auxiliary tasks for human pose reconstruction. Extensive evaluations on the AMASS dataset, a collection of human motion sequences, demonstrate the superior accuracy and smoothness of HMD-NeMo compared to state-of-the-art approaches.

HMD-NeMo represents a significant advancement in generating full-body avatar motion in mixed-reality scenarios. Its versatility, impressive performance metrics, and ability to handle different scenarios make it a pioneering solution in the field.

If you want to evolve your company with AI and stay competitive, consider using HMD-NeMo. It can redefine your way of work by automating customer interactions and improving sales processes. Connect with us at hello@itinai.com for AI KPI management advice and explore practical AI solutions at itinai.com.

List of Useful Links:

Itinai.com office ai background high tech quantum computing 0002ba7c e3d6 4fd7 abd6 cfe4e5f08aeb 0

Vladimir Dyachkov, Ph.D
Editor-in-Chief itinai.com

I believe that AI is only as powerful as the human insight guiding it.

Unleash Your Creative Potential with AI Agents

Competitors are already using AI Agents

Business Problems We Solve

  • Automation of internal processes.
  • Optimizing AI costs without huge budgets.
  • Training staff, developing custom courses for business needs
  • Integrating AI into client work, automating first lines of contact

Large and Medium Businesses

Startups

Offline Business

100% of clients report increased productivity and reduced operati

AI news and solutions