Scientists use WiFi signals to track human movement for the metaverse

A team of researchers from Nanyang Technological University in Singapore recently unveiled a new method for tracking human movement for the metaverse. 
One of the key features of the metaverse is the ability to represent real world objects and people in the digital world in real time. In virtual reality, for example, users can turn their heads to change their viewpoints or manipulate physical controllers in the real world to affect the digital environment.
The status quo for capturing human activity in the metaverse uses device-based sensors, cameras or a combination of both. However, as the researchers write in their preprint research paper, both of these modalities have immediate limitations.

A device-based sensing system, such as a hand-held controller with a motion sensor, “only captures the information at one point of the human body and thus cannot model very complex activity,” write the researchers. Meanwhile, camera-based tracking systems struggle with low-light environments and physical obstructions.

Enter WiFi sensing

Scientists have used WiFi sensors to track human movement for years. Much like radar, the radio signals used to send and receive WiFi data can be used to sense objects in space.
WiFi sensors can be fine-tuned to pick up heartbeats, track breathing and sleeping patterns, and even sense people through walls.

Metaverse researchers have experimented with combining traditional tracking methods with WiFi sensing to varying degrees of success in the past.

Enter artificial intelligence

WiFi tracking requires the use of artificial intelligence models. Unfortunately, training these models has proven to have a high degree of difficulty for researchers.
Per the Singaporean team’s paper:
In order to train the necessary models required to experiment with WiFi sensing for HAR, scientists have to build a library of training data. The data sets used to train AI can contain thousands or even millions of data points depending on the aims of the particular model.
Often, labeling these data sets can be the most time-consuming part of conducting these experiments.

Enter MaskFi

The team from Nanyang Technological University built “MaskFi” to overcome this challenge. It uses AI models built using a method called “unsupervised learning.”
In the unsupervised learning paradigm, an AI model is pretrained on a significantly smaller data set and then put through iterations until it’s able to predict output states with a satisfactory level of accuracy. This allows researchers to focus their energy on the models themselves instead of the painstaking effort it takes to build robust training data sets.

According to the researchers, the MaskFi system achieved about 97% accuracy across two related benchmarks. This indicates that this system could, through future development, serve as the catalyst for an entirely new metaverse modality: a metaverse that can provide a 1:1 real-world representation in real-time.