VID AI systems can track the movements of people hidden behind walls by inspecting radio waves reflected off their bodies, according to a new study.
The model dubbed “RF-Pose” starts off by transmitting low power radio signals that can penetrate through walls using a wireless Wi-Fi device. These waves bounce back when reflected off a human body, creating heatmaps, andd these are then processed by a neural network to build a two-dimensional stick figure representing the person behind the wall.
During the training process, RF-Pose is trained using both the heatmap images created from their device and images taken from cameras. The researchers from the Massachusetts Institute of Technology (MIT) collected more than 50 hours of footage of people in 50 different environments, including people walking along corridors, or engaging in a lesson in a classroom.
The positions of people sitting, eating, walking, talking or jogging from the camera images are extracted to create ‘confidence maps’, a series of blobs representing the head, torso, arms and legs, and converted into stick figures.
The movements of human body are extracted and encoded into confidence maps and 2D stick figures. Image credit: Zhao et al and MIT CSAIL.
Next, the corresponding radio signal for each image is also fed into the neural network. The combination of the camera images, stick figures, and radio heatmaps trains the system to map the correct pose of the stick figure to the radio signals.
“RF-Pose is trained with 70% of the data from visible scenes, and tested with the remaining 30% of the data from visible scenes and all the data from through-wall scenarios. We make sure that the training data and test data are from different environments,” the paper released on arXiv said.
It means that for the inference stage, the system no longer requires camera images and can build a two-dimensional stick figure using the radio signals alone. RF-Pose can accurately identify somebody 83 per cent of the time from a line-up of 100 individuals. A video demonstration shows that it can also handle more than one person in a room.
The potential for this kind of technology to be used for spyware is alarming. Since it only uses radio signals, it doesn’t rely on lighting conditions like normal cameras. But at the moment the resolution is only tens of centimeters and is constricted by the bandwidth of the radio signal.
The researchers don’t discuss the ethical implications in much detail, and said they were planning to use it to monitor diseases like Parkinson’s, multiple sclerosis (MS), and muscular dystrophy.
Looking through walls, now easier than ever
“We’ve seen that monitoring patients’ walking speed and ability to do basic activities on their own gives health care providers a window into their lives that they didn’t have before, which could be meaningful for a whole range of diseases,” said Dina Katabi, co-author of the paper and a professor at MIT’s Computer Science and Artificial Intelligence Laboratory.
In the paper they did acknowledge that it could be used for surveillance, activity recognition or gaming. Last week another paper on arXiv also used a neural network to estimate human poses from a camera attached on a drone in the hopes of detecting violent behaviour.
The researchers hope to create three dimensional models from radio signals so that they can detect movements in more detail. It could, for example, detect if an older person’s hands are shaking regularly enough that they may want to see a doctor.
“By using this combination of visual data and AI to see through walls, we can enable better scene understanding and smarter environments to live safer, more productive lives,” said Mingmin Zhao, first author of the paper and a PhD student at MIT. ®