Researchers at MIT have developed a groundbreaking wristband that uses ultrasound stickers to track hand movements in real time, enabling wireless control of robotic hands and virtual objects. This technology overcomes the limitations of previous motion capture systems by directly imaging the muscles, tendons, and ligaments inside the wrist, offering unprecedented accuracy.
How ultrasound stickers capture hand movements
Professor Xuanhe Zhao and his team designed a wearable device that combines a miniaturized ultrasound transducer with a hydrogel sticker that safely adheres to the skin. As the wearer moves their hand, the wristband produces detailed ultrasound images of the wrist's internal tissues. An artificial intelligence algorithm, trained on thousands of manually labeled images, continuously translates these images into the corresponding positions of the five fingers and the palm. Gengxi Lu, a former MIT postdoc and lead author of the paper, explains: "The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers. By taking a picture of the state of the strings, you can determine the state of the hand."
Sponsored Protocol
Real-time control of robotic hands and virtual objects
In demonstrations, a person wearing the wristband can wirelessly control a robotic hand, replicating gestures and movements. The system has been used to play a simple tune on a digital piano and to shoot a mini basketball into a desktop hoop. Additionally, the wearer can manipulate objects on a computer screen, for instance pinching fingers together to enlarge or shrink a virtual object. The current device is about the size of a cell phone, but the team is working on further miniaturization. They also plan to expand the movement dataset with volunteers of varying hand sizes and shapes to improve the algorithm's robustness.
Future applications in robotic surgery and virtual reality
This technology opens the door to numerous applications. The researchers envision building a large repository of hand motions to train humanoid robots in delicate tasks such as surgical procedures. In design and gaming, the wristband could enable intuitive interactions with virtual objects. Xuanhe Zhao states: "We believe this is the most advanced way to track dexterous hand motion through wearable imaging of the wrist. These wearable ultrasound bands can provide intuitive and versatile controls for virtual reality and robotic hands." To explore other MIT innovations in biotechnology, read the article on the breath test for pneumonia and on injectable mini livers. For more details on the original study, see the article on MIT Technology Review.
Sponsored Protocol
