‘Smart Gloves’ Teach Piano Playing through Touch

‘Smart Gloves’ Teach Piano Playing through Touch

By Riis Williams

Made of thin cotton and stitched together in only 20 minutes, an experimental pair of gloves isn’t particularly fashionable or useful for keeping anyone’s hands warm. Instead the accessory uses tactile sensors woven into its fabric to serve an entirely different purpose: teaching piano and other hands-on skills.

For a study in Nature Communications, Massachusetts Institute of Technology graduate student Yiyue Luo and her colleagues created these “smart gloves” using haptic technology, which incorporates physical sensations such as vibrations or force to help with tactile activities. Researchers used the gloves to record one pianist’s hand movements while playing a song. They then relayed those movements to a student through fingertip vibrations, helping the learner build muscle memory and perform the piece with greater precision. “Hand-based movements like piano playing are normally really subjective and difficult to record and transfer,” Luo says. “But with these gloves we are actually able to track one person’s touch experience and share it with another person to improve their tactile learning process.”

Using a computerized embroidery machine, the team embedded small wires linked to a pressure-sensing material in the gloves to detect hand motions. When a piano teacher wearing the gloves repeatedly performed a tune, a machine-learning algorithm processed their movement on the keys and translated it into instructional vibrations. Students wearing their own gloves then attempted to play the same tune, with the fingertip vibrations guiding them through proper movements. (The vibration intensity increased to correct fingering or rhythm mistakes.) By the trials’ end, students who had practiced with the gloves could play more accurately than those who had not. “This type of learning is sort of like when you’re first starting to ride a bike,” says Rice University mechanical engineer Marcia O’Malley, who was not involved in the study. “You use the training wheels to get the sensation of properly riding the bike. Eventually you take them off and start to ride independently.”

On supporting science journalism

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

The team also tested the gloves’ ability to aid people playing online games with a mouse and keyboard, recording motions from experienced players to guide novices. People who gamed with the gloves’ guidance scored better on average than those who did not.

Playing piano or video games is often just for fun, but O’Malley adds that with an improved algorithm, coded to identify and capture finer hand movements, the new glove technology could someday help to teach crucial practices such as surgeries. Instructional haptics “remove a step in the learning process that auditory and visual learning can’t,” she says. “When vibration is actually felt directly at the point of action, we can act and learn quicker—and with that, there’s so much potential.”