How They “Learn” To Win

Non-living hydrogels can play the video game Pong and improve their gameplay with more experience, researchers report on August 23 in the journal Cell Reports Physical Science. The researchers hooked hydrogels up to a virtual game environment and then applied a feedback loop between the hydrogel’s paddle—encoded by the distribution of charged particles within the hydrogel—and the ball’s position—encoded by electrical stimulation. With practice, the hydrogel’s accuracy improved by up to 10%, resulting in longer rallies. The researchers say that this demonstrates the ability of non-living materials to use “memory” to update their understanding of the environment, though more research is needed before it could be said that hydrogels can “learn.” Credit: Cell Reports Physical Science/Strong et al.

Bridging Biology and Robotics

The researchers were inspired by a previous study that showed that brain cells in a dish can learn to play Pong if they are electrically stimulated in a way that gives them feedback on their performance.

“Our paper addresses the question of whether simple artificial systems can compute closed loops similar to the feedback loops that allow our brains to control our bodies,” says corresponding author and biomedical engineer Yoshikatsu Hayashi of the University of Reading. “The basic principle in both neurons and hydrogels is that ion migration and distributions can work as a memory function that can correlate with sensory-motor loops in the Pong world. In neurons, ions run within the cells; in the gel, they run outside.”

Unveiling Hydrogel Mechanics

Hydrogels are complex polymers that become jelly-like when hydrated—gelatin and agar are natural examples. In this case, the researchers used an “electro-active polymer,” meaning a hydrogel that can respond to electrical stimulation thanks to the presence of ions (charged particles) in the media surrounding its polymer matrix. When the hydrogel is electrically stimulated, the ions move, dragging water molecules with them, and this movement causes the hydrogel to temporarily change shape.

“The rate at which the hydrogel de-swells takes much longer than the time it takes for it to swell in the first place, meaning that the ions’ next motion is influenced by its previous motion, which is sort of like memory occurring,” says Strong. “The continued rearrangement of ions within the hydrogel is based off of previous rearrangements within the hydrogel, continuing back to when it was first made and had a homogeneous distribution of ions.”

Advanced Gameplay and Future Applications

To test whether the hydrogel’s physical “memory” could enable it to play Pong, the researchers used electrodes to connect the hydrogel to a virtual game environment and started up the game by sending the ball in a random direction. They used electrical stimulation to inform the hydrogel of the ball’s position and measured the movement of ions within the hydrogel to determine the position of its paddle.

As the Pong games played out, the researchers measured the gel’s hit rate and examined whether its accuracy improved. They showed that, with more experience, the hydrogel was able to hit the ball more frequently, resulting in longer rallies. Whereas the Pong-playing neurons achieved their optimal ball skills within around 10 minutes, the hydrogel took closer to 20 minutes to reach its maximum Pong potential.

“Over time, as the ball moves, the gel gathers a memory of all motion. And then the paddle moves to accommodate that ball within the simulated environment,” says Strong. “The ions move in a way that maps a memory of all motion over time, and this “memory” results in improved performance.”

Because most existing AI algorithms are derived from neural networks, the researchers say that hydrogels represent a different kind of “intelligence” that could be used to develop new, simpler algorithms. In the future, the researchers plan to further probe the hydrogel’s “memory” by examining the mechanisms behind its memory and by testing its ability to perform other tasks.

“In our follow-up projects, we are thinking about how to extract the algorithm from the hydrogels that allows memory acquisition,” says co-author William Holderbaum of the University of Reading.

“We’ve shown that memory is emergent within the hydrogels, but the next step is to see whether we can also show specifically that learning is occurring,” says Strong.

For more on this research, see “Hydrogel Brain” Defies Expectations With Deep Learning.

Reference: “Electro-active polymer hydrogels exhibit emergent memory when embodied in a simulated game environment” by Vincent Strong, William Holderbaum and Yoshikatsu Hayashi, 22 August 2024, Cell Reports Physical Science.
DOI: 10.1016/j.xcrp.2024.102151

This research was supported by Process Vision Ltd.