A team of researchers at the Istanbul Technical University (ITU), have developed a robotic system that can detect the emotions and stress of children with autism through games by using sensors and artificial intelligence.
The project was initiated by professor Hatice Köse, a faculty member of the ITU Computer and Informatics, Artificial Intelligence and Data Engineering Department. Köse and the project team developed a robotic system that uses sensors and artificial intelligence solutions to determine the emotions and stress of children with autism through games. As part of the project, therapy games have been developed with a child-like robot named "Kaspar." In the course of the games, emotions and stress in the children were determined.
Köse stated that children with autism have difficulty expressing their emotions properly. "When they are stressed, especially, they do not respond to treatment or therapy. It is not easy to understand their emotions and stress, they reflect them much differently compared to adults," she said.
"Studies and datasets developed in this area always aim to detect stress and emotions of adults. These systems cannot be used efficiently for children. In the therapy session, we aimed to understand their emotions during their interactions with the robot by processing data from different sensors with artificial intelligence," she outlined.
In this context, Köse explained that they developed a robotic system and an artificial intelligence-supported emotion recognition module using the Kaspar robot. "We developed the Kaspar humanoid robot and its software in England for therapy sessions of children with autism. It is currently used in autism therapy in many countries. We also used it in our country as part of our project," Köse stated.
"Within the scope of our project, while children with autism play therapy games with the Kaspar robot, we track eye movement, facial expressions, voice and posture through various smart sensors. We also wear a smart bracelet to monitor physiological data. Expert psychologists analyze the data from these smart sensors, and develop artificial intelligence models to read children's stress, emotions and attention," she added.
Emphasizing that they worked with different groups of children in Türkiye, Poland, the United Kingdom and North Macedonia within the scope of the project and made comparisons, Köse stated that they worked with therapists, doctors and engineers from different countries in the project group.
"Children don't normally look at people's faces much, but they looked at the robot's face more. We observed that they looked and became more interested, especially when they sang with the robot. Children communicated more with the robot while they were playing. We detected this both through the observation and analysis of experts, and through artificial intelligence models intertwined with sensor data," she highlighted.