Researchers at the Technical University of Munich (TUM) and the University of Cologne have developed an AI-based learning system that recognizes strengths and weaknesses in mathematics by tracking eye movements with a webcam to generate problem-solving hints. This enables teachers to provide significantly more children with individualized support.
An up-to-date PC, a good graphics card and a standard webcam: according to research by Prof. Achim Lilienthal, that's all you need to identify pupils' strengths and weaknesses in mathematics. The principle: a webcam tracks the eye movements. Depending on the task, specific patterns emerge that can be displayed digitally on a heatmap, with red indicating areas where the children look frequently and green the areas where they glance only briefly. This helps the researchers analyze the data. "The AI system classifies the patterns," says the TUM robotics professor. On this basis, the software selects learning videos and exercises for the pupil.
Identify learning strategies via heat maps
"Tracking eye movements in a single system using a webcam, recognizing learning strategies via patterns and offering individual support, and finally creating automated support reports for teachers is completely new," says Maike Schindler. The Professor of Mathematics in Inclusive and Special Education Contexts at the University of Cologne has worked with TUM Professor Lilienthal for ten years. She also heads the recently completed KI-ALF research project, which was funded by the German Federal Ministry of Education and Research (BMBF) and in which the webcam-based eye-tracking system was developed. Her research focuses on pupils "who have great difficulties in learning mathematics." Prof. Lilienthal believes that "individually customized lessons" for high-achieving children are also possible in the future.
Prof. Schindler – who holds a teaching degree – and her team have defined hundreds of tasks in which children add, subtract, multiply and divide numbers, or have to recognize or represent them. "Tasks involving visually presented, digital learning materials are particularly suitable for this approach," says Schindler. For example, the children are asked to count the dots in a ten-row table with a few dots missing only in the bottom row.
The pupils who catch on quickly jump to the bottom row and only count backwards. Those who count the rows and dots individually are among the ones who need support. The digital system uses a heat map to show where the children look and the AI translates the patterns into individual practice programs.
Simplified, high-precision eye tracking
To develop the simplified eye tracking system, which now registers eye movements, TUM Professor Lilienthal benefits from the fact that he also works with corresponding systems in robotics research. In that work, he currently uses eye trackers with the small humanoid robot Nao. This enables it to communicate better with humans. However, these very precise systems cost many thousands of euros.
To find a more cost-effective solution for schools, the researchers cleverly combined technical expertise with knowledge from mathematical didactics. While advanced systems work with a maximum deviation of one degree, webcams have a lower accuracy of three to four degrees. The solution: "With the AI-ALF math tasks, we know that the students are ultimately looking at the on-screen display of the problems," says Lilienthal. "We use this to automatically readjust the eye tracking with the webcam." The system has gradually learned to deal with inaccuracy. "Today it makes no difference to our application whether we work with our webcams or high-end eye trackers," says the professor. This makes the AI system developed with Prof Maike Schindler affordable and, therefore, increasingly important for school use.
Wulfen Comprehensive School: first school in Germany to use the system
This is one reason why the first school to use the AI-based learning system is the Wulfen Comprehensive School in Dorsten, North Rhine-Westphalia. Here, a standardized math test revealed that one-third of 180 children at the start of Year 5 had "arithmetic difficulties." "We are delighted that we can now support significantly more children in their basic math skills with the help of the AI-based learning system. This means we can help more learners improve their math performance than in the past due to a lack of teachers."
In the comprehensive school, five pupils can work with the KI-ALF system simultaneously in individual remedial lessons and are supported and accompanied by a math teacher. Normally teachers can give individual support to only one child at a time. "Especially in times of scarce resources and teacher shortages, our system for promoting basic math skills is simply an excellent support for schools," says Schindler.
