Since the early 21st century, metamaterials have revolutionized electromagnetic wave manipulation, starting with passive devices designed for beam control applications such as invisibility cloaking and perfect lensing. Over time, these systems have evolved into programmable metasurfaces capable of adapting to tasks like wireless communication and sensing. Despite these advancements, current metamaterials still rely on predefined functions determined by human operators, lacking the ability to autonomously carry out complex, real-time tasks without human intervention.
In a new paper published in Light: Science & Applications, a team of scientists, led by Professor Lianlin Li from Peking University, China, and Pazhou Laboratory (Huangpu), China, and Professor Tie Jun Cui from Southeast University, China, and Pazhou Laboratory (Huangpu), China , along with co-workers, have developed a revolutionary electromagnetic metamaterial agent system—metaAgent. This system autonomously handles complex electromagnetic (EM) tasks through reasoning and decision-making. Specifically designed for practical scenarios like smart homes and smart factories, metaAgent leverages various sensors (audio, text, visual, and radio) to interpret environmental information and independently execute actions, such as real-time user tracking, monitoring vital signs like respiration and heartbeat, and controlling robots to provide assistance.
The core innovation of the system lies in its ability to reason using natural language, powered by large foundation models (LFMs). This enables metaAgent to autonomously plan and execute tasks without human intervention, while continuously learning from human feedback. For example, upon detecting that a user is unwell, metaAgent can utilize semantic programmable metasurfaces to monitor the individual and coordinate with robots or other devices to deliver aid based on user instructions. This autonomous coordination of multiple long-term tasks marks a significant breakthrough in EM manipulation and human-device interaction, offering dynamic, real-time solutions for intelligent environments. By moving beyond static, predefined functions, metaAgent opens new possibilities for smart environments, healthcare, and advanced human-machine interaction.
At the core of metaAgent is a sophisticated autonomous agent system that independently responds to dynamic and uncertain environments through the intricate interactions between electromagnetic waves, matter, and information. MetaAgent consists of two key components: the first is the "Cerebellum," which employs multiple semantic programmable metasurfaces (SPMs) operating across different frequency bands to execute specific electromagnetic manipulation tasks. The second component is the "Cerebrum," which, through a multi-expert discussion mechanism powered by LFMs, formulates high-level strategies. This mechanism includes sensing expert, planning expert, grounding expert, and coding expert, who together delegate specific task sequences to the Cerebellum for processing. The scientists summarized the operating principles of their metaAgent system as follows:
"metaAgent autonomously accomplishes the user's command by taking sequentially the four-step operations: i) the sensory expert summons the multi-modality sensor data (radio, audio, text, image) and synthesizes the information in natura language, ii) the planning expert generates a body of executable subtasks, iii) the grounding expert assigns each subtask with an action function and associated devices, iv) the coding expert generates the action policy. The coding expert produces two types of outputs in natural language: one 'external' output for the communication with the external human user, and one 'inner' output for consideration by the planning and grounding experts."
"Our metaAgent has capability to understand a variety of environmental cues (spoken instructions, body language, vital signs), to make long-horizon plans involving interactions with robots and humans, and to successfully execute the plans based on the available hardware and EM wave manipulation skills." they added.
"The capabilities of the metaAgent even exceed those of a human assistant because of the metaAgent's microwave perception that can sense through visually opaque layers and around corners." the scientists forecast.
