Neural networks are one typical structure on which artificial intelligence can be based. The term ›neural‹ describes their learning ability, which to some extent mimics the functioning of neurons in our brains. To be able to work, several key ingredients are required: one of them is an activation function which introduces nonlinearity into the structure. A photonic activation function has important advantages for the implementation of optical neural networks based on light propagation. Researchers in the Stiller Research Group at the Max Planck Institute for the Science of Light (MPL) and Leibniz University Hannover (LUH) in collaboration with Dirk Englund at MIT have now experimentally shown an all-optically controlled activation function based on traveling sound waves. It is suitable for a wide range of optical neural network approaches and allows operation in the so-called synthetic frequency dimension.
Artificial intelligence (AI) is widely used and designed to augment human skills such as data analysis, text generation and image recognition. Its performance has surpassed that of humans in many areas, for example in terms of speed. Tasks which would take many hours of work when performed manually can be completed in seconds.
Among other options, AI can be based on artificial neural networks inspired by the brain. Similar to neurons in the human brain, the nodes of the neural networks are linked in a very complex structure. Currently, they are most commonly implemented using digital connections. Recent experience in training artificial intelligence such as large language models has made it clear that their energy consumption is vast and will increase exponentially in the upcoming years. Therefore, scientists are researching a solution intensively and considering different physical systems which could support or partially replace electronic systems for certain tasks. These networks could be based on optical materials, on structures of molecules, on DNA strands, or even the development of mushroom structures.
Optics and photonics have many advantages over conventional electronic systems
Optics and photonics have the advantage of high bandwidths and information encoding in high-dimensional symbols -- both reasons for the speed-up of our communication system. Photonic systems are already quite advanced and often allow parallel processing and connection to established systems such as the optical fiber-based world-wide internet. When scaling up, photonics also holds the promise of lower energy consumption for complex problems. Now research groups are tapping into these resources and knowledge to implement optical neural networks in many different ways. However, many key challenges must be addressed, for example the up-scaling of the photonic hardware and the reconfigurability of the neural networks.
All-optically controlled activation function based on sound waves demonstrated for the first time
Researchers in the Stiller Lab work on optoacoustics and specifically on the challenge of optical neural networks mediated by acoustic waves. For the upscaling of the optical neural networks, they have now developed an activation function which can be controlled all-optically. The information does not need to be converted back from the optical to the electronic domain. This development is an important step for photonic computing, a physical analog computing alternative which promises to be able to realize energy efficient artificial intelligence in the long term. A simple form of a neural network consists of a weighted sum of bits of the incoming information and a nonlinear activation function. The nonlinear activation function is essential for deep learning models to learn to solve complex tasks. In optical neural networks, these parts are ideally implemented in the photonic domain as well. For the weighted sum -- a matrix operator -- a plethora of photonic approaches already exist. This is not the case for the nonlinear activation function, for which few approaches have been demonstrated experimentally.
"The long-term prospect of creating more energy efficient optical neural networks depends on whether we are able to scale up the physical computing systems, a process potentially facilitated by a photonic activation function." says Birgit Stiller, head of the research group ›Quantum Optoacoustics‹.
A photonic nonlinear activation function is the optical equivalent of the nonlinear activation functions used in artificial neural networks, but implemented using photonic devices instead of electronics. It introduces nonlinearity into photonic computing systems, enabling all-optical neural networks and optical machine learning accelerators. Examples of activation functions are ReLU, sigmoid, or tanh functions and they can transform the weighted sum of inputs into an artificial neural network.
Sound waves as a mediator for an effective photonic activation function
The scientists from the Stiller Research Group at MPL and LUH, in collaboration with Dirk Englund from MIT, have now demonstrated that sound waves can be the mediator for an effective photonic activation function. The optical information does not have to leave the optical domain and is directly processed in optical fibers or photonic waveguides. Via the effect of stimulated Brillouin scattering, the optical input information undergoes a nonlinear change depending on the level of optical intensity.
"Our photonic activation function can be tuned in a versatile way: we show the implementation of a sigmoid, ReLU and quadratic function and the concept also allows for more exotic functions on demand, if needed for certain types of tasks," says one of the two lead authors Grigorii Slinkov. The other lead author Steven Becker adds: "An interesting advantage comes from a strict phase-matching rule in stimulated Brillouin scattering: different optical frequencies -- for parallel computing -- can be addressed individually, which may enhance the computational performance of the neural network."
Including a photonic activation function in an optical neural network preserves the bandwidth of the optical data, avoids electro-optic conversion and maintains the coherence of the signal. The versatile control of the nonlinear activation function with the help of sound waves allows the implementation of the scheme in existing optical fiber systems as well as photonic chips.
