"What could be more interesting than how the mind works?" Nearly every person has once in life deliberated how this three-pound organ right beneath the skull controls all bodily functions, interprets information from the outside world, embodies the essence of the mind and soul, and how the "neural-circuit" acts as an information highway between different areas of it.
While the human brain finds ways to exceed our physical capabilities, combining mathematics, algorithms, statistical models, and computational methods accelerates our scientific pursuit. In a major step in this direction, Dr. Manan Suri, an Associate Professor with the Department of Electrical Engineering at IIT Delhi, listed as one of the World's Top 35 Innovators under the age of 35 by MIT Technology Review, USA, along with research scholars- Mr. Ahmed Shaban and Mr. Sai Sukruth Bezugam developed a new spiking neuron model called DEXAT- Double EXponential Adaptive Threshold neuron which may prove to be a significant milestone in neuromorphic artificial intelligence technology.
Imperative Ideation
Taking about the process followed while finding a valuable topic to develop and brainstorm upon, which is often one of the biggest challenges in research, Prof. Suri said,
"The early seeds of this work were laid back in 2018 while performing regular tests on nanoscale memory devices. We started scratching the surface of the idea somewhere in 2019 when we observed some very peculiar behavior in the nanodevices. We asked ourselves- Can we utilize this behavior for building something useful? That is how the idea was initiated. The first step was to validate our drawing board intuition through some simulations followed by experiments."
Talking Tech
Asking about the model's functioning to gain better insights, Prof. Suri explains that the DEXAT neuron model improves the performance of neuromorphic Recurrent Spiking Neural Networks - RSNNs (a neurochip hardware that is highly efficient and provides real-time solutions) by providing faster convergence, higher accuracy, and flexible long short-term memory. The team presents a hardware-efficient methodology to realize the DEXAT neurons, which uses tightly coupled circuit-device interactions. Using experimentally extracted parameters, they simulated a full RSNN that achieves a classification accuracy of 96.1% on the SMNIST (A subsampled MNIST data used for training the model) dataset and 91% on the Google Speech Commands (GSC) dataset.
The demonstration also includes complete end-to-end real-time inference for speech recognition using actual fabricated resistive memory circuit-based DEXAT neurons. The final task investigates the impact of nanodevice variability and endurance, illustrating the robustness of DEXAT based RSNNs.
Planning the process and solving the pain points
Conducting the scientific research process requires you to dedicate time and effort to the planning process. When asked about how was the process on way, getting to know that one is on the right track to solve the problem, Prof. Suri replies,
In research, when you are up against new concepts or first-time observations, the process is fascinating and tricky at the same time. You need to have a good amount of trust in your hypothesis and apply a very systematic, rigorous, and meticulous verification to pass the observations. Multiple repeated experiments, detailed statistical and theoretical confirmation of the observations gave us the confidence that we are on the right track.
Not to forget, there often exist difficulties while carrying out significant research works, for in the pandemic time. It poses problems and makes working laborious.
Pulling through this project amid 2 COVID waves was the toughest part. Given that a significant portion of the work was experimental in nature, during the lockdown conducting physical laboratory experiments became extremely challenging. However, with planning and not leaving any moment unutilized, we were able to complete the required study.
said Dr. Suri.
Seeking Sustainability
While deploying an AI model, one needs systems that dissipate a lot of energy and which consumes a lot of power. Traditionally, most of the models are very energy-hungry systems, posing a question on the sustainability or the carbon footprint of the technology.
While technology might be beneficial, we need to be mindful about what is its impact on nature, climate, and all those things. That takes us to the point of energy efficiency. The idea was that we have to design AI systems, models, or hardware in such a way that it is energy-efficient and more sustainable.
The outcome successfully demonstrated state-of-the-art results in terms of accuracy, latency, and hardware implementation for adaptive spiking neurons applicable to the domain of recurrent spiking neuromorphic networks (RSNNs), being energy efficient at the same time.
Bringing up rear
Further talking about what comes next, the possible improvisations and to where does the work fit into the progress of the field, Prof Suri concludes,
"We can exploit new theoretical models only by building systems based on advanced nanoelectronic semiconductor devices. We have also filed a patent for the invention. The next step is to build full scale neuromorphic hardware based on the research for deployment and product development."
As the study has attained positive results not only in the hardware part, but also the theoretical aspect of the technology, it was recently published by the international journal, Nature Communications.
Learnings and takeaways
Our success in mimicking nature has been related to advances in science and technology. The world will be a better place for successive generations if technology works in transformative and unseen ways. While the human race's inherent intelligence has produced ubiquitous inventions, the convergence of human intelligence and artificial intelligence will amplify growth and deliver sustainable progress.
