These days, as we know, large language models (LLMs) are very popular in the field of artificial intelligence, AI. However, as we know, these require intensive computational resources to operate. Therefore researchers at Computer Science and AI Lab at MIT worked on an adaptable and efficient solution, especially useful in robotics and autonomous cars. They have come out with Liquid Neural Networks (LNNs). The research team was led by Rus. They were inspired by biological neurons found in small worms (such as C.Elegans) which perform complex tasks with just 302 neurons. These LNNs differ from traditional models as they do not use expensive computational hardware. The neurons are stabilized during training.
Actually, these models make use of dynamically adjustable differential equations. These adapt to novel situations after training. Typical traditional networks do not have this ability.
In essence, we enhance the representation learning capacity of a neuron. To begin with, the neuron stability is increased in training. Then non-linearities are introduced over synaptic inputs. It enhances the expressivity of the model, both during training and inference.
LNNs make use of a wiring architecture different from traditional models. There are lateral and recurrent connections within the same layer. Therefore, the maths and wiring both contribute to learning on continuous-time basis and adjusting behaviour dynamically.
The adaptation is in response to the inputs seen.
These are compact models. A classic model has around 1 lac artificial neurons, and 50 lac parameters to do a task of keeping a car in its lane. Rus and her colleagues were able to train an LNN to do the same task with just 19 neurons.
This reduction in size has its advantages. It runs on small computers. (used in robots and edge devices).
As it uses fewer neurons, the results are easily interpretable. It is easy to decipher how it has reached the decision. Just 19 neurons involved makes it possible to draw a decision tree corresponding to the firing patterns and the flow in the system. Is it possible to do so if we have 1 lac neurons?
The models also make us understand causal relationships. Traditional models struggle to learn this, and move over to spurious patterns unrelated to the problem. LNNs have a firm grasp of causal relationships.
Their generalisations are better for unseen situations. They focus on the task and not the context.
Attention maps drawn for LNNs show their focus on the task, say the road while driving or target object while detecting object. These adapt to the task, even when the context changes.
These models can be trained in one environment, but work fine in a different environment without further training.
LNNs are designed for continuous data streams — including video streams, temperature measurements and so on. There should be a time series of data. They do not work well for static data.
In robotics and autonomous cars, data is continuously fed to ML models. So LNNs work well.
LNNs have been tested for a single robot system, and Rus and her MIT collaborators would like to test them for multi-robot system.