Researchers strive to build better and better AI, but three Google researchers have found out that there is a limit to what has been currently called generative AI. It could be a damp squib on the plans of their superiors to travel to more advanced AI systems.
The paper is authored by a trio of Steve, Lyric Doshi and Nilesh. This paper still has not been peer-reviewed. The paper concludes that AI is not capable of going beyond its training dataset. The paper considers the transformer model — a transformer here converts one type of input into a different type of output.
This model’s architecture was first theorized by a group of researchers in 2017 (Vaswani et al) who wrote a paper called Attention Is All You Need. The researchers thought that the model generates text and other output, and therefore the model can do intuitive thinking on its own. If this is further refined, it can lead to human-level AI, called AGI.
Transformer models did create a lot of buzz — it was felt that they can go beyond their training data.
However, when these are assigned tasks or functions not covered by their training data, they fail, and their generalization is not extended even for simple tasks.
Thus a transformer model is not able to cope with anything if it is not pre-trained on it.
Since the models have been trained on billions of parameters, it was natural to expect that they would be able to cope with tasks on their own. These models have crammed so much knowledge into them, and there is not a whole lot they have not been trained on.
Can a model have some sort of emergent property in AI with enough training data? The research pertains to GPT-2. It is now obsolete. Further models GPT-3, GPT-3.5 and GPT-4 have appeared. Maybe, there could be an emerging AI. Or else, further research could adopt a new approach that overcomes the limitations of the present paradigm.
Of course, this research will sober the sizzle of AI. So far the model depends on the expertise humans already have. We will have to temper the AI expectations.
AGI presumptions require both time and further research. AI still cannot take leaps of thought that separates human beings from machines.
The voice has reached the ears of Sam Altman and Satya Nadela, and they have decided to put in a joint effort towards developing AGI.
Leave a Reply