Blog

Yan LeCun, Facebook’s AI chief and considered to be one of the godfathers of AI reacted to Elon Musk’s contention that AI will supersede human intelligence in the next five years. The world has seen ChatGPT (2022), AI chatbots of Google and Microsoft. AI has been progressing and one idea that emerges every now and then is AI surpassing human intelligence and then taking control of the world. LeCun counters this idea by saying that if this is so, then the AI systems could have learnt driving cars with 20 hours of practice, just as the teenagers do. However, we are far away from a fully autonomous car though we have millions of hours labelled training data. As for AI taking over the world, the idea is ridiculous.

AI may pose an existential threat. This concept is based on the assumption of a hard take off. This theory means, the moment a super-intelligent system is turned on, it will improve upon itself on its own, and be more intelligent than humans, ultimately leading to the destruction of the whole world. This is a joke, since no process in the world is exponential for very long. Such systems would have to recruit all the resources of the world. And infinite power.

As we know, a large chunk of retail revenue is generated by the unorganized sector. In 2023, it is 80 per cent of the total revenue. However, as modern retail and e-commerce picks up, the contribution of unorganized retail sector will go down to 71 per cent in 2030. Currently, modern trade contributes 12 per cent of the total retail revenue, and e-commerce 8 per cent (2023). By 2030, the share of e-commerce will rise to 17 per cent.

The new commerce model consists of D2C (18 per cent share), social commerce (46 per cent share), quick commerce (20 per cent share) and live commerce (17 per cent share). It is a $15 billion market in 2023.

Retail will have to improve the ease of product discovery, prompt of point of purchase support and service and customized offers and communications.

Facebook has introduced Mobile LLM. It is a less-than-billion parameter model for use on devices. It avoids sheer quantity of data and parameters. The architecture is deep and thin. There is sharing of embedding and group-query attention mechanism.

Accuracy is achieved by block-wise weight sharing. It is suitable for chat and API calling. There is shared information between different parts of AI. It makes the phone smarter, and it does not slow down. This approach enables deployment of powerful AI models directly on consumer devices.

Organizations are already adding generative AI features on smartphones. Mobile LLM extends it beyond. It is a shift to sustainable and accessible AI with good computational possibilities on user’s device.

A pathbreaking paper in the field of artificial intelligence (AI) is the 2017 paper Attention Is All That You Need jointly produced by eight research workers. The team leader Ashish Vaswani was assisted by seven other persons — Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser and Illia Polosukhin.

Ashish Vaswani is co-founder and CEO at Essential AI, San Francisco, California since 2022. Previously, he worked with Google (Nov 2021 he left Google). Later he was co-founder and chief-scientist at Adept AI Labs.

He is a PhD from University of Southern California. He passed out from Birla Institute of Technology and Science (1998-2002).

Naom Shazeer is CEO, Character AI, Palo Alto, California, USA. He personally designed multi-head attention, the residual architecture, sparsely-gated Mixture of Experts (2016), Mesh-Tensorflow (2018), TS (2019), LaMDA (Google, major contributor).

He builds personalized systems at Character AI to enable all humans to lead their best lives.

He left Google (he was principal Software Engineer) in 2021.

It is a matter of pride for Indians that Niki Parmar is from Pune, India. She got her bachelor’s in engineering degree from Pune Institute of Computer Technology followed by a Masters in Computer Science from University of Southern California. She worked in Google and left in 2022. November 2021. Right now she works at Essential AI which she joined in January, 2023.

Jakob Uszkoreit is CEO and co-founder, Inceptive Nucleics, USA. As on intern, he left Google in 2007. He rejoined Google as Senior Software Engineer in 2008, and again left Google in 2021. He founded Inceptive in 2022.

He has master’s degree in computer science and Math from University of Berlin.

Lion Jones is cofounder — AI Research Scientist, researcher at University of Birmingham, Sakana AI, Tokyo, Japan.

He left Google in 2023. He was AI research scientist at Stealth. He joined Sakana in 2023.

He has earned master’s in advanced computer science from University of Birmingham.

Adian Gomez is a British-Canadian computer scientist working in AI with a focus on NLP. He is cofounder and CEO of Cohere.

He left Google in 2018 and became a researcher. He cofounded Cohere in 2019.

He is an Oxonian — a PhD in computer Science. He earned his bachelor’s in computer science from University of Toronto.

Lukasz Kaiser is at present Member Technical Staff at OpenAI, San Francisco, California.

In the last few years, he has worked on ML using neural networks. He left Google in 2021, and joined OpenAI.

He has a doctorate in computer science from RWTH Aachan. He did his master’s from University of Wroclaw (Poland) in computer science. He also earned Masters in Math from University of Wroclaw (Poland).

Illia Polosukhin is cofounder, NEAR Protocol. He is an AI researcher. He was engineering manager at Google. He joined NEAR Protocol in June 2017.

He has master’s from University of Kharkiv Polytechnic Institute in Applied Math and Computer Science.

Indian advertising industry will go beyond Rs.1.1 trillion by the close of 2025. It shows a compound growth rate (CAGR) of 9.86 per cent. (Dentsu Digital Report, 2024). In the same period, digital advertising will rise to Rs.62000 crore, showing a CAGR of 23.49 per cent.

In 2023, the advertising market size was of Rs.93,100 crore.

Indian advertising was fueled by events such as IPL, ICC Cricket World Cup, Women’s World Cup, Asia Cup and other cricketing events. In addition, there were Assembly Elections. In 2024, the positive momentum will continue with the general elections and IPL.

In 2023, digital media surpassed the TV media spend. Its share is 44 per cent in total Indian advertising (Rs. 40,600 crore). TV’s share is now 32 per cent (Rs. 29,800 crore). Print media stands at 20 per cent (Rs. 18,600 crore).

In digital media, social media contributed 30 per cent share (Rs. 11,960 crore), followed by online video with 29 per cent and paid search with 23 per cent.

By the end of 2024, TV’s share will decline from 32 per cent to 28 per cent. It may reach 25 per cent by the end of 2025. Audiences may be engaged through connected TVs (CTVS). There will be a shift to streaming access. By 2027, CTV households will be 100 million, and ad spend on CTVs is likely to touch $400 million.

Print media will shrink further. By the end of 2025, it will have a share of 16 per cent.

Radio will continue to maintain a share of 2 per cent in the coming years.

AI will affect the advertising field by making ad campaigns targeted and personalized. Some instances where AI will affect advertising follow:

Personalized recommendations: Algorithms powered by AI continuously analyze consumer preferences and behaviour and accordingly offer personalized ad recommendations.

Chatbots: Chatbots and virtual assistants would answer customer queries, engage with the customers, provide them guidance and information.

Predictive analytics: Predictive analytics could be leveraged to get the most out of their strategies.

Programmatic advertising: AI-assisted programmatic algorithms would analyze user data and behaviors and make real time bidding decisions. It ensures precise targeting of the ads.

Dynamic creative optimization: AI assists to create personalized messages in the light of user attributes and behaviour so that the content resonates with specific interests of the audience.

Fraud detection and prevention: ML identifies patterns in the data to identify fraudulent activities.

Voice search optimization: There are voice-activated devices. AI optimizes ad content for voice search.

LLMs are built on a corpus of vast data, and in order to build more capable models the entire public web has been used. All other public and private data sources have also been tapped (books, research papers, private data).

It is possible that in order to search for more data, firms could have taken transcripts of YouTube videos, though doing so could breach the law. In fact, when OpenAI was questioned about this, its CTO responded by saying that she was not sure. Google itself has scraped transcription data (private data) from YouTube videos to train its own models.

In fact, data harvesting has been ingrained in the business models of large tech firms such as Google and Facebook (Meta). Though licensing of copyright book material is possible, it takes a long time.

The crucial difference between an efficient model such as ChatGPT and other models is the data volume.

In past, there were instances of Facebook sharing user data with third parties. Of course, Facebook is in an advantageous position in the AI race as it sits on mountain of data available on Facebook and Instagram.

It is also important to train LLMs on unique data. On social media, one comes across billions of posts, images and videos contributed by the public. Social media tries to remain transparent about the ways such data is used to build products and features.

This traffic of data sharing runs across the players-one using the data of the other player and vice versa. It is just scratch and grab mentality. Mining data has become a multi-trillion business.

After all, all these firms live in the glass houses. Who will throw stones on others? Maybe, the thrower could be at the receiving end tomorrow.

There is niche called brain computer interface (BCI) — it consists of electrodes that either penetrate the brain or sit on its surface to provide direct communication to computers.

Such brain implants are useful for patients who are paralyzed due to neurodegenerative disease –amyotrophic lateral sclerosis, stroke and multiple sclerosis.

Elon Musk, as we know, has set up a startup called Neuralink, a brain implant unit. A rival company Synchron also operates in this space. It is roping in patients for clinical trials at 120 centers. Synchron is New York-based company. It received US authorization for preliminary testing in July 2021. It has tested in the US and Australia with no adverse side effects. The aim is to help paralyzed patients type on a computer using devices that translate brain signals.

A larger trial awaits the nod from the US FDA. Neurosurgeons at University of Buffalo and University of Pittsburgh are collaborating on preliminary study. Data is being collected while the brain implant is used.

Successful testing could benefit a significant population.

A model which has several billion parameters can cost a lot in terms of training. There are several factors that affect the training cost — infrastructure used, the complexity of the architecture, the size of the training dataset and the duration of the training process.

This training involves extensive computational resources — high-performance GPUs or TPUs, and substantial amount of electrical energy to power these chips. This continues for the entire training period.

It could cost millions of dollars, primarily due to high cost of chips and energy consumption. These costs fluctuate over time due to advancements in hardware efficiency, changes in energy costs and improvements in training algorithms.

LLM’s capability is expressed in terms of parameters since these relate to the model’s complexity and capacity to understand and generate language.

Parameters are essentially weights and biases learnt during training. These determine the model’s ability to process and generate text.

A large number of parameters generally enable a model to capture nuanced patterns in data. It leads to improved performance in NLP tasks. It is thus a practical measure of a model’s size and complexity, reflecting its potential.