Blog

These days industry is receiving STEM graduates well. STEM stands for Science, Technology, Engineering and Mathematics. All STEM professional’s are not on par. Alumni of elite premier institutes such as IITs, NITs, IISC are very well-received here in India, but they go abroad for better prospects.

The quality of STEM education in India is not uniform. Top tier institutions do maintain standards. Many institutes have poor standards, outdated curricula, shortage of faculty. Management of these institutes are interested in commercializing education.

India produces 15 lac engineering graduates a year, of which only 2.5 lacs gets worthwhile employment. As against this, US produces 70000 engineers annually. The starting salaries for engineers there is $50000-$70000 per annum. The salaries increase and keep pace with the cost of living. By 2020, the average salary of engineers in the US hovered around $70000-$100000 per annum. The salaries are higher in Silicon Valley. In India, the salaries for the last two decades are Rs.2.5 lac to Rs.3.5 lac per annum. It is sheer exploitation. Most of our engineers are underpaid and underemployed.

There are face search engines, say Clearview AI and PimEyes, which have the capability to pair photos from the public web. These tools are available to the police which can identify a snapshot of someone by comparing it with online photos where the face appears. The identification reveals the name, social media profiles and other information which perhaps the person would not reveal to the public, e.g. risque photos.

These are technological breakthroughs. And the breakthroughs are ethical.

Tech giants such as Facebook and Google had developed the face recognition technology years earlier but they had held back the technology, thinking that it is too dangerous putting a name to a stranger’s face.

As far back as 2017, at Menlo Park, California, the HQ of Facebook, Tommar Leyvand, an engineer demonstrated a facial recognition software. It identified with a mobile camera the face of Zach Howard, and Howard confirmed the identification is right. The phone then identified several people correctly. This technology is a god-send for a person with vision problems or face blindness. However, it was risky.

Facebook had previously deployed facial recognition technology to tag friends in photos, but there was a hue and cry about the privacy. It was in 2015. Facebook faced a lawsuit costing company $650 million.

The new facial recognition software of Leyvand could enable users to recall the name of a colleague at a party or search someone at a crowded place. Still, Facebook did not release the version. In the meantime, Levyland left Facebook and joined Apple to work on its glasses — Vision Pro AR.

As early as 2011, a Google engineer worked on facial recognition tool. Months later, Google Chairman Eric Schmidt declared that the technology has been withheld.

With recent releases of the startups, the taboo has been broken. Facial recognition technology has the potential to become ubiquitous.

It helps the police to solve crimes. Authoritarian governments use it for surveillance of their citizens. It can soon become an app on our phones, or in Augmented Reality (AR) glasses. We will usher in a world with no strangers.

We already know that AI models in training and operations consume lot of electricity since the hardware used has GPUs as accelerators. In order to make data centres greener, we have to focus on power consumption. AI will account for 20 per cent of the total electricity consumption by 2030.

To begin with, we should make hardware efficient by using server virtualisation and containerization. As far as possible, we should use renewable energy, e.g. use of solar panels, wind turbines etc. Data centres should use effective cooling mechanisms, e.g. using hot aisle and cold aisle containment, air flow control, liquid cooling systems. There should be optimised resource allocation. Lastly, data centre infrastructure management (DCIM) should be deployed.

Building an AI model such as ChatGPT is highly resource consuming. It is difficult to measure all the costs. Most people are not aware of the resource usage underlying ChatGPT.

While building an LLM, we have to train the model on vast human-written text. It takes lot of computing, and hence lot of electricity. That results into heat generation. To keep it cool on hot days, data centres need to pump in water — often to a cooling tower outside its warehouse-sized buildings.

Microsoft’s water consumption increased 37 per cent from 2021 to 2022. The consumption reached 1.7 billion gallons — equal to 2500 Olympic-sized swimming pools. Most of this added consumption is due to AI.

Microsoft sourced the water from Raccoon and Des Monies rivers in central Iowa to cool a powerful supercomputer used to teach AI systems how to mimic human writing.

The growing demand for AI tools carries hefty costs, from expensive semiconductors to increase in water consumption. Few people in Iowa knew about its status as a birth place of GPT-4.

When users ask ChatGPT anything in a series of 5 to 50 prompts or questions, it gulps up 500ml of water. It varies depending on where the servers are located and the season.

There is indirect water usage, such as to cool power plants that supply electricity to the data centres.

Google too uses water. Its consumption doubled outside Las Vegas. It was also thirsty at Iowa, drawing more potable water to its data centres.

Microsoft is doing research to measure AI’s energy and carbon footprint. It is also working on ways to optimise resource consumption.

Of course, building a generative AI model from scratch requires a lot of effort. First it is necessary to build a foundation model of your own. India has the National Programme on Artificial Intelligence and it is working to build a generative AI model. India plans to build its own version of ChatGPT. It will be a large language model (LLM). India wants to make generative AI available as a digital public good on the lines Aadhar, UPI, Digilocker etc. to other countries.

The Ministry of Electronics and IT is organising the Global IndiaAI 2023 conference where a wide range of topics will be covered including Next Generation Learning and Foundational AI models, AI’s applications in healthcare, governance, and next generation electric vehicles, future AI research trends, AI computing systems, investment opportunities and nurturing AI talent.

The government is showcasing the AskGITA(Guidance, Inspiration, Transformation and Action) generative AI interface at G20 Summit. This model is just like ChatGPT and is based on GPT-4. This model will answer questions related to life, based on the feed from the holy Bhagwat Gita.

India has developed Bhashini, its digital repository of digital content in 22 Indian languages. India’s generative AI model will use Bhashini.

Private companies such as Amazon, Tech Mahindra, Infosys, TCS, Wipro, HCL and others too have announced generative AI models to benefit their customers. CoRover.AI, UK has announced its generative AI model BharatGPT which support 12 Indian languages and over 120 foreign languages, through text, and even video.

There is a rise in the share prices of Nvidia, the company making AI chips. The growth of AI has escalated the demand for GPU chips. Big Tech vie with each other to buy as many GPU chips as possible. Nvidia chips are being bought by the companies in Saudi Arabia and UAE. Chinese companies such as Tencent and Alibaba stand at the door of Nvidia. It shows the great demand for AI chips.

The demand will rise further because of the chatbots and other tools.

Jenson Huang is the CEO of Nvidia. Recently he met our PM.

RIL and Tata Group entered into separate pacts with Nvidia to develop advanced AI applications in India. The tie-up is for creating AI infrastructure in the country and to develop foundation LLM for generative AI solutions as well as for training and skilling the workforce.

Nvidia will provide Jio end-to-end AI supercomputer technologies, including CPU and GPU, networking, AI operating systems and frameworks — all this to build most advanced AI models. Jio will manage and maintain AI cloud infrastructure and oversee customer engagement and access.

To the Tata Group, Nvidia will provide its expertise in upskilling 6 lac TCS employees in AI. It will collaborate with Tata Communications and Tata Motors to build AI infrastructure and build AI across design and style for vehicles.

Jio will create super centres to provide catalytic growth.

Nvidia will provide access to the most advanced Nvidia GH200 Grace Hopper Superchip. Nvidia will also provide access to DG Cloud, an AI Supercomputing service in the cloud.

As India has scale, data and talent, given the most advanced infrastructure, it can build its own LLMs that power generative AI applications made in India. To handle computational demands, RIL will create 2000 MW of AI-ready computing capacity, across both cloud and edge locations.

AI will be accessible to researchers, startups and enterprises across India.

Nvidia has operations in Bangalore since 2004. These have been set up almost two decades ago. There are four engineering development centres — Gurugram, Hyderabad, Pune and Bangalore. Nvidia has almost 4000 employees in India. In addition, it has 3.2 lac India-based developers.

Mainframes had run the world for more than 70 years with organisations such as large banks and financial institutions, oil and gas companies, aviation business, manufacturing and mission critical apps all being maintained on mainframes.

Mainframes in business apps used Cobol. As more platform-agnostic computer languages developed, there happened a migration — either partial or full. Some still continue to use mainframes for their core operations.

Many thought mainframes are obsolete. However, there is renewed interest in mainframes — it has lately been recognised that both on-premise and cloud deployments should co-exist.

In addition, mainframes market is likely to grow on account of adoption of Internet of Things (IoT) and production of massive data.

It is not economical to transfer the entire applications to cloud. To achieve economies of scale, companies continue to use mainframes. Mainframe systems are robust, support volume, variety and velocity of data. These days there is focus on data analytics, and maintaining data on mainframes lead to quick data integration solutions.

However, manpower working on mainframes is retiring. New professionals are not skilled in mainframes to replace the outgoing manpower. Academic institutions do not teach mainframes and Cobol curricula. Out of top 100 banks, 92 have invested in mainframes. And 70% of large corporations have invested in mainframes. To serve this sector, there is dearth of manpower.

Prior to moving apps from mainframes to cloud, organisations examine whether the front end could be maintained with new tools while retaining backend on the mainframes with connectors.

Organisations can tap the retired manpower and ask them back to work to meet the manpower shortage. The tenure of retiring persons can be extended. IBM and universities should continue to teach this curriculum. New generation should be motivated to build careers in mainframes.

For the first time, Cobol, the programming language was used in 1959. It still powers many critical systems, but the number of Cobol developers are declining. The way out is to convert Cobol codes into Java codes. There are many Java coders but the conversion task is very laborious.

IBM announced in September 2023 that generative AI will be used to do this conversion. ChatGPT was introduced towards the end of November 2022. Generative AI models are excellent at code generation. They can be made to learn codes written in past decades. All that one has to do now is to set the content, set the direction and the machine will do the conversion. The costs come down dramatically.

Huawei, a Chinese company released Mate 60 Pro smart phone carrying a chip (advanced system-on-chip processor) manufactured by Semiconductor Manufacturing International Corporation, Shanghai. The smart phone has speeds in excess of 350 megabits per second. It is thus on par with Apple’s iPhones with 5G speeds.

There are restrictions on export of manufacturing tools that can turn out processors at 16 nm or below. China has not published specifications of the chip inside Mate 60 Pro. However, its performance points to a processor at 7nm or better.

Has China found a way to bypass the sanctions led by US? Has Chinese electronics found a way to circumvent the curbs? The US efforts focused on Chinese restricting Chinese abilities in semiconductor field to chips larger than 14nm, eight years behind the current technological frontier.

SMIC, China has already achieved 7 nm capability as per TechnoInsights on an earlier chip for Bitcoin mining (MinerVA7). It means the latest development is evolutionary. There is no reason for China to cheer up and for the US to get worried.

Older tools are capable of being used for such progress. They can make more advanced semiconductors. There is an approach called multi-patterning. Here silicon is exposed multiple times to light for marking the circuit design, instead of a single exposure. SMIC could achieve 7 nm by running this lithography step four times or more. Of course, there is increase in the number of tools required and costs. The whole manufacturing throughput slows down. However, the extra costs being marginal can be compensated by economising manufacturing process elsewhere.

Mate 60 Pro could have been powered by a new Kirin 9000 chip. Policy makers will have to consider whether equipment curbs do really work. The current restrictions may not prevent China from getting 5 nm. Of course, China will still trail behind the leaders by many years. And there is a limit to squeeze more out of old equipment. China will be stuck, while the rivals advance. And the US and allies have many other ways to tighten the sanctions. They can further add materials to the list.

SMIC has considerable finance (about $30 billion) for technological catch up. US-based groups who lobby for its electronics industry estimate the building up of 23 facilities for semiconductors on the mainland. It has investment estimated to be worth $100 billion. There is greater government support. Washington will have to do some hard thinking.

As a concept AI is with us since decades, but has taken a big leap in the last decade with the surge in computation power, new algorithms and ML models, and the availability of vast amounts of digital data to train AI models.

AI is a good predictor, for instance whether we will lose a particular customer, or whether the engine is likely to fall. AI is a good classifier, for instance, whether something is spam, AI is good in clustering, for instance grouping customers or students so as to serve them better. Of late, we have been using generative AI with the capability to generate content, thus making it useful for so many new applications.

Large language models can be pre-trained, fine tuned models. Further they can be instruction-tuned or RL-tuned. This write-up offers you the implications of these terms.

Pretrained LLMs

Pretrained models have been trained on vast datasets. These are foundational models. Their learning includes the learning of patterns, grammar, facts and some reasoning abilities.

Pretrained models leverage the accumulated knowledge over the years. It is the beginning and makes sure that the model has mastered the nuances of language.

Pretrained models are similar to a library with many many books in the mind. They are a repository of knowledge.

Fine-tuned LLMs

A pre-trained model is further trained on a specific dataset. It makes the model ready for a specific task. A fine-tuned model retains its vast general knowledge, but also becomes a specialist in a specific domain. It could be healthcare. Imagine a physician fine tuning model in cardiology.

Instruction-tuned LLMs

These models are fine tuned using textual instructions. These models do not rely on vast data, but they rely on the instructions provided to them. They are in fact a bridge between generic responses and task specific outputs. Their answers are aligned to the intent of the user. Imagine a model that teaches a cooking recipe. With the instructions received by the model, it can teach anyone the art of making a good dish. It is like directing the narrative.

RL-tuned LLMs

RL, as we know, is reinforced learning. Here the model learns from the feedback. While interacting with the environment, the model either receives rewards or penalties based on its actions. It refines its behaviour over a period of time. This feedbacks is iterative loop. It can be adapted in real-time. The responses are honed and the performance is improved.

A musician may hit a wrong note occasionally, but with such an error, he adjusts to make the next rendition better. RL-tuned LLMs work on these lines, by refining the output in the light of feedback received.