Blog

These are the days of generative AI which performs natural language processing functions. You must have come across chatGPT and must have experienced its magic. That makes you interested in natural language processing or NLP.

How can you go about to gain knowledge about NLP?

1. As a first step, you must learn Python and Machine Learning basics. In ML basics, you must know about data pre-processing, exploration, evaluation and selection. Get familiar with both supervised and unsupervised learning. ML in Python’s library is Scikit-learn.

2. Move to deep learning fundamentals by learning about neural networks and the way they process data. Here you should learn the concepts of activation, loss and optimization in training the neural network.

After forward pass, the backpropagation is used in neural networks Gradient decent is used as an optimization. Make yourself familiar with TensorFlow and PyTorch.

3.It is time for you to understand tokenization of words. Also read about stemming and lemmatization –these techniques reduce words to their root forms.

NLP consists of several tasks — sentiment analysis, machine translation, question-answer and so on.

Learn about part-of-speech tagging and named entry recognition.

You should know the characteristics of a language — syntax, semantics, dependency parsing.

4. Traditional techniques prior to deep learning such as Bag of Words, conversion of data into numerical form and nGrams were used.

5. Learn about word embeddings using Word2Vec. Learn about Glove too. Learn about RNNS, CNNs, Long Short Term Memory (LSTM).

6. Learn about Transformers, and attention mechanism.

Many are under the wrong impression that learning a few ML algorithms is all about data science. In fact, data science expects you to work with data. The focus is on data processing. The first step is to understand the data, its types (say numerical, textual etc.) and methods of analysis. The data structures such as tables, arrays and lists require your attention.

The data we are dealing with must be clean. Here we are focusing on data quality. There could be missing values, some removal of records, and dealing with outliers. The accuracy and consistency of data is required. Cleaning makes you do data preprocessing and outlier detection.

From the clean data, we have to extract relevant information for analysis. It is called data filtering.

Data is then transformed — range adjustment or scaling, normalization, encoding.

Data sets may have date and time information which requires parsing.

There should be data merger or combining of rows and column joining. This is consolidation of information. There are data aggregation techniques. These facilitate summarization and analysis of sub-sets. This is useful in integrating data from various sources. One can also handle relational data.

The above theoretical background must be applied to real-world projects. One can start with small datasets. Gradually more complex data can be tackled . Application hones your skills.

AI has made available chatbots which could be used as teaching bots. Harvard University plans to employ an AI chatbot to teach some parts of its popular course — Computer Science 50 : Introduction to Computer Science.

Upgrad, an edtech company, has acquired Harappa Education. It has launched a chatbot called thriveBOT which has been fine tuned for three languages — English, Hindi and Hinglish. ThriveBOT is a virtual teaching assistant. It provides instant support to learners. It encourages in-depth thinking by addressing questions posed by learners.

Some skills have been identified as those which help you in work place — these are called ‘thrive skills’. These include teamwork, public speaking, effective communication, taking ownership, prioritization, credibility, resilience, crisis management etc. These skills are as important as technical skills. Harappa Education provides these ‘thrive skills’.

Harappa Education caters to MNCs and other corporates to train their employees. It also trains individual employees and small teams and runs management development programmes for working professionals.

GitHub Co-pilot, generative AI development tool has transformed the coding function. The tools automate the routine work or assist programmers with complex tasks. GitHub Co-pilot enhances the productivity of the developer, and the time saved can be utilised in doing more satisfying creative work.

It was necessary for tech industry to communicate with machines in natural language, say English, rather than programming languages, say Python and Java. With GitHub Co-pilot, the world has come closest to that dream.

GitHub Co-pilot commenced a new world of software development. There is pairing of AI and programming. It auto-completes comments and code for the developer. GitHub Co-pilot, two years after its birth, is writing 46 per cent of code. It accelerates coding process.

Co-pilot chat answers the questions. One can receive additional context. There is immediate content-specific support. All this happens in Editor or IDE.

It is trusted by a large number of organisations.

GitHub is Microsoft-owned. It stores, hosts and shares code. Many Indian developers and organisations use GitHub.

Just like GitHub Co-pilot, Copilot chat is a developer first. It is everywhere in the development cycle. Besides it is an enterprise ready product. Microsoft is the first-to-market such a tool. It keeps on refining it in the light of the feedback received by its user base. It is an AI assistant that can efficiently handle the entire DevOps Workflow.

APIs enable different software apps to communicate with each other. In fact, they consist of a set of rules and protocols. In short, they intermediate between the programmes. It allows access to a particular programme or service even though the developer is not aware of its internal workings. AI APIs are those which provide access to AI capabilities and functionalities.

The following are the AI APIs catering to different domains.

Computer Vision API

Here the vision APIs lead to comprehension of images and videos. There are facial recognitions, scene descriptions and object identification. The two such candidates are MS Azure Computer Vision API and Google Vision AI.

ML and Data Analytics APIs

Tools are provided by these APIs to train ML models. Predictive models are built by the developers. To illustrate, sci-kit-learn and TensorFlow.

Speech Recognition and Synthesis APIs

These convert spoken language into text and vice versa. We find them in voice assistants. They can be used in transcriptions. For instance, IBM Watson Speech-to-Text and Amazon Polly.

Natural Language Processing (NLP) APIs

Developers can analyze and manipulate language. The tasks they can execute are translation, summarization, and sentiment analysis. To exemplify, GPT API and Cloud Natural Language API (Google).

Working of APIs

APIs require a key or credentials provided by the service provider. There should be requesting for API services, say a text string as an input. AI APIs process the request — input data using their training and algorithms. Such work requires complex computation using GPU chips or TPU chips.

AI model ultimately infers patterns, extracts information and executes the intended task.

Response

The developer’s application gets back a response after AI processing. The response depends on what our prompts are.

Integration to Applications

The response given is integrated into the applications. This way users interact with the AI-assisted features seamlessly. This integration happens on internet or mobile apps, IoT devices, chatbots and so on.

Advantages of AI APIs

Developers have not to create models from scratch. Models which are pretrained are used to take benefit from APIs. These APIs are scalable. They are cost efficient, as businesses pay as per usage. They are accessible to broader audience. They are constantly updated.

ChatGPT was launched last November (2022). It has added a new feature now — ChatGPT Vision. It opens up limitless possibilities. The Vision feature introduced towards the end of September, 2023 enhances the multi-modal capabilities of ChatGPT. Users can now upload images, and ask questions. It can read the cluttered sign-board with ease. It can write code from a screenshot of a SaaS dashboard. It can break down diagrams. It can interpret cartoons and comics. All techies from Silicon Valley to Bangalore are impressed.

There is of course, a new gold rush. Excited we are about the possibilities and paranoid we are about its ramifications.

Generative AI, as we know, uses the computational power of Graphics Processing Units (GPUs). Unlike the traditional central processing units (CPUs), GPUs can process multiple calculations simultaneously (parallel processing) and are therefore optimal for training AI and deep learning models. Previously, GPUs were used for image processing.

There is demand for GPUs from established firms as well as startups. There is scarcity of GPUs, as both the business enterprises as well as startups are trying to leverage AI.

By global comparison, India’s requirement for GPUs is small. The US AND China have big demands.

Companies can access GPUs from cloud service providers. Here too those with big demands for GPUs get priority in getting access as they can pay faster. Alternatively, we should get the GPU chips directly from the GPU makers such as Nvidia. There is a waiting time involved that runs into months.

Scarcity also makes GPUs expensive. Big firms receive their supplies directly from the makers. They are hesitant to ship GPUs to India, since the ticket sizes are small and the payment capacity correspondingly smaller. Such big firms include Amazon Web Services (AWS), Microsoft Azure and Google Cloud.

Governments acquire GPUs by committing large finance. Governments make available GPUs for research purposes and for commercial use. The UK has acquired GPUs from Nvidia and vital components from Nvidia, AMD and Intel. Saudi Arabia has acquired 3000 H100 Nvidia chips (at $40000 a piece) to run a supercomputer at a science university. The UAE too acquires Nvidia chips through state-funded business. Chinese companies buy 1 lac high performance A800 GPUs, and have contracted for further purchases in 2024.

India should seriously think about having sovereign AI capability. The government can also persuade GPU making firms to set up plants in India. CDAC in India is providing controlled access to GPUs to AI startups.

TCS, Infosys and Jio are having tie-ups with Nvidia, the Santa Clara-based company.

India requires almost a million experts to develop AI and generative AI systems. India requires 20-30 million domain experts who can use AI and generate AI tools. India must train most of its population to become AI-literate. The government must also draft regulatory framework so as to make AI ethical and responsible. IT companies must build safeguarding features in the AI systems. AI systems should be classified according to the risks they pose to users. There should be some algorithmic accountability including bias.

AI laws must not stifle innovation. There should be deliberations among policy makers, regulators and technologists.

Though poets all over the world in general and Urdu poets here in India in particular have generally praised the moon, in reality it is a pockmarked satellite of the earth. Still the moon is so fascinating that it pulled cosmonauts to it. The last flight to the moon was Apollo 17 mission where astronauts moonwalked for 75 hours. The world watched with awe on TV their expedition on the moon some 2 and a half lac miles away from the earth. They returned to the earth in December 1972. Since then not a single soul from the earth has ever returned to the moon.

NASA has planned the return to the moon mission called Artemis. Artemis is the twin sister of Apollo. In November 2022, Artemis I, the first moon mission circled the moon and came back. It was an unmanned flight. Artemis II proposes to carry four crew members, of which one will be a woman, and one black person for the first time. The mission is proposed in November, 2024.

One year later, Artemis III will follow. Two more manned missions are planned by 2030.

NASA will explore the possibility of inhabiting the moon. Housing is a necessity. Perhaps this dream could be realised by 2040. Later, the possibility of Living on Mars can be explored. Though ambitions, this goal is not unattainable.

A 3D Printer can be used to create houses. It is additive making — layer by layer. The raw material can be derived from lunar soil on the surface of the moon. Such a project can be executed in collaboration with private sector and universities.

Future space missions need new programmes, machinery and equipment.

Lunar dust is fine and abrasive. It swirls around in circles. It is toxic if breathed in. Dust is both a problem and a solution. It could be used in housing as raw material.

Lunar environment poses problems to both constructions and dwellers. Younger generation may soon have their dream come true — living on the moon.

As we know, the Writers Guild of Hollywood was on strike, and one of the grievances was the use of their copyright works to train the LLMs. The strike ended with a new contract which carries an AI clause that allows explicitly, as per reports, the LLMs to use scripts written by members of the Writers Guild. However, the permission must be purchased from the writer, and not simply taken.

Users later create fan fiction using featuring characters from a particular TV series or film. Authors do not want to sue the readers for fan fiction. On the contrary fan fiction excites the audience. However, fan fiction is created by human beings working at a human pace. Amazon restricts authors of self-published books to not more than three a day in Kindle store. There is possibility of people uploading AI-generated material. Such material is created at unhuman pace.

There is a Google case precedent when the US Court of Appeals allowed it to scan copyrighted works into its database by invoking the doctrine of fair use. The court held by and large this generates only snippets for the users.

LLM companies can argue that they have just developed a tool, and are not responsible for its misuse.

Generative AI poses a risk of derivative works. An author can create new stories of the same characters for his or her excited audience. However, what if such stories are generated without any practical limit due to generative AI?

AI-generated material is not entitled to copyright protection (one federal court). Human authorship is the essence of copyright. Non-human actors cannot be incentivized by legal protection, especially of the copyright law.

Algorithms too these days smartly respond to certain queries saying I am sorry but I am not permitted to create fictional works derived from the copyrighted works.

LLMs contend that the use of copyright works for training is fair use, and any other rule that bans this will set back AI research. The copyright system should not put undue burdens on AI, and allow it to continue the innovation to proceed.

Generative AI and ChatGPT are magical and still throw many issues. If they are pressed hard for prolonged period, they may provide erroneous responses called hallucinations. OpenAI which launched its generative AI model scraped the internet without bothering about the consent of the content creators. Such scraping is necessary for training its model. OpenAI’s competitors in generative AI field did the same thing. The issue is whether these companies have the right to scrape content on the internet without express permission. Later this content is used for generating fresh content through the models. There is no payment made to content creators.

Prominent authors have sued OpenAI and many more are likely to join the suit. Even content creators, painters and photographers are aggrieved.

Google has now provided the website publishers a switch that enables websites to be available for web crawlers for searches but not for generative AI training. By using the tool, websites will have the freedom to be available only for search purposes or be available for search as well as training the model purposes.

Google may be sincere in its effort to bestow control of published material to websites, or it might have done so to avoid litigation. In any case, the issue of content ownership is highlighted in the midst of generative AI models.

Media abroad has taken steps to protect their websites and have incorporated tools not to allow lifting of their content for being used for model training. Indian media too have safeguarded their websites from the crawlers of generative AI companies.

Though such denial is good news, for the development of AI models they must be fed with content to train and refine them. Big tech has the capability to bypass the safeguards introduced by the websites. Still doing so will weaken their legal position.

Another option is to compensate the authors, content creators and web publishers. The issue here is whether this is practical — can they afford to pay for all the content they need for training and fine tuning? AI models are hungry for more and more content.

The idea of using synthetic data was put forward. It means to use machine generated content. However, it has not yielded good results. Ultimately, the solution of a compensation being paid to access the content from the net legally will be accepted by AI companies though not willingly but under pressure from court room nudging. It may not benefit small time content creators, and may work in favour of big fish.

Countries are grappling with framing laws to take care of such issues, but India lags behind here. Indian policy makers must realise that India generates huge digital data, second only to China. This is valueable for AI companies. Therefore, there should be a legal framework to deal with this.

China too has zealously pursued generative AI. Every other day we hear a new product announcement either from some startup or an established tech giant. Perhaps all of them may not survive, and a shakeout is imminent.

OpenAI ChatGPT introduced in late November, 2022 set China on a course of war of AI models. Tencent, Alibaba and Baidu all entered the race with their own offerings.

You will be surprised to know that there are at least 130 large language models (LLMs) in China. It constitutes 40 per cent of the global total. It is just a notch behind the US share of 50 per cent of the global total. China is in a close race. In addition, there are industry- specific fine tuned LLMs linked to the core model.

All these models may not be viable businesses. They are clones of each other, and they are not cost effective. It is difficult to speculate which models will survive in the long run. Maybe, only two or three general purpose LLMs will remain viable and will command the market.

Investors rely upon the experience of the founders before investing in these startups. Z&Y decided to back Baichuan Intelligence who has introduced an open source AI model on the lines of Llama 2. Baichuan’s founder is Wang, who floated a search engine — Sogou, and whose company was approved for releasing a chatbot. Google’s employees in China too are setting up some startups. Chines big tech Alibaba, Tencent and Baidu can have a head-start, and can offer generative AI as an additional plug-in.