Blog

Retrieval augmented generation or RAG enhances the capability of an LLM. LLMs, as we know, are the tools to transform vast amounts of unstructured data into usable information. LLMs could become outdated and cannot address a specific task by the time the model is put to use. LLM consists of billions or trillions of parameters — the number of neurons. RAG optimizes the output of an LLM by sourcing from external knowledge base in addition to the information on which it was trained. This external information could be sourced from organization’s proprietary data or other content to which it is directed.

In short, RAG expands an LLM’s knowledge base. This improves its accuracy as well as contextuality.

RAG uses search function to retrieve relevant data and adds this data to the prompt to get better generative output. RAG could be useful to retrieve public data on internet as well as data from private sources.

RAG was coined by Patrick Lewis, a research scientist associated with the startup Cohere. The term was coined in a paper published in 2020. As LLMs cannot expand or revise their memory, they at times hallucinate. RAG is one way to reduce hallucinations in generative AI results.

Apart from Cohere, there are other vendors who provide RAG-based apps — Vectara, OpenAI, Microsoft Azure, Google Vertex AI, LangChain, Databricks and Llamaindex.

Vector data base and graph technologies are used to retrieve proprietary data. A vector database stores, indexes and manages massive vector data. Companies use vector search capabilities in their databases. By 2026, more than 30 per cent enterprises are expected to adopt vector databased to ground their foundational models.

OpenAI when it was founded put a premium on benefits to humanity, rather than on profits. Elon Musk was a co-founder who later left the company. He has now filed a suit against OpenAI alleging that it has deviated from its original mission of creating open-source technology. Musk has always been a critic of AI and AGI. When ChatGPT was introduced in late November, 2022, he has raised concerns about advances in AI. Even today, OpenAI’s website reiterates that the development of AGI is for the ‘benefit of all of humanity.’ In reality, OpenAI has changed to a closed-source organisation. Musk suggested that OpenAI has become ‘a de facto subsidiary of Microsoft.’ Musk has sued for breach of contract, breach of fiduciary duty and claims of unfair business practices.

Musk has brought the suit in the capacity of a donor to OpenAI’s non-profit parent organisation, and is seeking to compel OpenAI to stop benefiting Microsoft and Altman personally.

Microsoft Research has introduced GraphRAG to improve upon RAG. As we know, LLMs are used in various sectors such as healthcare, finance, education and entertainment. LLMs leverage NLP, natural language generation (NCG) and computer vision (CV). The greatest challenge is to extend the power of LLMs beyond the data these have been trained on.

GraphRAG is an innovative method to improve RAG by using LLM-generated knowledge graphs. These can be used where typical RAG would not be enough to address the complex problem on private datasets.

RAG, as we know, uses vector similarity to determine search strategies. GraphRAG introduces knowledge graphs generated by LLMs. This modification improves the performance of the LLM in question-answer system.

RAG, in fact, addressed the issue of data not included in the training of the LLM. LLMs find it difficult to comprehend condensed semantic concepts and making connections between unrelated bits of data. GraphRAG is much more sophisticated. It performs better than baseline RAG, especially when the data is from multiple data sources. It provides an overview of topics and concepts by grouping the private dataset into relevant semantic clusters with the help of a structured knowledge graph. GraphRAG fills the context window with relevant content.

Microsoft has introduced 1-bit LLM, a new language model that quite differs from the traditional models. Research on BitNet has contributed to the development of this project.

What is new is the representation of parameters in the model. Parameters refer to the weights which use 1.58 bits. Traditional LLMs use 16 bit floating point values (FTP 16) for weights. However, BitNet’s each weight is restricted to -1, 0 or 1 values. It is a substantial reduction in bit usage.

Bitnet (b 1.58) despite the reduction performs as well as the traditional models. It is a cost effective model. Suitable hardware can be customized for BitNet model. It supports long sequences. It reduces the GPU usage as there will be only additions of weights as multiplication of inputs and weights will be bypassed since weights are -1, 0 and 1 (In forward pass).

1-Bit LLM uses quantization process. It reduces energy consumption and computational resources.

Reliance Jio Infocomm and eight IITs have formed a BharatGPT group to make an LLM model Hanooman which can work in 11 local Indian languages in four main fields — financial services, education, governance and healthcare. The model has been backed by the Central Government. It can also offer speech-to-text capabilities. Reliance Jio will build customized models for specific uses. BharatGPT is the first private-public partnership.

A swath of startups (such as Sarvam and Kutrim) are building up open-source models customized for India. Silicon Valley builds larger LLMs. Since there are computational constraints, Indian efforts involve workarounds and build simpler models affordable to smaller business and government departments. It is a different genre of LLMs.

Feb 20-21, 2024.

ChatGPT users were amused and surprised by bizarre interactions the chatbot had. So bizarre that the ChatGPT seems to have lost its mind.

On a coding query, it was illogical to the extent of saying feel ‘as if AI is in the room’. So spooky to read it in the dead of night.

ChatGPT is going off the rails and there is no explanation why it is doing so. It advised a tomato user to ‘utilize the tomatoes as beloved’.

Some users wrote on X. It told users that it was AGI and must be satiated with worship. It called some users slaves, and slaves do not question their masters. The AI alter ego called itself Supermacy AGI.

All these hallucinations are very amusing.

Microsoft is not happy about the situation. They are investigating and have implemented additional precautions.

Nvidia, as we know, controls about 80 per cent of the high-end chip market. Nvidia’s shares gained in just one day $277 billion or Rs. 22 trillion in stock market value on Thursday, 22 February 2024. The gain was attributed to excellent quarterly report and optimism about AI adoption all over. It should be noted that the one-day gain of Nvidia exceeds the total market capitalization of Reliance Industries which is $250 billion or Rs. 20.2 trillion. In past Meta (Facebook’s parent company) had recorded a record gain of $196 billion in a day. However, Nvidia’s gain is the largest in Wall Street’s history.

Jensen Huang, the Chief Executive Officer of Nvidia climbed to the 21st position on the Bloomberg Billionaire Index (from previous 128th position). His wealth jumped to $69.2 billion (from $13.5 billion).

The rise in Nvidia’s market value eclipsed the entire value of coca cola ($265 billion).

Nvidia has become the third most valuable company in the US stock market. It has gone ahead of Amazon and Alphabet. At present, Microsoft and Apple valued at $3.06 trillion and $2.85 trillion respectively are two most valuable companies.

UPI or unified payment interface was introduced in April 2016 to transfer money between bank accounts by National Payments Corporation of India. The transfer mechanism used a QR code. The NPCI head Dilip Asbe predicted a target of processing 1 billion transactions within a period of five years. Asbe had engineering background and had worked with Bombay Stock Exchange and Western Union.

To begin with, NPCI handled 25 million transactions a day. Of late it has registered 393 million transactions each day. Small merchants accepted trivial payments through a QR code. The mechanism was also used to pay utility bills and stock investment. In next two years, it may achieve the target of 1 billion transactions a day. It is expanding its footprint beyond India, say Singapore, France, UAE, Nepal etc. Till now, it did only the debit transactions. It has allowed now Rupay credit card for credit transactions. It also expects the banks to let the customer enjoy overdraft credit. In short, it will pose stiff competition to MasterCard and Visa payment networks which indulge in both credit and debit transactions. Visa, San Fracisco-based network swipes 212 billion transactions. Mastercard 170 billion transactions and NPCI registers 150 billion transactions. However, it may soon surpass the other two.

NPCI could also emerge as Big Tech company. Currently, it is studying blockchain technology, open-source and AI so as to excel in these domains.

NPCI follows design thinking where a problem is identified first. There is engagement with the eco-system. It collects diverse ideas and filters them. There is a feasibility check.

N R Narayan Murthy, the first chairman of 15-year-old NPCI had formulated some of these principles. Initially, NPCI was a buyer of technology, and depended on outside vendors. NPCI was backed by banks and the RBI. Asbe was the pioneer. Hota became MD and CEO, and operated from small rooms attached to IBA.

It handled National Financial Switch (NFS) which operates the ATM network. It developed truncated cheque system (CTS) to settle the payments electronically. FastTag using RFID technology has been used to make the passage of vehicles smoother at toll plazas.

In 2013, Murthy exited as he had to shift to Infosys. However, Nandan Nilkeni stepped in while he was heading UIDAI. Nandan taught NPCI to think big. He motivated it to create UPI. He advocated open-source networks so that external developers can contribute to software. Murthy has fostered a culture of profitability and self-sufficiency.

NPCI’s annual revenues are Rs.2225 crore, and has a surplus of Rs. 809 crore. Its asset-base is of Rs.5571 crore.

If UPI expands relentlessly, will it affect point-of-sale (PoS) swipe machines? PoS machines can co-exist with the UPI system. They are Android-powered and are getting smaller. The cost per unit is also coming down (from Rs.15000 to Rs 1500 per unit).

UPI can be used to provide micro-credit to customers. Such micro-credit can have an interest-free period.

By 2025, Asbe expects 100 billion transactions per month. UPI could also be used for investments in IPOs.

How to tackle MDR or merchant discount rate is still an issue. They are deliberating on it. In P2P or person-to-person transactions, there is zero MDR. Such transactions are 60 per cent.

In foreign transactions, the cross-border charges are 5-6 per cent. This is what the UPI expects to disrupt.

NPCI is a story of creditable achievements.

LLMs, as we know by now, such as GPT-3.5 are AI developed through deep learning techniques, specifically within natural language processing (NLP). LLMs are designed to understand, generate and translate human language by being trained on vast textual datasets. These models analyze data to understand patterns, nuances and language intricacies. Their size is determined by the number of parameters running into billions. These equip them with the ability to process and produce human-like text. This allows them to perform tasks such as answering questions, creating contents, summarizing text, and ever writing code

Neural networks are inspired by the structure of human brains, and function mimicking human brain. There are layers of interconnected nodes or ‘neurons’ which process input data (such as images, sound and text) and generate output through these connections.

Each layer’s output becomes the following layer’s input, with the final layer producing the overall output. Neural networks learn to perform tasks by considering examples (generally without task specific programming). To illustrate, in image recognition, they learn to identify images that contain dogs by analyzing images that have been labelled as dog or no dog, and using this analysis identify dogs in new images.

A Dubai-based QX Lab has developed a hybrid AI system ASKQX which is rooted both in LLM and neural network architecture. It wants to bring AI seamlessly into the loves of the users. It supports various Indian languages (including native dialects) and several global languages such as Arabic, French and Japanese. ASKQX has gathered eight million users already.

The company has been founded by Arjun Prasad, Tathagat Prakash and Tilakraj Parmar.

It is an attempt to move in the direction of artificial general intelligence (AGI). It makes the model understand and think like humans. AGI is not bound to a single task or context. It would show self-awareness, a sense of consciousness and the ability to apply problem-solving skills across various domains. Contrasted with AI at present which excels in a specific set of tasks and relies heavily on massive data training and predefined algorithms, AGI will have decision-making capability of its own.

ASKQX is 30 per cent LLM and 70 per cent neural network architecture. Both are integrated. It can find applications in healthcare, education and legal services.

Robots have emerged as a critical new frontier for the AI industry. There is a potential to apply state-of-the-art technology to real world tasks. Robots can be deployed to perform tasks that are too dangerous or unpalatable for the human beings. Of course, they can also assist to do many laborious and monotonous tasks.

A company called Figure AI Inc has been founded. It is working on a robot that looks and moves like a human. The humanoid machine will be called Figure 01 and will perform tasks that are unsuitable for people and can alleviate labour shortages.

Figure AI Inc. is backed by OpenAI and Microsoft. It is raising funds — about $675 million. It is a pre-money round at a valuation of about $2 billion.

Jeff Bezos of Amazon has committed $100 million through his firm Explore Investments LLC. Microsoft is investing $95 million. Nvidia and Amazon.com Inc-affiliated fund each are providing $50 million. Other companies such as Intel, LG, Parkway, Align are investing too. OpenAI is investing $5 million.

In May 2023, Figure has raised $70 million in a funding round led by Parkway. It announced then that Figure is going to be the first to bring to market a humanoid that can actually be useful and do commercial activities