In deep learning, the number of hidden layers can be as large as 1000 layers.

Deep nets process data by employing sophisticated math modelling, Here a model takes a set of inputs and gives an output.

Using a deep net is very simple–as simple as copying and pasting a line of code for each layer.

Weight optimization is a process of fine tuning the algorithms used to assign values to individual parameters that influence a neural networks output.

An optimizer is an algorithm that adopts the neural network’s attributes (learning rate and weights). It assists in improving accuracy and reduces the total loss.

Weights are modified in training deep learning optimizers model. This reduces the loss function. The weights are updated using the backpropagation of error algorithm.

Compare an expected output with a predicted output. You will set the error. Propagate these errors backward to update weights and biases.

Backpropagation enables us to calculate the gradient of the loss function with respect to each of the weights of the network. Every weight is thus updated individually to reduce slowly the loss function over many training iterations.

Gradient descent is optimization algorithm. Its aim is to adjust parameter function to minimum.

In linear regression, gradient descent finds weights and biases. In deep learning, backward propagation uses this method.

The three types of gradient descent are — batch, stochastic and mini-batch gradient descent. Batch gradient descent is called a training epoch.

Input layer receives inputs from external sources. It passes these to the hidden layer. Each neuron in the hidden layer gets information from the previous layer. It computes the weighted total. This is transferred to the neurons in the next layer.

Weighted means the impacts of the inputs from the preceding layer are more or less optimised by giving each input a distinct weight. These weights are adjusted during the training process to enhance the performance of the model.

Neurons are also called units. Whether a layer has a dozen or million units depend on the complexity of the data — the patterns in the dataset.

Each neuron receives input from the previous layer. It then gives output which becomes the input of the neuron in the next layer. The process continues till the final layer. The final layer produces the output of the network.

Units (neurons) are linked to one another. Each of these links has weights that control how much one unit influences the other. The neural network continues learning as the data moves from one unit to another.

It is free, open source software library for ML and AI. It can be used for a variety of tasks but has a focus on training and inference of deep neural networks.

TensorFlow enables you to deploy the models on any environment, say servers, browsers, edge devices, cell phones, CPUs, GPUs, FPGAs and microcontrollers.

It is basically a software library for numerical computation making use of data flow graphs. In the graphs, nodes represent mathematical operations. Edges in the graph represent multi-dimensional data arrays, called tensors which communicate between them.

TensorFlow Applications

It is used for voice recognition, image recognition, sentiment analysis, deep neural network, autonomous cars, optical character recognition, video processing and many more.

It is a powerful tool for wide-ranging ML tasks — NLP, time series analysis, generative models, RL, and anomaly detection.

Learn TensorFlow

One has to setup TensorFlow and load a dataset. Then build an ML model. Train and evaluate this model.

There are several pre-trained models at TensorFlow Hub from Google and research models in the Model Garden.

Before you set up TensorFlow, you must have knowledge of programming concepts, linear algebra and calculus.

You can use 3 Blue 1 Brown’s Essence of Linear Algebra or EdX’s Introduction to Linear Algebra.

DeepMind is an AI company acquired by Google in 2014. DeepMind will be merged with Google Brain team to form Google DeepMind. Hassabis, DeepMind’s CEO will head this new collaboration.

DeepMind is known for developing AI in such a way that it has outperformed humans in games like Chess and Go. It has achieved several breakthroughs in this field. DeepMind’s research team is Google Brain. It is dedicated to deep learning. It was set up in 2011. It has combined open-ended. ML research with information systems and computing resources.

The team developed tools such as TensorFlow. It allows the public use of AI research projects.

Google Brain throws opportunities in the field of ML and NLP.

Google DeepMind leaves an indelible mark on the AI landscape.

Deep learning and machine learning should be distinguished. It is necessary to understand that deep learning is machine learning. In fact, it is an evolution of machine learning. Deep learning uses a neural network which is programmable. It enables machines to make accurate decision or make accurate predictions without help from humans.

Machine learning, by definition, is an application of artificial intelligence (AI). It includes algorithms which parse data. Then they learn from data. They then apply what they have learnt to make informed decisions. Machine learning involves a lot of complex math and coding. By machine learning, a function is performed with the given data, and this gets better over a period of time. Machine learning facilitates all types of automated tasks.

Deep learning model analyses data continually with a logical structure. It is a layered structure of algorithms called artificial neural network inspired by the biological networks of neurons in the human brain. It is tricky to make the model draw correct conclusions. To make it flawless, there is lot of training involved.

However, when it works as desired it is magical. An image recognition application can identify flower, fruits, animals and human beings. It can recognise speech and can translate and can drive autonomous cars.

Though deep learning is a subset of machine learning, both function similarly, and hence the terms are used interchangeably. However, they have different capabilities.

Basic machine learning models progressively improve while performing specific functions in the light of new data. However, they still require human intervention. In deep learning, an algorithm decides whether the prediction is accurate or not. No human help is sought.

DeepMind’s CEO Demis Hassabis has ventured to say that human level artificial intelligence will be developed in a few years.

There are references to artificial general intelligence (AGI). Here the machines or algorithms match human intelligence. As such AI so far was used for specific tasks. However AGI would be able to learn and adapt to new situations just like human beings.

It will heavily impact finance, transportation and healthcare sector.

There is considerable progress towards developing AGI in recent years. The pace will accelerate, and maybe we can achieve AGI within a decade.

At the same time, there are concerns about the rapid development of AI. Hinton has hinted at the dangers of AI, and has called for a pause on its development. Hassabis is for responsible AI. Jeff Dean from Google too shares his view.

In early years, AI performed functions such as classsification of data (whether the animal is a tiger or lion), grouping the data (say customers with similar income) or choosing actions ( whether a self-driving car should stop or speed up). This was a cumbersome job. You required large manpower of data scientists. You had to collect a lot of data. The data had to be labelled. There was training involved. Using AI then, a chatbot assisting home loan customer could be created in 6-8 months. The process was expensive. A chatbot assisting credit card users could be created if we start again from scratch. Thus AI’s adoption was slow.

Foundation Model

Traditional AI systems were trained for a specific purpose. The way out is to develop Foundation Model –an AI system with a broad set of capabilities, and which can be adapted to a variety of purposes.

Of course, Foundation Model are developed by big organisations, with resources and expertise.

Foundation Model uses an ML process in which the model trains itself to learn one part of the input from another part of the input. This is called self-supervised learning.

Thus if the Model is fed adequately, it will predict the word after ‘I love’ in all likelihood will be ‘You’. It is not necessary to label and train the data.

Once we have a collection of Foundation Models, these can be exploited to create derivative AI models and applications. This can be done economically. A single chatbot can answer queries related to housing loans, credit cards and many other banking services. To make it more effective, one can add a few more documents.

Large Language Model

When a foundation model is based on language model, it is called large language model (LLM). In fact, LLMs are a subset of Foundation Models. These models, in fact, automate language. They can generate dialogues, summarisation, text, translation and so many other things.

OpenAI’s LLM crawled over a lot of publicly available information on internet to create ChatGPT.

Of course, the publicly available information is not always accurate. ChatGPT, therefore, can give wrong answers. OpenAI’s GPT is fascinating and magical.

Enterprises need accurate information. They need trustworthy outputs. They need datasets on which they have rights. The data must be curated carefully — it should be free from hate and profanity. There should be simultaneous use of internal data and external data. Many Foundation Models can be created this way. IBM is working on this.

Generative AI is a new class of AI systems that can generate content in terms of text, images and video. Researchers are now working on models which have Artificial General Intelligence (AGI) which make them as smart as human beings and at time smarter,

Crowdsouraing of Data

India is a country of multiple languages. Under a project called Bhasha Daan, data is being crowd sourced. People can contribute to datasets in their own language. These are then subjected to LLMs to build tool sets for translation. These models are then published on API Setu Platform.

Government Contributing Data

Apart from crowdsourcing, the government can provide data from its various departments.

Translations for Data

English texts could be translated. Wikipedia could be translated into Hindi. However, translation itself requires LLM. It is a kind of chicken-and-egg situation.

Synthetic Data

Computers can be used to create data.

India has to build LLMs in the native languages.

India is very particular about the balanced approach in AI regulation. The government is aware of the risks of algorithmic biases and copyright issues, and a regulatory framework could emerge in future, but this process cannot be hurried up. It is so pragmatic to think that AI regulation cannot be an isolated one country issue. The digital world is borderless, and AI could be controlled with global coordination. India’s regulatory framework could emanate from international deliberations.

The European Union is contemplating a draft regulation for AI — the EU AI Act. The US is opting for a disaggregated oversight spread over various agencies. The governments cannot ignore common threats and will have to target these. There should be a risk-based approach, and an agreement over the main principles of trustworthy AI. There should be endorsement for international standards. There should be greater cohesion in AI strategies regarding applications of AI.

Unfettered AI could harm the employment market, even though it has the potential to create many new jobs. The governments will have to deal with this disruption. Bard is capable of coding in more than 20 programming languages. It is really scary to think of the impact it can have on IT services.

Thus AI regulatory framework cannot be avoided. China has announced draft regulations prescribing the registration of AI products with its cyberspace agency. There should be pre-release security assessments.

AI regulations do require private guardrails since AI is being increasingly embedded in so many products and services, processes and decision-making. There should be oversight on how data is being used by software which is evolving, and is becoming complex. The private sector has to come forward to suggest strong protective measures.

Big Tech companies do take cognisance of the technologies newly emerging, and support them. Facebook, renamed as Meta evinced interest in a 3D virtual world accessible through hardware. It is called metaverse. The journey started in October, 2021. Facebook had realised that it had attracted huge amount of digital advertising. However, Apple introduced user option for tracking as a privacy feature. That affects the advertising revenues of Facebook in a big way. Facebook was not comfortable entering the hardware field. Facebook re-oriented itself to what was declared the future of internet — the metaverse.

Later, Facebook got interested in blockchain technology and cryptos. It thought of introducing a crypto currency Libra. But that projected was shutdown. Facebook also was interested in a video calling device-Portal, but closed the project in 2022.

Instagram NFT support was announced in May 2022. It enables creators to showcase NFTs on Instagram. However, it has disabled this feature in March 2023.

Facebook has not closed down Metaverse project as yet, but has started shifting its focus to AI. Its foray into metaverse happened at a time when it was tainted by the Cambridge Analytica case of the data breach. The company put $10 billion in 2022 in Reality Labs to focus on hardware. However, the Labs has a reported a loss of $13.7 billion in 2022. Metaverse can gain traction in future but not in the short term. There is decline in users and share prices. Facebook has laid off employees.

Facebook is focusing on AI and is building algorithms that furthers its original tenet — bringing the world closer.

There is a history of battles between open source and proprietary (or closed) systems in software technology. There is a tendency to hoard the most important tools that could shape tomorrow’s computing platforms. Some other companies opt for giving those tools away. The most popular example is that of Google’s open source Android mobile operating system. It challenged the Apple’s dominating position in the field of smart phones.

Meta is the company from Silicon valley that owns Facebook, Instagram and WhatApp. It has created AI technology LLaMA that can power online chatbots. LLaMA stands for ‘large language model Meta AI’. LLMs pinpoint patterns in text and generate text of their own. LLMs are systems that learn skills by analysing vest amounts of text (including books), Wikipedia articles and chat logs. ChatGPT and Bard are built on top of LLMs. Instead of keeping the technology to itself, Meta has released the computer code underlying the system. Those whose email addresses are with Meta could download the source code after vetting by Meta.

In essence, it is making its AI open source software. It makes the software freely copiable, modifiable and amenable to reuse. It will give capability to public at large to build chatbots of their own very quickly.

One of the godfathers of AI, Yann LeCun (the other two were Geoffrey Hinton and Youshua Bengio) is the chief scientist of Meta. LeCun has declared that Facebook’s AI will be an open source platform. Thus Meta has taken a different approach from its rivals. Meta supremo Zuckerberg considers this to be a smart thing to do. This approach could be contrasted with Google and OpenAI which are in the AI race. They are being increasingly secretive about the methods and software that underlie their AI products. In fact, they are critical of the unfettered open source approach of Meta. It could adversely affect the job market. It can lead to the misuse of AI technology. Meta feels that keeping the code to itself is a ‘huge mistake’. LeCun feels that the technology will be widely welcome when it is outside the control of companies like Google and Meta.

Meta has invested in AI technology for nearly a decade. It has spent billions of dollars building the software and hardware needed to realise chatbots and other generative AI. It has worked behind the scenes to infuse its other products with AI. Zuckerberg wants Meta to be a leader in AI space.