Blog

John Schulman, one of the founders of OpenAI, quits to join rival Anthropic. Schulman had joined OpenAI in December 2015, just before his doctorate in electrical engineering and computer science at UC Berkeley. He announced his exit on Tuesday, August 6, 2024. He is making the move ‘to deepen his focus on AI realignment and to resume technical work, thus opening a new chapter of his career’.

Greg Brockman, OpenAI’s cofounder, has announced his sabbatical till the end of the year. He intends to relax and would like to work for achieving AGI goal.

Schulman at OpenAI has led the team of reinforcement learning that developed ChatGPT powered by GPT-3, a language model.

After the departure of Schulman, only three of OpenAI’s eleven original founders remain in the company — Sam Altman, Brockman and Zaremba.

Ilaya Sutskever, another founder, left the company in May 2024. Andrej Karpathy, another founder left in February 2024.

The most notable early exit was that of Elon Musk on the ground that OpenAI is deviating from its public good priority to commercial priorities.

Peter Deng from Product Management area who joined in 2023 also exited this year.

As we know, there is a demand in the market for anti-obesity drugs Wegovy (Navo Nordisk) and Zepbound (Eli Lilly). There are several other versions of such drugs in the market, not necessarily the generics of these two. There are promotional messages on a continuous basis for Wegovy (semiglutide) and Zepbound (tirazepatide). There are anti-diabetic counterparts of these — Ozempic and Mounjaro. There is wide-spread media coverage and outdoor advertising.

Several websites sell these drugs — some 1100 websites mentioned semaglutide. Many illegal online pharmacies sell these drugs. Here these could be obtained without a prescription. These could be counterfeit drugs — contaminated and with varying proportions of semiglutide. The big issue is the uncertainty of the quality of these drugs. Pharmacies can take advantage of a regulatory grey zone which allows them to sell alternative version of a branded drug during a drug shortage. There is doubtful sourcing of the active ingredient in the products of these pharmacies. And in case of infections, whether these pharmacies maintain the sterile manufacturing process.

Some offer fancy dosage forms such as gummies, lozenges and oral drops. The additives, they claim, purport to improve weight loss or minimize side effects. However, they could affect a drug’s efficacy.

Demand outstrips the supply. There may not be insurance coverage. That pushes the customers to buy the cheaper alternatives. There is confusion between genuine pharmacy products, compounded products and counterfeit stuff.

The WHO warned about vials of anti-obesity medicines that contain undeclared ingredients, including insulin. There are dosing errors on account of compounding of medicines. Many end up in hospitals. Novo Nordisk sells single use pens, whereas vials of semiglutide sold by some pharmacists are prone to overdosing on the part of patients. There is a rise in the cases of overdosing and poisoning.

There should be sufficient supply of these medications by approved pharmacies such as Novo Nordisk and Eli Lilly. That will abate the problem. The end of shortage will end the free-for-all compounding. Consumers should know that fake drugs are no bargain.

A US court in Washington ruled in August 2024 that Google illegally monopolised the search market. This gives the first win to the government in its anti-trust case against a tech giant in more than two decades.

The court referred to the $26 billion in payment to make its search engine the default option on the smartphones and web browsers. This move by Google effectively blocks any other competitor from succeeding in the market. Such monopolizing by Google has been able to raise consistently the prices of online advertising. The distribution agreements foreclose a substantial portion of the general search services market and impair the opportunities for the competitors. The default position has allowed Google to build up the most-used search engine in the world and Google could attract more than $300 billion in annual revenue largely generated by search ads.

The Justice Department will continue to enforce vigorously the anti-trust laws. Google plans to appeal the decision.

The Judge passed a harsh comment, “Google is a monopolist and it has acted as one to maintain its monopoly.”

Such a significant ruling was passed against Microsoft in the 1990s when the company bundled its browser as the default browser in Windows-based devices. It crowded out Netscape, the pioneer in browsing technology. The US authorities took notice, and a landmark decision against Microsoft was passed. Microsoft avoided the specter of breakup (since prior to this, AT&T and IBM were badly hit by breakup).

The US authorities could ask Google to share data. It can affect Google’s finances adversely. If competitors get access to similar data, advertisers will diversify ad spends across multiple platforms. It will result into a potential decline in Google’s advertising market share. Besides, there could be costs associated with sharing data. Even while sharing data, privacy cannot be compromised, and hence Google will have to work with stakeholders, regulators and privacy advocates.

There could be another remedy — divestiture or selling parts of the firm. It is a structural remedy. There could be a ban on the practice of paying billions to make Google a default search engine. A choice could be given to users. However, this is like shutting the stable after the horse has fled. Even the users can exercise choice in favour Google.

There are certain misgivings about the LLMs. Some feel they have been overhyped. Some consider them as an impediment in achieving AGI. In fact, LLM have limitations. There is lack of trust of the users. LLMs are not that accurate and reliable. The discussion is cliche, since the current LLMs do have reasoning and logical problems.

Despite the skepticism, big tech has not stopped building the best LLMs. Each company tries to demonstrate one-upmanship.

Yan LeCun believes LLMs will not lead to AGI. In fact, OpenAI has slowed the achievement of AGI by 5-10 years. It is necessary to try some novel approaches to AGI through abstraction and reasoning corpus (ARC). LeCun advises to attain animal-level intelligence first.

LLMs at times struggle with the apparently simple tasks. Youshua Bengio, one of the godfathers of AI, says in AI, some ingredients of human intelligence are missing.

LLMs fail to play common games such as tic-tac-toe. GPT- 4 struggled with Sudoku puzzles. DeepMind contends LLMs lack genuine understanding. Consequently, they cannot auto correct or adjust their responses. LLM-based chatbots are poor in math problems.

Still, the time is not ripe to write them off. Though at present, we have not reached human-level intelligence, it does not rule out the possibility of reaching it in future.

GPT-4 understands complex emotions. It beats human psychologists. GPT-5, according to Mira Murati, OpenAI CTO, will have Ph.D.-level intelligence.

According to Sutskever, text is the projection of the world. LLMs build cognitive architecture from scratch. They trace the evolution of learning and resonate with real-time learning.

There is research on LLMs too. In future, they can understand cause-and-effect relationships. LLMs can use neurosymbolic AI.

LLMs are worth another shot on the road to AGI.

AI can facilitate business processes, thereby cutting costs. The irony is that AI per se is very costly.

If AI is scaled up, the costs rise — on account of capital investments in data centers and acquisition of costly GPUs. these costs cannot be passed on to the clients. AI tools are not economical.

Even the cost of deployment of AI into a company’s systems costs a lot — ranging from $5 million to $20 million.

The silver lining is that AI costs are declining apparently, and the gap between investments and returns is narrowing.

The crux of the issue is the benefits AI can offer. Ther is continuing investment for the growth of AI. The companies prefer overinvestment, rather than err by underinvestment.

Over a period of time, the costs of training AI models have risen. These too are coming down. Gemini from Google is a powerful model but is available at a lesser cost. Similarly, OpenAI’s GPT-$o is faster, but is available at half the cost of the previous model GPT-4 Turbo.

The cost of accessing the models has dropped substantially — this is measured by processing of tokens.

Researchers focus on techniques such as ‘sparsity’ and ‘quantization’ to do the cost cutting.

Even enterprises now focus on choosing the right model for their needs, not necessarily the costliest model. They can opt for open-source models or smaller models.

Silicon Valley in past has accepted a margin hit in order to grow the market share. The aim is to be competitive, and skim the market later.

However, the issue is how the benefits of generative AI outweigh its costs for the businesses. Some analysts suggest a 30 per cent project drop by the end of 2025. If generative AI remains confined to summerization and deployment of chatbots, it may not be worth even the lower price tags. AI compnies will have to ponder over this issue.

In the Silicon Valley, Palo Alto is the heart of the valley. It is home to many high tech companies and startups. It is a rich city in the San Francisco Bay area. Companies such as Tesla, HP and VHWare are based here. It is the home to Stanford University, which is one of the leading research institutes. Even schools located here are esteemed institutions. Many move to Palo Alto to avail of the educational opportunities.

It is one of the wealthiest cities in the USA. The real estate market is highly competitive. Property values are very high. The cost of living too is high. It shows how desirable the area is.

You can have a good quality of life here. There are recreational facilities and parks. The downtown area houses boutique shops, cafes and restaurants. The demography is cosmopolitan and diverse.

The city is known for its environmental initiatives.

There could be a total capital investment of $1 trillion in AI infrastructure over the next several years. Already billions of dollars have sunk into Big Tech companies trying to create AI infrastructure. Still, what is awaited are the big sales to justify these investments.

Wall Street has been disappointed on this count. Google’s shares have fallen by approximately 6 per cent and Microsoft shares too have declined since its own results.

2024 is considered to be a year for generative AI. Generative AI’s mass adoption should result into meaningful profits of Big Tech. However, the returns are not yet seen. It raises concerns about the worth of AI.

No doubt, the technology offers an enormous opportunity. And as time advances, the opportunity grows bigger. That results into greater investments. However, will these investments lead to an increase in profits?

It requires just-one winning product that could justify the massive investments. Google, Microsoft and Amazon have shown healthy pace of growth in cloud computing. Cloud computing grows on account of AI since it requires huge computational resources. However, these gains are insufficient to satisfy the investors. The returns fell shy of analysts’ expectations.

Amazon spent in capital investment $30.5 billion, mostly in AWS cloud in the first half of the year. It wants to build capacity so as to meet future demand without affecting profits. Google’s cloud investment growth could be seen, but how much of it could be attributed to investment in AI is not sure. Capital expenditures overshadow better-than-expected sales. Microsoft too disappoints. It declares that Azure’s growth to the extent of 8 per cent could be attributed to AI. (In previous quarter, it was 7 per cent). Can this growth justify heavy capital expenditure?

Facebook is the hero in this scenario. It has raised its capital expenditure, citing AI investments. However, it shows improvements in ad targeting and content recommendations. Face book’s high spending on AI is a short-term sacrifice for long-term gain. Apple too is powering its handsets with AI to overcome slow sales in China.

Our pencil leads are graphite. Graphite is prized for its inertness. These days it has attained the position of a critical mineral on account of its role in energy sector. As a form of carbon, graphite is the largest element by weight in an EV battery. It constitutes most of anode — the electrode that receives and holds lithium ions during charging and releases them when energy is needed.

China has a good hold over graphite supply chain. It not only produces graphite but refines it too.

The US has not mined graphite for many years. Of late, there is a a deal between Graphite One, a startup and Lucid, a maker of high-end EVs. Graphite One has conducted a feasibility study on major graphite deposits in Alaska.

Unscript is a startup based in San Francisco and Bangalore. It has developed capability of converting a single photo into a full-fledged video — complete with facial expressions, eye and head movements, voice modulations and body language. The results are astonishing — the video is of studio quality.

This happens in a matter of minutes, say a couple of minutes. It curtails the shooting efforts which otherwise would have gone into it.

The new upgrade amazingly competes Microsoft’s VASA-1 and Alibaba’s EMO. It outshines Google’s Vlogger. It is suitable for brands, advertising and marketing organizations and influencers. They have roped in 50 corporates as clients already. OpenAI’s SORA generates abstract videos. Unscript’s software generates human videos.

Unscript was founded in 2021. Its chief is Rituika Chowdhry. Apuru Jain is a co-founder. Its clients include Bombay Shaving Company, RadioCity, HUL, HealthifyMe and Amazin Graze.

The video content integrates easily with client’s systems.

AI hardware has been highlighted in Chris Miller’s book Chip War which discusses advanced chip manufacturing. Semiconductor industry is the most precise and complex industry. In early days, there was vertical integration. These days, we have fabless chipmaking and foundries.

These days we also have Nvidia’s GPUs, Google’s TPUs and emerging neural processing units — NPUs. These are all advanced accelerators. It is an evolution from 7nm to 5nm to 3nm fabrication. While giving more compute power, it manages power consumption and heat dissipation.

GPUs are good at parallel processing and are good for training LLMs. TPUs are application-specific integrated circuits (ASICs). They are good for low precision computations. NPUs are a type of ASICs designed for accelerating neural network computations for specific tasks in mobile and edge-computing.

India is entering into semiconductor space. There is continuous advancement in semiconductor technology. The demand for AI hardware is soaring — from accelerator for training to inference chips.THere is a demand for specific chips — power management, telecom, digital signal processing, cryptography and so on. There are advances in ASICs. At the same time, material science and quantum computing do help in these endeavors. There could be combinations of GPUs and NPUs on a single chip. There are AI-optimized field-programming gateways.

There are neuromorphic chips that mimic CNS and ANS. These could be used in robotics and complex sensor networks.

India has a talent pool of STEM graduates. They could be leveraged to maintain our competitive edge in AI space. The government is planning to have conducive infrastructure — a cluster of 25000 GPUs. AI hardware is a promising field.