First of all, the total costs of building an LLM consists of manhours spent by highly talented manpower, who use expensive chips to train them and the various operational costs. We are leaving aside the fixed overheads here.

LLMs and even smaller models are expensive to train and deploy. There is hardware cost. Training prerequisites are the GPUs or graphics card. Nividia’s A 100 which is commonly used costs $10000. The computation requires tens of thousands of these GPUs. A GPT-3 model has 175 billion parameters. It takes around 285 plus years of computation. To make it manageable, OpenAI used thousands of GPUs to reach its computation goals. According to one estimate, OpenAI may have used more than 30,000 GPUs to commercialise ChatGPT. It wold have cost $30 million. While integrating it to Bing, Microsoft could have spent over $4 billion in the hardware cost. Bard-powered Google could have cost Alphabet $100 billion.

Running costs of such a model is also very high. An exchange with an LLM costs several times more than the search on search engine. A ChatGPT-like model receives millions or billions of daily queries. Basic running costs are too high even for organisations with deep pockets. According to one estimate, OIpenAI spends about $7 lac per day to run ChatGPT.

Talented manpower’s salary (with compensations over a million dollars per annum) is another cost component. Skilled talents come at premium cost.

There is environmental cost on account of carbon emissions.

There is constant research on LLMs. There are data collection costs. There is electricity cost. And a host of administrative costs.

All these costs take the best model out of the reach of public. All factors are antithetical to mass adoption. Even Big Tech will have to ration the services.

Not-for-profit models care not sustainable.

Organisations must give serious thought to how these models can be monetised. OpenAI has converted itself into for-profit.

Research will have to focus on reducing the training cost and hardware cost. Already, organisations such as Google produce their own chips, e.g. TUP or Tensor Processing Unit, Amazon’s Inferentia and Trainium, Facebook’s MTIA etc. There should be research on computer memory.