Facebook made a bold claim on 28th June, 2023 that the recommendation models it is working on could surpass the biggest LLM models of today. Facebook is into research of multi-modal AI, say visual and auditory to better comprehend a piece of content. Some such models are in public domain, and some are used internally to improve relevance or targeting of messages. These advanced models understand people’s preferences, and have tens of trillions of parameters. In other words, orders of magnitude larger than the biggest language models of today. Is it talking about a theoretical possibility of the potential of a model? The company is clear that these very large models available at present can be trained and deployed efficiently at scale. Is the company ready to create infrastructure for such model? Perhaps, what they are aiming at is aspirational.

Preference understanding and modelling is a sort of behavioral analysis. Are they aiming at training the models on practically every written work available?

The 100 trillion parameter claim, though somewhat exaggerated, still shows that Facebook is aiming at something scarily big.

Facebook is conceiving a model larger than anything yet created. Facebook would like to dazzle advertisers with science. There would be large-scale attention models. There could be graph neural networks. There could be few shot learning and other techniques. An architecture that is hierarchical deep neural retrieval network.

Researchers may not be impressed. They are familiar with such ideas. The users either do not understand or care. However, an advertiser does feel it is better to put money on media where it is well-spent. Facebook is trying to convince them that it excels in understanding consumer behaviour. The primary aim of social media and tech platforms is to sell ads with better granular and precision targeting. Despite the users revolting against all this, the platforms try to impress upon advertisers the value and legitimacy of targeting. Advertising becomes prolific but the issue is whether it improves.

These platforms do not do market research to help their users. Have they ever done research to tell us which 10 advertising books are the best for media students? Instead they look over our shoulders when we are surfing the net, and buying some toffees to bombard us with ads of toffees the next day.

Do we really need a model with 10 trillion parameters just to tell what people like? And spend a hell of amount on building it.