In October, 2023, Apple in collaboration with researchers of Columbia University released an open source multi-modal LLM called Ferret. It was released on 30th October 2023 without any commercial license.

Apple is secretive as well as protects its proprietary systems. However, it has released an open-source model. The model can run on small devices such as iPhones and iPads. It is released under non-commercial licence.

The model is powered by 8 Nvidia A 100 chips. It is trained on GRIT dataset. Whether the model would be able to compete with larger models such as GPT-4 is to be seen.

However, this introduction is paradigm shift in Apple’s AI strategy. Open-source invites collaboration and innovation. It departs from its traditional closed-door approach.

Apple is in its early stages of generative AI journey with Ferret. Mobile handsets have a limited capacity to handle models, say they can handle models with 10 billion parameters. Apple researchers have made a breakthrough — a smartphone can be supplemented with RAM with onboard flash storage. It is a small cache for LLM data.

Ferret identifies elements within an image. Beyond that, it answers user’s queries. There are possibilities of image search. It is spatial-aware. The model is capable to analyze and interpret images and text together. It is much like a smart assistant. It can look at pictures and read descriptions.

Open sourcing is a strategic move. Apple can tap global AI talent. It can accelerate Ferret’s capabilities ahead of its rivals such as Google and Microsoft. Of course, the challenge is to remain protective about its IP.