Microsoft has announced the next iteration of its lightweight AI model Phi-3 Mini, the first of three small models the company has plans to launch in the coming days. Phi-3 Mini has 3.8 billion parameters and is fed on a data set that is smaller than LLM (large language model) such as GPT-4.
Microsoft’s Phi-3 mini can now be accessed on Azure, Ollama, and Hugging Face. Microsoft plans to release Phi-3 Small (7B parameters) and Phi-3 Medium (14B parameters). Parameters are the number of complex instructions a model can understand.
Microsoft released Phi-2 in December last year, which performed as well as bigger models, including Llama 2. Microsoft claims Phi-3 performs better than the last iteration and is capable of responding close to how a model is 10 times bigger than it is capable of doing. Eric Boyd, corporate vice president of Microsoft Azure AI Platform told The Verge that Phi-3 Mini is as capable as LLMs such as GPT-3.5 but “smaller form factor.”
Also Read: Rockstar Games Parent Company Lays Off 5% Workforce, Cancels Games
What Are The Advantages Of Having Small AI Models
When compared to the bigger AI models, the smaller ones are mostly cheaper to run and perform better on personal devices such as laptops and phones. The Information reported earlier this year that Microsoft was creating a team that focuses specifically on lighter-weight AI models. Along with Phi, Microsoft has built Ocra-Math, a model that will focus on solving math problems.
Microsoft’s competitors also have their own small AI models and most of them target simpler tasks such as coding assistance and document summarization. Google has Gemma 2B and 7B work great for basic chatbots and language-related work. Anthropic’s Claude 3 Haiku is capable of reading dense research papers with graphs and quickly summarizing them.
Eric Boyd said that developers trained Phi-3 with a “curriculum.” They took inspiration from how children learn from bedtime stories, and books with simpler language, and sentence structures. “There aren’t enough children’s books out there, so we took a list of more than 3,000 words and asked an LLM to make ‘children’s books’ to teach Phi,” Boyd says.
Also Read: New Mafia Game Announcement May Be Coming Soon
Microsoft’s Phi-3 Is Better At Coding And Reasoning
Boyd added that the new AI model was built by learning from previous iterations. While Phi-1 focused on coding and Phi-2 started to learn to reason, Phi-3 can code and reason better.
While the Phi-3 family of models has general knowledge, it might not be capable of beating GPT-4 or other LLM. Users will notice a big difference in the kind of answers they will get from an LLM trained on the entirety of the internet against a smaller model like Phi-3.