We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of 200K tokens to better support multilingual applications, as well as group query attention for more efficient long-sequence generation. Phi-4-Multimodal is a multimodal model that integrates text, vision, and speech/audio input modalities into a single model. Its novel modality extension approach leverages LoRA adapters and modality-specific routers to allow multiple inference modes combining various modalities without interference. For example, it now ranks first in the OpenASR leaderboard to date, although the LoRA component of the speech/audio modality has just 460 million parameters. Phi-4-Multimodal supports scenarios involving (vision + language), (vision + speech), and (speech/audio) inputs, outperforming larger vision-language and speech-language models on a wide range of tasks. Additionally, we experiment to further train Phi-4-Mini to enhance its reasoning capabilities. Despite its compact 3.8-billion-parameter size, this experimental version achieves reasoning performance on par with or surpassing significantly larger models, including DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B.
核心贡献 · Key contributions
提出 Phi-4-Mini，一个 38 亿参数的语言模型，在数学和编码任务上超越两倍大小的模型。 Introduces Phi-4-Mini, a 3.8B parameter language model outperforming models twice its size on math and coding.
提出 Phi-4-Multimodal，采用新颖的 LoRA 混合技术实现文本、视觉和语音的统一处理。 Proposes Phi-4-Multimodal with a novel Mixture-of-LoRAs technique for unified text, vision, and speech processing.
在 OpenASR 排行榜上取得最先进语音识别性能，以紧凑的 56 亿参数模型排名第一。 Achieves state-of-the-art ASR performance, ranking first on the OpenASR leaderboard with a compact 5.6B model.
在 38 亿参数模型中展示强大推理能力，匹配或超越 DeepSeek-R1-Distill-7B 等更大模型。 Demonstrates strong reasoning capabilities in a 3.8B model, matching or surpassing larger models like DeepSeek-R1-Distill-7B.
通过冻结基础模型并使用模态特定 LoRA 保持语言性能，避免退化。 Maintains language performance by freezing the base model and using modality-specific LoRAs, avoiding degradation.
在视觉语言、语音和推理基准上提供广泛评估，优于类似规模模型。 Provides extensive evaluations across vision-language, speech, and reasoning benchmarks, outperforming similar-sized models.
局限 · Limitations
语音摘要数据仅占后训练数据的 1%，限制了摘要质量。 Speech summarization data constitutes only 1% of post-training data, limiting summarization quality.
由于通用知识数据较少，SQQA 性能落后于 Gemini-2.0-Flash 和 GPT-4o。 SQQA performance lags behind Gemini-2.0-Flash and GPT-4o due to less general knowledge data.
音频安全数据集仅包含语音，缺少非语音声音和音频特定越狱训练。 Audio safety datasets are voice-only, lacking non-speech sounds and audio-specific jailbreak training.
视觉-语音联合训练使用合成 TTS 数据，可能无法完全捕捉真实世界的变异性。 Vision-speech joint training uses synthetic TTS data, which may not fully capture real-world variability.
推理增强版 Phi-4-Mini 处于预览阶段，未与主模型同时发布。 Reasoning-enhanced Phi-4-Mini is in preview and not released concurrently with the main models.