AI models released in May 2024
21 releases from 15 labs in May 2024, including Text to Sound Effects, Codestral (22B) and Aya 23. Every entry is dated to the lab's official announcement and links to it.
- Releases
- 21
- Labs
- 15
- Open weights
- 12
May 2024
21 releasesText to Sound Effects
ElevenLabs' first sound model, generating sound effects, short instrumental tracks, soundscapes and character voices from text prompts, available to all users.
- Modalities
- text → audio
- At launch
- Consumer app, API
- Generates effects, short instrumentals and soundscapes
- Aimed at film, TV, games and social creators
- Succeeded by SFX v2 and SFX v3
Codestral (22B)
Mistral AI's first code model, a 22B open-weight model trained on more than 80 programming languages with fill-in-the-middle completion through a shared instruct and completion endpoint.
- Modalities
- text → text
- Context
- 32K context
- Parameters
- 22B
- License
- Mistral AI Non-Production License
- At launch
- Open weights, API
- Trained on 80+ programming languages
- Released under the new Mistral AI Non-Production License
- API model ID codestral-2405
Aya 23
Multilingual open-weights research models from Cohere For AI that trade Aya 101's breadth for depth, covering 23 languages in 8B and 35B sizes.
- Modalities
- text → text
- Parameters
- 8B and 35B
- License
- CC-BY-NC-4.0
- At launch
- Open weights
- Builds on the Aya open science initiative
- Released with a technical report
- Weights published under a non-commercial license
Mistral 7B v0.3
Update of Mistral 7B that extends the vocabulary to 32,768 tokens, released as base and instruct checkpoints and served as open-mistral-7b.
- Modalities
- text → text
- Context
- 32K context
- License
- Apache-2.0
- At launch
- Open weights, API
- Vocabulary extended to 32,768 tokens
- 32K-token context window
- Released under Apache 2.0
Phi-3-small and Phi-3-medium
Phi-3-small (7B) and Phi-3-medium (14B) are larger instruction-tuned members of Microsoft's Phi-3 family, first announced with Phi-3-mini and made available on Azure and Hugging Face in May 2024.
- Modalities
- text → text
- Context
- 125K context
- Parameters
- 7B, 14B
- License
- MIT
- At launch
- Open weights, API, Cloud platforms
- Phi-3-small offers 8K and 128K context variants
- Phi-3-medium offers 4K and 128K context variants
- First announced as coming soon on April 23, 2024
Phi-3-vision
Phi-3-vision is the first multimodal model in Microsoft's Phi-3 family, a 4.2B model that combines language and vision to reason over images, extract text and interpret charts and diagrams.
- Modalities
- text, image → text
- Context
- 125K context
- Parameters
- 4.2B
- License
- MIT
- At launch
- Open weights, API, Cloud platforms
- 4.2B parameters with a 128K token context length
- Weights released May 21, 2024
- Optimized variants via ONNX Runtime, DirectML and NVIDIA NIM
CogVLM2
CogVLM2 is Z.ai's next-generation open vision-language model, based on Llama 3 8B, with English and Chinese chat models. It supports image resolutions up to 1344 by 1344.
- Modalities
- text, image → text
- Context
- 8K context
- Parameters
- 19B
- License
- CogVLM License
- At launch
- Open weights
- 8K content length
- A video understanding version, CogVLM2-Video, followed on July 8, 2024
- Weights under the CogVLM License
DeepSeek-V2-Lite
DeepSeek-V2-Lite is a smaller mixture-of-experts model in the DeepSeek-V2 family, released as a base model and an SFT chat model. It uses the same economical training and efficient inference architecture as DeepSeek-V2.
- Modalities
- text → text
- Context
- 32K context
- Parameters
- MoE: 16B total, 2.4B active
- License
- DeepSeek Model License
- At launch
- Open weights
- 16B total parameters with 2.4B activated
- 32K context length
- Released as DeepSeek-V2-Lite and DeepSeek-V2-Lite-Chat
Doubao (Doubao-pro and Doubao-lite)
ByteDance launched the Doubao model family on Volcano Engine's Ark platform, with Doubao-pro and Doubao-lite language models in 4K, 32K, and 128K context versions plus a Doubao text embedding model.
- Modalities
- text → text
- Context
- 125K context
- At launch
- API
- Doubao-pro offered in 4K, 32K, and 128K context versions
- Doubao-lite offered in 4K, 32K, and 128K context versions
- Function-calling and role-play variants of Doubao-pro-4k released the same day
- Doubao-embedding supports Chinese and English with 4K context
Gemini 1.5 Flash
Gemini 1.5 Flash is a lighter-weight member of the Gemini 1.5 family, designed to be fast and efficient to serve at scale while keeping the breakthrough 1 million token long context window.
- Modalities
- text, image, audio, video → text
- Context
- 1M context
- At launch
- API, Cloud platforms, Consumer app
- Introduced at Google I/O 2024 in public preview in Google AI Studio and Vertex AI
- Optimized for high-volume, high-frequency tasks; the fastest Gemini model served in the API at launch
- Stable gemini-1.5-flash-001 listed as generally available in the Gemini API changelog on May 23, 2024
Gemini 1.5 Pro (2M context update)
At Google I/O 2024, Google released an improved Gemini 1.5 Pro with better code generation, logical reasoning, planning and multi-turn conversation, and extended its context window to 2 million tokens.
- Modalities
- text, image, audio, video → text
- Context
- 2M context
- At launch
- API, Cloud platforms, Consumer app
- The 2 million token window was offered by waitlist to API developers and Google Cloud customers at announcement
- General availability of the 2M context window was recorded in the Gemini API changelog on June 27, 2024
- Also improved audio and image understanding
Hunyuan-DiT
Hunyuan-DiT is an open text-to-image diffusion transformer with fine-grained understanding of English and Chinese prompts, using bilingual CLIP and multilingual T5 text encoders and supporting multi-turn, dialogue-driven image generation.
- Modalities
- text → image
- Parameters
- 1.5B
- License
- Tencent Hunyuan Community License Agreement
- At launch
- Open weights, Consumer app
- 1.5B-parameter diffusion transformer
- Version 1.1 followed June 13, 2024 and version 1.2 on July 8, 2024
- Text-to-image also offered in the Tencent Hunyuan Bot web app
Imagen 3
Imagen 3 is Google DeepMind's text-to-image model announced at I/O 2024 as its highest-quality image model at the time. It started in private preview in ImageFX and later reached Vertex AI and the Gemini API.
- Modalities
- text → image
- At launch
- Consumer app, API, Cloud platforms
- Generally available on Vertex AI for approved users from July 31, 2024 and for all users from December 10, 2024
- Reached the Gemini API as imagen-3.0-generate-002 on February 6, 2025
- Initially offered in private preview to select creators in ImageFX
PaliGemma
PaliGemma is Google's first open vision-language model in the Gemma family, combining the SigLIP vision encoder with a Gemma 2B language model. It targets captioning, visual question answering, OCR, object detection and segmentation.
- Modalities
- text, image → text
- Parameters
- 3B
- License
- Gemma Terms of Use
- At launch
- Open weights, Cloud platforms
- Inspired by PaLI-3; image encoder initialized from SigLIP-So400m
- Pretrained, fine-tuned and mixed-task checkpoints at multiple resolutions
- Available on GitHub, Hugging Face, Kaggle and Vertex AI Model Garden
GPT-4o
GPT-4o is OpenAI's omni model, trained end to end across text, vision, and audio and accepting any mix of text, audio, image, and video input. It launched in ChatGPT, including the free tier, and in the API as a text and vision model.
- Modalities
- text, image, audio → text, audio
- Context
- 125K context
- At launch
- API, Consumer app
- API model ID gpt-4o-2024-05-13 at launch
- API audio and video capabilities went first to a small group of trusted partners
- A new Voice Mode built on GPT-4o was planned as an alpha for ChatGPT Plus
- 128,000-token context window per the gpt-4o docs model page
Yi-1.5
Yi-1.5 is an upgraded version of 01.AI's Yi models, continuously pre-trained on a 500B-token high-quality corpus and fine-tuned on 3M samples to improve coding, math, reasoning, and instruction following.
- Modalities
- text → text
- Parameters
- 6B, 9B, 34B
- License
- Apache-2.0
- At launch
- Open weights
- Base and chat models in 34B, 9B, and 6B sizes
- Context variants of 4K, 16K, and 32K tokens
- Code and weights distributed under Apache 2.0
Qwen-Max-0428
Qwen-Max-0428 is an instruction-tuned Qwen model for chat services, larger than the open Qwen1.5-110B. It reached the top 10 of the Chatbot Arena leaderboard shortly after release.
- Modalities
- text → text
- At launch
- API
- Available through the DashScope API
- DashScope API became compatible with the OpenAI API format
- Outperforms Qwen1.5-110B-Chat on MT-Bench, per Qwen
Amazon Titan Text Premier
Amazon Titan Text Premier is the most advanced model in the Titan Text family, optimized for retrieval-augmented generation and agent-based applications through integration with Knowledge Bases and Agents for Amazon Bedrock.
- Modalities
- text → text
- Context
- 32K context
- At launch
- API
- 32K token context window
- Best suited for English-language tasks
- Integrates with Knowledge Bases, Agents and Guardrails for Amazon Bedrock
DeepSeek-V2
DeepSeek-V2 is a mixture-of-experts language model designed for economical training and efficient inference, pre-trained on 8.1T tokens. Compared with DeepSeek 67B, it saves 42.5% of training costs and reduces the KV cache by 93.3%.
- Modalities
- text → text
- Context
- 125K context
- Parameters
- MoE: 236B total, 21B active
- License
- DeepSeek Model License
- At launch
- API, Consumer app, Open weights
- 236B total parameters with 21B activated per token
- 128K context length
- Released as a base model and an RL-tuned chat model
Granite Code (3B, 8B, 20B, 34B)
Family of open code models released as base and instruct versions in four sizes, trained on permissively licensed code and instruction data under Apache 2.0.
- Modalities
- text → text
- Parameters
- 3B, 8B, 20B and 34B
- License
- Apache-2.0
- At launch
- Open weights
- Context lengths range from 2K tokens for 3B to 8K tokens for 20B and 34B
- Instruct versions tuned for logical reasoning and problem solving
- Released under Apache 2.0
Jamba-Instruct
Instruction-tuned version of the Jamba open model launched in public preview on the AI21 platform for commercial use with a 256K-token context window.
- Modalities
- text → text
- Context
- 250K context
- At launch
- API
- Built on the Jamba SSM-Transformer base model
- Later offered on Azure, Amazon Bedrock and Snowflake Cortex
- Public preview on the AI21 Platform at launch
Cite and reuse this data
These release records are free to reuse under CC BY 4.0, including commercially. Credit The AI Engineering and link to this page's URL.
Dates are the labs' official announcement dates, and every record links to its source. List fields in the CSV are separated by " | ".
Citation
The AI Engineering Editors. "AI models released in May 2024." The AI Engineering, updated May 31, 2024. https://theaiengineering.com/releases/2024/05
HTML link
<a href="https://theaiengineering.com/releases/2024/05">AI models released in May 2024</a> (The AI Engineering)- Full dataset
- 1,167 releases from 55 labs
- Coverage
- to
- Updated
- License
- CC BY 4.0
- Publisher
- The AI Engineering