Gemini, Google’s latest foundation model, is primed for success. It’s recognized for enhanced capability, flexibility, smartphone optimization, and achieving a groundbreaking 90% on the MMLU benchmark, showcasing advanced language understanding and problem-solving skills.
A Message from Sundar Pichai
A Message from Sundar Pichai, CEO of Google and Alphabet:
In the realm of technological advancements, each shift provides a gateway to propel scientific discovery, expedite human progress, and enhance lives. I firmly believe that the ongoing transition with AI holds the potential to be the most profound in our lifetimes, surpassing even the shifts to mobile or the web that came before. AI stands poised to create opportunities, both mundane and extraordinary, for individuals worldwide. It is set to usher in new waves of innovation and economic advancement, fostering knowledge, learning, creativity, and productivity on an unprecedented scale.
What excites me most is the prospect of making AI beneficial for everyone, irrespective of their location around the globe.
As we celebrate nearly eight years of being an AI-first company, the momentum of progress is not only sustained but accelerating. Generative AI, integrated across our products, is now empowering millions of individuals to accomplish tasks that were inconceivable just a year ago—whether it’s finding answers to complex questions or utilizing novel tools for collaboration and creation. Simultaneously, developers worldwide leverage our models and infrastructure to craft innovative generative AI applications, contributing to the growth of startups and enterprises.
This momentum is remarkable, yet we are merely scratching the surface of what AI can achieve.
Our approach to this endeavor is both bold and responsible. We aspire to be ambitious in our research, striving for capabilities that offer immense benefits to individuals and society. Concurrently, we incorporate safeguards, collaborating closely with governments and experts to address the evolving risks as AI becomes more capable. Our commitment includes ongoing investments in top-notch tools, foundational models, and infrastructure, extending their integration into our products and sharing them with others, guided by our AI Principles.
Today, we embark on the next phase of our journey with Gemini, our most advanced and versatile model to date, exhibiting state-of-the-art performance across numerous leading benchmarks. The inaugural release, Gemini 1.0, comes optimized in various sizes: Ultra, Pro, and Nano. These models mark the initiation of the Gemini era and materialize the vision conceived when we established Google DeepMind earlier this year. This new generation of models represents one of the most substantial science and engineering endeavors undertaken by our company. I am genuinely excited about what lies ahead and the opportunities that Gemini will unlock for individuals across the globe.
Introducing Gemini
Authored by Demis Hassabis, CEO and Co-Founder of Google DeepMind, on behalf of the Gemini Team:
AI has been the central focus of my life’s work, a shared dedication among many of my research colleagues. From my early days programming AI for computer games to my years as a neuroscience researcher delving into the intricacies of the brain, I’ve maintained a steadfast belief that the creation of more intelligent machines could lead to incredible benefits for humanity.
The commitment to realizing a world responsibly empowered by AI, epitomized by projects like Gemini, continues to propel our endeavors at Google DeepMind. We’ve long aspired to develop a new era of AI models, drawing inspiration from how individuals comprehend and engage with the world. Our goal has been to fashion AI, exemplified by models like Gemini, that transcends the conventional notion of smart software, instead resembling something both useful and intuitive — an adept helper or assistant.
Today marks a significant stride toward this vision as we proudly unveil Gemini, our most advanced and versatile model to date.
Gemini stands as the culmination of extensive collaborative efforts involving teams throughout Google, including our esteemed colleagues at Google Research. Conceived from the ground up, Gemini is designed to be multimodal, possessing the ability to generalize seamlessly and comprehend various forms of information, spanning text, code, audio, image, and video.
Unveiling Gemini: Our Most Robust and Advanced AI Model
Gemini represents an unparalleled level of flexibility, adeptly running on diverse platforms, from expansive data centers to nimble mobile devices. Its cutting-edge capabilities are poised to revolutionize how developers and enterprise clients construct and expand their AI applications.
Our inaugural release, Gemini 1.0, is meticulously optimized in three distinct sizes:
- Gemini Ultra: Our most extensive and powerful model, ideal for tackling highly complex tasks.
- Gemini Pro: Our optimal model for seamlessly scaling across a diverse array of tasks.
- Gemini Nano: Our most efficient model tailored for on-device tasks, ensuring streamlined performance.
Cutting-edge performance
We have conducted rigorous testing on our Gemini models, evaluating their performance across a diverse range of tasks. Whether it’s natural image, audio, and video comprehension or mathematical reasoning, Gemini Ultra demonstrates exceptional performance, surpassing current state-of-the-art benchmarks on 30 out of 32 widely-used academic benchmarks in large language model (LLM) research and development.
Scoring an impressive 90.0%, Gemini Ultra stands as the first model to outperform human experts in Massive Multitask Language Understanding (MMLU), a comprehensive test encompassing 57 subjects such as math, physics, history, law, medicine, and ethics. This evaluation assesses both world knowledge and problem-solving abilities.
Our innovative benchmark approach to MMLU empowers Gemini to leverage its reasoning capabilities, enabling more thoughtful responses to challenging questions and resulting in significant improvements over relying solely on initial impressions.
Gemini excels beyond the current state-of-the-art performance across various benchmarks, encompassing both text and coding evaluations.
Notably, Gemini Ultra attains a state-of-the-art score of 59.4% on the newly introduced MMMU benchmark, a comprehensive assessment involving multimodal tasks across diverse domains that demand intentional reasoning.
In our evaluations of image benchmarks, Gemini Ultra demonstrates superior performance compared to previous state-of-the-art models, achieving this without reliance on object character recognition (OCR) systems for extracting text from images. These results underscore Gemini’s inherent multimodality and provide early indicators of its advanced reasoning capabilities.
For more details in our Gemini technical report.
Gemini outperforms current state-of-the-art benchmarks across a variety of multimodal tasks.
Cutting-Edge Advancements: The Next Wave of Capabilities
Traditionally, creating multimodal models involved training separate components for various modalities and then integrating them to emulate certain functionalities. While effective for tasks like image description, these models often falter in more intricate and conceptual reasoning.
Gemini breaks from this approach by being inherently multimodal, undergoing initial pre-training on diverse modalities. Subsequently, fine-tuning with additional multimodal data enhances its effectiveness. This unique design enables Gemini to effortlessly comprehend and reason about diverse inputs, surpassing existing multimodal models across nearly every domain with state-of-the-art capabilities.
Learn more about Explore the functionalities of Gemini and observe its operational mechanisms.
Advanced reasoning
Gemini 1.0 possesses advanced multimodal reasoning capabilities, enabling it to decipher intricate written and visual information with finesse. This distinctive proficiency makes it particularly adept at uncovering nuanced knowledge that might be challenging to discern within extensive datasets.
The remarkable capacity of Gemini 1.0 to extract insights from vast document sets through reading, filtering, and comprehending information positions it to facilitate groundbreaking advancements at digital speeds across diverse fields, spanning from science to finance.
Gemini reveals fresh scientific discoveries.
Comprehending text, images, audio, and beyond
Gemini 1.0 underwent training to simultaneously recognize and comprehend text, images, audio, and more. This comprehensive approach enhances its ability to grasp nuanced information and respond to inquiries on intricate subjects, making it particularly proficient in explaining reasoning within complex fields such as math and physics.
Gemini provides explanations for reasoning in both mathematics and physics.
Sophisticated programming
Our initial iteration of Gemini demonstrates proficiency in comprehending, elucidating, and generating high-quality code across the globe’s most widely used programming languages, including Python, Java, C++, and Go. Its versatility in working across diverse languages and its capability to reason through intricate information position it as a leading foundational model for coding on a global scale.
Gemini Ultra exhibits outstanding performance across various coding benchmarks, including HumanEval—an industry-standard for evaluating coding task performance—and Natural2Code, an internal dataset employing author-generated sources rather than web-based information.
Furthermore, Gemini serves as the core for more advanced coding systems. Two years ago, we introduced AlphaCode, the pioneer AI code generation system achieving competitive performance in programming competitions.
Utilizing a specialized version of Gemini, we have now developed an even more advanced code generation system, AlphaCode 2, excelling in solving competitive programming problems that extend beyond coding to encompass complex mathematical and theoretical computer science concepts.
Gemini demonstrates exceptional proficiency in both coding and competitive programming.
Assessed on the identical platform as the initial AlphaCode, AlphaCode 2 demonstrates significant enhancements, successfully solving nearly twice the number of problems. Our estimation suggests that its performance surpasses that of 85% of competition participants, a substantial increase from the approximately 50% achieved by AlphaCode. Moreover, when programmers collaborate with AlphaCode 2, specifying certain properties for the code samples, its performance is further enhanced.
We look forward to programmers increasingly leveraging highly capable AI models as collaborative tools. These models can aid in problem-solving, propose code designs, and assist with implementation, enabling the accelerated release of applications and the design of superior services.
Learn more details in our AlphaCode 2 technical report.
Enhanced Reliability, Scalability, and Efficiency
We conducted extensive training of Gemini 1.0 at scale on our AI-optimized infrastructure, leveraging Google’s specially crafted Tensor Processing Units (TPUs) v4 and v5e. With a focus on reliability and scalability during training, as well as efficiency during serving, Gemini 1.0 stands as our most robust and scalable model.
Running on TPUs, Gemini exhibits a substantial increase in speed compared to earlier, smaller models with limited capabilities. These custom-designed AI accelerators form the backbone of Google’s AI-driven products, serving billions of users across platforms such as Search, YouTube, Gmail, Google Maps, Google Play, and Android. They have also empowered global companies to cost-effectively train large-scale AI models.
Today, we are unveiling Cloud TPU v5p, the most powerful, efficient, and scalable TPU system to date. Tailored for training cutting-edge AI models, this next-generation TPU will expedite the development of Gemini and assist developers and enterprise customers in training large-scale generative AI models more rapidly, thereby accelerating the delivery of new products and capabilities to customers.
A lineup of AI accelerator supercomputers, specifically Cloud TPU v5p, within a Google data center.
Rooted in Responsibility and Safety: The Foundation of Our Design
At Google, our dedication to advancing bold and responsible AI permeates every aspect of our work. In alignment with Google’s AI Principles and the robust safety protocols governing our products, we are introducing additional safeguards to account for Gemini’s multimodal capabilities. Throughout the developmental stages, we meticulously assess potential risks and actively work to test and mitigate them.
Gemini undergoes the most comprehensive safety evaluations of any Google AI model to date, encompassing assessments for bias and toxicity. Our exploration extends to novel research in potential risk areas such as cyber-offense, persuasion, and autonomy. Leveraging Google Research’s premier adversarial testing techniques, we proactively identify critical safety issues before Gemini’s deployment.
To enhance our internal evaluation methodology, we collaborate with a diverse group of external experts and partners to stress-test our models across various issues. To identify blind spots, we actively engage in thorough assessments to ensure our models meet high safety standards.
During Gemini’s training phases, we employ benchmarks like Real Toxicity Prompts—consisting of 100,000 prompts with varying toxicity levels sourced from the web, developed by experts at the Allen Institute for AI. More details on this work will be shared soon.
For content safety issues, we deploy dedicated safety classifiers to identify, label, and segregate content involving violence or negative stereotypes. This layered approach, combined with robust filters, is designed to enhance Gemini’s safety and inclusivity. We continue to address challenges such as factuality, grounding, attribution, and corroboration.
Our commitment to responsibility and safety remains steadfast in the long term. We collaborate with the industry and broader ecosystem to define best practices and establish safety and security benchmarks through organizations like MLCommons, the Frontier Model Forum, its AI Safety Fund, and our Secure AI Framework (SAIF). The latter is specifically crafted to mitigate security risks unique to AI systems across the public and private sectors. As we develop Gemini, we persist in partnering with researchers, governments, and civil society groups worldwide.
Global Availability of Gemini: Expanding Access Worldwide
The deployment of Gemini 1.0 is underway across various products and platforms:
The inclusion of Gemini Pro in Google’s product lineup
We are extending the reach of Gemini to billions of users through various Google products.
Effective immediately, Bard will integrate a finely tuned version of Gemini Pro, marking the most substantial upgrade since Bard’s launch. This enhanced version will be accessible in English across more than 170 countries and territories, with plans to expand to additional modalities and introduce support for new languages and regions shortly.
Gemini is also being introduced to Pixel devices, with Pixel 8 Pro being the inaugural smartphone optimized to run Gemini Nano. This integration powers novel features such as Summarize in the Recorder app and is set to roll out in Smart Reply on Gboard, beginning with WhatsApp and extending to more messaging apps in the coming year.
Over the next few months, Gemini will become available in an expanding array of our products and services, including Search, Ads, Chrome, and Duet AI.
We have commenced experimentation with Gemini in Search, leading to a 40% reduction in latency for users in the United States within our Search Generative Experience (SGE), accompanied by enhancements in quality.
Constructing with Gemini
Commencing on December 13, developers and enterprise customers can utilize Gemini Pro through the Gemini API, accessible in either Google AI Studio or Google Cloud Vertex AI.
Google AI Studio, a web-based developer tool, provides a free platform for rapid app prototyping and launch with an API key. For a fully-managed AI platform, Vertex AI offers customization options for Gemini along with comprehensive data control. It also leverages additional Google Cloud features, enhancing enterprise security, safety, privacy, data governance, and compliance.
Android developers will gain the capability to work with Gemini Nano, our most efficient on-device model, through AICore—a new system feature available in Android 14, starting with Pixel 8 Pro devices. Early access to AICore can be secured by signing up for an early preview.
The imminent arrival of Gemini Ultra
Gemini Ultra is undergoing comprehensive trust and safety assessments, including red-teaming by trusted external entities. We are further enhancing the model through fine-tuning and reinforcement learning from human feedback (RLHF) before its wide release.
As a crucial step in this process, Gemini Ultra will be accessible to select customers, developers, partners, safety experts, and responsibility experts for early experimentation and feedback. This precedes the broader rollout to developers and enterprise customers scheduled for early next year.
In tandem, early next year, we will introduce Bard Advanced—a cutting-edge AI experience providing access to our premier models and capabilities, debuting with Gemini Ultra.
Empowering Innovation: Welcome to the Gemini Epoch
This marks a momentous stride in AI development, heralding the dawn of a new era for us at Google as we consistently drive innovation and responsibly enhance the prowess of our models.
Our journey with Gemini has seen substantial advancements, and we are dedicated to further amplifying its capabilities in upcoming versions. This includes progress in planning and memory, along with expanding the context window to process a wealth of information for more refined responses.
The prospect of a world responsibly empowered by AI exhilarates us, offering incredible possibilities for a future marked by innovation. This future promises to elevate creativity, deepen knowledge, propel scientific endeavors, and revolutionize the lifestyles and work dynamics of billions around the globe.