AlphaGeometry: An Olympiad-level AI system for geometry - Related to iclr, 2024, deepmind, a, an

A glimpse of the next generation of AlphaFold

Research A glimpse of the next generation of AlphaFold Share.

Progress improvement: Our latest AlphaFold model displays significantly improved accuracy and expands coverage beyond proteins to other biological molecules, including ligands Since its release in 2020, AlphaFold has revolutionized how proteins and their interactions are understood. Google DeepMind and Isomorphic Labs have been working together to build the foundations of a more powerful AI model that expands coverage beyond just proteins to the full range of biologically-relevant molecules. Today we’re sharing an improvement on progress towards the next generation of AlphaFold. Our latest model can now generate predictions for nearly all molecules in the Protein Data Bank (PDB), frequently reaching atomic accuracy. It unlocks new understanding and significantly improves accuracy in multiple key biomolecule classes, including ligands (small molecules), proteins, nucleic acids (DNA and RNA), and those containing post-translational modifications (PTMs). These different structure types and complexes are essential for understanding the biological mechanisms within the cell, and have been challenging to predict with high accuracy. The model’s expanded capabilities and performance can help accelerate biomedical breakthroughs and realize the next era of 'digital biology’ — giving new insights into the functioning of disease pathways, genomics, biorenewable materials, plant immunity, potential therapeutic targets, mechanisms for drug design, and new platforms for enabling protein engineering and synthetic biology.

Watch Series of predicted structures compared to ground truth (white) from our latest AlphaFold model.

Above and beyond protein folding AlphaFold was a fundamental breakthrough for single chain protein prediction. AlphaFold-Multimer then expanded to complexes with multiple protein chains, followed by [website], which improved performance and expanded coverage to larger complexes. In 2022, AlphaFold’s structure predictions for nearly all cataloged proteins known to science were made freely available via the AlphaFold Protein Structure Database, in partnership with EMBL's European Bioinformatics Institute (EMBL-EBI). To date, [website] million individuals in over 190 countries have accessed the AlphaFold database, and scientists around the world have used AlphaFold’s predictions to help advance research on everything from accelerating new malaria vaccines and advancing cancer drug discovery to developing plastic-eating enzymes for tackling pollution. Here we show AlphaFold’s remarkable abilities to predict accurate structures beyond protein folding, generating highly-accurate structure predictions across ligands, proteins, nucleic acids, and post-translational modifications.

Performance across protein-ligand complexes (a), proteins (b), nucleic acids (c), and covalent modifications (d).

Accelerating drug discovery Early analysis also exhibits that our model greatly outperforms [website] on some protein structure prediction problems that are relevant for drug discovery, like antibody binding. Additionally, accurately predicting protein-ligand structures is an incredibly valuable tool for drug discovery, as it can help scientists identify and design new molecules, which could become drugs. Current industry standard is to use ‘docking methods’ to determine interactions between ligands and proteins. These docking methods require a rigid reference protein structure and a suggested position for the ligand to bind to. Our latest model sets a new bar for protein-ligand structure prediction by outperforming the best reported docking methods, without requiring a reference protein structure or the location of the ligand pocket — allowing predictions for completely novel proteins that have not been structurally characterized before. It can also jointly model the positions of all atoms, allowing it to represent the full inherent flexibility of proteins and nucleic acids as they interact with other molecules — something not possible using docking methods. Here, for instance, are three lately published, therapeutically-relevant cases where our latest model’s predicted structures (shown in color) closely match the experimentally determined structures (shown in gray): PORCN: A clinical stage anti-cancer molecule bound to its target, together with another protein. KRAS: Ternary complex with a covalent ligand (a molecular glue) of an critical cancer target. PI5P4Kγ: Selective allosteric inhibitor of a lipid kinase, with multiple disease implications including cancer and immunological disorders.

Predictions for PORCN (1), KRAS (2), and PI5P4Kγ (3).

Isomorphic Labs is applying this next generation AlphaFold model to therapeutic drug design, helping to rapidly and accurately characterize many types of macromolecular structures essential for treating disease.

New understanding of biology By unlocking the modeling of protein and ligand structures together with nucleic acids and those containing post-translational modifications, our model provides a more rapid and accurate tool for examining fundamental biology. One example involves the structure of CasLambda bound to crRNA and DNA, part of the CRISPR family. CasLambda shares the genome editing ability of the CRISPR-Cas9 system, commonly known as ‘genetic scissors’, which researchers can use to change the DNA of animals, plants, and microorganisms. CasLambda’s smaller size may allow for more efficient use in genome editing.

Predicted structure of CasLambda (Cas12l) bound to crRNA and DNA, part of the CRISPR subsystem.

Technologies GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy Share.

AI is revolutionizing the landscape of scientific research, enabling advancements at a pace that was once unimaginable — from accelerating drug discov...

This morning, Co-founder and CEO of Google DeepMind and Isomorphic Labs Sir Demis Hassabis, and Google DeepMind Director Dr. John Jumper were co-award...

AlphaGeometry: An Olympiad-level AI system for geometry

Research AlphaGeometry: An Olympiad-level AI system for geometry Share.

Our AI system surpasses the state-of-the-art approach for geometry problems, advancing AI reasoning in mathematics Reflecting the Olympic spirit of ancient Greece, the International Mathematical Olympiad is a modern-day arena for the world's brightest high-school mathematicians. The competition not only showcases young talent, but has emerged as a testing ground for advanced AI systems in math and reasoning. In a paper , we introduce AlphaGeometry, an AI system that solves complex geometry problems at a level approaching a human Olympiad gold-medalist - a breakthrough in AI performance. In a benchmarking test of 30 Olympiad geometry problems, AlphaGeometry solved 25 within the standard Olympiad time limit. For comparison, the previous state-of-the-art system solved 10 of these geometry problems, and the average human gold medalist solved [website] problems.

In our benchmarking set of 30 Olympiad geometry problems (IMO-AG-30), compiled from the Olympiads from 2000 to 2022, AlphaGeometry solved 25 problems under competition time limits. This is approaching the average score of human gold medalists on these same problems. The previous state-of-the-art approach, known as “Wu’s method”, solved 10.

AI systems often struggle with complex problems in geometry and mathematics due to a lack of reasoning skills and training data. AlphaGeometry’s system combines the predictive power of a neural language model with a rule-bound deduction engine, which work in tandem to find solutions. And by developing a method to generate a vast pool of synthetic training data - 100 million unique examples - we can train AlphaGeometry without any human demonstrations, sidestepping the data bottleneck.

With AlphaGeometry, we demonstrate AI’s growing ability to reason logically, and to discover and verify new knowledge. Solving Olympiad-level geometry problems is an essential milestone in developing deep mathematical reasoning on the path towards more advanced and general AI systems. We are open-sourcing the AlphaGeometry code and model, and hope that together with other tools and approaches in synthetic data generation and training, it helps open up new possibilities across mathematics, science, and AI.

“ It makes perfect sense to me now that researchers in AI are trying their hands on the IMO geometry problems first because finding solutions for them works a little bit like chess in the sense that we have a rather small number of sensible moves at every step. But I still find it stunning that they could make it work. It's an impressive achievement. Ngô Bảo Châu, Fields Medalist and IMO gold medalist.

AlphaGeometry adopts a neuro-symbolic approach.

AlphaGeometry is a neuro-symbolic system made up of a neural language model and a symbolic deduction engine, which work together to find proofs for complex geometry theorems. Akin to the idea of “thinking, fast and slow”, one system provides fast, “intuitive” ideas, and the other, more deliberate, rational decision-making.

Because language models excel at identifying general patterns and relationships in data, they can quickly predict potentially useful constructs, but often lack the ability to reason rigorously or explain their decisions. Symbolic deduction engines, on the other hand, are based on formal logic and use clear rules to arrive at conclusions. They are rational and explainable, but they can be “slow” and inflexible - especially when dealing with large, complex problems on their own. AlphaGeometry’s language model guides its symbolic deduction engine towards likely solutions to geometry problems. Olympiad geometry problems are based on diagrams that need new geometric constructs to be added before they can be solved, such as points, lines or circles. AlphaGeometry’s language model predicts which new constructs would be most useful to add, from an infinite number of possibilities. These clues help fill in the gaps and allow the symbolic engine to make further deductions about the diagram and close in on the solution.

AlphaGeometry solving a simple problem: Given the problem diagram and its theorem premises (left), AlphaGeometry (middle) first uses its symbolic engine to deduce new statements about the diagram until the solution is found or new statements are exhausted. If no solution is found, AlphaGeometry’s language model adds one potentially useful construct (blue), opening new paths of deduction for the symbolic engine. This loop continues until a solution is found (right). In this example, just one construct is required.

AlphaGeometry solving an Olympiad problem: Problem 3 of the 2015 International Mathematics Olympiad (left) and a condensed version of AlphaGeometry’s solution (right). The blue elements are added constructs. AlphaGeometry’s solution has 109 logical steps.

Generating 100 million synthetic data examples.

Geometry relies on understanding of space, distance, shape, and relative positions, and is fundamental to art, architecture, engineering and many other fields. Humans can learn geometry using a pen and paper, examining diagrams and using existing knowledge to uncover new, more sophisticated geometric properties and relationships. Our synthetic data generation approach emulates this knowledge-building process at scale, allowing us to train AlphaGeometry from scratch, without any human demonstrations. Using highly parallelized computing, the system started by generating one billion random diagrams of geometric objects and exhaustively derived all the relationships between the points and lines in each diagram. AlphaGeometry found all the proofs contained in each diagram, then worked backwards to find out what additional constructs, if any, were needed to arrive at those proofs. We call this process “symbolic deduction and traceback”.

Visual representations of the synthetic data generated by AlphaGeometry.

That huge data pool was filtered to exclude similar examples, resulting in a final training dataset of 100 million unique examples of varying difficulty, of which nine million featured added constructs. With so many examples of how these constructs led to proofs, AlphaGeometry’s language model is able to make good suggestions for new constructs when presented with Olympiad geometry problems.

Pioneering mathematical reasoning with AI.

The solution to every Olympiad problem provided by AlphaGeometry was checked and verified by computer. We also compared its results with previous AI methods, and with human performance at the Olympiad. In addition, Evan Chen, a math coach and former Olympiad gold-medalist, evaluated a selection of AlphaGeometry’s solutions for us.

Chen noted: “AlphaGeometry's output is impressive because it's both verifiable and clean. Past AI solutions to proof-based competition problems have sometimes been hit-or-miss (outputs are only correct sometimes and need human checks). AlphaGeometry doesn't have this weakness: its solutions have machine-verifiable structure. Yet despite this, its output is still human-readable. One could have imagined a computer program that solved geometry problems by brute-force coordinate systems: think pages and pages of tedious algebra calculation. AlphaGeometry is not that. It uses classical geometry rules with angles and similar triangles just as students do.”.

“ AlphaGeometry's output is impressive because it's both verifiable and clean…It uses classical geometry rules with angles and similar triangles just as students do. Evan Chen, math coach and Olympiad gold medalist.

As each Olympiad capabilities six problems, only two of which are typically focused on geometry, AlphaGeometry can only be applied to one-third of the problems at a given Olympiad. Nevertheless, its geometry capability alone makes it the first AI model in the world capable of passing the bronze medal threshold of the IMO in 2000 and 2015. In geometry, our system approaches the standard of an IMO gold-medalist, but we have our eye on an even bigger prize: advancing reasoning for next-generation AI systems. Given the wider potential of training AI systems from scratch with large-scale synthetic data, this approach could shape how the AI systems of the future discover new knowledge, in math and beyond.

AlphaGeometry builds on Google DeepMind and Google Research’s work to pioneer mathematical reasoning with AI – from exploring the beauty of pure mathematics to solving mathematical and scientific problems with language models. And most lately, we introduced FunSearch, which made the first discoveries in open problems in mathematical sciences using Large Language Models. Our long-term goal remains to build AI systems that can generalize across mathematical fields, developing the sophisticated problem-solving and reasoning that general AI systems will depend on, all the while extending the frontiers of human knowledge.

Technologies Generating audio for video Share.

Video-to-audio research uses video pixels and text prompts to generate rich soundtra...

Since founding Towards Data Science in 2016, we’ve built the largest publication on Medium with a dedicated community of readers and contributors focu...

Research Discovering novel algorithms with AlphaTensor Share.

First extension of AlphaZero to mathematics unlocks new possibilities...

Google DeepMind at ICLR 2024

Research Google DeepMind at ICLR 2024 Share.

Developing next-gen AI agents, exploring new modalities, and pioneering foundational learning Next week, AI researchers from around the globe will converge at the 12th International Conference on Learning Representations (ICLR), set to take place May 7-11 in Vienna, Austria. Raia Hadsell, Vice President of Research at Google DeepMind, will deliver a keynote reflecting on the last 20 years in the field, highlighting how lessons learned are shaping the future of AI for the benefit of humanity. We’ll also offer live demonstrations showcasing how we bring our foundational research into reality, from the development of Robotics Transformers to the creation of toolkits and open-source models like Gemma. Teams from across Google DeepMind will present more than 70 papers this year. Some research highlights:

Problem-solving agents and human-inspired approaches Large language models (LLMs) are already revolutionizing advanced AI tools, yet their full potential remains untapped. For instance, LLM-based AI agents capable of taking effective actions could transform digital assistants into more helpful and intuitive AI tools. AI assistants that follow natural language instructions to carry out web-based tasks on people’s behalf would be a huge timesaver. In an oral presentation we introduce WebAgent, an LLM-driven agent that learns from self-experience to navigate and manage complex tasks on real-world websites. To further enhance the general usefulness of LLMs, we focused on boosting their problem-solving skills. We demonstrate how we achieved this by equipping an LLM-based system with a traditionally human approach: producing and using “tools”. Separately, we present a training technique that ensures language models produce more consistently socially acceptable outputs. Our approach uses a sandbox rehearsal space that represents the values of society. Pushing boundaries in vision and coding.

Our Dynamic Scene Transformer (DyST) model leverages real-world single-camera videos to extract 3D representations of objects in the scene and their movements.

Until lately, large AI models mostly focused on text and images, laying the groundwork for large-scale pattern recognition and data interpretation. Now, the field is progressing beyond these static realms to embrace the dynamics of real-world visual environments. As computing advances across the board, it is increasingly key that its underlying code is generated and optimized with maximum efficiency. When you watch a video on a flat screen, you intuitively grasp the three-dimensional nature of the scene. Machines, however, struggle to emulate this ability without explicit supervision. We showcase our Dynamic Scene Transformer (DyST) model, which leverages real-world single-camera videos to extract 3D representations of objects in the scene and their movements. What’s more, DyST also enables the generation of novel versions of the same video, with user control over camera angles and content. Emulating human cognitive strategies also makes for improved AI code generators. When programmers write complex code, they typically “decompose” the task into simpler subtasks. With ExeDec, we introduce a novel code-generating approach that harnesses a decomposition approach to elevate AI systems’ programming and generalization performance. In a parallel spotlight paper we explore the novel use of machine learning to not only generate code, but to optimize it, introducing a dataset for the robust benchmarking of code performance. Code optimization is challenging, requiring complex reasoning, and our dataset enables the exploration of a range of ML techniques. We demonstrate that the resulting learning strategies outperform human-crafted code optimizations.

ExeDec introduces a novel code-generating approach that harnesses a decomposition approach to elevate AI systems’ programming and generalization performance.

Looking hundreds of millions of years into a protein’s past with AlphaFold to learn about the beginnings of life itself.

Responsibility & Safety An early warning system for novel AI risks Share.

New research proposes a framework for evaluating general-...

Former intern turned intern manager, Richard Everett, describes his journey to DeepMind, sharing tips and advice for aspiring DeepMinders. The 2023 in...