FACTS Grounding: A new benchmark for evaluating the factuality of large language models - Related to now, here's, a, chatgpt, an

FACTS Grounding: A new benchmark for evaluating the factuality of large language models

Our comprehensive benchmark and online leaderboard offer a much-needed measure of how accurately LLMs ground their responses in provided source material and avoid hallucinations.

Large language models (LLMs) are transforming how we access information, yet their grip on factual accuracy remains imperfect. They can “hallucinate” false information, particularly when given complex inputs. In turn, this can erode trust in LLMs and limit their applications in the real world.

Today, we’re introducing FACTS Grounding, a comprehensive benchmark for evaluating the ability of LLMs to generate responses that are not only factually accurate with respect to given inputs, but also sufficiently detailed to provide satisfactory answers to user queries.

We hope our benchmark will spur industry-wide progress on factuality and grounding. To track progress, we’re also launching the FACTS leaderboard on Kaggle. We’ve already tested leading LLMs using FACTS Grounding and have populated the initial leaderboard with their grounding scores. We will maintain and enhancement the leaderboard as the field advances.

Quantum computers have the potential to revolutionize drug discovery, material design and fundamental physics — that is, if we can get them to work re...

In my professional life as a data scientist, I have encountered time series multiple times. Most of my knowledge comes from my academic experience, sp...

Impact MuZero, AlphaZero, and AlphaDev: Optimizing computer systems Share.

As part of our aim to build increasingly capable and gen...

Using AI to fight climate change

How we’re applying the latest AI developments to help fight climate change and build a more sustainable, low-carbon world.

AI is a powerful technology that will transform our future, so how can we best apply it to help combat climate change and find sustainable solutions?

Our climate & sustainability lead, Sims Witherspoon, who recently spoke about how AI can accelerate our transition to renewables at TED Countdown, explains, “Climate change is a multifaceted problem with no single solution. We need to move beyond discussing what we can do and start focusing on how we can do it.”.

The effects of climate change on Earth’s ecosystems are incredibly complex, and as part of our effort to use AI for solving some of the world’s most challenging problems, here are some of the ways we’re working to advance our understanding, optimise existing systems, and accelerate breakthrough science of climate and its effects.

Understand weather, climate, and their effects.

more effective understanding the core problems and their effects is a critical first step to tackling climate change. Working together with the UK Met Office, we developed a precipitation nowcasting model to more effective understand changing weather. This nowcasting model is more accurate than the existing state of the art, and much preferred by the Met’s expert meteorologists. Our climate and weather research spans short-range (under two hours) to medium-range (ten days) forecasting, which can hugely impact how we optimise renewable energy systems that are based on natural resources.

From modelling the behaviour of animal species across the Serengeti to supporting machine learning projects that advance conservation projects in Africa, we’ve been helping scientists track and improved understand the effects of climate change on ecosystems and biodiversity. Going forward, our team is also building upon AI systems used to identify bird song in Australia, helping advance tools that monitor changing wildlife at scale.

Moreover, we’re partnering with non-profit Climate Change AI to close critical gaps in climate-related data. Currently, this partnership focuses on building a comprehensive wishlist of datasets whose availability would advance AI solutions for climate change. We’ll make this wishlist available to the wider public when it’s complete.

While we transition to more sustainable infrastructure, we need to optimise the systems the world depends on today. For example, today’s computing infrastructure, including AI itself, is energy-intensive. To help solve some of these issues, we’ve been developing AI that can enhance existing systems, including optimising industrial cooling and more efficient computer systems.

Given our energy grids are not yet running on clean energy, it's critical we use our resources as efficiently as possible while we work on the transition to renewables. Accelerating the global transition to renewable energy reports can also greatly reduce carbon emissions.

In 2019, our climate & sustainability team collaborated with domain experts at a Google-owned wind farm to increase the value of wind energy – ultimately, aiming to support growth across the broader industry. By developing a custom AI tool to better predict wind power output and another model to recommend commitments to supply this anticipated energy to the electricity grid, this tool greatly boosted the value of the wind energy. Cloud is now developing a software product using this model, which is being piloted by French power company ENGIE.

Research Google DeepMind at ICLR 2024 Share.

Developing next-gen AI agents, exploring new modalities, and pioneering foundational l...

Applying our AI research to help enrich the lives of billions of people around the world.

Building useful products with new technologies has always be...

Technologies Pushing the frontiers of audio generation Share.

Our pioneering speech generation technologies are helping people arou...

You can access ChatGPT Search without an account now - here's how

The best way to get the most out of an AI chatbot is verifying its findings and letting it direct you to more robust and useful content regarding the topic. Therefore, AI search engines are a powerful tool for everyday workflows, and, now, accessing ChatGPT Search just got a whole lot easier.

Also: How to make ChatGPT provide improved information and citations.

ChatGPT Search, originally launched in October, lets individuals search the web directly within the ChatGPT interface for timely, up-to-date information, complete with citations linked to insights. There are two big advantages that differentiate it from a traditional search engine: using conversational search queries, and getting a more personalized results page.

With ChatGPT Search, you can enter a fragmented or unorganized sentence that follows your train of thought, and the AI can still understand the meaning of your query by leveraging its natural language processing capabilities. The result is a simplified search experience in which you can spend less time crafting a tailored search query or thinking of specific keywords.

Also: The best AI for coding in 2025 (and what not to use).

For example, if you wanted to know what the technical name for a bee humming is, instead of generally searching "bee hums" on Google, you could say something like, "What is the scientific term for the sound a bee makes when it flies around you? It kinda sounds like a hum."

You can try ChatGPT Search for yourself for free now. You can also compare it to the many AI search engines available on the market for free, which we rounded up for you here.

When you're inviting friends, family, or co-workers to an event, the most convenient option i...

Firefox people can now tap into their favorite AI directly from th...

SOPA Images / Contributor / Getty Images.

Although you may be tuning into the Super Bowl for the foo...