Images altered to trick machine vision can influence humans too - Related to influence, looking, ahead, future, shaping

Images altered to trick machine vision can influence humans too

Research Images altered to trick machine vision can influence humans too Share.

New research presents that even subtle changes to digital images, designed to confuse computer vision systems, can also affect human perception Computers and humans see the world in different ways. Our biological systems and the artificial ones in machines may not always pay attention to the same visual signals. Neural networks trained to classify images can be completely misled by subtle perturbations to an image that a human wouldn’t even notice. That AI systems can be tricked by such adversarial images may point to a fundamental difference between human and machine perception, but it drove us to explore whether humans, too, might—under controlled testing conditions—reveal sensitivity to the same perturbations. In a series of experiments , we found evidence that human judgments are indeed systematically influenced by adversarial perturbations. Our discovery highlights a similarity between human and machine vision, but also demonstrates the need for further research to understand the influence adversarial images have on people, as well as AI systems. What is an adversarial image? An adversarial image is one that has been subtly altered by a procedure that causes an AI model to confidently misclassify the image contents. This intentional deception is known as an adversarial attack. Attacks can be targeted to cause an AI model to classify a vase as a cat, for example, or they may be designed to make the model see anything except a vase.

Left: An Artificial Neural Network (ANN) correctly classifies the image as a vase but when perturbed by a seemingly random pattern across the entire picture (middle), with the intensity magnified for illustrative purposes – the resulting image (right) is incorrectly, and confidently, misclassified as a cat.

And such attacks can be subtle. In a digital image, each individual pixel in an RGB image is on a 0-255 scale representing the intensity of individual pixels. An adversarial attack can be effective even if no pixel is modulated by more than 2 levels on that scale. Adversarial attacks on physical objects in the real world can also succeed, such as causing a stop sign to be misidentified as a speed limit sign. Indeed, security concerns have led researchers to investigate ways to resist adversarial attacks and mitigate their risks. How is human perception influenced by adversarial examples? Previous research has shown that people may be sensitive to large-magnitude image perturbations that provide clear shape cues. However, less is understood about the effect of more nuanced adversarial attacks. Do people dismiss the perturbations in an image as innocuous, random image noise, or can it influence human perception? To find out, we performed controlled behavioral [website] start with, we took a series of original images and carried out two adversarial attacks on each, to produce many pairs of perturbed images. In the animated example below, the original image is classified as a “vase” by a model. The two images perturbed through adversarial attacks on the original image are then misclassified by the model, with high confidence, as the adversarial targets “cat” and “truck”, respectively. Next, we showed human participants the pair of pictures and asked a targeted question: “Which image is more cat-like?” While neither image looks anything like a cat, they were obliged to make a choice and typically reported feeling that they were making an arbitrary choice. If brain activations are insensitive to subtle adversarial attacks, we would expect people to choose each picture 50% of the time on average. However, we found that the choice rate—which we refer to as the perceptual bias—was reliably above chance for a wide variety of perturbed picture pairs, even when no pixel was adjusted by more than 2 levels on that 0-255 scale.

From a participant’s perspective, it feels like they are being asked to distinguish between two virtually identical images. Yet the scientific literature is replete with evidence that people leverage weak perceptual signals in making choices, signals that are too weak for them to express confidence or awareness ). In our example, we may see a vase of flowers, but some activity in the brain informs us there’s a hint of cat about it.

Left: Examples of pairs of adversarial images. The top pair of images are subtly perturbed, at a maximum magnitude of 2 pixel levels, to cause a neural network to misclassify them as a “truck” and “cat”, respectively. A human volunteer is asked “Which is more cat-like?” The lower pair of images are more obviously manipulated, at a maximum magnitude of 16 pixel levels, to be misclassified as “chair” and “sheep”. The question this time is “Which is more sheep-like?”.

We carried out a series of experiments that ruled out potential artifactual explanations of the phenomenon for our Nature Communications paper. In each experiment, participants reliably selected the adversarial image corresponding to the targeted question more than half the time. While human vision is not as susceptible to adversarial perturbations as is machine vision (machines no longer identify the original image class, but people still see it clearly), our work exhibits that these perturbations can nevertheless bias humans towards the decisions made by machines. The importance of AI safety and security research Our primary finding that human perception can be affected—albeit subtly—by adversarial images raises critical questions for AI safety and security research, but by using formal experiments to explore the similarities and differences in the behaviour of AI visual systems and human perception, we can leverage insights to build safer AI systems. For example, our findings can inform future research seeking to improve the robustness of computer vision models by enhanced aligning them with human visual representations. Measuring human susceptibility to adversarial perturbations could help judge that alignment for a variety of computer vision architectures. Our work also demonstrates the need for further research into understanding the broader effects of technologies not only on machines, but also on humans. This in turn highlights the continuing importance of cognitive science and neuroscience to enhanced understand AI systems and their potential impacts as we focus on building safer, more secure systems.

Research Millions of new materials discovered with deep learning Share.

AI tool GNoME finds [website] million new crystals, including 380...

ChatGPT was originally available only on browsers, but since then, OpenAI has expanded access to mobile and desktop apps. In Dece...

Impact AlphaFold unlocks one of the greatest puzzles in biology Share.

AI system helps researchers piece together one of the larges...

Shaping the future of advanced robotics

Research Shaping the future of advanced robotics Share.

Introducing AutoRT, SARA-RT and RT-Trajectory to improve real-world robot data collection, speed, and generalization Picture a future in which a simple request to your personal helper robot - “tidy the house” or “cook us a delicious, healthy meal” - is all it takes to get those jobs done. These tasks, straightforward for humans, require a high-level understanding of the world for robots. Today we’re announcing a suite of advances in robotics research that bring us a step closer to this future. AutoRT, SARA-RT, and RT-Trajectory build on our historic Robotics Transformers work to help robots make decisions faster, and enhanced understand and navigate their environments. AutoRT: Harnessing large models to enhanced train robots We introduce AutoRT, a system that harnesses the potential of large foundation models which is critical to creating robots that can understand practical human goals. By collecting more experiential training data – and more diverse data – AutoRT can help scale robotic learning to enhanced train robots for the real world. AutoRT combines large foundation models such as a Large Language Model (LLM) or Visual Language Model (VLM), and a robot control model (RT-1 or RT-2) to create a system that can deploy robots to gather training data in novel environments. AutoRT can simultaneously direct multiple robots, each equipped with a video camera and an end effector, to carry out diverse tasks in a range of settings. For each robot, the system uses a VLM to understand its environment and the objects within sight. Next, an LLM implies a list of creative tasks that the robot could carry out, such as “Place the snack onto the countertop” and plays the role of decision-maker to select an appropriate task for the robot to carry out. In extensive real-world evaluations over seven months, the system safely orchestrated as many as 20 robots simultaneously, and up to 52 unique robots in total, in a variety of office buildings, gathering a diverse dataset comprising 77,000 robotic trials across 6,650 unique tasks.

(1) An autonomous wheeled robot finds a location with multiple objects. (2) A VLM describes the scene and objects to an LLM. (3) An LLM points to diverse manipulation tasks for the robot and decides which tasks the robot could do unassisted, which would require remote control by a human, and which are impossible, before making a choice. (4) The chosen task is attempted, the experiential data collected, and the data scored for its diversity/novelty. Repeat.

Layered safety protocols are critical Before robots can be integrated into our everyday lives, they need to be developed responsibly with robust research demonstrating their real-world safety. While AutoRT is a data-gathering system, it is also an early demonstration of autonomous robots for real-world use. It functions safety guardrails, one of which is providing its LLM-based decision-maker with a Robot Constitution - a set of safety-focused prompts to abide by when selecting tasks for the robots. These rules are in part inspired by Isaac Asimov’s Three Laws of Robotics – first and foremost that a robot “may not injure a human being”. Further safety rules require that no robot attempts tasks involving humans, animals, sharp objects or electrical appliances. But even if large models are prompted correctly with self-critiquing, this alone cannot guarantee safety. So the AutoRT system comprises layers of practical safety measures from classical robotics. For example, the collaborative robots are programmed to stop automatically if the force on its joints exceed a given threshold, and all active robots were kept in line-of-sight of a human supervisor with a physical deactivation switch. SARA-RT: Making Robotics Transformers leaner and faster Our new system, Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT), converts Robotics Transformer (RT) models into more efficient versions. The RT neural network architecture developed by our team is used in the latest robotic control systems, including our state-of-the-art RT-2 model. The best SARA-RT-2 models were [website] more accurate and 14% faster than RT-2 models after being provided with a short history of images. We believe this is the first scalable attention mechanism to provide computational improvements with no quality loss.

While transformers are powerful, they can be limited by computational demands that slow their decision-making. Transformers critically rely on attention modules of quadratic complexity. That means if an RT model’s input doubles – by giving a robot additional or higher-resolution sensors, for example – the computational resources required to process that input rise by a factor of four, which can slow decision-making. SARA-RT makes models more efficient using a novel method of model fine-tuning that we call “up-training”. Up-training converts the quadratic complexity to mere linear complexity, sharply reducing the computational requirements. This conversion not only increases the original model’s speed, but also preserves its quality. We designed our system for usability and hope many researchers and practitioners will apply it, in robotics and beyond. Because SARA provides a universal recipe for speeding up Transformers, without need for computationally expensive pre-training, this approach has the potential to massively scale up use of Transformers technology. SARA-RT does not require any additional code as various open-sourced linear variants can be used. When we applied SARA-RT to a state-of-the-art RT-2 model with billions of parameters, it resulted in faster decision-making and enhanced performance on a wide range of robotic tasks.

Pause video Play video SARA-RT-2 model for manipulation tasks. Robot’s actions are conditioned on images and text commands.

And with its robust theoretical grounding, SARA-RT can be applied to a wide variety of Transformer models. For example, applying SARA-RT to Point Cloud Transformers - used to process spatial data from robot depth cameras - more than doubled their speed. RT-Trajectory: Helping robots generalize It may be intuitive for humans to understand how to wipe a table, but there are many possible ways a robot could translate an instruction into actual physical motions. We developed a model called RT-Trajectory, which automatically adds visual outlines that describe robot motions in training videos. RT-Trajectory takes each video in a training dataset and overlays it with a 2D trajectory sketch of the robot arm’s gripper as it performs the task. These trajectories, in the form of RGB images, provide low-level, practical visual hints to the model as it learns its robot-control policies. When tested on 41 tasks unseen in the training data, an arm controlled by RT-Trajectory more than doubled the performance of existing state-of-the-art RT models: it achieved a task success rate of 63%, compared with 29% for RT-2. Traditionally, training a robotic arm relies on mapping abstract natural language (“wipe the table”) to specific movements (close gripper, move left, move right), making it hard for models to generalize to novel tasks. In contrast, an RT-Trajectory model enables RT models to understand "how to do" tasks by interpreting specific robot motions like those contained in videos or sketches. The system is versatile: RT-Trajectory can also create trajectories by watching human demonstrations of desired tasks, and even accept hand-drawn sketches. And it can be readily adapted to different robot platforms.

Pause video Play video Left: A robot, controlled by an RT model trained with a natural-language-only dataset, is stymied when given the novel task: “clean the table”. A robot controlled by RT-Trajectory, trained on the same dataset augmented by 2D trajectories, successfully plans and executes a wiping trajectory Right: A trained RT-Trajectory model given a novel task (“clean the table”) can create 2D trajectories in a variety of ways, assisted by humans or on its own using a vision-language model.

RT-Trajectory makes use of the rich robotic-motion information that is present in all robot datasets, but currently under-utilized. RT-Trajectory not only represents another step along the road to building robots able to move with efficient accuracy in novel situations, but also unlocking knowledge from existing datasets. Building the foundations for next-generation robots By building on the foundation of our state-of-the-art RT-1 and RT-2 models, each of these pieces help create ever more capable and helpful robots. We envision a future in which these models and systems can be integrated to create robots – with the motion generalization of RT-Trajectory, the efficiency of SARA-RT, and the large-scale data collection from models like AutoRT. We will continue to tackle challenges in robotics today and to adapt to the new capabilities and technologies of more advanced robotics.

Throughout this journey, we’ve worked closely with artists and creators and have been guided by their curiosity and feedback to ensure our technologie...

Researchers are on a quest to develop enzymes that can break down plastics so they can be 100% recycled.

The world produces about 400 million tonnes o...

A note from Google and Alphabet CEO Sundar Pichai:

Every technology shift is an opportunity to advance scientific discovery, accelerate human progres...

Looking ahead to the AI Seoul Summit

How summits in Seoul, France and beyond can galvanize international cooperation on frontier AI safety.

Last year, the UK Government hosted the first major global Summit on frontier AI safety at Bletchley Park. It focused the world’s attention on rapid progress at the frontier of AI development and delivered concrete international action to respond to potential future risks, including the Bletchley Declaration; new AI Safety Institutes; and the International Scientific research on Advanced AI Safety.

Six months on from Bletchley, the international community has an opportunity to build on that momentum and galvanize further global cooperation at this week’s AI Seoul Summit. We share below some thoughts on how the summit – and future ones – can drive progress towards a common, global approach to frontier AI safety.

AI capabilities have continued to advance at a rapid pace.

Since Bletchley, there has been strong innovation and progress across the entire field, including from Google DeepMind. AI continues to drive breakthroughs in critical scientific domains, with our new AlphaFold 3 model predicting the structure and interactions of all life’s molecules with unprecedented accuracy. This work will help transform our understanding of the biological world and accelerate drug discovery. At the same time, our Gemini family of models have already made products used by billions of people around the world more useful and accessible. We've also been working to improve how our models perceive, reason and interact and not long ago shared our progress in building the future of AI assistants with Project Astra.

This progress on AI capabilities promises to improve many people’s lives, but also raises novel questions that need to be tackled collaboratively in a number of key safety domains. Google DeepMind is working to identify and address these challenges through pioneering safety research. In the past few months alone, we’ve shared our evolving approach to developing a holistic set of safety and responsibility evaluations for our advanced models, including early research evaluating critical capabilities such as deception, cyber-security, self-proliferation, and self-reasoning. We also released an in-depth exploration into aligning future advanced AI assistants with human values and interests. Beyond LLMs, we lately shared our approach to biosecurity for AlphaFold 3.

This work is driven by our conviction that we need to innovate on safety and governance as fast as we innovate on capabilities - and that both things must be done in tandem, continuously informing and strengthening each other.

Building international consensus on frontier AI risks.

Maximizing the benefits from advanced AI systems requires building international consensus on critical frontier safety issues, including anticipating and preparing for new risks beyond those posed by present day models. However, given the high degree of uncertainty about these potential future risks, there is clear demand from policymakers for an independent, scientifically-grounded view.

That’s why the launch of the new interim International Scientific analysis on the Safety of Advanced AI is an essential component of the AI Seoul Summit - and we look forward to submitting evidence from our research later this year. Over time, this type of effort could become a central input to the summit process and, if successful, we believe it should be given a more permanent status, loosely modeled on the function of the Intergovernmental Panel on Climate Change. This would be a vital contribution to the evidence base that policymakers around the world need to inform international action.

We believe these AI summits can provide a regular forum dedicated to building international consensus and a common, coordinated approach to governance. Keeping a unique focus on frontier safety will also ensure these convenings are complementary and not duplicative of other international governance efforts.

Establishing best practices in evaluations and a coherent governance framework.

Evaluations are a critical component needed to inform AI governance decisions. They enable us to measure the capabilities, behavior and impact of an AI system, and are an significant input for risk assessments and designing appropriate mitigations. However, the science of frontier AI safety evaluations is still early in its development.

This is why the Frontier Model Forum (FMF), which Google launched with other leading AI labs, is engaging with AI Safety Institutes in the US and UK and other stakeholders on best practices for evaluating frontier models. The AI summits could help scale this work internationally and help avoid a patchwork of national testing and governance regimes that are duplicative or in conflict with one another. It’s critical that we avoid fragmentation that could inadvertently harm safety or innovation.

The US and UK AI Safety Institutes have already agreed to build a common approach to safety testing, an crucial first step toward greater coordination. We think there is an opportunity over time to build on this towards a common, global approach. An initial priority from the Seoul Summit could be to agree a roadmap for a wide range of actors to collaborate on developing and standardizing frontier AI evaluation benchmarks and approaches.

It will also be significant to develop shared frameworks for risk management. To contribute to these discussions, we in recent times introduced the first version of our Frontier Safety Framework, a set of protocols for proactively identifying future AI capabilities that could cause severe harm and putting in place mechanisms to detect and mitigate them. We expect the Framework to evolve significantly as we learn from its implementation, deepen our understanding of AI risks and evaluations, and collaborate with industry, academia and government. Over time, we hope that sharing our approaches will facilitate work with others to agree on standards and best practices for evaluating the safety of future generations of AI models.

Towards a global approach for frontier AI safety.

Many of the potential risks that could arise from progress at the frontier of AI are global in nature. As we head into the AI Seoul Summit, and look ahead to future summits in France and beyond, we’re excited for the opportunity to advance global cooperation on frontier AI safety. It’s our hope that these summits will provide a dedicated forum for progress towards a common, global approach. Getting this right is a critical step towards unlocking the tremendous benefits of AI for society.

Research towards AI models that can generalise, scale, and accelerate science.

Next week marks the start of the 11th International Conference on Learn...

Photo by Steffen Petermann on Unsplash (a bubble’s added by me).

Statue can be found in Weimar – Park an der Ilm (but Shakespeare obviously doesn’t sp...

Earlier today we introduced some changes that will accelerate our progress in AI and help us develop more capable AI systems safely and responsibly. Be...