The ERA Project: Building a Benchmark for Error-Reasoning Agents in Science
The main objective of this Leverhulme-funded project is to build the first benchmark that specifically assesses the error-reasoning ability of AI agents and human-AI teams in science.
In addition, our project will assess how automation changes science. How does AI-driven automation impact researchers' lived experience and sense of identity? What level of autonomy should AI Scientists be given, and what place should humans have in the newly emerging research workflows?
Defining error
The ERA Project will build a new theory of error and error-reasoning in science, with a focus on error in the life sciences. We will use this framework to build at least two different error-reasoning benchmarks that can assess isolated AI agents and human-AI teams.
Our goal is to enable a critical, reliable and trustworthy implementation of AI-driven automation in science.