This post is part of Epoch AI’s Gradient Updates newsletter, which shares more opinionated or informal takes on big questions in AI progress. These posts solely represent the views of the authors, and do not necessarily reflect the views of Epoch AI as a whole.
In this post, I lay out some big questions I have about AI capabilities: questions I think govern the impact that AI will have on the world. These questions animate much of our benchmarking work at Epoch: we aim to build benchmarks, and conduct analyses using benchmarking data, that help answer these questions.
The questions broadly get at a mix of economic impact as well as the more fundamental drivers of AI capabilities. For each question I’ll say why it matters and mention some current benchmarking work trying to make progress toward answering it.
If you find these questions interesting and would like to help us answer them, apply to work at Epoch!
1. Can AI do my job?
More specifically, can AI go from doing narrowly-scoped tasks to doing messier, open-ended jobs? Epoch’s polling shows that, when people use AI for work, they still mostly only use it for part of a task. If this were to change, it would presage more significant labor market disruption — as well as continued fast revenue growth for AI model developers.
The majority of benchmarking to date has focused on relatively narrowly-scoped tasks: fixing bugs, solving math problems, writing reports. Some benchmarks push this farther. MirrorCode asks for large software packages to be implemented from scratch, albeit in a highly structured setting. Remote Labor Index extracts real projects from freelancing platforms and has humans grade AI deliverables vs. human reference work. My favorite, though, bites every bullet: Andon Café is a real cafe owned and operated by an AI agent. I expect we’ll see a lot more of this type of benchmarking, though perhaps in the form of repeated case studies rather than traditional benchmarks.
2. Is AI making progress in areas where we eventually expect it to have a big economic impact?
The “ChatGPT moment” came once AI could have a reasonably coherent conversation about almost any topic. The “Claude Code moment” came when AI could take a reasonable crack at almost any coding task. What will the next moment be? We can’t expect to predict exactly, but we can place some bets as to what capabilities will unlock near-term impact, and benchmark those capabilities.
Perhaps the most salient area right now is cybersecurity: when will AI be able to hack almost any vulnerable system? Cybersecurity benchmarks arguably were already ringing alarm bells, even before the Hugging Face incident.
Another area that comes to mind is computer use. Speaking proverbially, “my dad” uses AI all the time now, but only via chat apps — nothing agentic. My guess is that fast, reliable computer use will unlock another leg up for AI adoption, even if we can’t say exactly what degree of speed and reliability is necessary. Computer use benchmarks can help us track this.
I also think it’s worth tracking physical industries. Not necessarily robotics, but: can AI walk a less-skilled technician through repairing a broken machine on a factory floor? This doesn’t seem within reach yet, but would presumably be highly valuable. Benchmarks in this category might help clarify the picture.
3. How consistent is the gap between frontier and trailing models across domains?
This question applies to open vs. closed weights, the US vs. China, and even the US frontier (OpenAI, Anthropic) vs. the US near-frontier (xAI, Meta). All of these questions have an economic valence: can the leading developers capture enough value vs. their competition to sustain the frenetic infrastructure build-out? The US/China framing has additional geopolitical implications.
One way closed-weight developers could capture this value is if the open- vs. closed-weight capabilities gap is somehow larger than it seems — perhaps because open-weight model developers optimize their more limited resources to a few central domains, like coding, while neglecting a long tail of domains that are highly economically valuable. An extreme case of this is “benchmaxxing”, where developers prioritize achieving high benchmark scores even while their models lag at the capabilities those benchmarks are intended to measure.
Carefully designed benchmarks can shed some light on the economic question, though probably not answer it entirely. For instance, if winner-take-all dynamics are very strong, even a small capabilities gap could lead to closed-weight developers capturing most value.
4. Why are benchmark scores all correlated?
The Epoch Capabilities Index (ECI) combines scores from multiple benchmarks into a single general measure of AI capability. This works because, as it turns out, benchmark scores are highly correlated with each other — even across nominally different domains. How should we interpret this single, underlying dimension that statistical analysis reveals?
One possibility is that AI companies work hard to make sure every benchmark score goes up with every model release, but that this work is fairly independent from benchmark to benchmark. For instance, the work could consist of procuring training data for the domain of each benchmark. If so, there’s no deeper explanation here.
But there could be a deeper explanation: some underlying factor of general capability, akin to IQ in humans. If so, how might we interpret this factor? There may be no definitive answer, though some connections are notable: before it saturated, (the log of) METR’s Time Horizons measurement was highly correlated with ECI. Or, perhaps ECI gains are driven in large part by the maximal context length over which AI systems can sustain coherent reasoning.
ECI growth trends are very useful for detecting whether AI capabilities growth has accelerated. If we can say more clearly that ECI is measuring a quantity that translates directly to real-world impact, then an acceleration is easier to interpret. We might even hope for an economic metric: maybe a marginal frontier ECI point translates into revenue.
5. Can AI do AI R&D?
The first question in this list about more fundamental drivers of AI capabilities is the classic question of recursive self-improvement. Some versions of automated AI R&D lead to runaway capabilities growth — an intelligence explosion. A comprehensive suite of AI R&D benchmarks would serve as a leading indicator for this.
The process of AI R&D is complex, and automating it may require a wide range of capabilities. We can try to build benchmarks covering many aspects of that process. These could focus more on a single metric, or test for the ability to generate larger-grained research results.
Two notable challenges with benchmarking in this area are realism and cost. Realism refers to the fact that the most consequential AI R&D occurs within frontier AI companies, which are somewhat opaque to outside observers. Cost refers to the fact that AI R&D is resource-intensive when done at realistic scale, e.g. using considerable GPUs, and provisioning such resources for benchmarking is costly at some scales and impractical at others.
6. Can AI learn on the fly?
Related to the concept of continual learning, this asks whether AI can improve at a task through repeated attempts — typically only by managing its context, not by updating its weights. The implications are significant: pre-deployment testing fails to bound capabilities if AI can improve on the fly, whether at economically valuable tasks or dangerous ones.
Our benchmark EBR-bench tries to get at this by having AI repeatedly play a long, strategically rich, campaign-style board game. AI isn’t bad at it out of the box, but has so far failed to show much, if any, of a positive learning trajectory from playing the game repeatedly. This benchmark helps us recognize a continual learning capability, should one emerge.
7. What are the returns to inference scaling?
A relative of the previous question, this asks whether AI systems can accomplish any task if they think long enough. It’s vexing that we aren’t quite sure. Experiments tend to show logarithmic gains at best, so the requisite scale may be impractical for many tasks. But a capability that “merely” costs an impractical amount of tokens may soon be within reach, e.g. due to falling inference costs or additional training.
Beyond resource intensiveness, there are at least two challenges to answering this with benchmarks. First, we don’t know the optimal way to scale inference compute. Serial thinking time is an obvious strategy, but perhaps multi-agent architectures or specialized harnesses are more efficient. Second, the answer may differ across domains. It may be, for instance, that “hill-climbable” domains show steady logarithmic gains far beyond where harder-to-verify domains plateau.
8. How well does reinforcement learning generalize across domains and out of distribution?
Improvements in AI capabilities are likely due in part to improvements in training data: more of it, and higher quality. But to what extent is this the primary driver? Are AI systems bad at tasks that are far from their training distribution in some sense?
Not in the strongest sense: AI systems have improved at our benchmark of “puzzles” drawn from an undisclosed game (analogous to chess puzzles), despite it seeming unlikely that they were post-trained on this game. AI systems also do nearly as well on MirrorCode when working in low-resource languages as high-resource languages. The sort of reasoning that AI systems can do is already quite flexible.
The degree of generalization is less clear the farther one goes from the training distribution, including, for instance, to harder-to-verify tasks. It’s entirely possible that there isn’t much degradation, and that AI systems have crossed some sort of threshold where they are now “getting better at everything all at once” — sometimes described as a “GPT-3 moment for RL”. Big, if true.
Benchmarks with strong cases of being far from the training distribution, and otherwise not confounded with reasons AI might be bad at them, could help nail this down.
9. Can AI come up with “new ideas”?
We’re used to a certain degree of technological innovation: humans come up with new ideas, they are tested and elaborated, and eventually some have a significant impact on the world. AI could plausibly significantly increase the rate and value of these ideas, leading to unprecedented impact. It’s far from obvious that other bottlenecks — e.g., physical testing — wouldn’t slow this process. But who knows what a data center of geniuses can come up with?
We don’t quite have a data center of geniuses yet. Mathematical innovation may be the farthest along, but my sense of mathematicians’ characterization of AI math insights so far is that they are all the sort of things that humans can and do regularly come up with.
Benchmarks can help us look out for whether this remains the case. Indeed, a big part of our goal for FrontierMath: Open Problems is to make it easier to find such cases. Problems are selected to have a reasonable chance of requiring what humans would recognize as “new ideas”. Thus, we hope, any AI solution is at least worth looking into: maybe it came up with something surprising.
"Benchmaxxing" strikes as quite an interesting phenomenon, which is why independent benchmark evaluation is as equally important. Like eval the evals. A good place to start is could be to develop a new set of generic metrics, rather than not self-defined ones that merely minimize (or maximize) a specific target.
One cafe surviving is n=1 though, no counterfactual for what would fail under it. Case studies solve the narrow-task problem but reintroduce survivorship, which benchmarks were built to avoid