AI can now do math. But can it ask good questions? - Ken Ono
When mathematicians make breakthroughs, they hallucinate too.
When mathematicians make breakthroughs, they hallucinate too.
They reach beyond established results. But unlike AI, they’ve learned to tell a promising hallucination from a dead end.
Number theorist Ken Ono on AI, creativity, and mathematical discovery.
Watch the interview here:
Transcript:
Why Predicting Problem Difficulty Is So Hard [00:00:00]
Daria
Could you introduce yourself briefly, what field you’re coming from and where you’re coming from?
Ken
My name is Ken Ono. I work in number theory, and I’m on the faculty at the University of Virginia.
Daria
What makes the problem hard for an AI?
Ken
It’s very difficult to predict which problems are the difficult ones. Earlier today, Elliott, who is the head of the whole enterprise, carried out a little survey to see if we could predict which problems have been hard to solve. And we’re not very good at that.
How Mathematicians and AI Both “Hallucinate” [00:01:00]
Ken
So what are some of the features that AI struggles with? Well, some of the features that AI struggles with that won’t come as a surprise are the creative bits. The leaps of faith, which are often called hallucinations to make fun of AI. But make no mistake, as pure mathematicians, when we are at our best, when we make discoveries that seem to be a little bit beyond what’s already in the literature, let me make something very clear: we are hallucinating. And from time to time, we’re right.
What have we learned about AI’s ability? Well, the way I describe it now is that I don’t think I can ask a question that AI can’t identify with a particular area of mathematics, because it seems like ChatGPT and others just have at their fingertips the accumulation of human knowledge. And it continues to surprise me how quickly that seems to be the case. Three years ago, ChatGPT or all of these large language models would get things wrong that a five year old could get right. And now we’re asking questions that a PhD student at our universities wouldn’t even know where to start to look.
AI as a Copilot for Mathematical Discovery [00:02:15]
Daria
Extending into the future, where do you expect AI to land in math in, say, three years?
Ken
I’m going to answer in kind of a funny way. Here we are in tier four, this Frontier Math symposium, with the idea of assembling a benchmark of 50 difficult mathematical problems. And what has been proposed to us is to think of a large language model as an opponent. We want to come up with insidiously difficult problems to oppose and test the capabilities of these entities. But you know what I’ve discovered today already, after just 5 or 6 hours, is that we’ve seen a transformation on the part of the participants. When the participants see and test out the pro version of ChatGPT that has been provided to us, you can see their eyes light up thinking it is amazing. This could be really, genuinely my copilot.
So where do I think we’re going? We’re probably going to talk about the performance on these problems later this summer, and that’ll be a big deal. But perhaps the bigger deal will be the realization for the mathematicians that are here and the colleagues, when we all go back to our universities, that AI is really meant to be a copilot. And will it assist our scientific discovery? Absolutely. Just like a large language model can discover new openings in chess or new strategies in Go, imagine the possibilities when that kind of power is applied to our mathematical problems.
How AI Helped Reveal New Formulas for Primes [00:04:00]
Daria
And do you use AI in your research?
Ken
I do use AI in my research, and I use it in a number of ways. There are some rather boring ways that I use it. It’s an assistant to me. ChatGPT helps me with gaps I might have in the literature. As much as I read the papers, and as much as I think I’m on top of the literature, papers are coming out all the time. AI helps me with the gaps. But increasingly I’m using AI as a discovery tool. Is there phenomena out there that I’m just not aware of? And that’s happening all the time.
I wrote a paper last year where in a certain area of mathematics called partition theory, we discovered and then proved infinitely many expressions that detect the primes. Not that certain numbers are primes, but literally lists all primes in order without needing to know how to factorize numbers. And a few years ago I would have thought that was impossible, and the first hints of those formulas were discovered because computers said, I think there’s this pattern here of detecting primes. I remember just being floored by that.
The reason why this theorem on primes is important is largely based in history. There’s a famous problem called Hilbert’s 10th problem. Hilbert in 1900 assembled a list of about two dozen math problems that he laid out for the mathematicians of his future, namely mostly the 20th century. Some of those problems remain open to this day, such as the Riemann hypothesis. But the 10th problem was very innocuous. And it’s essentially, is there an oracle that can answer yes or no when given a Diophantine equation, it can ask yes, there will be integer points or no, there won’t be.
It wasn’t required to list the points for you, just say yes or no and it’s up to you to find the points if there are any. And in the 1970s, it was famously proven by logicians, largely based on the work of Yuri Matiyasevich, that no, there is no oracle, there is no computer algorithm out there that can answer whether or not a given Diophantine equation will have a solution. And as disappointing as that might be, that’s actually quite reaffirming for mathematics, because it shows how complicated the nature of numbers happens to be. But along the way, one of the consequences of Matiyasevich’s work is that there has to be a polynomial in some number of variables whose positive values, when you substitute in for the arguments, just positive integers gives you the set of prime numbers on the nose. One of these polynomials was found. And it’s really quite objectionable because it’s like 20 some variables, degree 20 something. So good luck trying to find the prime number 37 by substituting in the values.
So the work that I just described is very much along those lines, except where the polynomials are replaced by these functions coming from partitions. And unlike polynomials, they spit out the primes rapidly in order, not so rapidly that you have to worry about the security of the internet, make no mistake. But as a theoretical device, the computers help us discover an area of mathematics that I don’t think any of us thought would exist. So that’s a good partner. A good partner is one that is going to give you clues to conjectures that you can then either prove or build a body of work from.
The Dream of Proving the Riemann Hypothesis [00:07:40]
Daria
Since you mentioned the Riemann hypothesis, if you had to guess, will it be solved by a human, a human assisted by an AI, or by an AI?
Ken
I would like to see the Riemann hypothesis proven in my lifetime. I’ve spent quite a bit of time thinking about it, and my dream for the future is that yeah, maybe AI will help us find a pattern that will, if not prove the Riemann hypothesis, prove something that is close enough to it, so that all of the corollaries, all of its consequences can be established.
So, as you probably know, the Riemann hypothesis is often used to prove other theorems. So they’re conditional. There are probably 1000 papers written in the last 20 years, 25 years that say if the Riemann hypothesis is true, then we would know this, this, this, this, this and that. What isn’t so well known is that the Riemann hypothesis is like a sledgehammer. Many of those consequences could still be justified without having the full Riemann hypothesis. So my dream would be that some of those lower, but still very difficult open problems can be resolved because of patterns that are revealed to us. Can that happen? I don’t know, but maybe.
Fears and Guardrails for AI in Academia [00:09:00]
Daria
Do you have any worries when it comes to AI in the future?
Ken
Yes. And let me answer this very carefully. There was that famous Arnold Schwarzenegger film, Terminator: The Rise of the Machines, which I don’t know if you’ve seen it, but it was about the oncoming of AI and perhaps the end of humanity. Well, AI is everywhere around us, and if you work in the university like I do, the emergence of AI has essentially dominated my work at the University of Virginia, not my research, although I talked a little bit about that. I’m the STEM advisor to the university, and we have spent probably the last two and a half years in very high level discussions trying to prepare our university for the future. What do we want the university to look like ten years from now because of AI, the Internet of Things, and so on and so forth.
And so, yes, I’m deeply worried about what AI will do to universities, both positive and negative. I’m a member of the National Security Agency Advisory Board, and we are really worried about the future of AI, whether it’s misinformation or whether it’s losing control of basic infrastructure. I think there’s reason to be concerned about all of that.
So returning to what we are doing here today in Frontier Math, we’re not just asking questions of these systems. We are studying their responses. We are studying their transcripts. You can ask a computer a question, ask the same question five times and get five completely different transcripts, responses to the same question. Five years ago, I would have told you a computer is running a program. That program executes a certain number of lines and given a fixed input, will always do and carry out the same procedures. That’s not true here. And when you recognize that that’s not true and you don’t know why that is happening, then we should be worried.
How Education Must Adapt to AI [00:12:00]
Daria
Do you think the world as a whole, are we focusing enough on the societal impacts of AI that are about to come? Or should more people be thinking about this?
Ken
I’m an optimist. So my optimistic outlook is something like this. When the personal computer came out, a lot of industries should have died, but on the contrary, they were reinvented and usually within a few years ended up being far more productive than we could have possibly imagined. Will productivity go up? Yes. Will certain occupations be unnecessary? Oh, absolutely. And to be aware of that and to be prepared for that is what we should, I think, all be doing.
For us as instructors, we have to rethink the value of the exercises we give. We have to rethink the purpose of the projects or the research projects that we assign to our students. Because, you know, what we’re learning is that large chunks of that work are easily automated. And we have to think now about how important the skills that are gained from doing certain tasks truly are. So I think many of us that work in education will be forced to think very deeply about what and why we teach what we teach and adapt and plan for the future.
And this is happening. Earlier this month, the National Academy assembled a committee of about 20 educators and mathematicians to be the first mathematical sciences education board. Ravi Vakil, who is here, is on it. I’m on it. And as are many very distinguished educators and we will be thinking very deeply about what education from K through 12, through post-doctorate work across the sciences should be.
Daria
Now, I’d like to ask a few quick questions where the answers are meant to be like yes or no, or a number on a scale. So the first question is how long do you expect this benchmark, the problems put together today to withstand AI systems?
Ken
I still think most of the problems that are in tiers one through three are still too difficult for AI. So I think the jury is still out on them. The problems that we are writing, I hope the score is zero for quite some time. Will it be zero by 2030? I doubt that very much.
Daria
And how much do you expect AI to reshape math research where zero on a scale from 0 to 10, where zero is the level of a pocket calculator and then ten is math researchers are obsolete.
Ken
Well, I don’t think math researchers will ever be obsolete because an AI doesn’t know how to generate good questions. If that happens, well, that would be really a profound moment. I do know, and we probably use ChatGPT or Perplexity, and they offer natural questions as follow ups to a given transcript. That’s not what I’m talking about. Asking good questions that drive a theory takes skill. And I think we will always need people who can do that. And we don’t have many people who can do that. These are generally very special people. It’s on a scale of 0 to 10, probably seven. I think AI will be very important. Maybe eight.
What Makes a Great Mathematical Question [00:15:30]
Daria
A bit of a tangent here. How does one learn to write good questions? Can one learn this?
Ken
It usually comes from humility. What don’t I know? Here are the problems I wish I could solve. But I’m not lucky enough to solve any of them now. So as a mechanism to move forward in your research, well, what baby step question could I ask that might shed some light on the deeper questions that I really want to answer. And it’s very rarely the case that a difficult moonshot problem is solvable because you asked a question just the right way. Sometimes it happens, but it’s very rare. Usually there could be 30 intermediate questions that you might ask along the way over 20 or 30 years.
And earlier in this conversation, I intentionally used the word hallucination. I know it’s common for us to say, oh, the AI hallucinates. How dare it? And it’s like a horrible thing. I actually think it’s a good thing when I study these transcripts of AI trying to solve a difficult math problem. This is what it typically looks like. There could be a couple pages of oh my God, I can’t believe the computer knew this, or knew that, or recognized it. Made a mistake and went back earlier and redid a calculation. I feel like I’m talking to a graduate student.
But then maybe a couple pages later you’ll see a couple lines of Chinese and then a couple of lines later some words in French, but interspersed with actual formulas. It’s searching the literature for what it has recognized before. And, you know, I think that’s what we actually do when we do research. There’s this swimming around in uncertainty, being confused a lot. But then every once in a while you hallucinate. I wonder if such and such a thing is true. Maybe nine times out of ten it was just a dumb question. But even if it was the dumb question, you might figure out, oh, I asked a dumb question so I never have to go down that path again. You learn something from that, and every once in a while you hit on a question that you can’t rule out as being relevant, and maybe that will lead to a solution. Hallucinations are good. But it’s equally important that you know how to recognize when a hallucination is just false. And AI isn’t always great at that. But also to recognize, well, I can’t rule that out. Maybe I need to probe further.
Teaching by Making Deliberate Mistakes [00:18:00]
Daria
Do you ever deliberately make mistakes in your teaching?
Ken
It’s funny that you say that. Sometimes, yes. And I’ll give you a great example of this. At the University of Virginia, I teach a class called Math 3000, which is typically like your introduction to proofs class. And I tell the students from day one that there will be a session of class where I will spout off garbage and their homework will be to nail me on it and fix everything that I said wrong. And it’s super fun.
So let me give you an example. An easy example of this is the sum of the geometric series. One plus one half plus one fourth plus one eighth. If you sum up the reciprocals of the powers of two, it converges by the geometric series to the formula two. And you can give a false proof for that. Let S be the sum of one plus one half plus a fourth plus dot dot dot. So then two times two S rather would be what it would be two plus one. So you have that extra two. So well you can solve for S and get correctly that S equals two. But then I’d write down exactly the same argument. Let’s add up all the positive integers in order. And you can solve a similar algebra equation, but sum of the positive integers add up to infinity. And then I can go into, well, this is why I have to be very careful with infinite processes. So to answer your question, do I deliberately sometimes say false things? Absolutely.
Future-Proofing Higher Education [00:20:00]
Daria
Do you have advice for math students right now? Or what should or whether we should do something different or how we should use AI?
Ken
Maybe at UVA, part of my work is in the provost office, and as I mentioned earlier, we’ve been thinking very deeply about how to future proof higher education, or at least at the University of Virginia. And with the emergence of AI early on, it was how do we make sure our students are doing their homework? Two years ago, three years ago, we were talking about a company called Chegg. How do we know Chegg isn’t doing homework for our students? And now nobody talks about Chegg anymore because you can use a large language model. And at the outset, this was very important to us. How do we make sure that our students are learning to write? But we’ve adapted to that, right? You can certainly use AI tools to help students learn to write very easily if you give the right kind of prompts.
How do you write down proofs? Well, we can break down and maybe do some forensic analysis on correct versus true proofs very rapidly with an AI assistant. How do you think critically about the axioms you have to work with? Maybe you struggle to write a proof, but then you finally get a proof to work. But it’s wonky and you want to make a clean, compact argument. That’s another kind of editing. And yeah, I think AI will help us with that. We don’t talk very much about it yet, but I can easily see this will be important moving forward.
Will AI Reshape the World Like the Industrial Revolution? [00:22:30]
Daria
Thank you. How much will it reshape the world? Also, on a scale from zero, pocket calculator to ten, like industrial revolution level.
Ken
Yes, ten. I’m 57 years old. I’m the youngest of three. My two older brothers are older than me. And seeing them go through their schooling and then watching them start their careers, it’s kind of eye opening in terms of how technological advances have affected our lives. It went from there’s an electric typewriter, oh my gosh, to, oh, there’s something called a word processor to when I was by the time I was in college, I had a whole Macintosh computer that I get to bring to school. And now we have cell phones and so on and so forth. So, ten.
Daria
What is your outlook on AI progress from zero? You would stop AI entirely if you could just, like, erase the idea of inventing intelligent systems from humans minds. And then ten, accelerate as quickly as possible.
Ken
I do not believe that we should be developing AI without paying careful attention to the need for guardrails. And I can’t say that with confidence. I hope that I am wrong, but I think it would be naive to just plow on full speed ahead without some concerns. There are so many things in tech that are possible today that lead to misinformation, so definitely not ten, but certainly not one.
It’s interesting that you ask that question. We’re filming today in the home office of a company called Constellation. They are a growing and very important outfit. It’s a research institute, largely funded, I believe, by advocates for AI, but their mission is to support all initiatives that make AI safe. So as long as this company is called Constellation and outfits like it are given their due support and given a voice not to be just the rubber stamp, but if they are given ample support and have a seat at the table when policymakers are making decisions, then yeah, maybe seven.
Creativity, Neural Networks, and the Human Mind [00:23:30]
Ken
When we do our work in research, mathematics or just any scientific endeavor, we’re immersed in these problems for months, years at a time, and from time to time we have this moment and we don’t know where it comes from. And I have gone back through some of my works where I felt like I had those moments, but with a different pair of glasses. Right now, I thought, given what I’ve now seen in AI, I wonder if I had just gotten to the point where I did what the neural networks are doing, but just biologically.
And I finally made that connection. And, you know, a lot of the discoveries that I’ve made or people have said I’ve made probably actually fall in that category. Now, that’s not a negative thing, because the ability to make those connections I consider to be creative. But it was, but in many cases, it’s a different kind of creativity than I thought I was actually performing. Now that I think about it. And I don’t say that to say that this is humbling, which it is. But on the other hand, it’s also quite liberating because it says that if you’re genuinely curious and you’re learning all the time, then you can look forward to that moment where this will happen to you.
And I like that when I can say it wasn’t actually a moment, I just reached that threshold so that the neurons had exactly the right number of connections to make something happen. I feel a whole lot better about that, because then if you work on a problem for many months without making progress, you can still have faith that you’re making progress. You just haven’t yet reached the threshold. And yeah, from that perspective, I think it is quite liberating.