AI can learn logic. But can it learn folklore knowledge? - Svetlana Jitomirskaya

Large language models can imitate reasoning steps and even verify formal proofs. But Svetlana Jitomirskaya argues they lack folklore knowledge: the implicit priors mathematicians build from experience

Large language models can imitate reasoning steps and even verify formal proofs.

But mathematical physicist Svetlana Jitomirskaya argues they lack folklore knowledge: the implicit priors mathematicians build from experience.

Humans can form intuition from a handful of examples; models still need far more data to see the same patterns.

Watch the interview here:

Transcript

Why folklore knowledge stumps AI [00:00:00]

Daria

Could you briefly introduce yourself, where you’re coming from, and what’s your field of study?

Svetlana

I’m Svetlana. I work at UC Berkeley and I study mathematical physics, spectral theory, equations in mathematical physics.

Daria

And what brings you here?

Svetlana

I know I am very fascinated by what AI can do in mathematics. I’m a little worried whether it will put us out of work. And I came here to try to find out what’s going on, the latest, and maybe contribute to this big thing.

Daria

Is your problem submission for this event also in mathematical physics?

Svetlana

It is related to this.

Daria

In broad strokes, could you describe the problem?

Svetlana

It is related to the interplay of number theory and some questions of spectral theory and mathematical physics. It has to do with some effects that happen at very large scales. So I think it is something that machines would need to really know—they would not be able to approximate, they would not be able to compute. So in order to solve my problem, a machine would really need to argue in a way that is not even written in any paper, but may be understood by some people in the field.

Daria

If you took an army of grad students and gave them this problem...

Svetlana

Oh yeah, this problem would be solved without that much difficulty. Maybe with some computation by a student knowledgeable in the subject. That is the point. It is not a very difficult problem for someone who has certain knowledge, but this knowledge is not widespread. This knowledge is not in the literature. This is a little bit of folklore knowledge.

“It tried to pretend it had solved it” [00:02:46]

Daria

And would you be surprised if this system was able to solve it?

Svetlana

I’ll be very surprised. We have just tested easier versions and it had no clue. Also, it tried to pretend it had solved it.

Daria

As they always do. When would you expect—if you had to guess, how long will your problem stand the test of time?

Svetlana

I hope forever? No, I’m not that optimistic. I am slowly coming to terms that AI may end up being smarter than every human in every sense. So that may happen.

Daria

Do you use it in your research at all?

Delegating lengthy arguments to AI [00:03:42]

Svetlana

I haven’t been using it much in my research, but because I so far haven’t used the strong versions. But just today I saw something that quite amazed me. I used Codel Mini High to help me create a problem. And there was some part that I was lazy to do in advance because it required lengthy computations and not just computations, but sort of arguments. And it did it momentarily. I was so impressed today.

Daria

Do you think you’ll use it more?

Svetlana

Yes, yes, yes. I’ve been using it a lot.

Automating routine math [00:05:05]

Daria

Do you expect the field of math to change?

Svetlana

Yes, yes, I expect this will lead to big improvements. This may lead to the fact that there will be fewer mathematicians. What will now probably happen is that you get a certain paper that is done by analogy, and you just ask ChatGPT and maybe it does it right. And then the paper is not very interesting. I think this will arrive. So the work of mathematicians will become more creative. There will be less time spent on routine, because we would be able to delegate it to the machines. And I think it will become even more exciting.

The 3-example problem of human creativity [00:05:50]

Daria

What does it take to get good at the creative parts of math, or do you think AI might learn that part of it?

Svetlana

When it learns it, I don’t know what will be left for humans. But so far, I don’t see it. So it’s still okay. At least the current language learning models—I was proven wrong. I always thought that they were just regurgitating, but they developed logic right now. And actually the idea is, in retrospect, quite simple. If you train them on summarizing things, they just extrapolate the key ideas. But how do you train them on being creative? I don’t know. I have no idea about human creativity, where it comes from. We can develop good intuition from 2 or 3 examples. We immediately see abstract patterns and not even patterns, but the ability to think abstractly. This kind of—it’s very hard for me to imagine how AI would try something that nobody has tried. And that’s creativity. But who knows?

The end of mathematical errors [00:07:28]

Daria

What are your biggest hopes and worries when it comes to AI in math and otherwise?

Svetlana

My biggest hope is that Lean is developed in such a way that all the math papers will be immediately verifiable, because it’s clear that most math papers have some errors and some of them can be minor, but some can be not so minor. And my biggest hope, and I think it’s not too far off in time, is that all the mathematics will be put in the verified database, this Lean database. And when you submit a paper to a journal, you need to prove that it was verified. And then at least it will remove a lot of work from the referees to verify correctness. Machine will verify that. Referees would only say whether it’s interesting or not. So I hope this period comes before the work of mathematicians becomes obsolete because of AI.

Sergey has been saying that Lean’s translator is a couple of years away, an automatic translator. And once it happens, because currently the biggest bottleneck is translating written texts into Lean. Translating a paper, an argument into Lean, it takes seven times the time to type it in LaTeX. Nobody wants to do it. But if there is an automatic translator, everything will be automatically translated into whatever is verifiable will be verified. Everything will change.

Is a superhuman proof still math? [00:09:33]

Daria

If AI were to prove something important, resolve the Riemann hypothesis, and then verified fully through Lean so that we all knew it’s correct, but then the proof would not be accessible to any human—is this still valuable? Is this math?

Svetlana

A different kind. There already exist computer-assisted proofs, which is similar. Humans cannot verify it without another machine. But we accept it. So this would be in a similar category. So it is math definitely. But computer-assisted proofs are in a different category than so-called proofs from the book, proofs that convey the beauty. Computer-assisted proofs are not quite right, even though they may have elements of conveying the beauty. It would be similar probably to this.

Daria

Do you expect the Riemann hypothesis to be resolved with the help of AI?

Svetlana

Not in my lifetime.

Daria

Do you have any advice for new math students in the age of AI?

Svetlana

I think it’s important to learn the fundamentals the old-fashioned way, but use it like people use computers for computer experimentation. You know what to experiment with, but computers can do it for you faster and with large numbers. So use it similarly. Make it your friend and helper, but not an enemy of your development.

Daria

How long do you expect until AI solves your problem?

Svetlana

More than ten years, maybe 20.

Daria

That’s the largest number I’ve heard so far. And then how much will AI change math research on a scale from 0 to 10, where zero is the level of a pocket calculator and ten is human mathematicians are absolutely obsolete?

Svetlana

Five.

Daria

And then how much will AI change the world where zero is once again the level of a pocket calculator and ten is the Industrial Revolution or even more?

Svetlana

No, I agree with ten.

We’ll still do math when AI is smarter [00:12:23]

Daria

And then if you have an extremely capable AI assistant that can do all the math, even the creative parts for you instantly, do you keep doing math?

Svetlana

That’s a tough one. So you see people keep playing chess, for example. So math may become something like sports, like chess. We are also running even though there are other species faster than us and machines and cars can go faster than us and bicycles, but we keep running. And you see, math is infinite. You can always create some model, some world where this will stay. But then once you formulated things, AI won’t be able to answer your questions. But maybe it will transform into formulating models and whatnot. I don’t know. It depends again. Whatever I can think of, ostensibly AI can also learn to do that. But it may keep adding levels to what it does. So if math will become creating new models, then maybe AI will learn to do that. And math may become something like creating models that create models or whatnot.

Daria

So what’s your outlook attitude towards AI progress where zero is you would stop it entirely if you could, and ten is accelerate at the fastest possible pace?

Svetlana

Of course there should be safeguards presumably. But maybe like eight. I’m an optimist. I believe in progress.