Sorry, but FunSearch probably isnt a milestone in scientific discovery
Over the last few days, you probably saw a bunch of enthusiastic news reports and tweets about a new Google Deepmind paper on math, like this at The Guardian:
and this
§
Well, yes, and no.
Google DeepMind, along with the mathematician Jordan Ellenberg really did use some AI to help solve a math problem, in a very clever paper that is worth reading, on how to use “program search with large language models” (as the accurate and not at all hypey title explains).
6
But then Google DeepMind’s PR team oversold the paper, claiming e.g., that they had “solved a notoriously hard challenge in computing” and ending with this exuberant but immensely speculative set of claims:
As clever as FunSearch is, it’s unlikely to be a major factor in solving cancer or making lightweight batteries.
In reality:
An LLM didn’t solve the math problem on its own; the LLM was used in a very narrow, prescribed way inside of larger system. (This is very different from the usual explain-your-complete-problem-in-English-and-get-an-answer.)
Human mathematicians had to write problem-specific code for each separate mathematical problem.
There is no evidence that the problem was heretofore “unsolvable”
“Going beyond human knowledge” is a bit oversold here.
The problem was not exactly the biggest outstanding problem in math
And the solution probably isn’t all that general.
It’s also hardly the first time AI is helped with mathematics.
NYU computer scientist Ernest Davis has just written a terrific new paper about all this, going into depth. Highly recommended.
Gary Marcus has been worrying aloud about hype in AI for decades.
ncG1vNJzZmifkafGrq3RnKysZqOqr7TAwJyiZ5ufonyxe9KoqauxXZfCtXnFrqWsnZGnsKl5z6umm5mSocZutdKnqw%3D%3D