I'll share something as a former solar researcher.
Scientific progress is heavily influenced by how many bodies you can throw at a problem.
The more experiments you can run, with more variety and angles the more data you can get, the higher the likelihood of a breakthrough.
Several huge scientist are famous not because they are geniuses, but because they are great fundraisers and can have 20/30/50 bodies to throw at problems every year.
This is true in virtually any experimental field.
If LLMs can be de facto another body then scientific progress is going to sky rocket.
Robots also tend to be more precise than humans and could possibly lead to better replication.
But given that LLMs cannot interact with the real world I don't see that happening anytime soon.
fennecbutt 38 minutes ago [-]
>But given that LLMs cannot interact with the real world
Yes they can...VLAs exist.
bonoboTP 1 days ago [-]
> But given that LLMs cannot interact with the real world
What type of interaction do you envision? Could a non-domain-expert, but somewhat trained person provide a bridge? If the LLM comes up with the big ideas and tells a human technical assistant to execute (put the vial here, run the 3D printer with this file, put the object there, drive in a screw), would that help? But dexterous robots are getting more and more advanced, see CoRL demos right now.
epolanski 18 hours ago [-]
> If the LLM comes up with the big ideas and tells a human technical assistant to execute (put the vial here, run the 3D printer with this file, put the object there, drive in a screw), would that help?
No, because the bottleneck isn't the thinking but running experiments.
I worked in solar research, assembling a cell to test implied 40 different steps and from beginning to testing it was around 4 to 5 days.
This means that in one year working full time I will realistically run 40ish different experiments. Many of those will need to be done multiple times, and when you have 40 different steps that can go wrong and kill your efficiency this further compounds.
Thus realistically are running 5 to 10 different experiments (or better, a handful plus their variations).
At no point in this process you're like "yeah, if only LLMs could provide ideas", it's just not true, you get millions of ideas, time and bodies are the limit.
alphydan 14 hours ago [-]
Not to worry. With the millions unemployed by AGI, you can get hundreds of thousands of unskilled "hands" for your IA. Even with reasonable failure rates you will get a few hundred experimenters past the 40 steps.
Not sure if I should finish with /s or /fear or /uncertainty
_0ffh 16 hours ago [-]
In biochemistry there are multiple vendors that sell semi-to-fully automated setups that do large numbers of experiments in parallel.
I have no idea what solar research experimentation looks like in detail, is it theoretically possible to build similar setups for that use case? Where exactly is the bottleneck?
nosianu 15 hours ago [-]
This may be a problem of scale. biochem is a much wider field, and I guess, depending on what devices exactly you mean, a lot of it is usable in other bio fields (like https://www.faulhaber.com/en/markets/laboratory-automation/l... - the pictures showing a common general bio-lab scenario, there can be thousands of such assays to test)?
So it depends on if the same machinery can be used for more general material science research and testing work.
epolanski 8 hours ago [-]
Of course robotics can do a lot, it's process dependent.
kjkjadksj 22 hours ago [-]
Someone needs to evaluate the big ideas spat out by the llm is the big issue. Lab work can already be automated. And bs holders are even cheaper than an automated machine.
marbro 1 days ago [-]
Can these robots move a chess piece from one square to another?
I agree that there is power in numbers for science, but not all science is lab work. Sometimes the bottleneck is purely computational.
a_bonobo 1 days ago [-]
In my computational niche the bottleneck was always writing up the results :) And wow does AI help there.... It's not hard to get a decent first draft written by AI based on my existing results.
kjkjadksj 22 hours ago [-]
Computational can be a huge bottleneck. Some steps they really do take dozens of hours to run on cluster. And you are not the main character, others might be using the cluster and your jobs might be waiting in a queue. You might not be able to appreciate parameters need adjustment until the run is over and you evaluate output.
Another delay point is getting collaborators schedules to align for meetings on progress or potential directions.
Placing the results in context takes some time but not so much as you might guess if you are constantly reading and writing sourced paragraphs and skeleton papers needing only results plopped in when they are ready and some exposition in the discussion section.
Writing the code might be the fastest step in the process already.
joe_the_user 19 hours ago [-]
Seems you're burying the Lede in your post - yes, AIs aren't scientists.
What can be said about scientists and bodies is interesting but ultimately irrelevant.
Edit: I'd add that various LLMs/neural-nets have turned out to be great tools for research. I simply find the scientist-equivalent position problematic.
They set up scoreable computational science problems and do search over solutions.
ekianjo 20 hours ago [-]
> But given that LLMs cannot interact with the real world
Pair LLMs with machines and robotics and you are getting closer
the__alchemist 1 days ago [-]
I am reposting something along the lines of a flagged and dead comment: This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain. This is not a critique of his speech or points, but I think the lead implied by the (especially Youtube) title would hit harder if it came from someone whose work wasn't AI-centered.
Jumper's work is the poster child of AI success in science; this isn't about a new domain being revolutionized by it.
I will throw out an idea I've been thinking about recently about a far less ambitious idea, but related: Amber (MD package) provides Force Field names and partial charges for a number of small organic molecules in their GeoStd set. I believe these come from its Antechamber program. Would it be possible to infer useful FF name and Partial charge for arbitrary organic molecules using AI instead, trained on the GeoStd set data?
squigz 19 hours ago [-]
I don't understand how a scientist being awarded a Nobel prize in their field, using AI, does not add to AI's credibility as a useful tool?
mattkrause 10 hours ago [-]
It's a bit circular if the scientist's field (and prize) are for contributions to AI.
ants_everywhere 24 hours ago [-]
> This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain.
No it wouldn't. I've seen anti-AI people try to make this sort of argument repeatedly and it doesn't make any sense.
It's an attempt to smuggle in an ad-hominem. It's relying on the fact that people who hate AI also hate people who work in AI.
the__alchemist 22 hours ago [-]
Hey! I think I misused words, or was otherwise unclear. Sorry about that. I'm not Anti-AI and don't hate AI or people who work on it. I don't mean to conflate my post with someone who is Anti-AI. No ad-hominem intended.
ants_everywhere 21 hours ago [-]
Hey, sorry I don't mean to imply any of those things about you.
Before commenting I did check out your profile and saw that you weren't AI hater. My intention was the opposite -- I was trying to point out that some arguments don't survive on logic, but on emotional appeal. When enough smart people repeat something, it can slip past our skepticism. But of course my comment was terse and you had no way of knowing that.
In my view, among all people who make claims about what is on the forefront of science or knowledge, the most credible claims come from people whose research is on the forefront of science or knowledge. That's why the claims of experts are considered more credible in their fields than non-experts, even when the non-experts are experts in nearby fields.
So, applying that general rule to this particular case, we would expect the people to most credibly talk about the application of AI to science to be people who know the most about the application of AI to science.
There are other scientists whom I really respect but whose opinions on this particular topic I would find less weighty. For example, Terry Tao is a brilliant mathematician and has consulted on using AI to do research level math. But he was just a few months ago figuring out how to set up ChatGPT with VS Code, so I wouldn't expect him to be the most up to date on how AI is impacting science.
On the other hand, a Nobel laureate who has shown an aptitude for making important scientific discoveries is exactly the sort of person I believe would be able to talk knowledgeably about how to carve up science problems and about how to apply the current generation of AI to solve them. Especially if they've seen the internal world of a company that has a top tier foundational model and a track record for making scientific discoveries. Because in that case they see how science is being done if you have unlimited funds and access to some of the smartest scientists on the planet.
In contrast, I would be much more skeptical of a similar claim made by a company like OpenAI or Microsoft that doesn't have the same track record of producing new science.
For those reasons I don't think it's true that the claim would be more credible from someone with more distance (and hence less expertise). And I think similar claims made in other contexts would strike most people as bizarre. For example, if I said that claims about medicine are more credible if they come from people who aren't medical doctors.
bgwalter 23 hours ago [-]
It is not at all an ad hominem. Disclosure of interests and conflicts for interest were assumed to be declared in the open even a decade ago.
Carter sold his peanut farm to avoid conflicts of interest, Trump launched a coin pump & dump on his first day.
If a Nobel Prize winner works for a corporation, that should be disclosed (the original title contained "Nobel Prize Laureate" instead of "DeepMind Director").
But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
ants_everywhere 23 hours ago [-]
You are making a completely different point than the person I'm responding to. An equally bad point, but a different point nonetheless.
His title is in the video thumbnail.
> But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
Surely you agree this is a veiled ad hominem
NedF 1 days ago [-]
Awful title, great video.
Three points jumped out
1) "really when you look at these machine learning breakthroughs they're probably fewer people than you imagine"
In a world of idiots, few people can do great things.
2) External benchmarks forced people upstream to improve
We need more of these.
3) "the third of these ingredients research was worth a hundredfold of the first of these ingredients data."
Available data is 0 for most things.
hodgehog11 1 days ago [-]
> We need more of these.
> Available data is 0 for most things.
I would argue that we need an effective alternative to benchmarks entirely given how hard they are to obtain in scientific disciplines. Classical statistics has gone very far by getting a lot out of limited datasets, and train-test splits are absolutely unnecessary there.
bobmarleybiceps 23 hours ago [-]
I kind of dislike the benchmarkification of AI for science stuff tbh. I've encountered a LOT of issues with benchmark datasets that just aren't good...
In a lot of cases they are fine and necessary, but IMO, the standard for legit "success" in a lot of ML for science applications should basically be "can this model be used to make real scientific or engineering insights, that would have been very difficult and/or impossible without the proposed idea."
Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo
esjeon 13 hours ago [-]
Yeah the title was the worst and the most disgusting part, and I'm saying this in totally positive sense. As soon as you catch that "DeepMind" name there, you know it's really about science and a crafted model, not a wannabe general intelligence.
The cynists will comment that I've just been sucked in by the PR. However, I know this team and have been using these techniques for other problems. I know they are so close to a computationally-assisted proof of counterexample that it is virtually inevitable at this point. If they don't do it, I'm pretty sure I could take a handful of people and a few years and do it myself. Mostly a lot of interval arithmetic with a final application of Schauder that remains; tedious and time-consuming, but not overly challenging compared to the parts already done.
PontifexCipher 1 days ago [-]
This is not just PR and is very interesting. However, in my view, (and from a quick read of the paper) this is actually a very classical method in applied math work:
- Build a complex intractable mathematical model (here, Navier-Stokes)
- Approximate it with a function approximator (here, a Physics Informed Neural Network)
- Use the some property of function approximator to search for more solutions to the original model (here, using Gauss-Newton)
In a sense, this is actually just the process of model-based science anyway: use a model for the physical world and exploit the mathematics of the model for real-world effects.
This is very very good work, but this heritage goes back to polynomial approximation even from Taylor series, and has been the foundation of engineering for literal centuries. Throughout history, the approximator keeps getting better and better and hungrier and hungrier for data (Taylor series, Chebyshev + other orthogonal bases for polynomials, neural networks, RNNs, LSTMs, PINNs, <the future>).
You didn't say anything to the contrary, and neither did the original video, but it's very different than what some other people are talking about in this thread ("run an LLM in a loop to do science the way a person does it"). Maybe I'm just ranting at the overloading of the term AI to mean "anything on a GPU".
hodgehog11 1 days ago [-]
This is absolutely true, but it still makes use of the advantages and biases of neural networks in a clever way. It has to, because computationally-assisted proofs for PDEs with singularities is incredibly difficult. To me, this is not too similar from using them as heuristics to find counterexamples, or other approaches where the implicit biases pay off. I think we do ourselves a disservice to say that "LLMs replacing people" = "applications of AI in science".
I also wouldn't say this is entirely "classical". Old, yes, but still unfamiliar and controversial to a surprising number of people. But I get your point :-).
nybsjytm 1 days ago [-]
> I know they are so close to a computationally-assisted proof of counterexample that it is virtually inevitable at this point.
That's a strong claim. Is it based on more than the linked work on some model problems from fluid mechanics?
I will say that I dread the discourse if it works out, since I don't believe enough people will understand that using a PINN to get new solutions of differential equations has substantially no similarity to asking ChatGPT (or AlphaProof etc) for a proof of a conjecture. And there'll be a lot of people trying to hide the difference.
hodgehog11 24 hours ago [-]
It's based on knowledge of the related estimates, applying similar techniques to geometric problems, knowledge of all the prior works that lead to the current work, and speaking with members of the team themselves. They are much further along than it appears at first glance. All of the major bottlenecks have fallen; the only concern was whether double precision accuracy is good enough. The team seems to have estimates that are strong enough for this, but obviously keep them close to their chest.
PINNs are different in concept, yes, but clearly no less important, so the additional attention will be appreciated. Asking LLMs for proofs is a different vein of research, often involving Lean. It is much further behind, but still making ground.
nybsjytm 24 hours ago [-]
> PINNs are different in concept, yes, but clearly no less important
If anything I think they're more important! Whether or not it works out for Navier-Stokes, this kind of thing is an extremely plausible avenue of approach and could yield interesting singularities for other major equations. I am however extremely concerned about public understanding. I know you are well aware that this is worlds away from the speculative technologies like 'mathematical superintelligence' but, if it works out, it'll be like a nuclear bomb of misinformation about AI and math.
11101010001100 23 hours ago [-]
I think the PR is making it seem that Deepmind is not standing on the shoulder of giants, when in fact it very much is. The paper itself makes this clear. I wish them luck!
hodgehog11 18 hours ago [-]
To add to this, I think it is important to recognise that this is not fundamentally a Deepmind project; engineers from Deepmind came to help with the computational aspects to bring the error down after the lead and last authors brilliantly realised this approach would work with a proof of concept back in 2023. A good amount of work from Deepmind was involved, but I don't like the idea that they could get all the credit for this.
nybsjytm 7 hours ago [-]
> brilliantly realised
Can you say more about this? Nothing about this approach seems very amazing to me. Construct an approximate solution by some numerical method (in this case neural networks), prove that a solution which is close enough to satisfying the equation can be perturbed to an exact solution. Does the second half use some nonstandard method?
hodgehog11 1 hours ago [-]
This is one of those things that seems easy in retrospect, but wasn't particularly obvious at the time.
1. Proving existence to a differential equation using a numerical approximation is quite a bit more difficult than it seems at first. For Euler or NS, it seems almost absurd. Not only do you need a rather substantial amount of control over the linearisation, you need a way to rigorously control a posteriori error. This is easy for polynomials, but doing it for other models requires serious techniques that have only been created recently (rigorous quadrature via interval arithmetic).
2. Further to that, neural networks are far from an obvious choice. They are not exactly integrable and it is certainly not clear a priori that their biases would help to search for a blowup solution. In fact, I would have initially said it was a poor choice for the task. You also need to get it to a certain degree of precision and that's not easy.
3. The parameterisation of the self-similar solution turns out to be completely appropriate for a neural network. To be fair, solutions of this type have been considered before, so I'm willing to chalk this down to luck.
It's difficult to explain how challenging it is to fully derive a plan for a computationally-assisted proof of this magnitude unless you've tried it yourself on a new problem. At the end it seems completely obvious, but only after exhausting countless dead ends first.
rossant 21 hours ago [-]
They are building a formally-defined counter example to this? Am I understanding correctly?
> In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.
hodgehog11 18 hours ago [-]
They find a very good approximate blowup solution (non-smooth) using a NN. When the solution is good enough, you just need careful fixed point theory to infer an actual solution.
mlpoknbji 19 hours ago [-]
If it was a sure thing, why publish the paper they did? Why not just solve NS?
hodgehog11 18 hours ago [-]
It will take another few months at least, and the rest of the argument will comprise a fair few pages. But the hardest part is over.
When working toward a problem of this magnitude, it is natural to release papers stepwise to report progress toward the solution. Perelman did the same for the Poincare conjecture. Folks knew the problem was near a solution once the monotonicity proof of the W functional came out.
nybsjytm 7 hours ago [-]
> Folks knew the problem was near a solution once the monotonicity proof of the W functional came out.
This isn't true, it was a major accomplishment but by far the easiest part of Perelman's papers and not actually even part of the proof of the Poincaré conjecture.
hodgehog11 49 minutes ago [-]
Interesting, this was the big 'a-ha' moment in the grad course I took on the subject. Surgery was clearly important too, but this seemed to be more apparent from Hamilton's work. The W functional was what provided the control needed. Also, calling it 'easy' feels like it dismisses the insights necessary to have developed the functional in the first place.
Happy to be corrected on this; I wasn't an active mathematician at the time, so everything I know comes from other accounts.
Fraterkes 15 hours ago [-]
What would the implications of solving NS actually be? Would we get much more accurate weather predictions?
geordiew 15 hours ago [-]
I think it is fair to say that any blowing-up solution of navier stokes would be of purely intellectual interest. NS is a problem that many smart people have thought about and failed to solve. The current picture is that there are probably self-similar blowing up solutions that exist at extremely high codimension in the space of initial conditions. Thus they are "probability zero", and, if essentially never arise in nature. (A nice analogy: imagine a movie of a stone falling into a pond, and the ripples created afterwards. Now run the movie of the ripples backwards. We never see this in nature!)
TZubiri 14 hours ago [-]
Worth noting that deep mind is a world class pre-LLM team. And the techniques used are completely unrelated to LLMs
ants_everywhere 23 hours ago [-]
> The cynists will comment that I've just been sucked in by the PR
You can just ignore them. I see a lot of science-literate folks try to meet the anti-science folks as if they're on equal footing and it's almost always a waste of time. Imagine if every time you talked about biology you had to try to address the young earth creationists in the room and try to pre-rebut their concerns.
10 hours ago [-]
some_guy_nobel 1 days ago [-]
NVIDIA published the Illustrated Evo2 a few days ago, walking through the architecture of their genetics foundation model:
It's nice to see more and more labs using ai for drug discovery, something truly net positive for society.
hodgehog11 1 days ago [-]
As someone who works in the field, it really doesn't feel like more money (proportionally speaking) is going to this. A little bit is done here and there for PR. The number that are working on net positive applications for AI is still shockingly low compared to everything else.
tim333 11 hours ago [-]
On the same topic of AI helping scientific discovery there was this tweet yesterday
>...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections
>...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning...
Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?
_heimdall 11 hours ago [-]
If and when these tools become self-improving we have absolutely no idea what comes next.
Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.
cwmma 10 hours ago [-]
The architecture of the current models where learning is a separate and very expensive process make runaway self improvement seem like something that will require a bunch of breakthroughs in order to happen.
_heimdall 10 hours ago [-]
That seems reasonable, but I'm not sure if we really know yet. If current models are given direction and control to change how the next model is trained or architected it seems plausible that they could stumble into such a breakthrough.
The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.
tim333 1 hours ago [-]
I doubt there will be a sudden run away. Probably more of a gradual shift where humans and AI combine to improve the AI and gradually the AI does more of the work.
10 hours ago [-]
Inviz 24 hours ago [-]
I have a mildly psychotic friend who think that he uncovered the secrets to everything with AI. Quantum theory and Jungian archetypes, together with 4 dimensions - great mix
cantor_S_drug 20 hours ago [-]
Let's say the "Secrets of the Universe" broadly consists of Graph of 100 "abstract" interconnected concepts. The concepts have to be abstract because it is describing everything. It has to be limited in number because we cannot be endlessly chasing the definitions till we reach the levels of atoms. Is it possible to get glimpse of that Graph just toying with abstract ideas. The exact nodes / concepts used in the graph maybe different (depending on field) but the structure will be isomorphic. It has to be discoverable in any field since we started with the assumption that the Graph is "Secret of the Universe" so it should apply to any subset as well and should be discoverable from that subset. This is like analytic functions where knowing its derivatives in a small enough interval can lead us to the exact function.
Hilift 16 hours ago [-]
AI really is good at finding new viruses, due to simple DNA sequences look like noise to humans. But then you may have created a different problem.
jgalt212 1 days ago [-]
I see this sort of work as a natural extension of Combinatorial Chemistry or bootstrapping and Monte Carlo methods in stats.
First jump that computers gave us : speed. With excess of speed came the ability to brute force many problems.
Next jump given by AI (not LLMs specifically, I mean “machine learned systems” in general) is navigation. Even with large amounts of speed some problems are still impractically large, we are using AI to better explore that space, by navigating it smarter, rather than just speeding through it combinatorially.
whatever1 1 days ago [-]
No evidence so far that "AI" has improved our general optimization capabilities. At all.
Still at the top of the benchmarks of integer optimization by huge margin are the traditional usual suspects. Same in constraint programming and SAT.
Not published just yet are experiments for finding solutions to mathematical problems traditionally found with SAT solvers, at much larger scale than was previously possible.
whatever1 1 days ago [-]
This is just meta heuristic relying on local search :facepalm:
You could call it artificial ant colony optimization.
People come up with such ideas all the time. Sorry, but nothing groundbreaking here.
hodgehog11 1 days ago [-]
Um, okay? Isn't that how most optimisation involving AI is supposed to go?
Perhaps some much needed context. Mathematicians are not stupid; we are very much aware of all the existing forms of genetic optimisation algorithms, cross entropy method, etc. Nothing works on these problems, or at least not well at scale. As I said, state of the art for many of these was SAT-related. The problem is that the heuristic used for exploring new solutions always required very careful consideration as the naive ones rarely worked well.
Here, the transformer is proving effective at searching for good heuristics, far more so than any other existing technique. In this sense, it is achieving far, far. far better performance in optimisation than previous approaches. That is a breakthrough, at least for us mathematicians. If this doesn't constitute improvements in optimisation, I don't know what does.
Saying it's "just meta heuristic relying on local search" is akin to saying these tasks are "just optimisation". If it's so procedural, why weren't we making ground on these things before?
Also, by the way, a :facepalm: is not exactly the pinnacle of academic rebuttal, no matter how wrong I could have been.
whatever1 23 hours ago [-]
Apologies I didn’t mean to be cocky / dismissive.
It’s just that the paper cited is no different than any other paper in the meta-heuristic community.
Some idea for guiding the local search. Some limited sample results. No promises on bounds or generalizability of the method.
If this is ground breaking, then every legitimate meta heuristic paper in the past 50 years was also ground breaking.
I will change my mind if I see a wide set of benchmark results where it consistently beats or is even head-to-head with the SoTA. Then we would know that we have a game changer.
hodgehog11 21 hours ago [-]
So for reference, I am a mathematician, I work in probability theory. I don't work in optimisation theory, but saw a talk a few days ago by a prominent member in that field ranting about the obsession with pointless bounds when industry does not care, Gurobi team does not care, and for OR, NN-centric heuristics coupled with massive GPU compute are eating the lunch of academia. But I digress.
The key difference is that this one approach (and the improvements that follow up on it) seems to be generating counterexamples and sharper estimates at such a rapid speed that they simply can't publish them fast enough. It almost seems pointless to do so given how easy they now are to find. Mathematicians are not stupid, we know about meta-heuristics. It takes forever to get a single result with existing techniques, so long that each result is often its own paper. Getting everything in the PatternBoost paper took a few months, as I understand. Local search is problem-specific, but the point is that the NN is doing a lot of heavy-lifting so the choice of local search is not as critical.
Here is a frivolous example mentioned in the paper itself: largest subset of a 2D lattice with no embedded isosceles triangle. For larger lattices (I don't remember the exact size under consideration), SoTA are SAT solvers which work up to n=32. Previous meta-heuristics cannot even do this, of course, you can try if you don't believe me. PatternBoost has just recently gotten to ~96% of the expected size for n=64 and is still improving by the day. Once it reaches 100%, there are techniques to show local optimality. Does this work as a benchmark? There are plenty more in there and unpublished.
For more serious cases that are difficult to explain here, the group in Sydney have counterexamples to hundreds of graph-theory and rep. theory conjectures that have stood for many decades. I also disagree on the "no different" aspect; Geordie Williamson is a very strong mathematician, and does not tend to jump on trivial things. He is very receptive to discussion on these matters, so you can ask him yourself how this is actually a game-changer downstream.
Yes, it is a meta-heuristic. But almost all meta-heuristics have been useless so far for these problems. This one is not, and for the people downstream, that's really all that matters.
lomase 1 days ago [-]
If you only know how to use a hammer, everything looks like a nail.
whatever1 1 days ago [-]
Combinatorics will always be a (tough) nail regardless of what a random HN commentator thinks.
Bring any tool you wish, but the problem is very well defined and very real.
malux85 5 hours ago [-]
> No evidence so far that "AI" has improved our general optimization capabilities. At all.
Uh, ok, I didn't claim that. At All.
Deep Learning machine learned features have definitely helped us (meaning my company) over hand engineered features, allowing us to navigate our problem space significantly faster
bgwalter 1 days ago [-]
[flagged]
signatoremo 23 hours ago [-]
> Literally no one who is not connected to the "AI" industrial complex praises "AI".
Why lying when one can easily find examples of exactly that?
Scientific progress is heavily influenced by how many bodies you can throw at a problem.
The more experiments you can run, with more variety and angles the more data you can get, the higher the likelihood of a breakthrough.
Several huge scientist are famous not because they are geniuses, but because they are great fundraisers and can have 20/30/50 bodies to throw at problems every year.
This is true in virtually any experimental field.
If LLMs can be de facto another body then scientific progress is going to sky rocket.
Robots also tend to be more precise than humans and could possibly lead to better replication.
But given that LLMs cannot interact with the real world I don't see that happening anytime soon.
Yes they can...VLAs exist.
What type of interaction do you envision? Could a non-domain-expert, but somewhat trained person provide a bridge? If the LLM comes up with the big ideas and tells a human technical assistant to execute (put the vial here, run the 3D printer with this file, put the object there, drive in a screw), would that help? But dexterous robots are getting more and more advanced, see CoRL demos right now.
No, because the bottleneck isn't the thinking but running experiments.
I worked in solar research, assembling a cell to test implied 40 different steps and from beginning to testing it was around 4 to 5 days.
This means that in one year working full time I will realistically run 40ish different experiments. Many of those will need to be done multiple times, and when you have 40 different steps that can go wrong and kill your efficiency this further compounds.
Thus realistically are running 5 to 10 different experiments (or better, a handful plus their variations).
At no point in this process you're like "yeah, if only LLMs could provide ideas", it's just not true, you get millions of ideas, time and bodies are the limit.
Not sure if I should finish with /s or /fear or /uncertainty
I have no idea what solar research experimentation looks like in detail, is it theoretically possible to build similar setups for that use case? Where exactly is the bottleneck?
So it depends on if the same machinery can be used for more general material science research and testing work.
Another delay point is getting collaborators schedules to align for meetings on progress or potential directions.
Placing the results in context takes some time but not so much as you might guess if you are constantly reading and writing sourced paragraphs and skeleton papers needing only results plopped in when they are ready and some exposition in the discussion section.
Writing the code might be the fastest step in the process already.
What can be said about scientists and bodies is interesting but ultimately irrelevant.
Edit: I'd add that various LLMs/neural-nets have turned out to be great tools for research. I simply find the scientist-equivalent position problematic.
https://arxiv.org/abs/2509.06503
They set up scoreable computational science problems and do search over solutions.
Pair LLMs with machines and robotics and you are getting closer
Jumper's work is the poster child of AI success in science; this isn't about a new domain being revolutionized by it.
I will throw out an idea I've been thinking about recently about a far less ambitious idea, but related: Amber (MD package) provides Force Field names and partial charges for a number of small organic molecules in their GeoStd set. I believe these come from its Antechamber program. Would it be possible to infer useful FF name and Partial charge for arbitrary organic molecules using AI instead, trained on the GeoStd set data?
No it wouldn't. I've seen anti-AI people try to make this sort of argument repeatedly and it doesn't make any sense.
It's an attempt to smuggle in an ad-hominem. It's relying on the fact that people who hate AI also hate people who work in AI.
Before commenting I did check out your profile and saw that you weren't AI hater. My intention was the opposite -- I was trying to point out that some arguments don't survive on logic, but on emotional appeal. When enough smart people repeat something, it can slip past our skepticism. But of course my comment was terse and you had no way of knowing that.
In my view, among all people who make claims about what is on the forefront of science or knowledge, the most credible claims come from people whose research is on the forefront of science or knowledge. That's why the claims of experts are considered more credible in their fields than non-experts, even when the non-experts are experts in nearby fields.
So, applying that general rule to this particular case, we would expect the people to most credibly talk about the application of AI to science to be people who know the most about the application of AI to science.
There are other scientists whom I really respect but whose opinions on this particular topic I would find less weighty. For example, Terry Tao is a brilliant mathematician and has consulted on using AI to do research level math. But he was just a few months ago figuring out how to set up ChatGPT with VS Code, so I wouldn't expect him to be the most up to date on how AI is impacting science.
On the other hand, a Nobel laureate who has shown an aptitude for making important scientific discoveries is exactly the sort of person I believe would be able to talk knowledgeably about how to carve up science problems and about how to apply the current generation of AI to solve them. Especially if they've seen the internal world of a company that has a top tier foundational model and a track record for making scientific discoveries. Because in that case they see how science is being done if you have unlimited funds and access to some of the smartest scientists on the planet.
In contrast, I would be much more skeptical of a similar claim made by a company like OpenAI or Microsoft that doesn't have the same track record of producing new science.
For those reasons I don't think it's true that the claim would be more credible from someone with more distance (and hence less expertise). And I think similar claims made in other contexts would strike most people as bizarre. For example, if I said that claims about medicine are more credible if they come from people who aren't medical doctors.
Carter sold his peanut farm to avoid conflicts of interest, Trump launched a coin pump & dump on his first day.
If a Nobel Prize winner works for a corporation, that should be disclosed (the original title contained "Nobel Prize Laureate" instead of "DeepMind Director").
But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
His title is in the video thumbnail.
> But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
Surely you agree this is a veiled ad hominem
Three points jumped out
1) "really when you look at these machine learning breakthroughs they're probably fewer people than you imagine"
In a world of idiots, few people can do great things.
2) External benchmarks forced people upstream to improve
We need more of these.
3) "the third of these ingredients research was worth a hundredfold of the first of these ingredients data."
Available data is 0 for most things.
> Available data is 0 for most things.
I would argue that we need an effective alternative to benchmarks entirely given how hard they are to obtain in scientific disciplines. Classical statistics has gone very far by getting a lot out of limited datasets, and train-test splits are absolutely unnecessary there.
Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo
The cynists will comment that I've just been sucked in by the PR. However, I know this team and have been using these techniques for other problems. I know they are so close to a computationally-assisted proof of counterexample that it is virtually inevitable at this point. If they don't do it, I'm pretty sure I could take a handful of people and a few years and do it myself. Mostly a lot of interval arithmetic with a final application of Schauder that remains; tedious and time-consuming, but not overly challenging compared to the parts already done.
- Build a complex intractable mathematical model (here, Navier-Stokes)
- Approximate it with a function approximator (here, a Physics Informed Neural Network)
- Use the some property of function approximator to search for more solutions to the original model (here, using Gauss-Newton)
In a sense, this is actually just the process of model-based science anyway: use a model for the physical world and exploit the mathematics of the model for real-world effects.
This is very very good work, but this heritage goes back to polynomial approximation even from Taylor series, and has been the foundation of engineering for literal centuries. Throughout history, the approximator keeps getting better and better and hungrier and hungrier for data (Taylor series, Chebyshev + other orthogonal bases for polynomials, neural networks, RNNs, LSTMs, PINNs, <the future>).
You didn't say anything to the contrary, and neither did the original video, but it's very different than what some other people are talking about in this thread ("run an LLM in a loop to do science the way a person does it"). Maybe I'm just ranting at the overloading of the term AI to mean "anything on a GPU".
I also wouldn't say this is entirely "classical". Old, yes, but still unfamiliar and controversial to a surprising number of people. But I get your point :-).
That's a strong claim. Is it based on more than the linked work on some model problems from fluid mechanics?
I will say that I dread the discourse if it works out, since I don't believe enough people will understand that using a PINN to get new solutions of differential equations has substantially no similarity to asking ChatGPT (or AlphaProof etc) for a proof of a conjecture. And there'll be a lot of people trying to hide the difference.
PINNs are different in concept, yes, but clearly no less important, so the additional attention will be appreciated. Asking LLMs for proofs is a different vein of research, often involving Lean. It is much further behind, but still making ground.
If anything I think they're more important! Whether or not it works out for Navier-Stokes, this kind of thing is an extremely plausible avenue of approach and could yield interesting singularities for other major equations. I am however extremely concerned about public understanding. I know you are well aware that this is worlds away from the speculative technologies like 'mathematical superintelligence' but, if it works out, it'll be like a nuclear bomb of misinformation about AI and math.
Can you say more about this? Nothing about this approach seems very amazing to me. Construct an approximate solution by some numerical method (in this case neural networks), prove that a solution which is close enough to satisfying the equation can be perturbed to an exact solution. Does the second half use some nonstandard method?
1. Proving existence to a differential equation using a numerical approximation is quite a bit more difficult than it seems at first. For Euler or NS, it seems almost absurd. Not only do you need a rather substantial amount of control over the linearisation, you need a way to rigorously control a posteriori error. This is easy for polynomials, but doing it for other models requires serious techniques that have only been created recently (rigorous quadrature via interval arithmetic).
2. Further to that, neural networks are far from an obvious choice. They are not exactly integrable and it is certainly not clear a priori that their biases would help to search for a blowup solution. In fact, I would have initially said it was a poor choice for the task. You also need to get it to a certain degree of precision and that's not easy.
3. The parameterisation of the self-similar solution turns out to be completely appropriate for a neural network. To be fair, solutions of this type have been considered before, so I'm willing to chalk this down to luck.
It's difficult to explain how challenging it is to fully derive a plan for a computationally-assisted proof of this magnitude unless you've tried it yourself on a new problem. At the end it seems completely obvious, but only after exhausting countless dead ends first.
> In three space dimensions and time, given an initial velocity field, there exists a vector velocity and a scalar pressure field, which are both smooth and globally defined, that solve the Navier–Stokes equations.
When working toward a problem of this magnitude, it is natural to release papers stepwise to report progress toward the solution. Perelman did the same for the Poincare conjecture. Folks knew the problem was near a solution once the monotonicity proof of the W functional came out.
This isn't true, it was a major accomplishment but by far the easiest part of Perelman's papers and not actually even part of the proof of the Poincaré conjecture.
Happy to be corrected on this; I wasn't an active mathematician at the time, so everything I know comes from other accounts.
You can just ignore them. I see a lot of science-literate folks try to meet the anti-science folks as if they're on equal footing and it's almost always a waste of time. Imagine if every time you talked about biology you had to try to address the young earth creationists in the room and try to pre-rebut their concerns.
https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/
It's nice to see more and more labs using ai for drug discovery, something truly net positive for society.
https://x.com/DeryaTR_/status/1972115494787338484
>...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections
>...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning...
Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?
Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.
The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.
https://en.wikipedia.org/wiki/Combinatorial_chemistry
Next jump given by AI (not LLMs specifically, I mean “machine learned systems” in general) is navigation. Even with large amounts of speed some problems are still impractically large, we are using AI to better explore that space, by navigating it smarter, rather than just speeding through it combinatorially.
Still at the top of the benchmarks of integer optimization by huge margin are the traditional usual suspects. Same in constraint programming and SAT.
Not published just yet are experiments for finding solutions to mathematical problems traditionally found with SAT solvers, at much larger scale than was previously possible.
You could call it artificial ant colony optimization.
People come up with such ideas all the time. Sorry, but nothing groundbreaking here.
Perhaps some much needed context. Mathematicians are not stupid; we are very much aware of all the existing forms of genetic optimisation algorithms, cross entropy method, etc. Nothing works on these problems, or at least not well at scale. As I said, state of the art for many of these was SAT-related. The problem is that the heuristic used for exploring new solutions always required very careful consideration as the naive ones rarely worked well.
Here, the transformer is proving effective at searching for good heuristics, far more so than any other existing technique. In this sense, it is achieving far, far. far better performance in optimisation than previous approaches. That is a breakthrough, at least for us mathematicians. If this doesn't constitute improvements in optimisation, I don't know what does.
Saying it's "just meta heuristic relying on local search" is akin to saying these tasks are "just optimisation". If it's so procedural, why weren't we making ground on these things before?
Also, by the way, a :facepalm: is not exactly the pinnacle of academic rebuttal, no matter how wrong I could have been.
It’s just that the paper cited is no different than any other paper in the meta-heuristic community.
Some idea for guiding the local search. Some limited sample results. No promises on bounds or generalizability of the method.
If this is ground breaking, then every legitimate meta heuristic paper in the past 50 years was also ground breaking.
I will change my mind if I see a wide set of benchmark results where it consistently beats or is even head-to-head with the SoTA. Then we would know that we have a game changer.
The key difference is that this one approach (and the improvements that follow up on it) seems to be generating counterexamples and sharper estimates at such a rapid speed that they simply can't publish them fast enough. It almost seems pointless to do so given how easy they now are to find. Mathematicians are not stupid, we know about meta-heuristics. It takes forever to get a single result with existing techniques, so long that each result is often its own paper. Getting everything in the PatternBoost paper took a few months, as I understand. Local search is problem-specific, but the point is that the NN is doing a lot of heavy-lifting so the choice of local search is not as critical.
Here is a frivolous example mentioned in the paper itself: largest subset of a 2D lattice with no embedded isosceles triangle. For larger lattices (I don't remember the exact size under consideration), SoTA are SAT solvers which work up to n=32. Previous meta-heuristics cannot even do this, of course, you can try if you don't believe me. PatternBoost has just recently gotten to ~96% of the expected size for n=64 and is still improving by the day. Once it reaches 100%, there are techniques to show local optimality. Does this work as a benchmark? There are plenty more in there and unpublished.
For more serious cases that are difficult to explain here, the group in Sydney have counterexamples to hundreds of graph-theory and rep. theory conjectures that have stood for many decades. I also disagree on the "no different" aspect; Geordie Williamson is a very strong mathematician, and does not tend to jump on trivial things. He is very receptive to discussion on these matters, so you can ask him yourself how this is actually a game-changer downstream.
Yes, it is a meta-heuristic. But almost all meta-heuristics have been useless so far for these problems. This one is not, and for the people downstream, that's really all that matters.
Bring any tool you wish, but the problem is very well defined and very real.
Uh, ok, I didn't claim that. At All.
Deep Learning machine learned features have definitely helped us (meaning my company) over hand engineered features, allowing us to navigate our problem space significantly faster
Why lying when one can easily find examples of exactly that?
1) https://www.ecmwf.int/en/about/media-centre/news/2025/ecmwfs...
2) https://pmc.ncbi.nlm.nih.gov/articles/PMC11510778/
But I’m sure you could find the connections to the AI industry complex somehow.
But the important part is that the funding for "AI" skeptics is practically zero, whereas the funding for "AI" boosters is basically unlimited.
Many people have tried, many people have been let down.
https://news.ycombinator.com/newsguidelines.html
https://www.youtube.com/@ycombinator