great to read your blog, quite a pretty job you are doing!
The issue you raise about machine learning is really true. When I first heard about machine learning and physics, I thought that by definition it is the very contrary of science. Science is about using our intelligence to nail down some specific observations to a simple model that starts from a given fact and gets to the observation. For instance: I could train my machine-learning algorithm by observing the tides of several see locations around the world and probably at the end it would be able to predict the tides of a given uncharted location. This might be great from an engineering viewpoint: as a scientist, I think that we understand more about tides once we make the connection with the Moon, the Sun, the Newtonian gravity and the laws of water flow. And we will not be significantly wiser, without understanding that!
However, the more I think about it, the more I have the impression that machine learning is just another numerical tool. We can use it to guide our intuition, but at the end we need to prove the results it gives us, at least partially. I do my research activity in condensed-matter, and I’m fine if machine learning tells to my colleague (I don’t do that) that in a condensed-matter model there are three very exotic quantum phases of matter, but then he (we?) should be able to understand those phases and link them with pen and paper to the original model. I feel that only at that stage we have produced knowledge. I feel that only at that stage we have identified the key properties of the model, and that somebody could use them to see what happens when I perturb the model by adding, for instance, an electromagnetic field. Many people might be happy with the machine-learning result, I feel it is just a hint.
I have an observation concerning the point where you say: “At its core, science has always been about convincing people. The rigorous web of logical deductions that constitutes the backbone of math and of all exact sciences, is –ultimately- a rhetorical device.” I have the impression that here you are biased by your research on black holes in complex astrophysical objects. I think that science is about producing models that are able to predict the behavior of nature under certain conditions. It’s not just about rhetoric, it’s also about numbers and data!
Ciao Leo, machine learning is a big big field. I am actually surprised that you did not run into it on your job. Applications are countless and it is spreading everywhere. There is also lots of multidisciplinary work at the interface with physics. There are models than can learn systems of differential equations describing data, automatically https://www.pnas.org/doi/10.1073/pnas.1517384113. Neural nets that can learn hamiltonians https://arxiv.org/abs/1906.01563. And even if you have a very strong theoretical framework you still can do Bayesian inference on your parameters (that counts as machine learning too). Unless you are very very pen and paper you can run, but you cannot hide :-D
In my field, I've seen crazy crap as well as cool stuff done with machine learning (as well as stuff where machine learning was used because it was cooler than any another technique and would bring more citations).
I imagine that in astrophysics you now have tons of data on celestial objects and since you have to look for regularities you are trying to find them with machine learning. It sounds plausible and interesting, it will probably guide you more effectively than other methods.
In the field of out-of-equilibrium dynamics for quantum systems (among the things I do), I have seen people using machine learning to predict the time evolution of an observable at a precision that any algorithm we had in hand 10 years ago could handle. Obviously, I don’t find this particularly interesting.
I’ve also read papers about the use of machine learning to understand which effective hydrodynamic equations are satisfied by a specific quantum model; it sounds cool, but if you go and look in detail it doesn't seem to me that in the end we know more than before.
It's not a question of being pen & papers or wanting to hide from the present, it's also a question of evaluating what is being done in terms of the production of new interesting knowledge.
Indeed in astronomy -at least for what I can see from my galactic astronomy point of view- I did not witness any major science breakthrough due to machine learning, yet. By science breakthrough I mean learning something new and important on some actual physical system out there. I have seen breakthroughs in data analysis and modeling by orders of magnitude in speed, for instance https://ui.adsabs.harvard.edu/abs/2017Natur.548..555H/abstract ; attempts at having the ML model ask questions (implicitly, by finding anomalous objects) such as this one https://ui.adsabs.harvard.edu/abs/2017MNRAS.465.4530B/abstract ; and lots of proofs-of-concept of potentially cool ideas which are in need of serious work to actually be viable (most of my papers). Here is a short review: https://arxiv.org/pdf/2212.01493.pdf. But yeah, nothing I would describe as a core science breakthrough. So the situation is not much dissimilar to what you describe in your field. Is this because we are just at the beginning, or is this a major limitation of the method? I do not know. It would be an interesting comparison to take a look at the literature a few decades ago when computer simulations started to become widespread to check whether we are following a similar pattern of adoption. Did the breakthroughs from computer simulations lag widespread adoption by a few years? Can we expect the same to happen with machine learning?
I also suspect that there is an astronomy specific effect: lots of discoveries in astronomy were enabled by the tedious collection of large volumes of seemingly uninteresting data (e.g. stellar spectra) coupled to classification tasks that had to rely on human judgment (again, stellar spectra). Now this is increasingly being outsourced to ML models, sometimes trained on crowdsourced data (did you hear of galaxyzoo?).
Also to keep in mind is the fact that people do not feel allowed to conclude new results from new methods until the method is widely accepted by the community. Thus most ML-in-astro papers confirm what we already knew, because we are still in the stage where people want to show that these methods work but are wary to go out on a limb and say that the new method solved some controversial problem.
concerning computer simulations, I do not think there was ever a sharp discontinuity from a scientific community that did not use them to one that used them massively (as we are currently doing). I think they became more and more common, as well as computers got more and more powerful… but all of this was slow and continuous. Just one day you look back and you realise that ten or twenty years before things were done differently :)
At the end the use of computers in science is probably a true methodological revolution that took place silently.
This is in principle quantifiable. Whether machine learning adoption and capabilities evolve slowly (like numerical simulations) or abruptly is a question of great interest for forecasting.
Hi Mario,
great to read your blog, quite a pretty job you are doing!
The issue you raise about machine learning is really true. When I first heard about machine learning and physics, I thought that by definition it is the very contrary of science. Science is about using our intelligence to nail down some specific observations to a simple model that starts from a given fact and gets to the observation. For instance: I could train my machine-learning algorithm by observing the tides of several see locations around the world and probably at the end it would be able to predict the tides of a given uncharted location. This might be great from an engineering viewpoint: as a scientist, I think that we understand more about tides once we make the connection with the Moon, the Sun, the Newtonian gravity and the laws of water flow. And we will not be significantly wiser, without understanding that!
However, the more I think about it, the more I have the impression that machine learning is just another numerical tool. We can use it to guide our intuition, but at the end we need to prove the results it gives us, at least partially. I do my research activity in condensed-matter, and I’m fine if machine learning tells to my colleague (I don’t do that) that in a condensed-matter model there are three very exotic quantum phases of matter, but then he (we?) should be able to understand those phases and link them with pen and paper to the original model. I feel that only at that stage we have produced knowledge. I feel that only at that stage we have identified the key properties of the model, and that somebody could use them to see what happens when I perturb the model by adding, for instance, an electromagnetic field. Many people might be happy with the machine-learning result, I feel it is just a hint.
I have an observation concerning the point where you say: “At its core, science has always been about convincing people. The rigorous web of logical deductions that constitutes the backbone of math and of all exact sciences, is –ultimately- a rhetorical device.” I have the impression that here you are biased by your research on black holes in complex astrophysical objects. I think that science is about producing models that are able to predict the behavior of nature under certain conditions. It’s not just about rhetoric, it’s also about numbers and data!
Leonardo
Ciao Leo, machine learning is a big big field. I am actually surprised that you did not run into it on your job. Applications are countless and it is spreading everywhere. There is also lots of multidisciplinary work at the interface with physics. There are models than can learn systems of differential equations describing data, automatically https://www.pnas.org/doi/10.1073/pnas.1517384113. Neural nets that can learn hamiltonians https://arxiv.org/abs/1906.01563. And even if you have a very strong theoretical framework you still can do Bayesian inference on your parameters (that counts as machine learning too). Unless you are very very pen and paper you can run, but you cannot hide :-D
Hi Mario,
I attend one machine-learning seminar a week...
In my field, I've seen crazy crap as well as cool stuff done with machine learning (as well as stuff where machine learning was used because it was cooler than any another technique and would bring more citations).
I imagine that in astrophysics you now have tons of data on celestial objects and since you have to look for regularities you are trying to find them with machine learning. It sounds plausible and interesting, it will probably guide you more effectively than other methods.
In the field of out-of-equilibrium dynamics for quantum systems (among the things I do), I have seen people using machine learning to predict the time evolution of an observable at a precision that any algorithm we had in hand 10 years ago could handle. Obviously, I don’t find this particularly interesting.
I’ve also read papers about the use of machine learning to understand which effective hydrodynamic equations are satisfied by a specific quantum model; it sounds cool, but if you go and look in detail it doesn't seem to me that in the end we know more than before.
It's not a question of being pen & papers or wanting to hide from the present, it's also a question of evaluating what is being done in terms of the production of new interesting knowledge.
Indeed in astronomy -at least for what I can see from my galactic astronomy point of view- I did not witness any major science breakthrough due to machine learning, yet. By science breakthrough I mean learning something new and important on some actual physical system out there. I have seen breakthroughs in data analysis and modeling by orders of magnitude in speed, for instance https://ui.adsabs.harvard.edu/abs/2017Natur.548..555H/abstract ; attempts at having the ML model ask questions (implicitly, by finding anomalous objects) such as this one https://ui.adsabs.harvard.edu/abs/2017MNRAS.465.4530B/abstract ; and lots of proofs-of-concept of potentially cool ideas which are in need of serious work to actually be viable (most of my papers). Here is a short review: https://arxiv.org/pdf/2212.01493.pdf. But yeah, nothing I would describe as a core science breakthrough. So the situation is not much dissimilar to what you describe in your field. Is this because we are just at the beginning, or is this a major limitation of the method? I do not know. It would be an interesting comparison to take a look at the literature a few decades ago when computer simulations started to become widespread to check whether we are following a similar pattern of adoption. Did the breakthroughs from computer simulations lag widespread adoption by a few years? Can we expect the same to happen with machine learning?
I also suspect that there is an astronomy specific effect: lots of discoveries in astronomy were enabled by the tedious collection of large volumes of seemingly uninteresting data (e.g. stellar spectra) coupled to classification tasks that had to rely on human judgment (again, stellar spectra). Now this is increasingly being outsourced to ML models, sometimes trained on crowdsourced data (did you hear of galaxyzoo?).
Also to keep in mind is the fact that people do not feel allowed to conclude new results from new methods until the method is widely accepted by the community. Thus most ML-in-astro papers confirm what we already knew, because we are still in the stage where people want to show that these methods work but are wary to go out on a limb and say that the new method solved some controversial problem.
Hi Mario,
concerning computer simulations, I do not think there was ever a sharp discontinuity from a scientific community that did not use them to one that used them massively (as we are currently doing). I think they became more and more common, as well as computers got more and more powerful… but all of this was slow and continuous. Just one day you look back and you realise that ten or twenty years before things were done differently :)
At the end the use of computers in science is probably a true methodological revolution that took place silently.
Leonardo
This is in principle quantifiable. Whether machine learning adoption and capabilities evolve slowly (like numerical simulations) or abruptly is a question of great interest for forecasting.