Why do theoretical physicists write papers explaining preliminary results?
|
|
| The CERN auditorium Image Credit: Skeie |
How does physics work? Do experimental results drive theory, or do theoretical results drive new experiments? When new experimental results are announced, the process of how and why the physics community reacts the way it does can seem opaque. PhysicsBuzz shines some light on the stops and starts of physics discovery by investigating two different cases where exciting new experimental results gained media attention and generated theoretical papers, and then those experimental results were invalidated.
On December 15, 2015, the ATLAS and CMS experimental particle physics research collaborations that examine data from the Large Hadron Collider at CERN in Geneva, Switzerland joined together to announce the latest results from their data analysis.
In 2 slides out of 40 from the ATLAS presentation, and in 3 out of 52 from the CMS presentation, there was a tentative but resonant announcement. The two collaborations (which do separate analyses and serve as checks on each others’ findings) found, after subtracting the background and running their complex data analysis, a “bump” in the data.
The detectors at the LHC can measure particles of light, called photons. When two photons of the same energy are observed close enough together in the LHC detectors, it suggests that they might have been formed by a massive particle (called a boson) decaying into two photons. This phenomenon is called a “diphoton event.” And in 2015, ATLAS and CMS observed more diphoton events than they expected at 750 billion electron volts (GeV).
This reminded the physics community of the Higgs boson discovery from 2012. The Higgs, a long-theorized particle that helps explain where mass comes from, was first seen as a diphoton excess at 125 GeV.
But in this case, the bump in the data was small. That is, ATLAS and CMS reported low confidence values for the excess, values below the 5-sigma marker of “discovery” in particle physics. Sigma indicates statistical significance; 5-sigma means that scientists can be 99.99994% sure an anomaly in some data is actually evidence of an anomaly in nature.
ATLAS measured the bump at 750 GeV at a 3.6 sigma significance. CMS found a significance of 2.6 sigma at 760 GeV. A significance of 3 sigma means 99.73% confidence; 2 sigma means 95.5% confidence. Those values might not seem that different, but with so much data coming out of high-energy physics experiments, these differences can be crucial. (Read our 2012 article explaining more about sigma)
Still, it seemed like these could be the first results of what physicists call “new physics”: experimental data that does not disprove or work against previous physics data, but points to the existence of something—usually a new particle—not already predicted by the Standard Model. (Article continues below)
What is the Standard Model?
The Standard Model is a well-proven way to understand most of the experimental results the world has seen from particle physics. Along with a sense of the scientific structure (leptons, bosons, neutrinos) of this model, it’s important to have an idea of the story behind it.
|
|
| The Standard Model Image Credit: Wiki user MissMJ CC BY 3.0 |
The Standard Model was not created by just one person, or just one collaboration, at one time. It is a combination of Albert Einstein’s 1905 and 1915 theories of relativity, and Max Planck’s 1900 theory of quantum mechanics, as well as the 1954 Yang-Mills field equations. It also includes the 1969 theory of the charm quark that led to the November Revolution in 1974 when experimentalists found charm quarks. Experimental findings that confirm the Standard Model span the twentieth and twenty-first centuries, including the bottom quark in 1977, tau lepton and neutrino in 1978, top quark in 1995 and Higgs boson in 2012.
Physicists know that Standard Model accurately predicts the behavior of particles from about 100 GeV to 1 TeV.
But if the Standard Model is so long-lived and well proven, why are physicists so eager to find “new physics” beyond the Standard Model?
Part of the reason is that the Standard Model doesn’t explain everything (it doesn’t account for gravity, for instance). But another part of the reason is that it’s possible for the Standard Model to be expanded without invalidating previous data or previous theories. As high-energy particle accelerators generate particles at higher and higher energies, it’s entirely possible to see a particle exist at 750 GeV that would not have been detectable in any of the previous lower-energy experiments.
Physicists around the world, as well as the press, reacted to the news from Geneva about the 750 GeV diphoton excess.
There are 445 papers reacting to the excess in the online paper database, arXiv.org. It’s important to note that while most of these papers offer extensions of existing theoretical models that can be used to predict the 750 GeV excess, some of them also explained why, theoretically or experimentally, the excess could not be possible. While most papers on the archive are unpublished (the archive requires a university email address and rarely rejects papers), many of those papers went on to be published. APS’s Physical Review Letters (PRL) published four in April 2016.
When the news about the excess first broke, press coverage emphasized how excited physicists were about the possibility of “new physics.” At the same time, many news outlets were sure to include quotes from physicists saying that they weren’t sure the excess was going to be turn out to be true. The excess was small, so it was always possible that it would end up being a statistical fluctuation when the LHC collaborations collected more data.
Then, on August 5, 2016, at the International Conference on High-Energy Physics (ICHEP) in Chicago, the ATLAS and CMS collaborations announced that when they were able to work with more data at even higher energies, the bump in the data they had seen before flattened out; there was no more excess of diphoton events at 750 GeV. The bump they had reported in December 2015 was not evidence of “new physics,” but neither was it a malfunction in equipment or an error in the data analysis. It was a statistical fluctuation.
So why did theoretical physicists write so many articles about something they knew might not really exist? Do theorists jump at every data anomaly coming from CERN?
Theoretical physics has a complex relationship with experimental data, and several prolific theorists disagree over the fundamental nature of the field. But to see the thought process behind one of the theory papers published about the 750 GeV diphoton excess, let’s start with the source: one of the authors.
Dr. Yuichiro Nakai, a theoretical physicist at Harvard, co-authored of one of the four theory papers published by PRL in April 2016 that explained how the 750 GeV bump could be explained as an extension of the Standard Model.
“We [Nakai and his collaborators on the paper] knew from experience that there are many papers that appear immediately,” after an announcement like the one in December, Nakai said. “So we were rushing to work and finish this project and we succeeded.”
Nakai and his collaborators proposed that the 750 GeV bump could be explained by a new heavy particle, or boson, that decayed to two photons.
“I had never considered pursuing this kind of model before, because there was no such gamma excess of diphoton to merit it…we had no motivation to consider it before,” Nakai said. “But at the same time we, of course, knew that some of the particles in Standard Model can decay to two photons.”
When Nakai heard about the excess, “I didn’t believe it much, but I hoped that it would turn out be true.”
And when ATLAS and CMS confirmed their first analysis results (using the same data from 2015) in March 2016, Nakai started to feel hopeful about the results, even though he remained skeptical.
“Of course we hoped that this one was true, but we’ve had many, many experiences when an excess appears, there’s many papers, and then the excess goes away,” Nakai said.
“Of course we hoped that this one was true, but we’ve had many, many experiences when an excess appears, there’s many papers, and then the excess goes away,” Nakai said.
Nakai did not see this as just a case of the press over-hyping something physicists were not excited about. Not all data anomalies are created equal.
For example, in 2014, CMS reported a small excess in its data. Then, in 2015, ATLAS also showed a similar excess. That time it was a pair of bosons, not photons, and 2 trillion electron volts, not 750 billion electron volts. Researchers called it a “diboson excess.” But this excess was a different thing, in both the particle physics community and the press, than the 750 GeV diphoton excess.
“Many more theorists jumped in and wrote papers about the diphoton excess than the diboson excess,” Nakai said. “Part of the reason is, I think, that for the diphoton excess theorists could more easily construct a model to explain the data.”
Dr. Joanne Hewett, a theoretical physicist at the SLAC National Accelerator Laboratory in Menlo Park, CA, saw the theoretical situation behind the diboson excess differently. She published theory papers on both the 2 TeV diboson excess and the 750 GeV diphoton excess.
“When I first looked at the 2 TeV excess it was brain-dead easy to come up with a theory to explain it,” Hewett said. It was immediately apparent to her that the diboson excess could be incorporated into the left-right symmetric model, an extension of the Standard Model from the mid- to late- 1970s.
But she agreed that fewer papers were written about this bump than the 750 GeV diphoton bump. Even though both excesses were found at near three sigma, the sigma level is not the whole story.
“The signal for the diboson excess was much less clean than the diphoton excess,” Hewett said.
Even though the names “diboson” and “diphoton” seem analogous, the diboson case actually referred to two different boson particles: W and Z. Whereas in the diphoton case, the name refers to two identical photons.
Most of the high-energy particles produced in the LHC have very short lifetimes and are found in the CMS and ATLAS detectors only by the energy deposits of their decay products (the smaller, slower particles that remain after a high-energy particle decays). W and Z bosons can decay a number of different ways, in what particle physicists call different “channels.”
Photons, on the other hand, can be directly detected in the LHC.
“Identifying photons is something the LHC detectors were designed to do very well,” Hewett said, “because they were designed to discover the Higgs in the diphoton channel.”
And it’s not like diboson case is the only time this has happened before.
In fact, this narrative has a long history. Many intriguing particle physics “bumps” are found in experiments, only to be statistically disproven with more data later. In fact, before the Higgs was proven in 2012, there were many bumps in the data from many different particle experiments that seemed to be the first hints of the Higgs, only to flatten out later.
|
|
| The Crystal Ball detector in Mainz, Germany Image Credit: School of Physics and Astronomy, University of Edinburgh |
For example, in 1984, the Crystal Ball collaboration at the Deutsches Elektronen-Synchrotron (DESY) research center in Hamburg found an excess at 8.3 GeV. The excitement from the physics community prompted the article in Physics Today, “Has the Higgs boson been seen at Crystal Ball?” But in 1985, after the collaboration examined more data, it was clear that the bump was a statistical fluctuation. There was no new particle at 8.3 GeV.
“There have been many larger fluctuations than the 750 GeV bump that have gone away in the past. I mean, the pentaquark [a particle made up of five quarks that was recently discovered and confirmed by CERN’s LHCb experiment] came and went a few times in the last 15 years before it was confirmed,” Dr. Sabine Hossenfelder said. Hossenfelder is a theoretical physicist at the Frankfurt Institute for Advanced Studies who works on models to explain quantum gravity.
Hossenfelder and Nakai agreed that the diphoton excess could be described as more “theorizable” than other excesses.
“Theorizable” means that it was surprising enough to be notable (it was potential evidence of a new particle or particles that would constitute an expansion of the Standard Model), but not enough to be revolutionary (its existence wouldn’t require the re-working of existing scientific theories).
This idea is consistent with some ideas Hossenfelder has written about on her blog, Backreaction, in a post titled “Surprise me, but not too much.”
Hossenfelder suggested that physicists, especially theorists, might shy away from game-changer theories in a way that is contrary to the idea of an objective search for explanations of nature. For her, the issue is both scientific and sociological.
“Even though they [theorists] will tell you, tell journalists, oh it’s only two sigma, oh it’s probably wrong…they’ll still write a paper about it anyway,” Hossenfelder said.
“Even though they [theorists] will tell you, tell journalists, oh it’s only two sigma, oh it’s probably wrong…they’ll still write a paper about it anyway,” Hossenfelder said.
Hossenfelder described the number of theoretical papers that appeared online immediately after the ATLAS/CMS announcement as “ridiculous.”
But she does not think this is all necessarily just the theorists’ fault (“that’s their job,” she said). In general, she is worried about the cycle of experimental press attention and theory publication, if only because she felt it keeps physicists’ attention on a narrow band of physics.
“The variations we currently see in phenomenology models or in theory development in general are very small,” Hossenfelder said. “People are kind of afraid to surprise someone too much.”
“Afraid to surprise someone too much”
Hossenfelder’s comments echo the words of Dr. Thomas Kuhn. Kuhn, a historian and philosopher of science, wrote in his 1962 book The Structure of Scientific Revolutions that scientific discovery followed a certain process. According to Kuhn, a certain paradigm, or theoretical basis for scientific thought (such as the Ptolemaic, earth-centric view of the galaxy) leads to “normal science” that searches for details in that scientific paradigm. At some point, “normal science” encounters enough observations that violate the accepted paradigm, and after some debate—which can be contentious—on the part of the scientific community, a new paradigm is eventually accepted (such as Copernicus’s heliocentric model).
Kuhn’s ideas, while incredibly influential, are not without controversy, and Kuhn went on record many times to plead that his ideas should not be used to de-legitimize science. But he did specifically talk about how scientists deal with anomalies in data. He wrote, “No part of the aim of normal science is to call forth new sorts of phenomena: indeed those that will not fit in the box are often not seen at all…normal scientific research is directed to the articulation of those phenomena and theories that the paradigm already supplies.”
Proving the Standard Model wrong is very different from proving Einstein wrong or existing data wrong. At a certain stage, something about how we think about the Standard Model has to be wrong, because it doesn’t explain everything.
“The rigidity of the Standard Model could be a sign that we’re closing in on the final theory,” Hossenfelder said, “or it could mean that if we want to improve on this theory, this rigidity is telling us that we’ll need to make a big change. Because if small changes ruin things, if you want do something new, it needs be entirely different.”
“So it’s a double-edged sword,” Hossenfelder said. “It could mean it is the exact right thing to do or the exact wrong thing to do.”
But it depends on how you look at the current state of physics. Hewett disagrees with the idea that the Standard Model is currently too rigid to change.
“Whatever the ultimate theory ends up being, it has to contain the Standard Model at the energy scale where it works,” Hewett said.
Dr. Andrew Cohen, a theoretical physicist at Boston University, thought that the number of papers published might not be the best way to judge the scientific reaction to any particular result, but agreed that the papers about the 750 GeV excess were a special case.
“Partly, CERN was looking for an excess because theorists had been talking about it prior to when the experiment started,” Cohen said.
“It’s very easy—it takes no effort at all—to produce [theoretically] the signature that the LHC had seen,” Cohen said. “So, nobody bothered to write about it before.”
Even before there was any hint for the 750 GeV excess, Cohen thinks theorists could have written very similar papers about potential new particles at 600 GeV, 605 GeV, and kept increasing the energy level until they coincided with experimental evidence.
“I, like any other theorist, knew when the [750 GeV] data was released that the most likely thing was going to be that the excess would not be seen again. I, and probably every other theorist, spent an afternoon and wrote down several models that could produce [the 750 GeV diphoton excess]. I thought none of them were exciting enough to publish…other people made other judgments. The 400 different theory papers [written about the 750 GeV diphoton excess] were mostly the same ideas,” Cohen said.
Dr. Robert Garisto is an editor of PRL. In April 2016, PRL ran an editorial note explaining the situation. The note stated that “…we think that this set [of papers] gives readers a sense of the kind of new physics that would be required to explain the [750 GeV] data, if confirmed.”
“There were three possibilities of what we could have done: we could have thrown open the gates and published all the papers about the 750 GeV bump, we could have completely closed the gates and not published any of the papers, or published some sub-set of the papers,” Garisto said. In the end, PRL published a subset.
Garisto agrees about the theorizability of the 750 GeV bump.
“This was the kind of unexpected hint that one might expect,” Garisto said. “The 750 GeV bump was not an invalidation of previous physics. It just would have meant that there could be this other stuff in addition to what we already know.”
“This was the kind of unexpected hint that one might expect,” Garisto said.
So what’s the ideal that we’re looking for? How should theorists react to preliminary experimental data?
“Theory and experiment work hand in hand,” Garisto said. “You can’t expect theorists to wait until experiments are completely confirmed, nor would it be helpful. It’s like a conversation, not a monologue…you can’t have one waiting until the other one completely finishes to start talking.”
Nakai agrees. “If you want to publish a paper on an experimental excess, you have to rush because so many people want to publish the paper…we cannot wait for the experimental data to be verified.”
And, he adds, hundreds of papers reported to be on the physics pre-print archive does not mean that there are hundreds of robust theories out there explaining it.
Nakai thought that another factor explaining why there can be so many theory papers published after initial experimental results is how well those results fit into possible answers for the bigger open questions in physics.
“Many papers…want to explain another phenomenon like dark matter…and so they want to explain something other than the diphoton excess in a way which also explains the diphoton excess,” Nakai said.
And it’s important to note that sometimes theorists’ bets on initial experimental results pay off.
For example: neutrino oscillations.
Neutrinos are the lightest particle in the Standard Model, and there are three types: electron, muon, and tau. Neutrino oscillations means that one type of neutrino can oscillate into another type of neutrino. That is, the type of an individual neutrino can change. Dr. John Bahcall began theorizing the existence of neutrino oscillations in 1968 to explain the low number of neutrinos detected in the Homestake gold mine in Lead, South Dakota. He was proven correct by data from the Kamiokande-II detector in Japan in 1989 and the Sudbury Neutrino Observatory in Ontario, Canada in 2001.
But there are also cases where very surprising, unbelievable results are just that—unreal.
On September 23, 2011, the OPERA particle physics experimental collaboration at an Italian national lab in Gran Sasso, Italy presented their latest results, which had just been posted online the night before, in Geneva at CERN. In fact, they were in CERN’s main auditorium—the same room where ATLAS and CMS collaborations would present their 750 GeV bump results 5 years later.
|
|
| The OPERA High Precision Tracker near Gran Sasso mountain, Italy Image Credit: OPERA Collaboration at LGNS |
Whereas the ATLAS and CMS announcements where hidden in a few slides in the annual releases of their work, the entire OPERA announcement was specifically about a particularly surprising result: the collaboration had found evidence for a neutrino particle moving faster than the speed of light. The collaboration reported its results at 6 sigma, well over the 5-sigma gold-standard. 6 sigma means that there is a 99.9999998% certainty that this result was not due to a random statistical fluctuation.
But theorists did not appear to jump for the new data in the same way they jumped for the diphoton excess.
Dr. Alan Chodos, former Yale physicist and APS Associate Executive Officer was one of the first theorists to work on a theory of tachyon (faster-than-light) neutrinos.
“I thought the magnitude of the effect that OPERA found seemed way too big to be consistent with tachyon neutrinos. I hoped and expected it would go away,” Chodos said.
Then, in February 2012, after running more tests, the collaboration reported that the anomaly could be due to a faulty wire in their GPS system. After more tests (and a check from a previous experiment that measured expected results for neutrino speed), the OPERA collaboration eventually changed their reported measurement of neutrino speed. The collaboration ended up publishing a new limit on that speed which was slower than the speed of light, and therefore not a stunning new result.
“In this way I found the OPERA case very much like the recent 750 GeV bump. When it [the anomaly in the data] went away, the air goes out of the balloon, and all the people who were extremely interested…it turns out they didn’t really have interest in it after all,” Chodos said.
“Most physicists have a intrinsic prejudice against that anything that could go faster than the speed of light,” Chodos said, “so a lot of the experiments that are done automatically exclude tachyons. If they measure something looking like a tachyon they would say, oops we must have made a mistake or say oops we didn’t find anything,” and take another look.
The Standard Model can be extended and changed without violating Einstein’s theory of relativity. But for the larger physics community to seriously consider either a model or data that allows for particles to travel faster than the speed of light would take a major shift in the convictions of most physicists.
The OPERA case, “was unexpected, it was not easy to understand, it violated well-known laws of physics,” Chodos said. “Whereas the 750 GeV would have been very interesting, but would not have been what I would call revolutionary.”
So we could say theorists are looking for something between scientific revolution and scientific boredom. But how do we differentiate between the two?
Many scientists talk about “ambulance chasing” in particle physics, a common term that means, roughly, the conjecture that theoretical physicists race each other to publish papers about every new bump in experimental data, whether or not it has been confirmed or will turn out to be accurate.
Chodos was conflicted about the idea. Whereas in experimental physics, he mentioned, physicists have to make sure to be correct in all of their data and there is a huge penalty to being associated with an experiment that published incorrect results, in theory it’s a different story.
“Ambulance chasing in particle physics…it looks bad when it doesn’t work, when we chase something that doesn’t turn out to be true…but on the other had, there are rewards if the phenomenon holds, if you were the one who comes up with the right explanation for it….as a theorist, the rewards for being right are much greater than the penalty for being wrong,” Chodos said.
Cohen, who in 2011 published a PRL paper explaining why the OPERA results could not describe nature, takes the idea even further.
“It’s not that there’s a small penalty for being wrong, it’s that there’s no penalty of any kind,” Cohen said. “It’s the theorist’s job to suggest ways in which the world might work.”
“It’s not that there’s a small penalty for being wrong, it’s that there’s no penalty of any kind,” Cohen said. “It’s the theorist’s job to suggest ways in which the world might work.”
“Theorists should be encouraged to look at every possibility, whether or not it’s right or wrong. That’s a good thing, not a bad thing,” Cohen said.
All together, PRL published three papers explaining how the faster-than-light neutrino result could not be theoretically correct. Compared to the 750 GeV diphoton bump (or even the 2 TeV diboson excess), it was not a “theorizable” preliminary experimental result.
“Most of the [theory] papers that were written explaining and justifying the superluminal neutrino thing were wrong,” Garisto said, “not even in the hey-it-turns-out-OPERA-was-wrong sense but wrong from the standpoint that the arguments were not consistent, did not pass the validity test. Plenty of papers justifying the 750 GeV thing could have been right if it did turn out to be there.”
“It’s easier to add new particles to existing theories than to get around an existing theory, something well known like relativity,” Garisto said. He cited Richard Feynman’s famous quote that physics is “imagination, in a tight straightjacket,” meaning that any new theories in physics have to explain all the existing evidence.
The issue of theory vs. experiment is a practical, pressing, issue for Hossenfelder. She helped organize a conference in Frankfurt in September 2016, bringing together the people who do quantum gravity theory and high-energy experimentalists.
“The reality of whether or not your theory works or not should matter,” Hossenfelder said. “How else do you know if your model is about nature or not?”
Hossenfelder adds that she knows complex theories can take a long time to develop fully.
“You first need to really understand how your argument works. Only then can you start thinking how about to test it. It’s a difficult question because nobody knows how long you should wait.” But she adds that, in her opinion, it’s better to start sooner rather than later.
“Unfortunately, theorists don’t often actually know what’s possible, experimentally. They might overestimate or underestimate what experimentalists can do, or study something that is far off reality. To get people together right here [at this conference] to talk to each other has been useful for both sides,” Hossenfelder said.
For Hewett, the physicist from SLAC who published on both the diphoton and diboson excess, the issue is about the ways she can use her 750 GeV diphoton work to continue her research into theoretical extensions of the Standard Model.
“Some people might think that these 400-500 papers [about the 750 GeV diphoton excess] were a waste of time, and it’s actually not true. I don’t regret the time we spent working on this paper at all, because we have found some new properties of this model that nobody had found before, [and] we only found these properties because we focused on the 750 GeV.”
Her group is currently working on a new paper about these new properties. This process happened, she explained, because explaining a 750 GeV diphoton excess required a re-working of larger theories beyond the Standard Model.
“We had to fit this piece of data somehow…it didn’t pop out from any existing theory. [We had to] learn the ways we could and could not stretch existing theories,” Hewett said.
So even if there never is another diphoton excess around 750 GeV, Hewett has found new, fruitful theoretical directions from invalidated data. Intriguing indeed.