In the world of quantum mechanics, knowledge comes in fits and starts. In between explosive findings, like the Higgs boson in 2012, and illuminating theories, like Albert Einstein’s concept of general relativity, is a large gap. Why do big things follow certain laws of nature that very small things don’t? Lee Smolin, an iconoclast in the world of theoretical physics, says that “in all these years of experiments, [there] is better and better and better confirmation of the predictions of the Standard Model, without any insight into what may be behind it.”

Since he was a boy, Smolin has been on a path to figure out what’s behind it. The 63-year-old theoretical physicist decided to take up Einstein’s unfinished business—making sense of quantum physics, and unifying quantum theory with general relativity—back when he was a teen. He dropped out of high school from boredom. And this quest for truth has kept him up at night and sustained his work, through college, graduate school, and his current tenure at the Perimeter Institute in Ontario, Canada, where he has been part of the faculty since 2001.

In his latest book, Einstein’s Unfinished Revolution, Smolin remembers thinking “he was unlikely to succeed, but perhaps here was something worth striving for.” Now, it seems, he may have found a way to construct the elusive “theory of everything.”

During our phone conversation, Smolin explained from his home in Toronto how he got into the world of quantum physics and how he views the quest he’s been on for most of his life. Now, as ever, he is a teacher. Quantum mechanics, Schrodinger’s cats, bosons, and dark energy might be difficult to access for most, but it’s clear from the careful and organized way Smolin explains complex ideas and history in his writings and conversations, they don’t have to be.

Your latest work, Einstein’s Unfinished Revolution, which was just released, takes a realist approach to quantum mechanics. Can you explain the significance of that approach?

A realist approach is one that takes the old-fashioned point of view that what is real in nature is not dependent on our knowledge or description or observation of it. It simply is what it is and science works by observing evidence or a description of what the world is. I’m saying this badly, but a realist theory is one where there is a simple conception, that what is real is real and depend on knowledge or belief or observation. Most importantly, we can find out facts about what’s real and we draw conclusions and reason about it, and therefore decide. It isn’t a way most people thought of science before quantum mechanics.

The other kind of theory is an anti-realist theory. It’s one that says there are no atoms independent of our description of them or our knowledge of them. And science is not about the world as it would be in our absence—it’s about our interaction with the world and so we create the reality that science describes. And many approaches to quantum mechanics are anti-realist. These were invented by people who didn’t think there was an objective reality–instead, they understooreality to be determined by our beliefs or our interventions in the world.

So the most important thing that the book explains is this debate or even contest between realist and non-realist approaches to quantum mechanics since the beginning of the theory in the 1910s, the 1920s. The book explains some of the history that has to do with the philosophical schools of thought and trends which were popular during that period when quantum mechanics was invented.

Since the beginning, since the 1920s, there have been versions of quantum mechanics that are completely realist. But these are not the forms of quantum mechanics which are usually taught. They’ve been de-emphasized but they have existed and they’re equivalent to the standard quantum mechanics. By their very existence, they negate many of the arguments that the founders of quantum mechanics gave for their abandonment of realism.

The issue of whether there can be objective truths about the world is also important because it is at the core of a number of key public debates. In a multicultural society, there’s a lot of discussion about how and whether you talk about objectivity, reality. In a multicultural experience, you might tend to say that different people with different experiences, or different cultures have different realities, and that’s certainly true in a certain sense. But there’s another sense in which each of us just exists and what’s true of nature should be true independent of what culture or background or belief we bring to science. This book is part of that argument for that point of view, that in the end, we can all be realists and we can have an objective view of nature, even as we are multicultural with expectations in human culture and so forth.

The key idea, in society as well as physics, is that we must be relationalists as well as realists. That is, the properties we believe are real are not intrinsic or fixed, rather they concern relationships between dynamical actors (or degrees of freedom) and are themselves dynamical. This switch from Newton’s absolute ontology to Leibniz’s relational view of space and time has been the core idea behind the triumph of general relativity. I believe this philosophy also has a role to play in helping us shape the next stage of democracy, one suited to diverse, multicultural societies, which are continually evolving.

So, this book is trying to intervene in both debates about the future of physics and debates about the future of society. This has been true, really, of all six of my books.

In your 2013 book, Time Reborn, you describe your rediscovery of time, this revolutionary idea that “time is real.” How did this journey contemplating time and space begin? 

I’ve always been interested in time and space, even when I was a child. When I was 10 or 11, my father read a book about Albert Einstein’s theory of relativity with me and, at that time, I wasn’t originally thinking of being a scientist. But years later, when I was 17, I had a kind of magical moment one evening, when I read the autobiographical notes of Albert Einstein, Philosopher-Scientist and got the strong feeling that that was something I would be interested in following and doing.

I read that book because I was interested in architecture during those years. I became pretty interested in architecture after meeting Buckminster Fuller. I got interested in his geodesic domes and the idea of making buildings with curved surfaces, so I began to study the mathematics of curved surfaces. Just kind of out of rebellion, I went through the exams for mathematics even though I was a high school dropout. That gave me the opportunity to study differential geometry, which is the mathematics of curved surfaces, and every book I was studying to do the kind of architecture projects I was imagining had a chapter on relativity and the general theory of relativity. And I got interested in relativity.

There was a book of essays about Albert Einstein, and in it was the autobiographical notes. I sat down one evening and read them through and just got a strong feeling that that’s something I can do. I basically decided to become a theoretical physicist and work on fundamental problems in space-time and quantum theory that evening.

Your decision to drop out of high school propelled you on your path toward theoretical physics. What other circumstances supported your decision to be a physicist?

I lived in Manhattan in New York City until I was about 9. Then we moved to Cincinnati, Ohio. With the help of a friend of the family who was a professor of mathematics at a little college in Cincinnati, I was able to jump ahead three years and do calculus. And I did that totally as a gesture of rebellion. And then, I dropped out of high school. My motive was to start taking college courses early because I was very bored with high school.

Young PhDs face a lot of pressure in academia’s publish-or-perish environment. In your 2008 book, The Trouble with Physics, you wrote about an additional obstacle that plagues theoretical physicists at the beginning of their career. “String theory now has such a dominant position in the academy that it is practically career suicide for young theoretical physicists not to join the field.” Does that pressure still exist today for young PhDs?

Yes, but perhaps not quite as much. As always, the job situation for new PhDs in physics is not great. There are some jobs but there are not as many as there are people who are qualified for them. A new PhD student who does their work within a well-defined, well-known framework, where they can be judged on their problem-solving ability rather than their ability to, say, discover new ideas and new directions, is a safer path at the beginning of your career.

But I think in the long run, students should ignore that and should do what they love and what they’re most suited to do. There is room also for people who have their own ideas and who’d rather work on their own ideas. It’s a harder path in the beginning for those young people, but on the other hand, if they’re fortunate and they get a toehold in the system and they really do have original ideas—which are good ideas—they will often find they have a place in the academy.

I think there’s no value in trying to game the system. People may disagree, but that’s my sense. You might try to game it and say “Look, there are five times more positions in condensed matter physics than there are in quantum gravity”—so then you would choose to go into condensed matter physics, but there are ten times more people going into condensed matter physics. So you face much more competition.

At some point, you were a proponent of string theory. When and how did string theory become too problematic in your mind?

I would say there are several issues that seemed very difficult to address. One of them is the landscape problem, why there appear to be a vast number of different ways that this world of dimensions can curl themselves up.

So one of the problems that we have with the standard model of particle physics is that it doesn’t specify the value of many of the important properties of the particles and forces it describes. It says that elementary particles are made up of quarks and other fundamental particles. It doesn’t specify the masses of the quarks. Those are free parameters, so you tell the theory what the masses of the different quarks are or what the masses of the neutrinos are, the electrons, what are the strength of the different forces. There are altogether about 29 free parameters—they’re like dials on a mixer and they turn up and down masses or strengths of forces; and so there’s a lot of freedom. This is once the basic forces and basic particles are fixed, you still have all this freedom. And I started to worry about this.

When I was in graduate school, and into the 1980s, and then string theory was invented, there was that brief moment when we thought that string theory would resolve those questions because it was believed to be unique—to come into only one version. And all those numbers, such as the masses and the strengths of the forces, would be predictions of the theory unambiguously. So that was for a few weeks in 1984.

We knew part of the price of the theory is that it doesn’t describe 3 dimensions of space. It describes nine dimensions of space. There are six additional dimensions. And to have anything to do with our world, those six extra dimensions have to shrink down and curl up into spheres or cylinders or various exotic shapes. Sixth dimensional space can curl up into a lot of different things it would take the language of a mathematician to even describe. And there turned out to be at least hundreds of thousands of ways to curl up those six extra dimensions. Additionally, each of those corresponded to a different kind of world with different elementary particles and different fundamental forces.

Then my friend, Andrew Strominger, found that actually, that was a vast undercounting and there were a vast number of possible ways to curl up the extra dimensions leading to a vast number of possible sets of predictions for the properties of the elementary particles. So it seemed like string theory couldn’t make any predictions or explanations of why the particles came out and the forces came out the way that they did in the standard model.

Another problem is that they don’t stay curled up, since this geometry of spacetime is dynamical under general relativity or under string theory. It seems the most likely thing is that the dimensions that you make smaller can either collapse the singularities or start to expand and evolve in ways that manifestly don’t look like our universe.

There are also some problems of mathematical consistency where the theory actually predicts infinite answers to questions that should be finite numbers. And there are foundational interpretational problems. So it was a kind of crisis. At least, I felt there was a crisis right away, which was 1987. Most people working on string theory didn’t recognize that crisis until about the mid-2000s, but I felt it acutely so I began looking for ways that the universe might pick its own parameters.

It’s a beautiful idea but it faces these fundamental obstacles. There hasn’t been much progress on it for many years.

Was it around that point when you came up with the idea of “cosmological natural selection?”

I began thinking about this like an evolutionary biologist because at the time I was reading books by the great evolutionary biologists who wrote popular books. Steven J. Gould, Lynn Margulis, Richard Dawkins. And I was very influenced by them, to try to seek a way that the universe could be subject to some kind of process of natural selection that would fix the parameters of the standard model.

The biologists had this notion that they called the fitness landscape. A landscape of different possible sets of genes. On top of this set, you imagined a landscape in which the altitude was proportional to the fitness of a creature with those genes. That is, a mountain was taller at one set of genes if those genesresulted in a creature that had more reproductive success. And that was called the fitness. So I imagined a landscape of string theories, a landscape of fundamental theories, and some process of evolution going on on it. And then it was just a question of identifying a process that should work like natural selection.

So we required some kind of duplication and some kind of means of mutation and then some kind of selection because there had to be a notion of fitness. And at that point, I remembered an old hypothesis of one of my postdoctoral mentors, Bryce DeWitt, who had speculated that inside of black holes were the seeds of new universes. Now, ordinary general relativity predicts that to the future of the event horizon is a place that we call singular, where the geometry of space and time break down and time just stops. And there was evidence then—and it’s stronger now—that quantum theory leads to a situation where that collapsed object becomes a new universe, that instead of being a place where time ends, the interior of a black hole—due to quantum mechanics—has a kind of bounce where a new region of space and time could be created, which is called a “baby universe.”

So, I imagined that that mechanism, if true, would serve as a kind of reproduction for universes. In the instance this happens in black holes, universes that created many black holes during their history would be very fit, would have a lot of reproductive success, and would be reproducing many copies of its “genes,” which were by analogy, the parameters of the standard model. It just kind of came together. I saw that if we adopt the hypothesis that black holes bounced to make baby universes—you’ve got a mechanism of selection that might work in the cosmological context to explain the parameters of the standard model.

Then I came home and a friend called me from Alaska, and I told her my idea and she said, “You have to publish that. Somebody else will if you don’t. Somebody else will have the same idea.” Which, indeed, you know, a lot of people did publish versions of it later. So that’s the idea of cosmological natural selection. And it’s a beautiful idea. Of course, we don’t know if it’s true. It does make a few predictions, so it is falsifiable. And so far it has yet to be falsified.

You’ve also said there has been less progress over the last thirty years than over the last century in fundamental physics. How far are we into what you’ve called, this current revolution?

If you define a major advance as when either a new experimental result verifies a new theoretical prediction based on a new theory or a new experimental result suggests a theory—or interprets a suggested theory that goes on and survives other tests, the last time there was such an advance was the early 1970s. Since then there have been several experimental findings which were not predicted—like that the neutrinos would have mass; or that dark energy would not be zero. Those are certainly important experimental advances, for which there was no prediction of or preparation for.

So in the early 1970s there had been formulated what we call the standard model of particle physics. The question has been how to go beyond that, because that leaves a number of open questions. A number of theories have been invented, provoked by those questions, which made various predictions. And none of those predictions have been verified. The only thing that has happened in all these years of experiments is better and better and better confirmation of the predictions of the standard model without any insight into what may be behind it.

It’s getting on 40-something years—without a dramatic development in the history of physics. For something like that, you’d have to go back to the period before Galileo or Copernicus. This current revolution was commenced in 1905 and so far we have taken about 115 years. It is still unfinished.

Within physics today, what findings or answers would spell the end of the current revolution that we’re in?

There are several different directions which people are exploring as roots to take us beyond the standard model. In particle physics, in the theory of the basic particles and forces, they made lots of predictions from a number of theories, none of which have been confirmed. There are people studying the fundamental questions that quantum mechanics presents us with and there are some experimental theories there that attempt to go beyond fundamental quantum physics.

Within fundamental physics, there are some mysteries that we get easily confused about, that the standard formulation of quantum mechanics brings up, and so there are experimental predictions that are related to going beyond quantum mechanics. And there are predictions related to unifying quantum mechanics with Einstein’s theory of general relativity, to have the whole theory of the universe. In all those domains, there are experiments and the experiments so far have failed to reproduce either a hypothesis or a prediction that went beyond the theories that we now understand.

There has not been a real breakthrough in any of the directions that I’m most concerned with. It’s very frustrating. What’s happened since the Large Hadron Collider found the Higgs boson and all of its properties, verified the predictions so far of the standard model? We don’t discover any additional particle. There were experiments which might have found evidence for the atomic structure of space that we were talking about under certain hypotheses. Those experiments have not shown that either. So they’re still all consistent with space being smooth and not having atomic structure. They’re not quite after it enough to completely rule out the depiction of quantum gravity but they’re going in that direction.

It’s a frustrating period to be working on fundamental physics. It’s important to emphasize that not all fundamental science, not all physics is in this situation. There are certainly other areas where progress is being made, but none of them really probe the fundamental questions as to what are the fundamental rules of nature.

Do you think there are conditions that allow revolutions to occur, some sort of methodology?

I don’t know that there are any general rules. I don’t think there’s a fixed method to science. In the twentieth century, there was a lively debate that continues among philosophers and historians of science today, about why science works.

One view about why science works that many of us are taught in elementary school and high school, that my son is being taught, is that there’s a method. You’re taught if you follow the method, you make your observations, and you take notes in a notebook, you log your data, you draw a graph, I’m not sure what else, it is supposed to lead you to the truth—apparently. And I think that specifically, versions of that were put forward under forms related to psychological positivism, which argued that there was a methodology to science, and that distinguished science from other forms of knowledge. Karl Popper, a very influential philosopher, argued that science was distinguished from other forms of knowledge if it made predictions that were falsifiable, for example.

On the other end of this debate, was an Austrian, a fellow named Faul Feyerabend, one of the important philosophers of science, and he argued very convincingly that there is no method in this universe for all sciences, that sometimes one method works in one part of science and sometimes it doesn’t work and another method works.

And for scientists, just like with any other part of human life, the goals are clear. There’s an ethics and a morality behind everything. We move closer to the truth rather than further from the truth. That’s the kind of ethical principle that guides us. In any given situation there is a wiser course of action. It’s a shared ethics within a community of scientists regarding knowledge and objectivity and telling the truth over fooling ourselves. But I don’t think that’s a method: it’s a moral condition. Science, it works because we care to know the truth.

What do you say to the idea promoted by some theoretical physicists like Stephen Hawking that there could be no grand unifying theory of everything?

Nature presents itself to us as a unity and we want to understand it as a unity. We don’t want one theory to describe one part of a phenomenon and another theory to describe another part. It doesn’t make sense otherwise. I’m searching for that single theory.

Why can’t quantum physics be melded with general relativity?

One way to understand it is that they have very different concepts of time. They have concepts of time that seem to contradict each other. But we don’t know for sure that they can’t be melded together. Loop quantum gravity appear to have succeeded, at least partly, in melding them together. And there are other approaches that go some distance. There’s an approach called causal dynamical triangulation—Renate Loll, Jan Ambjørn, and colleagues in Holland and Denmark—as well as an approach called causal set theory. So there are several different ways to get at least part of the picture.

Then we seem to be in a “blind men and the elephant” situation in which you ask about a quantum theory of gravity through different thought experiments, through different questions, and you get different pictures. Maybe their job is to put those different pictures together; none of them by themselves seems to have the ring of truth or to go all the way to make a complete theory. We’re not there but we have lots to think about. There are lots of partial solutions. It can be very inspiring and also, it can be very frustrating.

The idea of loop quantum gravity you mentioned is one you developed along with others, including Carlo Rovelli. How can loop quantum gravity connect quantum mechanics and general relativity? 

Loop quantum gravity is one of several approaches which have been invented to try to unify quantum physics with general relativity. This approach came about through several developments being pursued by several people.

I had a set of ideas I was pursuing which had to do with trying to use a physical picture that had been developed in the standard model of elementary particle physics. In this picture, there were loops and networks of fluxes or forces which became quantized and the flux—say, if a magnetic field had a superconductor that breaks up into discrete flux lines—that was one of the roads to quantum gravity. Another one was Abhay Ashtekar making a reformulation of the theory of general relativity by Einstein to make it look more like the forces in the standard model of elementary particles. And those two developments fit together nicely.

These came together to give us a picture in loop quantum gravity in which there becomes an atomic structure of space just like with matter—if you break it down small enough, it’s composed of atoms which go together through a few simple rules into molecules. So if you look at a piece of cloth, it may look smooth, but if you look small enough, you’ll see that it’s composed of fibers made of various molecules and those in turn are made of atoms tied together, so on and so forth.

So similarly, we found by basically solving the equations of quantum mechanics and general relativity simultaneously, a kind of an atomic structure to space, a way to describe what the atoms in space would look like and what properties they would have. For example we discovered that atoms in space would take up a certain discrete unit of volume and this came from a certain set of allowable volumes the same way that in regular quantum mechanics the energy of an atom lies in a discrete spectrum—you can’t take a continuous value. We found out that areas and volumes, if you look small enough, come in fundamental units and so we predicted the value of those units. And then we started to get a theory, a picture of how these shapes, which were kind of atoms in space, could evolve in time and we got an idea of how to—it’s pretty complicated—but how to at least write down what the rules were for those objects to change in time.

Unfortunately, all of this is on an extremely small scale and we don’t know how to make an experiment to test whether what’s really going on when a gravitational wave travels through space, for example. To do experiments which are falsifiable, you need to be able to make measurements of geometry and length and angles and volumes at extremely small distances—that we’re definitely not able to do. We’re working on it, and I’m pretty confident we’ll get there.

Can researchers like yourself still uncover deep truths like these in the midst of government shutdowns and funding cuts?

Science certainly and properly, in most countries of the world, depends on public funding—on public funding through the government, typically. There is a component which is paid for by philanthropy and I think there is a role for private support and philanthropy, but by far the core of science is and I believe properly should be, funded publicly by the government.

I think that science is a public function and having a healthy scientific research sector is as important to the well-being of a country as having a good education or having a good economy, so I feel very comfortable being publicly supported. The Perimeter Institute, where I work, is partly publicly supported and partly privately supported.

You certainly want to have a healthy amount of funding of science by governments and interruptions of that or cuts to that obviously make science harder to do. You can certainly question, is a lot of money well spent? You can also question, well shouldn’t we be spending 10 or 20 times more? There’s justification for both. Certainly an agency like, in my field, the United States National Science Foundation or Natural Sciences and Engineering Research Council (NSERC) of Canada has to make difficult choices over different proposals, but that’s the nature of anything that’s worth doing. You have to make choices.

What advice do you have for young physicists, or even scientists in general, beginning their careers?

We should see having a career in science as a wonderful privilege and you should try as hard as you can to become somebody who can contribute to making progress to solving problems. The most important question is: What are you curious about? If it’s something that you really must understand, that keeps you up at night, that drives you to work hard, then you should study that problem, study that question! If you go into science to have a decent, well-paid career, you’re better off going into business or finance or technology, where all that intelligence and energy that you put in will just go to advancing your career. I don’t want to be too cynical, but if your motives are careerist, there are easier ways to have careers.