When/how did quantum mechanics become what it is?
October 31, 2021 10:18 PM   Subscribe

My brain is entirely the layman’s, so keep that in mind when offering me guidance. Here is where I always, inevitably get stuck: every article or book I’ve ever read that addresses quantum mechanics offers a description that reads to me only as “our math must be probabilistic because we actually can’t observe a quantum system without disrupting it” but nonetheless that description always stands right next to the flat declaration that “reality itself is actually probabilistic”. When and how did scientific consensus go from “our math cannot be definitive” to “reality is not definitive”?
posted by bixfrankonis to Science & Nature (16 answers total) 11 users marked this as a favorite
 
To my understanding, it was a gradual collection of confusing results which classical physics couldn't address:
* Rutherford's experiments showing that the atomic nucleus was highly concentrated - the nucleus would want to tear itself apart due to electromagnetic repulsion of the positive protons - something was holding it together.
* Einstein's explanation of the photoelectric effect shows that light wasn't solely a wave
* Double-slit experiments, where single photons were still showing interference patterns going one at a time through a double slit
* Double-slit experiments with electrons showing that matter also had wave-like properties
* Investigations into radioactivity, and trying to understand why and when the nucleus would emit radiation
posted by coberh at 10:39 PM on October 31, 2021


Speaking of the double slit experiment, it's a remarkably old one. Like 1800s old. Look it up!

OP might you be interested in reading books that answer your question? The story is compelling and long, and in the final stages there's so much back-and-forth particularly between Einstein, Rosen, and Podolsky vs. Team Quantum, that I can't possibly report it here. But it's told particularly well in books like Brian Greene's The Fabric of the Cosmos and many others. I'm re-reading tFotC as an ebook from my local library right now which is why the name is on the tip of my tongue :) I promise it's super accessible.

Look up the E-P-R paradox and the story surrounding them, before and after, fire a start if you can't wait for the book.
posted by MiraK at 10:44 PM on October 31, 2021 [3 favorites]


Sorry when I said 1800s above that was a typo. I meant to write 1700s.

Crucially the question you're asking involves the debate between a set of people who argued that quantum probabilities work the same way as regular probability (i.e. there is an underlying "real" reality that can be discerned if only we had the tools to measure all of the factors and variables involved the same way we can predict whether a coin will land heads or tails if we know the exact forces acting on it during a toys like momentum and velocity of spin and gravity and wind disturbance and so on, and the only reason we use probabilities to understand outcomes is because we don't have those measurements), and people who were arguing that there is no underlying reality and probability is the literal state in which the universe exists, like, the electron is not a particle but rather a smear that's spread across literally the whole universe all at once - spread very very thin in most places and thickly in only some small areas.

I'll try to stop posting multiples here at 3 in the fucking morning but no promises because this shit is my jam, thickly spread in my localized area, lol.
posted by MiraK at 11:09 PM on October 31, 2021 [7 favorites]


The case for it going from "it's just math" to "this is reality itself" arose, in the wisdom I received, with Bell's Theorem: "Bell’s theorem shows that no theory that satisfies the conditions imposed can reproduce the probabilistic predictions of quantum mechanics under all circumstances." Before that set of results, as I understand it, there was a formal possibility for an Einsteinian hidden variables model to explain apparent indeterminacy with ultimate deterministic characteristics. After Bell's, that was seemingly ruled out.
posted by jjray at 11:12 PM on October 31, 2021 [7 favorites]


The "ultraviolet catastrophe" is an important part of the story I don't see mentioned yet. In about 1900, a classical physics calculation on the frequencies of light emitted by an idealized object in thermal equilibrium by Rayleigh and Jeans predicted infinite energy in ultraviolet photons. A way to correct this problem was found by Max Planck, who discovered that if you assume that light comes only in discrete units, i.e. quanta, of energy based on the frequency, then the math works out to predict the thermal emission of objects really well.

I'm pretty sure it was the first major clue that there might be this whole quantization thing happening.
posted by Zalzidrax at 11:43 PM on October 31, 2021 [3 favorites]


that description always stands right next to the flat declaration that “reality itself is actually probabilistic”

and the trouble with that declaration is that it contains the word "reality", which means so many different things to so many different people as to make any flat declaration containing it instantly dubious. Generally I will interpret this kind of declaration as coming from an author who is trying to talk a bigger game than they actually have.

Given my personal referent for the word "reality" - i.e. the totality of what is, what was and what will be, tightly synonymous with "everything" and "universe" - I can make the flat and tautologous declaration that reality is what it is, and behaves as it does, and that the notion of probability has nothing to offer that clarifies that tautology.

I then follow that up with the observation that if we want to understand and predict reality's features and behaviours then we need to label those features and behaviours, and in order to prevent those labels from exploding into unmanageable multitudes we will often need to use the same labels for pieces of reality that might not in fact be identical, just near enough to be going on with.

If we choose good enough labels then we can achieve a labelling that means we see such pieces of reality as we label the same way exhibit behaviours that are also the same. We can then make a reasonable claim to have understood whatever parts of reality have been so labelled, and we can expect that predictions we make about their behaviour will turn out to be usefully accurate.

Again, probability doesn't really enter into any of that.

One of the most useful reasoning tools we have is the idea of causality. Causality applies to patterns of behaviour where we can confidently assert that thing A happens because of things B, C, D and E. What that means in practice is that if we recognize B, C, D and E happening then it's a pretty solid bet that we're going to see A happen as well.

When all you have is a hammer then everything looks like a nail, and causality is such a useful hammer that many people don't even bother questioning whether or not it's universally applicable. I don't think it is - not by a long chalk - but quite alarming numbers of people do seem to; when surprised by something happening out of the blue, these people will go "well, there must have been some reason for that" and go on to construct elaborate and often completely non-testable stories about how whatever it was came about.

The idea that everything that happens has causes is super comforting: it allows us to feel that even stuff we're manifestly incapable of controlling is under some kind of control and therefore seemingly less likely to blindside us. It's so comforting that many, many people use it to construct an overarching Architecture of Everything inside their own heads and then mistake that architecture for the reality we all inhabit.

So when faced with the success of quantum theory - a model of reality that flatly rejects the idea that all events have causes, and only ever makes probabilistic predictions about which events will occur and which will not - there is inevitable cognitive dissonance. People are so attached to the idea of causality as a universal organizing principle that they will insist on creating "interpretations" of quantum theory that complicate it way more than it needs to be.

MiraK refers to a worldview many people hold, that says

there is an underlying "real" reality that can be discerned if only we had the tools to measure all of the factors and variables involved the same way we can predict whether a coin will land heads or tails if we know the exact forces acting on it during a toys like momentum and velocity of spin and gravity and wind disturbance and so on, and the only reason we use probabilities to understand outcomes is because we don't have those measurements

From my point of view, where I'm content to let the word "reality" label the totality of what is and was and will be regardless of whether or not I or anybody else understands it, this looks to me like a position that could only be held by somebody who has mistaken their personal causality-based worldview for reality itself. Further, it seems to me that what they're doing in this instance amounts to trying to reserve a place to slide a yet-to-be-formulated theory based on that worldview in "underneath" quantum theory.

To me this has long seemed a doomed quest. Quantum theory is the best description we have of what is and of what happens on the finest scales we can measure. As for things we can't measure: whatever we say about those amounts to Just So stories - pretty and comforting, but it doesn't pay to give them too much credence.

The way I personally prefer to reconcile quantum theory's enormous success with its fundamentally probabilistic nature is not to tell myself stories about how reality actually is, but instead conclude that causality is not a universally applicable organizing principle, merely an extremely useful one. This locates the architecture of my worldview back inside the realm of my own reasoning, where I think it properly belongs, rather than projecting it outward onto everything else. If my predictions disagree with reality then it's my observations and/or conceptual models that are wrong, not reality. Having accepted that, there's no cognitive dissonance involved in accepting that not only sometimes do things just happen: on the finest scales it's most of the time, always and everywhere, and that's OK.

But again, this cannot justify a statement that "reality itself is actually probabilistic". To my way of thinking the only attribute one can confidently ascribe to reality is that of being real: i.e. susceptible to being experienced (and if we're skilled or lucky or both, perhaps even measured).

As soon as we say anything else about reality as a whole, we're back in the realm of theory. Claims that reality is this way or that way - probabilistic or deterministic or made of energy or made of information or made of God's thoughts or whatever - are frameworks for models, not definitive statements about whatever it is that we all find ourselves living in. The only certainty I can have is that something is going on. All else involves various degrees of guesswork.

In summary: anything we can say about reality necessarily involves having conceptual models of its parts and their relationships, and the claim that "reality itself is actually probabilistic" is less confusingly, less pithily and far less arrogantly re-cast as "our best available understanding of what can and will happen on the finest scales is demonstrably probabilistic now, and appears very likely to remain so indefinitely".
posted by flabdablet at 12:53 AM on November 1, 2021 [6 favorites]


Sean Carroll did a series on YouTube last year called The Biggest Ideas in the Universe that covers the question you're asking, among many other aspects of physics. One of the best things about it is that includes a simplified version of a lot of the math that demonstrates the reasons why the modern consensus is what it is and how it all fits together.

He's also really good about noting underlying assumptions like those raised by flabdablet and discusses various possible philosophical interpretations of what it all might mean.

It really helps in clarifying why the probabilities don't really have anything to do with the measurement problem. It's not about measurement affecting the system so much as fundamental limits of what can be known about a quantum system no matter how careful you are to ensure the measurement doesn't affect the system. (or at least the property being measured)
posted by wierdo at 1:15 AM on November 1, 2021 [6 favorites]


fundamental limits of what can be known about a quantum system

or possibly even fundamental limits on the extent to which selectively ignoring relationships between any thing and everything it isn't can actually improve the success rate of predictions about its behaviour.
posted by flabdablet at 3:58 AM on November 1, 2021


The question you ask rose to be the most important outstanding issue in physics sometime in the 19290s. The best known one-liner was Einstein's:.

The theory produces a good deal but hardly brings us closer to the secret of the Old One,' wrote Albert Einstein in December 1926. 'I am at all events convinced that He does not play dice. '
posted by SemiSalt at 5:14 AM on November 1, 2021


You might also want to check out the first lecture in MIT's intro quantum mechanics class (starts with a lot of course business but gets to work around 10:00), which does an excellent job laying out the experimental facts you've got to deal with, using almost no math. The whole point of the lecture is to reorient folks to a worldview that can deal with these facts. He summarizes the perspective really well around 32:30 -- "The miracle is not that electrons behave oddly. The miracle is that when you take 1027 electrons, they behave like cheese."

The turning point, I think, was the one-two punch of EPR and Bell. The EPR paradox is only a paradox because its consequences are too weird to stomach. Then Bell arrives with a testable experiment that shows that yes, it's exactly that weird and we need to get used to it.
posted by range at 5:47 AM on November 1, 2021 [4 favorites]


In the 1920s in Copenhagen, Bohr and Heisenberg gradually developed a way of thinking about quantum mechanics as a theory about the physical reality of the universe rather than just a kind of math. This is the interpretation that Einstein didn't like ("God does not play dice") and that Schrodinger invented his cat for and so for the past hundred years, people have been trying to come up with alternatives.

The most comforting thing to believe is that even if you can't tell what state an electron is in, it is "actually" in some non-probabilistic hidden state. As mentioned above, Bell's theorem states that this is unequivocally false. So if you want you can say, 1964 is when physicists were forced to accept that the probabilistic nature of quantum mechanics isn't just a barrier between us and some ultimate reality but is in fact the ultimate reality we live in.

Now Bell's theorem doesn't rule out nonlocal hidden variable theories...but these are by far the minority view among physicists today.
posted by goingonit at 8:03 AM on November 1, 2021 [4 favorites]


The most comforting thing to believe is that even if you can't tell what state an electron is in, it is "actually" in some non-probabilistic hidden state.

For me, the most comforting thing to believe is that quantum particles such as electrons only ever resemble objects that we would normally think of as particles - that is, little lumps of stuff - when they're interacting with other things in ways that have causal consequences. Between interactions, I like to think of electrons as tendencies - propagating temporary imbalances or perhaps even loans of order, if you like - in the electromagnetic fields that can reasonably be said to have existence between events of causal particle interaction (and likewise for the fields associated with other kinds of particle).

So the way I see it, the reason electrons appear to behave paradoxically in experiments like those outlined in the MIT lecture that range linked above is that people intuitively conceive of them as having the same kind of object persistence, identity and locality properties as the little billiard balls that intuition interprets particles as.

But if you get rid of that intuition and learn to work with the idea that one simply cannot say not only what a particle is doing but even what properties it can reasonably be said to have - including the fundamental property of existence as a distinguishable entity - except when it interacts causally with something else, then quantum theory starts becoming a whole lot less indigestible. A particle then becomes a bookkeeping quantity (when you emit one over here then it's gonna end up absorbed, later, probably over there somewhere) rather than anything like an actual physical subsystem, and the book entries are exactly those particle-particle interactions that it does make sense to reason about using the when-this-then-that of causality.

The whole question of what attributes (including the fundamental attribute of identity) it is reasonable to ascribe to particles as they interact is also conceptually very interesting. Again, billiard-ball intuitions from classical physics are completely and actively unhelpful here, I think.

My personal intuitions about this stuff rest on the idea that everything that can happen does happen - not in some tree of parallel universes with a literally inconceivable branching rate, but right here, right now. There's a lot of cancelling out, and particle-particle interactions happen in exactly those places that don't cancel out.

But my brain is entirely a layman's too. I am in no way qualified to offer an authoritative opinion on any of this stuff, so if those better informed disagree with anything I've said here then you should believe them, not me.
posted by flabdablet at 9:15 AM on November 1, 2021 [1 favorite]


The theory produces a good deal but hardly brings us closer to the secret of the Old One,' wrote Albert Einstein in December 1926. 'I am at all events convinced that He does not play dice. '

I'm convinced that He does, but only once, and it's the only game in town, and all any of us can ever do is enjoy the pretty patterns they make as they roll because they never gonna land.
posted by flabdablet at 9:26 AM on November 1, 2021 [1 favorite]


It's worth noting that the Copenhagen Interpretation mentioned above is frequently summarized and/or lampooned as the "shut up and calculate" school of quantum mechanics. This is part of what bugged Einstein et al -- quantum mechanics, at least for now, demands that we accept that there's a hard limit to our ability to ask why something happens. Eventually the answer is "I don't know, they just do that" and there is no conceivable experiment that would distinguish one why hypothesis from another.

Although we've found a way to model what happens with staggering accuracy, we're still just answering what questions, not why. If you got into physics because you were hoping to discern the True Face of the Cosmos, that's kind of a letdown.
posted by range at 9:42 AM on November 1, 2021 [1 favorite]


Re the Copenhagen Interpretation being "shut up and calculate", one other thing to add is that since the 20s we've built machines for "shutting up and calculating" and these machines have turned out to have some remarkable properties, among them an equivalent level of computational power that's independent of their physical realization. This has led some people (particularly computer scientists who wind up exposed to quantum mechanics, of whom there are many more today than even a decade ago) to think that the computation is the "real" part and the physical realization is a convenient mnemonic device to connect it to our phenomenology. Stephen Wolfram wrote a book that's kind of about this. For a computer scientist, I think the idea that reality "is" a vector in Hilbert space being evolved by the Hamiltonian operator is easier to accept than for a physicist.

Anyway when you hear people debating "is reality a simulation?" they are more or less talking about the above question, namely whether the physics of our universe can be fully described by a Turing-computable algorithm and the implications if so.
posted by goingonit at 11:31 AM on November 1, 2021


quantum mechanics, at least for now, demands that we accept that there's a hard limit to our ability to ask why something happens. Eventually the answer is "I don't know, they just do that" and there is no conceivable experiment that would distinguish one why hypothesis from another.

The expectation that "why" is a question reasonably applicable to every observable event does appear to be fundamental to many people's worldviews. For me, the most interesting "why" question is why that is so; I suspect, though I can't be sure, that that one might actually have some useful answers.

For a computer scientist, I think the idea that reality "is" a vector in Hilbert space being evolved by the Hamiltonian operator is easier to accept than for a physicist.

This will be because computer scientists, in aggregate, spend even less time outdoors than physicists :-)

There does come a point where "in-principle" hypotheses about the computability of physics become so esoteric and so divorced from any prospect of being tested as to amount to arguments over how many angels can dance on the head of a pin, and the ensuing strong convictions that reality is actually an instance of this or that or the other kind of pattern assume the structure and function of religious faith.

I prefer to think of pattern recognition itself as a kind of lossy compression step in the process of representing reality that goes on in our brains. Patterns are things we find and make, and we do that by ignoring whatever aspects of reality don't fit into them. Experimental physics is a discipline that rests largely on learning what can feasibly and usefully be ignored. I also think that the huge dopamine hit one gets from discerning a widely applicable pattern, especially a genuinely novel one, is a big part of the reason why so many people like to imagine that the Ultimate Pattern is a thing they might discern some day.
posted by flabdablet at 10:58 PM on November 1, 2021


« Older Restaurant recommendation in Nashville for a gift...   |   Book recommendation: not your typical MBA... Newer »
This thread is closed to new comments.