Tools of science: Induction and Occam’s razor

As philosophers are fond of pointing out, induction is logically unsound: no track record, however lengthy, of observing that swans are white can validate the conclusion that all swans are certainly white and that no-one will ever encounter a black swan. Yet science uses induction every day, and it works. Our sampling of information is always partial, and yet that partial information tells us enough about the world around us to generate highly successful predictions and to produce engineering and technology that works. One can thus ask on what basis science uses the principle of induction.

Some would argue that induction is an example of a basic assumption of science that cannot be further justified. They might claim that all “ways of knowing” depend on such unverified assumptions, that science is just one example of such a system, and that other assumptions can lead to equally valid domains of understanding, such as theology.

A scientist, though, would argue that tools of science such as induction are not arbitrary, but are themselves justified by science. The scientific method is itself the product of science, deriving from a long historical process of working out what works. Thus, by bootstrapping, science arrives at methods that produce good predictions about the world, and produce engineering and technology that works.

Black and white swans

Thus induction is verified by its demonstrated utility. However, science has long accepted that it cannot attain absolute certainty, and that it is always open to revision given new and unexpected data, such as a black swan. Thus the lack of a formal proof of the absolute reliability of induction is not a problem for science. Good-enough reliability is sufficient — all of science is like that.

Induction is thus a part of sampling theory, the well-tested and well-understood principle that a partial sample from a larger ensemble gives you reliable information about the larger ensemble. For example, an opinion poll of a few thousand voters gives reliable information about the forthcoming votes of millions. Statistics, the study of the information obtainable from such sampling and of how reliable it is, gives results that are always probabilistic, which is good enough for science.

To take an example, what is the likelihood that the sun will rise tomorrow, just as it always has? The logically minded philosopher will assert that you have no certainty of such an event, since the past history cannot guarantee the future. Yet everyone regards that extrapolation as so reliable that they entirely discount any other possibility. Is everyone wrong?

We have pretty good evidence that the basic laws of physics have remained constant for the past 5 trillion days (13.8 billion years) and that the world has been rotating on its axis and thus producing a sunrise for at least a trillion days. This sequence could come to an end in a “black swan” event; the laws of physics could suddenly change, or the Earth could collide with a wandering planet big enough to destroy it.

What are the likely time-scales for such events? The likely time-scales cannot be short compared to the past record, otherwise their non-occurrence in the past would be unlikely. Nor is it likely that the time-scales closely match the known record; that would be a highly unlikely coincidence. That means that, in all likelihood, the time-scales for such events are long compared to a trillion days. And that means that the likelihood of the world ending before tomorrow dawns is less than one-in-a-trillion. And that is a sufficiently small probability for us to ignore.

Another way of phrasing that is to imagine an ensemble of possible worlds, each with some span of time of stable laws of physics, followed by a change after some unknown time. Now pick out all the inhabitants of the ensemble whose world has stayed the same for at least a trillion days. Of these inhabitants, the overwhelmingly vast majority will live far from the end point. That’s because the number of unremarkable days in the middle of the sequence (trillions) vastly outnumber the number of “end” days.

Occam’s razor can be justified by a similar probabilistic argument. The principle says that one should only claim the existence of something for which there is good evidence. It’s a principle of parsimony, that one should not invoke entities unless they are demanded by evidence.

Occam's razor cartoon

The justification is that the number of conceivably possible entities vastly exceeds the number that actually exist. For example, there are tens of millions of species on Earth, but the number of possible species that could exist by permuting sequences of DNA is trillions of times vaster.

Thus, if you pick at random from the pool of “possible entities”, the chance that you hit on one that actually exists is vastly smaller than the chance of picking a winning lottery ticket. Thus, if you want to pick something that does exist you need to be guided to it by empirical evidence. If there is no such evidence then, in Christopher Hitchens’s version of Occam’s razor, “what is postulated without evidence can be dismissed without evidence”.

Occam’s razor is most precisely formulated in terms of information. One can consider long strings of 1s and 0s specifying some aspect of the universe. The particular information strings that specify our actual observable universe are an infinitesimal subset of the number of possible strings, which is infinite, and thus the chance of picking out by chance a string that matches our universe is negligible. Thus Occam’s razor tells us to adopt the minimum information-content demanded by the evidence, and to dispense with any that is superfluous.

The formulation in terms of information explains, for example, why the idea of a multiverse is not a violation of Occam’s razor. The information content of “take one universe, duplicate it a vast number of times, assign physical constants at random” is actually much less than the information needed to take one universe and specify all the physical constants. Thus a multiverse is a highly parsimonious concept.

9 thoughts on “Tools of science: Induction and Occam’s razor

  1. Neil Rickert

    Count me as a skeptic of inductionism.

    According to the story, Captain Cook (or a member of his crew) observed a black swan.

    That’s just wrong. What they observed was a black waterfowl of a hithertoo unknown species. It was not a black swan until they so named it.

  2. Pingback: The multiverse as a scientific concept | coelsblog

  3. Pingback: The Multiverse is parsimonious and not ad hoc? | Hugh Jidiette

  4. Pingback: A scientific response to the Brain in a Vat | coelsblog

  5. YF

    Great post as usual, Coel. I admire your razor-like ability to shave away unnecessary fluff and get to the core of the issues.

  6. YF

    Coel, I’ve been wondering how you would respond to the inductive skeptic’s point that the statistical arguments you presented above depend on the assumption that one has sampled ‘randomly’ from the population and that the sample is therefore ‘representative’. If we cannot know that our sample has been drawn randomly from the population (e.g., the ensemble of possible worlds) then the statistical arguments in favor of the reasonableness of induction cannot go through. Or so the critique goes. For instance, see here:

    Some remarks on the rationality of induction

    1. Coel Post author

      Hi Yon,

      If we cannot know that our sample has been drawn randomly from the population (e.g., the ensemble of possible worlds) then the statistical arguments in favor of the reasonableness of induction cannot go through.

      I’d reply along these lines. We can’t assume that the sample is drawn randomly, indeed we don’t really know anything about the sampling. But, we can consider the set of all possible samplings. And only certain rather special samplings would cause the argument to fail. Most of the possible samplings would effectively be sufficiently random. So, it is more likely than not that the sampling is sufficiently random and that the argument holds.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s