Now that we have superfast computers capable of storing and processing a near-infinite amount of data, you might think that given enough information, we could predict the results of anything far into the future. Some would even argue that is the purpose of science and the greatest promise of computing.

However, as the 20th-century philosopher Karl Popper articulated, there is, in fact, a hard cap on our ability to predict the future. The key is the growth of knowledge, the generation of new ideas. The proof is strikingly simple and profound.

  1. Definitionally, we cannot predict the content of our next idea, be it technological innovation, political ideology, or scientific discovery. The reason simple. To predict the next idea, we have to know what that idea is. Therefore, in predicting it, we will have discovered it and it will cease to be the next idea.

  2. Ideas affect the future events of the world, as much, if not more than seemingly stable factors like demography and resources. New technologies change how we use resources. Political ideologies change the demographies of nation-states. Scientific discoveries change our fundamental understanding of the world and what’s even possible.

  3. Therefore, because we cannot predict the content of ideas and those ideas affect the outcome of future events, we cannot predict those future events.

As an example, imagine viewing the world from the perspective of someone living in the year 1900. Imagine you’re trying to predict what will happen over the next 100 years - where will humanity be in the year 2000, (plausibly within the lifetime of someone born that year). To do so, you would have to understand the impact of technological changes like the transistor that you did not even know were possible.

You would also have to predict the impact of political ideologies like Nazism, Soviet-style Marxism and liberalism, and their conflicts. These ideologies even radically changed seemingly stable factors like the demography of countries with deaths from war and famine.

You would also have to predict the impact of scientific hypotheses that had not yet been discovered, like E=mC2, and their resulting technologies like nuclear power.

We can see from our perspective in the 21st century that doing so is clearly impossible. But the point of Popper’s proof is that, when trying to predict the future, we’re always doing so through the same veil of ignorance as the person in 1900.

The fact is, we should expect that in the next 100 years we will experience massive changes to our current world and ideas, just like the last 100 years. We just don’t know, and cannot know, what those changes will be.

So what does that mean?

First, it means we should obviously be very skeptical about long term predictions. While for short term predictions, those anticipating the next few months or years, the likelihood of radically new ideas emerging is low, over a long enough time horizon, radically new ideas almost guaranteed. (My rough and ready heuristic is any prediction that’s longer than 10 years is useless.)

Second, we shouldn’t focus exclusively on predictions. Some people have made the mistake of assuming that the only way to think clearly, logically, and scientifically is to make predictions. But that’s not so. Science generates explanations. Understanding those explanations helps us solve problems and create new knowledge. Newtonian physics was discovered in the 1600s, but it’s still useful today for everything from building rockets to designing bridges. Economics may not be able to predict when the next recession will happen, we can still use its tools to resist depressions.

Finally, and most importantly, we should not get distracted by predictions about the future. Rather, we should work to create a better one.