We can confidently and successfully predict the future under some circumstances. Thus it is not the case that, starting from the current state of affairs, anything at all can happen next. Put another way, the future is at least constrained or partially determined.
We can also be sure that the extent to which the future is constrained exceeds our current ability to discern it. Our attempts at prediction are always hampered by casually incomplete knowledge, that is, knowledge that we could possibly obtain but have not. Incomplete knowledge comes in at least two flavors, the first of which relates to facts about the present state of affairs (such as the exact locations of objects of interest). In these cases, we know what kind of information is needed, how to collect it, and how we would use it to refine our predictions. We simply have not done so.
Our failure to collect such information may not be due to indolence. As finite beings we have finite resources, whereas the number of possible facts about the present is very large or perhaps unlimited. Further, these facts change over time, adding to the challenge of gathering temporally consistent measurements. As our objects of interest become smaller, there are more of them and the precision with which we might like to measure their state grows finer. Meteorology provides an excellent example: the more granular our measurements of temperature and wind speed, the more accurate our short-term forecasts, but the more effort is required to collect the data. Nevertheless, we can envision that such information could be collected by, say, the application of some appropriate technology. Because we can determine with apparently arbitrary precision a small set of the desired facts, we have good reason to believe that similar facts exist regarding those that we did not collect. In other words, these facts about the present exist but are merely unmeasured.
The second flavor of casually incomplete knowledge relates to the rules or patterns by which events transpire. To the extent that we are unable to predict phenomena accurately, we tend to suspect that there are new conceptual organizations, causal chains, formulae, and the like that will enable us to improve such predictions. Our inductive evidence for this is that such discoveries are made every day. Even in fields with extremely complex agents and interactions, such as human individual and social behavior, we seek and sometimes find patterns that can enable successful prediction.
The importance of casually incomplete knowledge to our present topic is that it greatly expands the extent to which the freedom of the universe is constrained, as compared with what we can actually predict. In the wake of the discovery of Newton’s laws, this led to the notion of the “clockwork universe” and eventually to a scientific determinism:
We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes. (LaPlace, A Philosophical Essay on Probabilities, 1825)However, subsequent scientific discoveries showed that there are areas of inherently incomplete knowledge. The Heisenberg uncertainty principle states that the combined position and momentum of a particle cannot be measured beyond a particular accuracy. Thus, with respect to the first flavor of incomplete knowledge, we are limited not only by our ambition and resources, but also by an intrinsic feature of how the physical universe operates.
The development of chaos theory showed that in certain types of systems, small differences in initial conditions result in highly divergent outcomes. In such systems, any lack of precision in measurement of the present state of affairs eventually results in considerable prediction error. This creates a limit on the second flavor of incomplete knowledge, because it means that the conceptual schemes under which we see convergent, predictable behavior are oversimplified with respect to such systems. Such a limit is more extensive than it first appears: given enough time, all systems are chaotic, because none of them are genuinely isolated from the rest of the universe.
Combined, the uncertainty principle and chaotic dynamics make perfect prediction even theoretically impossible. This evidently has no bearing on whether there is a fact of the matter about the future. It does, however, mean that a claim that the universe is or is not deterministic cannot be tested.
The only tool we have available to us for determining the validity of theories is to make predictions and observe whether or not they hold true. But prediction is the very capability that we have just ascertained to exhibit an inherent shortfall. No matter how a theory of determinism or freedom is posed, and what verification or falsification criteria we specify, we cannot distinguish between failures to predict that are due to inherent knowledge limitations, and those that are due to a genuine under-determination of the future.
It is possible that there is something important that we do not understand about the uncertainty principle or chaotic dynamics. We could also discover a method of validating theories that does not involve prediction. Otherwise, the question of freedom, regarding the universe and a fortiori with respect to individuals, is solidly in the realm of metaphysics. It is a question that cannot be answered empirically and about which any beliefs are tantamount to faith.