Do all observations count equally in testing scientific models?

The third hallmark of science may make you wonder about what counts as an “observation” against which a prediction can be tested. Consider the claim that aliens are visiting Earth in UFOs. Proponents of this claim say that thousands of eyewitness reports of UFO encounters provide evidence that it is true. But should these personal testimonials count as scientific evidence?

On the surface, the answer isn’t obvious, because all scientific studies involve eyewitness accounts on some level. For example, only a handful of scientists have personally made detailed tests of Einstein’s theory of relativity, and it is their personal reports of the results that have convinced other scientists of the theory’s validity.

However, there’s an important difference between personal testimony about a scientific test of relativity and a UFO. Although it might require many years of study before you had the necessary background to conduct a test of relativity for yourself, you could in principle then confirm the results reported by other scientists. In other words, while you may currently be trusting the eyewitness testimony of scientists about relativity, you always have the option of studying enough so that you could verify their testimony for yourself. In contrast, there is no way for you to verify someone’s eyewitness account of a UFO.

Scientific tests of a model only make sense when we test the model’s predictions against observations that anyone could in principle verify. That is why something like a test of relativity counts, while a simple eyewitness report does not.