Correlation does not equal causation!

Oh really, now.

First of all, there is no such thing as a "non debatable fundamental truth (an observation that happens 100% of the time with no failure)". For example, in some circles of physics even the "fundamental truth" of gravity is flawed, because it fails relative to so-called "known" fundamental conditions of quantum mechanics. Gravity itself is still only a scientific theory, of which statistics form an integral component for observable and theoretical characteristics to form high probability hypotheses.

And being a physicist, you should have known that many experiments deal with complex or difficult to measure phenomena. As such, determining what a sample set of observations tells us about a proposed explanation or hypothesis in general requires us to make an inference, or as we statisticians call it, to "Reason With Uncertainty". You perhaps have worked with at least one such statistical method, such as the method of least squares (or should know it), which is commonly used by scientists to minimize errors in data measurement.

Reasoning with uncertainty is the core of statistical inference and is typically done using a method called Null Hypothesis Significance Testing (NHST). I agree that undeviating replication IS required in order to make a firm conclusion, however almost all pure science is theoretical - precisely due to variances in empirical data and limitations of observation. Also generally known as probability analysis, this is covered in Physics 101.

[History lesson] The use of statistical methods dates back to at least the 5th century BCE, when the Romans used it for collating censuses of the population for taxation. The earliest writing on statistics was "Manuscript on Deciphering Cryptographic Messages", by Al-Kindi which gave a detailed description of how to use statistics and frequency analysis to decipher encrypted messages. An excellent summary of the essential role math and statistics played throughout history in science is "Lost Discoveries: The Ancient Roots of Modern Science", by Dick Teresi (available on Amazon or other fine book stores near you).