StatQuest: Bootstraping

During the "Numerical Methods I" course together with students at Faculty of Physics we talk about experimental data fitting. Matlab's polyfit function will cover most common use cases, but in Physics we are often interested not only in the point estimates of measurements. We also care about associated errors and the documentation of polyfit function fails to deliver on that. Unless you are well versed and statistics and know that you need to calculate covariance matrix and take square root of values on its diagonal.

I was caught of guard by the students who were interested in actually estimating the measurement error using polyfit. As at the time I wasn't prepared, I have told them about Bootstrap method, which I like a lot. I like this method, because it has natural interpretation (it gives us a confidence interval) and is applicable even in highly complex situations, for which no analytical error estimation formulas exist.

More about this method in the following video by StatQuest with Josh Starmer.

eagereyes: The DataSaurus

Summary statistics, whether mean, standard deviation or any other single number, are useful, but sometimes they can mislead - vastly different data sets can have similar or even identical summary statistics. In the video below Robert Kosara discusses this phenomenon in more detail.

Thus here we give you an opposite advice to the one we gave previously. You should plot the data to see whether summary statistic you have obtained make sense.

Well, you should actually always do both things - visualize the data and do some tests or summarization. If intuition from the visuals and the numbers differs, then you should look for things that may have gone wrong.

Kolmogorov-Smirnov test

Last time we have explored the distribution of increments obtained from the interpolated Brownian motion. We have seen that the correct interpolation formula generates proper increments. Yet we have seen that with just our eyes. While the eyes are a good tool, they sometimes can lie. Thus it is wise to use a formal method to verify our intuition.

Here in this post we will discuss one of the simplest methods to verify whether data follows certain distribution - Kolmogorov-Smirnov test.