Bootstrapping – updated 2019-03-27
This description is extracted from the book, Nonlinear Regression Modeling for Engineering Applications: Modeling, Model Validation, and Enabling Design of Experiments, by R. R. Rhinehart, John Wiley & Sons, 2016, ISBN 9781118597965.
In regression, we adjust model coefficient values to best fit the data. But, uncertainty in experimental data leads to uncertainty on the regression model coefficient values, which leads to uncertainty in the model-calculated outcomes. We need a procedure to propagate the uncertainty from the experimental data to that on the modeled values, so that aspects of model uncertainty can be appropriately reported.
In linear regression, this is relatively straight forward, and mathematical analysis leads to methods for calculating the standard error of the estimate and the 95% confidence limits on the model. If the following conditions are true: 1) the model functional form matches the experimental phenomena, 2) the residuals are normally distributed (which assumes the experimental vagaries are the confluence of many, small, independent, equivalent sources of variation), 3) model coefficients are linearly expressed in the model, and 4) experimental variance is uniform over all of the range (homoscedastic), then analytical statistical techniques have been developed to propagate experimental uncertainty to provide estimates of uncertainty on model coefficient values, and on the model.
However, if the variation is not normally distributed, if the model is nonlinear in coefficients, if variance is not homoscedastic, or the model does not exactly match the underlying phenomena, then the analytical techniques are not applicable. My, relevant regression applications have those attributes. In this case, numerical techniques are needed to estimate model uncertainty. Bootstrapping is the one I prefer. It seems to be understandable, legitimate, simple, and is widely accepted.
Bootstrapping is a numerical, Monte Carlo approach that can be used to estimate the confidence limits on a model prediction. This r3eda site Bootstrapping Method 2016-06-14 provides a description. This r3eda Bootstrapping 2017-04-23 is a VBA application, and this r3eda site Bootstrapping File Directions 2016-06-14 is the description of how to use the Excel VBA file.