Application of Sequential Probability Ratio Test for Reliability and Validity Studies

Suppose that a new blood pressure cuff has been created. Before this device can be accepted in the medical community, doctors would need to be sure that the new device is fairly consistent in its measurements and as accurate as the old blood pressure cuff. Likewise, to compare an old (standard) device with a new device, reliability and validity studies are performed in practice. In reliability and validity studies, a device is said to be reliable when it has low variance and a device is said to be valid when it has zero or small bias. Among many approaches for comparing two devices in the literature, the method of Bland and Altman analysis has been popular due to its simple calculation and interpretation under the assumption of normal distribution for the difference between two devices’ measurements. Note that researchers often collect data sequentially. In this chapter we present alternative statistical approaches to save researchers’ resources if possible. First, we present the application of a statistical method called the sequential probability ratio test which allows an early conclusion before reaching a fixed sample size. Next, we relax the normality assumption (which is not guaranteed to be satisfied in practice) by transforming an observed difference between two devices’ measurements into a Bernoulli random variable, and it provides a slightly different interpretation than the original Bland Altman analysis.