An application of Markov chain Monte Carlo (MCMC) methods in alcohol research: item parameter recovery for the Protective Behavioral Strategies Survey

Purpose: This study was motivated by the measurement challenges of Project INTEGRATE, a large-scale synthesis study of aggregate data and individual participant data (IPD) from brief alcohol intervention trials for young adults. Methods: The present study utilized Markov chain Monte Carlo (MCMC) methods to help address the measurement challenges using the Protective Behavioral Strategies Survey. We aimed to recover item parameters for a two-parameter logistic item response theory (2PL-IRT) model. We tested the viability and feasibility of the custom-developed MCMC algorithm in R under study conditions that varied the number of items (J=5, 10, 20 and 40) and sample size (N=300, 500 and 1000). For each condition, 25 replications were conducted. We evaluated the accuracy of parameter recovery based on the mean bias and root mean square error (RMSE). Results: The MCMC algorithm for the 2-PL IRT model adequately recovered item parameters: for J=5 items and N=300, the bias and RMSE of the item discrimination parameter were -.039 and .073, respectively; and of the item severity parameter were .033 and .065, respectively. As the number of items and sample size increased, both item parameters were more accurately estimated. Conclusions: We presented the outcomes of MCMC methods for a 2PL-IRT model in the recovery of item parameters as the first step toward obtaining commensurate latent trait scores for participants from multiple studies and ensuring the same data interpretation across multiple studies for IPD meta-analysis or integrative data analysis.