ANALYSIS OF IMPACT SCORES OBTAINED AS A FUNCTION OF REVIEW CYCLES (FROM JUNE 2010 TO JUNE 2012 REVIEW CYCLES)

Purpose: To examine the trend of scientific quality of research grant applications submitted to AHRQ with time, we assessed the overall impact scores provided by each of 5 standing study sections as a function of times during the past 7 review cycles (from June 2010 to June 2012). Methods: As at the NIH, the Scientific Peer Review Committees at AHRQ use 9 points overall impact score system to evaluate the scientific merit of research grant applications submitted to AHRQ for funding opportunity. This analysis is based upon the enhanced score criterion http://grants.nih.gov/grants/guide/notice-files/NOT-HS-10-002.html using the final impact scores assigned to the applications that were discussed at the review meetings from five study sections and Special Emphasis Panel (SEP) that were reviewed from June 2010 to June 2012 (a total of 7 review cycles). The data used in this analysis was from the National Institutes of Health (NIH) Query View & Report Database (QVR system). Microsoft Excel and SPSS were the tools used for analysis. Results: The means impact scores trend to increase with time in terms of review cycles (from June 2010 to June 2012). The means impact scores trend to increase with time in terms of review cycles (from June 2010 to June 2012). For example, for the HCRT, HQER or HSR study section, the means impact scores decrease to a lowest values of about 31-35 the exception of February 2011 review cycle meeting. The means of overall impact scores gradually increased to higher values at the subsequent review cycles to about 38 to 53 for all 5 study sections. The means for the June 2010 cycle ranged from 36.26 - 40.76 and the means rose to upward trend for the June 2012 cycle at 38.48 - 53.46 range showing an upward trend. Although the mean values of impact scores implicated an upward trend from June 2010 to June 2012, there was no statistical significance among these mean values. Conclusions: Our analysis of impact scores among study sections found no statistically significant differences, indicating that the AHRQ peer review process is consistent in terms of quality of the scientific review among study sections. Our findings could provide useful information to the AHRQ leadership team, as well as the extramural health services research community regarding the scientific peer process for grant applications submitted to AHRQ for funding opportunity.