We are pleased to announce that Insight Policy Research was acquired by Westat, effective June 15, 2022. Branded as “Westat Insight,” we are currently operating as a wholly owned subsidiary of Westat and continue to provide our clients with the best-in-class services they have come to expect. Our combined experience now offers expanded expertise in health, education, and social policy as well as deeper methodological skills in survey research, evaluation, data analytics, and technical assistance. Learn more here.
We are pleased to announce that Insight Policy Research was acquired by Westat, effective June 15, 2022. Branded as “Westat Insight,” we are currently operating as a wholly owned subsidiary of Westat and continue to provide our clients with the best-in-class services they have come to expect. Our combined experience now offers expanded expertise in health, education, and social policy as well as deeper methodological skills in survey research, evaluation, data analytics, and technical assistance. Learn more here.
powered by bulletin
Careers Contact Us
Menu

Projects

703-504-9480

Merit Review: Assessment of Investigator and Reviewer Experiences

Download

Project Overview

Insight is leading an evaluation of the National Science Foundation’s (NSF) Merit Review Process. Principal investigators (PIs) submit approximately 50,000 proposals to NSF each year, and reviewers evaluate each one against two main criteria: intellectual merit and broader impacts. The goal of this study is to conduct a biennial web survey of PIs’ and reviewers’ experiences with NSF’s review process, with particular focus on burden, satisfaction, and quality. Each survey also includes an examination of new pilot programs aimed at increasing the quality and efficiency of the review system that year.

Survey results have indicated one important component of the merit review process is the quality of written reviews. Recently, NSF set an annual goal to improve the quality of written reviews of NSF proposals but has no way to assess the quality of written reviews. As a secondary component of this study, Insight is—

  • Developing, validating, applying, and automating a tool to assess review quality
  • Using the new review quality tool to assess the impact of several previous pilot approaches on review quality using difference-in-differences analyses
  • Testing the effect of several new, incremental changes to the merit review process using a Plan-Do-Study-Act approach
  • Conducting value stream mapping to identify possible strategies for streamlining the merit review process to reduce dwell time

Products

Final reports, Merit Review—Assessment of Investigator and Reviewer Experiences (2017); Merit Review—2017 Assessment of Investigator and Reviewer Experiences (2018); Merit Review—2019 Assessment of Investigator and Reviewer Experiences (anticipated 2020)

Back to Project Listing