Projects

Pilot Study of the Emerging Frontiers in Research and Innovation (EFRI) Program’s Monitoring System

Project Overview

The Emerging Frontiers in Research and Innovation (EFRI) Program promotes innovative, interdisciplinary initiatives in engineering research and education. EFRI pursues research projects that embrace new frontiers in engineering research and supports those projects that generate transformative ideas in areas where there is a demonstrated need. EFRI aims to create a new outcome monitoring system to streamline the data collection process and provide clear measures of how funded projects achieve their long-term goals after the funding period ends.

The National Science Foundation (NSF) contracted with Insight to launch a new survey on key program outcomes for past, present, and future EFRI grantees. To establish a valid and reliable survey, Insight evaluated EFRI goals and the logic model for the last survey, reformulated survey questions to meet program objectives, checked for redundancy with other NSF databases, tested survey logic through cognitive interviews, and pretested the survey instrument.

Insight completed the following tasks to establish a reliable survey monitoring system for EFRI grantees:

  • Refinement of the logic model, targeting measurable outcomes for the program
  • Review and refinement of the existing survey instrument
  • In-depth interviews with NSF staff members to identify existing data sources and key items for
    data collection
  • Cognitive interviews with a targeted sample of EFRI grantees and transcription and coding of
    completed interviews
  • Online data collection with EFRI principal investigators (PIs), co-PIs, and students
  • Collection of the first wave of survey results as well as an assessment of survey and item response rate

In May 2015, Insight delivered a final report on the survey process with recommendations for improvement. The final report included analysis of the first wave of survey responses from PIs, co-PIs, and students. A nonresponse bias analysis illustrated that the sample reflected characteristics of the EFRI population. Results indicated the majority of grantees begin partnerships with businesses after their grants end, typically to license their products. They also reported using their NSF EFRI funding and results to secure additional funding to refine their efforts and innovations further.

Core Activities

Qualitative and Quantitative Analysis; Cognitive Interviews; Web Survey; Data Collection; Report Development and Presentation

Products

  • Report on cognitive interview results (February 2015)
  • Online survey instrument (November 2014)
  • Database housing the first wave of EFRI grantee responses (April 2015)
  • Final report summarizing survey results and future data collection recommendations (May 2015)