We are pleased to announce that Insight Policy Research was acquired by Westat, effective June 15, 2022. Branded as “Westat Insight,” we are currently operating as a wholly owned subsidiary of Westat and continue to provide our clients with the best-in-class services they have come to expect. Our combined experience now offers expanded expertise in health, education, and social policy as well as deeper methodological skills in survey research, evaluation, data analytics, and technical assistance. Learn more here.
We are pleased to announce that Insight Policy Research was acquired by Westat, effective June 15, 2022. Branded as “Westat Insight,” we are currently operating as a wholly owned subsidiary of Westat and continue to provide our clients with the best-in-class services they have come to expect. Our combined experience now offers expanded expertise in health, education, and social policy as well as deeper methodological skills in survey research, evaluation, data analytics, and technical assistance. Learn more here.
powered by bulletin

White Papers, Issue Briefs, and Articles

Our white papers, issue briefs, and articles spotlight areas of improvement for social policies and programs, changing the way we see the world a little bit at a time.

Family Support

Enhancing Rigor, Relevance, and Equity in Research and Evaluation Through Community Engagement

This brief, authored by Megan Worden, Hilary Wagner, and Debra Wright summarizes key themes from the 2021 Office of Planning, Research, and Evaluation Methods Meeting on Community-Engaged Research. Not only is engaging communities in research and evaluation critical for implementing effective programs, community members can bring valuable input to program evaluation, improving the rigor and relevance of the research. Engaging communities in research and evaluation—especially when identifying research questions, designing and implementing the study or intervention, recruiting study participants, and interpreting findings—may lead to more relevant research questions, greater recruitment success, increased external validity, greater retention of participants, and enhanced understanding of findings (Viswanathan et. al 2004). In this brief we define community engaged research, discuss the motivation for incorporating this approach, and provide examples, tools, and strategies for implementing community-engaged research.

Download

Exploring Core Components Research in Social Services Settings

This brief, authored by Rachel Holzwart, Hilary Wagner, and Megan Worden, summarizes key themes from the 2020 Office of Planning, Research, and Evaluation Methods Meeting on using core components approaches in social services settings. Core components of effective interventions are the “essential functions and principles that define the program and are judged as being necessary to produce outcomes in a typical service setting” (Blase & Fixsen). Core components approaches can complement traditional social policy evidence building, provide information needed to successfully adapt programs across a variety of conditions, and determine which intervention elements are critical to achieving desired results and delivering benefit to participants.

Download

Methods, Challenges, and Best Practices for Conducting Subgroup Analysis

This brief, authored by Insight’s Andrew Breck and Breanna Wakar, discusses the key features of a well-designed and implemented subgroup analysis. Subgroup analysis enables researchers to determine whether policy or program impacts vary across groups. While the brief focuses mainly on a multiple regression framework, it also provides an overview of alternative approaches to conducting subgroup analyses.

Download

OPRE 2019 Methods Meeting Resources List

This resource list, compiled by Insight’s Dani Hansen and Rachel Holzwart, documents references, tools, and websites for readers wanting to learn more about open science methods. “Open science” represents a broad movement to make research, data, and findings more transparent, accessible, and replicable throughout the research process. This document highlights federal initiatives and policies that support open science, the process of registering study plans in a repository before beginning a research project, reproducibility and replicability, and managing and sharing data.

Download

National Evaluation of HUD’s ConnectHome Initiative

The brief, written by Insight’s Carole Trippe and Brittany McGill, summarizes the results of one of three components of the National Evaluation of ConnectHome, a collaborative effort of communities, the federal government, and the private sector to bring high-speed internet access to low-income families in HUD-assisted housing. The brief describes the Baseline Access Survey—which assessed the baseline level of internet access among families with children in 22 pilot communities prior to implementing the initiative. Insight examined the level of in-home internet access, the types of internet connection, the number and types of devices used (e.g., phone, laptop, tablet), the reasons for lack of internet access, and awareness of the ConnectHome initiative. Results show only about one-third of households have high-speed internet access at home. Another third are “underconnected,” relying, for example, on a cell phone data plan for in-home internet access, and one-third have no internet access at all at home. Most households without internet access report cost as the primary reason.

Download

Methods for Promoting Open Science in Social Policy Research

This brief, authored by Rachel Holzwart and Hilary Wagner, summarizes key themes from the 2019 Office of Planning, Research, and Evaluation Methods Meeting on promoting open science in social policy research. “Open science” is a broad movement to make all phases of research—from design to dissemination—more transparent, accessible, and replicable. Open science methods have gained momentum as highly publicized news stories related to data manipulation (e.g., p-hacking), publication bias (e.g., no publication of null results), and inability to replicate or reproduce research results have cast doubt on research credibility. Proponents of open science strive to transform the research culture using a range of methods, such as preregistering evaluation plans and providing open access to code and data, to encourage open sharing of research information and enable researchers to verify and build on one another’s work.

Download

How Even the Best Evidence Can Yield Bad Decisions, and What We Can Do About It

While evaluation evidence is the best evidence for identifying “what works,” it is still imperfect. In evidence-based policymaking, the imperfections create a significant risk we will implement programs that do not work, and we will suffocate programs that do—or at least can—work. Numerous factors, including limitations of external validity, an overreliance on p-values and hypothesis testing, underpowered research, and even our intolerance for false positives can lead to incorrect conclusions. This paper discusses various ways evidence-based decisions can still be bad decisions. The paper identifies two trends that can address these shortcomings: the use of Bayesian statistical methods and continuous quality improvement. The paper concludes with six recommendations for strengthening evidence-based policymaking.

Download

Understanding Rapid Learning Methods: Frequently Asked Questions and Recommended Resources

This brief, authored by Rachel Holzwart, Robbie Skinner, and Debra Wright, highlights a list of frequently asked questions about rapid learning methods. The list includes links to resources for further information on topics such as the purpose of rapid learning methods, which rapid learning methods are appropriate for which contexts, and practical resources to help programs implement these methods. If you’ve been wanting to learn more about rapid learning methods and their application in social service settings, this is a great place to start.

Download

Rapid Learning: Methods for Testing and Evaluating Change in Social Service Programs

This brief, authored by Rachel Holzwart and Hilary Wagner, summarizes the 2018 Office of Planning, Research, and Evaluation (OPRE) Innovative Methods Meeting discussions on rapid learning methods (e.g., rapid cycle evaluation and continuous quality improvement). It describes considerations when selecting these methods, how researchers have successfully used them, and how to build a lasting culture of learning within an organization.

Download