While evaluation evidence is the best evidence for identifying “what works,” it is still imperfect. In evidence-based policymaking, the imperfections create a significant risk we will implement programs that do not work, and we will suffocate programs that do—or at least can—work. Numerous factors, including limitations of external validity, an overreliance on p-values and hypothesis testing, underpowered research, and even our intolerance for false positives can lead to incorrect conclusions. This paper discusses various ways evidence-based decisions can still be bad decisions. The paper identifies two trends that can address these shortcomings: the use of Bayesian statistical methods and continuous quality improvement. The paper concludes with six recommendations for strengthening evidence-based policymaking.