The Role of Theory in Understanding and Resolving the Reliability Crisis

In the behavioral and social sciences especially, theoretical ambiguity is particularly acute due to the complexities of human interaction and societal organization. The goals of this proposed project are to address the potential role of theory in the reproducibility crisis. Specifically: (1) Test the extent of theoretical ambiguity in an area of study. (2) Test if computer-assisted comparison of causal models can identify the specific sources of empirical unreliability in that area. (3) Test if crowdsourcing theoretical claims can improve reliability and the replicability of findings in that area. (4) Develop computer systems that improve and economize this process so that tests and solutions can be deployed across cognitive, behavioral and social sciences.