What Works Clearinghouse: Statistics, Website, and Training (SWAT)

The What Works Clearinghouse (WWC) is a leading federal source of evidence-based information about education programs, policies, and interventions that show promise for improving student outcomes.

The WWC is the standard-bearer for identifying high-quality studies and reliable, evidence-based solutions. AIR supports the WWC as its leading contractor and partner in distributing this valuable information.

Since becoming the the Statistics, Website, and Training (SWAT) contractor in 2018, AIR has assisted the Institute of Education Sciences (IES), in the U.S. Department of Education, in methodological discussions, training, and dissemination, and helps facilitate deliberations about the WWC Standards and Procedures across all WWC contracts.

These projects included:

Updating the WWC Standards and Procedures Handbooks

The WWC has published multiple updates to the Standards and Procedures Handbook since AIR became the SWAT contract.

  • In January 2020, the WWC published Version 4.1 of the Standards and Procedures Handbooks. The updated Handbooks included new procedures for synthesizing findings in WWC products using meta-analytic techniques, estimating intervention effects in single-case design studies, creating effectiveness ratings for Intervention Reports, and other statistical and methodological innovations.
  • In late 2020, the WWC published a supplement to the Handbooks that derived several new effect size standard error formulas and updated previously outdated calculations.
  • In 2022, the WWC published the draft Version 5.0 of the Handbook with plans to finalize the release in the summer of 2022.

Including ESSA Evidence Tier Tags and Data

Prior to the 2018, the WWC’s website and products did not include ESSA Evidence Tier information. Since becoming the SWAT contract, AIR has helped the WWC launch several ESSA Evidence Tier initiatives, including:

Revising Online Training Materials

The WWC’s rigor depends on the quality of the information selected from studies included within WWC products, a process the WWC refers to as “study reviews”. The WWC relies on individuals to conduct study reviews, so that the information can be translated and disseminated accurately. The individuals assigned to conduct the study reviews must therefore receive training to ensure high quality. AIR has helped advance this mission by:

Conducting a WWC Rapid Review of Evidence

The onset of the COVID-19 pandemic resulted in an immediate transition of in-person to online, distance learning. The Institute of Education Sciences called on researchers and educators to bring forth evidence-based programs and practices that could support the use of technology in online classrooms. AIR assisted the Department in conducting what IES referred to as the “first-ever cooperative rapid evidence synthesis.” Through this process, AIR assisted in the conduct of:

Maintaining and Updating the WWC’s Infrastructure

Ensuring the WWC is a primary resource for all education stakeholders requires a team of professionals who have relevant experience and expertise across numerous impacted sectors. In addition to the projects listed above, AIR also assisted in various other activities, such as:

  • Maintaining and updating the Online Study Review Guide and supporting documentation, which allows WWC reviewers to upload study review information in a reliable and efficient manner;
  • Responding to WWC Help Desk questions about various issues related to conducting study reviews, the systematic review process, and how to identify appropriate resources. Through March 2022, the Help Desk team has responded to more than 3,200 individual requests for information, assistance, or feedback;
  • Conducting peer-reviews of WWC products and study reviews, which ensures that the information listed on the WWC’s website and supported databases remains high-quality; and
  • After a series of discussions with education stakeholders, AIR assisted in the re-launching of the WWC’s website to bring enhanced navigation and user experiences online. More updates and enhancements are planned for 2022 and 2023.