2 Methods

Our methods and reporting are informed by guidance from the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015) Statement (Moher et al., 2015), the Preferred Reporting Items for Overviews of Reviews (PRIOR) Statement (Gates et al., 2022), the PRISMA extension for reporting literature searches in systematic reviews (Rethlefsen et al., 2021), JBI guidance on umbrella reviews (Aromataris et al., 2020), and Cochrane guidance on overviews of reviews (Pollock et al., 2022).

2.1 Eligibility Criteria

We included systematic reviews with a meta-analysis examining the effects of school-based anxiety prevention interventions delivered directly to primary and secondary school students. We used the definition of a “systematic review” provided in the PRISMA-P 2015 reporting guideline: “a systematic review attempts to collate all relevant evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods to minimize bias in the identification, selection, synthesis, and summary of studies” (Moher et al., 2015). We used this definition to identify reviews that have the potential to provide reliable findings for evidence-informed decision-making. Based on this definition, our operational criteria for a systematic review were: (i) clear objectives, (ii) an explicit and reproducible methodology, (iii) a systematic search strategy for attempting to identify all studies that meet the review eligibility criteria, (iv) critical appraisal of the included studies, and (v) systematic synthesis of the characteristics and findings of the included studies. We excluded systematic reviews that did not report the results for a meta-analysis of school-based anxiety prevention interventions on at least one anxiety outcome (anxiety diagnosis, subsyndromal anxiety, or anxiety symptoms). We also excluded reviews on interventions delivered in school settings but only during out-of-school time (e.g., after school, summer break). Due to limitations in resources and review team language proficiency, we excluded reviews published in a language other than English. Although grey literature was eligible, we excluded reviews published only in a summary format (e.g., only as a conference abstract or executive summary).

For our meta-analysis of data from primary studies in eligible systematic reviews, we included evaluations of a school-based anxiety prevention intervention directly delivered to primary and secondary school students during normal school hours. We excluded studies that included pre-school or college students if less than 50% of the sample was primary and secondary school students. We used the National Research Council (2009) definitions for primary (universal) and secondary (selective and indicated) prevention of prevention; we consequently excluded evaluations of interventions characterized by (1) a focus on broad mental health and well-being but not specifically on anxiety (even if anxiety was a measured outcome), (2) screening for or identifying students with anxiety, or (3) treating students with anxiety (based on either a verified diagnosis or use of an established cutoff score for a validated measure of anxiety). We also excluded interventions that did not have a component (1) delivered directly to students, (2) delivered during normal school hours, and (3) focused on anxiety specifically. Although we excluded primary studies without a comparison group, we did not restrict inclusion based on type of comparator intervention or method of assignment to intervention group.

2.2 Information Sources

We searched for eligible systematic reviews published using the following electronic databases (with search platforms in parentheses): PubMed (National Library of Medicine), ProQuest Dissertations & Theses A&I (ProQuest), and Social Science Premium Collection (ProQuest). The Social Science Premium Collection includes the Criminology Collection, Education Collection (which includes Education Resources Information Center, or “ERIC”), International Bibliography of the Social Sciences, Library & Information Science Collection, Linguistics Collection, Politics Collection, Social Science Database, and Sociology Collection. Given the rapid timeframe of our overview, we did not search systematic review registries, web search engines, specific web sites, or journal tables of contents, nor did we conduct a search for primary studies missed by included systematic reviews.

2.3 Search Strategy

Two authors experienced in conducting systematic reviews (KSF, ETS) developed the search strategy, and another author experienced in systematic reviews (SG) subsequently reviewed the strategy prior to execution using the Peer Review of Electronic Search Strategies (PRESS) Guideline (McGowan et al., 2016). This PRESS peer review led to elimination of two terms: “NOT (universities[Mesh])” and “NOT subject(“university students”)” due to concerns that they would exclude studies of interest on K-12 students that make reference to universities and/or university students. We used Yale MeSH Analyzer to inform our search terms based on reviews identified in our preliminary scans of the literature (Ahlen et al., 2015; Caldwell et al., 2019; Dray et al., 2017; Feiss et al., 2019; Garber et al., 2016; Gee et al., 2020; Johnstone et al., 2018; Stockings et al., 2016; Werner-Seidler et al., 2021; Zhang et al., 2023). We also used PubMed PubReMiner for term frequency analysis. To focus on literature published after the conceptualization of subsyndromal psychiatric disorders (Volz et al., 2023) and following JBI recommendations (Aromataris et al., 2020), we limited our search to publications from 1990 until the date of the search (May 3, 2023). Our search strategy did not involve the use of any published search filters.

2.4 Selection Process

One team member (SDT) uploaded citations into our web-based systematic review data management software (DistillerSR) and used the “Duplicate Detection” function to identify and remove duplicates. We then used standardized forms in DistillerSR for the citation screening. After training in the standardized forms (led by SG) and DistillerSR software (led by SDT and SG), two reviewers (a combination of ED, EETS, MSC, and SG) independently screened the title and abstract of each citation for potentially eligible systematic reviews. Reviewers were not blind to journal titles nor to study authors and their institutions. We retrieved the full text for each citation deemed potentially eligible by at least one reviewer. Two reviewers (a combination of ED, MSC, and SG) then independently assessed each full text for eligibility. We included all systematic reviews deemed eligible by both reviewers. Reviewer pairs resolved any disagreements about full-text eligibility via discussion, conferring with a third reviewer as needed. We recorded reasons for excluding citations at the full-text stage. Rather than excluding overlapping but eligible systematic reviews at this stage, we identified and managed any overlap in the populations, interventions, comparators, and outcomes of included systematic reviews at later stages of the review (i.e., data collection and analysis). Two reviewers (MSC and SG) independently assessed all primary studies included in eligible reviews for eligibility following the same process described above for systematic review eligibility assessment.

2.5 Data Collection Process and Items

We used standardized forms in DistillerSR for the data collection process based on a data collection codebook with all variables for which data were sought. After training in the data collection codebook (led by SG) and DistillerSR software (led by SDT), two reviewers (a combination of SG and either ED or MSC) independently collected data from eligible systematic reviews and primary studies. Reviewer pairs resolved any disagreements about data collection via discussion, conferring with a third reviewer as needed. Variables included bibliographic information (e.g., authors, year published), information related to the review question (i.e., population, interventions, comparators, outcomes, and settings), and methodological characteristics (e.g., risks of bias). Our primary outcome is anxiety diagnosis. Our secondary outcomes are subsyndromal anxiety and anxiety symptoms. Our tertiary outcomes are depression, educational achievement and attainment, self-harm, self-medication, stress, substance use, suicidal ideation, and wellbeing. We collected all outcome data compatible with one of the above domains from all eligible primary studies (Gates et al., 2022). SG and EETS discussed and decided on any additions, modifications, or clarifications to the codebook after the data collection process commenced.

2.6 Risk of Bias

After training in the assessment criteria (led by SG) and DistillerSR software (led by SDT), two reviewers (a combination of SG and either ED or MSC) independently assessed the methodological quality of systematic reviews using AMSTAR-2 (Shea et al., 2017), risk of bias in systematic reviews using ROBIS (Whiting et al., 2016), and the risk of bias of primary studies using the RoB 2 for randomized trials (J. A. C. Sterne et al., 2019) and ROBINS-I for nonrandomized trials (J. A. Sterne et al., 2016).

2.7 Synthesis Methods

We narratively summarized descriptive information and risk of bias assessments about eligible systematic reviews and primary studies. We quantified the overlap of primary studies across systematic reviews using Corrected Covered Area (CCA) calculations (Bougioukas et al., 2022) and visualized overlap using a citation matrix and CCA pairwise tables (Pieper et al., 2014). To estimate intervention effectiveness, we conducted random effects meta-analyses using a robust variance estimation (RVE) approach (Hedges et al., 2010) implemented via the general workflow for modelling dependent effect sizes in the R metafor package (Viechtbauer, 2024). This includes using the clubSandwich package v. 0.5.10 (Pustejovsky, 2023) to apply a small sample correction to obtain cluster robust tests and associated confidence intervals. For domains involving inadequate degrees of freedom for RVE, we specified random effects meta-analysis in metafor without RVE. We ensured that all analyzed effect sizes were statistically independent using the following decisions rules: (a) if a study had multiple timepoints, we selected the post-intervention timepoint or the follow-up timepoint that was closest to the completion of the intervention; (b) because statistical dependencies remained due to multi-arm trails, we followed guidance from Cochrane to combine treatment arms into a single intervention group effect estimate (Higgins et al., 2022). We report effect estimates as standardized mean differences for continuous data and risk ratios for dichotomous data.

We assessed the extent of heterogeneity statistically using \(I^2\) and \(\tau^2\). We conducted meta-regressions to investigate sources of statistical heterogeneity in effect estimates by race/ethnicity, grade/school level, school type, geographic region/country, level of prevention, cultural specificity of interventions, comparator type, study/publication year, and risk of bias. We also conducted equivalence testing using two one-sided test (TOST) procedures (Lakens, 2017) and the TOSTER v. 0.8.3 package for R (Lakens, 2024). Using TOST, we specified upper and lower bound limits for each effect size (SMD and risk ratios): i.e., is, a minimal effect size of interest to test whether effects obtained in this meta-analysis falling within this range are deemed not to be practically different (even in cases of statistically significant or non-significant differences between groups). We used SMDs of +/-0.10 as equivalence bounds for continuous outcomes. For dichotomous outcomes, we converted these SMDs to odds ratios using the logit method, and then converted these odds ratios to risk ratios using 11% as the non-exposed prevalence rate of a current anxiety disorder diagnosis (Ghandour et al., 2019), yielding risk ratios of 0.85 and 1.17 for the lower and upper bounds. In addition, we estimated the probability that the true effect of school-based anxiety prevention interventions will be null or higher in a new study using the cumulative distribution function for the t-distribution (IntHout et al., 2016).