2 Methods

Our methods and reporting are informed by guidance from the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015) Statement (Moher et al., 2015), the Preferred Reporting Items for Overviews of Reviews (PRIOR) Statement (Gates et al., 2022), the PRISMA extension for reporting literature searches in systematic reviews (Rethlefsen et al., 2021), JBI guidance on umbrella reviews (Aromataris et al., 2020), and Cochrane guidance on overviews of reviews (Pollock et al., 2022).

2.1 Eligibility Criteria

We included systematic reviews with a meta-analysis examining the effects of school-based depression prevention interventions delivered directly to primary and secondary school students. We used the definition of a “systematic review” provided in the PRISMA-P 2015 reporting guideline: “a systematic review attempts to collate all relevant evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods to minimize bias in the identification, selection, synthesis, and summary of studies” (Moher et al., 2015). We used this definition to identify reviews that have the potential to provide reliable findings for evidence-informed decision-making. Based on this definition, our operational criteria for a systematic review were: (i) clear objectives, (ii) an explicit and reproducible methodology, (iii) a systematic search strategy for attempting to identify all studies that meet the review eligibility criteria, (iv) critical appraisal of the included studies, and (v) systematic synthesis of the characteristics and findings of the included studies. We excluded systematic reviews that did not report the results for a meta-analysis of school-based depression prevention interventions on at least one depression outcome (depression diagnosis, subsyndromal depression, or depression symptoms). We also excluded reviews on interventions delivered in school settings but only during out-of-school time (e.g., after school, summer break). Due to limitations in resources and review team language proficiency, we excluded reviews published in a language other than English. Although grey literature was eligible, we excluded reviews published only in a summary format (e.g., only as a conference abstract or executive summary). For our meta-analysis of data from primary studies in eligible systematic reviews, we included evaluations of a school-based depression prevention intervention directly delivered to primary and secondary school students during normal school hours. We excluded studies that included pre-school or college students if less than 50% of the sample was primary and secondary school students. We used the National Research Council (2009) definitions for primary (universal) and secondary (selective and indicated) prevention of prevention; we consequently excluded evaluations of interventions characterized by (1) a focus on broad mental health and well-being but not specifically on depression (even if depression was a measured outcome), (2) screening for or identifying students with depression, or (3) treating students with depression (based on either a verified diagnosis or use of an established cutoff score for a validated measure of depression). We also excluded interventions that did not have a component (1) delivered directly to students, (2) delivered during normal school hours, and (3) focused on depression specifically. Although we excluded primary studies without a comparison group, we did not restrict inclusion based on type of comparator intervention or method of assignment to intervention group.

2.2 Information Sources

We searched for eligible systematic reviews published using the following electronic databases (with search platforms in parentheses): PubMed (National Library of Medicine), ProQuest Dissertations & Theses A&I (ProQuest), and Social Science Premium Collection (ProQuest). The Social Science Premium Collection includes the Criminology Collection, Education Collection (which includes Education Resources Information Center, or “ERIC”), International Bibliography of the Social Sciences, Library & Information Science Collection, Linguistics Collection, Politics Collection, Social Science Database, and Sociology Collection. Given the rapid timeframe of our overview, we did not search systematic review registries, web search engines, specific web sites, or journal tables of contents, nor did we conduct a search for primary studies missed by included systematic reviews.

2.3 Search Strategy

One author who is an experienced systematic reviewer (EETS) developed the search strategy, and another author experienced in systematic reviews (SG) subsequently reviewed the strategy prior to execution using the Peer Review of Electronic Search Strategies (PRESS) Guideline (McGowan et al., 2016). This PRESS peer review led to including additional prevention terms in subject headings (primary prevention, selective prevention, indicated prevention), as well as adding “indicated” and “meta” to our free text search of titles and abstracts. As a result of the PRESS peer review, we also explored several subject headings and exploding broad subject headings that we ultimately did not include due to concerns of significantly increasing the number of false positives yielded by our search strategy. We used previous overviews identified in our preliminary scans of the literature to inform our search terms (Hoare et al., 2021; Hu et al., 2020; Levinson et al., 2019; Salazar de Pablo et al., 2021). To focus on literature published after the conceptualization of subsyndromal depression (Volz et al., 2022) and following JBI recommendations (Aromataris et al., 2020), we limited our search to publications from 1990 until the date of the search (October 20th, 2022). Our search strategy did not involve the use of any published search filters.

2.4 Selection Process

One team member (SDT) searched for duplicates using the “Duplicate Items” function in Zotero prior to uploading citations into our web-based systematic review data management software (FileMaker Pro). We then used standardized forms in FileMaker Pro for the citation screening. After training in the standardized forms (led by SG) and FileMaker Pro software (led by SDT and EETS), two reviewers (a combination of ED, EETS, MSC, and SG) independently screened the title and abstract of each citation for potentially eligible systematic reviews. Reviewers were not blind to journal titles nor to study authors and their institutions. We retrieved the full text for each citation deemed potentially eligible by at least one reviewer. Two reviewers (a combination of ED, MSC, and SG) then independently assessed each full text for eligibility. We included all systematic reviews deemed eligible by both reviewers. Reviewer pairs resolved any disagreements about full-text eligibility via discussion, conferring with a third reviewer (EETS) as needed. We recorded reasons for excluding citations at the full-text stage. Rather than excluding overlapping but eligible systematic reviews at this stage, we identified and managed any overlap in the populations, interventions, comparators, and outcomes of included systematic reviews at later stages of the review (i.e., data collection and analysis).

2.5 Data Collection Process and Items

We used standardized forms in FileMaker Pro for the data collection process based on a data collection codebook with all variables for which data were sought. Variables included bibliographic information (e.g., authors, year published), information related to the review question (i.e., population, interventions, comparators, outcomes, and settings), and methodological characteristics (e.g., risks of bias). In a revision to our protocol, we also collected information directly from eligible primary studies (rather than relying on information reported in systematic reviews about primary studies). Our primary outcome is depression diagnosis. Our secondary outcomes are subsyndromal depression and depressive symptoms. Our tertiary outcomes are anxiety, educational achievement and attainment, self-harm, self-medication, stress, substance use, suicidal ideation, and wellbeing. We collected all outcome data compatible with one of the above domains from all eligible primary studies (Gates et al., 2022).

We provided reviewers with a FileMaker Pro instruction manual for using our standardized forms to collect data. After training in the data collection codebook (led by SG) and FileMaker Pro software (led by SDT and EETS), two reviewers (a combination of SG and either ED, EETS, or MSC) independently collected data from eligible systematic reviews and primary studies. Reviewer pairs resolved any disagreements about data collection via discussion, conferring with a third reviewer as needed. SG and EETS discussed and decided on any additions, modifications, or clarifications to the codebook after the data collection process commenced.

2.6 Risk of Bias

After training in the assessment criteria (led by SG) and FileMaker Pro software (led by SDT and EETS), two reviewers (a combination of SG and either ED, EETS, or MSC) independently assessed the methodological quality of systematic reviews using AMSTAR-2 (Shea et al., 2017), risk of bias in systematic reviews using ROBIS (Whiting et al., 2016), and the risk of bias of primary studies using the RoB 2 for randomized trials (J. A. C. Sterne et al., 2019) and ROBINS-I for nonrandomized trials (J. A. Sterne et al., 2016).

2.7 Synthesis Methods

We narratively summarized descriptive information about eligible systematic reviews and primary studies. We quantified the overlap of primary studies across systematic reviews using Corrected Covered Area (CCA) calculations (Bougioukas et al., 2022) and visualized overlap using a citation matrix and CCA pairwise tables (Pieper et al., 2014). To estimate intervention effectiveness, we conducted random effects meta-analyses with a robust variance estimation approach using data that we extracted from eligible primary studies included in eligible systematic reviews (Tanner-Smith et al., 2016). We assessed the extent of heterogeneity statistically using \(I^2\) and \(\tau^2\). We estimated the probability that the true effect of school-based depression prevention interventions will be null or higher in a new study using the cumulative distribution function for the t-distribution (IntHout et al., 2016). We also conducted meta-regressions to investigate sources of statistical heterogeneity in effect estimates by baseline differences in depression symptoms, race/ethnicity, grade/school level, school type, geographic region/country, level of prevention, cultural specificity of interventions, comparator type, study/publication year, and risk of bias. We report effect estimates as standardized mean differences for continuous data and risk ratios for dichotomous data. Narrative statements to communicate findings for each meta-analytic effect estimate are based on guidance from the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (Santesso et al., 2020).