1st author, year | Selection bias (A) | Study design (B) | Confounds (C) | Blinding (D) | Data collection methods (E) | Withdrawal & drop-outs (F) | Intervention integrity (G) | Analysis (H) | Summary rating |
---|---|---|---|---|---|---|---|---|---|
Mixed method studies (n = 8) | |||||||||
Smith (2004) [53] | Moderate | Moderate | NA | Weak | Weak | Moderate | Collection of quantitative data stopped before real community action started. | Community-level allocation, individual-level analysis. No appropriate analysis of change in child growth over time. | Weak |
Lee (2008) [42]a | Weak | Weak | NA | Moderate | Weak | Moderate | Many youth involved in the interventions, no information on consistency, other community initiatives were running simultaneously (including stricter supply controls and rewards linked to school attendance). | Community-level allocation and analysis. Statistical methods described in other publication. Dates of data collection (2001–2004) do not line up with dates of intervention (2003–2005), no post-test data. | Weak |
Tyrell (2003) [43]a | Weak | Weak | NA | Moderate | Weak | Moderate | No description of who was exposed to the project and who weren’t, nor of possible external influences on outcomes. | Allocation on community and individual level. Evaluation on community, organisational and individual level. No statistical analysis (outcomes as percentages only). | Weak |
Guenther (2011) [58] | Strong | Weak | NA | Weak | Weak | N/A | All participants were part of the project; not all participants attended every session; it is likely that the results were influenced by other interventions put on the families. | Individual-level allocation and analysis; statistical analyses (frequencies and t-test) were appropriate; analysis performed on actual intervention status. | Weak |
Salisbury (1998) [44]a | Weak | Moderate | NA | Weak | Weak | Moderate | No description of exposure or consistency; no mention of other interventions influencing outcomes; tested for population growth (which didn’t grow) | Unit of allocation and analysis are on organizational level. No statistical analysis. | Weak |
Hunt (2010b) [59] | Weak | Weak | Weak | Weak | Weak | Weak | No description of exposure or consistency, potential influence of other interventions running in the communities at the same time. | Unit of allocation community and organisational level, unit of analysis individual level, no statistical analysis. | Weak |
Moran (2003) [45]a | Moderate | Moderate | NA | Weak | Moderate | Weak | 92 % of participants reported awareness of town plan. Outcomes may be influenced by the cycle of optimism and pessimism. | Unit of allocation is community level, analysis is done on community and individual level. No statistical analysis. | Weak |
McCalman (2005) [54] | Moderate | Moderate | NA | NA | Weak | Weak | All evaluation participants were exposed to intervention; consistency was not measured; outcomes likely influenced by other factors. | Project was allocated at organizational level, data were collected on community level, cannot be sure whether changes at community level are caused by changes at the organizational level. No statistical analysis | Weak |
Quantitative only studies (n = 2) | |||||||||
Jarvie (2008) [46]a | Weak | Weak | Weak | Weak | Weak | Strong | One community withdraw temporarily, others stayed. There’s a chance that outcomes are influenced by other developments going on at the same time | No statistical analysis, intervention offered at community level, data gathered at population level. | Weak |
Shannon (2001) [47]a | Weak | Moderate | N/A | Weak | Weak | Moderate | No description of exposure to intervention or consistency in delivery. Outcomes may be influenced by other factors | Community-level allocation and analysis. Appropriate statistical analysis. | Weak |