This article has Open Peer Review reports available.
An exploratory review of HIV prevention mass media campaigns targeting men who have sex with men
© French et al.; licensee BioMed Central Ltd. 2014
Received: 27 February 2014
Accepted: 11 June 2014
Published: 18 June 2014
Men who have sex with men (MSM) are at increased risk of HIV infection in both high- and low-income settings. Mass media campaigns have been used as a means of communicating HIV health promotion messages to large audiences of MSM. There is no consensus on which designs are most appropriate to evaluate the process and outcomes of such interventions.
An exploratory review was conducted to assess research examining awareness, acceptability, effects on HIV testing, disclosure and sexual risk, and cost-effectiveness of HIV mass media campaigns targeting MSM. We searched for quantitative and qualitative studies published between 1990 and May 2011 via the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, Psych Info, ISI Web of Science, OpenGrey and COPAC, and contacting experts. No exclusions were made on the basis of study design or methods because our primary aim was to map evidence. We appraised study quality and present a narrative synthesis of findings.
Sixteen reports from 12 studies were included. All were from high-income countries and most examined multi-media interventions. Half of the studies were single cross-sectional surveys. Three repeat cross-sectional studies collected data pre and post the campaign launch. The remaining three studies monitored routine data. Three studies included a nested qualitative component. Campaign coverage was the most commonly reported outcome (9 studies). Imagery, tone of language, content and relevance were identified in the qualitative research as factors influencing campaign acceptability. HIV testing rates (or intention to test) were reported by five studies. Two studies reported that testing rates were higher among men who had seen the campaigns compared to men who had not, but this may reflect confounding. Findings were less consistent regarding reductions in sexual risk behaviours (4 studies). None of the studies examined cost-effectiveness.
Campaigns aim to provide MSM with information to help prevent transmission of HIV and to address increasing motivation and changing norms towards precautionary behaviours. However, the limitations of mass media in imparting skills in effecting behaviour change should be recognised, and campaigns supplemented by additional components may be better-suited to achieving these goals.
Worldwide, sex between men accounts for between 5 and 10% of HIV infections, but the proportion is far higher in much of the developed world where sex between men is the most common means of transmission . Sex between men is also a prominent feature in the epidemiology of HIV in other regions such as Latin America where men who have sex with men (MSM) are at increased risk of HIV infection.
Mass media interventions have the potential to reach large audiences, providing them with information and raising awareness. Campaigns can also present role models and aim to change normative beliefs, as well as helping put health issues on policy-makers’ agenda . Mass media interventions can potentially reach individuals or groups who may not be accessing other statutory or community-based interventions. They may use broadcast media, such as television, radio or film; print media, such as posters and newspapers; outdoor media, such as billboards; or digital media, such as the internet .
Although HIV prevention mass media campaigns have been criticised for using weak evaluation design [4, 5], there is no consensus on which designs are most appropriate for evaluating process and outcomes . While randomised controlled trials (RCTs) provide the most rigorous means of evaluating public health interventions, they are not generally applicable to mass media interventions because of obvious challenges concerning contamination or the lack of sufficient units for statistical power . Alternatives include interrupted time-series analyses comparing repeat data on outcomes from a population before and after exposure, and cross-sectional studies comparing outcomes among exposed and unexposed individuals. Which of these provide the least biased estimates of effect is likely to depend on context; for example intervention coverage, secular trends in outcomes and likely effect sizes . The evaluation of the effects of mass media interventions on MSM is further complicated by the lack of a sampling frame for this population necessitating convenience samples . Furthermore, other questions of importance to policy-makers such as intervention coverage and acceptability are also not amenable to experimental designs, and in the case of acceptability may be examined via quantitative and qualitative research.
A Cochrane systematic review conducted by Vidanapathirana et al. in 2005 assessed the effects of mass media on HIV testing among the general population and specific target groups, including MSM . The authors concluded mass media campaigns were effective in increasing testing in the general population in the short-term, although no long-term impacts on HIV testing were observed. However, only one of the studies included in the review targeted MSM .
Given the lack of previous reviews of HIV prevention mass media interventions targeting MSM, we aimed to examine literature in this area. Given our interest in examining questions of awareness and acceptability as well as effectiveness and cost-effectiveness, and given the lack of consensus on which designs are most appropriate, our exploratory review aimed to systematically map evidence in this area, appraise its quality, and narratively synthesise its findings. The following research questions (RQ) are examined: (RQ1) How successfully do HIV prevention mass media interventions achieve awareness among their target audience(s) of MSM? (RQ2) Does mode of delivery affect campaign awareness among MSM? (3) Are HIV prevention mass media campaigns acceptable to MSM? (RQ4) What influences campaign acceptability to MSM? (RQ5) Are HIV prevention campaigns effective or cost-effective in modifying HIV knowledge or attitudes, reducing sexual risk behaviour, and promoting HIV testing and HIV disclosure among MSM, when compared with pre-intervention or non-exposed participants? (RQ6) In what ways do intervention characteristics appear to influence awareness, acceptability or effectiveness?
The following electronic bibliographic databases were searched (from January 1990 to May 2011): the Cochrane Library, MEDLINE, EMBASE, Psych Info and ISI Web of Science. Two databases of grey literature, OpenGrey and COPAC, were searched in December 2012. A search strategy using thesaurus and non-thesaurus terms as appropriate to each database relating to the concepts of MSM, media and HIV, adapted from the strategy used by Vidanapathirana et al. (see Additional file 2) . Relevant websites were also searched, including the World Health Organization, Centers for Disease Control, Diffusion of Effective Behavioral Interventions and the Joint United Nations Programme on HIV/AIDS. The reference lists of related reviews and included articles were searched for additional citations. Authors of included studies and other experts in the field were contacted by email to identify further studies.
Criteria for selecting studies
Both published and unpublished literature was included. Included reports met the following criteria:
Published in English between 1990 and May 2011.
Any study where MSM were an intervention’s target group, irrespective of sexual identity. This included interventions that solely targeted MSM or where interventions also targeted other groups but study results were reported for MSM as a subgroup. Evaluations where it was not possible to disaggregate the intervention’s awareness, effectiveness or acceptability amongst MSM from other target groups were excluded. Studies of campaigns targeting health professionals were excluded.
Mass media campaigns relating to HIV health promotion that targeted MSM were included. Unpaid for media coverage and interactive media health promotion interventions (such as use of internet chat rooms) were excluded. Interventions that only included small media, such as leaflets, were excluded, but those where mass media were complemented with small media were included. Outcome evaluations of complex programme interventions which included both a mass media component and non-media components were excluded, as were mass media campaigns relating solely to other aspects of sexual health. Laboratory studies which artificially exposed a research sample to an intervention were also excluded.
Pre-intervention or non-exposed study participants.
As explained above, our review sought to map and appraise a variety of study designs. Therefore, in relation to each of our research questions, studies were not excluded on the basis of their design. Instead, our appraisal systematically assessed the potential internal and external validity of studies (see below).
Descriptions of mass media campaigns with no form of evaluation and studies limited to piloting or pre-testing were excluded.
Evaluations were included which examined at least one of the following outcomes at any post-intervention time-point: HIV knowledge or attitudes, HIV testing, HIV disclosure and sexual risk behaviours. No limitations were put on length of follow-up.
References identified through our search were downloaded into an Excel (Microsoft 2010) spreadsheet. Titles and abstracts were screened by one reviewer. Full texts were obtained for review when titles and abstracts met our inclusion criteria or when there was any ambiguity about the decision for inclusion. These were screened by one reviewer, using a screening sheet detailing inclusion and exclusion criteria, and checked by a second with no disagreements occurring.
Data extraction and quality assessment
Data on intervention (media employed, aims, theory of change, any initial formative research or piloting, setting, target population) and study (aims, design, sampling, response rates, data collection, analysis) was extracted from studies by one reviewer and checked by another with any disagreements being resolved by discussion. Data extraction forms had been piloted on two studies.
Quality assessment was conducted at the study level using tools developed specifically for this review. Existing Cochrane  and TREND  (Transparent Reporting of Evaluations with Non-Randomized Designs) could not be used because these focus respectively on RCTs and on non-randomized studies with external control groups, whereas, as discussed above, our evaluations used a range of designs to examine multiple questions with no consensus in the field as to which are the most rigorous. Quantitative studies were assessed in terms of mimimizing confounding, selection and information bias, reverse causality and random error. Qualitative studies were assessed using established criteria  addressing sampling, data collection, data analysis, the extent to which the study findings are grounded in the data, whether the study privileges the perspectives of participants, and the breadth and depth of findings. All reports were quality assessed by one reviewer and checked by another with any differences resolved by discussion.
Synthesis of findings
Given the exploratory aims of this review and the lack of homogeneity in study design and aims, measures and interventions, it was not appropriate to undertake meta-analysis and narrative synthesis was instead undertaken, using similar approaches to those used in previous well-conducted reviews.
Given the small number of qualitative studies found in this review and the lack of overlap in the substantive topics addressed, the decision was taken not to attempt a systematic synthesis, for example via meta-ethnography, and instead to limit reporting to presentation of findings and conclusions of the studies on their own terms.
Nine studies examined campaign awareness [14–18, 21–28]. Seven assessed acceptability or influences on acceptability [15, 16, 18, 21–28]. None assessed effects on knowledge or attitude outcomes; five examined effects on HIV testing [9, 16, 19, 21] or intention to test ; and four examined effects on sexual behaviour, including unprotected anal sexual intercourse , condom use [16, 29], median number of sexual partners  and changes in sexual practice [15, 28]. No studies examined cost-effectiveness.
Study context and target population
Seven interventions targeted all MSM [16–19, 21, 23–26, 29]. The remainder targeted specific sub-groups of MSM by age [9, 27] or ethnic sub-groups , recent unprotected sexual intercourse with men of unknown or discordant HIV status [15, 22, 28] and perceived HIV-negative serostatus . None of the interventions reported aiming to target MSM according to sexual identity.
Location of delivery
Hecht et al. 
“Are you Iffy?” a social marketing campaign to encourage MSM to reassess their HIV-negative status. 140 posters, 7 billboards and 10 newspaper ads
Health belief Model
Focus groups and community field-testing
San Francisco, USA
Bars/clubs, toilets, subways, bus terminals and newspaper kiosks
HIV negative MSM
Hilliam et al. *
“HIV Wake-Up Campaign” to provide information on HIV, benefits of prevention and regular testing and where to get further advice. Posters, web adverts and dedicated website, complemented with small print media, i.e. leaflets
MSM and non-HIV specialist professionals
Not stated where print media distributed
16 mass media adverts placed in press, on websites and as posters, complemented with small media, such knik-knaks
2007: Themes for interventions identified in meetings with partners. Pre-testing with focus groups.
National and regional press and where appropriate national HIV positive press. Posters in gay bars, saunas and clubs
Adverts and their aims (number of images, launch date, display costs):
Assume nothing (4 images, July 1997, £73,000)
Think, Talk, Time Test (3 images, Jan. 1998, £67,000)
What am I?.. See to it. (1 (1 image, July 1998, £70,000)
What on Your Mind? (1 image, Jan. 1999, £70,000)
Homophobia – (1 image in gay media, Oct 1999; £4,934; 3 images in general media, Aug 1999, £74,616)
Better of knowing – (5 images, Feb 2000; £52,254)
Facts for life – to provide information on HIV risks (9 images, Sept 2000, £39, 808; Jan 2003)
In two minds - to illustrate dilemmas between “thoughts connected to the head (relating to risk reduction) and with the cock/crotch (less rational…)” (10 images, Nov. 2000; £31,114)
Just unbelievable - to highlight the presumption that all HIV positive partners will disclose their status (3 images, Oct 2001, £20, 252)
Clever Dick - to promote condom use (5 images, March 2002, £22,046)
Biology of transmission – to increase awareness of rectum’s and anus’s fragility and absorbency (3 images, Oct 2002, £20,661)
Think again – to show divergent thoughts and concerns about HIV transmission and exposure (6 images, Nov 2003, £20, 326)
Infection situations – to illustrate possible adverse outcomes associated with sexual risks (5 images, April 2004, £24, 191)
Be confident, be covered – to promote use of condoms (3 images, Feb 2005 £16,923)
PEP – to increase knowledge of post-exposure prophylaxis (PEP) and is availability after sexual exposure to HIV (1 image. June 2004 and July 2005, £9, 428)
Closer – to increase awareness of local and national HIV prevalence and to get men to reconsider their HIV risk (6 images, Jan 2006, cost not reported)
Roedling et al. 
A campaign to provide MSM with information about PEP and where it is available
London and Brighton, UK
Adverts placed in gay press and ‘other material’
Hartfield et al. *
“It’s the Little Prick You Can Deal With” campaign to encourage MSM at high risk of HIV to have HIV test every 3 months. Gay-orientated websites, billboards, pavement chalk drawings, complemented with small print media and knick-knacks (e.g. coasters)
Pre-testing with community members
Outside and inside gay bars and saunas
MSM at high risk of HIV infection, defined as those who have had unprotected sexual intercourse with a partner of unknown or discordant HIV status in last year
Katzman et al. 
“Community Manifesto” to identify sexual health issues and to promote positive sexual health for MSM
Task force meetings with involvement from health agencies and community members
Seattle & King County, USA
Placed in 2 weekly Seattle papers, freely distributed. Also posted on a gay health web site in English and Spanish
“Think Again” campaign (adapted from the US “Assumptions” campaign) to encourage men to challenge assumptions around partners’ HIV status. Ultimate goal to reduce unprotected anal intercourse between men of discordant HIV status and thereby reduce HIV incidence. Multi-media complemented with small print media and knick-knacks (e.g. coasters). Cost of campaign = $250,000
Used social marketing concepts
9 focus groups (47 participants)
National campaign - billboards, gay venues and Internet
HIV positive and HIV negative men having unprotected sexual intercourse with men whose HIV status is unknown to them
The TASC Agency *
“Equal” campaign to promote safer sex, condom and lube use, and regular sexual health check ups. Posters
MSM aged 25-40 years
McOwan et al. 
HIV testing campaign ‘gimme 5 minutes’ which ran between March-May 2000. ‘Peer’ images (different photos representing each of the target groups) with same accompanying text covering topics relating to pre-test discussions, making decision to test and information about testing services at the campaign clinic. Newspaper adverts and posers, complemented with small print media. Cost around £10,000
Full page advertisements in a free tabloid newspaper, 100 posters in Central London bars
MSM, particularly targeting men of Black and Southern European origin and men under 25-years-old
Sherr et al. 
“Try this HIV test” campaign to encourage homosexual men to consider having an HIV test in light of recent advances in HIV treatments.
Dawson & Hartfield 
Newspaper comic strip “Stella Seattle” “to clarify information about controversial transmission issues and to encourage HIV testing”
Mention formative research was undertaken and focus groups set up to test first few comic strips were tested
Comic strip ran weekly for 4 months in 2 local newspapers with a large gay readership
Campaign began Aug 1993. Cost $9,500 (included media placement of 15-episode strip, artist fees and staff time)
Griffith et al. 
Statutory HIV educational campaigns, including television and radio. Early campaigns were general and latter ones were targeted to specific subpopulations, including gay and bisexual men. Of the 38 media interventions 6 aimed at gay men and 2 at bisexual men
Numerous media campaigns
Across TV, radio and gay press
Gay and bisexual men
Across all interventions, posters were mainly placed in gay commercial venues and other gay community settings. Other studies described placing posters in other settings, such as around transport hubs [14, 15, 28].
Formative research was used to inform the development of the campaign concepts, content and materials in five interventions, including: use of focus groups with the target population [14, 15, 18, 23–26, 28], meetings with health agencies or community members [16, 23–26], and community field testing [14, 22]. This formative work helped ensure that campaign content addressed men’s preferences and needs, and as described in the study by Lombardo & Léger, it also helped to adapt an existing intervention to a new geographical setting while maintaining message consistency [15, 28].
Most interventions aimed to provide information on HIV prevention strategies and encourage HIV testing (Table 1). While most of the campaigns had moved away from simple ‘use a condom’ messages, few aimed to provide men with information on negotiating safer sex or disclosing their HIV status to a sexual partner. Furthermore, only two studies described the theory of change underlying the intervention; one drawing on the Health Belief Model  and the other on social marketing concepts [15, 28].
Campaign costs were reported in four studies [9, 15, 18, 23–26, 28]. Costs ranged from $250,000 for the national campaign in Canada [15, 28] to £9,500 for media placement, artists’ fees and staff time for the “Stella Seattle” newspaper comic strip .
Study design and quality
Study process and outcome measures
Hecht et al. 
To compare certainty of HIV negative status before and after the campaign
Repeat pre-test/post-test cross-sectional study (campaign May - July 2008, post-test July - Oct 2008). Part of National HIV Behavioral Surveillance
Recruitment: MSM venue-based, time-space sampling
45% reported they had seen the campaign. Of these:
Response rate: Not reported
64% correctly identified the subject of the campaign
Sample size: 316 men
Analysis confined to 255 men who reported being HIV negative
Hilliam et al. *
To measure campaign awareness; HIV awareness; attitudes toward testing, prevention and safe sex; and behaviour change
Repeat pre-test/post-test cross-sectional study - post-test 4-5 months after launch
Recruitment: Via LGBT and HIV organisation websites, and Gaydar (post-test only). Men recruited via Gaydar (G) were analysed separately from those recruited via other non-Gaydar (nG) websites due to differences observed between samples
3-13% reported non-prompted awareness of campaign
Response rate: Not reported
69%nG and 82%G reported prompted awareness of campaign (men most commonly reported seeing the web adverts 50%nG and 77%G, followed by the web site 17%G and 29%nG, and finally by the posters 27%G and 32%nG)
Sample size: Pre-test sample = 88, Post-test sample = 775
Campaign attributes most commonly reported by men: clear message (62%nG and 63%G) and relevance (51% G and 61%nG). Campaign attributes least commonly reported by men: motivating (11%nG and 20%G) and trustworthy (23%nG and 25%G)
Men who had seen the campaign were more likely to report having had an HIV test in the last 6 months than those who had not seen the campaign, 33%G and 38nG% versus 9%nG and 16%G, respectively
Repeat cross-sectional surveys – Gay Men’s Sex Survey (GMSS) questionnaire
Recruitment: Via Pride-type events across UK and the Internet
Focus groups and interviews
A variety of recruitment methods described for nested qualitative component, including established agency networks, e-newsletters, fliers and use of snowballing techniques. Purposive sampling, UK cities
Be confident, be covered: 32% recognised, and of those 52.4% had read
Response rate: Not reported
PEP: 16.1% recognised, and of those 56.6% had read
Infection situations: 18.6% recognised, and of those 51.6% had read
Between 1997-2000 data gathered via face data collection at Pride events, then from 2001 via booklets and online
Think again: 29.1% recognised, and of those 54.0% had read
2005: N = 12,322
2004: N = 11,909
Biology of transmission: 26.1% recognised, and of those 50.3% had read
2003: N = 9,482
2002 N = 11,046 (booklet = 3515, online = 7531)
Just as unbelievable: 30.7% recognised (40.4% of booklet users and 26.5% online users), and of those that recognised 64.6% (booklet) and 58.2% (online) had read
2001 N = 9226 (Pride attendees = 2401, booklet = 2384, web = 4441)
Clever dick: 31.9% recognised (43.9% of booklet users and 26.7% online users), and of those that recognised 72.2% (booklet) and 64.8% (online) had read
2000 N = 312
1999 N = 313
Facts for life: Recognised by 42.1% of Pride attendees, 43.0% of those using the booklet and 24.8% of those online, and of those that recognised 54.5% (booklet) and 41.4% (online) had read
1998 N = 294
In two minds? Recognised by 62.9% of Pride attendees, 59.6% of those using the booklet and 39.3% of those online, and of those that recognised 71.5% (booklet) and 63.1% (online) had read
2009: 6 groups, 49 men
Better off knowing: 48.1% recognised
2004: 7 groups, 33 men
What’s on Your Mind: 31.7% recognised
2003: 5 groups, 46 men
What am I?.. See to it: 47.1% recognised
2001: 5 groups, 37 men
Homophobia: 34.1% recognised image in gay press, 36.5% recognised images in general media
What’s on Your Mind: 54.3% recognised
What am I?.. See to it: 47.9% recognised
Think, Talk, Time to Test: 35.1% recognised
Think, Talk, Time to Test: 40.5% recognised, and of those 62.5% had read
Assume nothing: 44.6% recognised, and of those 56.1% had read
Roedling et al. 
To compare clinical data, exposure characteristics, follow-up and awareness of post-exposure prophylaxis following sexual exposure to HIV pre and post campaign
Retrospective case note review from 2004 pre and post campaign
Recruitment: Not applicable. Case notes for all those attending for PEP in 2004 included
Campaign launched in July 2008
Response rate: not applicable
Pre-campaign 20/33 (61%)
Sample size: 216 attendees requested PEP, data available on 197 (91%)
Post-campaign 39/66 (59%), p = 1.00
Analysis: Confined to 112 MSM commencing PEPSE, pre-campaign n = 36 and post-campaign n = 76
Median number of sexual partners in the previous 3 months
Pre-campaign 3 (range 1-50)
Post-campaign 4 (1-100), p = 0.51
Hartfield et al. *
To evaluate campaign coverage and impact
Repeat pre-test/post-test cross-sectional study. Campaign launched June-Aug 2008. Pre (March – May 2008) and post campaign (June-Dec. 2008) survey
Recruitment: Via MSM venues
75% reported exposure (24% unaided and a further 50% prompted)
Response rate: Not reported
80% of those who saw campaign very positive/positive. Only 3% negative
Sample size: Baseline survey n = 197
Intention to have HIV test
38% of those who had seen the campaign and were HIV negative (n = 279) said they would test more frequently due to the campaign
Post-campaign survey n = 464
Katzman et al. 
To examine the potential impact of the manifesto
Cross-sectional survey and focus groups (in English and Spanish)
Recruitment: Survey posted on web, left in 38 gay venues for mail-in and distributed by street intercepts. Method of focus group recruitment not stated
84% seen or heard manifesto
Response rate: 2506 surveys distributed in gay venues and 137 surveys returned (5.5%). 69 surveys from women and men without a partner excluded
Of those who had seen it:
61% strongly agreed/agreed with the manifesto, 19% disagreed/strongly disagreed
Sample size: Survey n = 103
13% had HIV test
9 Focus groups (139 participants) with representation of gay men both HIV positive and negative
HIV status disclosure
12% disclosed HIV status
10% asked partner to disclose status
16% increased condom use
To assess impact
Recruitment: Via “gay spaces” across Canada
79% average national exposure
Top messages received “rethinks risks” 47%, “protect self and partner” 37% and “use condoms”35%
Response rate: Not reported
73% found messages appealing
Sample size: N = 417
48% report message prompted them to change “something” about sexual practices, but men were not asked about the direction of change
The TASC Agency *
Cross sectional surveys (2006) and online survey (2007), focus groups and in-depth interviews
Recruitment: For surveys via Internet and a Pride event in Glasgow. Not stated for focus groups and interviews
Survey 2006: 82% had seen phase 1 and 2 poster images. 5.3% had visited the Equal website
Response rate: Not reported
Online survey: 8/26 had previously seen the Phase 1 posters; 9/24 the Phase 2 posters and 15/25 the Phase 3 ones
Sample size: Survey 2006: 222 men responded. Analysis confined to men aged 25-40 years, N = 116
Survey 2006: Phase 1 posters 57% reported that they “Love’em” or “They’re good”; Phase 2 posters 53%
Online survey: N = 27
6 focus groups with 28 participants
McOwan et al. 
To evaluate effect of an HIV testing campaign
Retrospective case note review of GUM attendees – comparing the same time points across two years and comparing campaign clinic with two other sexual health clinics
Recruitment: Not applicable. Those testing in the three clinics were retrospectively identified through a central laboratory
In the campaign clinic 4.5 fold increase in numbers of men testing in 2000, n = 292, compared to 1999, n = 65, (p < 0.001), 14.0 fold increase in men of Southern European origin (n = 42 in 2000 vs. n = 3 in 1999, p < 0.001), 6.5 increase in Black men (n = 13 in 2000 vs. n = 2 in 1999, p = 0.003) and 9.5 increase in men under 25 (n = 57 in 2000 vs. n = 6 in 1999, p < 0.001)
Response rate: Not applicable
No significant differences for these outcomes observed in two control clinics. Total number of men testing in 2000 = 236 and in 1999 = 239 (p = 0.982), Southern European men testing n = 37 and 25, respectively (p = 0.341), Black men testing n = 3 and 5, respectively (p = 0.864) and men aged less than 25 years n = 32 and 36, respectively (p = 0.807)
Sample size: See Results
Unprotected anal intercourse (campaign clinic only)
Pre campaign 35/65 (53.8%, 95% CI 41.0 - 66.3%)
Post campaign 156/292 (53.4%, 95% CI 47.5-59.1%)
Sherr et al. 
To evaluate permeation (picture recognition), recall of message, endorsement, and decision to have a test
Cross sectional survey
Recruitment: Two sexual health clinics
80.1% reported seeing the campaign pictures
Clinics attendees having an HIV test. In one clinic this included heterosexual men and women, and homosexual men requesting an HIV test and in the other all homosexual men
Decision to have HIV test
25.5% recalled the message (half of this group had correct recall)
Response rate: Not reported
9.3% reported campaign played important part in decision to have a test
667 individuals completed questionnaire, of these 339 reported they were homosexual or bisexual
Dawson & Hartfield 
To look at exposure to and satisfaction with campaign
Repeat cross sectional surveys
Recruitment: Gay bars and a Pride event at different points in time
1993 interviews – 73% familiar with comic
Response rate: Not reported
1993 clinic forms - 32% gay and bisexual clients who tested reported seeing the campaign
Review of clinic HIV testing and voice mail calls
1994 bar survey – 47% had seen comic strip
Oct 1993 gay bar structured interviews - number unknown
1994 rally survey – 44% had seen comic strip
July-Nov 1993 clinic review – number of case notes reviewed not reported
1993 interviews - Of those familiar, 89% positive. Negative comments mainly around format, such as confusing and hard to read
May 1994 gay bar survey, n = 662
1994 rally survey – Of those who had seen it, 57% liked a lot and 41% thought OK
June 1994 Gay Pride Rally survey, n = 198
Voice mail – weekly average of 200-400 calls. Report that most were positive
Griffith et al. 
To describe association between HIV educational campaigns and long-term testing trends between Sept 1985 – Sept. 1993. Of the 38 media interventions, 6 aimed at gay men and 2 at bisexual men
Continuous – prospective collection of demographic and behavioural data from all GUM attendees having an HIV test
Recruitment: Not applicable
Unable to extract data - trends in testing amongst homosexual and bisexual men annotated with media campaigns shown graphically. Authors report periods of peak testing generally corresponded temporally with increased media coverage
Response rate: Not applicable
19 242 tested in three London sexual health clinics, UK
12 183 men (37.6% homosexual and 7.9% bisexual)
Limited information was provided on recruitment and sampling methods. As expected, none of the studies recruited men using random probability sampling. The study by Lombardo & Léger reported that men were recruited at random from convenience samples in ‘gay spaces’ rather than using representative sampling frames [15, 28]. Other studies reported using convenience sampling to identify men in gay venues, such as bars and clubs, or locations where gay men were likely to congregate, such as on streets or in parks [14–16, 18, 22]. Three studies described recruitment via the Internet [16, 21, 23–26], five accessed GUM attendees [9, 17–19, 29], and three recruited men at gay community events, such as Pride [18, 23–27]. The study by Katzman et al. was the only one to provide information on response rates and reported a low response rate for surveys distributed at gay venues . Thus, the potential for selection bias is strong resulting in over-estimates of coverage and effects. These limitations also suggest the need for caution regarding the external validity of study findings. Information bias is also likely with many of the studies. Men completing questionnaires face-to-face with an interviewer may have been more likely to provide socially desirable responses compared to men self-completing questionnaires, thereby inflating estimates of study coverage, acceptability and effects. Several studies relied on participants’ own attributions to assess intervention effects [9, 21, 22], and these are likely to be vulnerable to information bias.
Confounding was another major source of potential bias. None of the included studies described attempts to control for confounding either via matching or adjustment of comparison groups. The cross–sectional studies were also vulnerable to potential reverse causality [15–18, 23–28].
Three studies included a nested qualitative component [16, 23–27]. None of these studies aimed to build theory. One reported methods of recruitment [23–26]. Men were purposively sampled to ensure, for example, representation of different ages and HIV statuses. None of the studies reported on methods used to analyse data. Nor did the studies provide in-depth quotes and thick description of context or aim to build theory.
Findings are presented in Table 2.
RQs 1 and 2: Campaign awareness and how this was affected by mode of delivery
Campaign awareness was examined in nine studies [14–18, 21–28]. A variety of designs were used to address the question of campaign awareness: four single cross-sectional studies [15–17, 27, 28] and five repeat cross-sectional studies [14, 18, 21–26]. Coverage ranged from 3% to 84%, but this is partly an artefact of different methods of elicitation, such as whether prompted or unprompted and the different time-frames used. Recognition of campaign imagery was in every study more prevalent than recall of campaign messages [14, 17, 23–26, 28].
Differences were noted in the characteristics of men who reported campaign awareness compared to those who did not in two studies. Sigma Research observed across surveys that campaigns were more likely to be seen by gay rather than bisexual-identified men, those with more male partners and those who had tested HIV positive compared to those who had not [23–26]. Men under 20 years of age and those over 50 were less likely to recall campaigns compared with men aged 20-50 years, as were those who had not tested for HIV, and those with low educational attainment. Hilliam et al. reported that non-gay identified men were less likely than gay-identified men to report awareness of the campaign when prompted (69% versus 82%, respectively) .
Due to heterogeneity of interventions and methods, it was impossible to determine whether or how mode of delivery affected campaign awareness.
RQs 3 and 4 Campaign acceptability and influences on this
Seven studies reported on acceptability or on the attributes that may affect acceptability [15, 16, 18, 21–28]. These drew on single cross-sectional [15, 16, 27, 28] and pre/post test cross-sectional design [18, 22], as well as focus groups [16, 23–26]. Hilliam et al. reported that in their evaluation of the “HIV Wake-up campaign” men were most likely to agree the campaign had a clear message (around a third of men) but least likely to agree that the campaign was motivating (less than 20% of men) . Four themes relating to increased acceptability were evident from the focus group studies: imagery (such as the use of models representative of the gay community, the benefits of comics for explicit material, and the importance of ensuring imagery and the campaign message complement one another); content (such as ensuring messages are not too complex); tone (such as not being patronising or blaming); and relevance (such as making certain messages are appropriate to the target audience) [16, 23–27]. The included studies looked at overall acceptability of the campaign rather than the acceptability of different modes of delivery.
RQs 5 and 6 Campaign effectiveness and cost-effectiveness, and effect of intervention characteristics on these
Knowledge and attitudes
None of the included studies reported on knowledge or attitude outcomes.
HIV testing and HIV status disclosure
Five studies reported on HIV testing [9, 16, 19, 21] or intention to test . Two of these examined campaigns where the primary aim was to encourage HIV testing [9, 22]. A repeat cross-sectional study with data on HIV testing behaviour pre- and post-intervention reported that testing increased significantly post-intervention, from 16% to 33% among gay-identified men and from 9% to 38% in non-gay-identified men . Another study found that 38% of HIV-negative men reported that they would test more frequently as a result of the intervention . However, another study reported that only 9.3% of men attending a sexual health clinic identified a campaign to encourage HIV testing as an important factor in their decision to have a test, although HIV testing data are not presented . In the study comparing intervention and control clinics , increases in HIV testing were observed among MSM in general in the campaign clinic, but the greatest increases were observed among Black, southern European and young men, images of whom featured prominently in the campaign. The authors report that whereas in the year prior to the campaign only one of the 65 MSM testing reported that they did so as a result of an advertisement, 162/292 did so during the campaign. No similar increases in testing were observed in the two comparison clinics. However, these findings should be interpreted with caution: men in the control clinics were not asked about the reasons for testing; other confounding factors could have affected the observed increases in testing and there may have been contamination across clinics. A limitation of the two studies in sexual health clinics was that the samples only included individuals having an HIV test [9, 19].
HIV status disclosure was examined in one cross-sectional study, which observed that 12% of men who reported seeing the campaign reported disclosing their HIV status to a partner . However, as there is no comparison group it is difficult to attribute disclosure of HIV status to the campaign.
Sexual behaviour outcomes
Four studies reported sexual behaviour outcomes: unprotected anal sexual intercourse , condom use [16, 29], number of sexual partners , and change in sexual practice [15, 28]. Two of the studies were single cross-sectional studies asking men about sexual behaviour post-intervention [15, 16, 28]. The remaining two studies compared pre- and post-intervention measures [9, 29], with neither study observing any significant differences. However, both these studies were retrospective case note reviews and are therefore limited by what information was recorded in the case notes.
None of the included studies examined cost-effectiveness of the campaigns.
Due to the broad range of interventions, designs and outcomes, it was not possible to examine how intervention characteristics affected outcomes.
Summary of key findings
The studies included examined different aspects of intervention process and diverse outcomes. Key limitations in methodology included unrepresentative samples, information bias and lack of control of confounding. We note particular problems with the cross-sectional studies comparing post-intervention measures of exposure, process and outcomes, in that these were likely to be subject to strong confounding and information bias [15–18, 23–28]. Repeat interrupted times series, a design used by three of the included studies [14, 21, 22], are likely to provide less biased estimates of intervention effects.
Intervention awareness was variable and recall of key messages among exposed men was generally poor. There was some evidence of lower awareness among non-gay than gay-identified men. Campaign acceptability was variable, and there was some evidence that attention to imagery, content, tone and campaign relevance could enhance acceptability. We found little rigorous evidence of significant effects of mass media interventions on MSM. There was some evidence of short-term increases in HIV testing. Our exploratory review found no rigorous evidence of intervention effects on sexual behaviour outcomes and on HIV status disclosure. As none of the studies were conducted beyond six months, it was not possible to assess sustained impact on behavioural outcomes. Changes in knowledge or attitudes or cost-effectiveness of the campaigns were not reported by any of the studies. We found no evidence addressing how intervention characteristics might influence effectiveness.
Because of our multiple research questions and the lack of consensus of which designs are most appropriate to examining these, we undertook an exploratory systematic mapping and appraisal of studies of HIV prevention mass media interventions targeting MSM. In the section below, we reflect on the potential for future systematic reviews of mass media interventions to define more focused questions and inclusion criteria for studies. Other limitations should be noted when interpreting our findings. First, most of the published evidence has come from high income countries. Second, only English language papers were included in the review, so literature in other languages would have been missed. Third, although we searched for unpublished literature in multiple ways, those reports we found came mostly from the UK, which may reflect our stronger domestic networks. Unpublished reports which were older or came from other settings are more likely to have been missed, especially perhaps if they reported null or negative findings. Finally, although interventions that only included small media were excluded, some of the multi-media campaigns did include small media, and it is possible that these may have affected outcomes of interest.
Implications for research
We highlight above the particular weakness of the included studies which drew on post-intervention measures of intervention exposure and outcomes in order to assess intervention effects. Even had these studies attempted to control for confounding differences between those reporting exposure to interventions and those not doing so, substantial residual confounding would very likely have remained because of the subtle differences between those recalling and not recalling campaigns. Therefore, we conclude that any future systematic reviews of mass media should focus on interrupted time-series studies examining pre- and post-intervention measures, drawing either on longitudinal data or from repeat cross-sectional data where this involved consistent sampling methods. Although such studies are vulnerable to confounding by secular trends in the outcomes in question, we conclude that this is a less important source of bias. We therefore also recommend that primary evaluations of mass media effects adopt this design. Nonetheless we stress the importance of cross-sectional studies in assessing awareness and a combination of cross-sectional studies and qualitative research in assessing acceptability.
Our review did not include any studies using interactive media such as smart phone applications or website-based risk assessment tools. Very few studies of such media have yet been published. The use of new technologies warrants further investment and research .
Evaluation surveys for the most part found that campaigns were acceptable to MSM. Campaigns need to be relevant to the target audience’s needs and formative work during development with target (and non-target groups if material might be viewed outside gay venues) was seen as key to ensuring imagery, language, tone and content were acceptable. Piloting and pre-testing of campaigns should be considered a prerequisite to any campaign launch.
Implications for policy
Mass media interventions have the potential to reach large audiences, and their cost is low per individual reached; though it should be noted that most of the costs provided in the studies focused on display and distribution only, and not on staff time and other development costs. We found insufficient evidence to determine whether mass media interventions represent an effective or cost-effective strategy in the prevention of HIV infection amongst MSM. We recommend that further research is required to investigate this, drawing on interrupted time-series designs and focusing on new/interactive media in addition to traditional/static media.
The aims of the studies in our exploratory review were generally focused on behaviour change, such as HIV disclosure or HIV testing, rather than information provision. A previous systematic review concluded that effective behaviour change interventions require a focus on interpersonal skills development rather than merely the provision of knowledge . The strength of mass media interventions is that they may have a small influence, but on a relatively large portion of the target population. They can also signpost more in-depth interventions, such as one-to-one interventions that are better at addressing motivation and skills. They can set the context in which norms can be changed and stigma addressed, but they cannot affect these things in isolation. This may be an argument for focusing mass media interventions on raising awareness and knowledge and delivering them alongside other more in-depth interventions in order to enable behaviour change.
We would like to thank the following people for their help in identifying potential studies for our review: Will Nutland, Catherine Dodds, Ford Hickson, Paul Flowers, Nicky Coia, Magaly Blas, Karen Hartfield and Lorraine Sherr.
This review was commissioned by Terrence Higgins Trust (THT) on behalf of CHAPS, a national HIV prevention partnership funded by the Department of Health for England (1996-2012). CHAPS was a partnership of community-based organisations carrying out HIV prevention and sexual health promotion with gay men, bisexual men and other MSM in England. Alongside THT it included The Eddystone Trust (South West England), GMFA (London), Healthy Gay Life (Birmingham), The Lesbian & Gay Foundation (Manchester), The Metro Centre (London), NAM, TRADE Sexual Health (Leicester), and Yorkshire MESMAC.
- UNAIDS: UNAIDS report on the global AIDS epidemic. 2011, New York NY: UNAIDSGoogle Scholar
- Wakefield MA, Loken B, Hornik R: Use of mass media campaigns to change health behaviour. Lancet. 2010, 376: 1261-1271.View ArticlePubMedPubMed CentralGoogle Scholar
- Wellings K, Macdowall W: Evaluating mass media approaches to health promotion: a review of methods. Health Educ. 2000, 100 (1): 23-32.View ArticleGoogle Scholar
- Noar SM, Palmgreen P, Chabot M, Dobransky N, Zimmerman R: A 10-Year Systematic Review of HIV/AIDS Mass Communication Campaigns: Have We Made Progress?. J Health Commun. 2009, 14 (1): 15-42.View ArticlePubMedGoogle Scholar
- Myhre SL, Flora JA: HIV/AIDS communication campaigns: progress and prospects. J Health Commun. 2000, 5 (Suppl): 29-45.View ArticlePubMedGoogle Scholar
- Bonell C, Hargreaves J, Cousens S, Ross D, Hayes R, Petticrew M, Kirkwood BR: Alternatives to randomisation in the evaluation of public-health interventions: design challenges and solutions. J Epidemiol Community Health. 2011, 65: 582-587.View ArticlePubMedGoogle Scholar
- Evans AR, Wiggins RD, Mercer CH, Bolding GJ, Elford J: Men who have sex with men in Britain: comparison of a self-selected internet sample with a national probability sample. Sex Transm Infect. 2007, 83: 200-205.View ArticlePubMedGoogle Scholar
- Vidanapathirana J, Abramson MJ, Forbes A, Fairley C: Mass media interventions for promoting HIV testing. Cochrane Database Syst Rev. 2005, Issue 3: Art. No.: CD004775-doi:10.1002/14651858.CD004775.pub2Google Scholar
- McOwan A, Gilleece Y, Chislett L, Mandalia S: Can targeted HIV testing campaigns alter health-seeking behaviour?. AIDS Care. 2002, 14 (3): 385-390.View ArticlePubMedGoogle Scholar
- Moher D, Liberati A, Tetzlaff J, Altman DG, for the PRISMA Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-View ArticlePubMedPubMed CentralGoogle Scholar
- Cochrane Handbook for Systematic Reviews of Interventions. Edited by: Higgins JPT, Green S. 2009, Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009. Available from http://www.cochrane-handbook.org Google Scholar
- Des Jarlais DC, Lyles C, Crepaz N, the TREND Group: Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND Statement. Am J Public Health. 2004, 94: 361-366.View ArticlePubMedPubMed CentralGoogle Scholar
- Shepherd J, Harden A, Rees R, Brunton G, Oliver S, Oakley A: Young people and healthy eating: a systematic review of research on barriers and facilitators. 2001, London: EPPI-CentreGoogle Scholar
- Hecht J, Riggs J, Hargraves H, Wei C, Raymond HF: “Are you iffy?”: A social marketing campaign to address uncertainty in HIV status communication among men who have sex with men. Sex Transm Dis. 2011, 38 (5): 457-458.View ArticlePubMedGoogle Scholar
- Lombardo AP, Léger YA: Thinking About “Think Again” in Canada: Assessing a Social Marketing HIV/AIDS Prevention Campaign. J Health Commun. 2007, 12 (4): 377-397.View ArticlePubMedGoogle Scholar
- Katzman J, Gulati H, Higa DH, Welch Q, Wood RW: A “Community Manifesto” for gay and bisexual men: an appeal to control HIV/STDs. J Public Health Manage Pract. 2007, 13 (3): 244-251.View ArticleGoogle Scholar
- Sherr L, Nardone A, Leaity S, Wells H, Mercey D, Elford J: “Try this HIV test”- an evaluation of a mass media campaign targeting homosexual men. Sex Transm Inf. 1999, 75 (4): 273-Google Scholar
- Dawson C, Hartfield K: Developing a cost-effective media campaign addressing unprotected anal sex among gay men. AIDS Educ Prev. 1996, 8 (4): 285-293.PubMedGoogle Scholar
- Griffith R, Mandalia S, Beck EJ, Kenny C, Watkins P, Claydon E, Harris JRW: HIV media campaigns and HIV-1 testing trends at a London genitourinary medicine clinic, 1985-1993. AIDS. 1995, 9 (12): 1367-1372.View ArticlePubMedGoogle Scholar
- Anonymous: Community education, Mexico. AIDS Action. 1990, 10: 5-Google Scholar
- Hilliam A, Fraser L, Turner L: HIV Wake-Up Campaign. 2011, NHS Health Scotland, http://www.hivscotland.com/downloads/1309967009-15922-HIV%20Wake%20Up%20Campaign%20Evaluation.pdf. Accessed 30th November 2011.Google Scholar
- Hartfield K, Burt R, Thiede H: HIV/AIDS Epidemiology Report. “It’s the Little Prick You Can Deal With” – Evaluation of an HIV Testing Promotion Campaign. 2009, Washington State: Seattle & King County, 41-44.Google Scholar
- Sigma Research: CHAPS end user evaluation: 2005-2009. 2010, London: CHAPS R&D ProgrammeGoogle Scholar
- Weatherburn P, Dodds C, Branigan P, Keogh P, Reid D, Hickson F, Henderson L, Nutland W: Form and focus: Evaluation of CHAPS national interventions, 2003 to 2006. 2007, London Sigma Research, http://www.sigmaresearch.org.uk/reports,Google Scholar
- Weatherburn P, Dodds C, Branigan P, Nutland W, Reid D, Keogh P, Henderson L, Hickson F, Stephens M: Successful measures: evaluation of CHAPS national HIV prevention campaigns targeted at gay men, 2001-2003. 2003, London Sigma Research, http://www.sigmaresearch.org.uk/reports,Google Scholar
- Weatherburn P, Henderson L, Reid D, Branigan P, Keogh P, Hickson F: Advertising awareness: evaluation of CHAPS national HIV adverts and leaflets targeted at gay men, 1996-2000. 2001, London Sigma Research, http://www.sigmaresearch.org.uk/reports,Google Scholar
- The TASC Agency: Report 1. December. Gay and Bisexual men’s views on the Equal campaign. 2007, Scotland: The TASC AgencyGoogle Scholar
- Trussler T, Marchand R: Prevention Revived: Evaluating the Assumptions Campaign. 2005, Vancouver: AIDS VancouverGoogle Scholar
- Roedling S, Reeves I, Copas AJ, Beattie A, Edwards SG, Fisher M, Benn P: Changes in the provision of post-exposure prophylaxis for HIV after sexual exposure following the introduction of guidelines and publicity campaigns. Int J STD AIDS. 2008, 19: 241-242.View ArticlePubMedGoogle Scholar
- Lewis MA, Uhrig JD, Ayala G, Stryker J: Reaching Men Who have Sex with Men for HIV Prevention Messaging with New Media: Recommendations From an Expert Consultation. Ann Forum Collab HIV Res. 2011, 13 (3): 1-9.Google Scholar
- Johnson WD, Diaz RM, Flanders WD, Goodman M, Hill AN, Holtgrave D, Marlow R, McClellan WM: Behavioral interventions to reduce risk for sexual transmission of HIV among men who have sex with men. Cochrane Database Syst Rev. 2008, Issue 3: Art no: CD001230-doi:10.1002/14651858.CD001230.pub2Google Scholar
- The pre-publication history for this paper can be accessed here:http://0-www.biomedcentral.com.brum.beds.ac.uk/1471-2458/14/616/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.