Table 4 Expected outcomes, evaluation activities and results for each of the 9 sites participating in Ophelia Victoria

Longer term: Improved knowledge of falls prevention. Increased motivation to undertake health promoting behaviours; Medium term: Community members feel cared for; gain practical support and information; mentors improve communication skills and understanding of specific health problems; mentors and community members have increased social connectedness; Short term: Mentors engage with community members.

1) Evaluation of HLQ scales 2, 3 & 4 pre-post intervention in mentors and senior citizens (including Arabic speaking women’s group)

2) Satisfaction surveys – mentors; 3) Interviews with mentors and all clients.

8 mentors, 18 senior citizens participated in evaluation. Mean (SD) age of mentors = 69.8 (5.8) years, 100% female and 100% spoke English as their first language.

In HLQ scales, mentors showed small to large improvements with ES ranging from 0.26 (95% CI −0.73, 1.24) for scale 3 to 0.92 (−0.13, 1.94) for scale 2. For the senior citizens group, HLQ scores showed no improvement in scale 4 (ES 0.10 (−0.95, 1.14)).

Interviews and focus groups with 18 senior citizens and mentors found most participants reported regularly applying what they learnt, increased mobility, and benefits from the social engagement. Mentors also reported an increase in their own confidence to support others and all reported a desire to continue in the mentorship role.

Longer term: Increased community members’ capacity to navigate and engage with health services; improved health literacy and engagement of volunteers; increased social connectedness; Short to medium term: Community members are educated about the local health service, including navigation and engaging with GPs; reduced social isolation

1) Administration of HLQ scales 2, 5 & 6 at pre and post intervention with community members and volunteers.

2) Interviews with community members and volunteers.

3) Capturing of potential wider community effects via interviews.

14 mentors participated in training and evaluation; 7 community members participated in evaluation, with an estimated n = 100 reached by the intervention. Demographic data on participants not collected.

In the HLQ scales, participants completing both pre and post questionnaires (n = 18) showed moderate increases, with moderate ES ranging from 0.52 (95% CI −0.13, 1.16) for scale 5 to 0.56 (−0.09, 1.20) for scales 2 and 6.

In interviews participants reported some GP's provided positive feedback on the Good Questions form. The form helped participants feel prepared and assertive during GP visits. The Better Health Channel: Improved awareness was evident. Some participants sought the help of a family member to gain access. Using volunteers who were active community members to deliver simple, word-of-mouth messages was reported as successful. Volunteers reported feeling useful and proud. Discussing one's health within immediate circles (family and community groups) reported frequently suggesting a ripple effect in terms of spread of the intervention's messages within existing circles

Longer term: Increased management of health and adherence to recommendations; able to find out about supports/services and information as required; Medium term: Open and insightful exchange between clients and their GP; Short term: Clients use new skills and strategies during GP visits

1) Pre and post questions from HLQ scales 6 & 9. Scale scores not calculated as questions were modified.

2) A brief survey of the utility of the tool for clients

3) Focus group with assessment officers

8 clients completed modified HLQ scales pre-post intervention; 5 completed the utility survey. 88% were female; age >65 years.

Focus group with four assessment officers

Overall client results showed slight increase in modified HLQ question scores.

All 5 clients completing the utility survey felt discussions with the assessment officer about how to talk with the GP were useful. There were mixed responses to resources; some clients reported they were useful and others reported they were too long.

Assessment officers reported that clients initially said they were happy with their relationship with their GP, but further questioning revealed many felt unheard by the GP. Case studies of positive outcomes when clients were encouraged to raise issues such as incontinence with their GP were discussed. Assessment officers reported being more aware of the need to question clients on this topic.

Longer term: Clients are able to apply learnings to future situations; Medium term: Improved ability to find health information on the web; improved capacity to understand and appraise health information; Short term: Targeted participants (older adults) attend and or/or access information (wider community)

1) Administration of HLQ scales 1, 2, 5 & 7 pre and post-intervention (scale 1 as comparison in which no change expected);

2) Client interviews at 2–4 weeks post intervention 2

3) Number of people attending computer course

11 clients participated in intervention 1 (computer course), 27 in intervention 2 (presentation of DVD and checklist during planned activity groups). Pre-post HLQ scales collected on 32.

Interviews with 12 participants from intervention 2. Demographic data not collected.

Changes in HLQ scales showed moderate increases with ES ranging from 0.43 (95% CI −0.07, 0.92) for scale 2 to 0.50 (0.00, 0.99) for scale 7. No change was seen in the comparison scale.

Interviews with participants from intervention 2 found 4 participants reported an increase in using the internet to search for health-related information post-intervention. Barriers were not having a computer/internet and a lack of need for any health related information; 6 participants reported Increased levels of confidence or increased awareness in ability to appraise online information. The checklist was described as a useful resource

Longer term: Community are optimally engaged with the service. Clients feel empowered to self-manage their health; Medium term: Increased awareness about the service; Staff are using a range of tools and strategies to engage and communicate with clients; The ‘My Health Diary’ is being used by 50% of eligible clients; Short term: Community engagement activities and promoting the service more broadly; Staff training around understanding the importance of health literacy and effective communication

My Health Diary: 1) Number of diaries taken; 2) Number of diaries being used, assessed by brief interviews with clients who consented to interview; 3) Clinician interviews

My Health Diary: 44 clients participated, 26 (62%) contacted for interview; mean age = 59 (17.0) years, 71% female; 92% with chronic condition. Interviews with n = 5 clinicians

My Health Diary: Of 26 clients interviewed, 6 reported using the diary. Interviews with clinicians found that staff felt uptake was low as diary was not formally promoted to clients, most of whom did not bring the diary with them to appointments. Different parts of the diary were felt to be more or less useful, with some replicating existing record systems. Two of the 5 clinicians interviewed reported the diary was easily understood by clients, who appeared to value having a concise record of health information.

Longer term: Clients feel understood and supported by healthcare providers; clients have sufficient information to manage their health; clients understand health information well enough to know what to do; Medium term: Nurses integrate resources and techniques into everyday practice; Short term: Increased awareness of the resources and techniques among nurses; nurses have sufficient knowledge and confidence to apply appropriately

1) Administration of HLQ scales 2, 5, 9 pre and post intervention (scale 2 was comparison scale in which no change was expected)

2) The Diabetes Knowledge Questionnaire (DKN) (pre and post intervention)

3) Interviews with clinicians

24 clients participated in the intervention; 15 provided pre-post HLQ data. Mean age 75 (13.2) years, 67% female. Mean years with diabetes 9.8 (9.5), 96% had type 2 diabetes.

Interviews with 9 clinicians

Client results for pre and post HLQ scales showed no improvement with ES of 0.08 (95% CI −0.64, 0.79) for scale 9 to 0.15 (−0.57, 0.87) for scale 5. Change in scale 2 = 0.04 (−0.67, 0.76). DKN scores indicated a small trend of improvement (ES = 0.24 (95% CI −0.43, 0.79).

Interviews with clinicians found the diabetes education checklist was user-friendly and helped staff reframe education content/delivery to suit needs of individual clients. Using teach-back helped staff identify clients’ learning requirements and built a rapport. Using the learning styles tool reinforced the importance of the learning trajectory to both clinicians and clients. Staff discussed case studies of clients who became more proactive, asked more questions or showed improvements in self-management of their care.

Longer term: Optimal use of health services by clients, preventing readmissions; Medium term: Clients have increased confidence to self-manage health and health crises; Short term: Improved client capacity to understand and use new health information and navigate health service.

1) Pre and post questions from HLQ. Scale scores not calculated as questions were modified;

2) Identification of client learning preferences;

3) Interviews with participating clinicians

In total, 70 clients participated; mean age = 76, 49% female, mean number of health conditions = 3.

Interviews with clinicians (n = 8)

Preferred methods of learning information were: Talking through with someone (83%); writing down (53%). Least popular methods were brochures (33%) and pictures or diagrams (26%). Preferred methods for receiving information were face to face (93%). Email was least preferred (9%).

Interviews with clinicians found teach-back 1) ensures client has an accurate understanding of what they need to do; 2) identifies gaps in clients' understanding; and 3) allows for better rapport between client and clinician. The health service navigation plan provided clients with a better knowledge of their services at the point of discharge. The learning styles tool was useful particularly for identifying clients with reading and language issues.

Longer term: Increased appropriate demand for early intervention health services; Medium term: Improved client capacity to understand and appraise new health information relevant to their needs; Increased confidence to self-manage health and health crises; increased capacity to effectively and appropriately engage with health services and providers; Short term: HARP clinicians collaborate with clients

1) Administration of HLQ scales 2, 4, 8 pre and post-intervention (scale 4 as comparison in which no change was expected).

2) Interviews with clients

2) Focus group and interviews with clinicians

48 clients completed the HLQ pre-post intervention; 11 participated in the interviews; mean age 63.9 (15.7) years; 45% female; mean number of health conditions 6.3 (4.3);

11 clients and 10 clinicians participated in interviews

Changes in HLQ scales showed no to small increases with ES ranging from 0.02 (95% CI −0.41, 0.45) for scale 2 to 0.24 (−0.19, 0.67) for scale 8. No change was seen in the comparison scale (scale 4).

Findings from the client interviews showed clients felt comfortable with the experience and with showing their understanding through actions or words. 4 clients expressed confidence using the appointment planner and reported it was a helpful resource.

Clinician interviews found the benefits of using teach-back were: 1) allows clients to take more ownership of their health; 2) builds on client's capabilities; 3) revealed clinicians’ misconceptions about client's level of understanding. The appointment planner was used less often. Clinicians noted it was a useful tool, but needed to be embedded into their practice. Clients appeared to have their own systems of managing appointments, although forgetfulness played a prominent role in recalling appointments. The Learning Styles Tool was praised by clinicians who felt it alerted them to client's literacy needs, and allowed tailoring their practice to the client's requirements. Others felt it helped focus on client preferences in contrast to clinician's expectations and assumptions.

Longer term: Increased access and links with local health services; strengthened relationship, trust and engagement with local health service; Medium term: improved client access local health services; Short term: Referral pathways between services are developed and clinicians undertake referrals.

1) Focus group with central intake staff; 2) Telephone survey with dental clients

7 clients, 3 dentists and 3 intake staff participated in the study and evaluation activities. Telephone survey with 7 dental clients. Demographic data not collected on clients. Focus group with 3 central intake staff

Telephone survey with dental clients indicated all clients were comfortable with the dentists raising health issues, and all thought the intervention was a good idea. In total, 4 clients were referred to new services of which 3 were pleased with the outcome. One person reported waiting a long time for their initial appointment with the primary health service provider.

Focus group with central intake and dental staff found the referral process between dental services and primary health care was efficient and not overly time-consuming. Staff reported the process increased clients' awareness of services available to them.

Longer term: Improved quality of life and health outcomes; Medium term: Increased capacity of clients to navigate the healthcare system resulting in early response to declining health to prevent unplanned readmission.

Short term: Increased knowledge of clients in engaging with the health system

For Care coordination: 1) Client case studies and interviews; 2) Clinician focus group

Care coordination: Focus group with 4 staff,

Care coordination: Staff focus group found the intervention avoided the need to repeatedly question clients and allowed recording of case-management information more efficiently, especially for short term clients with more acute needs. A case study of one client found that over 8-months, 22 episodes of care coordination were documented by 5 separate nurses, resulting in closer engagement with the GP and avoidance of one hospital admission