This article has Open Peer Review reports available.
Efficacy and cost-effectiveness of minimal guided and unguided internet-based mobile supported stress-management in employees with occupational stress: a three-armed randomised controlled trial
© Ebert et al.; licensee BioMed Central Ltd. 2014
Received: 12 July 2014
Accepted: 17 July 2014
Published: 7 August 2014
Internet- and mobile based stress-management interventions (iSMI) may be an effective means to address the negative consequences of occupational stress. However, available results from randomised controlled trials are conflicting. Moreover, it is yet not clear whether guided or unguided self-help iSMI provide better value for money. Internet-based mental health interventions without guidance are often much less effective than interventions including at least some guidance from a professional. However, direct comparisons in randomised controlled trials are scarce and, to the best of our knowledge, the comparative (cost)-effectiveness of guided vs. unguided iSMI has not yet been studied. Hence, this study investigates the acceptability and (cost-) effectiveness of minimal guided and unguided iSMI in employees with heightened levels of perceived stress.
A three-armed randomised controlled trial (RCT) will be conducted to compare a minimal guided and unguided iSMI with a waiting list control condition (WLC). Both active conditions are based on the same iSMI, i.e. GET.ON Stress, and differ only with regard to the guidance format. Employees with heightened levels of perceived stress (PSS ≥ 22) will be randomised to one of three conditions. Primary outcome will be comparative changes in perceived stress (PSS). Secondary outcomes include changes in self-reported depression, work-engagement, presenteeism and absenteeism. Moreover, a cost-effectiveness analysis will be conducted from a societal perspective, including both direct medical costs and costs related to productivity losses. In addition, a cost-benefit analysis will be conducted from the employer’s perspective. Incremental net-benefit regression analyses will address the question if there are any baseline factors (i.e. subgroups of employees) associated with particularly favorable cost-effectiveness when the experimental intervention is offered. Assessments take place at baseline, 7 weeks post-treatment and 6 months after randomisation.
Online-based (guided) self-help interventions could be an acceptable, effective and economically sustainable approach to offer evidence-based intervention alternatives to reduce the negative consequences associated with work-related stress. This study evaluates the (cost-) effectiveness of two versions of an iSMI, minimal guided and unguided iSMI. Thus, the present study will further enhance the evidence-base for iSMI and provide valuable information about the optimal balance between outcome and economic costs.
German Clinical Trial Registration (DRKS): DRKS00005687.
Occupational stress has shown to be a major risk factor for a range of health outcomes, such as depression , coronary disease ( and related mortality . Moreover, stress is also associated with substantial economic costs due to absenteeism, staff turnover and reduced work performance .
In the last decades, a number of interventions for occupational stress have been developed for which efficacy has been demonstrated in a large number of randomised controlled trials. When comparing different types of occupational stress management interventions (SMI) it has been shown that cognitive-behavioural programmes aimed at the individual consistently produced larger effects compared to other interventions (e.g., relaxation, organisational interventions) [5–7]. However, the majority of affected individuals remains untreated .
Using the Internet to provide self-help interventions may help to overcome some of the limitations of traditional SMI such as limited availability, high threshold and costs. Advantages of Internet-based interventions are besides others that: (1) they are easily accessible at any time and place, (2) anonymity is assured when employees want to avoid stigmatisation or self-disclosure in group settings, (3) participants can work at their own pace and review materials as often as they want, and (4) such interventions may reach affected employees earlier than traditional mental health services, hence preventing the onset of more severe mental health problems. Finally (5), internet-based interventions are easily scalable, implying that only a small increase of therapeutic resources is required for reaching a greater proportion of the eligible population using these interventions [9–13].
Internet-based interventions have shown to be effective in community and clinical settings, including the treatment of depression [14–16], anxiety [17, 18] and sleep disorders . However, only a few interventions have been developed and evaluated to address the specific needs of the working population. So far, RCTs on Internet-based SMI show conflicting results, with some studies reporting significant results with moderate effects sizes on perceived stress [12, 20] and others not finding significant results [21, 22].
Although it is often assumed that Internet-based interventions offer good value for money, evidence for their cost-effectiveness from randomised trials is still scarce and, to the best of our knowledge, completely absent when it comes to internet-based SMI in the work setting. Studies currently investigating the cost-effectiveness of mental health interventions for workers include a study on a guided self-help course for workers with depressive symptoms  and a study on a guided self-help regeneration training for stressed employees with work-related rumination and sleeping problems . Our group recently conducted a randomised controlled trial testing the efficacy and cost-effectiveness of an internet-based SMI in employees with heightened levels of perceived stress . Results are still pending. However, that latter study only evaluated an intervention including substantial professional support (up to 4 h per participant) which is thus very time-consuming and expensive. This clearly limits the possible reach of the intervention and, consequently, its potential to reduce the negative consequences of occupational stress at population level. Once developed, costs of internet-based interventions are substantially linked to professional guidance time. Thus, evaluating whether interventions with less intensive guidance up to 1 h per participant (minimal guidance, or unguided interventions) may still be effective appears promising.
Internet-based mental health interventions without guidance are often much less effective than interventions including at least some guidance from a professional [15, 16]. However, direct comparisons in randomised controlled trials are scarce. A recent systematic review identified only five of such randomised trials evaluating internet-based interventions for mental health related problems (Reichler et al. in prep). Guided interventions were significantly superior to unguided interventions (d = 0.22). Nevertheless, despite the possibly lower effectiveness of unguided interventions compared to guided interventions, unguided self-help may still produce larger effects at a population level with regard to the reduction of disease burden, given that more individuals can be reached for comparable costs. However, to the best of our knowledge, no study on the comparative efficacy and cost-effectiveness of guided and unguided SMI has been published so far.
Thus, this study aims at strengthening the evidence-base for internet- and mobile-based SMI (iSMI) by investigating the acceptability and (cost-) effectiveness of minimal guided and unguided iSMI in employees with heightened levels of perceived stress.
The study has the following specific aims: (1) To assess the effectiveness of minimal guided and unguided versions of iSMI for reducing perceived stress when compared to a waiting list control group (WLC). (2) To assess the comparative effectiveness of minimal guided and unguided iSMI. (3) To assess the cost-effectiveness of minimal guided and unguided iSMI compared to WLC. (4) To assess the comparative cost-effectiveness of minimal guided and unguided iSMI from a societal perspective. (5) To assess the cost to benefit ratio and return-on-investment of offering the interventions from an employer’s perspective. (6) To assess and compare acceptability of minimal guided and unguided iSMI. (7) To investigate in explorative analyses if there are any baseline factors (i.e. subgroups of employees) associated with particularly favourable cost-effectiveness when the experimental intervention is offered.
We hypothesise minimal guided and unguided iSMI both to be more effective and cost-effective than WLC; minimal guided iSMI to show higher effectiveness, cost-effectiveness and acceptability than unguided iSMI. Likewise, we expect guided iSMI to represent a better business case as seen from an employer’s perspective than unguided iSMI and WLC, respectively.
Participants & procedure
Inclusion and exclusion criteria
We will include a) currently employed workers b) above the age of 18 c) with scores ≥ 22 on the Perceived Stress Scale (PSS-10 ), d) who have internet access, e) sufficient German skills in reading and writing (self-report) and f) who are willing to give informed consent. We will exclude subjects a) reporting to have been diagnosed with psychosis or dissociative symptoms in the past, b) showing a notable suicidal risk as indicated by a score greater than 1 on BDI  Item 9 (“I feel I would be better off dead”).
The cut-off on the PSS was chosen to select participants with a heightened level of subjective stress as identified by one standard deviation (SD = 6.2) above the mean (PSS-10 = 15.3) in a large working population .
The study is integrated in the primary prevention and occupational health management programme of a large health insurance company in Germany. Participants are recruited via a) the insurer´s homepage, b) all regional health insurance offices of the cooperating insurance company (n = 918), c) advertisements in newspapers and d) an article in the members-journal (quarterly circulation 5.8 Million). Recruitment takes place between January and May 2014. Interested people can sign in for participation on http://www.geton-training.de. The research website provides information about the GET.ON stress training and details about the study. The trial is open to all individuals fulfilling the inclusion criteria and is not restricted to members of the cooperating health insurance company. People interested in participating in the study can apply online by providing the research team with their e-mail address.
Assessment of eligibility and randomisation
People who apply for study participation will receive an online information letter with detailed information about the study procedures and will be asked to provide an e-mail-address and a first and last name (which can be pseudonyms if desired) to participate. They will be informed that they can withdraw from the intervention and/or study at any time without any negative consequences. Applicants who continue to participate in the study will be asked to complete online screening questionnaires that assess the severity of their stress level (PSS ≥ 22), and to state whether they have a high suicidal risk (BDI Item 9 > 1) and whether they have been diagnosed with psychosis or dissociative symptoms in the past.
Participants meeting all of the inclusion and none of the exclusion criteria who have completed the baseline assessment and returned the informed consent form will enter the study and will be randomly allocated to study conditions. Randomisation will take place at an individual level. The allocation will be performed by an independent researcher not otherwise involved in the study using an automated computer-based random integer generator (randlist). During the randomisation process, allocation will be concealed from participants, researchers involved in recruitment, and eCoaches. Participants will be informed about the outcome of the randomisation and participants in the intervention group will receive immediate access to the GET.ON Stress training. However, the participants in the control group will receive the login data required to complete the training six months later than the intervention group.
Self-report assessments will take place at baseline, post-intervention (7 weeks), and at the 6-months follow-up. See Figure 1 for a detailed overview. Self-report data will be collected using a secured online-based assessment system (AES, 256-bit encrypted).
Content of the GET.ON stress training
Emotion regulation I
Muscle- and breathing relaxation
Emotion regulation II
Acceptance and tolerance of emotions
Emotion regulation III
Effective self-support in difficult situations
Plan for the future
Reflection on goal attainment and learning experiences. Implementation intentions until booster session
Reflection on goal attainment and learning experiences. Implementation intentions for the coming months
Participants of the “unguided” treatment-arm have contact with the study-administration team during the study period and are not supported by an eCoach. They are provided, however, with an e-mail address which can be contacted in case of any technical problems.
Minimal guided iSMI
Participants of the “minimal guided” treatment-arm are supported by an accompanying eCoach. The guidance manual is mainly based on the supportive-accountability model of providing guidance in internet interventions . In this model, it is argued that adherence to an Internet-intervention (and thereby effectiveness) can be enhanced via human support through accountability to a coach who is seen as legitimate, trustworthy, benevolent, and having expertise. In the current study, the purpose of the guidance will thus be to support participants to adhere to the treatment modules but will be kept to a minimum to minimise costs. Every participant will be assigned to one eCoach throughout the study. The eCoaches are trained psychologists and will follow guidelines about the feedback process that are defined according to the standardised manual for the intervention. Coach guidance consists of two elements: (a) adherence monitoring and (b) feedback on demand.
Adherence monitoring includes offering participants to support them to adhere to the intervention by regularly checking whether participants have completed intervention sessions on time, and if not, to remind them to do so. The eCoaches will send reminders in case that participants did not complete at least one session within 7 days. Both, personal and automatic reminders have shown to improve adherence to self-guided health promotion and behaviour change interventions [30, 31], but in the model of supportive accountability, it is assumed that personal reminders from a coach are perceived as benevolent and are more effective than automatic reminders. According to the model of supportive accountability, it is made clear to the participant that the aim of adherence monitoring is to provide feedback and that feedback in turn provides opportunities for self-reflection, thus aiming to help to achieve personal goals rather than exposing or punishing the participant.
Feedback on Demand includes offering participants the opportunity to contact the coach via the internal messaging system of the platform and receive individual support/feedback whenever such a need may arise. Within 48 hours, the participants will receive personalised written feedback. In contrast to other guidance concepts, coach guidance will only take place on initiative of participants. Feedback is not assumed to directly have an influence on the effectiveness of the intervention. Instead it aims at creating perceived legitimacy of the coach and a sense that the coach has the participant’s best interest at heart . It is assumed that people respond more positively to adherence demands from a coach who is perceived as legitimate [29, 33]. Hence, perceived legitimacy of the coach is assumed to further increase adherence to the intervention and to be a necessary precondition that adherence monitoring will have positive effects.
Primary and secondary outcomes
Primary outcome will be perceived stress at post-treatment. In secondary analyses, we will explore the effects of the intervention on depression, anxiety, emotional exhaustion, quality of life, absenteeism/presenteeism and numbers of participants displaying a reliable positive change. Economic analyses will be conducted assessing cost-effectiveness and cost-utility from a societal perspective including the costs of all types of health services and the costs that stem from productivity losses. Additionally, cost-benefit analyses from an employer’s perspective will be conducted. In explorative analyses we will also investigate whether there are any baseline factors (i.e. subgroups of employees) associated with particularly favorable cost-effectiveness when the experimental intervention is offered.
The German version of the ten-item-Perceived Stress Scale (PSS-10) [26, 34] will be used as a primary outcome measure. The PSS is “the most widely used instrument for measuring perceived stress” and assesses the degree to which people perceive their lives as stressful, particularly regarding how “unpredictable, uncontrollable and overloading respondents find their lives” . Cronbach’s alphas range for this scale from .78 to .91 . The scale is based on Lazarus’ transactional model of stress and, therefore, fits well with the theoretical basis of the intervention. Participants in this study will be asked to answer questions relating to the past week as opposed to the past month to avoid confounding with the training period. Similar procedures have been adopted in previous studies .
Depressive symptoms will be measured with the short version of the German version of the Center for Epidemiological Studies’ Depression Scale (CES-D) [36–38]. This frequently used self-report instrument consists of 15 items that are answered on a four-point Likert scale referring to the previous week. Total scores range from 0 to 60. The internal consistency of this measure has been found to be excellent (α = .95) .
The German version of the Maslach Burnout Inventory (MBI-GS-D) [40, 41] will be used to measure emotional exhaustion, the basic stress dimension of burnout. This commonly used self-report instrument consists of five items and uses a six-point Likert-type scale anchored by 1 = “never” and 6 = “very often”. The internal consistency of this subscale was α = .85 in a German sample .
The Utrecht Work Engagement Scale (UWES)  assesses work engagement defined as a “positive, fulfilling, work-related state of mind that is characterised by vigor, dedication, and absorption” . This scale has nine items, and the internal consistency of the total score is α = .91 . The reference period involves the last two weeks.
Quality of life
We will use the Short Form 12 (SF-12)  and the (EuroQol EQ-5D)  to assess quality of life. The SF-12  covers eight health domains (physical functioning, role limitations, pain, general health perception, vitality, mental health, emotional role and social functioning) and allows for the calculation of two sum scores for physical and mental health. We will also use the EQ-5D which is a widely applied, valid and reliable measurement of quality of life and consists of five items related to mobility, self-care, common activities, pain/discomfort and anxiety/depression. Furthermore, this measurement contains a visual analogue scale concerning health state valuation. The EQ-5D is only assessed for use in the health economic evaluation and is no secondary outcome itself.
Information on the participants’ use of health services will be obtained with the German Version of the Trimbos and institute of Medical Technology Assessment Cost Questionnaire for Psychiatry (TiC–P-G; ). With this questionnaire participants register the number of “work loss” days (absenteeism from work) and the number of “work cut-back” days (reduced efficiency at work while feeling ill), general practice visits, sessions with psychiatrists, hospital days, etc. will also be measured with help of the TiC–P-G. The questionnaire has shown to have a good retest-reliability and to achieve comparable results between patient-reported data and data derived from medical registrations .
In absence of a standardised measure for evaluating course satisfaction in internet-based treatments, user satisfaction will be measured with a self-designed questionnaire based on the Client Satisfaction Questionnaire (CSQ-8 , German Version ). This self-report scale consists of 8 items measuring the global client satisfaction with the internet-based training. Previous research indicated a high internal consistency .
To determine the numbers of participants achieving a reliable positive outcome we will code participants as responders or non-responders according to the widely used reliable change Index (RCI ). Participants will be considered responders, when they display a RCI score of above 1.96. To determine potential negative effects of the intervention on course of symptom  we will also report number of participants with reliable symptom deteriorations according to the RCI .
Perceived stress scale
Center for epidemiological studies depression scale
Maslach burnout inventory – emotional exhaustion
Emotion regulation skills questionnaire – general distress
Utrecht work engagement scale
Trimbos/iMTA questionnaire for costs associated with psychiatric illness
EuroQol, SF-12 quality of life
Demographic variables questionnaire
Effort reward imbalance questionnaire – short form
Client satisfactory questionnaire
Potential dropout reasons
Big five inventory – short form
Volitional components questionnaire – short form
General self-efficacy scale
Psychotherapy motivation questionnaire – short form
Internet affinity questionnaire
Work limitations questionnaire
The single-item presenteeism question
A single item question on work ability
Connor-Davidson resilience scale
Sample size calculation
We aim to include 408 participants. The study was designed based on the expected superiority of the active groups (minimal guided and unguided internet-based self-help) compared to the waiting-list control group on the primary outcome variable (i.e. perceived stress) at post-test. Based on pilot-evaluation data we expect a mean effect for the minimal guided treatment arm compared to the waiting list control group of d = 0.70. However, unguided internet-based interventions consistently produce lower effect sizes as guided interventions  and a meta-analysis on traditional face-to-face interventions for work-related stress  yielded an overall combined effect size of only d = 0.34. As we also want to examine the difference in the effectiveness of minimal guided and the unguided iSMI we aim to be able to detect difference between groups with an effect size of d = 0.30. Thus we need to include 408 participants. This sample will allow us to detect a between-group effect size (ES) of d = 0.30 with a power (1-ß) of 80 % and an alpha of .05 (calculated using PASS 12) in a one-tailed test (for uni-directional hypotheses H1: WLC < unguided < minimal guided iSMI).
The trial will be conducted in compliance with the study protocol, the Declaration of Helsinki and GCP. Aiming at an intention-to-treat design we will include all participants who will be randomly assigned to conditions. Additionally, per protocol analyses (PPA) will be conducted, including only participants’ satisfying protocol treatment. Analyses of variance will be conducted to explore the effects of the treatments on all primary and secondary outcomes. Missing data will be handled using multiple imputations (MI). MI is especially robust with respect to missing data . For all analyses, Cohen’s d  will be calculated by standardising the differences between baseline and follow-up scores by the pooled standard deviation of the baseline scores. We will also calculate the number needed-to-be-treated (NNT) with GET.ON Stress minimal guided and unguided to achieve one response compared to the control group. For all statistical analyses, significance level will be set at p < .05, one-sided. The effectiveness evaluation will be conducted in accordance with the CONSORT statement .
Cost-effectiveness and cost-utility analyses
In the cost-effectiveness analyses, treatment response (reliable positive change) will be the primary outcome, whereas quality adjusted life years (QALYs) will be the outcome in the cost-utility analyses. A cost-effectiveness acceptability curve  will be calculated, when better outcomes are associated with higher costs. Such an acceptability curve represents the probability that the intervention is cost-effective relative to the control group, given varying thresholds for the willingness to pay (WTP) for gaining one additional treatment responder or gaining one additional QALY. Nonparametric bootstrapping will be applied to estimate confidence intervals for mean differences in costs and effects between groups.
Cost-benefit analysis from the employer’s perspective
As indicated, a cost-benefit analysis will be conducted from the employer’s perspective. Costs, C, will be equated with intervention costs; benefits, B, with greater productivity owing to lesser absenteeism and lesser presenteeism. The ratio C/B is the cost-benefit ratio, and its inverse, B/C equals the return-on-investment, while the net-benefit is defined as B-C, which is the payout of offering the intervention. All health-economic evaluations will be conducted in accordance with the CHEERS statement for reporting health economic studies . Sensitivity analyses directed at uncertainty in the main cost-drivers will be conducted to ascertain the robustness of the outcomes.
Incremental net-benefit regression analysis
Effect modification can be studied in the context of a health-economic evaluation to address the question if there are any baseline factors (i.e. subgroups of employees) associated with particularly favourable cost-effectiveness when the experimental intervention is offered. The data analytic approach taken here consists of incremental net-benefit regression analysis (INBRA). First, incremental net-benefits, Δ(NB), are calculated as Δ(NB) = WTP × Δ(QALY) - Δ(Costs), where WTP × Δ(QALY) is the willingness to pay for gaining one QALY and Δ(Costs) are the extra costs owing to offering the intervention. Δ(NB)s are calculated for each participant in the dataset and can then be analysed in a regression framework. In its simplest form the incremental net-benefits are regressed on the treatment indicator variable to evaluate the cost-effectiveness of the intervention relative to the control condition. When Δ(NB) exceeds €0, then the intervention is deemed to be cost-effective. The regression models can be expanded to include interaction terms of the treatment indicator variable and putative effect modifiers [74, 75].
Occupational stress has shown to be a major risk factor for a range of health outcomes, and is moreover associated with substantial economic costs. Online-based (guided) self-help interventions could be an attractive, efficient and cost-effective approach to offer evidence-based intervention alternatives to reduce the negative consequences associated with work-related stress. However, results on iSMI are still conflicting. Moreover, it is unclear whether guided or unguided self-help approaches would provide better value for money when implemented at a large scale. The marginal costs of treating a person with a pure unguided Internet-intervention will get smaller and smaller when more people would make use of the e-health intervention alone. However, the costs of guidance by a professional coach are fixed and remain the same for every other person being treated. Thus, research on the dose–response relationship (e.g. guidance yes/no) is one of the most important research questions from an economic viewpoint. However, to date, the response–dose relationship has not been examined for iSMI nor for other internet-based occupational health management interventions.
A previous study on GET.ON Stress evaluated an intervention version with weekly guidance from an accompanying eCoach . This study now evaluates the acceptability, comparative efficacy and cost-effectiveness of two more economic versions of support in iSMI, namely minimal guidance and no guidance. Thus, the present study will provide valuable information about optimal pay-off between treatment outcome and economical costs necessary before wide dissemination of such interventions.
This study will also have some limitations. First, this study does not include any objective measurement of stress (e.g. cortisol levels). Due to feasibility limitations, only self-report measurements will be examined. Although self-reports always carry the risk of introducing subjective biases, it has been suggested that replacing self-reports with stress-related physiological measurements is not promising . Second, we only include participants experiencing heightened levels of perceived stress, as indicated by a cutoff score of ≥ 22 on the Perceived Stress Scale. Thus, our results will only be generalisable to employees fulfilling this criterion. In occupational health routine practice SMI are often offered to the whole working population (i.e. universal prevention approach) instead of selecting participants based on baseline stress level. Therefore, future studies should evaluate this iSMI using a universal prevention approach to allow for a reliable estimation of efficacy and cost-effectiveness of the intervention for this target group. Finally, the study is powered to detect differences between groups in the reduction of perceived stress. Hence, the sample size will be too small to draw reliable conclusion for potential moderating effects between guided and unguided groups with regard to the clinical effectiveness of the intervention. Which participants are thus likely not to profit from unguided but from minimal guided iSMI will remain unclear. However, health-economic evaluations do not test hypotheses. Instead, a probabilistic decision-making approach is undertaken. Thus, this study will address the question “who benefits most?” in explorative analyses by relying on an incremental net-benefit regression analysis.
There will also be several strengths of this study, including the randomised controlled design with the direct comparison of two active conditions, an appropriate statistical analysis plan and handling of missing data with state of the art methods. Moreover, to the best of our knowledge, direct comparisons of the costs and benefits of different levels of support in internet-based occupational health management interventions have not been published so far.
Overall, to overcome the gap between the need for effective strategies for managing negative consequences of occupational stress and evidence-based treatment availability/utilisation, (cost-) effective low-threshold interventions are needed that are accessible for as many people as possible. Internet-based occupational health interventions might be a promising strategy to overcome some of the limitations of face-to-face occupational health interventions. This study will enhance the evidence-base for iSMI and provide information about the differential acceptability, efficacy and cost-effectiveness of two economic support forms in iSMI.
This study is funded by the European Union (EFRE: ZW6-80119999, CCI 2007DE161PR001) and the BARMER GEK. Furthermore, we would like to acknowledge Angelina Scheel and Torsten Tarnowski for contributing to the study administration.
Recruitment began on February 1, 2014 and will continue approximately through 30.10.2014.
- Hammen C: Stress and depression. Annu Rev Clin Psychol. 2005, 1: 293-319. doi:10.1146/annurev.clinpsy.1.102803.143938PubMedGoogle Scholar
- Richardson S, Shaffer JA, Falzon L, Krupka D, Davidson KW, Edmondson D: Meta-analysis of perceived stress and its association with incident coronary heart disease. Am J Cardiol. 2012, 110 (12): 1711-1716. doi:10.1016/j.amjcard.2012.08.004PubMedPubMed CentralGoogle Scholar
- Steptoe A, Kivimäki M: Stress and cardiovascular disease. Nat Rev Cardiol. 2012, 9 (6): 360-370. doi:10.1038/nrcardio.2012.45PubMedGoogle Scholar
- Kalia M: Assessing the economic impact of stress–the modern day hidden epidemic. Metabolism. 2002, 51 (6 Suppl 1): 49-53. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/12040542 PubMedGoogle Scholar
- Richardson KM, Rothstein HR: Effects of occupational stress management intervention programs: a meta-analysis. J Occup Health Psychol. 2008, 13 (1): 69-93. doi:10.1037/1076-8922.214.171.124PubMedGoogle Scholar
- Van der Klink JJ, Blonk RW, Schene AH, van Dijk FJ: The benefits of interventions for work-related stress. Am J Public Health. 2001, 91 (2): 270-276. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1446543&tool=pmcentrez&rendertype=abstract PubMedPubMed CentralGoogle Scholar
- Bhui KS, Dinos S, Stansfeld SA, White PD: A synthesis of the evidence for managing stress at work: a review of the reviews reporting on anxiety, depression, and absenteeism. J Environ Public Health. 2012, 2012: 515874-doi:10.1155/2012/515874PubMedPubMed CentralGoogle Scholar
- American Psychological Association: Stress in America™: Missing the Health Care Connection. [https://www.apa.org/news/press/releases/stress/2012/full-report.pdf]
- Cuijpers P, van Straten A, Warmerdam L, van Rooy MJ: Recruiting participants for interventions to prevent the onset of depressive disorders: possible ways to increase participation rates. BMC Health Serv Res. 2010, 10: 181-doi:10.1186/1472-6963-10-181PubMedPubMed CentralGoogle Scholar
- Buntrock C, Ebert DD, Lehr D, Cuijpers P, Riper H, Smit F, Berking M: Evaluating the efficacy and cost-effectiveness of web-based indicated prevention of major depression: design of a randomised controlled trial. BMC Psychiatry. 2014, 14 (1): 25-doi:10.1186/1471-244X-14-25PubMedPubMed CentralGoogle Scholar
- Andersson G, Titov N: Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014, 13 (1): 4-11. doi:10.1002/wps.20083PubMedPubMed CentralGoogle Scholar
- Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B: E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther. 2007, 36 (3): 179-192. doi:10.1080/16506070701381863PubMedGoogle Scholar
- Ebert DD, Lehr D, Boß L, Riper H, Cuijpers P, Andersson G, Thiart H, Heber E, Berking M: Efficacy of an Internet-based problem-solving training for teachers: Results of a randomized controlled trial. Scand J Work Environ Health.Google Scholar
- Hedman E, Ljótsson B, Lindefors N: Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012, 12 (6): 745-764. doi:10.1586/erp.12.67PubMedGoogle Scholar
- Richards D, Richardson T: Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012, 32 (4): 329-342. doi:10.1016/j.cpr.2012.02.004PubMedGoogle Scholar
- Johansson R, Andersson G: Internet-based psychological treatments for depression. Expert Rev Neurother. 2012, 12 (7): 861-869. quiz 870. doi:10.1586/ern.12.63PubMedGoogle Scholar
- Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G: Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther. 2009, 38 (2): 66-82. doi:10.1080/16506070802694776PubMedGoogle Scholar
- Mayo-Wilson E, Montgomery P: Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev. 2013, 9: CD005330-doi:10.1002/14651858.CD005330.pub4PubMedGoogle Scholar
- Cheng SK, Dizon J: Computerised cognitive behavioural therapy for insomnia: a systematic review and meta-analysis. Psychother Psychosom. 2012, 81 (4): 206-216. doi:10.1159/000335379PubMedGoogle Scholar
- Zetterqvist K, Maanmies J, Ström L, Andersson G: Randomized controlled trial of internet-based stress management. Cogn Behav Ther. 2003, 32 (3): 151-160. doi:10.1080/16506070302316PubMedGoogle Scholar
- Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A: A field test of a web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: randomized controlled trial. J Med Internet Res. 2007, 9 (2): e17-doi:10.2196/jmir.9.2.e17PubMedPubMed CentralGoogle Scholar
- Shimazu A, Kawakami N, Irimajiri H, Sakamoto M, Amano S: Effects of web-based psychoeducation on self-efficacy, problem solving behavior, stress responses and job satisfaction among workers: a controlled clinical trial. J Occup Health. 2005, 47 (5): 405-413. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/16230834 PubMedGoogle Scholar
- Geraedts AS, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P: Web-based guided self-help for employees with depressive symptoms (Happy@Work): design of a randomized controlled trial. BMC Psychiatry. 2013, 13: 61-doi:10.1186/1471-244X-13-61PubMedPubMed CentralGoogle Scholar
- Thiart H, Lehr D, Ebert DD, Sieland B, Berking M, Riper H: Log in and breathe out: efficacy and cost-effectiveness of an online sleep training for teachers affected by work-related strain–study protocol for a randomized controlled trial. Trials. 2013, 14: 169-doi:10.1186/1745-6215-14-169PubMedPubMed CentralGoogle Scholar
- Heber E, Ebert DD, Lehr D, Nobis S, Berking M, Riper H: Efficacy and cost-effectiveness of a web-based and mobile stress-management intervention for employees: design of a randomized controlled trial. BMC Public Health. 2013, 13: 655-doi:10.1186/1471-2458-13-655PubMedPubMed CentralGoogle Scholar
- Cohen S, Kamarck T, Mermelstein R: A global measure of perceived stress. J Health Soc Behav. 1983, 24 (4): 385-396. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/6668417 PubMedGoogle Scholar
- Beck AT, Steer A, Brown GK: BDI-II: Beck Depression Inventory Manual. 2nd ed. San Antonio, TX: Psychological Corporation; 1996. 1996, San Antonio: Psychological Corporation, 2Google Scholar
- Lesage F-X, Berjot S, Deschamps F: Psychometric properties of the French versions of the perceived stress scale. Int J Occup Med Environ Health. 2012, 25 (2): 178-184. doi:10.2478/S13382-012-0024-8PubMedGoogle Scholar
- Mohr DC, Cuijpers P, Lehman K: Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. 2011, 13 (1): e30-doi:10.2196/jmir.1602PubMedPubMed CentralGoogle Scholar
- Titov N, Dear BF, Johnston L, Lorian C, Zou J, Wootton B, Spence J, McEvoy PM, Rapee RM: Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. Andersson G, ed. PLoS ONE. 2013, 8 (7): e62873-PubMedPubMed CentralGoogle Scholar
- Fry JP, Neff RA: Periodic prompts and reminders in health promotion and health behavior interventions: systematic review. J Med Internet Res. 2009, 11 (2): e16-doi:10.2196/jmir.1138PubMedPubMed CentralGoogle Scholar
- Tyler TR: Psychological perspectives on legitimacy and legitimation. Annu Rev Psychol. 2006, 57: 375-400. doi:10.1146/annurev.psych.57.102904.190038PubMedGoogle Scholar
- Tyler TR: The psychology of legitimacy: a relational perspective on voluntary deference to authorities. Pers Soc Psychol Rev. 1997, 1 (4): 323-345. doi:10.1207/s15327957pspr0104_4PubMedGoogle Scholar
- Michalsen A, Jeitler M, Brunnhuber S, Luedtke R, Büssing A, Musial F, Dobos G, Kessler C: Iyengar yoga for distressed women: a 3-armed randomized controlled trial. Evid Based Complement Alternat Med. 2012, 2012: 408727-doi: 10.1155/2012/408727PubMedPubMed CentralGoogle Scholar
- Cohen S, Janicki-Deverts D: Who’s stressed? Distributions of psychological stress in the United States in probability samples from 1983, 2006, and 20091. J Appl Soc Psychol. 2012, 42 (6): 1320-1334. doi:10.1111/j.1559-1816.2012.00900.xGoogle Scholar
- Radloff LS: The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977, 1 (3): 385-401. doi:10.1177/014662167700100306Google Scholar
- Hautzinger M, Bailer M: Allgemeine Depressions Skala. Manual. 1993, Göttingen: Beltz Test GmbH, Available at: http://www.psycontent.com/content/3x081j2m2732r345/ Google Scholar
- Lehr D, Hillert A, Schmitz E, Cut-off-werten EE Von: Screening depressiver Störungen mittels Allgemeiner Depressions- Skala ( ADS-K ) und State-Trait Depressions Scales ( STDS-T ). 2008, 1-10. doi:10.1026/0012-1924.54.1Google Scholar
- Lehr D, Hillert A, Schmitz E, Sosnowsky N: Assessing depressive disorders using the Center for Epidemiologic Studie. Diagnostica. 2008, 2: 61-70. doi:10.1026/0012-19126.96.36.199Google Scholar
- Schaufeli W, Leiter MP, Maslach C, Jackson SE: Maslach Burnout Inventory - General Survey (MI-GS). Maslach Burnout Inventory Manual. Edited by: Maslach C, Jackson SE, Leiter MP. 1996, Palo Alto, CA: Consulting Psychologists PressGoogle Scholar
- Neubach B, Schmidt K: Gütekriterien einer deutschen Fassung des Maslach Burnout Inventory (MBI-D). Eine Replikationsstudie bei Altenpflegekräften. Zeitschrift für Arbeits-und …. 2000, Available at: http://www.psycontent.com/index/98276L18337760W3.html Google Scholar
- Schaufeli W, Bakker A: Utrecht Work Engagement Scale: Test Manual. 2003, Utrecht: The Netherlands Departmant of Psychology, Utrecht UniversityGoogle Scholar
- Schaufeli W, Salanova M: The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. J Happiness …. 2002, Available at: http://0-link.springer.com.brum.beds.ac.uk/article/10.1023/A:1015630930326 Google Scholar
- Ware J, Kosinski M, Keller SD: A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34 (3): 220-233. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/8628042 PubMedGoogle Scholar
- Balestroni G, Bertolotti G: [EuroQol-5D (EQ-5D): an instrument for measuring quality of life]. Monaldi Arch Chest Dis. 2012, 78 (3): 155-159. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/23614330 PubMedGoogle Scholar
- Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan SS, Hakkaart-van Roijen L: Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013, 13: 217-doi:10.1186/1472-6963-13-217PubMedPubMed CentralGoogle Scholar
- Attkisson CC, Zwick R: The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982, 5 (3): 233-237. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/10259963 PubMedGoogle Scholar
- Schmidt J, Lamprecht F, Wittmann WW: Satisfaction with inpatient management. Development of a questionnaire and initial validity studies. Psychother Psychosom Med Psychol. 1989, 39 (7): 248-255.PubMedGoogle Scholar
- Jacobson NS, Truax P: Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991, 59 (1): 12-19. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/2002127 PubMedGoogle Scholar
- Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, Ljótsson B, Kaldo V, Titov N, Carlbring P: Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv. 2014, 1 (1): 12-19. doi:10.1016/j.invent.2014.02.001Google Scholar
- Ebert DD, Lehr D, Baumeister H, Boß L, Riper H, Cuijpers P, Reins JA, Buntrock C, Berking M: GET.ON Mood Enhancer: efficacy of Internet-based guided self-help compared to psychoeducation for depression: an investigator-blinded randomised controlled trial. Trials. 2014, 15 (1): 39-PubMedPubMed CentralGoogle Scholar
- Li J, Loerbroks A, Jarczok MN, Schöllgen I, Bosch JA, Mauss D, Siegrist J, Fischer JE: Psychometric properties and differential explanation of a short measure of effort-reward imbalance at work: a study of industrial workers in Germany. Am J Ind Med. 2012, 55 (9): 808-815. doi:10.1002/ajim.22018PubMedGoogle Scholar
- Berking M, Znoj H: Entwicklung und Validierung eines Fragebogens zur standardisierten Selbsteinschätzung emotionaler Kompetenzen (SEK-27). Zeitschrift für Psychiatr Psychol und …. 2008, Available at: http://www.psycontent.com/index/K300NR32641827R4.pdf Google Scholar
- Ebert D, Christ O, Berking M: Entwicklung und Validierung eines Fragebogens zur emotionsspezifischen Selbsteinschätzung emotionaler Kompetenzen (SEK-ES). Diagnostica. 2013, Available at: http://www.psycontent.com/index/F224M028N22W4175.pdf Google Scholar
- Rammstedt B, John OP: Measuring personality in one minute or less: a 10-item short version of the big five inventory in English and German. J Res Pers. 2007, 41 (1): 203-212. doi:10.1016/j.jrp.2006.02.001Google Scholar
- Forstmeier S, Rüddel H: Measuring volitional competences: Psychometric properties of a short form of the volitional components questionnaire (VCQ) in a clinical sample. TOPSYJ. 2008, 1: 66-77.Google Scholar
- Schwarzer R, Jerusalem M: Generalized Self-Efficacy scale. Measures in health psychology: A user’s portfolio. Causal and control beliefs. Edited by: Weinman J, Wright S, Johnston M. 1995, Windsor, UK: nferNelson, 35-37.Google Scholar
- Schwarzer R, Diehl M, Schmitz GS: Self-Regulation Scale. 1999, [http://www.fu-berlin.de/gesund/skalen]Google Scholar
- Bertrams A, Dickhäuser O: Messung dispositioneller Selbstkontroll-Kapazität. Diagnostica. 2009, 55 (1): 2-10. doi:10.1026/0012-19188.8.131.52Google Scholar
- Tangney JP, Baumeister RF, Boone AL: High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. J Pers. 2004, 72 (2): 271-324. Available at: http://0-www.ncbi.nlm.nih.gov.brum.beds.ac.uk/pubmed/15016066 PubMedGoogle Scholar
- Schulz H, Lang K, Nübling R, Koch U: Psychometrische Überprüfung einer Kurzform des Fragebogens zur Psychotherapiemotivation - FPTM-23. Diagnostica. 2003, 49 (2): 83-93. doi: 10.1026//0012-19184.108.40.206Google Scholar
- Schulz H, Nübling R, Rüddel H: Entwicklung einer Kurzform eines Fragebogens zur Psychotherapiemotivation. Verhaltenstherapie. 1995, 5: 89-95.Google Scholar
- Schwarzer R, Sniehotta FF, Lippke S, Luszczynska A, Scholz U, Schüz B, Wegner M, Ziegelmann JP: On the assessment and analysis of variables in the health action process approach: Conducting an investigation. 2013, Free university of berlin. Available at: http://www.fu-berlin.de/gesund/hapa_web.pdf Google Scholar
- Papacharissi Z, Rubin AM: Predictors of Internet Use. J Broadcast Electron. 2000, 44 (2): 175-196.Google Scholar
- Campbell-Sills L, Stein MB: Psychometric analysis and refinement of the connor-davidson resilience scale (CD-RISC): Validation of a 10-item measure of resilience. J Trauma Stress. 2007, 20 (6): 1019-28.PubMedGoogle Scholar
- Lerner DJ, Amick BC, Rogers WH, Malspeis S, Bungay K: The work limitations questionnaire: a self-administered instrument for assessing on-the-job work disability. Med Care. 2001, 39: 72-85.PubMedGoogle Scholar
- Kigozi J, Lewis M, Jowett S, Barton P, Coast J: Construct validity and responsiveness of the single-item presenteeism question in patients with lower back pain for the measurement of presenteeism. Spine (Phila Pa 1976). 2014, 39 (5): 409-416. doi:10.1097/BRS.0000000000000162Google Scholar
- Ahlstrom L, Grimby-Ekman A, Hagberg M, Dellve L: The work ability index and single-item question: associations with sick leave, symptoms, and health – a prospective study of women on long-term sick leave. Scand J Work Environ Health. 2010, 36 (5): 404-412. doi:10.5271/sjweh.2917PubMedGoogle Scholar
- Schafer JL, Graham JW: Missing data : our view of the state of the art. Psychol Methods. 2002, 7 (2): 147-177. doi:10.1037//1082-989X.7.2.147PubMedGoogle Scholar
- Cohen J: Statistical Power Analysis for the Behavioral Sciences. 1988, Available at: http://books.google.de/books/about/Statistical_Power_Analysis_for_the_Behav.html?hl=de&id=Tl0N2lRAO9oC&pgis=1 Google Scholar
- Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T: Academia and clinic the revised CONSORT statement for reporting randomized trials. Ann Intern Med. 2001, 134 (8): 663-694.PubMedGoogle Scholar
- Fenwick E, Byford S: A guide to cost-effectiveness acceptability curves. Br J Psychiatry. 2005, 187: 106-108.PubMedGoogle Scholar
- Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, Augustovski F, Briggs AH, Mauskopf J, Loder E, CHEERS Task Force: Consolidated health economic evaluation reporting standards (CHEERS) statement. BMJ. 2013, 346 (mar25_1): f1049-doi:10.1136/bmj.f1049PubMedGoogle Scholar
- Hoch J: All dressed up and know where to go: An example of how to use net benefit regression to do a cost-effectiveness analysis with person-level data (The “A” in CEA). 2008, 175-183.Google Scholar
- Hoch JS, Briggs AH, Willan AR: Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ. 2002, 11 (5): 415-430. doi: 10.1002/hec.678PubMedGoogle Scholar
- Semmer NK, Grebner S, Elfering A: Beyond Self-Report: Using Observational, Physiological, and Situation-Based Measures in Research On Occupational Stress. Research in Occupational Stress and Well-being.Vol 3. Research in Occupational Stress and Well-being. 2003, Bingley: Emerald (MCB UP), 205-263. doi:10.1016/S1479-3555(03)03006-3Google Scholar
- The pre-publication history for this paper can be accessed here:http://0-www.biomedcentral.com.brum.beds.ac.uk/1471-2458/14/807/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.