Skip to main content

Table 2 Randomised controlled study options and key difficulties

From: Challenges in evaluating Welfare to Work policy interventions: would an RCT design have been the answer to all our problems?

Study design

Disadvantages specific to study design

Key difficulty

Outcome

Recruitment into study by GP followed by randomisation

Self-referral to PAS increases risk of contamination of comparison group

Service is available external to the study

Dilution bias

Underestimate of effect

 

GPs may refer those who they think most in need/most likely to benefit - rather than recruit to the study

Group being evaluated not representative of those using the service

 
 

This would half the flow of patients being referred to PAS

PAS may not be sustainable

 
 

Requires high levels of co-operation from GP and PAS

Resource implications for GPs/PAS

 

Cluster randomisation

Need to identify IB recipients in comparison practices who would be eligible for referral to PAS: it is likely that this would only be around 20% of the total sample

Non-specific criteria for referral to service limits our capacity to identify an appropriate comparison group

Possible selection bias depending on ability to match controls

 

Cluster level differences need to be accounted for

Requires high levels of collaboration with policy makers well before implementation of pilot

Not possible given that PAS had been rolled out by the time of this evaluation