Treatments in research studies outperform the real-world results

Rigorous randomized controlled clinical trials conducted under ideal clinical conditions often show much bigger treatment effects on substance use disorder than is seen in the real world application of those same treatments.

One often over-looked factor that may partly account for this discrepancy is the intensity of the research assessments conducted during the initial research studies.  This is because the ongoing monitoring of clinical research participants may unintentionally enhance clinical outcomes over and above the effects of the actual treatment they receive.

WHAT PROBLEM DOES THIS STUDY ADDRESS?

It has been shown repeatedly that sustained contact with alcohol use disorder patients following intensive treatment enhances and extends the effectiveness of those treatments for alcohol use disorder ^{1 2 3}. Even assessments performed prior to the first treatment sessions lead to decreased alcohol consumption (44% of a 2005 sample stopped drinking during the pretreatment assessment period). It is important to understand the role that research-based assessments play in the success of clinical interventions, because those repeated assessments are not carried over into clinical practice.  In the real world, this would then potentially overestimate the effectiveness of an intervention that was shown to be effective under ideal conditions, and then put into clinical practice without the repeated assessment components.

HOW WAS THIS STUDY CONDUCTED?

A blinded randomized trial with a 2 x 2 design was performed to compare the effects of assessment frequency (frequent vs infrequent) and comprehensiveness (comprehensive vs brief) over 12 months.

READ MORE ON STUDY METHODS

DESIGN:  Frequent assessments involved quarterly in-person assessments and monthly telephone interviews (16 contacts over the year) whereas infrequent assessments involved only twice yearly in-person assessments with no telephone interviews (only 2 contacts after the baseline assessment in one year).  Comprehensive assessments focused on Alcoholics Anonymous (AA) participation, legal problems, medical and psychiatric status, personal relationship and occupational functioning in addition to alcohol and other substance use and the associated negative consequences.  Only the latter two assessments (alcohol and other substance use and negative consequences of that use) were obtained in the brief assessment groups.  The comprehensive in-person assessments required 1.5-2 hours to complete, whereas the brief in-person assessments took 0.5-1 hours to complete. The four groups were stratified by age, degree of alcohol dependence and prior alcohol treatment. Data was collected in addition from all participants from their collateral contact person to enhance verification of the information provided.

 

SETTING:  Two hospital-based outpatient alcohol- and other substance use disorder treatment clinics located in northeastern USA were chosen in part because the treatment provided was deemed to be among the most comprehensive in the state and included individual, group, family-based and support group treatment modalities.  Approximately 20% of those admitted to these outpatient treatment programs were deemed to be study-eligible.  Enrollment occurred across a 35-month period.  The second site was added towards the end of the study, representing 10% of the study participants.

 

PARTICIPANTS: Participants had to be adults referred/admitted to one of the two outpatient programs who met criteria for DSM IV alcohol abuse or dependence and who were willing to provide contact information for a person who could provide confirmatory information about the subject.   Exclusion criteria included a more severe non-alcohol substance use disorder, court mandated treatment, or organic brain impairment.  Of the 293 subjects recruited, 235 subjects were enrolled. 189 subjects (80%) completed the 12 month study.

 

MEASUREMENT:  Alcohol use (both reported by Timeline Followback and using breathalyzer measurements) and related negative consequences; AA participation, legal problems, medical and psychiatric status, personal relationship and occupational functioning were assessed at the 12 month time point.  Data analysis focused on the proportion of days abstinent, mean number of drinks per day and the proportion of heavy drinking days (6 or more drinks).

WHAT DID THIS STUDY FIND?

Both greater frequency and more comprehensiveness of the research assessments led to decreased alcohol use and decreased negative consequences from alcohol use.  Both increased frequency and increased comprehensiveness of the assessments independently decreased alcohol use in a statistically significant manner.  The poorest drinking outcome was seen in the group receiving infrequent, brief assessments.

Increased frequency may produce a larger effect on decreased drinking than increased comprehensiveness of assessments performed.  The infrequent, brief group reported being abstinent 69% of the days by the seventh month, as compared with >80% of the days in any of the remaining three groups (frequent-comprehensive or FC was 84%; frequent-brief or FB was 88%; and infrequent-comprehensive or IC was 80%).  Those receiving infrequent-brief assessments reported heavy drinking 21% of the days, as compared with approximately 10% for both frequent (FC and FB) groups and almost 14% for the infrequent-comprehensive IC group.  For this latter measurement, frequency of assessment produced a greater effect than comprehensiveness of assessment.

WHAT ARE THE IMPLICATIONS OF THE STUDY FINDINGS?

The findings in this study may explain, in part, why alcohol-specific treatment modalities appear to be more effective in the more ideal clinical research settings than in the real-world clinical practice settings. Even brief assessments, performed frequently, were as or more effective as longer, more thorough assessments.

In a clinical setting, maintaining long-term therapeutic contact (greater than 6 months) with patients who have completed substance use treatment programs is predicted to lead to decreased long-term substance use, even if those contacts are brief and non-therapeutic in nature.  This finding is supported by significant research showing that recovery management approaches including check-ups significantly enhance recovery.  In other words, have some kind of brief regular clinically-focused encounter and being asked about one’s alcohol/drug use may be enough to activate and mobilize at least some further recovery-related behavior change.

LIMITATIONS

1. Even the brief assessments were quite comprehensive (30 minutes to 1 hour) which may explain why the frequency of interventions appears to be a more potent driver of change in alcohol-related behaviors as compared with the comprehensiveness of the assessments.
2. The effects of follow-up assessments were tested on a group of participants who had received high-quality comprehensive and multi-modal outpatient treatment so the assessments are reinforcing effective interventions. If the initial treatment is not sufficiently effective (not tested here), the intensity of follow-up may not produce additional effects.
3. There may be a ceiling effect in the effectiveness of increased follow-up assessments and that may explain the lack of additional effects of both increased frequency together with increased comprehensiveness. If either assessment type alone has “maxed out” the effect then there is no additional benefit to combining the two, an effect that might be evident at lower doses of either.

BOTTOM LINE

CITATIONS

Clifford, P. R., & Davis, C. M. (2012). Alcohol treatment research assessment exposure: a critical review of the literature. Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors, 26(4), 773-81.