Unblinded testers inflated strength gains in trials

Resistance training research in older adults underpins a lot of confident advice about how much strength can be regained with age. But strength testing involves a human being encouraging, positioning and recording a participant, and that person often knows which group the participant is in. Knowing the treatment assignment can subtly shift how a test is administered and scored, and failing to analyse everyone in the group they were randomised to can compound the problem. This study asked whether those design choices measurably change the size of the reported benefit.

The authors carried out a retrospective analysis of 73 randomised controlled trials of progressive resistance strength training in older adults, all published before 2007. Each trial's reported effect on lower limb muscle strength was extracted, along with whether the outcome assessors were blinded to group allocation and whether the analysis followed intention-to-treat principles. Eighteen trials used blinded assessors and 55 did not. Meta-regression was then used to estimate the difference in effect size attributable to assessor blinding, both unadjusted and after accounting for the use of intention-to-treat analysis.

Trials using blinded assessors reported smaller strength effect sizes than trials using unblinded assessors, with a difference of -0.80 (95% confidence interval -1.35 to -0.25). The association persisted after adjusting for whether an intention-to-treat analysis had been used, with an adjusted difference of -0.65 (95% confidence interval -1.26 to -0.04), although that confidence interval sits close to zero. The direction was consistent: effects in unblinded trials were exaggerated relative to blinded ones. The analysis does not report which specific strength tests or training protocols were involved, and it does not identify a single true effect size for resistance training.

This supports a broader methodological literature showing that lack of blinding tends to inflate treatment effects, and it locates the problem specifically in exercise trials, where blinding participants is impossible but blinding assessors is not. It complicates the way headline effect sizes from older resistance training trials should be read, since most of those trials did not blind assessors. The analysis is retrospective and compares trials rather than randomly assigning blinding, so trials that bothered to blind may differ in other ways too. Whether the same gap persists in more recent trials, with newer reporting standards, is not addressed.

This is a reason to treat the eye-catching strength percentages from older training studies with a little scepticism, and to look for whether the people doing the measuring knew who had trained. It does not mean resistance training fails to build strength in older adults; it means the true effect is probably more modest than the least rigorous trials suggested. When comparing two programmes on the strength of published numbers, study design is part of the comparison. Reading a study's methods section is worth as much as reading its abstract.