Cluster-randomized rehabilitation trials: count the groups as well as the people

A trial may include hundreds of people but only a small number of independently randomized clinics. Before trusting its confidence interval, ask what was assigned, what was measured and how the analysis connected those levels.

Your learning task is to explain why the participant count alone cannot establish precision, calculate a simple design effect, and identify what still requires a proper trial analysis. This lesson extends appraising an individually randomized rehabilitation trial; its focus is the extra dependence created when groups receive the allocation.

Start with the allocation map

Original fictional case: Twenty Canadian outpatient rehabilitation clinics enter a study. Ten are randomly assigned a new team training package and ten continue their usual training. Each clinic contributes thirty adults. The outcome is an invented activity score measured at twelve weeks. Higher scores represent better reported activity.

The training changes clinic routines, so assigning different packages to people treated by the same team could lead to spill-over. Group allocation is a plausible design choice, although it does not eliminate every source of contamination. The dated CONSORT cluster extension describes this rationale. Campbell and colleagues, 2012.

The study has three different counts:

People share staff, referral practices and local conditions. Their outcomes may therefore contain overlapping information. Counting every score as an independent randomized unit would miss that structure.

It is not automatically wrong to analyse individual scores. A suitable multilevel model or generalized estimating equation can retain those scores while accounting for clustering. Another approach analyses clinic summaries. The problem is ignoring dependence, rather than merely having different allocation and measurement units. Kahan and colleagues, 2016.

Give the intracluster correlation a specific meaning

For this teaching calculation, assume a simple continuous-outcome model with independent clinics, equal clinic sizes and a shared within-clinic correlation. Let the intracluster correlation coefficient, or ICC, be 0.04.

In a random-intercept model within a trial arm, the ICC is the between-clinic variance divided by the total variance. Equivalently, it describes the correlation of two different participants' outcomes within the same clinic under that model. It is not the proportion of people who improved, and it does not mean every pair of participants has identical outcomes. Its value belongs to the particular outcome, population and model; 0.04 is an assumed teaching input here.

For illustration, between-clinic variance of 4 score-points squared and within-clinic variance of 96 score-points squared give:

ICC = 4 ÷ (4 + 96) = 0.04.

The small decimal is easy to dismiss. Its consequence depends on how many observations share the clinic.

Work through the design effect

Under the equal-size, common-correlation approximation, the design effect is:

DE = 1 + (m − 1) × ICC, where m is the number of participants per clinic.

For thirty participants per clinic:

DE = 1 + 29 × 0.04 = 2.16.

An approximate effective participant count is 600 ÷ 2.16 = 277.8. This is an information comparison with an otherwise comparable independent-observation analysis. It does not mean 322 people should be deleted, that only 278 received treatment, or that the trial has 278 independently randomized units. It still has twenty randomized clinics. Cochrane describes effective sample size and standard-error inflation as approximate adjustments, with qualifications when cluster sizes differ. Chapter 23, sections 23.1.4–23.1.5.

Now suppose an analysis that ignored clinics reported a four-point group difference with a standard error of 1.5 points. Its normal-approximation interval would be:

4 ± 1.96 × 1.5 = 1.06 to 6.94 points.

To illustrate only the effect of dependence, multiply that standard error by the square root of 2.16:

1.5 × √2.16 = 2.205 points.

Keeping the same point estimate and normal multiplier gives −0.32 to 8.32 points. The apparent precision changes substantially.

This second interval is a teaching approximation, not a completed valid reanalysis of the twenty-clinic trial. A suitable analysis may change the point estimate, adjust for baseline variables and use different small-sample inference. Neither interval tells us whether four points would matter to participants; no importance threshold was supplied.

More people and more clinics are different design changes

Keep the fictional ICC fixed at 0.04 and compare two ways to increase the total from 600 to 1,200 people:

Under these assumptions, doubling people within the existing clinics increases the effective count from 277.8 to 357.1. Doubling the number of comparable clinics instead doubles it to 555.6.

This comparison isolates information under the simplified model. It does not establish the cheaper design or guarantee power: recruiting clinics may be expensive, their populations may differ, and the assumed ICC may be uncertain. For unequal clinic sizes, inserting a convenient average into the simple formula can misrepresent precision.

There is also a boundary worth recognizing. With one clinic in each arm, a difference between programmes is inseparable from the difference between those two clinics in the usual parallel design. Adding hundreds of people to each does not create replication of the clinic-level assignment. Campbell and colleagues, 2012.

Read past the words “adjusted for clustering”

A model name is useful information, but not a complete explanation of uncertainty. With limited clusters, some standard implementations can produce overly confident inference unless appropriate corrections are used. Kahan and colleagues demonstrated this in specified simulation scenarios and a trial reanalysis; their findings do not establish one universally safe cluster count or one correction suitable for every design. Kahan and colleagues, 2016.

For the fictional twenty-clinic report, locate the actual method, small-sample procedure, cluster sizes, estimated ICC and counts contributing to each result. If the method is unclear, record the uncertainty rather than assuming either that it was adequate or that it was absent.

Also distinguish the target average. Giving a small clinic and a large clinic equal weight describes a different average from giving each participant equal weight when clinic sizes and effects differ. Ask which question the reported weighting answers. Do not improvise new weights merely to obtain a narrower interval.

Trace recruitment after allocation

Imagine that clinics learned their assignments before identifying eligible people. Staff excited about the new package then preferentially invited people expected to attend consistently. Randomizing clinics would not, by itself, prevent that later selection from changing who entered each arm. Recruitment timing is a specific concern in cluster trials. Cochrane Handbook, section 23.1.2.

For this case, check whether eligibility lists existed before allocation, whether recruiters knew assignments and whether selection rules were applied consistently. Differences in participant characteristics are clues to investigate, not proof of deliberate manipulation.

Follow attrition at both levels. Losing thirty participants across several clinics differs from losing one entire clinic and its thirty participants. The same missing-person count can leave different information about the randomized comparison. A design-effect calculation repairs neither selective recruitment nor unexplained missing outcomes.

Transfer exercise: compare two proposals

Original fictional exercise: Both proposals recruit 480 people with equal-sized clinics and an assumed ICC of 0.05. Proposal A uses 24 clinics with 20 people each. Proposal B uses 12 clinics with 40 people each. All other features are held comparable for this calculation.

Which has the larger approximate effective count, and what does that establish?

For A, DE = 1 + 19 × 0.05 = 1.95, giving 480 ÷ 1.95 = 246.2. For B, DE = 1 + 39 × 0.05 = 2.95, giving 480 ÷ 2.95 = 162.7. A has more information under the stated approximation and more randomized clinics. Equal participant totals do not imply equal precision.

This does not prove that A meets a target power, avoids recruitment bias or has an appropriate analysis. Those require additional design and methods information.

For your next journal discussion, write three lines: the number of randomized clusters, how the analysis handles dependence, and the most consequential unresolved design question. That note makes the uncertainty visible without dismissing a useful study simply because its design is more complex.

Sources and further reading

• Cochrane Handbook, Chapter 23 — Cluster-specific appraisal and approximate adjustments; relevant full-text sections read.
• Campbell et al., 2012 CONSORT cluster extension — Dated methodological guidance; substantive indexed opening/Scope excerpt accessed, direct full text unavailable in this check.
• Kahan et al., 2016 — Small-sample inference and its limits; primary simulation and reanalysis study.