Analysis: Why Klempner Study is Not Useful to Rule Out Benefits of Long-Term Treatment

 

I. The clinical trial by Klempner et al. cannot be used to assess the effectiveness of re-treatment of Lyme disease in patients with continued symptoms following standard treatment

 

A. The trial was poorly designed

 

1.   The sample sizes were too small to detect clinically meaningful treatment effects (the study was underpowered).

a.   Determined by examining the ability of Klempner’s study to detect levels of improvement that were found to be of value in studies of other chronic illnesses.

2.   The (Klempner) presumed treatment effect for sample size calculation was so large for one of the two primary outcomes (mental score) that Lyme patients would have been forced to perform better than the general U.S. population at follow-up.

a.       E.g.− The trial was terminated early because it was unlikely that the presumed treatment effects would be statistically significant.

1)   Since people were not going to achieve Klempner’s unreasonable requirement for improvement (see #2), then this poorly designed trial should have been terminated.

 

3.      Lack of statistical significance cannot be used to infer ineffectiveness in a poorly designed study. This is especially true here since Klempner’s confidence intervals for the treatment effects contain clinically meaningful values.

a.   Because meaningful values are in the confidence intervals, you can’t say treatment is ineffective.

 

      B. The trial data were poorly analyzed (summarized rather than analyzed)

 

1.   The statistical method used was less likely to obtain a statistically significant result than recommended methods for analyzing clinical trials (e.g. omitted ½ the data, categorized the outcomes leading to loss of information).

      a.  30 day, 90 day measures of outcome were not included in the analysis

      b.   If a longitudinal analysis of uncategorized data was done (all data included), it would have increased the ability to detect a significant treatment effect.           

 

2.      Klempner’s treatment effects cannot be interpreted for clinical meaning. Patients with Lyme have symptoms that wax and wane. For example, patients classified as “improved” could have unchanged health status, patients classified as “unchanged” could have improved.

                   a.   They categorized people rather than examining individual scores− it is unknown how treatment affected the mean outcomes in the placebo vs. antibiotic groups.

 

3.      The findings are likely biased (i.e. wrong) because the analysis didn’t adjust for baseline, and baseline scores differed by arm, and there was an inadequate presentation and analysis of participants lost to follow-up. Klempner does not say how many were lost to follow-up, and they were simply placed in a “worsened” category.