Understanding Biostatistics and the STEP Trial

How are statisticians analyzing the data from the STEP trial?

AIDS vaccine candidates are tested in randomized, controlled, double-blind clinical trials to evaluate their safety and to determine whether or not a specific candidate induces immune responses against HIV (see VAX October-November 2007 Primer on Understanding Randomized, Controlled Clinical Trials). Late-stage clinical evaluation—including both Phase IIb test-of-concept and Phase III trials—look specifically at the efficacy of a vaccine candidate based on its ability to protect an individual against HIV infection or provide some degree of partial efficacy (see VAXMay 2007 Primer on Understanding Partially Effective AIDS Vaccines).

All of these trials are carefully planned by biostatisticians using mathematical formulas to determine key factors related to the design of the trial, such as the total number of volunteers that must be enrolled. Before a trial begins, biostatisticians also set an analysis plan detailing the types of statistical calculations that will be performed on the data. This is critical to the interpretation of the final results.

Statistical Significance

Once a trial is complete, researchers can compare the group of individuals who received the vaccine candidate to those who received an inactive placebo and see what effect, if any, the candidate had on either incidence of HIV infection or on certain markers of disease progression—such as the amount of virus in the blood, or viral load—in those individuals who were infected with HIV during the trial. If there is a difference between the two groups, statisticians can conduct a series of calculations to determine whether the difference was due to the vaccine candidate, or if it was merely the result of chance. This is referred to as determining the statistical significance of a result. A test of statistical significance provides a measure of credibility to the results. If the trial was designed and conducted properly, a statistically significant difference between the vaccine and placebo groups means the results were unlikely to have occurred by coincidence.

Trends

The STEP trial, which tested Merck’s AIDS vaccine candidate known as MRKAd5 in a Phase IIb test-of-concept trial involving 3,000 volunteers, is an example of a clinical trial in which further statistical analysis is required. In November 2007 researchers reported that this vaccine candidate offered no benefit. Data analysis indicated there was no statistically significant difference between the number of HIV infections or viral load levels in individuals in the vaccine and placebo groups. In addition, the data actually showed a trend toward more HIV infections occurring in individuals who received the vaccine candidate. This was an unexpected result. The initial statistical analysis plan for the trial was not designed to measure this effect and therefore statisticians could not rely on typical tests of statistical significance to determine if the vaccine enhanced the risk of HIV infection or if the difference occurred merely by chance. This makes interpretation of the observed trend very complicated.

Stratification

Volunteers in AIDS vaccine trials are usually randomly assigned to either the vaccine or placebo group (see VAXOctober-November 2007 Primer on Understanding Randomized, Controlled Clinical Trials). This reduces the chance that variables, such as age, ethnicity, gender, or other baseline characteristics of the volunteers will impact the final results of the trial. After a trial is complete, researchers can look at the background characteristics of the volunteers and determine how well the trial was actually randomized.

Statisticians can also design a trial by randomizing volunteers based on a specific variable that they think may confound the results. In this process, known as stratification, a pre-specified number of volunteers with a previously-identified characteristic are randomly placed into the vaccine and placebo groups. In the STEP trial, volunteers were stratified based on their level of pre-existing immunity to the naturally-circulating cold virus (adenovirus serotype 5, or Ad5), which was used in a disabled form as the vector in this vaccine candidate (see VAX September 2004 Primer onUnderstanding Viral Vectors). Initial analyses showed that the trend toward a higher number of HIV infections in vaccine recipients was apparent in the sub-groups of volunteers who had pre-existing Ad5 immunity.

Multivariate analysis

More complex analyses were then conducted to see how other factors, in addition to pre-existing Ad5 immunity, influenced the observed results. These so-called multivariate analyses allow statisticians to analyze several variables simultaneously. The most relevant risk factor identified so far for the STEP trial was male circumcision status. Volunteers who received the vaccine candidate were four times more likely than placebo recipients to become HIV infected if they were both uncircumcised and had some degree of pre-existing Ad5 immunity.

According to the investigators of the STEP trial, the trend toward an association between circumcision status and the risk of HIV infection seemed to be as strong, if not stronger, than the trend toward an association between HIV infection and pre-existing immunity to Ad5. However, these results must be interpreted with caution since the multivariate analyses were not part of the original statistical analysis plan for this trial, and were only performed because of the unexpected results. This is called a ‘post-hoc’ analysis, or one done after the fact. Post-hoc analyses provide much less reliable information.

Investigators are now in the process of analyzing the STEP data based on other variables as well. Information collected from these analyses may help researchers develop hypotheses that can then be investigated further.