Clinical Trial Designs for Target-Directed Therapies: Clinical Trial Designs for Target-Directed Therapies by Gary L Rosner, ScD

SLIDES & TRANSCRIPTS
Tuesday, March 6

Clinical Trial Designs for Target-Directed Therapies: Clinical Trial Designs for Target-Directed Therapies
Gary L. Rosner, ScD

Slide 1: Statistical Issues In Study Design

I would like to thank the organizers, as well, for inviting me. It has been a very good learning experience. I am a statistician and I am going to talk about some statistical issues that relate to designing studies where you have particular molecular targets that you have identified.
You think that your drug is going to affect these, and now you want to go and do a study, and these are some of the considerations that you need to address in your study design.

Slide 2: Molecular Markers/Targets

In particular, one needs to consider how you are going to measure the effect, obviously, and I will talk more about that.
As Rich very well delineated, there are different ways in which we might use the term surrogate, but ultimately the challenge is to show that, whatever effect you have on a particular target, correlates with an overall clinical benefit.

Slide 3: Molecular Targets

So, we have heard lots today about different possible targets that have been identified in terms of the initiation, in terms of angiogenesis, metastasis and some of the different receptors and targets that have been identified.

Slide 4: Heterogeneity

The first thing, though -- we have also heard a lot about heterogeneity, but a lot of different types of heterogeneity.
What do we mean by heterogeneity? One could say that these two slides, perhaps the overall fraction of the area that is labeled red, suppose that is about the same in the two. In this case, it is pretty well spread out across the whole slide. Here, it is quite focused to just one half. Is that an example of heterogeneity that we need to consider? Well, if we are doing simple biopsies, then obviously it matters, because on the one hand, having a transect this way will still give us a representative sample in the sense that the distribution of the marker of the target within that core is similar to the distribution in the whole slide. However, here is depends upon whether one goes across this way or goes vertically. So, that is an example of heterogeneity that we perhaps need to consider. That may argue for doing more than one biopsy.

Slide 5: Within-tumor

There is also heterogeneity within a tumor. Patterns change across tumors. So, a slide taken from this part of the tumor may display this sort of variation in how the targets are distributed, whereas a later section may have a different distribution.

Slide 6: Measuring Hypoxia

As an example, I am going to talk about some work that I did with some colleagues at the North Carolina State University College of Veterinary Medicine and also the University of North Carolina. I was at Duke at the time, so we had all triangle universities involved. Here, we are measuring hypoxia. One of the advantages of collaborating with people at NC State in the vet school is, we had access to pet dogs who came down with cancer. The dogs were brought in by the owners for treatment. We were able to excise tumors, but also do some clinical studies on these dogs.
In particular, here we were studying the distribution of hypoxia within canine sarcomas. Jim Raleigh, at the University of North Carolina, had developed several different immunohistochemical markers for hypoxia. Hypoxia is of interest because, as I think has been said already, hypoxia is associated with resistance. Hypoxic cells are resistant to radiation. The question is how common is hypoxia in spontaneously-occurring tumors and how well can we estimate the fraction of cells that are hypoxic within a tumor.

Slide 7: Biopsy Study

For this particular study, we had seven tumors. We had two pretreatment biopsies within each tumor. Each biopsy was then sectioned and we could evaluate the sections with respect to the immunohistochemical marker, dye, or staining, for hypoxia.
So, we could look at within-biopsy variation in percent of cells that are hypoxic. We could also look at between biopsy variation, and also we can look at differences between the different tumors, the different dogs.

Slide 8: Methods

As the method for this study, what we did is, we took all the available data for a given biopsy and assumed that that was, in fact, the true percent of cells that were hypoxic within that biopsy. As you see, it ranged from 11 to 27 slides per biopsy. The median was 19.

Slide 9: Tumor Sections

Then, we would select at random either one slide per biopsy, two slides per biopsy, or four, as separate simulation studies, estimate the fraction hypoxic from either the one, the two or the four slides, and compare it to what we were calling the true state of hypoxia for that particular biopsy.
The question is, how frequently were we within plus or minus .05 of the true fraction hypoxic. Could we do that well with just taking one slide per biopsy? Do we need to use two or do we need to use four slides per biopsy? So, we sectioned off the tumors into slides, and this is an example slide. Within each slide, we would place a grid in the microscope eyepiece. Then we could measure hypoxia within each of the little grids.

Slide 10: Within-tumor Variability

What we see here are, for three dogs, that across the slides, there was some variability in hypoxic fraction. The variation, in fact, is higher if the frequency or the percent of cells that are hypoxic is higher.
If a tumor or a biopsy seems to have relatively little hypoxia, there is not as much variation as when there is more hypoxia.

Slide 11: Variance Increases

This is another picture that shows that, where the variance increases with the mean labeled fraction within the biopsy.

Slide 12: Fraction Within 0.05 By Number of Biopsies

So, this is something we need to consider. How many slides we take per biopsy depends to some extent on what the frequency is of whatever we are trying to measure. The greater the percent, it is going to be more variable. So, to have as precise a measure we are going to need more slides.

Slide 13: Study Results

The bottom line from this study, though, is that if we took four slides per biopsy, we were generally, 90 percent of the time, within plus or minus five percent. If we only looked at one slide, we did okay, as I said before, when it was relatively low, say 10 percent hypoxic. For the group of tumors that had much higher levels of hypoxia, one or two slides per biopsy didn't do such a great job.

Slide 14: Conclusions

This repeats what I just said.

Slide 15: Designing the Study

When we are designing a study, we need to think, what is the objective. Is the objective estimation? Is the objective prediction? Do we want to predict, for some future patient, something about the marker, or are we just trying to estimate in a cross section of the population?

Slide 16: Sources of Variation

Well, there are different sources of variation, as I indicated before, or as Emily also indicated. We have the within tumor or lesion heterogeneity. We also have between tumors but within patients, if there are multiple metastatic lesions. We need to consider that source of variations. There is also between patient heterogeneity.
Then the measurement methodology itself may be another source of variation or imprecision.

Slide 17: Measure

How we measure or how we count, if we use an immunohistochemical marker, also matters. Again, going back to the sarcomas, the canine sarcomas, we had two choices. If we had a section that we prepared into four slides, then do we want to sum up the total number of cells that were labeled across all four, and then divide that by the total number of cells counted, or do we want to average the fractions for each slide, and call that the measure of hypoxic fraction in this example. Similarly, do we want to go through and correct? Look at the slides, figure out the slides, figure out which of the cells are obviously not tumor cells, which are the ones that are tumor cells, and only divide our labeled fraction by dividing by the number of tumor cells. We can see that it has an effect on variation.

Slide 18: Between-biopsy and Within-biopsy

These show the two or however many biopsies that were available per dog for all seven dogs. You can see there is quite a bit of heterogeneity going across the dogs. The separate bars correspond to a biopsy, and then these box plots show the heterogeneity within a particular biopsy. That is just using for a denominator all the cells that were in the slide.

Slide 19: Variation

Here is a close up that will show it a little bit better. If we don't correct -- that is, we include all cells for this particular tumor -- these two biopsies are that far apart. If we correct, however, notice that the medians are much closer to each other. So, this has an effect on variability. Similarly, over here, uncorrected and corrected, the medians are much closer to each other across the biopsies.

Slide 20: Design Considerations

When we want to design a study, do we want to estimate with some pre-specified precision, say, the fraction labeled or even an effect that is a change, because of the treatment? Especially after what Emily Bergsland just said, there is a cost involved. There is an obvious cost in terms of the preparation of the slides, in terms of the patient and so on. Another cost that we need to consider is whether or not patients will agree to participate in this study. If taking a pretreatment biopsy is not something that is done routinely and is not necessarily a part of the patient's therapy, then that may have a very high cost associated with it, and perhaps we need to incorporate these kinds of costs when we design our studies. If we are evaluating two competing designs, we should probably go with the one that has less cost for the same precision, or at least weighs cost against precision.

Slide 21: Design Decisions

So, our decisions when designing a study are to choose the number of patients, the number of samples per patient, and the timing of the samples.

Slide 22: Utility Function

A simple way of incorporating cost is to take the precision of whatever you are trying to estimate -- say a treatment effect -- this should be minus the cost. So, you have a cost per patient times the number of patients. Then, for each patient you have a certain number of slides or biopsies times the cost.
If you are incorporating pretreatment biopsies, that cost may be much higher, or would be much higher, than if you are just going to go in and take samples at the time that the patient goes in for resection.

Slide 23: One Biopsy Per Subject

A little exercise, another simulation exercise, to kind of show how variation or heterogeneity impacts on clinical trial design, is the following: We will assume a particular model. The particular model would say that, for each individual, there is a distribution of what percent of the tumors are going to be labeled. So, there is a population from which we pick tumors that have different percent label. Then, for an individual, there is a distribution within that tumor of what fraction is labeled. So, the biopsies are random samples from, say, within an individual, but the individual is a random sample from this larger population, two sources of heterogeneity. We want to compare a trial where we are looking at a before and after sample to measure treatment effect, to a trial where we just randomize patients, some get the treatment, some not, and just look at the difference between these two groups at time of surgery. So, there is no pretreatment biopsy.

Slide 24: Between (BAR) Within

Just showing this heterogeneity, the bar graph, the histogram, shows what the population distribution was. There is, on average, 10 percent fraction here.
The individual densities, the lines, show within individual variation for four individuals chosen at random from this population. You can see that, even though the population is about 10 percent, there is heterogeneity between patients. This patient has a higher percent exhibiting the marker than this one. The variation is also higher for this patient than for this one.

Slide 25: Pre-Treatment Biopsy

How does this affect the clinical trial? Well, the power for the two types of designs, or within-subject design versus a between-patient or between-group design, will get to be about the same if the within-tumor or within-biopsy or within-patient variation is the same or larger as the between variation in this particular setting. The reason is that, as the variation increases, the correlation within a tumor, the correlation of these samples, decreases. If the correlation gets very small, you don't derive much benefit from a within-individual test.

Slide 26: Example

So, again, in this example we have 10 percent of the cells assumed positive. I also incorporated a threshold which just said if, when I generated this number at random, if it was below .01, it is below our level where we can detect it, so we will call it a zero. At random, 10 percent of the cells or sections or whatever within these simulated biopsies, were going to be zero. So, if you are measuring something like hypoxia, maybe some of the sections could be near a blood vessel. So, they would exhibit no hypoxia.
We are comparing here 20 patients within subject, so before and after clinical trial, to a 40-patient between-patient study. Just for simplicity, I am going to take one biopsy at each time. So, for the randomized trial it is one biopsy, say, at surgery, after treatment. For the within patient, it is a biopsy before and a biopsy after, and you just look at the difference.

Slide 27: Results

I have taken two situations here corresponding to roughly similar between-patient variation and within-patient variation. In this case, the between-patient variation is much less than the within-patient variation. Here, the within is smaller than between.
You see that, in general, for these two situations, both studies have roughly the same power to detect the difference, and the difference that I did in the simulation was just an absolute number of adding five percent to the fraction exhibiting whatever it is that you want, whatever you are trying to effect.
It is only in this situation where the within variation is less than the between, where you see the pretreatment biopsy improving the power.

Slide 28: Results

If the population average increases to 50 percent, then there is going to be much more heterogeneity. We saw that before with the hypoxia.
The tumors that had relatively little hypoxia, there was very little variation within a biopsy. The tumors that had more, 30 to 40 percent hypoxic fraction, had more variability. If the population average is now 50 percent, regardless of whether the between or within variation, one is larger than the other, you have very little power for the same sample size. In fact, even going up to 200 patients you still don't have 50 percent power in this particular example.
So, we need to know, when designing a study, how heterogeneous this quantity is, as well as what the mean fraction is.

Slide 29: Relative Cost Factors

Similarly, I think we need to incorporate costs in some way.

Slide 30: Summary

In summary, taking, in certain situations, a randomized trial that is only looking at an after treatment biopsy but comparing two groups may be a viable and a reasonable way to design a study, if you want to evaluate whether or not a particular marker is affected by a certain agent, a new compound. Taking multiple measurements per subject may help. Knowing something about the distribution of these things within the population of tumors, between lesions within patients and between patients, I think, is critical before we can really figure out what is the best design. Thank you.