How TRACE works

Purpose

Therapy persistence is the time from starting a treatment to stopping it¹. Patient-level data show it directly, but they are often not available in time. Brand teams usually have monthly new patients (NBRx) and total prescriptions (TRx) from launch, and TRACE estimates a persistence curve from those two series alone.

The idea is that total prescriptions in a month come from every cohort of patients who started earlier and are still on therapy. If each cohort loses patients along a retention curve S(t), the curve that best reproduces the TRx series is an estimate of persistence. TRACE uses the Weibull curve², which has two parameters and a shape flexible enough to describe early, steady or late dropout.

Notation

Equations

Each equation is followed by what it means.

S(t) is the share of a cohort of new patients still on therapy at age t months, so S(0) = 1 and it only falls. α is the scale: at t = α the share is e⁻¹, about 37%, whatever β is. Larger α means patients stay longer. β is the shape and is explained under equation 2.

The hazard h(t) is the rate at which patients still on therapy at age t stop. With β below 1 it falls with age, so most dropout comes early. With β equal to 1 it is constant, and the curve is the exponential. With β above 1 it rises, so dropout speeds up. One family can therefore describe early-heavy, steady or late dropout.

A_t is the number of patients still on therapy in month t. Each earlier month's new patients NBRx_i is multiplied by the share of that cohort still on therapy, S(t − i), and the results are added. Cohorts older than the max lag L are treated as gone. Patients starting in month t count in full, because S(0) = 1.

Total prescriptions in a month are modelled as proportional to the patients on therapy. The multiplier k turns patients into prescriptions, so it absorbs fill length, refill gaps and adherence. It is not refills per patient.

The three parameters are the values that make the fitted line closest to the observed TRx, measured by the sum of squared differences (least squares). The search is limited to α from 0.1 to 240 and β from 0.1 to 10.

For any fixed α and β, the squared error is a simple quadratic in k. Setting its slope to zero gives this closed form. So only α and β need to be searched, not all three.

Median persistence is the age at which S(t) = 0.5. RMSE is the typical size of the miss in TRx units. R² is the share of the variation in TRx that the fit explains, and MAPE is the average percentage miss (months with zero TRx are left out of it).

Worked example

Take the example that loads in TRACE, which was made up from α = 11, β = 0.85 and k = 1.8. The retention curve gives S(1) = 0.878, S(2) = 0.791, S(6) = 0.550 and S(12) = 0.341, and a median persistence of 7.1 months. Suppose a launch had new patients of 120, 135 and 150 in its first three months. In month 3:

The 120 patients who started two months earlier are down to 95, those who started a month earlier to 119, and all 150 new patients count. They add up to 363 on therapy, and each fills about 1.8 prescriptions in a month on average.

Projection

To project TRx beyond the data, TRACE extends the new-patient series with an assumption and applies equations 3 and 4 to the extended series, with the fitted α, β and k. The default assumption is the average of the last three months of NBRx. Holding the last month, or a number you enter for each month, are the alternatives.

A projection has two sources of error. One is the fit itself. The other is the new-patient assumption, which TRACE does not forecast. The second is often the larger.

Finding the fit

• Objective. Least squares (equation 5). For each pair of α and β, k is set by equation 6.
• Search. The Nelder–Mead simplex method³, run on the logarithms of α and β so both stay positive. It starts from six different points and keeps the best result, then refines it twice. Several starts guard against settling on a poor local minimum.
• Limits. α is held between 0.1 and 240 and β between 0.1 and 10. A fit that ends at one of these limits is flagged, because the data did not pin the value down.
• Minimum data. At least 6 months, because three parameters are fitted. Twelve or more is recommended.

Reading the fit

• A high R² does not prove the curve is right. With a short series, many combinations of α, β and k fit almost equally well. A short, steady series can be matched by a fast-dropout curve with a large k or a slow-dropout curve with a small one. α and β then trade off against each other.
• The long tail is extrapolation. Retention at ages beyond the length of your data comes from the shape of the Weibull, not from observation.
• Compare with what you know. If you have a published or claims-based persistence figure for the product or its class, check the median and the 6- and 12-month retention against it.

Verification

Verification checks that the tool does the arithmetic it claims to. Validation checks that the model is a fair picture of the real world. The two are reported separately.

Done

• Independent implementation. TRACE was compared with an independent implementation of the same model on seven test datasets, covering front-loaded, accelerating and steady dropout, short and long series, and noisy data. The parameters agree to at least four decimal places and the projections to at least five significant figures.
• Recovery of known parameters. Where data were generated from known α, β and k, TRACE returns values close to them.
• Input handling. Tests cover pasted text, CSV and Excel input, headers in any case, different separators, negative or non-numeric values, and too few months.
• The tests run again on every change to the code.

Not done

• Comparison with patient-level persistence. The curve TRACE estimates from aggregate data has not been compared with Kaplan–Meier curves⁴ from patient-level claims for the same products.
• Forecast accuracy. The accuracy of projections against later actual TRx has not been measured.

Until that comparison has been done, treat TRACE as a way to see what persistence your aggregate numbers imply, and to project from it, and not as a replacement for patient-level analysis where you have it.

Limitations

• It is an aggregate estimate. It does not follow individual patients.
• Fill behaviour is folded into the curve. Refill gaps, late fills and compliance end up in S(t) and k, so the curve describes prescription-weighted persistence, not strictly time on therapy.
• NBRx must really be new patients. Counts that include switches, restarts or all new prescriptions mix groups that persist differently.
• All cohorts share one curve. Early adopters often persist differently from later patients.
• The series must start at launch. Each month's TRx depends on every earlier cohort.
• Data quality matters. Samples, free goods, copay-card or cash channels and stocking can distort TRx, especially soon after launch.

More practical guidance is on the Help page.

References

1. [1] Cramer JA, Roy A, Burrell A, et al. Medication compliance and persistence: terminology and definitions. Value in Health 2008;11(1):44–47. ISPOR
2. [2] Weibull W. A statistical distribution function of wide applicability. Journal of Applied Mechanics 1951;18(3):293–297.
3. [3] Nelder JA, Mead R. A simplex method for function minimization. The Computer Journal 1965;7(4):308–313.
4. [4] Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 1958;53(282):457–481.