Multi-state models are a popular method of describing medical processes that can be represented as discrete states or stages. They have particular use when the data are panel-observed, meaning they consist of discrete snapshots of disease status at irregular time points which may be unique to each patient. However, due to the difficulty of inference in more complicated cases, strong assumptions such as the Markov property, patient homogeneity and time homogeneity are applied. It is important that the validity of these assumptions is tested. A review of methods for diagnosing model fit for panel-observed continuous-time Markov and misclassification-type hidden Markov models is given, with illustrative application to a dataset on cardiac allograft vasculopathy progression in post-heart transplant patients.

For a continuous-scale diagnostic test, it is often of interest to find the range of the sensitivity of the test at the cut-off that yields a desired specificity. In this article, we first define a profile empirical likelihood ratio for the sensitivity of a continuous-scale diagnostic test and show that its limiting distribution is a scaled chi-square distribution. We then propose two new empirical likelihood-based confidence intervals for the sensitivity of the test at a fixed level of specificity by using the scaled chi-square distribution. Simulation studies are conducted to compare the finite sample performance of the newly proposed intervals with the existing intervals for the sensitivity in terms of coverage probability. A real example is used to illustrate the application of the recommended methods.

Interval-censored failure time data occur in many medical investigations as well as other studies such as demographical and sociological studies. They include the usual right-censored failure time data as a special case but provide much more complex structure and less relevant information than the right-censored data. This article reviews some basic concepts, issues and the corresponding statistical approaches related to the analysis of interval-censored data as well as recent advances. In particular, we discuss estimation of a survival function, comparison of several treatments and regression analysis as well as competing risks analysis and truncation in the presence of interval censoring. A well-known example of interval-censored data is described and analysed to illustrate some of the statistical procedures discussed.

We review recent work on the application of pseudo-observations in survival and event history analysis. This includes regression models for parameters like the survival function in a single point, the restricted mean survival time and transition or state occupation probabilities in multi-state models, e.g. the competing risks cumulative incidence function. Graphical and numerical methods for assessing goodness-of-fit for hazard regression models and for the Fine–Gray model in competing risks studies based on pseudo-observations are also reviewed. Sensitivity to covariate-dependent censoring is studied. The methods are illustrated using a data set from bone marrow transplantation.

In recent years, breakthroughs in biomedical technology have led to a wealth of data in which the number of features (for instance, genes on which expression measurements are available) exceeds the number of observations (e.g. patients). Sometimes survival outcomes are also available for those same observations. In this case, one might be interested in (a) identifying features that are associated with survival (in a univariate sense), and (b) developing a multivariate model for the relationship between the features and survival that can be used to predict survival in a new observation. Due to the high dimensionality of this data, most classical statistical methods for survival analysis cannot be applied directly. Here, we review a number of methods from the literature that address these two problems.

The Cancer Care Outcomes Research and Surveillance (CanCORS) Consortium is a multisite, multimode, multiwave study of the quality and patterns of care delivered to population-based cohorts of newly diagnosed patients with lung and colorectal cancer. As is typical in observational studies, missing data are a serious concern for CanCORS, following complicated patterns that impose severe challenges to the consortium investigators. Despite the popularity of multiple imputation of missing data, its acceptance and application still lag in large-scale studies with complicated data sets such as CanCORS. We use sequential regression multiple imputation, implemented in public-available software, to deal with non-response in the CanCORS surveys and construct a centralised completed database that can be easily used by investigators from multiple sites. Our work illustrates the feasibility of multiple imputation in a large-scale multiobjective survey, showing its capacity to handle complex missing data. We present the implementation process in detail as an example for practitioners and discuss some of the challenging issues which need further research.

Repeated measures and multivariate outcomes are an increasingly common feature of trials. Their joint analysis by means of random effects and latent variable models is appealing but patterns of heterogeneity in outcome profile may not conform to standard multivariate normal assumptions. In addition, there is much interest in both allowing for and identifying sub-groups of patients who vary in treatment responsiveness. We review methods based on discrete random effects distributions and mixture models for application in this field.

Complex intervention trials should be able to answer both pragmatic and explanatory questions in order to test the theories motivating the intervention and help understand the underlying nature of the clinical problem being tested. Key to this is the estimation of direct effects of treatment and indirect effects acting through intermediate variables which are measured post-randomisation. Using psychological treatment trials as an example of complex interventions, we review statistical methods which crucially evaluate both direct and indirect effects in the presence of hidden confounding between mediator and outcome. We review the historical literature on mediation and moderation of treatment effects. We introduce two methods from within the existing causal inference literature, principal stratification and structural mean models, and demonstrate how these can be applied in a mediation context before discussing approaches and assumptions necessary for attaining identifiability of key parameters of the basic causal model. Assuming that there is modification by baseline covariates of the effect of treatment (i.e. randomisation) on the mediator (i.e. covariate by treatment interactions), but no direct effect on the outcome of these treatment by covariate interactions leads to the use of instrumental variable methods. We describe how moderation can occur through post-randomisation variables, and extend the principal stratification approach to multiple group methods with explanatory models nested within the principal strata. We illustrate the new methodology with motivating examples of randomised trials from the mental health literature.

For a number of reasons, surrogate endpoints are considered instead of the so-called true endpoint in clinical studies, especially when such endpoints can be measured earlier, and/or with less burden for patient and experimenter. Surrogate endpoints may occur more frequently than their standard counterparts. For these reasons, it is not surprising that the use of surrogate endpoints in clinical practice is increasing.

Building on the seminal work of Prentice¹ and Freedman et al.,² Buyse et al.³ framed the evaluation exercise within a meta-analytic setting, in an effort to overcome difficulties that necessarily surround evaluation efforts based on a single trial. In this article, we review the meta-analytic approach for continuous outcomes, discuss extensions to non-normal and longitudinal settings, as well as proposals to unify the somewhat disparate collection of validation measures currently on the market. Implications for design and for predicting the effect of treatment in a new trial, based on the surrogate, are discussed. A case study in schizophrenia is analysed.

Nesting of patients within therapists in psychotherapy trials creates an additional level within the design. The multilevel nature of this design has implications for the precision, internal and external validity of estimates of the treatment effect. Prior to or during a trial, psychotherapies are allocated to therapists and therapists are assigned to patients such that the therapist becomes part of the causal pathway from the intervention to the patient. It is therefore important to consider not only the relationship between interventions and patients but also relationships between interventions and therapists and between therapists and patients. Research designs comparing the effects of therapeutic approaches, therapist characteristics and packages of the two can be unified by viewing therapists as an important source of variability within psychotherapy outcome studies. Methodological considerations arising from therapist variation will be discussed, drawing together and building upon the associated psychotherapy and statistical literatures. Parallels will also be made with related designs and methods of analysis.

A dynamic treatment regime is a set of decision rules, one per stage, each taking a patient's treatment and covariate history as input, and outputting a recommended treatment. In the estimation of the optimal dynamic treatment regime from longitudinal data, the treatment effect parameters at any stage prior to the last can be non-regular under certain distributions of the data. This results in biased estimates and invalid confidence intervals for the treatment effect parameters. In this article, we discuss both the problem of non-regularity, and available estimation methods. We provide an extensive simulation study to compare the estimators in terms of their ability to lead to valid confidence intervals under a variety of non-regular scenarios. Analysis of a data set from a smoking cessation trial is provided as an illustration.

Regression analysis of survival data, and more generally event history data, is typically based on Cox's regression model. We here review some recent methodology, focusing on the limitations of Cox's regression model. The key limitation is that the model is not well suited to represent time-varying effects. We start by considering classical and also more recent goodness-of-fit procedures for the Cox model that will reveal when the Cox model does not capture important aspects of the data, such as time-varying effects. We present recent regression models that are able to deal with and describe such time-varying effects. The introduced models are all applied to data on breast cancer from the Norwegian cancer registry, and these analyses clearly reveal the shortcomings of Cox's regression model and the need for other supplementary analyses with models such as those we present here.

Sample size determination is an essential component in public health survey designs on sensitive topics (e.g. drug abuse, homosexuality, induced abortions and pre or extramarital sex). Recently, non-randomised models have been shown to be an efficient and cost effective design when comparing with randomized response models. However, sample size formulae for such non-randomised designs are not yet available. In this article, we derive sample size formulae for the non-randomised triangular design based on the power analysis approach. We first consider the one-sample problem. Power functions and their corresponding sample size formulae for the one- and two-sided tests based on the large-sample normal approximation are derived. The performance of the sample size formulae is evaluated in terms of (i) the accuracy of the power values based on the estimated sample sizes and (ii) the sample size ratio of the non-randomised triangular design and the design of direct questioning (DDQ). We also numerically compare the sample sizes required for the randomised Warner design with those required for the DDQ and the non-randomised triangular design. Theoretical justification is provided. Furthermore, we extend the one-sample problem to the two-sample problem. An example based on an induced abortion study in Taiwan is presented to illustrate the proposed methods.

This paper aims to develop a probability-based model involving the use of direct likelihood formulation and generalised linear modelling in order to estimate important disease parameters from real data. The force of infection and the recovery rate or per capita loss of infection are the parameters of interest. The problem of dealing with time-varying disease parameters is also addressed in the paper by fitting piecewise constant parameters over time. The findings of the current paper are comparable and similar to estimates from an independent approach suggested by White et al.₂₁ that employed Bayesian MCMC modelling via WinBUGS.

The use of patterned covariance structures in the parametric analysis of longitudinal data is both elegant and efficient. However, this strategy has not been well studied for semi-parametric models for analysing such data. We propose to estimate the covariance matrix in the semi-parametric model by rearranging the non-parametric component as a profiled linear function of the data and using a local smoothing technique. This results in a parametric regression formulation that enables us to construct likelihood functions and use various information criteria to select the best fitting covariance matrix. We apply our method to reanalyse data from a two-armed clinical trial for Scleroderma patients and show our method is more efficient for estimating the parametric components in the semi-parametric model.

We present a relatively non-technical and practically orientated review of statistical methods that can be used to estimate dose–response relationships in randomised controlled psychotherapy trials in which participants fail to attend all of the planned sessions of therapy. Here we are investigating the effects on treatment outcome of the number of sessions attended when the latter is possibly subject to hidden selection effects (hidden confounding). The aim is to estimate the parameters of a structural mean model (SMM) using randomisation, and possibly randomisation by covariate interactions, as instrumental variables. We describe, compare and illustrate the equivalence of the use of a simple G-estimation algorithm and two two-stage least squares procedures that are traditionally used in economics.

Although considerable attention has been given to zero-inflated count data, research on zero-inflated log-normal data is limited. In this article, we consider a study to examine human sperm cell DNA damage obtained from single-cell electrophoresis (COMET assay) experiment in which the outcome measures present a typical example of log-normal data with excess zeros. The problem is further complicated by the fact that each study subject has multiple outcomes at each of up to three visits separated by six-week intervals. Previous methods for zero-inflated log-normal data are based on either simple experimental designs, where comparison of means of zero-inflated log-normal data across different experiment groups is of primary interest, or longitudinal measurements, where only one observation is available for each subject at each visit. Their methods cannot be applied when multiple observations per visit are possible and both inter- and intra-subject variations are present. Our zero-inflated model extends the previous methods by incorporating a hierarchical structure using latent random variables to take into account both inter- and intra-subject variations in zero-inflated log-normal data. An EM algorithm has been developed to obtain the Maximum likelihood estimates of the parameters and their standard errors can be estimated by parametric bootstrap. The model is illustrated using the COMET assay data.

Generalised additive models (GAMs) allow for flexible functional dependence of a response variable on covariates. The aim of this article is to provide an accessible overview of GAMs based on the penalised likelihood approach with regression splines. In contrast to the classical backfitting, the penalised likelihood framework taken here provides researchers with an efficient computational method for automatic multiple smoothing parameter selection, which can determine the functional form of any relationship from the data. We illustrate through an example how the use of this methodology can help to gain insights into medical research.

As a type of multivariate survival data, multistate models have a wide range of applications, notably in cancer and infectious disease progression studies. In this article, we revisit the problem of estimation of state occupation, entry and exit times in a multistate model where various estimators have been proposed in the past under a variety of parametric and non-parametric assumptions. We focus on two non-parametric approaches, one using a product limit formula as recently proposed in Datta and Sundaram¹ and a novel approach using a fractional risk set calculation followed by a subtraction formula to calculate the state occupation probability of a transient state. A numerical comparison between the two methods is presented using detailed simulation studies. We show that the new estimators have lower statistical errors of estimation of state occupation probabilities for the distant states. We illustrate the two methods using a pubertal development data set obtained from the NHANES III.²

Multi-state approaches are becoming increasingly popular to analyse the complex evolution of patients with chronic diseases. For example, the evolution of kidney transplant recipients can be broken down into several clinical states. With this application in mind, we present a flexible semi-Markov model. The distribution functions are fitted to the durations in states and the relevance of the generalised Weibull distribution is shown. The corresponding likelihood function allows for interval censoring, i.e. the times of transitions and the sequences of states are not available during the elapsed times between two visits. The explanatory variables are introduced through the Markov chain and through the probability density functions of durations. A goodness-of-fit test is also defined to examine the stationarity of the semi-Markov model.

This article reviews various recently suggested approaches to estimate the random effects distribution in a linear mixed model, i.e. (1) the smoothing by roughening approach of Shenand Louis,¹ (2) the semi-non-parametric approach of Zhang and Davidian,² (3) the heterogeneity model of Verbeke and Lesaffre³ and (4) a flexible approach of Ghidey et al.⁴ These four approaches are compared via an extensive simulation study. We conclude that for the considered cases, the approach of Ghidey et al.⁴ often shows to have the smallest integrated mean squared error for estimating the random effects distribution. An analysis of a longitudinal dental data set illustrates the performance of the methods in a practical example.

Several different rate functions of the recurrent event process have been proposed for analysing recurrent event data when the observation of a study subject can be terminated by a failure event, such as death. When the terminal event is correlated with the underlying recurrent event process, these rate functions have different interpretations; however, recognition of the differences has been lacking theoretically and practically. In this article, we study the relationship between these rate functions and demonstrate that models based on an inappropriate rate function may lead to misleading scientific conclusions in various scenarios. An analysis of data from an AIDS clinical trial is presented to emphasise the importance of cautious model selection.