CANCELLED | Ulric B. and Evelyn L. Bray Social Sciences Seminar

Title: Inference under Covariate-Adaptive Randomization

Abstract: This  paper  studies  inference  for  the  average  treatment  effect  in  randomized  controlled  trials  with covariate-adaptive randomization.  Here, by covariate-adaptive randomization, we mean randomization schemes  that  first  stratify  according  to  baseline  covariates  and  then  assign  treatment  status  so  as  to achieve "balance" within each stratum.  Our main requirement is that the randomization scheme assigns treatment status within each stratum so that the fraction of units being assigned to treatment within each stratum has a well behaved distribution centered around a proportion π as the sample size tends to infinity. Such schemes include, for example, Efron's biased-coin design and stratified block randomization. When testing the null hypothesis that the average treatment effect equals a pre-specified value in such settings, we first show the usual two-sample t-test is conservative in the sense that it has limiting rejection probability under the null hypothesis no greater than and typically strictly less than the nominal level. We show, however, that a simple adjustment to the usual standard error of the two-sample t-test leads to a test that is exact in the sense that its limiting rejection probability under the null hypothesis equals the nominal level.  Next, we consider the usual t-test (on the coefficient on treatment assignment) in a linear regression of outcomes on treatment assignment and indicators for each of the strata.  We show that  this  test  is  exact  for  the  important  special  case  of  randomization  schemes  with π=½,  but is otherwise conservative.  We again provide a simple adjustment to the standard error that yields an exact test more generally.  Finally, we study the behavior of a modified version of a permutation test, which we refer to as the covariate-adaptive permutation test, that only permutes treatment status for units within the same stratum.  When applied to the usual two-sample t-statistic, we show that this test is exact for randomization schemes with π=½ and that additionally achieve what we refer to as "strong balance."  For randomization schemes with π≠½, this test may have limiting rejection probability under the null hypothesis strictly greater than the nominal level.  When applied to a suitably adjusted version of  the  two-sample t-statistic,  however,  we  show  that  this  test  is  exact  for  all  randomization  schemes that achieve "strong balance," including those with π≠½.  A simulation study confirms the practical relevance of our theoretical results.  We conclude with recommendations for empirical practice and an empirical illustration.