• 
• 
' 
Marginally Restricted 
D-optimal Designs 
by 
R. Dennis Cook and L.A. Thibodeau 
Technical Report No. 331 
Department of Applied Statistics 
University of Minnesota 
Saint Paul, Minnesota 55108 
November 1, 1978 
This work was supported in part by Grant 
#l-R01-GM25587-0l from the National Insti-
tute of General Medical Science. 
' 
Abstract 
In experimental design it often happens that some of the relevant 
carriers cannot be specified by the experimenter. We consider the 
problem of obtaining approximate D-optimal designs when the design 
space is a product space and the carriers associated with one margin 
are not subject to control. An equivalence theorem for D-optimal 
designs is presented. The essential ingredients of iterative schemes 
for generating designs are discussed. 
:--, 
' 
'• 
1. Introduction 
In classical optimal design for regression models it is usually assumed 
that all relevant carriers (independent variables) can be controlled 
completely throughout the design space, (see, for example, Fedorov, 1972). 
However, in many areas of application it is common to find that some of 
the known carriers are not subject to control. This often happens when 
the experiment consists of applying levels of a "treatment" to experimental 
units which differ on known relevant quantitative variables. In this case 
the values of the carriers associated with the experimental units are 
restricted by the availability of the units. 
Harville (1974, 1975) discusses the problem of obtaining nearly optimal 
allocation of experimental units for analysis of covariance models. He 
presents algorithms which result in nearly D-optimal exact designs for 
inferences about the treatment effects in additive covariance models and 
discusses extensions to nonadditive models. 
Here,we consider the problem of obtaining D-optimal designs for 
regression models when the values of some of the carriers are restricted 
and not subject to control by design. We first present the general 
formulation and some relevant background information. 
Let f'(x) • (£1 , ••• ,f) denote a vector of p linearly independent 
- p 
continuous functions on some compact space X. An experiment consists of 
selecting an x in x and observing a random variable y(x) with 
regression function E(ylx) a!'! and constant variance a2 • We assume 
that the fi are known while the parameter vector ! is unknown. If ; 
is a probability measure on X then ~ defines an experimental design. 
1 
2 
Exact designs concentrate mass ;(x1) at points xi, i~l,2, ••• ,r, 
subject to the restriction that N((x1) • n1 is integral for all i. An 
exact design specifies that the experimenter is to take N uncorrelated 
observations, n1 at xi. The resulting covariance matrix of the least 
squares estimate of ! is of the form 
where the information matrix, ~(t), is 
M(f;) • f .f .f' df; 
X 
Approximate designs are not constrained by the requirement that ni be 
integral for all i. Here we consider only approximate designs. 
The choice of a design is often based on the minimization of some 
functional of the information matrix, M(;). Perhaps the two most commonly 
employed functionals are 
and 
(i) -1 determinant M (;) • 
(11) max d(x;E:) 
X€; 
where -1 d(x;;) ::r !:...' (x)M (~)f(x). Designs minimizing these functionals are 
called D and G-optimal designs, respectively. The following result due 
to Kiefer and Wolfowitz (1960) established the equivalence of D and 
G-optimal approximate designs and provided a way of verifying whether a 
given design is D-optimal: 
.. 
I 
-~, 
\ 
"' 
t" 
3 
Theorem 1: (Equivalence Theorem). The following conditions are equivalent. 
(i) IM-1 <tn> I • min IM-1(t)I 
E; 
(ii) max d(x;(D) a min max d(x;t) 
X ; X 
(iii) max d(x;~D) mp. 
X 
The set of all designs satisfying these conditions is convex and the 
corresponding information matrices are identical. 
In the next section we provide analogous equivalences for situations 
in which x is a vector, x • (x1 ,x2), and the values of ;_ to be 
included in the experiment are~ at the experimenter1s control. 
4 
2. Marginally Restricted D-Optimal Designs 
Let x m (x1,x2) and let ~(x) ·m ;(x1 ,x2) denote an arbitrary design on 
X m x1 x x2• We consider only designs for which IM(t)I; O. Let ;i, 
iml,2, denote the marginal design 
~i(xi): ..( d~(xl,x2) 
j 
, i ; j m 1, 2 • 
Since the values of xl to be included in the experiment are not subject 
* to control, we assume that they specify a ma_rginal design ; 1 , say, which places 
* mass at points of a finite collection s1 • 
* 
* Following Fedorov (1972), we refer to s1 
* as the spectrum of the design ~1. All permissible designs must have ; 1 = ; 1 and 
we assume that there is at least one permissible design ~ such that IM(;)I; o. 
Let C • {~1~1 • ;"'} 1 and note that C is convex. The associated 
family of information matrices, {M(~)l;£C}, has the same properties as the 
family of all information matrices (cf, Fedorov, 1972, p. 66). In particular, 
under the assumption that * s1 is finite we may, without loss of generality, 
restrict C to measures with finite·spectrum. Let·s2 denote the spectrum of ; 2 
The design problem is to choose the "best" design from C according to the 
following definition. 
~ 
Definition 1: The design ; is a marginally restricted D-optimal design 
if 
min jM-1(;)1 = IM-1 (€)J • 
t£C 
In the case that i,(x1 ,x2) • .z1 (x1) ® !2 (x2), where 0 denotes the Kronecker 
product, a marginally restricted D-optimal design is easily determined. 
I 
.... 
\ 
... 
5 
Lemma 1: If f (x1 ,x2) = .s,1 (x1 ) @ .s,2 (x2) on 
D * restricted D-optimal design is equal to ~2 x ~l 
design for ,s.2 on x2• 
X = x1 x x2 then a marginally 
D 
where ~2 is the D-optimal 
Proof: The result follows immediately from Hoel (1965). 
Recall that for any design 
and, thus, 
fx d(x1 ,x2; f;) df;(xl'x2) • p 
max d(x1,x2; ;) ~ p 
xl,x2 
The following lenuna establishes an analogous result for the maximum over 
the unrestricted margin, x2 • First, for ;EC, let; I (x Ix) • 2 1 2 1 
denote the associated conditional design on x2 given x1
ES~: 
;211 (x2lx1) = ~(x1 ,x2)/;;(x1) for tt(x1) > O. 
Lemma 2: For ;&C l max d("i_,x2; f;) df;~(x1) ~ p, 
X1 x2 
Proof: The result follows immediately from the relationship 
J, d(x1 ,x2; f;) df;211(x21x1) ~max d(x1 ,x2; f;) Xz x2 
* for all x1 E s1 . 
The following theorem presents equivalences for marginally restricted 
D-optimal designs analogous to those of Theorem 1 for D-optimal designs. 
Theorem 2: The following conditions are equivalent: 
(i) IM-1 (i)I = min IM-1(;)1 
~EC 
r ~ * (ii) Jv max d(x1,x2; ~) d~1 (x1) 
X1 x2 
6 
= min [ max d(x1 ,x2 ; t) dt~(x1) 
~EC X1 x2 
r A * (iii) Jv max d(x1,x2;t) dt 1(x1) ~ P. 
X1 x2 
The set of all designs satisfying these conditions is convex and the 
corresponding information matrices are identical. 
Proof: The proof follows along the same lines as that for Theorem 1. Only 
the main points will be sketched here. We first show that (ii) and (iii) 
follow from (i). 
A 
Let ~ satisfy (i) and let ; denote an arbitrary design in C. Then 
A A 
;a. = (1-a); + a; e: C for all O ~ a ~ 1. Since IM(;) I ~ )M(;) I 
for all t e:C we must have 
0 
- log IM(~ >II < 0 • arv - a, -""' a•O 
Or, after evaluation, 
Iv r d(~,x2; €> 
X1 Jx2 
* d;2(1<x2lx1) d;l(xl) ~ p 
I * * * where t 211 • t(x1,x2) t 1 (x1), x1 € s1 . Choosing, for each x1 E s1 , 
A 
t 211 cx21x1) to place mass 1 at the value of x2 which achieves maxx2
d(x1 ,x2; t) 
it follows that 
r A * J ~ max d(x1 ,x2; ~) d~1(x1) ~ P X1 x2 
This in combination with lemma 2 establishes results (ii) and (iii). 
J 
.. 
\ 
7 
,.. 
To show that (1) follows from (ii), let ~ satisfy (ii) and assume 
111-1 (€)1 > min l.!1-1 (t)I 
E;EC 
There is a design ~EC such that 
,.. 
However, since ~ satisfies (ii), 
and result (i) follows. Other equivalences follow in a similar fashion. 
As in the case of the equivalence theorem, Theorem 2 establishes 
equivalences between functionals based on the determinant and the variances 
of the predicted values, and provides conditions for verifying when a given 
design 1s the marginally restricted D-optimal design. However, the following 
necessary condition may be a bit easier to verify in practice. 
,.. 
Corollary 1: Let ~ denote a marginally restricted D-optimal design then 
Proof: By Theorem 2, v ~ p. Assume v < p, then 
and, thus, 
The result follows by contradiction • 
. ···------- .. ·----- -----
8 
According to this corollary, to verify that a given design is not the 
marginally restricted design we need consider only the points in s2• 
Additive models represent an important special case which frequently 
occurs in practice. If the experimenter can specify that the model is 
additive and contains a constant term then !. may be written as 
f'(x1 ,x2) = (1,.&i(x1), ,&2(x2)). The following lennna shows how to construct a 
marginally restricted D-optimal design in this case. 
Lennna 3: If !'(x1 ,x2) = (l,.&~ (x1), ,s.2(x2)) then a marginally restricted 
D * D D-optimal design is ; 2 x ; 1 , where ; 2 is the D-optimal design for 
(1,,s.2) on x2• 
Proof: Let ;(xl,x2) • ;l(xl) x t2<x2?, 
and 
where 
M = 
=-! h ~~ d;i (xi) 
X1 
- -1-di (xi; ;1) • .&j_ ~ £i 
~ =- (1,aI (xi)), i • 1,2. 
It is straightforward to verify that 
d(xl,x2; ;) = dl(xl; ;l) + d2(x2; ;2) - 1 • 
The result follows immediately from Theorems 1 and 2 by setting 
and D ;2 • (2. 
* tl - ;l 
In general, Lemma 3 will not hold for models without constant terms. 
This is easily seen by considering the case f'(x1 ,x2) = (x1 ,x2), X ::a [-1,1]
2
• 
Also, it is worth noting that Lemma 3 shows how to obtain a D-optimal design for an 
additive model in the unrestricted case. 
J 
' 
9 
The following examples illustrate the use of Theorem 2 in two cases not 
covered by Lemmas 1 or 3. 
2 Example 1: Let X = [-1,1] and !.'(x1 ,x2) = (l,x2x1 ,x2). Also, let 
D * D ;(x1 ,x2) = ; 2 (x2) x ; 1 (x1) where ; 2 is the D-optimal design for (1,x2) 
D D 
on [-1,1], i.e. ; 2(-1) • ; 2(1) • 1/2. It is easily verified that 
where 
Clearly, 
and 
2 2 2 d(x1,x2; ;) a 1 + x2 + x2x1/k 
f l 2 * k a x1 d;1(x1) • 
-1 
2 
max d(x1,x2; t) • 2 + x1/k 
x2 
r1 2 * J. (2+xl/k) d;l • 3. 
-1 
Thus, by condition (iii) of Theorem 2, t is a marginally restricted D-optimal 
design. 
2 2 Example 2: Let X = [-1,1] and !,'(x1 ,x2) = (l,x2 x1 ,x2). Consider the 
D * D 2 design ; m ; 2 x ; 1 where ; 2 is the D-optimal design for (1,x2,x2) on 
[-1,1), i.e. ~(-1) • ~(O) a ~(1) = 1/3. A little algebra will verify that 
2 3 2 2 9 4 d(xl,x2; ;) • 3 - 6x2 + 2k x2xl + 2 x2 
J i 2 * where k = x1d~(x1). 
-1 
Thus, 
max d(x1 ,x2; ;) a 3 
x2 2 3 xl 
- -(1 + -) 2 k 
if 
if 
2 
x1 ~ k 
2 
xl ~ k 
. ---·- ··---·------------
10 
and 
2 
f l * J 3 xl * max d(x1,x2; ~)d~1 (x1) • 3 + 2(1- k)d;1(x1) > 3. 
-1 x2 lx1l<v'k 
.l 
Therefore, by Theorem 2; ~ is not a marginally restricted D-optimal design. 
.. 
11 
3. Generating Marginally Restricted D-Optimal Designs. 
For every ; EC there exists a conditional design t
211 
such that 
~(xl,x2) = ~2j1<x2lx1) ;~(xl) 
* for all x1es1 and t 2 11 is an unrestricted design on x1xx2 . The followin~ 
lemma shows a parallel between the designs t 211 and D-optimal designs and 
indicates how to generate marginally restricted D-optimal designs. 
A 
Lemma 4: 
for all 
* ; Proof: For all ( EC and xl E sl max d(x X •l:°) > d(x el:') dl:' ( I ) l' 2'~ - ~ l'x2,~ ~211 x2 xl 
_x2 2 
Sufficiency follows by letting ; in this expression be a marginally 
i:-* restricted D-optimal design, integrating both sides with respect to ~1 
and then noting that in the resulting expression the left hand side 
equals p by (iii) of Theorem 2 and the right hand side equals p by 
construction. 
To show necessity, choose t £ C such that 
max d(x1,x2;i) • Jx d(x1 ,x2;~) d~211 (x2 1x1) 
x2 2 
* for all x1 E s1 The result follows by integrating both sides with 
* respect to t 1 and using (iii) of Theorem 2. 
* Lemma 4 shows that for any x1 E s1 we must have 
where x2 
,. * ,. 
d(x1,x2;;) • d{x1 ,x2;;) 
A 
* and x2 are points of support of t 211 <x21x1) • Thus, to 
iteratively generate marginally restricted D-optimal designs we focus 
on the conditional designs ~211 
12 
Let E; denote a design at some iteration. The design f;+l' say, at 
the next iteration is obtained by augmenting E; * * with a fixed point (x1, x2) 
* * * where xl £ sl and x2 e; X2 . Specifically, for O < a < 1 let 
~ * * * * 211 (x2lx1) • (1-a) t211 (x2lx1) + ao(xl, x2) 
* * where o places mass 1 at (x1 , x2) The design at the next iteration 
is now defined as 
· E;+l (xl ,x2) • 
* 
~2f1<x2fx1) (1 (xl) 
a * * * 
~211 (x2lx1) E;l (xl) 
* * 
* 
xl 1' xl 
* 
xl m xl 
The following lemma indicates how (x1 , x2) is to be chosen. 
Lemma 1= Let E; be any nonsingular marginally restricted design. Define 
* * ;+l as above with (x1 , x2) such that 
max [max dCx1, x2; t> - ( d(x1 ,x2; ;) d;211 Cx21x1)J • 
"i x2 *Jx2* r * * 
Then 
d(x1 , x2 ; t) - Jv d(x1, x2; ~)d;211(x21x1) 
X2 
_a_ lnlM(;+1> I 
a a I > 0 a•O 
with equality if and only if t is a marginally restricted D-optimal design. 
------------ .. ·----
J 
.( 
'." 
. 
• 
13 
Proof: From the definition of ~+l' 
I 
1:!<E:+1> * * * * , * * * •.!:!(~)+a tl(xl) [!,(xl,x2) !. (xl, x2) - 1:!<E:211• xl)] 
* where ]:!(~2 11, x1) r * , * * =JY !(xl, x2) ! (xl, x2) d~2J1<x2lx1) 
X2 
Thus, 
_L lnlM<E:+1> I a a -1 ) a Tr M (E:+l * * * *, * * t1 (x1) [.!,(x1, x 2) 1 (x1,x2) 
* 
- !!<E:2 I 1 • x1> 1 
(See, for example, Fedorov, 1972). 
It follows then that 
a I •• ** J. * aa ln(M(~+l~ I m ~l(xl) [d(xl, x2; ~) - d(~, x2; ~) d~211<x2lx1)l 
a~o x2 
Thus, 
a 
:fa lnlM (E:+1 ) I > 0 
a:::10 
and by Lemma 4 equality is achieved if and only if ~ is a marginally restricted 
I>-optimal design. This completes the proof. 
* * The method of choosing (x1 , x2) and generating ¼-l are the essential 
ingredients in an iterative scheme to generate a marginally restricted·D-optimal 
design. The sequence of weights {ai} and the termination criterion can be 
specified generally as in schemes for generating unrestricted D-optimal 
designs. See, Fedorov (1972) and Tsay (1976). 
14 
References 
Fedorov, V.V. (1972). The Theory of Optimal Experiments. Translated and 
edited by W.J. Studden and E.M. Klimko. New York. 
Harville, D. (1974). Nearly optimal allocation of experimental units 
using observed covariate values. Technometrics 16, 589. 
Harville, D. (1975). Computing optimum designs for covariance models. 
In A Survey of Stntistical Design and Linear Models. Edited by 
J.N. Srivastava. North-Holland. 
Hoel, P.G. (1965). Minimax designs in two dimensional regression. 
Ann. Math. Statist. 36, 1097. 
Kiefer, J. and Wolfowitz (1960), "The equivalence of two extremum problems," 
The Canadian Journal of Mathematics, 12, 363-366. 
Tsay, J. (1976). On the sequential construction of D-optimal designs. 
Journal of the American Statistical Association 71, 671-674. 
I 
J