choice-metrics.com

Hello all,

I am trying to generate the orthogonal design with 5 attributes(3 levels each) and try to estimate all 2-order interactions as well.The main goal is estimating interactions, I could also run the Bayesian efficient design, although we can't obtain priors. I have specified interactions in the utility function. You may see the syntax below. Although I was faced with the problem that the maximum number of rows it allows is 144, at least I tried to play around with figures greater than 144 and no design was found. I basically need not less than 300 rows. Could you assist me with solving this issue please?


Thanks in advance,
Anna

Design
;alts=alt1, alt2
;rows=144
;orth=seq
;eff=(mnl,d)
;model:
U(alt1)=b1+b2*Mobility[0,1,2]+
b3*selfcare[0,1,2]+
b4*usualactivity[0,1,2]+
b5*pain[0,1,2]+
b6*depression[0,1,2]+
i1*Mobility*selfcare+
i2*Mobility*usualactivity+
i3*Mobility*pain+
i4*Mobility*depression+
i5*selfcare*usualactivity+
i6*selfcare*pain+
i7*selfcare*depression+
i8*usualactivity*pain+
i9*usualactivity*depression+
i10*pain*depression
/
U(alt2)=b2*Mobility+
b3*selfcare+
b4*usualactivity+
b5*pain+
b6*depression+
i1*Mobility*selfcare+
i2*Mobility*usualactivity+
i3*Mobility*pain+
i4*Mobility*depression+
i5*selfcare*usualactivity+
i6*selfcare*pain+
i7*selfcare*depression+
i8*usualactivity*pain+
i9*usualactivity*depression+
i10*pain*depression

$

The full factorial in a sequential orthogonal design is 3^5 = 243 possible choice situations. This contains every possible combination within an alternative. With the full factorial you can estimate every single interaction effect (2-way interactions, 3-way interactions, etc). I am not sure why you need more than 300, since clin your experiment there do not exist more than 243 possible profiles.

Hello Michiel,

Thanks a lot for your response. Also we have the same issue with 5 attributes(5 levels each) where the number of possible pairs is 4 881 250, so the full factorial is not applicable. That is why we planned to select at least 300 paired comparisons out of full factorial. Could you please assist on how we can generate them using orthogonal design because currently it doesn't generate more than 144 pairs?

Why does the design need to be orthogonal? As far as I am aware, there are no known orthogonal designs larger than 144 so Ngene cannot find them. Most of us would use efficient designs which exist for any number of rows. Further, why would you need so many choice tasks? 300 is very large, we usually use much smaller designs.

So summarising, you are asking for a design that does not exist and such large designs are typically not used in practice. I suggest using an efficient design with zero priors with much fewer rows letting go of orthogonality.

Thanks a lot for your clarification regarding to the ortogonal design rows maximum number, that was exactly the answer to my question. We wanted to use 300 choice tasks and put them into blocks to present 10-12 choice tasks per respondent, we can have a big sample. Also we would like to estimate interactions, which requires more choice tasks to include, but less than full factorial. The issue with efficient design was the inability to get priors, but now I am trying to identify how to get them. I have read that having 0 priors is bad specification for the model, so I will need to get the more precise ones.

Thanks again for your help!

I think your information is incorrect.

1. An orthogonal design implicitly assumes zero priors. So why would this be bad? Assuming zero priors and finding an efficient design will give you a near orthogonal design that also optimises for standard errors and makes sure you can estimate all interactions you specified. Which is all you want.

2. You do not need so many rows to estimate interactions. The statistical rule for the number of degrees to freedom, and hence the minimum nunber of rows, is the number of parameters you wish to estimate (it does not matter whether it is a main effect or interaction effect) dividided by (number of alternatives minus 1). Hence in your case you could easily estimate all parameters in a balanced design with 12 choice tasks (the minimum is 16 / 2 = 8). The efficient design automatically chooses the appropriate choice tasks such that all effects can be estimated, otherwise it would generate an infinite D-error. If Ngene can find a finite D-error, it means the model can be estimated. So if you are concerned about not having enough choice tasks, you could double it to 24 and block it, or you can use a foldover, but using 300 choice tasks is really too much and will contain many not so smart questions leading to an inefficient design resulting in larger standard errors.