choice-metrics.com

Dear Ngene Support Team,

I have a question about how Ngene calculates utility when using effects coding, and also about the validity of using effects coding for a D-efficient design.

When I changed my model from dummy coding to effects coding, the D-error decreased dramatically—from around 1.0 to about 0.2–0.3.
I thought this suggests that effects coding may work better in my design.
However, I am not sure how the utility is calculated in this case, and I would like to confirm my understanding.

In my design, there are four attributes in total.
One attribute has three levels, and the other three attributes have two levels each, resulting in 3 × 2 × 2 × 2 = 24 possible combinations.

I am also wondering whether it is appropriate to apply effects coding when generating a D-efficient design, or if this approach could lead to misleadingly small D-error values.

Could you please explain how Ngene computes the utility when effects coding is used, and whether using effects coding in a D-efficient design is considered appropriate?
Since the Ngene manual does not include many examples using effects coding, it would be very helpful if you could provide a simple example code and explain how the utility is calculated in that example.

Thank you very much for your help.

Best regards,

First of all, it does not matter which coding scheme you select for generating an efficient design. Yes, the D-error changes, but D-errors are not comparable across models. So for different utility functions, different priors, different coding schemes, the D-errors are not comparable. So you should not pay attention too much at the absolute value of the D-error because it is meaningless. What matter is that you choose the design with the lowest D-error that can be achieved with that particular coding scheme.

The reason why dummy coded is usually preferred is because it is easier to interpret. Effects coding does not materially have any benefits, but it is fine to use effects coding if you want.

Let me explain how effects coding works. Suppose we have attribute colour with three levels: Blue (1), Red (2) and Yellow (3), where the numbers indicate the level coding used in the Ngene script, so something like:

Colour[1,2,3] ? 1=Blue, 2=Red, 3=Yellow

The last level in Ngene is assumed the base/reference level, in this case, Yellow(3). With dummy or effects coding, Ngene creates two variables, namely Colour1 and Colour2.

With dummy coding, the following scheme would be used:
Blue: Colour1 = 1, Colour2 = 0
Red: Colour1 = 0, Colour2 = 1
Yellow: Colour1 = 0, Colour2 = 0

Suppose that the following priors were specified:

b.dummy[0.1|0.5] * Colour[1,2,3] ? 1=Blue, 2=Red, 3=Yellow

This means that U(Blue) = 0.1, U(Red) = 0.5, and U(Yellow) = 0. In other words, Yellow is least preferred and Red is most preferred.

With effects coding, only the coding of the base level changes:
Blue: Colour1 = 1, Colour2 = 0
Red: Colour1 = 0, Colour2 = 1
Yellow: Colour1 = -1, Colour2 = -1

Therefore, an identical model with effects coding would be:

b.effects[-0.1|0.3] * Colour[1,2,3] ? 1=Blue, 2=Red, 3=Yellow

This means that U(Blue) = -0.1, U(Red) = 0.3, and U(Yellow) = -(-0.1) - 0.3 = -0.2. In other words, Yellow is again least preferred and Red is most preferred. With effects coding, the coefficients are interpreted against the grand mean instead of against a base level. This means that Blue has 0.1 less utility than the average utility for colour, and Red has 0.2 more utility than the average utility for colour.

This results in the same behavioural model because only differences in utility matter. So no matter whether you apply dummy or effects coding, the difference in utility is the same:
U(Red) - U(Blue) = 0.4
U(Red) - U(Yellow) = 0.5

And since it expresses the same behavioural model, you can choose which coding scheme you prefer.

Michiel

Thank you very much for your detailed explanation.
I now understand how the utility is calculated when using effects coding.

You mentioned that the D-error can vary greatly depending on the model specification.
I would like to ask one more question:
How can we judge whether a D-error value is sufficiently low to consider the design well D-efficient?
In other words, is there any general guideline or reference range for interpreting the magnitude of the D-error?

Thank you again for your kind support.

Best regards,

There is no way say what a good value for the D-error is. For one study, a D-error of 0.1 is very good, while for another study a D-error of 0.1 can be bad.

This means that you should not really pay much attention to the value of the D-error. A D-efficient design is a design with a relatively low D-error for that particular study, but the value itself is meaningless. The only thing that indicates a bad design is a very large D-error, such as 1000 (well above 1). This indicates that the model is ill defined and that model estimation will almost certainly fail with the generated design. This usually only happens in case there is multicollinearity imposed by the analyst (via reject, require, or conditional constraints) or when the model is over-specified (e.g. too many constants).

Michiel