choice-metrics.com

Dear NGENE developers,

I am writing concerning a design of a survey aimed at farmers, focusing on their willingness to accept payment incentives with several attributes. Given the infeasibility of conducting pilot surveys, we have specified priors based on our previous research and the existing literature.

We estimated an efficient design, optimizing for an MNL model and minimizing D-error. All attributes, except for payment, are qualitative and dummy coded. Initially, we tested the Federov algorithm, but we removed it because, as noted in the Ngene manual, combining a continuous attribute with dummy-coded attributes resulted in some payment levels not appearing in the design.

However, even after removing the Federov algorithm, we encountered a major issue: in almost all of the 18 choice situations, the payment levels are identical in alternative 1 and alternative 2. We are concerned that this may reduce the credibility of the survey for farmers and potentially increase status quo bias. Moreover, since we aim to estimate willingness to accept (WTA), we are concerned that respondents may not sufficiently trade off payment against other attributes, which could affect the robustness of preference estimates.

This is the design code:

Design
;alts = alt1*, alt2*, optout
;rows = 18
;block = 2
;eff = (mnl,d)
;con

;model:
U(alt1) =
herd.dummy[0.15| 0.10] * HERD[2,1,0] ? prefiero llevar todo el ganado que hacer lotes
+ dist.dummy[-0.4 | -0.2] * DIST[2,1,0]
+ coop.dummy[-0.3| 0.1] * COOP[2,1,0] ?cooperar con quién y quién me lo da
+ pay[0.05] * PAY[50,100,150,200,250,300]/

U(alt2) =
herd.dummy* HERD[2,1,0]
+ dist.dummy* DIST[2,1,0]
+ coop.dummy* COOP[2,1,0]
+ pay* PAY[50,100,150,200,250,300] $


We have tested alternative specifications, such as using the ;reject command to avoid equal payment levels across alternatives. However, this leads to higher D-error and S-estimates than would be considered acceptable. This is the code for this design:


Design
;alts = alt1*, alt2*, optout
;rows = 18
;block = 2
;eff = (mnl,d)
;con
;reject:
alt1.PAY=alt2.PAY
;alg=mfederov

;model:
U(alt1) =
herd.dummy[0.15| 0.10] * HERD[2,1,0] ? prefiero llevar todo el ganado que hacer lotes
+ dist.dummy[-0.4 | -0.2] * DIST[2,1,0]
+ coop.dummy[-0.3| 0.1] * COOP[2,1,0] ?cooperar con quién y quién me lo da
+ pay[0.05] * PAY[50,100,150,200,250,300]/

U(alt2) =
herd.dummy* HERD[2,1,0]
+ dist.dummy* DIST[2,1,0]
+ coop.dummy* COOP[2,1,0]
+ pay* PAY[50,100,150,200,250,300] $

Our questions therefore are:

1. Why does Ngene tend to assign equal payment levels to alternatives 1 and 2 in the initial design?
2. Is there a way to prevent this while maintaining reasonably low D-error and S-estimates?


Thank you very much for your help.
Kind regards,
Elsa

There is a good reason why Ngene generates such a design, namely your prior for PAY indicates that it is a dominant attribute. It has a major contribution to utility, e.g. 0.05 * 200 = 10, which is huge in terms of utility. If such a dominant attribute exists, then to avoid that respondents only look at the PAY attribute and ignore all other attributes, an efficient design will automatically create overlap in the PAY attribute to force respondents to consider the other attributes.

In reality, PAY is probably not a dominant attribute, which means that your prior value is wrong. A value of 0.05 is very large. You should generally only use priors from a pilot study and never try to guess these values. If you reduce the value to 0.005 then you will see the efficient design no longer has such strong overlap. So the solution is very simple: change the prior for "pay".

You could consider uninformative priors by using herd.dummy[+] and pay[+] for example.

Michiel