choice-metrics.com

Dear Michiel,

Hope everything goes well for you!

I am generating a Bayesian design (the code is below). The D error of the Bayesian mean is 9.99. Is it too high? I know the lower D error is better. What is your thumb rule of D error for Bayesian design? Is there any way to decrease the D error?
Also if I add the interaction term, will it cause an issue?

Thanks for your help.

Wei

Code: Select all

Design;
;alts = alt1*, alt2*, nobuy
;rows = 16
;block = 2
;eff  = (mnl, d, mean)
;bdraws = gauss(3)
;model:
U(alt1) = b0[(n,-2.739, 0.329)] + b1[(n,-0.123, 0.018)] * price[7.99, 11.99, 15.99, 19.99]
             + b2.dummy[(n, 0.359, 0.199) | (n,-0.332,0.223) |(n, -1.142, 0.223)] * mp [1,2,3,0]
             + b3.dummy[(n,-0.647, 0.132)]  * production[1,0]
             + b4.dummy[(n,0.09,0.131)] * sustainability [1,0]
             ?+ b5 * mp.dummy[1] * production.dummy[1]
             ?+ b6 * mp.dummy[2] * production.dummy[1]
             ?+ b7 * mp.dummy[3] * production.dummy[1]
             ?+ b8 * mp.dummy[1] * sustainability.dummy[1]
             ?+ b9 * mp.dummy[2] * sustainability.dummy[1]
             ?+ b10 * mp.dummy[3] * sustainability.dummy[1]
           
/

U(alt2) = b0 + b1 * price
             + b2 * mp
             + b3 * production
             + b4 * sustainability
             ?+ b5 * mp.dummy[1] * production.dummy[1]
             ?+ b6 * mp.dummy[2] * production.dummy[1]
             ?+ b7 * mp.dummy[3] * production.dummy[1]
             ?+ b8 * mp.dummy[1] * sustainability.dummy[1]
             ?+ b9 * mp.dummy[2] * sustainability.dummy[1]
             ?+ b10 * mp.dummy[3] * sustainability.dummy[1]
             
$


The Bayesian D-error is simply the average D-error over a range of priors, so it would be expected to be slightly higher than a D-error. A value of 10 is very high and indicates that there is an issue.

The issue in your case is that your constant has a strong negative value, and if you look at the choice probabilities of your choice tasks you can see that the opt-out is expected to be chosen with 99% probability, so there are hardly any trade-offs being made. If these priors are based on pilot study data, is it correct that the opt-out is almost always chosen? If not, then your priors are not correct.

If is not a problem to add interaction terms, but you may want to increase the number of rows since it will increase the number of parameters in your model.

Michiel

Thanks for your information, Michiel!

I checked the opt-out rate. The highest rate is 45% and lowest rate is 3% per choice task. I used OOD design (code is shown below) for pilot study and the priors are from the mnl result of pilot study.

Is there any issue for the OOD design?

Thanks for your help,

Wei

Code: Select all

Design;
;alts = alt1*, alt2*, nobuy
;rows = 16
;block = 2
;orth = ood
;model:
U(alt1) = b0 + b1 * price[7.99, 11.99, 15.99, 19.99]
             + b2 * mp [1,2,3,0]
             + b3 * production [1,0]
             + b4 * sustainability [1,0]
             + b5 * mp.dummy[1] * production.dummy[1]
             + b6 * mp.dummy[2] * production.dummy[1]
             + b7 * mp.dummy[3] * production.dummy[1]
             + b8 * mp.dummy[1] * sustainability.dummy[1]
             + b9 * mp.dummy[2] * sustainability.dummy[1]
             + b10 * mp.dummy[3] * sustainability.dummy[1]         
/
U(alt2) = b0 + b1 * price
             + b2 * mp
             + b3 * production
             + b4 * sustainability
             + b5 * mp.dummy[1] * production.dummy[1]
             + b6 * mp.dummy[2] * production.dummy[1]
             + b7 * mp.dummy[3] * production.dummy[1]
             + b8 * mp.dummy[1] * sustainability.dummy[1]
             + b9 * mp.dummy[2] * sustainability.dummy[1]
             + b10 * mp.dummy[3] * sustainability.dummy[1]


$



I don't think that there is anything wrong with the OOD design (unless you have a dominant attribute whereby respondents ignore all other attributes). In your OOD design I would have dummy coded the MP attribute, which would increase the number of parameters and also include the number of rows in your orthogonal design, but I don't think that there is an issue.

I think that the issue is in your priors, so perhaps there is an issue in your model estimation. Your specified priors indicate that 99% of respondents choose the opt-out, whereas your data tells you that this is much lower than 45%. So your priors are not consistent with your data. This could be because you coded the levels differently (did you use 7.99 to 19.99 also as levels in estimation for price? did you use the same base level for your dummy coded variables?).

Michiel

Yes, I use the same price levels in the estimation.
I defined three binary variables for MP variable: MP_low (when MP=1), MP_med (when MP = 2), and MP_high(when MP=3).

Could you please explain how you got 99% of respondents to choose the opt-out based on the specified priors?

Thanks,

Wei

Simply calculate the utilities based on your priors, and then compute the choice probabilities using the logit formula. This will give you choice probabilities of around 98-99% for the opt-out alternative,

You can also see this in Ngene. Run the script with your priors, open a design. Select to Design properties, MNL | Probabilities.

Michiel