Setting cost levels in Ngene

[mattw]

Hello, thank you for offering this forum. It has been some time since I have used Ngene and on behalf of my study team I wanted to post a question on a national survey we are working on. There are 3 attributes besides cost, the 1st is a negative on utility with 3 levels, the 2nd is a positive on utility also with 3 levels, the 3rd is also positive and is binary. From focus group pretests we have set priors on these so that their influence on utility is approximately equal. We have also set cost levels such that the middle cost level would approximately balance the maximum improvement offered. We borrowed some syntax from the stormwater demo project in Ngene.

We have 4 questions related to this design:
1) Is there any guidance on how to set cost levels other than offering a decent spread above and below the level that would exactly offset a "maximum" policy change? I have heard the lowest cost should be attractive to most respondents and the highest cost should deter almost everyone, but wondering if there are other factors practitioners use, including deciding on the total number of cost levels to offer. More levels seems like it would be easier to avoid dominance problems.
2) The below script does not run. However if I change "tax[24,56..] below to "tax[0,56,... ], in other words just replace 24 with 0, it does run. However I do not plan to offer policyA at zero cost. So I am wondering if there is another problem with my script that I am not understanding, please advise if anyone recognizes a problem here. I am trying to get the script to run without the need to add a 0 cost option for PolicyA.
3) The choice experiment is fairly sparse, and I am wondering if 32 runs is overly high (8 blocks 4 questions each). More runs means more statistical variability but also more dominance problems, are there rules of thumb here?
4) I was interested in the "s" efficiency option because it seemed it would have a more direct interpretation than "d" efficiency, such as to sample size but the "s" measure of 0.95, I am not sure if it has a direct interpretation.

Thanks much for any advice! -Matt

design
;alts = (policyA, sq)
;rows = 32
;block = 8
;eff = (mnl,s)
;alg = mfedorov(candidates = 68)
;require:
sq.HAB_area = 100, sq.OE = 0, sq.HAB_info = 0, sq.tax = 0
;model:
U(policyA) = b_HAB_area[-0.2] * HAB_area[100,50,0] ? sqmi of HAB area
+ b_OE[0.1] * OE[0,50,100] ? sqmi of high qual habitat
+ b_HAB_info[10] * HAB_info[0,1] ? 0 = low (base), 1 = high
+ b_tax[-0.1] * tax[24,56,112,173,240] ? 0, 68, 127, 173, 246 annual tax ($)
/
U(sq) = b_sq[0]
+ b_HAB_area * HAB_area
+ b_OE * OE
+ b_HAB_info * HAB_info
+ b_tax * tax
$

[Michiel Bliemer]

1) There is no definite guidance on which attribute levels to select. It does not hurt to increase the number of levels of a numerical attribute, especially if the range is large. What you want to avoid is that going from one price level to the next makes one of the alternatives "too expensive" compared to the other. Since different respondents have different willingness to pay, it is therefore often a good idea to have a sufficient number of "steps" in your price attribute.

2) You are using the same attribute name "tax" for the status quo and try to set it to zero, but you defined this attribute without zero. There are two solutions: (i) you define tax[0,24,56,...etc} and then impose require: sq.tax = 0 and reject: policyA.tax = 0, (ii) you give the tax attribute a different name, so you can give it different attribute levels, so simply tax_sq[0]. The second option is of course much easier, as adopted in the script below. You can do the same for all other attributes and remove the require constraints. You would need require constraints if you have categorical attributes.

3) There is usually no issue in using a large number of choice tasks in your design. A design can be too small, but not too large. You probably do not need 32 rows, but it is fine.

4) An "S-error" of 0.95 indicates that essentially a single respondent answering 32 questions would suffice to estimate your 5 parameters. Since you have 8 blocks, you would need to multiply the number of respondents by 8, so 0.95*8 is about 8 respondents to estimate all parameters at a statistically significant level. I generally advise against using S-efficiency as an optimisation criterion because it only optimises on the most difficult to estimate parameter. The D-error optimises over all parameters equally. So I would recommend using ;eff = (mnl,d) and then simply look at the S-estimates in the Ngene output to see how many respondents you need for each parameter to be statistically signifcant (again multiplying each value with 8).

IMPORTANT: Your priors are really bad: they are far too large compared to your attribute levels. Bad priors will lead to bad (inefficient) designs. If you insist on using informative priors without conducting a pilot study, then I would recommend dividing all of them by 10. This will likely give you very different (but more realistic) sample size estimates.

Other comments:
1. You may want to use PolicyA, PolicyB, SQ. This would increase variation and information in your data. But of course it makes the choice tasks more difficult for the respondent.
2. It may be best to use the default swapping algorithm, which maintains attribute level balance. See script below.

Finally, it may be best to post questions that contain Ngene syntax into the relevant Ngene forums (in your case Ngene Online) instead of the general forum, so I know which Ngene version you are using (as the syntax varies slightly).

Code: Select all

design
;alts = (policyA, sq)
;rows = 32
;block = 8
;eff = (mnl,d)
;model:
U(policyA) = b_HAB_area[-0.02] * HAB_area[100,50,0] ? sqmi of HAB area
+ b_OE[0.01] * OE[0,50,100] ? sqmi of high qual habitat
+ b_HAB_info[1] * HAB_info[0,1] ? 0 = low (base), 1 = high
+ b_tax[-0.01] * tax[24,56,112,173,240] ? 0, 68, 127, 173, 246 annual tax ($)
/
U(sq) = b_sq[0]
+ b_HAB_area * HAB_area_sq[100]
+ b_OE * OE_sq[0]
+ b_HAB_info * HAB_info_sq[0]
+ b_tax * tax_sq[0]
$

Michiel