choice-metrics.com

Posted: **Fri Jun 01, 2018 3:34 pm**

Dear all,
I am working on a CE study where we will present for each choice task two product alternatives and a no-buy alternative.
I have generated a D-efficient design, with constrains and priors = 0, that has been used in a pilot survey with 100 subjects. I have estimated a MNL model to obtain the priors for the generation of a Bayesian design. Specifically, following suggestions on the forum, I have divided by 2 the priors (both mean and st errors), and specified the mean values = 0 of the parameters which were statistically insignificant in the MNL model. I have assumed a normal distribution for all priors.
I would like to generate a Bayesian design on the basis of WTP efficiency criterion. I am not very familiar with this kind of design and I have some questions . Here is the syntax I have used:

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(*/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[(n,0.70,0.13)|(n,0.36,0.12)|(n,0.33,0.12)]*STANDARDS[3,2,1,0]
        + b3[(n,3.18,0.44)]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[(n,0,1.60)]*INV[0,0.05,0.10]
        + b5.dummy[(n,0.60,0.18)|(n,0.58,0.18)|(n,0,0.18)]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[(n,0,0.24)]$

First of all, I do hope I have used the appropriate syntax and please see below the efficiency estimates I have obtained:

Code: Select all

		       Bayesian								
       Fixed	Mean	Std dev.	Median	Minimum	Maximum				
D error	0.482119	0.491894	0.006665	0.490296	0.48222	0.521154				
A error	6.511585	6.676097	0.29205	6.647395	6.237344	7.796395				
B estimate	84.551825	81.475317	8.39142	83.43224	55.881892	95.484354				
S estimate	11.894459	115231.9194	840837.3707	1088.419697	34.102464	8485557.686				
WTP(wtp1) estimate	1134.533193	1231.279807	109.305549	1206.002631	1067.009648	1840.774782				
WTP(wtp1) n	Infinity	115228.9605	840822.1028	1092.704408	39.391768	8484252.056				
										
Prior	b1	b2(d0)	b2(d1)	b2(d2)	b3	b4	b5(d0)	b5(d1)	b5(d2)	b0
Fixed prior value	-0.23	0.7	0.36	0.33	3.18	0	0.6	0.58	0	0
Sp estimates	4.514377	4.114633	10.582062	11.894459	3.79845	Undefined	9.556447	10.049788	Undefined	Undefined
Sp t-ratios	0.92248	0.966252	0.602519	0.568308	1.005664	0	0.634027	0.618269	0	0
Sb mean estimates	4.611486	4.741807	21.746327	39.386371	4.155939	17954.6242	2164.039473	18.180973	92730.11347	3073.650829
Sb mean t-ratios	0.912947	0.954193	0.594327	0.558814	0.990834	0.17711	0.619799	0.603889	0.157729	0.235089

I am not sure whether in WTP efficient designs using priors=0 would be ok.
Second, as far as I understood, “WTP(wtp1) estimate” would indicate the C-error values (please correct me if I am wrong) and should suggest the minimum number or design replications necessary to obtain statistically significant WTP values. Basically, my output tells me that, assuming my priors are correct, I need a number of 1134 * #blocks (12) respondents to obtain statistically significant WTP estimates. Did I understand correctly?
Also, what is the difference between “WTP(wtp1) estimate” and “WTP(wtp1) n”?
I apologize in advance if this has already been source of discussion on the forum or if I have missed the explanation on the manual.
Thanks a lot,
Claudia

Posted: **Fri Jun 01, 2018 4:14 pm**

Hi Claudia

With the exception of the denominator in a defined WTP estimate using C-error, you can apply any prior to the design, including zero. Unfortunately, zeros can cause issues in calculating efficiency measures such as sample size requirements. The sample size estimate is basically stating the smallest sample size you need to detect whether a parameter is statistically significantly different from zero (assuming a t of 1.96) from zero. The smaller the parameter (prior), the larger the sample size required to detect if it is different from zero.

If you look through the first block of output, the first column of numbers starting with 0.482119 represents the efficiency measures using any fixed priors and the means of the Bayesian priors – for example, -0.23 for b1 and 3.18 for b3. The 0.482119 is the D-error assuming the fixed priors and treating the means of Bayesian priors as fixed priors. The 1134.533193 you refer to is the C-error estimate (i.e., (wtp1) estimate) using these priors. The second column in this block of output are the Bayesian errors. For the C-error, the Bayesian C-error is actually 12.31.279807. Note that this is the C-error, not the WTP sample size which is the last row within this block of output.
Note that for b4, the mean of the Bayesian distribution is 0, and hence the sample size required to detect whether this is statistically different from 0 is infinite. This is what is being reported – in the row labelled WTP(wtp1) n – if you have a zero prior, you need an infinite sample size. The next value, 115,228.9605 is the actually number of design replications needed. Again, this is very large number simply because you have zero priors (same reasoning only it is taking draws and didn’t hit 0, but in the draws, did take values close to zero – mathematically, this number should be 0 also, but the number reported is simply simulation error). But this is the number you are after.

To demonstrate, if you use only parameters with non-zero priors in your calculation (e.g., ;wtp = wtp1(b2,b3/b1), you will see the number of design replications drop to under 60 in a few seconds. Note I didn’t include b5 in this, as one of the dummies has a zero mean, which I don’t think can be isolated in the syntax.

John

Posted: **Sat Jun 02, 2018 4:01 pm**

John,
Very appreciated. Everything is clear. Thank you very much.
Finally, I followed your suggestion and ran the following syntax:

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(b2,b3/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[(n,0.70,0.13)|(n,0.36,0.12)|(n,0.33,0.12)]*STANDARDS[3,2,1,0]
        + b3[(n,3.18,0.44)]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[(n,0,1.60)]*INV[0,0.05,0.10]
        + b5.dummy[(n,0.60,0.18)|(n,0.58,0.18)|(n,0,0.18)]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[(n,0,0.24)]$

And yes, everything drops down! As far as I understood, the C-error relates to the efficiency of the design at the generation stage, but C-error estimates = 97.01 is still big, right?
Also, I have noticed that the four price levels are allocated in a way that respondents are always presented with the trade-off among the two product alternatives between only two out of all the possible combinations. I always have 1 - 4 and 2 - 3 combinations and never 2 - 4 or 2 - 1, for example. Am I doing something wrong?
Thanks again for everything,
Claudia

Posted: **Sun Jun 03, 2018 9:03 am**

Hi Claudia

Yes, the C-error is quite large which is an indication that there may be something not quite right with the design. However, before I get to that, I just noticed that you are not optimising for your Bayesian priors. The command

;eff = (mnl,wtp(wtp1))

will optimise on the fixed priors, even though you have specified Bayesian priors. The command as you have it will show the Bayesian priors but not optimise on them, You need to tell it what to do with the Bayesian priors as in:

;eff = (mnl,wtp(wtp1),mean)
;bdraws = halton(200)

Now back to your design. To look for issues, start simple. Remove the Bayesian priors and start with a fixed prior design as in

Code: Select all

Design
;alts = alt1, alt2, nobuy
;rows = 48
;block =12;
;eff = (mnl,wtp(wtp1))
;wtp = wtp1(b2,b3/b1)
;con
;cond:
if (alt1.INV=[0], alt1.COM=[0]),
if (alt2.INV=[0], alt2.COM=[0]),
if (alt1.COM=[0], alt1.INV=[0]),
if (alt2.COM=[0], alt2.INV=[0])
;model:
U(alt1) = b1[-0.23]*PPPRICE[1,2,3,4]         
        + b2.dummy[0.70|0.36|0.33]*STANDARDS[3,2,1,0]
        + b3[3.18]*FPRICE[0.05,0.1,0.2,0.3]
        + b4[0]*INV[0,0.05,0.10]
        + b5.dummy[0.60|0.58|0.18]*COM[3,2,1,0]
/U(alt2) = b1*PPPRICE
        + b2*STANDARDS
        + b3*FPRICE
        + b4*INV
        + b5*COM/
U(nobuy)= b0[0.24]$

Running this syntax, and computing the choice probabilities (you can get these out of Ngene), you will note that the average choice probability for the three alternatives is around 0.375, 0.375, 0.25. Now increase the range of the price levels to [1,3,5,7] and you will see the average probabilities for the alternatives go to 0.33,0.33,0.33 (as well as huge improvements in the efficiency obtained). Efficient designs typically have an optimal set of choice probabilities (sometimes referred to as magic Ps). Given that the choice probabilities of all alternatives are a function of the combination of all levels, the algorithm will try and find the combination of levels that produce these probabilities. In this case, it wants to find choice probabilities of 0.33,0.33,0.33 but given the ranges imposed on the levels, it can't. To get the best choice probabilities it can, it is choosing the price level combinations you are seeing.

As you will see, increasing the range of the levels will generally improve efficiency. But be careful, the study requirements should in the most part rule the design, not the math underlying the design rule the study requirements. Unless finding a design proves intractable for the study, I personally don't like changing the study inputs.

John

Posted: **Tue Jun 05, 2018 3:48 am**

John,
Thanks a lot for this. I really appreciate it. Ok.. if I thought everything was clear, now it is even more clear!

Re the magic Ps, I ran the syntax without the Bayesian Priors, but I did not obtain any improvements in terms of C-error though. However, I “solved” the problem of price levels allocation by eliminating the “;con” command (that, I think, would not be actually correct). Anyway, as you suggested, I got the best output by increasing the price levels range, but I do not think this is behaviorally feasible since they would not reflect the real market prices of the product.
Thank you! I learned a lot!
Ciao ciao,
Claudia

choice-metrics.com

C-error in WTP efficient designs

C-error in WTP efficient designs

Re: C-error in WTP efficient designs

Re: C-error in WTP efficient designs

Re: C-error in WTP efficient designs

Re: C-error in WTP efficient designs