choice-metrics.com

Hi Michiel,

I would really appreciate your advice on the final Bayesian design for my DCE.

I have an unlabelled experiment with two alternatives, 24 choice tasks in two blocks of 12, and five attributes. Based on the pilot data, the primary main-effects MNL specification treats TST, falls, and cost as continuous, while sedation (SED) and evidence (EV) are categorical. I also estimated a categorical sensitivity model in which TST and falls were specified categorically, given qualitative pre-testing evidence suggesting potential non-linearity in these attributes. However, the continuous specification provided a better fit according to both AIC and BIC and was therefore retained as the preferred specification. I did not investigate a dummy-coded specification for cost.

I therefore used the coefficients and Rao-Cramer standard errors from the continuous MNL model as Bayesian priors:

Design

;alts = (treatA, treatB)
? unlabelled design with two generic alternatives (treatment A and treatment B)
;rows = 24
;block = 2
? Each respondent completes 12 tasks in one of 2 blocks.
;eff = (mnl,d,mean)
? minimise the mean D-error for the multinomial logit model across Bayesian prior draws
;alg = mfederov
? Design generation algorithm: Modified Fedorov algorithm (row-based), selecting from 2000 random choice tasks (default)
;bdraws = gauss(3)
? Draws for Bayesian priors: Gaussian quadrature with 3 abscissas
;reject:
treatA.TST = treatB.TST
AND treatA.FALLS = treatB.FALLS
AND treatA.SED = treatB.SED
AND treatA.EV = treatB.EV
AND treatA.COST = treatB.COST

;model:
? Bayesian D-efficient fractional factorial design under a main-effects multinomial logit (MNL) specification
? Bayesian priors (n, mean, SD): n=normally distributed; mean, pilot-only MNL coefficients; SD = 1.0 x SE.
? Coding: Continuous TST (total sleep time), FALLS and COST.
? Coding: SED=daytime sedation: 0=Uncommon, 1=Common;
? Coding: EV=certainty of evidence: 0=Low, 1=Moderate, 2=High.

U(treatA) = b_tst[(n, 0.04356633, 0.01263462)] * TST[10, 20, 40]
+ b_falls[(n, -0.2292554, 0.04290887)] * FALLS[0,11,17]
+ b_sed.dummy[(n, -1.0790061, 0.30433279)] * SED[1,0]
+ b_ev.dummy[(n, 0.928756, 0.440479297)|(n, 1.089891, 0.46989917)] * EV[1,2,0]
+ b_cost[(n, -0.021118391, 0.00965448)] * COST[10,35,60]
/

U(treatB) = b_tst * TST
+ b_falls * FALLS
+ b_sed * SED
+ b_ev * EV
+ b_cost * COST
$

The resulting design has a Bayesian mean D-error of approximately 0.01623 and a Bayesian mean S-estimate of 23.413547, corresponding to a minimum sample size of 47.

My main concern is the level distribution for the continuous attributes. In particular, across the 48 alternative profiles:

TST 10/20/40 minutes appears approximately 20 / 4 / 24 times.
Cost AUD 10/35/60 appears 22 / 1 / 25 times.
Thus, the middle cost level ($35) appears only once, and the middle TST level is also used relatively infrequently. I understand that this can occur in D-efficient designs when attributes are treated as continuous. However, I am concerned that these middle levels are almost absent from the actual questionnaire.

Could you please advise on the preferred approach in this situation?

1. Should I retain the continuous specification and accept the unequal level frequencies, given that the primary analysis assumes linear effects for TST, falls, and cost?
2. Would it be reasonable to generate the design with TST, falls, and cost dummy coded, using the coefficients and SEs from my categorical pilot sensitivity model as Bayesian priors, with the aim of improving representation of all three levels, while still estimating these attributes as continuous in the primary analysis?
3. If I use that approach, should I then evaluate the resulting 24-task design under my original continuous Bayesian MNL specification and compare its Bayesian D-error and S-estimate with those of the current design? If so, how similar would these efficiency measures need to be for it to be reasonable to prefer the alternative design on the basis of better level representation?
4. Are there any other approaches you would recommend for addressing the very limited representation of the middle levels?

A related feature of the current design is that FALLS has the same level across the two alternatives in 6 of the 24 tasks. From your previous forum responses, I understand that some overlap can be desirable when an attribute is relatively dominant (Falls), so I have not imposed a non-overlap constraint. Is my understanding appropriate in this situation?

My main concern is therefore the very limited representation of the middle TST and cost levels.

Thank you very much for your time and advice.
Joyce


I previously sought advice on this issue and would like to document/clarify the recommendations here. The advice is as follows:

1. The modified Federov algorithm causes the issue. If you would reformulate your ;reject constraint into a ;cond constraint, you can use the swapping algorithm, which produces attribute level balanced designs. Alternatively, you can impose attribute level frequency constraints for the numerical attributes, e.g. TST[10,20,40](6-10,6-10,6-10), where each attribute level appears between 6 and 10 times, noting that (8,8,8) would mean perfect level balance.
2. Yes, that is acceptable, and something that I often do.
3. You can if you want. You can formulate two models in Ngene at the same time. You optimise for the model with dummy coding, but you can evaluate directly for the model with linear coding.

Thanks for Michiel's previous response.

One other point that came up during the qualitative pre-testing was that some participants found the combination of uncommon sedation and a higher risk of falls unexpected. However, we would still like to retain both attributes, as they are important to the study. Now that we have pilot data (n-14), do you have any suggestions on how best to address this?

During the pilot, 2 of 14 participants consistently chose the alternative with the lower FALLS risk in every choice task, regardless of the levels of the other attributes. This appears to reflect non-trading on FALLS. Would this raise concern that the FALLS attribute is too dominant in the design, or is some individual-level non-trading acceptable in a DCE? If so, would you recommend modifying the design/levels for the main DCE, or addressing this mainly at the analysis stage?

Thanks in advance.
Joyce