choice-metrics.com

Dear Prof. Bliemer,

I hope you are doing well.

I am working on a study that extends a previous study by introducing nudging into the choice tasks. I have access to prior parameter estimates from the earlier study. However, one of the six attributes was found to be statistically insignificant.

I would like to ask whether it would be methodologically appropriate to remove this attribute from the experimental design and still generate an efficient design using the modified Federov algorithm based on the remaining attributes and priors.

I would greatly appreciate your advice on whether excluding the insignificant attribute would be reasonable in this context, or whether it should still be retained for theoretical or design considerations.

Thank you very much.

Kind regards,
Rushdi

In most cases it will be fine to remove the insignificant attribute and use the remaining attributes and priors to generate an efficient design.

The only case where you need to be careful is when the coefficient of the removed attribute is quite large, despite being non-significant (i.e., it has a very large standard error). Then removing the attribute would influence the choice probabilities. But if the coefficient is such that the attribute has only a minor contribution to utility, then it should be fine.

Michiel

Thank you Prof. Bliemer.

I have another question regarding the priors. Do you recommend accounting for attribute non-attendance when generating priors for an efficient design? For example, would it be preferable to estimate an ECLC model on the pilot data, derive ANA-adjusted utility coefficients, and use those as design priors, or is it generally better to use the coefficients obtained directly from a standard logit model?

Best,
Rushdi

I would not recommend estimating such a complicated model to account for attribute non-attendance. Such a model has many parameters that with a limited pilot study sample will be far from statistical significance and therefore unreliable. Secondly, there exists no software that can optimise the design for such a model.

Therefore, I would strongly recommend that you optimise for an MNL model at the design stage.

Michiel

Thank you so much Prof. Bliemer

Best,
Rushdi

Dear Prof. Bliemer,

I have one remaining question regarding the same study on the number of blocks. If I can only collect data from 50 respondents after generating the efficient design, how many blocks should I select from the design?

Best,
Rushdi

I'm not sure I understand the question. The number of blocks in the design follows from the design size (total number of choice tasks) and the number of choice tasks you want to give to an individual respondent. It is beneficial to cover each choice tasks more or less evenly across the 50 respondents, so try to balance the number of observations per block.

You can also decide to forego blocking and simply randomly select choice tasks from the experimental design. This is often the default in survey software.

Michiel