Hello,
I am currently designing a discrete choice experiment to assess farmers’ willingness to reduce antibiotic use. My design is unlabelled, with two alternatives (A and B). I am trying to design a pilot study that will be delivered to 20 farmers to generate priors for designing the full survey.
I have 5 attributes that were developed following focus group discussions.
Health: supplement feed with minerals / supplement feed with vitamin C / no supplementation
Prevention: vaccinate / do not vaccinate
Treatment: use antibiotics / do not use antibiotics
Management: disinfect material / reduce animal density / increase ventilation / no management
Cost: 100 / 200 / 400 / 700
When I use weak/non-informative Beyesian priors in ngene (roughly 0.01–0.2 to reflect genuine uncertainty), the resulting designs perform poorly (S-estimates: 3626; Sp estimates: 0.4-3442; sp t-ratios: 0.03-2). To obtain designs with acceptable performance (S-estimates = 2.4; Sp estimate: 0.6-2.0; sp t-ratios: 1.4-2.4), I need to impose much stronger priors (around 0.4–0.6), which feels difficult to justify given the lack of evidence.
My question is:
For a pilot DCE with very limited prior information, is it generally preferable to use weak/non-informative priors and accept poorer design efficiency in order to reflect true uncertainty, then estimate priors for the main study; or impose stronger assumed priors in order to obtain a more statistically efficient and robust pilot design?
I do not want to design a "poor" experiment and be unable to extract any useful information from the pilot's data to design the full survey. On the other hand, I don't want to use priors that are too strong, as I genuinely do not have any information regarding preferences.
Any advice or practical experience with similar situations would be greatly appreciated!!
Many thanks,
Sara
PS: I even tried to run a much simpler design that removed the management attribute (which has 4 categorical variables), but I have the same problems for non-informative priors.
PS2: Unfortunately, I cannot get any information to help refining the priors before the pilot survey.
Pilot DCE Design: Weak vs Strong Bayesian Priors?
Moderators: Andrew Collins, Michiel Bliemer, johnr
-
Michiel Bliemer
- Posts: 2081
- Joined: Tue Mar 31, 2009 4:13 pm
Re: Pilot DCE Design: Weak vs Strong Bayesian Priors?
I recommend that you read Chapter 1 in the manual, especially Section 1.6 about pilot studies and Section 1.7 on sample size estimates.
https://files.choice-metrics.com/NgeneManual.pdf
For a pilot study, you would typically use either an orthogonal design, or an efficient design with (near) zero local priors or Bayesian priors as you do. When using (near) zero priors, you should NOT look at the sample size estimates, because they will not make sense. With a zero prior, the sample size estimates become zero. Sample size estimates are only reliable once you have informative priors. So in your case, you can just do what you do, use conservative Bayesian priors, and generate an efficient design that minimises the Bayesian D-error.
Michiel
https://files.choice-metrics.com/NgeneManual.pdf
For a pilot study, you would typically use either an orthogonal design, or an efficient design with (near) zero local priors or Bayesian priors as you do. When using (near) zero priors, you should NOT look at the sample size estimates, because they will not make sense. With a zero prior, the sample size estimates become zero. Sample size estimates are only reliable once you have informative priors. So in your case, you can just do what you do, use conservative Bayesian priors, and generate an efficient design that minimises the Bayesian D-error.
Michiel