Overall design concerns

[rushdi98]

Dear all,

I have written this efficient design script for my choice experiment, but since its my first time doing an experimental design, I would really appreciate it if someone could check it for me before I run it so that I avoid mistakes. I have read alot of posts on this forum which have been realy useful, and I'm really grateful for the help this community gives.

Code: Select all

design

;alts = car, put, cs

;rows = 30
;block = 3

;eff = (mnl,d,mean)

;con

;bdraws = sobol(2000)

;alg = mfedorov

;model:

U(car) =

    beta_car_accegr[(n,
    -0.012234929064086,
    0.00842010698569487)]
    * AccEgr_car[10, 15, 20]

  + beta_car_tt[-0.0252509890545476]
    * TT_car[45, 55, 65]

  + beta_car_search[(n,
    -0.00167278578948678,
    0.0100632621670443)]
    * Search_car[1, 5]

  + beta_car_tc[(n,
    -0.0431167293062408,
    0.0343356614865965)]
    * TC_car[3, 4, 5]

  + beta_car_pc[-0.0598716980956188]
    * PC_car[1, 5, 10]

/

U(put) =

    asc_put_d[(n,
    0.244648021102231,
    0.371983617164455)]

  + beta_put_accegr[-0.0175993317527122]
    * AccEgr_put[1, 5, 10]

  + beta_put_tt[-0.0192997948718738]
    * TT_put[55, 65, 75]

  + beta_put_tc[-0.0648116534705902]
    * TC_put[1, 3.5, 5.5]

/

U(cs) =

    asc_cs_d[-1.90766885586331]

  + beta_cs_accegr[(n,
    -0.0397663282829118,
    0.0204235424770084)]
    * AccEgr_cs[1, 5, 10]

  + beta_cs_tt[-0.0210844616143259]
    * TT_cs[45, 55, 65]

  + beta_cs_search[(n,
    -0.0100297329304847,
    0.02334225361749)]
    * Search_cs[1, 5]

  + beta_cs_tc[-0.0205571483243145]
    * TC_cs[9.5, 16, 24]

  + beta_cs_avail[(n,
    0.00001,
    0.00318118828169986)]
    * Avail_cs[10, 50, 100]

$

The Bayesian specification is applied to the statistically insignificant parameters which have the correct sign (except for beta_cs_avail which was insignificant and had the wrong theoretical sign, so I am using 0.00001 to indicate the expected direction). the parametes that were significant and had correct signs are specified as fixed point priors.

Is this correct please? especially that the mean of beta_cs_avail is really small (0.00001) in this case which is much smaller than its standard error?!

And also the number of sobol draws, is 2000 enough or I should change it? I am planning to let the design run for around 22 hours.

Another thing is that I am expecting to collect only around 50 samples (each respondent getting 10 choice tasks), is the number of rows and blocks sufficient in this case?

Sincerely,
Rushdi

[Michiel Bliemer]

That looks good to me. It is indeed a good idea to not use priors with unexpected sign, so setting them to (near) zero is fine. It is also totally fine to have a larger standard error/deviation value than the mean. But if you know for sure that availability should be positive, you could also consider a uniform distribution, something like (u,0,0.003). But that you result in moving the mean further away from zero, so what you propose is also fine.

2000 Sobol draws should suffice.

I would recommend that you do NOT use the modified Fedorov algorithm. The preferred algorithm is the default swapping algorithm, which ensures attribute level balance. The modified Fedorov would in your case likely rarely show the middle levels (because that is 'optimal' from an efficiency perspective). You can just remove the ;alg property.

It is best to post scripts in the Ngene forums because I am not sure if you are using the desktop version (v1) or the online version (v2). Some of the properties are slightly different. If using Ngene Online, 22 hours will not work (as runs will automatically be cut off after 10 hours) but it would likely be overkill anyway. The swapping algorithm is much faster than the modified Fedorov and you will likely see that after about 30 minutes the Bayesian D-error does not improve much anymore.

The number of rows is fine. And giving 10 choice tasks to each respondent seems reasonable. If you think you can give more (e.g. 12) then you could choose 36 rows and 3 blocks, which would increase the amount of data with 20%. If you think that 12 is too much, you could split it up in the survey and first give 6, then ask some other questions in between (e.g. age, gender, income, etc) and then another 6.

Michiel

[rushdi98]

Dear Prof. Michiel Bliemer,

Thank you so so much, this relieved my doubts.

Sincerely,
Rushdi

[Michiel Bliemer]

Perhaps good to realise that, according to these priors, option "cs" is only chosen by around 6% of the people.

[rushdi98]

Yes, it’s really undesirable, but it’s not necessarily a bad thing, as depending on the case studies in Munich, an increase in car-sharing demand means a reduction in public transport use by a much higher percentage than private car use. So, it’s basically taking demand away from public transport in Munich, which is contrary to other research (e.g., Shaheen and Chan (2016)).

I lost the case study paper. I will put it here when I find it .

Thank you indeed, Prof. Bliemer. Your efforts are appreciated

I have updated the script according to Prof. Bliemer’s recommendation, so that the other members can benefit from it as well:

Code: Select all

design

;alts = car, put, cs

;rows = 36
;block = 3

;eff = (mnl,d,mean)

;con

;bdraws = sobol(2000)

;model:

U(car) =

    beta_car_accegr[(n,
    -0.012234929064086,
    0.00842010698569487)]
    * AccEgr_car[10, 15, 20]

  + beta_car_tt[-0.0252509890545476]
    * TT_car[45, 55, 65]

  + beta_car_search[(n,
    -0.00167278578948678,
    0.0100632621670443)]
    * Search_car[1, 5]

  + beta_car_tc[(n,
    -0.0431167293062408,
    0.0343356614865965)]
    * TC_car[3, 4, 5]

  + beta_car_pc[-0.0598716980956188]
    * PC_car[1, 5, 10]

/

U(put) =

    asc_put_d[(n,
    0.244648021102231,
    0.371983617164455)]

  + beta_put_accegr[-0.0175993317527122]
    * AccEgr_put[1, 5, 10]

  + beta_put_tt[-0.0192997948718738]
    * TT_put[55, 65, 75]

  + beta_put_tc[-0.0648116534705902]
    * TC_put[1, 3.5, 5.5]

/

U(cs) =

    asc_cs_d[-1.90766885586331]

  + beta_cs_accegr[(n,
    -0.0397663282829118,
    0.0204235424770084)]
    * AccEgr_cs[1, 5, 10]

  + beta_cs_tt[-0.0210844616143259]
    * TT_cs[45, 55, 65]

  + beta_cs_search[(n,
    -0.0100297329304847,
    0.02334225361749)]
    * Search_cs[1, 5]

  + beta_cs_tc[-0.0205571483243145]
    * TC_cs[9.5, 16, 24]

  + beta_cs_avail[(u,
    0,
    0.003)]
    * Avail_cs[10, 50, 100]

$

Sincerely,
Rushdi