Heterogenous pivot design around multiple ref alts.

nayeem · Post by **nayeem** » Thu May 22, 2025 4:18 am

I am designing a labelled choice experiment to estimate WTP for more reliable freight transport along a particular corridor. There are three labelled alternatives:

Normal Road Transport
Normal Intermodal Transport
High speed Intermodal Transport

The different attributes are:

Transport Cost
Transport time (origin to destination)
Travel time reliability (mean-variance method)
Frequency (for intermodal transport only)
Safety (for road transport only)-categorical variable

My questions are:
1. In the Ngene manual (last updated on Feb 24, 2025), script 9.6 shows an efficient design pivoted around multiple sets of reference alternatives but where the reference alternative is shown in each case. I want a pivot design optimized around multiple sets of reference levels with the assumption that reference alternative is not shown in each design. How can this be done in Ngene?
2. For this type of experimental design (pivoted around multiple sets of reference alternatives), are there any challenges involved in estimating a single model based on choice data coming from difference classes of respondents?
3. Since I am using 3 labelled alternatives, I am already using 2 ASC constants in my utility equations. Do I need to specify another constant to estimate order effects if I am changing the order of my alternatives in the survey instrument?

Michiel Bliemer · Post by **Michiel Bliemer** » Thu May 22, 2025 9:18 pm

1. This is shown in Script 9.5, see also the last paragraph on page 148 where this is discussed. Alternatively, you can simply generate a design with absolute levels (around average attribute values) as usual, and then convert these into percentages in Excel.

2. This is similar to RP data where also distance classes etc are different per respondent, but in the end one pools all the data to estimate a model. You may want to estimate a heteroskedastic choice model that accounts for differences in scale (or error variance), e.g. based on travel distance.

3. You cannot simply add another constant as it would make the model unidentified. You need to randomise the order of the alternatives across respondents, e.g. some respondents see Normal Road Transport on the left and others see it on the right. In your data set, you add a variable that indicates the position of each alternative (i.e., position 1, 2 or 3). You convert this into a dummy coded variable and add it to the utility function of each labelled alternative (which means estimating two extra coefficients). Since the position varies across respondents, this way you can disentangle the order effect from the label-specific constants. So you need to randomise the order of the alternatives in the survey instrument, this is the only way to do it, as otherwise the two effects are confounded.

Michiel

nayeem · Post by **nayeem** » Fri May 23, 2025 12:15 am

Dear Michiel,

This was the original code:

Code: Select all

design   ? mode choice of shippers without reference alternative  
;alts(withref)=ref, truck, train, etrain
;alts(withoutref)=truck, train, etrain
;rows=24
;block=2
;eff= withoutref(mnl,d)
;con
;model(withref): ?auxiliary model with reference alternative
U(ref)=    brefcost[-0.000000001]     *  cost.ref[150]                                  
        +  breftime[-0.000000001]     *  time.ref[10]                                   
        +  brefttr[-0.000000001]      *  ttrzero.ref[0.8]                               
        /
U(truck)=  b1[-0.000000001]           *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2[-0.000000001]           *  time.piv[-2,-1,0,1,2]                          
        +  b3[-0.000000001]           *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy[-0.000000001]     *  risk[1,0]                                      
U(train)=  con_train                                                                    
        +  b5[-0.000000001]           *  cost.piv[ -15%,-10%,-5%]                       
        +  b6[-0.000000001]           *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11[0.000000001]           *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7[-0.000000001]           *  cost.piv[5%,10%,15%]                           
        +  b8[-0.000000001]           *  time.piv[-2,-1.5,-1]                           
        +  b9[-0.000000001]           *  ttr2[0.8,1.47]                                 
        +  b10[0.000000001]           *  frequency[2,3,5]                               

;model(withoutref): ?model without reference alternative
U(truck)=  b1                         *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2                         *  time.piv[-2,-1,0,1,2]                          
        +  b3                         *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy                   *  risk[1,0]                                      
        /
U(train)=  con_train                                                                    
        +  b5                         *  cost.piv[-15%,-10%,-5%]                        
        +  b6                         *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11                        *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7                         *  cost.piv[5%,10%,15%]                           
        +  b8                         *  time.piv[-2,-1.5,-1]                           
        +  b9                         *  ttr2[0.8,1.47]                                 
        +  b10                        *  frequency[2,3,5]                               
$

I tried to add another model using a different reference alternative with an aim to generate a heterogeneous pivot design.

Code: Select all

design   ? mode choice of shippers without reference alternative 

?First model
;alts(withref1)=ref1, truck, train, etrain
;alts(withoutref1)=truck, train, etrain


?Second model
;alts(withref2)=ref2, truck, train, etrain
;alts(withoutref2)=truck, train, etrain


;rows=8
;block=1
;eff= alldistances(mnl,d)
;fisher(alldistances)=des1(withoutref1[0.5])+ des2(withoutref2[0.5])
;con
? Model when cost=$150 and time=10 hours
;model(withref1): ?auxiliary model with reference alternative1
U(ref1)=   brefcost[-0.000000001]     *  cost.ref1[150]                                  
        +  breftime[-0.000000001]     *  time.ref1[10]                                   
        +  brefttr[-0.000000001]      *  ttrzero.ref[0.8]                               
        /
U(truck)=  b1[-0.000000001]           *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2[-0.000000001]           *  time.piv[-2,-1,0,1,2]                          
        +  b3[-0.000000001]           *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy[-0.000000001]     *  risk[1,0]                                       
        /
U(train)=  con_train                                                                    
        +  b5[-0.000000001]           *  cost.piv[ -15%,-10%,-5%]                       
        +  b6[-0.000000001]           *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11[0.000000001]           *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7[-0.000000001]           *  cost.piv[5%,10%,15%]                           
        +  b8[-0.000000001]           *  time.piv[-2,-1.5,-1]                           
        +  b9[-0.000000001]           *  ttr2[0.8,1.47]                                 
        +  b10[0.000000001]           *  frequency[2,3,5]                               

;model(withoutref1): ?model without reference alternative1
U(truck)=  b1                         *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2                         *  time.piv[-2,-1,0,1,2]                          
        +  b3                         *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy                   *  risk[1,0]                                      
        /
U(train)=  con_train                                                                    
        +  b5                         *  cost.piv[-15%,-10%,-5%]                        
        +  b6                         *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11                        *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7                         *  cost.piv[5%,10%,15%]                           
        +  b8                         *  time.piv[-2,-1.5,-1]                           
        +  b9                         *  ttr2[0.8,1.47]                                 
        +  b10                        *  frequency[2,3,5]                               




? Model when cost=$200 and time=10 hours
;model(withref2): ?auxiliary model with reference alternative2
U(ref2)=   brefcost[-0.000000001]     *  cost.ref2[180]                                  
        +  breftime[-0.000000001]     *  time.ref2[10]                                   
        +  brefttr[-0.000000001]      *  ttrzero.ref[0.8]                               
        /
U(truck)=  b1[-0.000000001]           *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2[-0.000000001]           *  time.piv[-2,-1,0,1,2]                          
        +  b3[-0.000000001]           *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy[-0.000000001]     *  risk[1,0]                                      
        /
U(train)=  con_train                                                                    
        +  b5[-0.000000001]           *  cost.piv[ -15%,-10%,-5%]                       
        +  b6[-0.000000001]           *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11[0.000000001]           *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7[-0.000000001]           *  cost.piv[5%,10%,15%]                           
        +  b8[-0.000000001]           *  time.piv[-2,-1.5,-1]                           
        +  b9[-0.000000001]           *  ttr2[0.8,1.47]                                 
        +  b10[0.000000001]           *  frequency[2,3,5]                               

;model(withoutref2): ?model without reference alternative2
U(truck)=  b1                         *  cost.piv[-10%, -5%,0%,5%,10%]                  
        +  b2                         *  time.piv[-2,-1,0,1,2]                          
        +  b3                         *  ttr1[0.8,1.47,1.71,1.96,2.4]                   
        +  b4.dummy                   *  risk[1,0]                                       
        /
U(train)=  con_train                                                                    
        +  b5                         *  cost.piv[-15%,-10%,-5%]                        
        +  b6                         *  time.piv[1,2,3,4]                              
        +  b3                         *  ttr1                                           
        +  b11                        *  frequency2[5,6,7]
        /
U(etrain)= con_etrain                                                                   
        +  b7                         *  cost.piv[5%,10%,15%]                           
        +  b8                         *  time.piv[-2,-1.5,-1]                           
        +  b9                         *  ttr2[0.8,1.47]                                 
        +  b10                        *  frequency[2,3,5]                               

$

But this does not work.

Michiel Bliemer · Post by **Michiel Bliemer** » Fri May 23, 2025 1:54 am

Ngene does not support optimising pivot designs for different reference levels simultaneously. You would need to generate them separately.

Noting that you have uninformative priors, the reference levels play no role in the optimisation, so you could just generate a homogeneous design using design coding as explained in Section 9.1 (of the Ngene Online manual) and apply to all reference levels. You can relabel the design coding with pivot levels. With (near) zero priors, the actual levels do not matter when generating an efficient design.

Michiel

nayeem · Post by **nayeem** » Sat May 24, 2025 5:27 pm

Dear Michiel,
The example given in section 9.1 is related to unlabeled alternatives. My design has labelled alternatives and all attributes are not generic. But still I tried to follow the code given in script 9.1:

Code: Select all

design   ? mode choice of shippers with HOMOGENOUS DESIGN CODING    
;alts=truck, train, etrain
;rows=24
;block=3
;eff=(mnl,d)
;con
;model: ?model using DESIGN CODING
U(truck)=  b1.dummy                *  COST1[0,1,2,3,4]                               
        +  b2.dummy                *  TIME1[0,1,2,3,4]                               
        +  b3.dummy                *  TTR1[0,1,2]                                    
        +  b4.dummy                *  risk[1,0]                                      
        /
U(train)=  con_train                                                                 
        +  b5.dummy                *  COST2[0,1,2]                                   
        +  b6.dummy                *  TIME2[0,1,2,3]                                 
        +  b3.dummy                *  TTR1                                           
        +  b7.dummy                *  frequency[0,1,2]
        /
U(etrain)= con_etrain                                                                
        +  b5.dummy                *  COST2                                          
        +  b8.dummy                *  TIME3[0,1,2]                                   
        +  b9.dummy                *  TTR2[0,1]                                      
        +  b7.dummy                *  frequency                                      
$

The error message shown is: An attribute has the wrong number of levels for dummy or effects coding. 'cost1'

I have also noticed that when I try to indicate the sign of the uninformed priors in my design as in script 9.1:

Code: Select all

design   ? mode choice of shippers with HOMOGENOUS DESIGN CODING    
;alts=truck, train, etrain
;rows=24
;block=3
;eff=(mnl,d)
;con
;model: ?model using DESIGN CODING
U(truck)=  b1.dummy[-]             *  COST1[0,1,2,3,4]                               
        +  b2.dummy                *  TIME1[0,1,2,3,4]                               
        +  b3.dummy                *  TTR1[0,1,2]                                    
        +  b4.dummy                *  risk[1,0]                                      
        /
U(train)=  con_train                                                                 
        +  b5.dummy                *  COST2[0,1,2]                                   
        +  b6.dummy                *  TIME2[0,1,2,3]                                 
        +  b3.dummy                *  TTR1                                           
        +  b7.dummy                *  frequency[0,1,2]
        /
U(etrain)= con_etrain                                                                
        +  b5.dummy                *  COST2                                          
        +  b8.dummy                *  TIME3[0,1,2]                                   
        +  b9.dummy                *  TTR2[0,1]                                      
        +  b7.dummy                *  frequency                                      
$

I get the error message: The 'model' property contains a prior that has an incorrectly configured Bayesian component. '-'

I am using Desktop version of Ngene 1.4.0 and Build 24011. Even the script 9.1 is not running in Ngene.

Code: Select all

design
;alts = pt1, pt2
;rows = 12
;eff = (mnl,d)
;model: 
U(pt1, pt2) = time.dummy[-]  * TIME[0,1,2]
            + wait.dummy[-]  * WAIT[0,1,2]
            + trans.dummy[-] * TRANSFERS[0,1]
            + cost.dummy[-]  * FARE[0,1,2]
$

The error message shown is: A model in the utility expressions was not specified in the ';alts' property. ''

Michiel Bliemer · Post by **Michiel Bliemer** » Tue Jun 03, 2025 1:43 am

If you are using Ngene version 1.4 you need to refer to the manual for Ngene Destop (Windows) version 1.4. You are now reading the manual for Ngene Online, which is the latest release (version 2) and has several new features. You can find both versions of the manual on the ChoiceMetrics website: https://www.choice-metrics.com/pricinganddownloads
Direct link to the version 1.4 manual: https://files.choice-metrics.com/NgeneManual140.pdf

Below I have changed the script to run on version 1.4.

Code: Select all

design   ? mode choice of shippers with HOMOGENOUS DESIGN CODING   
;alts=truck, train, etrain
;rows=24
;block=3
;eff=(mnl,d)
;con
;model: ?model using DESIGN CODING
U(truck)=  b1.dummy[0|0|0|0]       *  COST1[0,1,2,3,4]                               
        +  b2.dummy[0|0|0|0]       *  TIME1[0,1,2,3,4]                               
        +  b3.dummy[0|0]           *  TTR1[0,1,2]                                   
        +  b4.dummy[0]             *  risk[1,0]                                     
        /
U(train)=  con_train                                                                 
        +  b5.dummy[0|0]           *  COST2[0,1,2]                                   
        +  b6.dummy[0|0|0]         *  TIME2[0,1,2,3]                                 
        +  b3.dummy                *  TTR1                                           
        +  b7.dummy[0|0]           *  frequency[0,1,2]
        /
U(etrain)= con_etrain                                                               
        +  b5.dummy                *  COST2                                         
        +  b8.dummy[0|0]           *  TIME3[0,1,2]                                   
        +  b9.dummy[0]             *  TTR2[0,1]                                     
        +  b7.dummy                *  frequency                                     
$

The plus and minus does not work in version 1.4, you will need to define the preference order manually, see below. Note that indicating the preference order of attribute levels is only useful for unlabelled alternatives where dominance checks are needed, they are not useful for labelled alternatives as in your case because there is no issue with strict dominance.

U(truck)= b1.dummy[0.004|0.003|0.002|0.001] * COST1[0,1,2,3,4]

Regarding your last script, you need to specify:

U(pt1) = ... /
U(pt2) = ...

The shortcut U(pt1,pt2) only works in Ngene Online.

Michiel

nayeem · Post by **nayeem** » Sun Jun 08, 2025 4:27 pm

Thanks for pointing out the differences between Ngene Online and Ngene Desktop.
I am using the following design with real reference levels:

Code: Select all

design   ? mode choice of shippers with Real data  SurveyEngine (FINAL)
;alts(low)=truck, train, etrain
;alts(high)=truck, train, etrain
;rows=36
;block=3
;eff = alldistances(mnl,d)
;fisher(alldistances) = des1(low[0.5],high[0.5])
;con
;model(low): ? COST=$150, TIME=10 Hours
U(truck)=  b1[-0.000001]                *  COST1[135,145,150,160,165]                               
        +  b2[-0.000001]                *  TIME1[8,9,10,11,12]                               
        +  b3[-0.000001]                *  TTR1[1.71,1.96,2.4]                                    
        +  b4.dummy[-0.000001]          *  risk[1,0]                                      
        /
U(train)=  con_train                                                                 
        +  b5[-0.000001]                *  COST2[120,135,145]                                   
        +  b6[-0.000001]                *  TIME2[10,11,12,13]                                 
        +  b3                           *  TTR1                                           
        +  b7[0.000001]                 *  frequency1[5,6,7]
        /
U(etrain)= con_etrain                                                                
        +  b8[-0.000001]                *  COST3[160,165,180]                                          
        +  b9[-0.000001]                *  TIME3[8,8.5,9]                                   
        +  b10[-0.000001]               *  TTR2[0.8,1.47]                                      
        +  b11[0.000001]                *  frequency2[2,3,5]

;model(high): ? COST=$200, TIME=10 Hours
U(truck)=  b1[-0.000001]                *  COST1[180,190,200,210,220]                               
        +  b2[-0.000001]                *  TIME1[8,9,10,11,12]                               
        +  b3[-0.000001]                *  TTR1[1.71,1.96,2.4]                                    
        +  b4.dummy[-0.000001]          *  risk[1,0]                                      
        /
U(train)=  con_train                                                                 
        +  b5[-0.000001]                *  COST2[160,180,190]                                   
        +  b6[-0.000001]                *  TIME2[10,11,12,13]                                 
        +  b3                           *  TTR1                                           
        +  b7[0.000001]                 *  frequency1[5,6,7]
        /
U(etrain)= con_etrain                                                                
        +  b8[-0.000001]                *  COST3[210,220,230]                                          
        +  b9[-0.000001]                *  TIME3[8,8.5,9]                                   
        +  b10[-0.000001]               *  TTR2[0.8,1.47]                                      
        +  b11[0.000001]                *  frequency2[2,3,5]

The D-error is 0.070232 and A-error is 17.15102.
When I use design coding which you have fixed for Ngene Desktop since real reference levels without actual prior values plays no role in optmization:

Code: Select all

design   ? mode choice of shippers with HOMOGENOUS DESIGN CODING   
;alts=truck, train, etrain
;rows=36
;block=3
;eff=(mnl,d)
;con
;model: ?model using DESIGN CODING
U(truck)=  b1.dummy[0|0|0|0]       *  COST1[0,1,2,3,4]                               
        +  b2.dummy[0|0|0|0]       *  TIME1[0,1,2,3,4]                               
        +  b3.dummy[0|0]           *  TTR1[0,1,2]                                   
        +  b4.dummy[0]             *  risk[1,0]                                     
        /
U(train)=  con_train                                                                 
        +  b5.dummy[0|0]           *  COST2[0,1,2]                                   
        +  b6.dummy[0|0|0]         *  TIME2[0,1,2,3]                                 
        +  b3.dummy                *  TTR1                                           
        +  b7.dummy[0|0]           *  frequency[0,1,2]
        /
U(etrain)= con_etrain                                                               
        +  b5.dummy                *  COST2                                         
        +  b8.dummy[0|0]           *  TIME3[0,1,2]                                   
        +  b9.dummy[0]             *  TTR2[0,1]                                     
        +  b7.dummy                *  frequency                                     
$

The D-error is 0.543151 and A-error is 0.972021.
My question concerns the value of A-error in the first design. Is this indicative of any problems in the design since the A-error is not close to zero?

Michiel Bliemer · Post by **Michiel Bliemer** » Sun Jun 08, 2025 9:21 pm

No, A-errors can be large, especially when adding ;con because the alternative-specific constants may have large variances (you can see this when you inspect the covariance matrix). In your case, the variances of the constants are much smaller when using dummy coding.

I would recommend always looking at the D-error and not the A-error, which is also very sensitive to units of the attributes (i.e., a design that is A-efficient when using cost in dollars and time in hours may not be efficient when using cost in cents and time in minutes).

Michiel

nayeem · Post by **nayeem** » Fri Jun 20, 2025 12:00 am

My aim is to develop an MNL model which will help me estimate priors for my final model. Initially, I used real reference levels to generate two models: one in which cost of alternatives are pivoted around $150 and another in which cost of alternatives is pivoted around $200, the values of the rest of the attributes being same in the two models. Now that I know that real references levels make no difference to efficient designs without uninformative priors I want to use design coding. In this case should I use:
1.Two different experimental designs for the two models (in which only the cost attribute is different) OR
2.A single experimental design will suffice for both models since all the other attributes except cost are essentially same?
In either case does the data need to be combined in an H-MNL model?

Michiel Bliemer · Post by **Michiel Bliemer** » Sat Jun 21, 2025 5:13 pm

You can choose to use a homogeneous design or a heterogeneous design. A homogeneous design (using the same underlying experimental design for all respondents) is sometimes preferred if you want to compare different population segments and if you find any differences you can argue it is due to differences in preferences instead of differences in experimental design. If the main objective is not about comparing population segments then you can use different experimental designs for respondents, which is often a bit more efficient. You can still compare different population segments, but the statistical power would be somewhat lower because it takes a larger sample to detect whether differences are due to different population segments or different designs.

When you combine data sources you can always try to estimate a heteroskedastic model, that contains at least a scale parameter.

Michiel

choice-metrics.com

Heterogenous pivot design around multiple ref alts.

Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.

Re: Heterogenous pivot design around multiple ref alts.