From a zero D-error to a positive D-error in one search

[OscarFrison]

Dear Ngene developers,

I am constructing a choice experiment to document consumer preferences for electricity tariff design. The design consists of multiple reject constraints to ensure realism. However, I am afraid that the (combination of) constraints negatively impacts the validity of my design. More specifically, when running a search, the initial performance is zero and only becomes positive (= 0.389) after a design evaluation that equals 1965. From this point onward, the performance of the design improves over time.

What explains the initial zero performance and what are the implications on the generated experimental design (and eventually parameter estimation)?

Please find my Ngene script below:

design
;alts = (alt1, alt2), sq
;rows = 36
;block = 6
;eff = (mnl, d)
;alg = mfedorov

;reject:
alt1.peak_reduc > 0 and alt1.capacity = 1, ? impossible to have peak reduction when there is no capacity component
alt2.peak_reduc > 0 and alt2.capacity = 1,

alt1.timeb = 1 and alt1.priceb > 1, ? impossible to have multiple price blocks when there is only one time block
alt2.timeb = 1 and alt2.priceb > 1,

alt1.priceb = 1 and alt1.timeb > 1, ? impossible to have multiple time blocks when there is only one price block
alt2.priceb = 1 and alt2.timeb > 1,

alt1.diff_priceb > 0 and alt1.priceb = 1, ? impossible to have a difference between price blocks when there is only one price block
alt2.diff_priceb > 0 and alt2.priceb = 1,

alt1.diff_priceb = 0 and alt1.priceb > 1, ? impossible to have zero difference between the price blocks when having more than one price block
alt2.diff_priceb = 0 and alt2.priceb > 1,

alt1.priceb = 2 and alt1.timeb = 6, ? impossible to have 2 price blocks and 6 time blocks
alt2.priceb = 2 and alt2.timeb = 6

alt1.priceb = 2 and alt1.priceb = 2, ? impossible to have the same amount of time and price blocks (except if both are equal to one)
alt1.priceb = 3 and alt1.timeb = 3,
alt1.priceb = 4 and alt1.timeb = 4,
alt1.priceb = 5 and alt1.timeb = 5,
alt1.priceb = 6 and alt1.timeb = 6,

alt1.diff_priceb = 0 and alt1.timeb > 1, ? impossible to have no difference between the price blocks when multiple time blocks
alt2.diff_priceb = 0 and alt2.timeb > 1,

alt1.season > 0 and alt1.timeb < 5, ? impossible to have two seasons and less than 5 time blocks
alt2.season > 0 and alt2.timeb < 5,

alt1.season > 0 and alt1.priceb < 3, ? impossible to have two seasons and less than 3 price blocks
alt2.season > 0 and alt2.priceb < 3

;model:

U(alt1, alt2) = b1[+].dummy *comp[10,20,30,40,50,0] ? € reduction fixed component distribution cost on bill
+ b2 *diff_priceb[0,10,20,30,40,50] ? percentage point difference between the price blocks (SQ is 100%)
+ b3 *priceb[1,2,3,4] ? no. of price blocks grid tariff
+ b4.dummy *timeb[3,5,6,1] ? no. of time blocks grid tariff
+ b5.dummy *season[1,2,3,0] ? composition of season structure grid tariff [1 = Dec-Feb & Mar-Nov, 2 = Winter & other, 3 = Autumn-Winter & Spring-Summer, 0 = 1 season]
+ b6.dummy *capacity[1,0] ? presence capacity component [1 = No, 0 = Yes]
+ b7.dummy *peak_reduc[25,50,75,0] ? percentage reduction on kW price during 13-17h time window
/
U(sq) = b_sq ? ASC for SQ
? level of compensation = 0 (base of dummy variable)
+ b2 *diff_priceb_sq[0] ? difference between the price blocks = 0 (base of dummy variable)
+ b3 *priceb_sq[1] ? no. of price blocks = 1
? no. of time blocks grid tariff = 1 (base of dummy variable)
? no. of seasons = 1 (base of dummy variable)
? capacity component = Yes (base of dummy variable)
? peak reduction component = No (base of dummy variable)
$

Many thanks in advance for your insights.

[Michiel Bliemer]

If you hover the mouse over the zero point in the search graph, you can see that the design actually has an Undefined D-error, which means Infinite. Since we cannot plot Infinite on the graph, it is plotted as zero, but it is actually not zero. If you click on it, you can see that the covariance matrix for that design has several extremely large values in them, meaning that some parameters cannot be estimated in the initial design. After some iterations, Ngene is able to find a design that allows estimation of all parameters (despite your constraints), as indicated by the finite D-error.

So the design seems to work well and Ngene is able to satisfy all your constraints while still keeping the model identifiable. You can just ignore the initial "zero" design, which is just there for you to inspect but you should not use that design. You can also see in the counter that there are a number of "Invalid designs", which are other designs with an Undefined/Infinite D-error.

Note that the D-error of a design will never be zero.

Michiel