Analysis Result Becomes Undefined

[CMA]

When I run the program below, the MNL D-Error becomes "Undefined".

Could the reason be that there are impossible combinations between E and F within the reject conditions?

If that's the case, would it be necessary to revise the experimental design?

If this can be resolved by modifying the program, I would like to know how.

Thank you in advance.

Code: Select all

Design
;alts = alt1*, alt2*
;rows = 18
;eff = (mnl, d)
;alg = mfederov(stop = total(1000000 iterations))
;reject:
?alt1
alt1.E = 0  and alt1.F =< 2,
alt1.E =< 1 and alt1.F <= 0,
?alt2
alt2.E = 0  and alt2.F =< 2,
alt2.E =< 1 and alt2.F <= 0
;model:
U(alt1) = b1.dummy[0|0]     * A[1,2,0]
        + b2.dummy[0|0]     * B[1,2,0]
        + b3.dummy[0|0]     * C[1,2,0]
        + b4.dummy[0|0]     * D[1,2,0]
        + b5.dummy[0|0|0]   * E[1,2,3,0]
        + b6.dummy[0|0|0|0] * F[1,2,3,4,0]
        + b7.dummy[0|0]     * G[1,2,0]
        /
U(alt2) = b1                * A
        + b2                * B
        + b3                * C
        + b4                * D
        + b5                * E
        + b6                * F
        + b7                * G
$

[Michiel Bliemer]

I could not run the script because you wrote =< instead of <=. When I change to the constraints below the script runs fine and generates a design with a finite D-error.

?alt1
alt1.E = 0 and alt1.F <= 2,
alt1.E <= 1 and alt1.F <= 0,
?alt2
alt2.E = 0 and alt2.F <= 2,
alt2.E <= 1 and alt2.F <= 0

Michiel

[CMA]

Thank you for your prompt reply.

The reject conditions I mentioned earlier were different from the experimental design.

I've corrected them to match the experimental design, but the result is still "Undefined".

I would be grateful for your assistance.

Thank you.

Code: Select all

Design
;alts = alt1*, alt2*
;rows = 18
;eff = (mnl, d)
;alg = mfederov(stop = total(1000000 iterations))
;reject:
?alt1
alt1.E =  0 and alt1.F >= 2,
alt1.E >= 1 and alt1.F <= 1,
?alt2
alt2.E =  0 and alt2.F >= 2,
alt2.E >= 1 and alt2.F <= 1
;model:
U(alt1) = b1.dummy[0|0]     * A[1,2,0]
        + b2.dummy[0|0]     * B[1,2,0]
        + b3.dummy[0|0]     * C[1,2,0]
        + b4.dummy[0|0]     * D[1,2,0]
        + b5.dummy[0|0|0]   * E[1,2,3,0]
        + b6.dummy[0|0|0|0] * F[1,2,3,4,0]
        + b7.dummy[0|0]     * G[1,2,0]
        /
U(alt2) = b1                * A
        + b2                * B
        + b3                * C
        + b4                * D
        + b5                * E
        + b6                * F
        + b7                * G
$

[Michiel Bliemer]

The covariance matrix indicates issues with b5 and b6 (extremely large values), belonging to attributes E and F. So yes, your constraints seem to be the issue, your model parameters cannot be estimated when these constraints are applied. The model can be estimated if you only apply one of these constraints, but not both. So you will need to think about model identification when you specify your constraints.

Michiel

[CMA]

It's disappointing that the model cannot be estimated with the current experimental design.

Is there a way to conduct a DCE with this experimental design?

I would like to explore using a D-optimal design as much as possible.

Also, on the Ngene HP (https://www.choice-metrics.com/features), does the limitation of choice sets and attributes shown to respondents refer to 'reject'?

[Michiel Bliemer]

Note that this is not a limitation of Ngene; the model cannot be estimated WITH ANY EXPERIMENTAL DESIGN because the current formulation of your utility functions combined with your constraints make the model unidentified.

You have linked the following levels:
E=0 with F=[0,1]
E=[1,2,3] with F=[2,3,4]

This creates two separate, non-overlapping, sets of levels, and hence the model cannot be estimated. The way around this is usually to reformulate your variables by grouping or splitting the variables. For example, split the variable into an attribute for F=[0,1] and a different attribute for F=[2,3,4]. It is almost always possible to redefine your variables to obtain an identified model. This is very case-specific and I cannot help you with this, but give it a try.

I am not exactly sure what sentence you are referring to on the homepage, but The section on advanced constraints refers mostly to ;cond, ;reject, and ;require. This sentence refers to partial profile and partial choice set designs, which are a type of constraints: " This method can also be used for other advanced designs in which the user wishes to limit the number of alternatives or attributes shown to a respondent in each choice task in order to reduce choice task complexity".

Michiel

[CMA]

Thank you for your response. I learned a lot.

I have an additional question.
The following literature has been published.
https://www.tandfonline.com/doi/full/10.1080/03007995.2022.2129800?rfr_dat=cr_pub++0pubmed&url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org#d1e350

The following constraints are described in the literature (Attributes and levels are listed in Table 1).
> To avoid improbable combinations of attributes, the experi-mental design assumed that the following levels would notcoincide within the same treatment: “brain surgery” with an“external device (300 g)” or an “external device (500 g)”;“gastric surgery” with an “internal device” and “no surgery”with an “internal device”.

In this configuration, the levels for "Surgery requirement" don't appear to overlap. Is this a feasible scenario?

[Michiel Bliemer]

I can answer Ngene questions but I am unable to comment on other peoples' studies as I do not have time to read papers and look into their modelling. Each study is different and requires the analyst to specify an identifiable model that can be estimated. The formulation of an identified model is different in each study, and may depend on coding structures, normalisations, interactions, and prohibitions/constraints. So to be clear, I am talking about the model formulation here (utility functions), not the experimental design. Choice model identifiability is one of the most difficult aspects of choice models and when imposing constraints it requires expertise of the choice modeller to ensure that the model can be estimated after imposing constraints on attribute levels. As I said, in most cases you can account for it by changing the variables in the choice model, or by creating interaction terms.

If the D-error of a design is Undefined, it means it is infinite and the choice model you specified is not identified and you need to specify different utility functions (or adjust your constraints). But HOW to change the utility functions is case specific and I cannot tell.
From your prohibitions, it looks like you have two "groups", namely E=0 and E>0. You could define this in a new variable, say X, where X=0 if E=0 and X=1 if E>1. If X=0, then you have attribute F1=[0,1]. If X=1, then you have attribute F2=[2,3,4]. You could consider making interactions between X and F1, F2. So perhaps think about groupings of your variables to see if you can disentangle them to formulate an identifiable model.

Michiel