Process

Experiment design includes three different phases: define the goals of the experiment; identify and classify independent and dependent variables; choose an experiment design.

The second step in the experiment design process is to identify quantities in the simulation that can be set to desired values (independent variables, e.g. input variables defining the policy options to be tested in SEAMLES-IF impact assessments) and the resulting system performance measures that are of interest (dependent variables), which are the selected (policy impact) indicators. Another class of variables to be considered when designing the experiment are variables which are known to affect the behaviour of the system. They are affected by the settings of the independent variables, but they are not considered dependent variables. They are often called “nuisance variables or outlook parameters”.

It is necessary to identify all variables of all three types before planning the set of runs. Dependent variables are determined by the objectives of the study. All independent variables should be identified, not just the ones that will be varied in the experiment. Nuisance variables must be monitored so that variation in the experiment results can be understood. In order to be able to reproduce the results later, the (fixed) values of any independent variables that were not adjusted must be recorded, as well as the values of ones that were varied. Independent and nuisance variables whose values are actually changed during the experiment will be called factors. Process diagrams and cause-effect diagrams can be used to identify them (see Barton, 1999).

*In this activity, one determines the number
of distinct model settings to be run and the specific values of the factors for
each of these runs. There are many strategies for selecting the number of runs
and the factor settings for each run. Four types of experiment matrices are
available, including the Hadamard matrix, Rechtschaffner matrix, fractional
factorial matrix and the complete (full) factorial matrix.
*

In Table 1, an example of a fractional factorial design is presented. Factorial designs allow the modeller to analyse multi-level parameters influences and their dependencies, which is not possible in Hadamard and Rechtschaffner matrices. Factorial designs are based on a grid, with each factor tested in combination with every level of every other factor. Factorial designs are attractive for three reasons: 1) the number of levels that are required for each factor are one greater than the highest-order power of that variable in the model, and the resulting design permits the estimation of coefficients for all cross-product terms; 2) they are probably the most commonly used class of designs, and 3) the resulting set of run conditions are easy to visualize graphically for as many as nine factors.

Table 1 shows how resolution 3-factor fractional factorial designs can be constructed for a large number factor screening experiments (factor L = 6, resolution m = 3) where only main effects exist. The factorial design is based on the first 3 factors, A, B and C. The rest of the main effects (D, E and F) are assigned to the last three columns of AB, AC and BC respectively so their main effects are confounded with corresponding interaction effects. The level settings of main effects confounded with interaction effects at each design point are set to be the same as the level settings of the corresponding interaction effects. Thus, we have a design matrix and all main effects can be estimated.

Table 1. Resolution III Fractional Factorial Design.

The disadvantage of full factorial designs
is that they require a large number of distinct runs. If a design includes all
treatment combinations (with two levels per factor L), it is a full factorial
design; if only a fraction of those is included, it is a fractional factorial
design. Controlled sequential fractional-factorial designs (CSFD) have the
flexibility to stop the experiment early, to increase the model size when
further investigation is necessary, and to classify any desired main effect or
interaction effect. For detailed description of factorial design and analysis,
please refer to

Régi par la licence Creative Commons Attribution Non-commercial Share Alike 3.0 License

copyright Seamless 2009