4.3.3 Sensitivity and uncertainty analysis

Icône de l'outil pédagogique Authors

Frank Ewert and Martin van Ittersum


We consider that effective application of policy-relevant impact assessments can only be achieved, if uncertainty analysis effectively responds to information needs of model users and stakeholders of the problem (Gabbert, 2008). Accordingly, a concept is developed and applied to determine uncertainty for two aims (van Ittersum et al., 2008): 1) informing users of the model (or the model chain) about critical model assumptions and parameters and their uncertainties, and 2) providing guidance to the model developers in improving the model or model chain. Uncertainty of parameters and specific model assumptions can be tested within the present design of creating different experiments with different parameters and model assumptions.

The prime interest for (policy) users of SEAMLESS-IF lies in the assessment of new policies or agro-technological developments, i.e. how effective and (cost-)efficient are new policies and technologies in realising specified policy aims. The question to be addressed is whether it is likely that the indicator values of the policy scenarios are better than those in the baseline scenario and if so, how much. We hypothesize that the relative difference between the baseline and the policy scenario will be more important than a perfect assessment of the absolute outcomes of a baseline or policy scenario. We therefore propose to focus the uncertainty analysis on the ranking of scenario outcomes.

The critical interactions between the model components of SEAMLESS-IF backbone models are the following: Starting from CAPRI, the key input obtained from EXPAMOD are the price elasticities (price-supply relationships) for the various NUTS2 regions. Key inputs from FSSIM to EXPAMOD are agricultural supply (and eventually externalities) because of prices (and policies, farm structure and objectives). FSSIM uses agricultural activities which have been quantified in FSSIM-AM, using APES for assessing yields and externalities such as water use, nitrogen emissions and pesticide leaching. Only one feedback mechanism is anticipated, i.e. between CAPRI and FSSIM where prices simulated by CAPRI are an input to FSSIM models. In all other instances, models will not be used in an iterative scheme with the other models in the chain. This suggests that one could treat the uncertainty in for instance APES independently of the uncertainty in the other models.

We propose a four-step procedure for uncertainty analysis in SEAMLESS-IF to be illustrated with Test Case 1 (G20 trade liberalization proposal):

Icône de l'outil pédagogique Phase 1: Identification of model users’ uncertainty information needs
As a first step, a questionnaire has been distributed to users. The results obtained from phase 1 will give insight to uncertainty preferences of model-users. However, given the complexity of SEAMLESS-IF, different types and sources of uncertainties have to be taken into account. In order to evaluate which particular types and sources have been addressed by model users, uncertainty information needs identified in phase 1 will be sorted according to the scheme suggested by Walker et al. (2003).

Icône de l'outil pédagogique Phase 2: Identification of key uncertainties
In common situations of limited resources, for a given “budget” (in terms of money, research time, etc.) maximal “output” (in terms of information about uncertainties in IAMs) should be achieved. Thus, of the uncertainties, which are most relevant to model-users, we have to filter-out those which are in fact of main impact on model output. These are called “key uncertainties”. It should be stressed that key uncertainties do not necessarily be quantifiable uncertainties (such as model parameters) only. Uncertainties in model structure or data quality, for example, can also turn out to be of major impact on model output and therefore be denoted as key. For identifying key uncertainties, different approaches have been suggested and applied within recent years (Rypdal and Flugsrud 2001, ApSimon et al. 2002, Gabbert 2006). Depending on the outcomes of phase 1 it will be decided which of the approaches available seems to be most reasonable. This facilitates the choice and application of appropriate methods for uncertainty analysis within SEAMLESS-IF.

Icône de l'outil pédagogique Phase 3: Uncertainty analysis in components of SEAMLESS-IF
Key uncertainties will be investigated at component level through classical forms of sensitivity and uncertainty analysis. However, in the final version of SEAMLESS-IF delivered at the end of the project this will not be supported within the framework. Still, sensitivity and uncertainty of individual components should be tested outside the framework by the component developer. 

Icône de l'outil pédagogique Phase 4: Uncertainty analysis in the model chain of SEAMLESS-IF
In a final step, the accumulation of uncertainties throughout the model chain is investigated. SEAMLESS-IF (final version at the end of the project) will support this analysis through the design of experiments (scenarios). This will allow some evaluation of the uncertainty associated to different parameters (of one or more models) with respect to the final results for a specific model chain. 

Icône de l'outil pédagogique References and further reading
  1. ApSimon, H.M., Warren, R.F., Kayin, S., 2002. Addressing uncertainty in environmental modelling: a case study of integrated assessment of strategies to combat long-range transboundary air pollution. Atmospheric Environment 36, 5417-5426.
  2. Gabbert, S., 2006. Improving efficiency of uncertainty analysis in complex Integrated Assessment models: The case of the RAINS emission module.” Environmental Monitoring and Assessment 119 (1-3), 507-526.
  3. Gabbert, S., 2008. Towards a user-oriented uncertainty analysis in model-based decision-support. Environmental Monitoring and Assessment. In review.
  4. Rypdal, K., Flugsrud, K., 2001. Sensitivity analysis as a tool for systematic reductions in greenhouse gas inventory uncertainties. Environmental Science & Policy 4, 117-135.
  5. van Ittersum, M.K., Ewert, F., Heckelei, T., Wery, J., Alkan Olsson, J., Andersen, E., Bezlepkina, I., Brouwer, F., Donatelli, M., Flichman, G., Olsson, L., Rizzoli, A.E., van der Wal, T., Wien, J.E., Wolf, J., 2008. Integrated assessment of agricultural systems - A component-based framework for the European Union (SEAMLESS). Agricultural Systems 96, 150-165.
  6. Walker, W.E., Harremoes, P., Rotmans, J., Van der Sluijs, J.P., Van Asselt, M.B.A., Janssen, P., Krayer Von Krauss, M.P., 2003. Defining uncertainty. A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment 4, 5-17.

Régi par la licence Creative Commons Attribution Non-commercial Share Alike 3.0 License

copyright Seamless 2009