Robust ranking of universities evaluated by hierarchical and interacting criteria

In this chapter, we present a methodology of decision aiding that helps to build a ranking of a finite set of alternatives evaluated by a family of hierarchically structured criteria. The presentation has a tutorial character, and takes as an example the ranking of universities. Each university is generally evaluated on several aspects, such as Quality of Faculty and Research Output. Moreover, their performance on these macrocriteria can be further detailed by evaluation on some subcriteria. To take into account the hierarchical structure of criteria presented as a tree, the Multiple Criteria Hierarchy Process will be applied. The aggregation of the university performances will be done by the Choquet integral preference model that is able to take into account the possible negative and positive interactions between the criteria at hand. On the base of an indirect preference information supplied by the Decision Maker in terms of pairwise comparisons of some universities, or comparison of some criteria in terms of their importance and their interaction, the Robust Ordinal Regression and the Stochastic Multicriteria Acceptability Analysis will be used. They will provide the Decision Maker some robust recommendations presented in the form of necessary and possible preference relations between universities, and in the form of a distribution of possible rank positions got by each of them, taking into account all preference models compatible with the available preference information. The methodology will be presented step by step on a sample of some European universities.