Outcomes-focused evaluation of engineering programs - Inhibitor or stimulant for improvement?

Paper in proceeding, 2013

In Sweden, a new model for quality evaluation of higher education is being introduced. It is internationally unique. A very strong emphasis on assessing the quality of an entire education through its outcomes, defined by samples of final degree reports, is perhaps the most original feature of the model. This is claimed to enable a direct assessment of quality rather than the indirect assessment used in conventional models. The model has, however, been intensively debated during its development and first years of use. The aims of this paper are first to evaluate in what ways and to what extent the Swedish national model for evaluation of higher education contributes to the quality improvement of the evaluated programs, and second to describe and evaluate an approach for organizing the project to prepare the evaluation documentation for a large-scale education evaluation. The paper concludes that on an overall and averaged level, the project participants estimated that the model will contribute to positive effects. Specific areas are identified including ethics and focus of final degree reports. However, there is a strong variance in the data: There is a substantial fraction of participants who have not identified any improvements or weak spots, and who do not think that the evaluation preparation will positively affect education quality development. Chalmers University of Technology chose to approach the evaluation with a centrally managed project. The setup gave possibilities for rationalization through common training, supporting materials, a common document database and for information sharing across departmental boundaries. Nevertheless, the project was very time-consuming with an average of 540 hours spent for each program/degree.