The comprehensive evaluation of the performance of a system is a complex endeavor: many facets need to be considered in configuring an adequate and effective evaluation setting. A single evaluation method or measure is not able to evaluate all relevant aspects in a complex setting where a multitude of stakeholders are involved. Employing a multi-method evaluation, where multiple evaluation methods or measures are combined and integrated, allows for getting a richer picture and prevents blind spots in the evaluation outcome.
Together with Eva Zangerle, I maintain a website on multi-method evaluation where we collect resources that help to apply multi-method evaluation in research and development. Naturally, this collection is meant to grow over time and we are happy to share your resources and discuss best practices.
- Second workshop: perspectives on the evaluation of recommender systems (PERSPECTIVES 2022)
- Evaluating Recommender Systems: Survey and Framework
- Special Issue 'Perspectives on Recommender Systems Evaluation' @ TORS
- Co-Chair of PERSPECTIVES 2022
- Report on the 1st Workshop on the Perspectives on the Evaluation of Recommender Systems (PERSPECTIVES 2021) at RecSys 2021