CORE Webinars

When Measures Mislead: Proxies, Decoys, and Mirages

Abstract: We design and optimise systems that interact with humans, and the human world, through benchmarks and standardised measurements. But are these measurements truly reliable? There is a deep literature examining how best to make such measurements, but this literature has only rarely considered an underlying principle: that measured scores are inherently incomplete as a representation of human behaviour. This talk, drawing on arguments made in the field of economics, argues that neglect of this principle has consequences in practice – blind pursuit of performance gains based on optimisation of scores, and analysis based solely on aggregated measurements, can lead to misleading or even meaningless outcomes.

Justin Zobel is a Redmond Barry Distinguished Professor in the School of Computing and Information Systems at the University of Melbourne. Professor Zobel received his PhD from Melbourne in 1991 and, prior to returning to Melbourne in 2008, worked at RMIT University and NICTA. As a researcher, he is best known for development of algorithms underpinning web search. He is also known for work on measurement techniques, data structures and algorithms, bioinformatics, and research training. His contributions were recently recognised in Australia by election to the CORE Academy and internationally by being made a Fellow of the ACM.

Click here for the Presentation slides of the talk and the Session Recording

Tips for preparing an evidence-based teaching portfolio
To help those applying for awards, the chair of the Teaching awards committee, Associate Professor Claudia Szabo, presented a seminar with tips on preparing an evidence-based teaching portfolio.  The seminar was aimed at those applying for either of the CORE teaching awards, or preparing for promotion, job applications or any other teaching award.

Click here for the Presentation slides of the talk and the Session Recording