A recent Academic Anonymous post in The Guardian about how student surveys are affecting a young professor’s confidence got me thinking.
I use the term “smile scores” for surveys that ask students how much they enjoyed a piece of instruction – as a reminder of what such surveys do and don’t measure. For example, student evaluations of teaching – a typical measure of students’ perceptions of a course – can be unrelated or even negatively correlated with the effectiveness of instruction. See two studies:
Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors and Student evaluations of teaching (mostly) do not measure teaching effectiveness.
So, when you want a valid way to measure the effectiveness of instruction, don’t use smile scores and don’t just ask students for their opinions on the matter. Instead, consider what students should be able to do by the end of the instruction, and measure
that – ideally by giving students direct performance tasks to complete before and after the instruction. This approach will tell you the instruction’s effectiveness in terms of students’ learning – with a measure you can count on. Now, that’s something to smile about.