If we are talking about effect sizes based on student achievement, and if student achievement is measured through the flawed tools of standardized tests, then how accurate are these effect sizes?

Indeed, I raised this critical objection myself in Chapter 11 of Visible Learning (2009). I have continued to be concerned about the grammar of surface learning that has spread like a virus across our schools—The pursuit of the content, the facts, and the desire to know lots. On the one hand I welcome this as knowing is the precursor to understanding, creativity, and wisdom, but this other part – the deeper thinking, the relating and extending of ideas, the bringing together of two or more seemingly unrelated ideas (Koestler’s definition of creativity, which I like) is also desired – and based on knowing. As we become more skilled the two, surface and deep, tend to merge, but the point is both, not either or is needed. My observation is that over 90 percent of what we ask students to do in assignments and on assessments can be completed successfully by knowing lots – and this is not defensible.

Worse, many above average students applaud this grammar of surface learning as they know how to play this game; want teachers to focus more on the content, and want teachers to talk more. But maybe we need to shut up and listen to how students are processing, relating and exploring ideas; teaching them to adopt and use different strategies of learning, and move more from surface to deep to transfer and back again – more often. With 300 million+ students even the 10 percent focused on deep allows me to make claims and generalise my work. Our recent work (Hattie & Donoghue, 2016) explore this distinction much more deeply and included in this work is the observation that even most of our teaching methods are dominated by focusing on EITHER content or relating, whereas we need teaching methods to be smarter about WHEN to be surface (learn knowledge) and when to be deep (relate knowledge). I also note that not all but most effects in Visible Learning are based on standardized surface level tests – it includes many teacher-made, performance, oral and so many varieties (not that the type of measure seems to matter as much as the narrowness of width of the outcome being measured)” (John Hattie, personal communication, February 27, 2019).