I tried to find what percentage of the data used in calculating the Visible Learning list effect size for feedback was from elementary, secondary, etc. Is that something you calculate and track?

Full Question: A question of apples and oranges: because the data has been taken from so many sources at diverse levels and conditions, some individuals question the usefulness of the effect sizes. Once when I was preparing to facilitate some professional development on descriptive feedback I tried to find what percentage of the data used in calculating the Visible Learning list effect size for feedback was from elementary, secondary, etc. Is that something you calculate and track?

Answer: “The major point of using effect size is to help reduce the apples and oranges problem, by considering these as ‘fruit.’ Of course, there needs to be care which is why in every meta-analysis care is given to investigating the moderators to any conclusions about ‘fruit.’ And dissimilar studies – that is what is coded in the various meta-analysis (some well-done others not so well done). For example, it is possible to code for whether elementary, secondary – it is a moderator and then ask do the results effect size differ – and they do not across these levels of schooling. That is the beauty of effect size and meta-analysis – you get empirical answers to these questions. Yes, this is lost when you then go to the league table, but I tried to be careful in the Visible Learning book to make these points about checking for moderators” (John Hattie, personal communication, February 14, 2017).