How do you allow for variability in design and quality for the studies gathered in the Visible Learning book? How can we be sure the results are robust?

Others have investigated this issue and with one major exception quality of study is just not a big enough variable to make a difference to the conclusions. The difference is when the effect size is very small (<0.10) then quality of the studies becomes more important. In the second book Visible Learning for Teachers (2012) some of the low-quality meta-studies have been discarded. Again, the effects of low and high quality were investigated and in general across the meta-analysis level (certainly not necessarily at the individual study level) it is rare to find quality making a difference (except when the effect is low).