How to Read Peer-reviewed Articles on Experiments in Learning 

Reading peer-reviewed social science research can be quite challenging. But it’s a skill that lets you go beyond media reports of the latest study. Here’s 15 minutes of advice on how to do read experimental studies specifically.
0:00 An intro to reading social science research
0:39 What do learning researchers study?
1:54 The basics of experimental designs
5:01 What’s the goal of reading research?
5:16 The first big question: why?
6:58 The second big question: what happened?
8:12 What’s the participant’s experience?
9:07 A tip for reading complex graphs.
10:23 The third big question: the explanation.
11:04 Some complexities when measuring learning
13:45 What’s the question again?
14:45 Last thoughts
If you want to be the first to know about the courses I'm releasing, sign up here: forms.gle/px7ZmXkvJW26uFWp8
Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.com
REFERENCES
Clips come from the following pieces:
Example of a longitudinal study: Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363.
Example of qualitative data and mixed qualitative and quantitative data come from: Reading Abraham Lincoln: An Expert/Expert Study in the Interpretation of Historical Texts. (2003). 1-28.
Example of quantitative data comes from: Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113-1124. doi.org/10.1037/a0023549
Example of abstract/laboratory experiments comes from: Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of negative recency and related phenomena.
Example of an applied study (using social studies material in the classroom): Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382-395. doi.org/10.1037/a0026252
Examples of an experimental study, data visualization that describes the nature of the materials and one that describes the course of the experiment from: Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129-184.
Research studying the effects of delayed vs immediate feedback are legion. This provides good background reading: Fyfe, E. R., & Rittle-Johnson, B. (2016). Feedback both helps and hinders learning: The causal role of prior knowledge. Journal of Educational Psychology, 108(1), 82-97. doi.org/10.1037/edu0000053
The example of worked examples vs open problem solving comes from some of Sweller’s earlier experiments. Here’s a typical example. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.
The statistics game and passage examples I drew from Arena, D. A., & Schwartz, D. L. (2013). Experience and Explanation: Using Videogames to Prepare Students for Formal Instruction in Statistics. Journal of Science Education and Technology, 23(4), 538-548. doi.org/10.1007/s10956-013-94...
The screenshot of a statistics passage is from one of the first pages of Regression and Other Stories by Gelman, Hill, and Vehtari, a great textbook on regression.
The design-based research example comes from Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi‐user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56-68. They created a virtual environment called “River City” to facilitate science inquiry experiences.
The “delayed measures” being different than immediate measures graph is from Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24(6), 837-848. doi.org/10.1002/acp.1598
The “Stats Invaders” clip (the statistics game) comes courtesy of Can Zhang, here: • Education Game Stats I...