How does expectancy-value-cost motivation vary during a semester? An intensive longitudinal study to explore individual and situational sources of variation in statistics motivation

Claudia C. Sutter, Delaram A. Totonchi, Jamie DeCoster, Kenneth E. Barron & Chris S. Hulleman (2024).

We measured expectancy, value, and cost 10 times over a 10-week introductory statistics course (N = 219) to examine their overall trajectory as well as individual (between-student) differences and situational (within-student) variability. First, our findings revealed an initial decline in expectancy and value and an initial increase in cost. Second, expectancy, utility value, and cost demonstrated individual and situational variability of comparable size, while intrinsic value had higher individual variability. Third, individual and situational variability in expectancy and value predicted variability in performance. Lastly, the relation of situational variability in expectancy and utility value with performance was stronger for Black, Latinx, and other racially marginalized students than for White/Asian students. Our findings provide empirical evidence for the situational nature of motivational beliefs and have implications for practitioners, course curriculum designers, and policymakers who aim to create more supportive and motivation-enhancing environments, particularly for statistics courses and students from racially marginalized and underserved backgrounds. Download a copy.

Memorization and Performance During Pandemic Remote Instruction - Evidence of Shifts from an Interactive Textbook

Jose L. Salas, Xinran (Wendy) Wang, Mary C. Tucker, & Ji. Y. Son (2024).

Students believe mathematics is best learned by memorization; however, endorsing memorization as a study strategy is associated with a decrease in learning (Schoenfeld, 1989). When the world changed with the onset of the COVID-19 global pandemic, instruction transitioned to fully remote instruction where many assignments and examinations became open textbook, open note, and even open Internet. In this new world, did students change their beliefs about the role of memorization in learning? Did academic performance change? And did the relationship between memorization beliefs and academic performance change? The current study takes advantage of data collected in an online interactive statistics textbook used by courses before (in-person) and after (remote) the declaration of the COVID-19 pandemic at three institutions, each representing a part of the California Master Plan for Higher Education (e.g., University of California, California State University, and California Community Colleges). Results from 2668 students who used the textbook showed that the UC institution had lower memorization belief scores compared to both the CSU and CCC institutions. Even when controlling for institution and chapter of the textbook, lower memorization belief scores were related to higher performance. Surprisingly, there were no significant differences in either memorization beliefs nor performance before and after transitioning to online remote instruction due to the pandemic. Although much of educational research is conducted in one institution, this kind of research can identify differences across institutional contexts to understand how learning can be affected by different teaching formats, including in-person and online/distance, brought on by disruptive social changes such as a global pandemic. Download a copy.

The “Better Book” Approach to Addressing Equity in Statistics : Centering the Motivational Experiences of Students from Racially Marginalized Backgrounds for Widespread Benefit

Claudia C. Sutter, Matthew C. Jackson, Karen B. Givvin, James W. Stigler & Ji Y. Son (2024).

Although improving racial equity in critical college courses such as introductory statistics is a laudable goal, making research-based progress toward that aim in a scalable manner remains a chal- lenge. To translate psychological insights to benefit racially marginalized students, we implemented the “Better Book” approach, where instructors, researchers, and developers work together to improve an online textbook used in introductory statistics. The “Better Book” approach to equity assumes that racially marginalized students are a “canary in the coal mine”, alerting us to systemic issues that can affect a broader array of students. We started by finding places in the textbook where racially marginalized students reported higher perceptions of costs (the effort and time required to learn the content) than non-marginalized students. Then we drew upon suggestions from users to redesign the textbook where gaps in cost perceptions peaked. We then analyzed data from both the original and redesigned versions of the textbook to evaluate the impact on students who were subsequently enrolled in the course. Results showed that perceptions of cost were dramatically reduced in the experience of racially marginalized students but also the redesign resulted in an improved experience for all students. Download a copy.

Watching videos of a drawing hand improves students’ understanding of the normal probability distribution

Icy (Yunyi) Zhang, Xiaohan Hanna Guo, Ji Y. Son, Idan A. Blank & James W. Stigler (2024).

Understanding normal probability distributions is a crucial objective in mathematics and statistics education. Drawing upon cognitive psychology research, this study explores the use of drawings and visualizations as effective scaffolds to enhance students' comprehension in a sequence of three studies. Download a copy.

Embodying Similarity and Difference - The Effect of Listing and Contrasting Gestures During U.S. Political Speech

Icy (Yunyi) Zhang, Tina Izad, Erica A. Cartmill (2024).

Study 1 examined two rhetorical devices—contrasts and lists—used by three politicians during U.S. presidential debates and asked whether the gestures produced during contrasts and lists differ. Gestures produced during contrasts were more likely to involve changes in hand location, and gestures produced during lists were more likely to involve changes in trajectory. Study 2 used footage from the same debates in an experiment to ask whether gesture influenced the way people perceived the politicians’ arguments. Download a copy.

Prediction versus Production for Teaching Computer Programming

Mary C. Tucker, Xinran (Wendy) Wang, Ji Y. Son, & James W. Stigler (2024).

Background: Most students struggle when learning to program. Aims: In this paper we examine two instructional tasks that can be used to introduce programming: tell-and- practice (the typical pedagogical routine of describing some code or function then having students write code to practice what they have learned) and prediction (where students are given code and asked to make predictions about the output before they are told how the code works). Sample: Participants were 121 college students with no coding experience. Methods: Participants were randomly assigned to one of two parallel training tasks: predict, or tell-and-practice. Results: Participants in the predict condition showed greater learning and better non-cognitive outcomes than those in the tell-and-practice condition. Conclusions: These findings raise a number of questions about the relationship between programming tasks and students’ experiences and outcomes in the early stages of learning programming. They also suggest some pedagogical changes to consider, especially in early introductions to programming. Download a copy.

Latine Students’ Motivational and Emotional Experiences Related to Their Introductory Statistics Course - Differences by Institution Type Necessitate Tailored Interventions

Claudia C. Sutter, Karen B. Givvin, Paige L. Solomon, & Ana Leandro-Ramos (2024).

The present study developed a representation-mapping intervention designed to help students interpret, coordinate, and eventually translate across multiple representations. We integrated the intervention into an online textbook being used in a college course, allowing us to study its impact in a real course over an extended period of time. The findings of this study support the efficacy of the representation-mapping intervention for facilitating learning and shed light on how to implement and refine such interventions in authentic learning contexts. Download a copy.

Concerns and Challenges in Introductory Statistics and Correlates with Motivation and Interest

Claudia C. Sutter, Karen B. Givvin, & Chris S. Hulleman (2023).

We explore how students’ course concerns at the outset of their introductory statistics course predict their later perceived course challenges and future interest in statistics via a function of achievement motivation. Data were collected from undergraduate students (N1=4524; 70% female; 37.8% students from racially marginalized groups) during the COVID-19 pandemic, using both open-ended (concerns and challenges) and closed-ended (achievement motivation and future interest) questions. Overall, incoming course concerns positively predicted perceived costs during the course and challenges at the end of the course and negatively predicted success expectancy and utility value during the course and future interest in statistics at the end of the course. Patterns varied by individual concerns/challenges, gender, and race/ethnicity. Cost played an important mediating role for female students and students from racially marginalized groups (e.g., Black, Latinx, or Native American/Indigenous students) between course concerns and future interest in statistics. Our findings (a) add to the increasing body of research reporting differences in how female and male students as well as students from racially marginalized backgrounds and racial majority students experience STEM courses and help explain different levels of interest in pursuing STEM careers, and (b) suggest that increasing future interest in statistics might require different interventions. Download a copy.

Representational-Mapping Strategies Improve Learning From an Online Statistics Textbook

Icy (Yunyi) Zhang, Maureen E. Gray, Alicia (Xiaoxuan) Cheng, Ji Y. Son, & James W. Stigler (2023).

The present study developed a representation-mapping intervention designed to help students interpret, coordinate, and eventually translate across multiple representations. We integrated the intervention into an online textbook being used in a college course, allowing us to study its impact in a real course over an extended period of time. The findings of this study support the efficacy of the representation-mapping intervention for facilitating learning and shed light on how to implement and refine such interventions in authentic learning contexts. Download a copy.

Listening for Common Ground in High School and Early Collegiate Mathematics

Gail Burrill, Henry Cohn, Yvonne Lai, Dev P. Sinha, Ji Y. Son, & Katherine F. Stevenson (2023).

Solutions to pressing and complex social challenges require that we reach for common ground. Only through cooperation among people with a broad range of backgrounds and expertise can progress be made on issues as challenging as improving student success in mathematics. In this spirit, the AMS Committee on Education held a forum in May 2022 entitled The Evolving Curriculum in High School and Early Undergraduate Mathematical Sciences Education. This article is a report on that forum by the authors listed above, who were among the organizers and presenters. Download a copy.

Don't force a false choice between algebra and data science

Ji Y. Son & James W. Stigler (2022).

Download a copy.

The Better Book Approach for Education Research and Development

James W. Stigler, Ji Y. Son, Karen B. Givvin, Adam Blake, Laura Fries, Stacy T. Shaw, & Mary C. Tucker (2020).

This paper describes a new approach for education research and development - the better book approach - and reports on our initial development and application of the approach in the context of introductory college-level statistics. Download a copy.

Lower Socioeconomic Status is Related to Poorer Emotional Well-Being Prior to Academic Exams

Danny Rahal, Stacy T. Shaw, James W. Stigler (2023).

People of lower social status tend to have greater emotional responses to stress. This study assessed whether lower social status was related to greater emotional responses in anticipation of a naturalistic stressor: academic exams among college students. As hypothesized, multilevel models (ratings nested within participants) predicting emotion indicated that students with lower mother’s education had less positive emotion, more depressive emotion, and more anxious emotion the day prior to academic exams than students with higher mother’s education (proportional reductions in variance [PRV] = .013–.020). Specifically, lower mother’s education was associated with poorer well-being before but not after the exam. Exploratory models revealed that differences in emotion by mother’s education were strongest for students with lower exam scores (PRV = .030–.040). Download a copy.

Teaching Statistics and Data Analysis with R

Mary C. Tucker, Stacy T. Shaw, Ji Y. Son & James W. Stigler (2022).

In this exploratory study, we characterize the attitudes and experiences of 672 undergraduate students as they used our online textbook as part of a 10-week introductory course in statistics. Students expressed negative attitudes and concerns related to R at the beginning of the course, but most developed more positive attitudes after engaging with course materials, regardless of demographic characteristics or prior programming experience. Analysis of a subgroup of students revealed that change in attitudes toward Rmay be linked to students’ patterns of engagement over time and students’ perceptions of the learning environment. Download a copy.

Student Concerns and Perceived Challengesin Introductory Statistics, How the Frequency Shifted during COVID-19, and How They Differ bySubgroups of Students

Claudia C. Sutter, Karen B. Givvin, Mary C. Tucker, Kathryn A. Givvin, Ana Leandro-Ramos & Paige L. Solomon (2022).

We explored how the frequency of concerns changed with the onset of COVID-19(N=1417) and, during COVID-19, how incoming concerns compared to later perceived challenges(n=524).Students were most concerned about R coding, understanding concepts, workload, prior knowledge,time management, and performance, with each of these concerns mentioned less frequently during thanbefore COVID-19. Concerns most directly related to the pandemic—virtual learning and inaccessibility of resources—showed an increase in frequency. The frequency of concerns differed by gender and URM status.The most frequently mentioned challenges were course workload, virtual learning, R coding, and under-standing concepts, with significant differences by URM status. Concerns about understanding concepts,lack of prior knowledge, performance, and time management declined from the beginning to the end of theterm. Workload had the highest rate of both consistency and emergence across the term. Download a copy.

Watching a Hands-on Activity Improves Students’ Understanding of Randomness

Icy (Yunyi) Zhang, Mary Tucker, James W. Stigler (2022).

Given the growing interest in using statistical programming languages like R as pedagogical tools, the findings of this study provide important and encouraging insights into the use of hands-on demonstrations to complement computer simulation in remote teaching. It validates the importance of giving students some hands-on exposure to the simulation processes prior to the compu- tational simulation we want them to understand and makes it clear that at least some of the benefits of embodied activities can be retained even if students are not performing the hands-on activities themselves. Download a copy.

Reasoning Affordances with Tables and Bar Charts

Cindy Xiong, Elsie Lee-Robbins, Icy Zhang, Aimen Gaba, Steven Franconeri (2022).

We tested whether confirmation bias exists when people reason with visualized data and whether certain visualization designs can elicit less biased reasoning strategies. We asked crowdworkers to solve reasoning problems that had the potential to evoke both poor reasoning strategies and confirmation bias. Presenting the data in a table format helped participants reason with the correct ratio strategy while showing the data as a bar table or a bar chart led participants towards incorrect heuristics. Confirmation bias was not significantly present when beliefs were primed, but it was present when beliefs were pre-existing. Additionally, the table presentation format was more likely to afford the ratio reasoning strategy, and the use of ratio strategy was more likely to lead to the correct answer. Download a copy.

Effect of Feedback with Video-based Peer Modeling on Learning and Self-efficacy

Wadi Eghterafi, Mary C. Tucker, Icy (Yunyi) Zhang, Ji Y. Son (2022).

In this study, we examined the effect of video-based feedback designed to highlight a peer engaging in effective thinking processes on self-efficacy beliefs and learning outcomes (performance on a delayed quiz). Results indicated that students who watched the mastery videos, but not the coping videos, rated their self-efficacy higher and scored higher on a class quiz taken more than a day after the feedback intervention than students who viewed a worked example. Download a copy.

Modeling First Applying Learning Science to the Teaching of Introductory Statistics

Ji Y. Son, Adam Blake, Laura Fries, James W. Stigler (2021).

In this article we describe our attempt to apply theories and findings from learning science to the design of a statistics course that aims to help students build a coherent and interconnected representation of the domain. The resulting practicing connections approach provides students with repeated opportunities to practice connections between core concepts (especially the concepts of statistical model, distribution, and randomness), key representations (R programming language and computational techniques such as simulation and bootstrapping), and real-world situations statisticians face as they explore variation, model variation, and evaluate and compare statistical models. We provide a guided tour through our curriculum implemented in an interactive online textbook (CourseKata.org) and then provide some evidence that students who complete the course are able to transfer what they have learned to the learning of new statistical techniques. Download a copy.

"Utility Value Trajectories and their Relationship with Behavioral Engagement and Performance in Introductory Statistics"

Claudia C. Sutter, Chris S. Hulleman, Karen B. Givvin, Mary Tucker (2021).

This study examined utility value trajectories overall and by gender, race, and underrepresented racial minority (URM) status within an introductory statistics course and tested the relationships between utility value, behavioral engagement, and performance. Download a copy.

"Instructed Hand Movements Affect Students’ Learning of an Abstract Concept from Video"

Icy (Yunyi) Zhang, Karen B. Givvin, Jeffrey M. Sipple, Ji Y. Son, James W. Stigler (2021).

The two studies reported here investigate the impact of instructed hand movements on students’ subsequent understanding of a concept. Students were asked to watch an instructional video—focused on the concept of statistical model—three times. These two studies found that instructed hand movement—even when presented as an unrelated, secondary task—can affect students’ learning of a complex concept. This article was featured in Scientific American. Download a copy.

"Practicing Connections:A Framework to Guide Instructional Design for Developing Understanding in Complex Domains"

Laura Fries, Ji Y. Son, Karen B. Givvin, James W. Stigler (2020).

Research suggests that expert understanding is characterized by coherent mental representations featuring a high level of connectedness. This paper advances the idea that educators can facilitate this level of understanding in students through the practicing connections framework: a practical framework to guide instructional design for developing deep understanding and transferable knowledge in complex academic domains. Download a copy.

Removing Opportunities to Calculate Improves Students’ Performance on Subsequent Word Problems

Karen B. Givvin, Veronika Moroz, William Loftus, & James W. Stigler (2019).

This paper reports our investigation on whether removing opportunities to calculate could improve students’ subsequent ability to solve similar word problems Download a copy.

Exploring the Practicing-connections Hypothesis:Using Gesture to Support Coordination of Ideas in Understanding a Complex Statistical Concept"

Ji Y. Son , Priscilla Ramos , Melissa DeWolf , William Loftus and James W. Stigler (2018).

This paper presented a framework and approach for studying how students come to understand complex concepts in rich domains. Specifically, it explores the role that a teacher’s gesture might play in supporting students’ coordination of two concepts central to understanding in the domain of statistics: mean and standard deviation. Download a copy.

Expertise and Expert Performance in Teaching

James W. Stigler and Kevin F. Miller (2018).

This chapter tries to take a broader approach to understanding the nature and development of expertise and expert performance in teaching. The paper also tries to integrate a number of ideas and findings from literatures as diverse as cross-cultural compar- isons of teaching, cognitive psychology, and systems improvement, among others. Download a copy.

DOES VAM + MET = IMPROVED TEACHING?

James W. Stigler, James Hiebert, and Karen B. Givvin (2018).

The paper discusses the logic of the more traditional approach on which many current policies for improving teaching in the United States are based and then presents an alternative research approach, in which a different theory of improvement is assumed. Download a copy.

Online Learning as a Wind Tunnel for Improving Teaching

James W. Stigler, Karen B. Givvin (2017).

The chapter proposes an approach that combines the affordances of online learning with the methodologies of systems improvement. It discusses how online learning might be a wind tunnel for the study and improvement of teaching. Download a copy.

"What Community College Developmental Mathematics Students Understand about Mathematics, Part2:The Interviews"

Karen B. Givvin, James W. Stigler, and Belinda J. Thompson (2011).

Following the prior paper, this article presents findings from one-on-one interviews with a sample of community college developmental math students. These interviews were designed to further probe students’ mathematical thinking, both correct and incorrect. Download a copy.

What Community College Developmental Mathematics Students Understand about Mathematics

James W. Stigler, Karen B. Givvin, and Belinda J. Thompson (2010).

This paper investigates what community college students actually understand about the mathematics that underlie the topics they’ve been taught and seeks evidence that students used reasoning in answering mathematical questions. Download a copy.

"Improving Social and Conceptual Connections during Remote Statistics Classes [conference paper]"

Caylor R. Davis, Karen Givvin, Jinna Hwang, and Ji Y. Son (2021).

This paper reports on the implementation of a statistics lesson in a remote classroom that incorporates an embodied group activity as a way to improve perceptions of social connection and to enrich conceptual connections during learning. Download a copy.

How Much Students Value an Introductory Statistics Course, How Value Levels Change across the Term, and How They Predict Learning [conference paper]

Claudia C. Sutter, Mary Tucker, Karen Givvin, and Chris S. Hulleman (2021).

This paper examines changes in utility value in an introductory statistics course overall, by sex and underrepresented racial minority (URM) and tested the relationship between utility value and learning. Download a copy.