Times are displayed in (UTC-04:00) Eastern Time (US & Canada) Change
(LC) PER: Assessment, Grading and Feedback III
7/13/2022 | 1:50 PM to 2:50 PM
Room: CC: Grand Gallery C
Moderator: / Co-Organizer:
Session Code: LC | Submitting Committee: Committee on Research in Physics Education / Co-Sponsoring Committee:
LC01 (01:50 to 02:00 PM) | Contributed | Measuring changes in student reasoning: Theoretical framework and methodology*
Presenting Author: Brianna Santangelo, North Dakota State University
Additional Author | Mila Kryjevskaia, North Dakota State University
Additional Author | Alexey Leontyev, North Dakota State University
| ,
| ,
One of the goals of physics instruction is to help students develop and refine their reasoning skills. However, measuring changes in reasoning is challenging since both content understanding, and appropriate reasoning strategies are necessary to arrive at a correct response. An instrument that would allow us to disentangle these factors (to the degree possible) would also allow us to measure changes along these two dimensions. In this study, we are drawing on the Dual-process theories of reasoning (DPToR) to develop an instrument based on the screening-target methodology: screening questions probe student conceptual understanding, and target questions require the application of that understanding in more complex situations. Comparison of performance on screening and target questions makes it possible to infer changes in conceptual understanding and reasoning skills due to instruction. We will show illustrative examples of student performance and discuss the implications for instrument development and implementation.
*This material is based upon work supported by the National Science Foundation under Grant Nos. FAR-0035257, DUE-1431940, DUE-1431541, DUE-1431857, DUE-1432052, DUE-1432765, DUE-1821390, DUE-1821123, DUE-1821400, DUE-1821511, DUE-1821561.
LC02 (02:00 to 02:10 PM) | Contributed | Impact of Response-Shift Bias on Students' Sense of Relevance
Presenting Author: Ivy Shaw, University of North Florida
Presenting Author | Brendan McEnroe, University of North Florida
Additional Author | Dr. W. Brian Lane, University of North Florida
| ,
| ,
Response-shift bias (RSB) describes how instructional interventions induce a downshift in students’ self-assessment of their pre-instructional capabilities and interests, such as measured by the Colorado Learning Attitudes about Science Survey (CLASS). This downshift produces a misalignment between the students' perception of their learning experience and pre-to-post self-assessment shifts, possibly explaining why positive CLASS shifts are difficult to obtain. To further explore this downshift, we added a reflective question to four items on the CLASS asking students to explain how they determined their responses. We examine correlations between themes in students’ reflections, finding that the students made more connections with their conceptual knowledge at the end of the semester than at the beginning. This change highlights the role of RSB in their survey outcomes. These trends offer insight into additional dimensions of the student learning experience and could help physics educators design more engaging courses.
LC03 (02:10 to 02:20 PM) | Contributed | Analyzing students' sensemaking with algebraic inequalities
Presenting Author: AMOGH SIRNOORKAR, Kansas State University
Additional Author | James T Laverty, Kansas State University
| ,
| ,
| ,
‘Doing physics' entails extensive use of mathematical expressions to contain and convey contextual information. Consequently, students' sensemaking on mathematical formalisms has attracted considerable attention in physics education research (PER). In the current work, we explore how introductory students construct meaning using algebraic inequalities while sensemaking on a physics problem. We qualitatively analyze a case study of a student (pseudonym: Matthew) through the frameworks of Sensemaking Epistemic Game and Symbolic Forms. The case study highlights Matthew’s verbal and written references to inequalities ‘driving’ explanation generation during sensemaking. A preliminary analysis of Matthew and other students' references to inequalities reveals a cluster of `comparing' symbolic forms conveying the idea of a quantitative comparison between physical quantities. Observations made in this study shed insights on how students construct and extract meaning through algebraic inequalities.
LC04 (02:20 to 02:30 PM) | Contributed | Interpreting Item Response Theory Results Using a Thermodynamic Analogy
Presenting Author: Trevor Smith, Rowan University
Additional Author | Nasrine Bendjilali, Rowan University
| ,
| ,
| ,
Several recent studies have employed item response theory (IRT) to rank incorrect responses to commonly used research-based multiple-choice assessments. These studies use Bock's nominal response model (NRM) for applying IRT to categorical data, but the response rankings only utilize half of the parameters estimated by the model. We present a mathematical argument for why this practice of using half of the NRM parameters when ranking responses is appropriate based on the primary question of multiple-choice tests: How can we use students' responses to test items to estimate their overall knowledge? We provide additional motivation for this practice by recognizing the similarities between Bock's NRM and the probability function of the canonical ensemble with degenerate energy states. As physicists often do, we exploit these mathematical similarities to gain new insights into the meaning of the IRT parameters and a richer understanding of the relationship between these parameters and student knowledge.
LC05 (02:30 to 02:40 PM) | Contributed | Development of a Likert-style instrument to assess LA’s PCK-Q
Presenting Author: Beth Thacker, Texas Tech University
Additional Author | Stephanie Hart, Texas Tech University
Additional Author | Kyle Wipfli, Texas Tech University
| Jianlan Wang, Texas Tech University
| ,
As part of a project to develop a written instrument for assessing learning assistants’ (LAs’) pedagogical content knowledge (PCK) in the context of questioning (PCK-Q), we are experimenting with questions in a Likert-style format. Previously, we have developed and validated questions in free-response format. We are now using those questions as the basis for Likert-style questions. Likert-style questions are different because they require the LAs’ to evaluate possible LA responses to students in classroom scenarios. They are also beneficial because scoring can be automated. The instrument will examine a LA’s ability to identify appropriate responses that provide evidence of the application of PCK-Q in the classroom. We will discuss problem development, present sample problems, and outline ongoing plans regarding validation, reliability testing, and dissemination efforts.
Funded by NSF IUSE grant 1838339
LC06 (02:40 to 02:50 PM) | Contributed | Affordances of Articulating Assessment Objectives
Presenting Author: Michael Vignal, University of Colorado Boulder
Additional Author | Katherine Rainey, University of Colorado Boulder
Additional Author | Marcos D. Caballero, Michigan State University
Additional Author | Bethany Wilcox, University of Colorado Boulder
Additional Author | Heather Lewandowski, University of Colorado Boulder
Concise, specific, and measurable statements of desired student learning, often called learning objectives, can aid instructors during curricular development. Similar statements, which we call assessment objectives, can aid researchers in developing and validating research-based assessments. In this talk, we discuss the benefits of articulating assessment objectives during the development of two assessments, one for upper-division thermodynamics and one for lower-division physics labs. Specifically, assessment objectives helped us identify concepts and practices that are important to instructors, aided in the development of targeted assessment items, provided an additional means for establishing face and construct validity, and can serve as a way for instructors and researchers to determine if these assessments are appropriate for use in their courses and research studies in the future.
LC07 (02:50 to 03:00 PM) | Contributed | A Comparative Study of RTOP and T
Presenting Author: Dazhen Tong, East China Normal University
| Zengze Liu, East China Normal University
| Sudong Pan, East China Normal University
| ,
| ,
The new round of basic education curriculum reform in China recently will inevitably bring about the reform of classroom teaching. To find a method of testing the reformed classroom teaching effect so that it follows the new curriculum reform is a problem that needs to be solved urgently. On the basis of examining the evaluation concept and application of the "Reformed Classroom Teaching Observation Protocol (RTOP)" originated in the United States, we test the appropriateness of RTOP in China's physics classroom teaching evaluation environment from both qualitative and quantitative perspectives in a class on Newton's third law, with good results. By comparing the advantages and disadvantages of RTOP and China's "Toshiba Cup Scoring Table", it is found that RTOP can provide guidance for the transformation of classroom teaching evaluation concepts and the development of evaluation tools in the context of China's new curriculum reform.
(LC) PER: Assessment, Grading and Feedback III
Description