^{1,a)}, Edwin F. Greco

^{1}, Eric R. Murray

^{1}, Keith R. Bujak

^{2}, M. Jackson Marr

^{2}, Richard Catrambone

^{2}, Matthew A. Kohlmyer

^{3}and Michael F. Schatz

^{4,b)}

### Abstract

The performance of over 5000 students in introductory calculus-based mechanics courses at the Georgia Institute of Technology was assessed using the Force Concept Inventory (FCI). Results from two different curricula were compared: a traditional mechanics curriculum and the Matter & Interactions (M&I) curriculum. Both were taught with similar interactive pedagogy. Post-instruction FCI averages were significantly higher for the traditional curriculum than for the M&I curriculum; the differences between curricula persist after accounting for factors such as pre-instruction FCI scores, grade point averages, and SAT scores. FCI performance on categories of items organized by concepts was also compared; traditional averages were significantly higher in each concept. We examined differences in student preparation between the curricula and found that the relative fraction of homework and lecture topics devoted to FCI force and motion concepts correlated with the observed performance differences. Concept inventories, as instruments for evaluating curricular reforms, are generally limited to the particular choice of content and goals of the instrument. Moreover, concept inventories fail to measure what are perhaps the most interesting aspects of reform: the non-overlapping content and goals that are not present in courses without reform.

The authors would like to thank the two anonymous reviewers for their many helpful comments. The authors would also like to thank Andrew Scherbakov and Robert Hume (Office of Minority Education and Development) for their efforts in collecting and organizing the demographic data. This work was supported by National Science Foundation’s Division of Undergraduate Education (DUE0618519 and DUE0942076).

I. INTRODUCTION

II. INTRODUCTORY MECHANICS AT GEORGIA TECH

III. SUMMARY OF MEASUREMENTS FROM IN-CLASS TESTING

IV. ITEM ANALYSIS OF FCI MEASUREMENTS

V. CONTRIBUTIONS TO THE PERFORMANCE DIFFERENCES

VI. CLOSING REMARKS AND LESSONS LEARNED

### Key Topics

- Lectures
- 16.0
- Kinematics
- 9.0
- Physics education
- 9.0
- Learning theory and science teaching
- 4.0
- Physics education research
- 4.0

## Figures

Average pre- and post-instruction FCI scores at Georgia Tech. The average FCI pre- and post-test scores are shown for students who took a one-semester mechanics course with either the traditional (TRAD) or Matter & Interactions (M&I) curriculum. The number of students (*N*) tested for each curriculum is indicated in the figure. The error bounds represent the 95% confidence intervals (estimated from the t-statistic) on the estimate of the average score.

Average pre- and post-instruction FCI scores at Georgia Tech. The average FCI pre- and post-test scores are shown for students who took a one-semester mechanics course with either the traditional (TRAD) or Matter & Interactions (M&I) curriculum. The number of students (*N*) tested for each curriculum is indicated in the figure. The error bounds represent the 95% confidence intervals (estimated from the t-statistic) on the estimate of the average score.

Gain in understanding of mechanics as measured by the FCI. The increase in student understanding resulting from a one-semester traditional (TRAD) or Matter & Interactions (M&I) course is measured using (a) the average raw gain *G* and (b) the average normalized gain *g*. Only students with matched scores were used for this figure (see Table I). The error bounds represent the 95% confidence intervals (estimated from the t-statistic) on the estimate of (a) the raw gain and (b) the normalized gain.

Gain in understanding of mechanics as measured by the FCI. The increase in student understanding resulting from a one-semester traditional (TRAD) or Matter & Interactions (M&I) course is measured using (a) the average raw gain *G* and (b) the average normalized gain *g*. Only students with matched scores were used for this figure (see Table I). The error bounds represent the 95% confidence intervals (estimated from the t-statistic) on the estimate of (a) the raw gain and (b) the normalized gain.

FCI score distributions by curriculum. The distributions of FCI test scores for students before (a) and after (b) completing a mechanics course with either a traditional (dashed line) or M&I curriculum (solid line) are shown. The total number of students tested in each curriculum is the same as in Fig. 1. The plots are constructed from binned data with bin widths equal to approximately 6.7% of the maximum possible FCI score (100%).

FCI score distributions by curriculum. The distributions of FCI test scores for students before (a) and after (b) completing a mechanics course with either a traditional (dashed line) or M&I curriculum (solid line) are shown. The total number of students tested in each curriculum is the same as in Fig. 1. The plots are constructed from binned data with bin widths equal to approximately 6.7% of the maximum possible FCI score (100%).

Difference in performance for individual FCI items and mechanics concepts. The difference in performance between traditional and M&I students is shown for each question on the FCI. Positive (negative) indicates superior performance by traditional (M&I) students on individual questions. The numerical labels indicate the corresponding question number in order of appearance on the FCI. The items are grouped together into one of five concepts: Kinematics, Newton’s first law, Newton’s second law, Newton’s third law, and Force Identification. The horizontal line (dash) illustrates the value of , the mean difference in the item gains between curricula.

Difference in performance for individual FCI items and mechanics concepts. The difference in performance between traditional and M&I students is shown for each question on the FCI. Positive (negative) indicates superior performance by traditional (M&I) students on individual questions. The numerical labels indicate the corresponding question number in order of appearance on the FCI. The items are grouped together into one of five concepts: Kinematics, Newton’s first law, Newton’s second law, Newton’s third law, and Force Identification. The horizontal line (dash) illustrates the value of , the mean difference in the item gains between curricula.

## Tables

Georgia Tech FCI test results are shown for 22 traditional sections (T1–T22) and 6 Matter & Interactions sections (M1–M6). Different lecturers are distinguished by a unique letter in column L. The average incoming FCI score *I* for students entering the course is shown for sections in which the FCI was given prior to instruction. In those sections where data are available, the average outgoing FCI score *O* for students completing this course are indicated. is the number of students in a given section who took the FCI both at the beginning and at the end (i.e., matched data) of their mechanics course.

Georgia Tech FCI test results are shown for 22 traditional sections (T1–T22) and 6 Matter & Interactions sections (M1–M6). Different lecturers are distinguished by a unique letter in column L. The average incoming FCI score *I* for students entering the course is shown for sections in which the FCI was given prior to instruction. In those sections where data are available, the average outgoing FCI score *O* for students completing this course are indicated. is the number of students in a given section who took the FCI both at the beginning and at the end (i.e., matched data) of their mechanics course.

The average differences in item gains between curricula are computed for the items in each FCI force and motion concept, . Each is positive, indicating better average item gains for traditional students across all FCI force and motion concepts. Concepts with higher are those for which traditional students achieve higher normalized gains than M&I students. Traditional students achieve the highest values of on the Kinematics and Force Identification concepts and lowest on Newton’s first law concept. The measures are presented along with their variance.

The average differences in item gains between curricula are computed for the items in each FCI force and motion concept, . Each is positive, indicating better average item gains for traditional students across all FCI force and motion concepts. Concepts with higher are those for which traditional students achieve higher normalized gains than M&I students. Traditional students achieve the highest values of on the Kinematics and Force Identification concepts and lowest on Newton’s first law concept. The measures are presented along with their variance.

An estimate of the fraction of homework questions covering a particular FCI concept in the two mechanics curricula is compared. Subtopics for these homework questions were not mutually exclusive. The relative fraction of homework questions covering FCI force and motion concepts and some individual FCI concepts (i.e., Kinematics, Newton’s second law, Newton’s third law, and Force Identification) is greater in the traditional curriculum.

An estimate of the fraction of homework questions covering a particular FCI concept in the two mechanics curricula is compared. Subtopics for these homework questions were not mutually exclusive. The relative fraction of homework questions covering FCI force and motion concepts and some individual FCI concepts (i.e., Kinematics, Newton’s second law, Newton’s third law, and Force Identification) is greater in the traditional curriculum.

Comparison of the estimated fractions of lecture/reading topics in the two mechanics curricula. Subtopics for these lectures/readings were not mutually exclusive. The relative fraction of lectures/readings in the traditional course is greater for the Kinematics, Newton’s third law, and Force Identification topics, which is consistent with their superior performance in those concepts on the FCI. However, on Newton’s first and second laws, the relative fractions of lectures/readings are roughly similar.

Comparison of the estimated fractions of lecture/reading topics in the two mechanics curricula. Subtopics for these lectures/readings were not mutually exclusive. The relative fraction of lectures/readings in the traditional course is greater for the Kinematics, Newton’s third law, and Force Identification topics, which is consistent with their superior performance in those concepts on the FCI. However, on Newton’s first and second laws, the relative fractions of lectures/readings are roughly similar.

Article metrics loading...

Full text loading...

Commenting has been disabled for this content