
Download
XML
0.00MB

Download
PDF
0.00MB

Read Online
HTML
0.00MB
Abstract
This is the third of five papers detailing our national study of general education astronomy students’ conceptual and reasoning difficulties with cosmology. In this paper, we use item response theory to analyze students’ responses to three out of the four conceptual cosmology surveys we developed. The specific item response theorymodel we use is known as the partial credit model. Since readers may be unfamiliar with the partial credit model, we provide a pedagogical introduction to this model. We use the partial credit model to assess the reliabilities of the four survey forms and to determine the probabilities of students achieving different scores on survey items.
Gina Brissenden provided valuable feedback on earlier versions of this manuscript. This paper benefited from the anonymous reviewer who suggested edits that improved the final version of this paper. This material is based in part upon work supported by the National Science Foundation under Grant Nos. 0833364 and 0715517, a CCLI Phase III Grant for the Collaboration of Astronomy Teaching Scholars (CATS). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
1. INTRODUCTION
2. THE PARTIAL CREDIT MODEL
3. PARTIAL CREDIT MODELANALYSIS
3.1. Item Parameters
3.2. Wright Maps
4. SUMMARY
Key Topics
 Cosmology
 13.0
 Coupled channel methods
 11.0
 Big Bang theory
 8.0
 Geoinformatics
 8.0
 Data analysis
 5.0
Figures
The CRCs for Form B, Item 1 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, the purple short dashed line to a score of 3, and the solid black line to a score of 4. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form B.
Click to view
The CRCs for Form B, Item 1 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, the purple short dashed line to a score of 3, and the solid black line to a score of 4. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form B.
The CRCs for Form C, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form C.
Click to view
The CRCs for Form C, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form C.
The CRCs for Form A, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line corresponds to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form A.
Click to view
The CRCs for Form A, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line corresponds to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The xaxis shows the range of estimated student abilities as measured by the fall 2009 version of Form A.
Form A’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form A’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form B’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form B’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form C’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form C’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form A’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form A’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form B’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form B’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form C’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form C’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form A’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form A’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form B’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form B’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Form C’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Click to view
Form C’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β_{ i0} (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β_{ i1} (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.
Tables
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the fall 2009. All values are in logits
Click to view
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the fall 2009. All values are in logits
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the spring 2010. All values are in logits
Click to view
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the spring 2010. All values are in logits
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the fall 2010. All values are in logits
Click to view
The step difficulty b _{ ij } and Thurstonian threshold β_{ j } parameters for the items on Forms A–C for the fall 2010. All values are in logits
Abstract
This is the third of five papers detailing our national study of general education astronomy students’ conceptual and reasoning difficulties with cosmology. In this paper, we use item response theory to analyze students’ responses to three out of the four conceptual cosmology surveys we developed. The specific item response theorymodel we use is known as the partial credit model. Since readers may be unfamiliar with the partial credit model, we provide a pedagogical introduction to this model. We use the partial credit model to assess the reliabilities of the four survey forms and to determine the probabilities of students achieving different scores on survey items.
Full text loading...
Commenting has been disabled for this content