1887
banner image
No data available.
Please log in to see this content.
You have no subscription access to this content.
No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
f
A Study of General Education Astronomy Students’ Understandings of Cosmology. Part III. Evaluating Four Conceptual Cosmology Surveys: An Item Response Theory Approach
Rent:
Rent this article for
Access full text Article
/content/aas/journal/aer/11/1/10.3847/AER2011031
1.
1. Baker, F. B. , and Kim, S. 2004, Item Response Theory: Parameter Estimation Techniques, 2nd ed., New York, NY: Marcel Dekker, Inc.
2.
2. Embretson, S. E. , and Reise, S. P. 2000, Item Response Theory for Psychologists, Mahwah, NJ: Lawrence Erlbaum Associates.
3.
3. Hambleton, R. K. , and Jones, R. J. 1993, “Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development,” Educational Measurement: Issues and Practice, 12, 253.
4.
4. Harris, D. 1989, “Comparison of 1-, 2-, and 3-Parameter IRT Models,” Educational Measurement: Issues and Practice, 8, 157.
http://dx.doi.org/10.1111/j.1745-3992.1989.tb00313.x
5.
5. Kennedy, C. A. , Wilson, M. , Draney, K. , Tutunciyan, S. , and Vorp, R. 2006, ConstructMap Software, Berkeley, CA: Berkeley Evaluation and Assessment Research (BEAR) Center. Available at: http://bearcenter.berkeley.edu/ConstructMap.
6.
6. Masters, G. N. 1982, “A Rasch Model for Partial Credit Scoring,” Psychometrika, 47, 149.
http://dx.doi.org/10.1007/BF02296272
7.
7. Rasch, G. 1980, Probabilistic Models for Some Intelligence and Attainment Tests, Chicago, IL: University of Chicago Press. Originally published in 1960, Copenhagen, DK: The Danish Institute for Educational Research.
8.
8. Schmitt, N. 1996, “Uses and Abuses of Coefficient Alpha,” Psychological Assessment, 8, 350.
http://dx.doi.org/10.1037/1040-3590.8.4.350
9.
9. Thompson, B. 2003, “Understanding Reliability and Coefficient alpha, Really,” in Score Reliability, ed. B. Thompson, Thousand Oaks, CA: SAGE Publications, 3.
10.
10. Wallace, C. S. , and Bailey, J. M. 2010, “Do Concept Inventories Actually Measure Anything?” Astronomy Education Review, 9, 010116.
http://dx.doi.org/10.3847/AER2010024
11.
11. Wallace, C. S. , Prather, E. E. , and Duncan, D. 2011a, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part I. Development and Validation of Four Conceptual Cosmology Surveys,” Astronomy Education Review, 10, 010106.
http://dx.doi.org/10.3847/AER2011029
12.
12. Wallace, C. S. , Prather, E. E. , and Duncan, D. 2011b, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part II. Evaluating Four Conceptual Cosmology Surveys: A Classical Test Theory Approach,” Astronomy Education Review, 10, 010107.
http://dx.doi.org/10.3847/AER2011030
13.
13. Wallace, C. S. , Prather, E. E. , and Duncan, D. 2012a, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part IV. Common Difficulties Students Experience with Cosmology,” Astronomy Education Review, 11, 010104.
http://dx.doi.org/10.3847/AER2011032
14.
14. Wallace, C. S. , Prather, E. E. , and Duncan, D. 2012b, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part V. The Effects of a New Suite of Cosmology Lecture-Tutorials on Students’ Conceptual Knowledge” (in preparation).
15.
15. Wilson, M. 2005, Constructing Measures: An Item Response Modeling Approach, Mahwah, NJ: Lawrence Erlbaum Associates.
16.
16. Yen, W. M. 1993, “Scaling Performance Assessments: Strategies for Managing Local Item Dependence,” Journal of Educational Measurement, 30, 187.
http://dx.doi.org/10.1111/j.1745-3984.1993.tb00423.x
17.
journal-id:
http://aip.metastore.ingenta.com/content/aas/journal/aer/11/1/10.3847/AER2011031
Loading

Figures

Image of Figure 1.

Click to view

Figure 1.

The CRCs for Form B, Item 1 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, the purple short dashed line to a score of 3, and the solid black line to a score of 4. The x-axis shows the range of estimated student abilities as measured by the fall 2009 version of Form B.

Image of Figure 2.

Click to view

Figure 2.

The CRCs for Form C, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line correspond to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The x-axis shows the range of estimated student abilities as measured by the fall 2009 version of Form C.

Image of Figure 3.

Click to view

Figure 3.

The CRCs for Form A, Item 3 from the fall 2009. Each curve shows the probability of a particular score as a function of ability. The blue dashed and dotted line corresponds to a score of 0, the green dotted line to a score of 1, the red dashed line to a score of 2, and the purple short dashed line to a score of 3. The x-axis shows the range of estimated student abilities as measured by the fall 2009 version of Form A.

Image of Figure 4.

Click to view

Figure 4.

Form A’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 5.

Click to view

Figure 5.

Form B’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 6.

Click to view

Figure 6.

Form C’s Wright map for the fall 2009. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 7.

Click to view

Figure 7.

Form A’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 8.

Click to view

Figure 8.

Form B’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 9.

Click to view

Figure 9.

Form C’s Wright map for the spring 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 10.

Click to view

Figure 10.

Form A’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 11.

Click to view

Figure 11.

Form B’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Image of Figure 12.

Click to view

Figure 12.

Form C’s Wright map for the fall 2010. A histogram of students’ abilities (proficiencies) is shown on the left. On the right are the Thurstonian thresholds for each item. Blue corresponds to β i0 (i.e., the ability at which one has equal probability of earning a score < 1 and a score ≥ 1), green to β i1 (i.e., the ability at which one has equal probability of earning a score < 2 and a score ≥ 2), etc.

Tables

Generic image for table

Click to view

Table 1.

The step difficulty b ij and Thurstonian threshold β j parameters for the items on Forms A–C for the fall 2009. All values are in logits

Generic image for table

Click to view

Table 2.

The step difficulty b ij and Thurstonian threshold β j parameters for the items on Forms A–C for the spring 2010. All values are in logits

Generic image for table

Click to view

Table 3.

The step difficulty b ij and Thurstonian threshold β j parameters for the items on Forms A–C for the fall 2010. All values are in logits

Abstract

This is the third of five papers detailing our national study of general education astronomy students’ conceptual and reasoning difficulties with cosmology. In this paper, we use item response theory to analyze students’ responses to three out of the four conceptual cosmology surveys we developed. The specific item response theorymodel we use is known as the partial credit model. Since readers may be unfamiliar with the partial credit model, we provide a pedagogical introduction to this model. We use the partial credit model to assess the reliabilities of the four survey forms and to determine the probabilities of students achieving different scores on survey items.

Loading

Full text loading...

/deliver/fulltext/aas/journal/aer/11/1/1.3677389.html;jsessionid=7t8d4nd3cag5.x-aip-live-02?itemId=/content/aas/journal/aer/11/1/10.3847/AER2011031&mimeType=html&fmt=ahah&containerItemId=content/aas/journal/aer
true
true
This is a required field
Please enter a valid email address
752b84549af89a08dbdd7fdb8b9568b5 journal.articlezxybnytfddd
Scitation: A Study of General Education Astronomy Students’ Understandings of Cosmology. Part III. Evaluating Four Conceptual Cosmology Surveys: An Item Response Theory Approach
http://aip.metastore.ingenta.com/content/aas/journal/aer/11/1/10.3847/AER2011031
10.3847/AER2011031
SEARCH_EXPAND_ITEM