1887
banner image
No data available.
Please log in to see this content.
You have no subscription access to this content.
No metrics data to plot.
The attempt to load metrics for this article has failed.
The attempt to plot a graph for these metrics has failed.
f
A Study of General Education Astronomy Students’ Understandings of Cosmology. Part II. Evaluating Four Conceptual Cosmology Surveys: A Classical Test Theory Approach
Rent:
Rent this article for
Access full text Article
/content/aas/journal/aer/10/1/10.3847/AER2011030
1.
1. Bardar, E. M. , Prather, E. E , Brecher, K. , and Slater, T. F. 2007, “Development and Validation of the Light and Spectroscopy Concept Inventory,” Astronomy Education Review, 5, 103.
http://dx.doi.org/10.3847/AER2006020
2.
2. Cohen, J. 1960, “A Coefficient of Agreement for Nominal Scales,” Educational and Psychological Measurement, 20, 37.
http://dx.doi.org/10.1177/001316446002000104
3.
3. Cohen, J. 1968, “Weighted Kappa: Nominal Scale Agreement with Provision for Scaled Disagreement or Partial Credit,” Psychological Bulletin, 70, 213.
http://dx.doi.org/10.1037/h0026256
4.
4. Ding, L. , and Beichner, R. 2009, “Approaches to Data Analysis of Multiple-Choice Questions,” Physical Review Special Topics–Physics Education Research, 5, 020103.
http://dx.doi.org/10.1103/PhysRevSTPER.5.020103
5.
5. Ding, L. , Chabay, R. , Sherwood, B. , and Beichner, R. 2006, “Evaluating an Electricity and Magnetism Assessment Tool: Brief Electricity and Magnetism Assessment,” Physical Review Special Topics– Physics Education Research, 2, 010105.
http://dx.doi.org/10.1103/PhysRevSTPER.2.010105
6.
6. Fleiss, J. L. , Levin, B. , and Paik, M. C. 2003, Statistical Methods for Rates and Proportions, 3rd ed., Hoboken, NJ: John Wiley and Sons, Inc.
7.
7. George, D. and Mallery, P. 2009, SPSS for Windows Step by Step: A Simple Guide and Reference, Boston, MA: Pearson Education, Inc.
8.
8. Hambleton, R. K. , and Jones, R. J. 1993, “Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development,” Educational Measurement: Issues and Practice, 12, 253.
9.
9. Landis, J. R. , and Koch, G. G. 1977, “The Measurement of Observer Agreement for Categorical Data,” Biometrics, 33, 159.
http://dx.doi.org/10.2307/2529310
10.
10. Lord, F. M. , and Novick, M. R. 1968, Statistical Theories of Mental Test Scores, Reading, MA: Addison-Wesley.
11.
11. Maloney, D. P. , O’Kuma, T. L. , Hieggelke, C. J. , and van Heuvelen, A. 2001, “Surveying Students’ Conceptual Knowledge of Electricity and Magnetism,” American Journal of Physics, 69, S12.
http://dx.doi.org/10.1119/1.1371296
12.
12. McDermott, L. C. , Rosenquist, M. L. , and van Zee, E. H. , 1987, “Student Difficulties in Connecting Graphs and Physics: Examples From Kinematics,” American Journal of Physics, 55, 503.
http://dx.doi.org/10.1119/1.15104
13.
13. Otero, V. K. , and Harlow, D. B. 2009, “Getting Started in Qualitative Physics Education Research,” in Reviews in PER Vol. 2: Getting Started in PER, ed. C. Henderson and K. A. Harper, College Park, MA: American Association of Physics Teachers, 1.
14.
14. Schmitt, N. 1996, “Uses and Abuses of Coefficient Alpha,” Psychological Assessment, 8, 350.
http://dx.doi.org/10.1037/1040-3590.8.4.350
15.
15. Thompson, B. 2003, “Understanding Reliability and Coefficient alpha, Really,” in Score Reliability, ed. B. Thompson, Thousand Oaks, CA: SAGE Publications, 3.
16.
16. Wallace, C. S. , and Bailey, J. M. 2010, “Do Concept Inventories Actually Measure Anything?,” Astronomy Education Review, 9, 010116.
http://dx.doi.org/10.3847/AER2010024
17.
17. Wallace, C. S. , Prather, E. E. , and Duncan, D. K. 2011a, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part I. Development and Validation of Four Conceptual Cosmology Surveys,” Astronomy Education Review, 10, 010106.
http://dx.doi.org/10.3847/AER2011029
18.
18. Wallace, C. S. , Prather, E. E. , and Duncan, D. K. 2011b, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part III. Evaluating Four Conceptual Cosmology Surveys: An Item Response Theory Approach,” Astronomy Education Review (submitted).
19.
19. Wallace, C. S. , Prather, E. E. , and Duncan, D. K. 2011c, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part IV. Common Difficulties Students Experience with Cosmology,” Astronomy Education Review (submitted).
20.
20. Wallace, C. S. , Prather, E. E. , and Duncan, D. K. 2011d, “A Study of General Education Astronomy Students’ Understandings of Cosmology. Part V. The Effects of a New Suite of Cosmology Lecture-Tutorials on Students’ Conceptual Knowledge,” (in preparation).
21.
21. Wilson, M. 2005, Constructing Measures: An Item Response Modeling Approach, Mahwah, NJ: Lawrence Erlbaum Associates.
22.
journal-id:
http://aip.metastore.ingenta.com/content/aas/journal/aer/10/1/10.3847/AER2011030
Loading
View: Tables

Tables

Generic image for table

Click to view

Table 1.

The scoring rubric for Item 1 on Form B (fall 2010 version)

Generic image for table

Click to view

Table 2.

A sample of student responses to Item 1 on Form B and their overall scores and reasoning element codes. The student responses contain their original spellings, grammar, and punctuation

Generic image for table

Click to view

Table 3.

The pre-instruction difficulties (P-values) and discriminations of the items on Forms A–D

Generic image for table

Click to view

Table 4.

The post-instruction difficulties (P-values) and discriminations of the items on Forms A-D

Generic image for table

Click to view

Table 5.

Cronbach’s α (pre-instruction and post-instruction) for Forms A–D

Abstract

This is the second of five papers detailing our national study of general education astronomy students’ conceptual and reasoning difficulties with cosmology. This article begins our quantitative investigation of the data. We describe how we scored students’ responses to four conceptual cosmology surveys, and we present evidence for the inter-rater reliability of those scores. We devote the bulk of this article to a classical test theoryanalysis of the data. We calculate difficulties and discriminations for each item, and we compute Cronbach’s α as a measure of the reliability of the surveys. We also discuss the implications this analysis has for the validity of the surveys.

Loading

Full text loading...

/deliver/fulltext/aas/journal/aer/10/1/1.3663209.html;jsessionid=1n3tgcjkjm0uq.x-aip-live-03?itemId=/content/aas/journal/aer/10/1/10.3847/AER2011030&mimeType=html&fmt=ahah&containerItemId=content/aas/journal/aer
true
true
This is a required field
Please enter a valid email address
752b84549af89a08dbdd7fdb8b9568b5 journal.articlezxybnytfddd
Scitation: A Study of General Education Astronomy Students’ Understandings of Cosmology. Part II. Evaluating Four Conceptual Cosmology Surveys: A Classical Test Theory Approach
http://aip.metastore.ingenta.com/content/aas/journal/aer/10/1/10.3847/AER2011030
10.3847/AER2011030
SEARCH_EXPAND_ITEM