My implementation of Hot Seat questioning was in direct response to student feedback on a prior participation system in which individual students received points for asking or answering questions in class. In open-ended student evaluations from fall 2003, 9 out of 22 students commented that this was an area for improvement, and none noted it as helpful. To gauge student opinion of the Hot Seat experience, I administered both a multiple-choice survey asking questions about specific aspects of the course, and the university's official open-ended student evaluation form.

In the multiple-choice survey, students were asked which component of the class was most helpful in learning the material. Of the 50 students responding, many cited listening to the instructor (17 students), taking online prequizzes (16 students), or seeing PowerPoint presentations (13 students) as helpful. Only three students marked that preparing for the Hot Seat was most helpful. Clearly, the students felt that this was not the dominant mechanism in their learning of astronomy. However, in the open-ended student evaluations, 10 out of 46 remarked that being in the Hot Seat was helpful. This was an obvious turnaround from the previous semester, when as many students had deemed class participation to be in need of revision.

Only three students responded negatively to the Hot Seat. One wrote, “I didn't like how the Hot Seat was put on the spot in front of the class.” Another said, “The Hot Seat stuff eliminates the rest of the class from participating daily.” The third simply jotted, “No more Hot Seat; it doesn't help.” The first comment shows that some students can be sensitive to being “put on the spot,” even when it is with three other classmates in a scheduled event. The second comment was a concern I shared before the class began. Surprisingly, I found the opposite to be true. The remainder of the class was more vocal when asked to respond to questions that the Hot Seat missed than they were during standard lectures. This tone was similar to that of a game show, with the “audience members” sometimes shouting out answers before the Hot Seat “contestants” could respond. The third comment that it doesn't help is one that also appears false when we examine the test scores related to Hot Seat questions, as we do below.