Journal of Mechanical Design, Vol. 128, No. 3, pp. 513515, May 2006
©2006 American Society of Mechanical Engineers. All rights reserved.
Previous section: The Concept of the Program
Next section: Next Steps
Title Page
Formative Assessment
The process for developing this television program was iterative. Prototype episodes were filmed and edited and then subject to formative assessment from potential viewers and from a group of expert advisors. The feedback was then used in developing another prototype, and the cycle was repeated. The two cycles of development and evaluation are presently complete, and three cycles will be completed before filming the first season of 13 episodes in the summer of 2006.
For each prototype episode, focus group methodology was used to elicit feedback from viewers in the target demographics. Two episodes have been evaluated so far. Each episode was viewed by children in groups of three to five. Observations were made as the children were viewing and detailed discussions with the children were conducted afterward. Dozens of such focus groups were conducted in Cambridge, MA and Stamford, CT, so that, in total, over 100 children viewed each of the first two prototype episodes.
The evaluations indicated that the show strongly appealed to kids of both genders across all the ages sampled. The research participants (kids 912) exhibited enthusiasm while viewing and during discussion afterward. They showed high interest and formed opinions about the engineering designs and what would work best. Participants admired the skill and teamwork of the contestants making comments such as "Erin was really, really good and really smart. She made the..." and "George came up with the planshe figured out how to test the... ."
In addition to testing the appeal of the program, an effort was made to assess the educational effectiveness of the program. Focus groups revealed that participants understood the scientific/technical concepts presented. For example, children vividly remembered the fact that a servomotor was used in a design and that it provided rotary motion. With another prototype episode, children formed opinions about preferred techniques for sealing joints. For both episodes, children were able to articulate different parts of the engineering design process.
Despite generally positive reactions, some weaknesses were revealed. For example, viewers wanted more detail in some technical areas, especially what components were being used and where they came from. In addition, kids felt that the working title of the show, "Design Squad," did not fit. Many kids objected to the word "design," mentioning an association with home design or decoration rather than invention or technology. In the opinion of the authors, this underscores the need for public media to be involved in shaping the national dialogue. The very concept of design, so central to engineering, in being lost to us because other professions are using the term in publicly visible ways and we are not. We need to show the world what design means to us, as engineers.
Previous section: The Concept of the Program
Next section: Next Steps
Title Page