The Arthur Interactive Media (AIM) Buddy Project was designed to promote aspects of character such as intellectual humility in young students. Unlike the simplicity of measuring how a student’s height changes from the beginning of the year to the end, measuring an abstract concept such as a student’s level of grasping the skills and attitudes of honesty or forgiveness is not as simple.
To measure student’s change throughout the study, the AIM research team developed a number of instruments including a student survey that was administered before participation in the program and then again at the end of the program.
One concern when using self-report measures, is the validity of the questions—Do the items measure what researchers want them to measure? This concern is especially heightened when the subjects are children of varying ages who likely think about the questions differently.
Researchers collected various types of data as a way of checking the validity of the student survey items used in the study. This included collecting quantitative data (numbers on a scale in the student surveys) and qualitative data (children’s own words through interviews).
The qualitative method of cognitive interviewing was used in the study’s original pilot phase. Here, children were asked to ‘think aloud’ as they answered. Researchers wanted to know what children were thinking as they came up with their answers.
Student responses were then integrated with survey data, and based on this integration of the quantitative and qualitative data, revisions were made to strengthen or remove items in the survey.
There is so much richness to be gained in the use of both types of data—and the combined use of qualitative data (cognitive interviewing) and quantitative data (survey responses) was validating and critical to shaping the final items of the student surveys.
Now that the data collection is complete—the AIM team is eager to measure changes in students’ survey responses from the study’s fall beginning to its spring ending—and to learn what this data will tell us about students’ character development!