Instrumentation Process Reflection

Image source: blogs.seattleweekly.com

1.    As my group continues to construct our evaluation instrument, my thoughts and ideas changed from what I began formulating in my mind after viewing several different instruments as identified in my annotated bibliography. I always believed that important characteristics of any media that warrant evaluation included quality of the production, correlation to teaching and learning objectives, as well as appropriate level of content. However, I hadn't really put much thought into considering Bloom's Taxonomy when evaluating media. I certainly never really considered evaluating the extent in which media will be received by a student group in regards to learning context as well as learner background and culture.
      I also found it difficult to think broadly, especially in relation to the baseline requirements. While researching different evaluation instruments while working on my annotated bibliography, I came to realize that many of the instruments in which I felt would be easiest to use and most likely to be put into service in instructional planning were either some form of Likert scale or a checklist. Additionally, most commonly the instruments were standardized in formatting, something I noted added to the ease in which they could be used. Our group decided to build a Likert scaled rubric for the baseline requirements, and  I ended up finding out that some of the categories didn't seem to work well as a scale.

2.   Collaboratively building this instrument across both time and distance has proven to be challenging. While using a wiki as a shared space to collect thinking and track changes was effective, it wasn't the most efficient way to collaborate. For example, the first time visited the wiki to begin work, I had several ideas in my head. However, as I was not the first group member to begin work, a great deal of work was already started, in a direction different than anticipated. While an absolute affirmation for group work is combining multiple perspectives, using an asynchronous method of meeting, some ideas were marginalized or not shared.
      While trying to stick with a format that has great promise in it's ease of use, specifically using a rubric based upon a Likert scale, many categories didn't work well in that particular format. As categories were placed, the Likert scale felt artificial or forced. An example comes in the baseline requirement regarding the consideration of students' background. How can you objectively quantify how well media considers a student's background? While we did find it important to stick with a particular format to aid in the ease of use, it was difficult finding that balance between ease of use and effectiveness of the instrument.