 |
| Image from voiceinrecovery.wordpress.com |
Traditional Instruments
When looking across instruments created by groups, the first obvious similarity that stood out was the organization of sequence according to the baseline descriptors: learners, learning, context, standards, learning objectives, strategies, cognitive load, accommodations, student assessment, and accessibility. In fact, many instruments also organized the way the baseline descriptors were evaluated, frequently relying on a 3 point rubric in which the levels of proficiency were narratively described. What’s more, several instruments also included a space after each descriptor with space for narrative response.
Many groups collected a running total of points earned based upon specific scores in each category, unlike the instrument created by my group in which in lieu of points, the categories were identified as proficient, sufficient, or digressing. The groups had a final point total at the end of the instrument to aid in determining the value of the media. One particular group, Team Media Evaluation, also included a narrative description of what the media should look like. In addition to the rubric as created by my group, there was also a broad range of response strategies, including Likert scales, yes or no questions, and fill in the blanks. Shamelle Nash even included a way to collect point values for the narrative section of her instrument.
Looking at this broad range of evaluation instruments, a few ways to improve the instrument created by my group are now possible. For example, Team Social Studies Rules included a narrative response section after each rating. However, unlike our blank lines, they prefaced the writing space with a guiding question, thoughtfully designed to help the evaluator think deeply about the media in question. A big “ah-ha” moment for me came when viewing the end of Shamelle Nash’s instrument. At the end of the instrument, there was a benchmark goal. It was cut and dry: scores above this level indicate a quality media. No other group, including my own, had something like that. Most of the rest of us simply collected points and left the evaluator to determine what to do with the points.
Web 2.0 Instruments
Many of the Web 2.0 Instruments were designed in a similar fashion as the instrument designed by my team. For the most part, the Web 2.0 Instrument is a more specialized version of the Traditional Instrument. Most groups started with the same baseline descriptors, then moved into the more media specific descriptors, such as credibility of source and interface design and navigation. Again, many of the groups based the evaluation on a 3 point rubric.
Team Social Studies Rules had a dramatically different Web 2.0 instrument, which was not only dramatically different from their Traditional instrument, but also very different from the other Web 2.0 instruments. Not only did the group change the content being addressed, but they used a Web 2.0 tool, Google Forms, to deliver the instrument. A question I would have for the group is this: What is the purpose for the instrument? The form clearly collects a wide range of data, which is presumably aggregated into a Google Spreadsheet. I’m wondering if the purpose for the instrument is to collect data on a range of media, developing a database to refer to when looking for effective media. If the purpose is to ascertain the effectiveness of the instrument, I wonder how the user receives the feedback.
One easy way to improve both the Web 2.0 instrument as well as the Traditional instrument created by my group is to focus more on ease and aesthetics. My first view of the Web 2.0 instrument created by Team Social Studies Rules resulted in a gasp. The instrument was aesthetically pleasing, and seemed like something I could quickly fill out as I was reviewing media.
