We’re proud to announce that our second presentation to the Sloan-C/MIC conference (happening later this month) received one of the 2012 Effective Practice awards. According to the official announcement, “the winning practices were selected for recognition because they provide evidence of innovation and replicability. The practices advance the goals of access, learning effectiveness, faculty and student satisfaction, and scalability.” This presentation focused on the cross-disciplinary nature of the MELO 3D project. We’ve been fortunate to receive funding for this project from the University of Michigan’s LSA Information Technology Committee New Infrastructure/New Initiatives (NINI) grant. Receiving this support has been fundamental to providing our participants with the resources and support needed to effectively meet this project’s goals.
Read on to find the full text of the submission.
“Integration of Technology Into Undergraduate Education via Cross-Disciplinary Pollination”
Access, Faculty Satisfaction, Learning Effectiveness, Scale, Student Satisfaction
Nancy Konigsberg Kerner and Brenda Gunderson
Institution Where EP Occurred:
University of Michigan
Effective Practice Abstract/Summary:
The MELO (Michigan Education through Learning Objects) project is a cross-disciplinary project that facilitated development of technology infused courses for enhanced learning. The project became active in 2008-2009 with an initial impact in three academic gateway courses: General Chemistry, Intro Psychology, and Intro Statistics. In 2009, the project was recipient of the Innovative Use of MERLOT Award. The great success in the initial disciplines led to additional funding and the involvement of additional disciplines. The project is currently in its third year and involves students (undergraduate and graduate) and faculty from eight different disciplines and some select staff as the key collaborators.
Cross-discipline training workshops and meetings facilitate the cross-pollination of ideas and technology infusion models which allows participants to creatively experiment with less risk and time demands. For example, one discipline shared the use of a screen-capture technology (Jing) for wrapping imperfect online learning objects for course inclusion in online pre-lab assignments. Following this sharing, there was adoption and adaptation of the technology by other disciplines for delivery of content in innovative ways. Such sharing was formative for the development and integration of technology into many curricula. Some of the different discipline course and curricula technology infusion innovations for enhanced learning include:
- Shift from individual to collaborative online assignments using Voice Thread (Organic Chemistry)
- Shift from static hard-copy to dynamic online learning object assignments (Physics, Physical Chemistry, Statistics)
- Shift from written feedback to personalized audio/video feedback (Writing)
- Shift of live pre-lab lectures to a web-based multimedia interactive format (General Chemistry)
- Shift of a live lecture to an online wiki syllabus with infused learning objects (American History)
- Shift of teacher-centered curricula to a curricula where the students are co-teachers building the online curricula (Psychology, General Chemistry, Writing)
As a result of this project an estimated 15,000 students per academic year are exposed to technology infused courses for enhanced learning impact.
Description of the Effective Practice:
MELO is a cross-disciplinary project at the University of Michigan (UM) with a goal to facilitate the integration of curriculum-based sequences of online learning objects (LOs) that complement classroom pedagogy in large enrollment gateway courses to enhance student learning, engagement, and persistence in college. LOs are interactive web resources designed to support a learning objective and include such things as animations, simulations, tutorials, case studies, and games. In this project we work primarily with openly licensed and adaptable LOs, some created by students and instructors at UM as well as those freely available on the web and through MERLOT.
Research into the efficacy of online learning objects has demonstrated that, by offering students a sense of control and ownership of the learning process, students’ educational achievement improves compared to a control group (Windle, McCormick, Dandrea, & Wharrad, 2010). Other research suggests that educational strategies that improve understanding and provide learning feedback have a positive impact on student retention (Hershock & O’Neal et al., 2007; Seymour & Hewitt, 1997).
The project took a unique approach to overcoming barriers to technology-enriched instruction by involving undergraduate and graduate students in addition to select faculty and staff from across different disciplines as the key collaborators. By training these select students and faculty to find, evaluate, adapt, create, and integrate online learning objects (LOs), the project facilitated the incorporation of high quality inter-disciplinary and discipline-specific LOs into the curricula. We include the word “adapt” relative to just the integration of LOs because many useful online learning resources exist, but many must be adapted in some way to become a strong and useful match to a given curricula.
Working in a cross-disciplinary setting of graduate students, staff, and faculty allowed for a rich learning environment where participants could creatively experiment with less risk and time demands due to the cross-pollination of ideas and technology infusion examples. This unique approach of involving graduate students in such a process is akin to training future faculty. Also, many of the graduate student participants were more knowledgeable and motivated with regard to technology and creative technology infusion and thus trained the faculty. One graduate student participant gave a presentation at the 2009 MERLOT International Conference entitled “Bottom-Up Faculty Development” which emphasized the dual direction of the project.
Working in a collaborative cross-discipline environment was formative for shaping the training program and its scalability and adoption to new and diverse curricula. Project participants were asked to document their infusion accomplishments via video capture technology. These short video summaries, which provided a statement of the initial problem, how the problem was addressed, and sample results, became a key aspect in successive training workshops. Workshops have also been provided outside the project itself as part of the UM Enriching Scholarship (free annual workshops for any UM student, faculty or staff member). Evaluations of these workshops have been exemplary and have led to identifying new MELO project participants. We are in the process of “packaging” the various training workshops in the form of openly licensed materials. These materials address topics such as “Integration (and Creation) of Online Learning Resources: Why and How?”, “Open Educational Resources”, and “Web Adaptability”.
Cross-discipline sharing was not only formative for the development and integration of technology into curricula, but it also led to a shift in project participants’ views regarding who has the responsibility to craft curricula — from that of instructors only, to instructors and students. For example, a MELO project graduate student instructor challenged the undergraduate students in their section of Intro Psychology to identify difficult course concepts and find interactive online learning resources that would assist student learning with respect to the identified concept. These undergraduate students contributed superior online resources that the instructors had not previously identified. Following this sharing, another MELO graduate student instructor for General Chemistry extended a similar challenge to the entire class of 450 undergraduate students through an “Online LO Scavenger Hunt” (see link 8 in the References and Supporting Documentation section below). Over 100 new LOs were contributed to the course by these undergraduate students. The MELO writing discipline team would subsequently ask undergraduate students in their courses to author reflective videos with regard to the writing of papers. These reflective videos populate a website as a teaching tool for all students. Thus the students are not only engaged learners but also co-teachers with regard to curricula development.
Evidence of Effectiveness:
A “MERLOT Institutional Stewardship Award” (2008) and an “Innovative Use of MERLOT Award” (2009) honored our initial proposal and project work (see http://taste.merlot.org/MERLOTAwards/2008InstitutionalSteward.htm and http://taste.merlot.org/MERLOTAwards/2009InnovativeUse.html). The latter award is given to “an individual or program that demonstrates novel, comprehensive, or creative usage and/or application of MERLOT materials and services. Nominees must provide evidence of an impact on students that can be demonstrated through qualitative or quantitative research or anecdotal evidence”. The identified winners were the MELO project co-PIs (Nancy Konigsberg Kerner and Brenda Gunderson) along with the year one and two core staff and faculty (Lynne Crandall and Brian Malley).
Several project discipline participants have received awards for MELO project related accomplishments. Psychology graduate student Adena Rottenstein created an LO with an undergraduate student demonstrating the Frustration-Aggression Hypothesis. It is hosted at the following site: http://youtu.be/oe8PJ8rVV0M. She entered the video in the Cengage Psychology ‘Get Psyched’ Video Contest (see http://www.youtube.com/watch?v=2zmZhQWNhpw). The submission beat out 100 other videos to win first place with a grand prize of an all-expense-paid trip to the psychology conference of choice). Adena also submitted her Frustration-Aggression LO to MERLOT in the area of Psychology.
Chemistry graduate student Porscha McRobbie submitted her Physical Chemistry homework animations created using Wolfram Mathematica to MERLOT (see http://demonstrations.wolfram.com/author.html?author=Porscha+McRobbie). The work shifted what were previously static hard copy homework problems to online interactive animations. The MERLOT Chemistry Editorial Board nominated the animations for a Classic Award. Porscha graduated with a Ph.D. and accepted a job with Wolfram Mathematica as the Chemistry Specialist.
A Sloan-C/MERLOT 2012 conference presentation on the MELO project entitled “Online Learning Objects: Affecting Change Through Cross-Disciplinary Practices and Open Technologies” submitted by MELO staff project participants Emily Rogers and Steve Lonn received the “best in track” award for the “Innovation in Media and Tools” track. The abstract for the presentation can be found at:
The following links are reviews that feature summaries with anecdotal evidence of effectiveness for discipline outcomes in Organic Chemistry, Spanish, and History (see links 2 and 4 in the References and Supporting Documentation section below for the LOs for Organic Chemistry and History LOs).
In addition to such reviews, both qualitative and quantitative data has been collected by many disciplines relative to the impact of LOs on student attitudes and learning effectiveness. For example, the Statistics team led by Brenda Gunderson implemented a newly authored “Name That Scenario” Pre-lab in Fall 2011 semester (see http://sitemaker.umich.edu/stats350.prelab/lesson_10). This LO was released to the nearly 1400 enrolled students. A total of 91% utilized the LO and 75% completed an extra credit assignment based on it. A statistical analysis was conducted to assess the effectiveness of the LO as follows:
1. Students first completed a Name That Scenario Pre-Test Assessment (8 points possible) before any access to the LO.
2. The LO was made available starting the next week and students were given the opportunity to do an extra credit assignment based on the LO (very small amount of points). Access to the LO was tracked, providing two groups of students: NTS = students who did access/use the LO, and Control = students who did not access/use the LO.
3. The week after this access and optional assignment, the students completed a Post-Test Assessment (8 points possible).
For the NTS Group, pre-to-post performance went from an average score of 4.66 to 5.88 points, an increase of 1.22 points (or 26.2% from baseline). For the Control Group, performance moved from an average score of 4.92 to 5.29 points, an increase of just 0.37 points (or 7.5% from baseline).
In another example, the General Chemistry team led by Nancy Konigsberg Kerner has been involved in transitioning from a large live introductory general chemistry pre-lab lecture to a web-based multimedia format to offer more personalized, ultra-accessible teaching and learning opportunities. The link to the multimedia interactive introduction to experiments one and two can be found here:
These personalized multimedia web-based materials were available to a group of sections in Winter 2012 term via a course management website. The draft materials are undergoing revision based on feedback. Collected qualitative survey data indicated the most helpful aspects of these materials:
•Just the fact that it was interactive was the most beneficial.
•The interactive visuals because they allowed me to see the concepts in action and understand what was explained.
•Having the procedures with pictures were great!
•I definitely liked seeing the process done in demos. I am a visual learner and trying to understand the procedure from reading the lab manual is not helpful.
•Liked how you could go at your own pace. Liked all of the videos and self-quizzes.
•Being able to learn at my own pace.
•I could take my time to understand the concepts. I could re-watch the videos, go back to earlier pages to refresh my memory before moving on, and go at my own pace. I can’t do this during the live lecture.
Students were queried on different aspects of learning impact such as whether the multimedia materials “enhanced understanding”, “were a valuable addition to lecture”, “could replace the classroom lecture”, and “prepared me for lab”. The percentage of students agreeing with such statements is quite high as summarized in the graph below.
The MELO project is currently focusing on assessment of the impact of technology infusion on student learning across the different disciplines. A number of cross-discipline surveys have been recently conducted and results are currently being analyzed.
How does this practice relate to pillars?
Learning effectiveness: This pillar is concerned with high quality education. The goal of this project is to facilitate the integration of curriculum-based sequences of online learning objects (LOs) that complement classroom pedagogy in large enrollment gateway courses to enhance student learning, engagement, and persistence in college. Thus our project is primarily impacting learning effectiveness.
Scale: We have already demonstrated the scalability of this project in that it has grown and expanded from year one to year three to impact many disciplines at UM. Openly licensed workshop training materials will allow it to extend beyond the UM community.
Access: All project produced learning materials and learning objects are openly licensed and being made available to all.
Faculty Satisfaction: Project faculty have received support and faculty recognition awards. This project continues to be funded by the UM LSA-ITC (Literature, Science and Arts Instructional Technology Committee). Faculty and graduate students involved in this project have presented different aspects of this project at a variety of venues.
Student Satisfaction: Both graduate students directly involved in this project and the undergraduate students involved in the courses that this project impacts have provided positive and often enthusiastic feedback to the technology infusion innovations. Some of this support has been provided within this application.
Equipment necessary to implement effective practice:
All you need are meeting rooms, a web browser, and a projector to share ongoing project results.
Estimate the probable cost associated with this practice:
Workshop costs would depend on the campuses involved. Some start-up costs may be incurred for participant “buy-in” (as occurred in this project). We found that attendance at the Sloan-C/MERLOT conference was invaluable with regard to community bonding and generating ideas relative to the infusion of technology into courses and assessment of same.
In addition to the links provided above for “Evidence of Effectiveness”, additional supporting documentation is provided here.
The URLs below are links to related 2012 Sloan-C/MERLOT presentations:
“Online Learning Objects: Affecting Change Through Cross-Disciplinary Practices and Open Technologies” by Emily Rodgers and Steve Lonn and others:
“Deconstructing a Puzzle Using Writing and Technology” by Ginger Shultz, Grace Winschel, and Renata Everett:
The URLS below link to examples of project effectiveness as evidenced in different discipline outcomes.
1) Shift from individual static hard copy pre-labs to online “packaged” LO simulation pre-lab assignments (Statistics): http://sitemaker.umich.edu/stats350.prelab/tutorials
2) Shift from individual hard copy pre-labs to collaborative online pre-labs with Voice Thread (Organic Chemistry): http://voicethread.com/?#u1896211.b2170018.i13249990
3) Shift from written to personalized audio/video feedback on assignments (Writing): http://www.screencast.com/users/cmodey/folders/Jing/media/8bd96754-d693-4b5e-ba59-952afb2f2e4d
4) Shift from a live-lecture to a complete online syllabus/wiki/ with infused LOs (History): https://amwest.pbworks.com/w/page/43768438/Syllabus
5) Shift from live pre-lab lectures to a web-based multimedia interactive format (Chemistry): http://www.umich.edu/~chem125/softchalk/exp1_final/
7) Shift from hard copy homework problems to online interactive LOs with assignments (intro mechanics):
8) Shift from instructors searching for LOs for curricula infusion to students searching and contributing LOs to build the curriculum http://www.screencast.com/t/MTE1MG14MTgt
10) Shift from hard copy text questions to an interactive online quiz requiring application of concepts to real-life situations via “Name That Scenario (Statistics): http://sitemaker.umich.edu/stats350.prelab/lesson_10
11) Award winning authored (grad student and student) LO (Psychology): http://youtu.be/oe8PJ8rVV0M
12) Grad student authored interactive animations (Physical Chemistry): http://demonstrations.wolfram.com/author.html?author=Porscha+McRobbie
Evidence of the positive nature of the interdisciplinary collaboration is also shown in specific LO outcomes such as:
13) Creation of a Calculus LO, “How to Write Up Math Homework” that combined the disciplines of Writing and Math:
14) Creation of a preliminary cross-disciplinary collection of LOs to address the common cross-discipline needs of students entering gateway courses:
Contacts for this effective practice:
- Nancy Konigsberg Kerner Lecturer IV, Chemistry, University of Michigan; MELO Project CO-PI
- Brenda Gunderson Senior Lecturer, Statistics, University of Michigan; MELO Project CO-PI
- Monika R. Dressler Senior Manager, LSA Instructional Support Services