On Tuesday, August 9, the Statistics MELO3D team was one of many to attend the informative workshop on assessment. We’d like to extend our gratitude to Dr. Chad Hershock and Dr. Mary Wright of CRLT as well as to Dr. Steven Lonn of the USE Lab for leading the workshop.
During the workshop we had an opportunity to plan for how we will assess the Name That Scenario learning object that has been at the heart of this summer’s work. Here is a brief overview of our plan:
1) Cross-Term Comparison
- Identify homework/exam questions from previous semesters that require students to select an appropriate statistical technique.
- Use these or similar questions in the new term.
- Compare student performance on such questions pre/post implementing NTS.
2) Paired Design to Assess NTS Impact
- Assess student ability to identify the correct statistical procedure after initial presentation of material but prior to introducing NTS using (perhaps) an in-lab quiz.
- Introduce students to NTS and encourage them use it!
- Re-assess student’s ability to choose an appropriate technique.
3) Use sitemaker/ctools data to estimate when and how often students use the new LO.
Related to 2b) above we had a good amount of discussion around what types of data we would like access to as well as how we will incorporate NTS into the course this fall. On the former, we’ve talked about building a reporting tool into the backend of NTS that would send student data on the number of questions answered to a database. This would be useful for correlating use with student achievement.
We discussed various options for introducing NTS to students, but ultimately we’ll have to work around our plans for assessing its impact if we want to gather this information. If we opt for the paired study (2) we’ll probably need to introduce it in lab or as a prelab in the weeks separating 2a and 2c. Either way, we’ll need to make this plan concrete before the start of the semester. This also means making a final check of NTS content as well as designing the assessment (2a and 2c).
In the meantime, we will also be seeking a blanket IRB exemption so we can share the results of our findings. This also means incorporating opt-out/opt-in (as appropriate) procedures for students and making sure they are well-informed about how we will be making use of this data.
Finally, thanks to Emily and Adena of the psychology team for sharing their plans with us and helping us think through our own plans. Thanks too for giving us all a chance to wear our ‘statistician’ hats in discussing power as it relates to effect size.
MELO3D@Michigan 